Let’s briefly understand what these ten skills are and why should we care to learn them here:
Firstly, to clarify, manual exploratory testing is different from manual testing. The latter refers to verifying a given list of requirements and doesn’t necessarily demand an analytical mindset. In contrast, manual exploratory testing is the skill to delve into the application details, come up with different real-life scenarios apart from what is documented in user stories, simulate them in a test environment, and observe the application’s behavior. Such exploratory testing skill highly demands a logical and analytical mindset and is the most preliminary skill to create a bug-free application. To structure these exploration sessions, there are methodologies and approaches that can be learned, which we shall discuss in Chapter 2.
This is one of the core skills for shift-left testing, as discussed earlier. Doing automated testing also significantly reduces the manual testing efforts, especially when the application grows with more features. In simple terms, the skill is to write code to test feature requirements automatically without human intervention. To do the same, we need tools, and therefore, the knowledge of different tools to write tests at different application layers is necessary to acquire the skill. It doesn’t stop there. One also needs to know what antipatterns to look for in automated testing and how to stay clear of them. We shall discuss these aspects in Chapter 3.
Continuous delivery is a practice where features are delivered incrementally to end-users in short cycles instead of a single big-bang release. By continuously delivering, the businesses earn profits early and can assess and retune their product strategy quickly based on end- user feedback. To power continuous delivery, we have to test the application continuously so that it is always ready to be released. As obvious it may be, the wise way is to automate and integrate quality checks in the CI/CD pipelines and run them frequently to ease the testing process. And continuous testing is the skill to strategize which type of automated tests are to be run at what stage of the delivery cycle so that the team can get faster feedback and also be able to integrate them to the CI/CD pipelines. These essentials are discussed in detail in Chapter 4.
You may have heard the sayings, ‘data is money’ and ‘data is the new oil,’ which shows how important testing for data integrity is today. When users’ data are lost, or the application shows wrong data to the end-users, they acutely lose trust in the application itself. The data testing skill is the knowledge about the different types of data storage and processing systems typically used in web and mobile applications such as databases, caches, event streams, etc., and to be able to derive test cases based on their individual nature. Also, Chapter 5 discusses how data flow between the application components creates new test cases apart from the functional flows.
The UI look and feel of the application is a major contributor to the business’ brand value, and especially when it comes to big B2C (business to customers) products, such as Amazon, a low visual quality may impact their brand value instantly, as millions of end-users use them regularly. Therefore, it is essential to validate that the end-users can have a seamless visual experience of the application by conducting visual testing.Visual testing requires an understanding of how the UI components interact with each other and with the browsers. Such visual quality checks can be automated, too, using tools that are different from automated functional testing. We will also talk about the stark differences between these two types of automation in Chapter 6.
Security breaches have become too prevalent in today’s world, and not even giants like Facebook and Twitter are excluded from such attacks. Such security issues cost heavily for both the end-users and the business in terms of legal penalties, brand reputation, and loss of sensitive information. So far, security testing has been isolated as a niche skill in the industry, and only a set of penetration testers were engaged towards the end of the development cycle to find security issues. But with the imbalance of available professional security testing talent and growing security breaches, software teams are in a situation to incorporate basic security testing as part of day-to-day work. We will be discussing how to tune our mindset to think like a hacker and find security issues in the application functionality in Chapter 7, along with tools to automate security scans.
Even a slight slack in application performance leads to a huge revenue and reputation loss for businesses—recall the Flipkart example we discussed earlier. Performance testing is the skill to measure a set of key performance indicators at different application layers. Performance tests can also be automated and integrated with the CI pipelines to get continuous feedback. We will discuss a shift-left performance testing strategy along with tools in Chapter 8.
Web and mobile applications have become everyday commodities. Making them accessible to everyone with permanent or temporarydisabilities is not only mandated by legal regulations in many countries but also an ethical activity to do. We will also discuss why incorporating accessibility features may even be a lucrative option for businesses in Chapter 9.
In order to acquire the accessibility testing skill, we need to understand the accessibility standards required by law first. We can then use both manual and the automated accessibility auditing tools to validate if the standards are met.
We saw how the end-users and businesses have a tall list of quality requirements such as availability, scalability, maintainability, observability, and so on, apart from just needing bug-free functionalities. These are called cross-functional requirements of an application. They are also popularly referred to as non-functional requirements (NFR) by many in the industry. We will discuss the subtle differences between these two terms in Chapter 10. Although functional requirements grab their undivided attention generally, it is the CFRs that imbue high-quality into the application, and whenever CFRs are not incorporated properly, it will lead to either an unhappy business or software team or end-users! Therefore, CFRs testing skill is a fundamental testing skill. We will discuss the testing methodologies and tools for validating different CFRs in Chapter 10.
It may come across as a surprise that there were 5.7 million 6 apps across the Google play store and Apple store in 2021. This boom in the number of mobile apps is mainly stemming from our increased usage of mobile devices today. Indeed, the web analytics company, Global Stats, announced in 2016 that the mobile and tablet internet usage across the globe had surpassed desktop usage based on their data. 7 So learning to test mobile applications and compatibility of websites across mobile devices have become a critical skill today.
In today’s world, ‘digitalization’ is inevitable to sustain and grow any business. Many businesses are leading the way in this aspect, while some are in the early phases of modernizing their existing digital platforms. Digitalization is key to broadening a business’ reach from a local community to a global scale, translating to more adoption and more revenue. Almost all small and large-scale enterprises in various sectors, like health care, retail, travel, academics, social media, banking, entertainment, devise plans to advance their digital strategies as a critical measure to reach new customer segments and yield higher profits.
In this journey towards digitalization and modernization, innovation becomes a crucial driver. We can see the businesses that innovate constantly continue to stay relevant and thrive over many decades. Netflix is a classic example. It started as an online DVD rental portal in the 1990s. It ventured into online streaming in 2007, cannibalizing its own DVD rental business. It later started producing original content called Netflix Originals. As of 2021, Netflix is the largest online streaming service with over 200 million global subscribers!
The technology space has been evolving in parallel to these innovative businesses to accommodate their increasingly advanced needs. Gone are the days where we were willing to wait in line to buy movie tickets, visit a store in a remote location to buy a specialty product, or carry around a hand- written shopping list searching for specifics. Technology eases everyday tasks extensively. We can sit at home and stream our favorite entertainment programs in one touch, try a new dress virtually, schedule our shopping list to be delivered regularly, and even more—brew a coffee with voice command!
With such rapid evolution in technology, product strategy must be versatile, catering to different customer needs to survive the ongoing competition in their respective sectors. It has to expand its horizons beyond just building websites. Consider the ride-hailing companies, such as Uber and Lyft, which provide their services at our fingertips in various ways—the web, Android, and iOS mobile platforms. Uber even announced their WhatsApp chatbot to book a ride as an easier alternative to their apps. 2 This kind of versatile product strategy has helped them expand across the globe and outgrow their competitors.
With such versatile product strategy and innovation, the businesses become successful in acquiring a critical mass of customers. But now, the next challenge is to thrive further—earn more revenue and more customers. We can see the giants in the industry, such as Amazon, leveraging their existing customer base to cross-sell services & products as a strategy to expand beyond. Amazon, which started as an online bookstore, now cross-sells various products ranging from fresh pantry items to electronics to apparel, jewelry, and more, meeting customer demand for consumer goods in nearly every market segment.
And why are we discussing these details? Because today’s software industry caters to all these business needs: innovate new product ideas, bring them to life, and scale them to reach new customer segments acrossthe globe. Unquestionably, the software development teams are standing on the edge, especially when the need of the hour is to deliver with high quality! Indeed, software quality has become a non-negotiable criterion in today’s competitive market. We can even say that compromising on software quality is equivalent to running a race knowing that you will lose it. This is reinforced by many real-world examples too: In October 2014, on the “Big Billion Day” sale, the two Indian eCommerce giants, SnapDeal and Flipkart, went head to head on their seasonal sale after marketing the sale event for months. Unfortunately, Flipkart’s website crashed multiple times during the day due to overwhelming demand, losing its sales to Snapdeal and eventually losing its number one position in the Indian eCommerce sector. Similarly, the Iranian chat application, RayChat, lost all its customer data due to a bot attack in 2021, as a result of an oversight in their database configuration. Given the company was a booming startup just founded in 2017, it had to face noxious encounters to the attack. Such is the impact of software quality!
We can see many similar examples till date across the globe that keep reinforcing that businesses, albeit novel product ideas, will slip down a steep slope when quality is compromised as customers quickly move on to more reliable competitors. At times, businesses may be forced to choose time-to-market over software quality, but they only have created a debt for themselves that needs to be caught up before their competitors use that debt to their advantage. As a result, we can firmly say that quality is quintessential for sustaining a business in the long term!
Such high software quality can be reaped only by a combination of skillful development and meticulous testing. In particular, when I say meticulous testing, I refer to paying attention to every aspect of the application throughout its stack. This chapter will introduce the breadth of such full stack testing, which has to be performed for a typical web or mobile application.
So, to begin with, let us come together on a common understanding of software quality. Software quality was once equated to a bug-free application. Anyone in the software world today will agree that it’s not just that anymore. If you ask end-users to define quality, we will hear them speak of ease of use, look and feel, data privacy, swiftness in rendering information, and availability of services 24/7. If you ask businesses to define quality, we will hear them speak about return on investment, real-time analytics, zero downtime, no vendor lock-in, scalable infrastructure, data security, legal compliance, and more. All of these are what makes an application a high-quality software today. A slack in any of these aspects would affect the quality in some way or the other, which is why we need to test for them meticulously! Though the list of quality requirements looks tall, we have tools and methodologies to cater to most of these needs. So, the bridge to high quality is the knowledge around the tools and, more importantly, the skill to apply them in a given context, both from a development and a testing perspective. This book aims to serve as that bridge between learning the testing skills and delivering high-quality web and mobile applications.
Testing, in a nutshell, is a practice to validate the behavior of the application to be as expected throughout. For testing to be successful, it needs to be practiced at the micro and macro levels. It has to be entwined with the granular aspects of the application, such as testing every method in a class, every input data, log message, error code, etc. Similarly, it has to focus on the macro aspects such as feature testing, integration between features, end- to-end workflows, etc. But we cannot stop testing there! We need to further test for the holistic quality aspects of the application such as security, performance, accessibility, usability, and more from the list mentioned earlier to achieve the end goal of delivering high-quality software. We can encapsulate all that to say we need to do full stack testing! As represented in Figure 1-1, full stack testing is to test for different quality aspects of the application at each stack (the UI, services, and DB) and application as a whole.
Indeed, full stack testing and development should be inseparable, as the two rails of a railway track. We must build both rails simultaneously to build quality into the product; otherwise, we are guaranteed to derail. For example, suppose we are writing a small block of code to calculate the total order amount for an e-commerce application. Parallelly, we need to test if the code is computing the right amount and if it is a secure code. If we do not test, then we have a broken rail line. When we continue to develop on top of such a fractured rail-line, we will have poor integration and sub- optimal functionality. To ingrain testing at such elementary levels, teams need to stop conceiving testing as a siloed post-development activity, as it has been traditionally done. Full stack testing needs to begin parallel to development and be practiced throughout the delivery cycle, giving faster feedback. The practice of starting testing early in the delivery cycle is referred to as shift-left testing, and for full stack testing to yield the right results, we need to follow shift-left testing.
If we were to write down the sequence of activities in a traditional software development lifecycle (SDLC), they’d read: requirements analysis, design, development, and testing, where testing comes at the end. As seen in Figure 1-2, ‘shift left testing’ suggests shifting the testing activities to the beginning of the cycle instead to produce high-quality results.
Let us take an analogy to understand this concept better. Let us say your team is building a new house. Does it seem sensible to complete the construction fully and then check for quality at the end of construction? What if you find out the rooms are not of the correct measurements then? What if there is no space to place the door? Those are the issues shift-left testing tries to overcome. The approach believes the quality checks should start right from the planning stage and continue throughout the development phase for the end product to be of the highest quality.
When it says quality checks should continue throughout the development phase, it means the quality checks have to be repeated iteratively for every small chunk of work so that the changes can be incorporated smoothly. In the house construction analogy, it means to perform quality checks as each wall is getting built so that the issues are corrected immediately. To perform such extensive testing, shift-left testing heavily relies on automated testing and Continuous Integration and Deployment (CI/CD) practices, where the quality checks are automated at micro and macro levels and continuously run against every small chunk of work in the CI server. This ensures theapplication is continuously tested at less cost and effort than manually testing every small chunk of work for multiple quality aspects.
To further understand, let’s break down shift-left testing into day-to-day activities. Let’s say a software team follows an iterative development cycle, such as in Agile development. Some of the quality checks they may do in different phases of delivery to shift testing to the left are captured in Figure 1-3.
Reading the figure 1-3 from the left, it begins with highlighting a set of quality checks that are carried out by the team before a user story is considered ready for development:
A ceremony called the three amigos process 5 is conducted in the analysis phase. Here the business representatives, developers, and testers gather briefly to mull over the feature thoroughly. The process aims to collect all three roles’ perspectives so that integrations, edge cases, and other business requirements don’t get overlooked. This is the first step in shifting-left, where the requirements of a feature are validated, to begin with.
Parallely, the business representative on the team works with the user experience designer to validate and improve the application design.
Once the above two steps are over, an Iteration planning meeting (IPM) is conducted at the beginning of an iteration/sprint to discuss the user stories of that iteration in detail. This provides an open space for the team to collectively validate the requirements once again.
And during the iteration, just before a user story is picked up for development, a story kickoff happens. The story kickoff is a minified version of the three amigos process where the focus of discussion gets deeper into that particular user story’s requirements and edge cases. By this stage, we can fairly say that the team has tested/validated the requirements diligently.
Similarly, while developing a user story, the following quality checks are implanted and utilized to get fast feedback:
Developers write unit tests as part of each story and integrate them with CI. They also add linting tools and plugins for static code analysis and integrate them with CI to get continuous feedback.In some teams, developers also write the UI-driven functional tests as part of user story development and integrate with CI. In other teams, testers write them post-development, which are both common practices.
Before committing the latest changes, developers run a set of automated tests on their local machine to get the first level of feedback.
The second level of feedback is obtained from the suite of automated tests (unit, service, UI, etc.) that are run in the CI for every commit.
The third level of feedback is received from a process called dev- box testing, where the testers and the business representatives do a quick round of manual exploratory testing on the developer’s machine to quickly verify the newly developed functionality.
With such rigorous focus on providing faster feedback, the team will get almost half of the feedback that would have otherwise been gotten from performing manual testing post-development before the user story comes to the testing phase itself. In other words, the team just shifted testing to the left. And, this kind of shift-left testing will give the testers on the team the liberty to fully explore the user story for various quality aspects rather than just verifying the expected functional behaviors.
Thus, shift-left testing enables defect prevention to begin with, by having multiple rounds of validations on the requirements, then assists in catching defects early — as early as in the local developer’s machine and in the CI, and ensures delivering high-quality software by giving the space for testers to explore various quality aspects in depth.
This concept of incorporating testing earlier in the delivery cycle is not just restricted to functional application testing. It can be applied to testing in general, including security testing, performance testing, and others. For example, one of the many ways to shift security testing to the left is to add a pre-commit scanning tool like Talisman, which scans the commit for secrets and alerts even before checking in the code. In each of the upcoming chapters, you will get to see practical approaches to do shift-left testing.
Overall, shift left testing embodies the aphorism, ‘Quality is the team’s responsibility’ as performing quality checks at every phase of the software development life cycle—validating application design prototypes, requirements, and so on, as discussed earlier has to be owned by different team members. So we can say that building relevant testing skills to perform various quality checks is vital for all roles in a team to deliver high-quality software successfully!