Cross-Functional Requirements Testing ​

Businesses often think of hundreds of functional requirements to add value to the customers and gain revenue. As evident as it is, functional requirements refer to the core business services offered to the customers— for instance, the feature to book a ride in a ride-hailing app or the feature to pay with a net banking facility. However, functional requirements are just not enough to be successful. Imagine searching for a ride, and you had to wait for 5 minutes to see the ride availability. We would often get a taxi on the road by that time, and the app would eventually cease to exist. Or suppose the app is doing its job functionally well but it takes several steps to book a single ride. I am sure you would look for a user-friendly alternative sooner or later. Even more, what if the app is insensitive to your personal details and has exposed them on their site? I would assume that calls for a good-bye to the app. These are the fundamental reasons why businesses and software teams need to focus on cross-functional requirements. They make the application complete and, most importantly, imbue high quality.

Cross-functional requirements (CFRs) refer to those features of the application that have to be built into every functional feature. For example, a couple of CFR for the ride-hailing app could be that the app should respond to users within X seconds, users should be able to perform any action within N steps, and the app should transmit and store the user details securely. Only when CFRs are built and tested thoroughly across all features, will any app have a chance of becoming a good competitor in the market.

Indeed, we have been discussing CFRs even in the previous chapters: performance, security, accessibility, visual, and data testing. This chapter is specifically focused on bringing attention to a collection of CFRs, whichcertainly forms a tall list. In the process, we shall take a broader view on the CFRs, and discuss an overall CFRs testing strategy that can cater to provide continuous feedback for the team. We will also be discussing some essential testing methodologies and tools to assist in implementing the test strategy.

There could be more than the list in the table. But overall, the CFRs or the '-ilities,’ as they are colloquially called, define the executional and evolutionary qualities of the application. Executional qualities are the behavior of the application during its runtime, such as availability, authentication, monitoring, and others. Evolutionary requirements define the static application code quality like maintainability, scalability, extensibility, etc. When executional qualities are not embedded in the application, the end-users and the business shall witness the impact first-hand. When evolutionary qualities are not addressed, software teams get the blow first, which then slowly translates as an issue for the business. For instance, end-users get frustrated when the system is unavailable, and when the code is unmaintainable, team members get frustrated, leading to productivity loss. In order to avoid such frustrations, we should establish a set of CFRs for the application right at the beginning of development and continuously test for it throughout the delivery cycle, just like functional requirements.

To lend a hand there, we shall discuss an overall CFRs testing strategy now.

CFRs Testing Strategy ​

Let’s discuss the FURPS model 3 , 4 to begin with. We will be using the model to establish a high-level CFRs testing strategy, as basically, the model has abstracted five themes out of all the software requirements including the CFRs. FURPS stands for functionality, usability, reliability, performance and supportability. The themes can be elaborated as follows:

Functionality This category of software requirements can be experienced as user- flows on the application such as login flow, ride availability and booking flow.

Usability This represents the set of requirements that affects the user experience such as the visual quality, browser compatibility, accessibility, ease of use, and so on.

Reliability These set of requirements contribute towards making the application consistent, fault tolerant, and recoverable.

Performance These requirements talk about the backend KPIs as discussed in Chapter 8 and the front-end performance metrics.

Supportability This category includes all the evolutionary code qualities such as maintainability, testability, secure code, and so on.

All the row items in table 10-1 can be visualized along the same lines. For example, accessibility, as discussed in Chapter 9, is manifested as functional features such as adding a transcripts section for videos and also via the application design. So the testing approach for accessibility will comprise the methods and tools used for testing the functionality as well asusability. Similarly, one way to incorporate security is to add authentication related functional features such as user login and ensure it is performant, reliable, usable, and imbibe evolutionary qualities. This section has testing strategies for each of these five themes, as depicted in Figure 9-1. To formulate your project-specific CFRs testing strategy, decompose the different aspects of the CFRs and adopt the respective methods and tools based on project priorities.

Functionality ​

To test the functional aspects of the CFRs, the manual exploratory and automated functional testing tools and methods discussed as part of Chapter 2 and Chapter 3 can be employed at different application layers. To reiterate, tools like Postman, Selenium Webdriver, RestAssured, JUnit can be used to automate them and get continuous feedback in teams. Additionally, the data testing tools and methods discussed in Chapter 5 will become essential in testing a functionality.

A special callout while testing the compliance-related functional features such as the strong customer authentication (SCA) feature as part of PSD2 or GDPR related functional features is that it is extremely crucial to gather the right information on those regulations and to mandatorily involve the legal team in the testing and the requirements gathering phase. For quick reference, a section on a few regulatory requirements is appended as part of the next section of the chapter.

Usability ​

In order to approach usability testing methodically, we can deconstruct usability into a few aspects such as visual quality, cross-browser compatibility, localization/internationalization, user experience design, and accessibility. We have discussed the testing approaches and tools for visual quality and cross-browser compatibility in Chapter 6 and accessibility testing tools in Chapter 9. Let’s discuss the testing approaches for the remaining aspects here:

Localization / internationalization testing ​

Localization testing can be addressed in a few ways. If the UI skin varies for different locales, you can adopt visual testing. If the skin doesn’t vary but only the language and features like date and money format changes, you can adopt unit testing and manual testing. Somelanguage texts are too long, which results in changing the UI layout too.

In those cases, you can adopt visual testing again. The verification of language-specific text needs to follow a sequence of steps. The first step is to get the meaningful text for all elements, messages, etc., from a person who knows the language, followed by approval from the product owner or any business-approved representative. Next is to document the right text in each story to enable development and manual story testing. Most often, when these steps are missed, the developers are left with no other option to fill the text with a Google-provided translation, which results in double testing efforts before and after getting the approved strings. Also, procrastinating localization testing towards the end of the release poses a risk of a broken UI layout when the text doesn’t fit the elements’ layout. You can also add unit tests to compare the locale’s string files for missing keys. Additionally, unit tests in the front-end and back-end to validate each locale’s date and money formats will cover the common aspects in localization/internalization testing.

User experience ​

User experience talks about the design aspects of the application, such as how intuitive the user flow is, how many clicks it takes the user to get the necessary information, whether the icon conveys the right meaning, and whether the application’s color palette suits the end-users’ liking and so on. Such aspects are researched during the beginning of the project and incorporated into the design. For instance, in a retail mobile project I worked on, we found that people from Italy preferred to see bright colors, like dark red, and designed the application with such a color palette.

As a general practice, you should include user experience aspects in manual exploratory testing for every user story. Jakob Nielson and team from the Nielson and Norman group did extensive research on user experience design and have collated a list of 10 usability heuristics to follow, which can be incorporated into testing. Most often, the product owners and the user experience designers pair on such testing. There are also tools like Userzoom and Optimal Workshop to conduct user experience tests on the design prototypes with real end-users. I have seen such tests when conducted periodically during the delivery cycle with different end-user groups result in enhancing the user experience design significantly.

A/B testing is another way to get real-time feedback on the user experience in production. A/B testing, though it is called testing, is more of experimentation per se. It is the process of presenting different user experience designs of the same feature as prototypes to two different end-user groups in the production and collecting data on the user’s behaviors to enable the product team to decide the final design.

For example, a simple experiment would be to understand if the sale button has the highest likelihood of being clicked if it’s red or blue. In such an experiment, the red and blue buttons are presented to different user groups in production, observed for a period, the usage data is collected and analyzed. This kind of experiment may require data science capabilities, 5 and usually, a team of product owners, data scientists, developers, and user experience designers work together in such experiments.

Reliability ​

From Table 10-1, the CFRs that contribute to the application’s reliability are recoverability, resilience, auditability, archivability, reporting, monitoring, observability, and consistency. Many of these reliability aspects, such aserror handling, retry mechanisms, fall-back mechanisms for single-point failures, consistency of data, and integration with third-party tools for monitoring, observability, and reporting services, can be experienced as user flows. The functional testing approaches, as discussed earlier, can be deployed here. Apart from that, the other testing methods that contribute to reliability testing are the following:

Chaos engineering ​

Chaos engineering is a way to unearth inherent flaws in the application that would lead to system outages, failures, and other disasters making the application unreliable. Usually, this method uncovers the unknown unknowns and is immensely helpful in large-scale systems. Chaos Engineering is discussed in detail in the next section of the chapter.

Infrastructure testing ​

Infrastructure is one of the many important parts of an application which contributes to reliability and recoverability. If it goes down, everything goes down. Additionally, the infrastructure layer has to be wired appropriately in order to support auto-scaling, alerting/monitoring, load balancing, and archiving capabilities. Although focussed testing at the infrastructure layer hasn’t become prevalent yet, it is gaining traction due to the increased need for businesses to scale widely. We shall discuss this in detail in the next section of the chapter as well.

Performance ​

We have unboxed the need for performance, and the tools and metrics for both the front-end and back-end performance testing as part of Chapter 8. To reiterate, some of the metrics are availability, response time, and concurrency, and the tools that help in performance testing are JMeter, WebPageTest, and LightHouse. An additional point to note is that performance testing caters to fulfilling the scalability requirements byidentifying the system breakdown threshold and thereby contributes to enhancing the reliability of the application too.

Supportability ​

Supportability refers to all the evolutionary code qualities such as compatibility, configurability, extensibility, installability, interoperability, portability, maintainability, reusability, security, and testability. Some of their functional manifestations, such as the configurability of functional features, compatibility with required protocols, installability in appropriate operating systems, interoperability features, etc., can be tested using the functional testing approaches with proper environment setup and stubs. The other approaches to test the supportability aspect are:

Architecture Tests ​

Architecture tests are those that are added to assert a set of architecture characteristics, such as verifying the right classes are under the right package (thereby conforming to the reusability requirement in this example). These automated tests provide feedback to the team if they deviate from the essential architecture characteristics that were designed in the first place to cater to some of the CFRs, such as reusability, portability, maintainability, etc. Tools like ArchUnit can be used to write such tests in Java, which we will discuss in a bit.

Static Code Analyzers ​

Many tools do static code analysis and provide useful feedback that serves to enhance maintainability—for example, Checkstyle mandates the team to stick to a common coding style. PMD is a tool that reports issues such as unused variables, empty catch blocks, duplicate code, etc. It also allows the team to add custom rules specific to the project standards. ESLint is a similar tool for checking Javascript code for possible style and code errors. SonarQube is a widely adopted tool that helps in assessing code coverage and scanning for vulnerabilities. Also, to recall, we discussed more static code analyzers that analyze the code to be secure and accessible in earlier chapters.

Using these methods and tools, you can shift your CFRs testing to the left. As discussed in Chapter 4, these CFRs can be continuously tested along with functional tests as part of CI, which shall enable the team to get continuous feedback on all quality dimensions and, thereby, continuously deliver!

Other CFRs Testing Methods ​

To assist in that goal of shifting CFRs testing to the left and being able to do continuous delivery, the newly introduced CFRs testing methods from the strategy section, such as chaos engineering, architecture testing, and infrastructure testing, are discussed here. You can also read about a set of commonly implemented regulatory requirements towards the end of the section, which will support your compliance testing efforts.

Chaos Engineering ​

Application reliability is one of the critical CFRs, as any service outage results directly in a loss for the business. A study in 2014 suggested that the cost of downtime ranges from $140K - $540K per hour for some businesses. I won’t be surprised if the cost is even higher in 2022. As a matter of fact, established products in the market such as the Amazon Web Services strive to achieve an uptime of 99.999%, i.e., a downtime of 5 minutes and 15 seconds per year.

Some of the factors that lead to downtimes are bugs in the application, single point of failures in the architecture, network issues, hardware failures, high load of unexpected traffic, and dependent third party services’ issues. Most of these factors are considered while designing the architecture, and teams do take relevant preventive measures during development too. For example, the exponential back-off method is adopted to handle service downtimes. The method prescribes to exponentially decrease the request frequency to a down service so that it gets breathing time to recover quickly. The blue-green deployment model is implemented in order to avoid downtimes during system updates. The model works by having two identical production instances where one is live and the other is used for upgrading and then switched as the live instance. Apart from these, teams handle downtimes in several ways preemptively such as having replicas to share high load, auto-scaling infrastructure, proper error handling of inputs, and so on. Despite all the efforts, large-scale distributed systems pose discrete challenges to the reliability of the application in the form of convoluted workflows, multiple layer dependency, third-party failures, downstream systems errors, and so on, which cannot be easily foreseen and eventually lead to downtimes.

For example, a fifty-member team worked on a large-scale distributed application and set up two instances separately to cater to US and UK regions. They configured each instance to redirect to the other when one went down and built functional capabilities in the application to handle the requests from both regions. The team tested the functionality and the redirection flow. They also checked the performance of their application on load. However, when there was a peak sale in the US, and the UK instance went down due to technical issues, the requests were redirected to the US instance, making a particular feature in the application throw errors to all the users. The root cause was later spotted in one of the third-party downstream systems with a constraint on rate limit per hour, which exceeded and started throwing errors. Practically, this is one of those edge cases that is hard to pinpoint, given the team had done their due diligence in all aspects. Also in reality, it is hard for anyone to know all the nitty-gritties in such large-scale distributed systems.

The fifty-member team is not alone. Netflix, too, had troubling experiences when they became cloud-native as the cloud instances faced unplanned outages due to various issues, resulting in losses and extended working hours. They took that up as a challenge! They deliberately mimicked the outages and solved the issues in their application one-by-one until the application became entirely resilient to such unplanned outages. To help with that approach, they designed a tool called the ChaosMonkey, which brought down one random instance of a cluster every day during their working hours, and the engineers implemented safety measures appropriately. This approach ensured every one of their system’s inherent unpredicted flaws were addressed, leading to a resilient and reliable system. Based on this success, they further evolved and crystallized the practice to form the Chaos Engineering discipline.

A formal definition for Chaos Engineering from the book Chaos Engineering, published by O’Reilly in 2020, is as follows:

“Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.”

— Nora Jones and Casey Rosenthal

As they have quoted, Chaos Engineering is a method to conduct experiments on the application, i.e., to simulate errors, outages, and other unexpected scenarios, and observe the application behavior. This practice is now getting widely adopted in the software industry, and many companies have evolved them further to suit their needs. From the collective learnings of the industry, the fundamental characteristics of Chaos Engineering can be enunciated as the following:

• It is more of an experiment rather than a testing type, i.e., it is not the activity where we verify the expected behaviors of the system when there are known outages but to observe the behavior of the system under unexpected situations and gain insights.

• The purpose of experimenting is to gain confidence in the system’s reliability and resilience. You can choose not to experiment when you have enough confidence in your system’s ability to handle unknown turbulence.

• It is particularly beneficial when you are developing a large-scale distributed system.

• Chaos Engineering is not the responsibility of one particular role, such as the dev-ops engineers or the testers. It is a team activity where all stakeholders together design the experiment, conduct, and debug the behavior.

It might have been probable for the fifty-member team to have spotted the rate limit issue earlier from conducting such chaos experiments!

Chaos Experiment ​

So, if you’re planning to orchestrate such chaos experiments, the Netflix team recommends conducting them in production directly as the real-life variables are extremely tough to simulate in a test environment. They also recommend designing proper capabilities to pause such experiments in between and reverting the system back to normal. There are several tools today that assist in scripting chaos experiments such as the Chaos Toolkit and ChaosBlade, so you don’t have to manually bring things down and up in production.

So to perform a chaos experiment, first, develop a hypothesis that might challenge the reliability of the application along with a cross-functional team. Next, define a steady-state hypothesis, which is the predicted application’s behavior during the experiment. Script them using the tools and run them in production. If the tool alerts the experiment to have failed, i.e., the steady-state hypothesis has not been met, your cross-functional team can jump into action.

Architecture Testing ​

At the beginning of any project, a list of functional and cross-functional requirements are mulled over extensively, and a conducive architecture design is laid out. For example, let’s say, in order to fulfill maintainability and reusability, the architecture design proposes to have separate layers. Also, since performance is of priority, caching mechanisms are placed in the right layers. But a universal law that may disrupt these well thought- through design decisions shall be Conway’s law. In his paper “How Do Committees Invent?”, Melvin Conway states that the team structures, particularly the communication paths between people, inevitably influence final product design. We can decipher from the law that the teams working on smaller portions of a large system would inadvertently optimize theirrespective portions without factoring in the big-picture needs.

For example, they may choose to prioritize performance over reusability and bypass layers, which may result in rework later. This is where architecture tests, when rightly placed to guard the essential architectural characteristics, will provide feedback to teams as and when they deviate from the big picture.

Tools like ArchUnit for Java and NetArchTest for .Net can be used to write such architecture tests. For example, you can introduce architecture tests to check for cyclic dependencies so that maintainability is continuously prevailed. Similarly, you can add architecture tests to make sure the packages are independent so that they are reusable. The ArchUnit tests are similar to JUnit tests and can be run as part of CI pipelines. Example 9-2 shows an ArchUnit test to assert if all the classes in the order management service reside in the OMS package, to ensure reusability. So whenever there is a need to include a class outside the responsibilities of the order management service, the team will be pushed to discuss the big picture and decide where it fits in appropriately.

Similarly, JDepend is a tool that produces metrics on design quality in terms of extensibility, maintainability and reusability. It does a static code analysis on all the Java classes and gives different design scores for a given Java package. It uses the number of abstract classes and interfaces in a package as a measure of its extensibility; checks for dependencies on external packages and raises alerts when there are unwanted dependencies and similarly, checks for package cyclic dependencies. It can be written as a JUnit test and be integrated with CI for continuous feedback on architecture quality.

Infrastructure Testing ​

The term infrastructure on a high-level refers to the computational resources (e.g., machines, VMs, containers), network structures (e.g., VPNs, DNS entries, proxies, gateways), and finally, the storage resources such as AWS S3, SQL server, secrets management systems, etc. Infrastructure testing is to test the setup and configuration of these computational, network, and storage resources to be as needed by the application. Such infrastructure testing is an emerging area in testing.

The need for infrastructure testing mainly arises from the growing demand to scale the application. The demand is mainly because the businesses, oncesuccessful, want to quickly expand their online services to newer regions and start serving a huge customer volume. To enable such quick scaling, they should have the ability to replicate the existing end-to-end application stack, including the infrastructure setup, within a short time and, if possible, to automatically replicate in a single click. Although most software teams have an established automated process to test, bundle and deploy the application to any environment in a single click, they sometimes fail to have the same capability on the infrastructure side, which introduces a gap in their ability to scale the application in a single click. This is where the practice of Infrastructure as Code (IaC) becomes very useful.

Infrastructure as Code refers to the practice of designing the infrastructure setup and configuration as reusable code, just like application code, to power continuous delivery and scalability. For instance, code is written to spin a cloud instance with 3 GB memory using the cloud provider’s APIs, set up application-specific load balancer rules, and a firewall. These features need to be tested so that the same code can be used to spin new infrastructure instances whenever there is high load or expansion to new regions.

• Terraform is a widely adopted open-source tool to script infrastructure code using declarative coding style. It enables the code to work across multiple cloud providers. To test the infrastructure code written using Terraform in various stages of path to production:

• Terraform gives the terraform validate command to check for syntax errors in the terraform code, which can be applied as early as in the development stage.

• TFLint, a linting plugin for Terraform, can help in analyzing the static infrastructure code for deprecated syntaxes, deviation from best practices such as naming conventions, etc. TFLint can also check for the existence of specified image types against popular cloud providers such as AWS, Azure, etc.

• Once the development is complete, terraform compares the latest code changes against the environment state and presents a previewof the changes as a safety measure before execution. For example, when the code changes result in an unintentional deletion of the database, the preview feature will save the day. The command to do that in terraform is terraform plan. You can also write automated tests against the output of this command to verify some of the aspects like security policy compliance.

• The next step is to deploy the infrastructure code to create actual cloud instances and verify if the instances have the intended infrastructure resources—for example, if the instance is running within the private subnet and has the current disk space. These cases can be automated using tools like Terratest, AWSSpec, Inspec, Test-Kitchen, etc., and added to CI.

• Then comes the end-to-end testing of the infrastructure components. We need to check if the components can interact with each other in an expected way—for instance, if the webserver can make a call to the application services. This testing, in a way, gets covered when the deployment of your application code is successful and the functional tests run smoothly. But it’s also possible to catch such issues before deploying the application by writing infrastructure tests using a combination of tools mentioned earlier.

Kief Morris, author of the book Infrastructure as Code published by O’Reilly in 2020, suggests the distribution of the infrastructure tests in various layers may form a diamond pattern instead of a pyramid. This is because the unit tests for the low-level declarative code, such as in Terraform, may not be of much use and are recommended to be kept minimal. So depending on the nature of the infrastructure code, we should choose to add relevant tests.

Apart from the functional end-to-end testing, the other aspects to be tested when it comes to infrastructure are:

Scalability ​

Auto-scalability of instances based on load and if the application features work smoothly after scaling.

Security ​

Infrastructure security is a very critical aspect to be tested. Tools like Snyk check for potential vulnerabilities in infrastructure code during development. Some of the security test cases, like checking for unexpected open ports, appropriate public and private facing instances, and so on, can be tested manually and by writing automated infrastructure tests.

Compliance ​

Sometimes, there are policies and compliance features the infrastructure code needs to adhere to. For instance, to be PCI DSS compliant (PCI DSS is discussed as part of the next section in this chapter), appropriate firewalls should be set up. To check for compliance rules, Hashicorp offers Sentinel for enterprises. An open-source tool to check compliance features in Terraform is Terraform-compliance. The tool is based on Python and gives a BDD layer just like Cucumber to write tests. The tool runs the tests against the output of the terraform plan command instead of the actual instance.

Operability ​

All other operational features such as log archival for auditability, monitoring tool integration, automated maintenance features, and so on need to be tested as well.

Based on the complexity and nature of the infrastructure code, you can choose to write automated tests for these cases and integrate them with CI. Many of the tools for automated infrastructure testing are still evolving and require coding skills beyond just one language. For instance, Terratest uses GoLang, Terraform-compliance uses Python, and AWSSpec uses Ruby.Additionally, many automated testing tools may require real infrastructure to be up and running, incurring costs. So, given the constraints, you shall craft the infrastructure testing strategy specific to your application needs.

Compliance Testing ​

Two of the commonly implemented regulations on the web are GDPR and WCAG 2.0. We have already discussed WCAG 2.0 elaborately in Chapter 9. We shall discuss GDPR in this section.

General Data Protection Regulation (GDPR) ​

GDPR primarily aims to protect EU citizens’ private data. If you are trying to sell goods globally, i.e., including EU citizens, then the website is bound by GDPR legislation. Another example is if a school in the US allows admissions to EU citizens via its website, then it has to abide by GDPR requirements. Non-compliance with GDPR may result in heavy penalties as high as 4% of the company’s annual revenue.

According to GDPR, private data is any information on its own or combined with other information that can be used to identify a livingindividual. The individual’s racial or ethnic origin, religious or philosophical beliefs, political opinions, sexual orientation, genetic data, biometric data, past or present criminal convictions, etc., are classified as ‘sensitive personal data,’ which needs to be carefully protected. Even many online identifiers like IP addresses, MAC addresses, mobile device IDs, cookies, user account IDs, and other system-generated data that can be used to identify a living individual come under GDPR protection. In order to protect the data, GDPR recommends the ‘Privacy by Design’ principles to be implemented by development teams. Privacy by Design framework, developed by Dr. Ann Cavoukian essentially gives seven foundational principles that focus on preventing any privacy-invasive events.

Some of the technical measures that you can implement are the protection of data at rest using dynamic salt and hashing techniques, encryption of data in transit, adhering to the principle of least privilege, pseudonymization, anonymization of data, and other general data security measures discussed as part of Chapter 7.

GDPR also advocates user’s rights to control their data in various ways like the following:

• Right to be informed: You need to let the application’s users know how their personal data is used. This is typically handled through the site’s privacy policy.

• Right of access: Users have the right to request their stored personal records.

• Right to be Forgotten: The users can request to delete their personal data when there is no compelling reason for its continued processing.

• Right to restrict processing: Users can prohibit the processing of their personal data. The website can still store the data but no longer process it.

• Right to rectification: Users can correct incomplete or inaccurate information on the website.

• Right to portability: Users can obtain and reuse their personal data.

• Right to object: Users can object to using their personal information for marketing, research, and statistics.

• Rights related to automatic decision-making states that users should be asked for their consent to use their profiles for automated decision-making, like profiling.

Most of these requirements could be tested using the functional testing approaches. For instance, you can add automated micro and macro level tests to assert that there are no implicit opt-in, verify if the personal data is stored only after the user’s consent, and that personal information is not in application logs. Also, the security testing concepts and mindset discussed in Chapter 7 will fit right in here.

Given that we discussed the major ones — GDPR and WCAG, if your application deals with credit cards (which most websites do) or provide payment services to the EU region (which happens fairly often), PCI DSS and PSD2 regulations will come into play. Here is a brief on them:

PCI DSS ​

PCI DSS stands for Payment Card Industry Data Security Standard. It applies to any entity that stores, processes, or transmits cardholder data. This means, in general, it applies to all the sites that take credit card details—even a donation site. It is defined by the PCI Security Standards Council to protect online card transactions. It is not a legal requirement but a mandatory standard expected by banks and merchants for card transactions. There are fines for non-compliance as per the respective contracts between the company and the payment processor. Companies can usually validate their compliance through a self- assessment questionnaire.

The PCI DSS standards provide twelve guidelines to make the card transactions secure in an application, such as encrypting the transmission, having a firewall, updating anti-virus software, etc. So when you are testing, you should think of scenarios to protect the card details, such as masking the card details in UI and in all storage locations, restrictions to access the card data, avoiding card details in logs, and so on. A threat modeling exercise, as discussed in Chapter 7, shall come in handy here.

Payment Services Directive (PSD2) ​

PSD was the first implemented payment services directive in the EU region, which aimed to prevent online payment crimes. It also intended to increase competition in the payment industry apart from banks that monopolize payment services. PSD2 is an overhaul of the original PSD standards. It is mandated by law in the EU region for all payment service providers. If you are building an application that provides payment services to customers in the EU region, you should pay attention to PSD2 regulations.

PSD2 mainly focuses on Strong Customer Authentication (SCA) features and in extending the reach of PSD2 within and beyond the EU region, such as enforcing PSD2 even if one leg of the transaction involves the EU region. To be PSD2 compliant, options for the businesses are to choose an already compliant payment service provider like Stripe 11 , Paypal, etc., or to build the SCA features for payment services in their applications. In simple terms, SCA can be equated to multi-factor authentication. The European Commission defines SCA as an authentication mechanism that uses at least two of the following three verification elements:

• User’s unique knowledge of something, like a password.

• User’s unique possession of something, like a debit or credit card or mobile.

• User’s unique inherence such as the biometric identifiers—face or fingerprints.

Such features have to be tested thoroughly to be PSD2 compliant.

To summarize, the first step in compliance testing is to develop a thorough understanding of the legislation. Then, the respective CFR testing approaches from the five themes as discussed in the strategy section can be employed appropriately to holistically test them. Once the application is tested and ready, the legal team or an authorized entity shall get involved for compliance certification. Only on successful certification, the cycle of compliance testing shall be considered complete.

And by that, you are certainly equipped to test the tall list of CFRs that your application might require to thrive and enable your team to do continuous delivery by shifting CFRs testing to the left.

Perspectives: Evolvability and the Test of Time!

We have discussed how to harness quality by testing the functional and cross-functional requirements so far. It is important at this juncture to understand that software requirements are not set in stone once at the beginning of the project. As established before, software requirements change with market needs continuously and that is inevitable. Also, what is inevitable is that the new requirements almost always threaten the existing implementation when not shepherded wisely. For instance, a team member may hastily override encryption in order to improve performance and thereby compromise security of the application in entirety. This is the core premise of the book Building Evolutionary Architectures by authors Neal Ford, Rebecca Parsons and Patrick Kua, where they prescribe a new CFR, “evolvability”, which is the ability of the system to preserve the existing architecture characteristics (e.g., layered architecture, data persistence methods, encryption at rest and transit) that facilitate a given set of functional and cross-functional requirements, while incorporating new changes. In order to achieve evolvability, they recommend the implantation of appropriate guard rails for the essential architectural characteristics that may be non-tradeable, which shall provide the teams with instantaneous feedback whenever they deviate. These guard rails can be in the form of automated tests around each of the functional and cross-functional requirements such as performance tests, security scans, accessibility audit results, architecture tests, functional micro and macro level tests, and in the form of code coverage metrics, static code analyzer metrics and so on. This ensemble of tests and metrics, collectively referred to as fitness functions, 12 shall guide the teams in making incremental changes without compromising the existing implementation and in the process create an evolutionary architecture!

To consolidate the views presented here, all the functional and cross- functional testing methods and tools that we have been learning all throughout the book including what is bundled in this chapter, in turn, aid in building an evolutionary architecture that stands the test of time beyond imbuing high quality today!