End to End Testing

Introduction

End to end testing refers to the testing of the functional and data flow of an application consisting of several sub-systems working together from its start to end. As we know, the software applications of modern days are sophisticated systems. Usually, they consist of several sub-systems working together each performing a definite task. At times, these systems are developed in different technologies by different teams or organizations. Finally, they come together to form a functional business application.  Hence, testing a single system would not suffice. Therefore, end to end testing verifies the application from start to end putting all its components together.

Definition of End to End testing

End to end testing is testing a software application with all its interfacing systems together from its starting point to the endpoint for functional and data integrity. Undoubtedly, the objective of end to end testing is to simulate the production like environment for the system with all interfacing components working together. Clearly, end to end testing happens after the completion of unit, integration and system testing.

 

Why End to End testing

  • The software applications are complicated consisting of several sub-systems. Hence, if any one of the components breaks, the entire application will fail. Therefore, testing them together in real user scenario is important.
  • The components of the application can consist of varied technologies. Also, they can be developed by different teams or organizations.
  • System and regression testing verify only the system under test. But, they do not evaluate the integration of the system with outside systems. Subsequently, in the real environment, the system has to work together with other systems. Hence, end to end testing validates the flow from the starting point to the ending point.

Difference between End to End and System testing

Often, people get confused between End to end testing and system testing. Now that we already know what end to end testing is, let us consider the system testing. System testing is verifying the system under test as a whole giving relevant input to it and observing the output. Some of the key differences between both the testing are:

End to End TestingSystem Testing
This testing tests the application together with all its subsystems, network dependencies, servers, and databases.This testing tests a particular system as a whole after the integration of all its code modules.
It uses Production like real environment for the testingIt uses the QA test environment for the testing
Test data is a copy of real production dataThe data is any testing data setup on the QA environment for that particular system.
Test cases are the scenarios from the perspective of the userThe test scenarios verify the system’s features and functionalities from developers and QA testers perspective
End to end testing happens after system testingSystem testing happens before the end to end testing and after unit and integration testing
The test scenarios cover the application from its starting point to endpoint involving the subsystems.The tests only cover the system under test ignoring the dependencies on other interfacing systems.
It is a manual testingIt can be manual or automation testing.

Process of End to End Testing

Like any other testing, End to end testing also goes through formal planning, test execution, and closure phases.

End to end testing is done with the following steps:

Planning

  • Business and Functional Requirement analysis
  • Test plan development
  • Test case development
  • Production like Environment setup for the testing
  • Test data setup
  • Decide exit criteria

Pre-requisite

  • System Testing should be complete for all the participating systems.
  • All subsystems should be combined together to work as a complete application.
  • Production like test environment should be ready.

Test Execution

  • Execute the test cases
  • Register the test results and decide on pass and failure
  • Report the Bugs in the bug reporting tool
  • Reverify the bug fixes

Test closure

  • Test report preparation
  • Evaluation of exit criteria
  • Test phase closure

Test metrics

The tracing the quality metrics gives insight about the current status of testing. Some of the common metrics of end to end testing are:

  • Test case preparation status: Number of test cases ready versus the total number of test cases.
  • Weekly Test progress: Number of test cases executed in the week versus a weekly target number of the test cases. Thus, this shows week wise test cases execution progress through the percentage of test cases pass, fail, executed and the remaining number of test cases.
  • Defects Status: This metric represents the status of the defects found during testing. Defects should be logged into defect tracking tool and resolved as per their severity and priority. Therefore, the percentage of open and closed defects as per their severity and priority should be calculated to track this metric.
  • Test environment availability: This metric tracks the duration of the test environment used for end to end testing versus its scheduled allocation duration.

End to End Testing Framework

While designing an end to end testing framework, we need to consider three important aspects. These are

  1. User functions
  2. Conditions
  3. Test cases

User functions

User functions are the functions or activities a user can perform using the user interface of the application. As the end to end testing consists of real user scenarios, the first step is a list out what all user can do.

  • list out all the functions a user can perform
  • establish whether or not the functions are reusable
  • understand the input and output of the functions
  • the flow of data through the related subsystems
  • work out the relation between different functions

Conditions

Conditions are the different set of input data and other dependencies like environment settings which can cause the functions to provide different outputs. Thus, the behavior of the function depends on its inputs and other parameters.

  • list out the conditions for each user function
  • environment settings like type of system, time, date, network, input data, etc all together form different conditions which can affect functions behavior

Test cases

Test cases are the steps of doing the testing. Therefore, here we will make use of the above two steps, user functions, and conditions and come up with several different test cases.

  • write test case for every input scenario
  • all the behavior of the function should form different test cases
  • Test cases should cover all user functions

Example of End to End Testing

Let us take a simple example of an e-commerce web application. This application has:

  • a front end for the user interaction
  • central server
  • a database for inventory
  • users database
  • seller web application using which sellers can register and upload the items for sale.
  • payment gateway with several banks and online payment wallets

Let us come up with an end to end testing framework details for this e-commerce system

User functions

Some of the examples of user functions can be:

  1. Access the e-commerce web application
  2. Seller Login
  3. User login
  4. Edit profile
  5. Browse through the items
  6. Checkout

Conditions

For simple login user function, some of the conditions can be

  1. Valid user id and password
  2. Invalid user id and password
  3. A correct user id and wrong password
  4. Valid user id and password and wrong captcha
  5. Forgot password

Similarly, for other user functions like edit profile or checkout, there will be other corresponding conditions.

Test cases

So a few of End to end testing test cases can be:

Sn No.Test case details
1Seller will create a profile and register a product. Next, User will create a profile and login with correct details. Then, he will search for the product. After this, the user will add the item to the cart and do the checkout with payment.
2Seller will create a profile and register a product. User will log in with wrong details and will not be able to log in.
3Seller will update the product details. Next, the user will log in and go through the updated product. User will buy an item and make the payment. Then, the user will cancel the order and refund should be processed.
4Seller will add the item. User will buy the item. He would initiate a return request. The seller should receive the item back and refund the balance. The refund should get processed.

Conclusion

With new age software applications becoming more and more complex, thorough testing including all interconnected systems is necessary. Although the system under test can be working perfectly fine, other dependent systems may breakdown in a certain scenario to cause a failure. Hence, end to end testing comes to rescue verifying the entire application with all its interfacing components like other subsystems, network and firewall, servers, and databases.

References

https://en.wikipedia.org/wiki/Software_testing

Translate »