This report gives an analysis and evaluation of Testing tools like Selenium and JUnit used in Web Application Testing. An in depth study of Web Applications and the necessity to deliver them with the least number of errors is made. A sample web application is used to do the analysis and measure the efficacy of automating the test scripts. Selenium is a group of software tools which has distinct methods and approaches for automating the test script. This report analyzes the use of Selenium in different operations like locating UI elements and comparing the expected results to the actual results while running a script. The ability of Selenium to execute the same test scripts in different browsers is also analyzed here. The document elucidates the fact that Selenium is not just a tool but a combination of short concise applications which together constitute a Suite of Software’s. This suite is designed to cater to the needs of different requirements and customers depending upon the Testing strategy they adopt.
A study of how JUnit could be used for Unit Testing Web Applications is made here. JUnit is a unit testing framework. The use of JUnit in writing the test scripts that are intended for repeatable execution is made here. When the sample application is checked unit wise using JUnit, the bugs could be identified early in the Testing process. The Graphical User Interface of JUnit helps in makng the Testing procedure more easy by denoting the progress of testing in a bar that turns red if a test is failed. Testing the sample application with JUnit ensures that the software can be easily rectified of errors as soon as they are found. The main advantage of automating the test scripts using tools like Selenium and JUnit is that it increases the efficiency of the Testing Process to a great extent. Even the most time consuming and redundant Test Cases that are avoided normally during Manual Testing can be carried out if the test scripts are automated. This becomes all the more important in Web Testing as the cost and time involved is very crucial during project delivery.
The intent of Software Testing is to find the errors in a Software Application while it is being executed. These errors are called by different terminologies such as errors, bugs, deviations or defects (Bertolino, 2007). A complete and comprehensive testing strategy checks each and every requirement that the software application intends to satisfy. Both Verification and validation are two processes that ensures that software meets the business requirements of the client as well as it works exactly as it is expected to.
Software systems are not error free. The reasons and chances of program errors are plentiful. Thus companies around the world are spending a big portion of their resources to make the software systems robust, reliable, scalable, safe, secure and defect free. This is where the significance of an economical and decisive software testing methodology becomes important. In spite of the best effort in the design, development and coding of software programs, some errors may persist in the software after its completion. Ever increasing faults in the software applications at various stages of development has invoked interest in Software Testing. As a result, the importance of software testing as an integral part of the software development process has been steadily growing and it has been regarded as the standard technique in the Software Industry to ensure quality and verify the correctness of the software.
Formally, Software Testing is defined as “the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements and actual results”. Software testing plays a vital role in the software development process. Nearly one-third of the loss caused due to software errors can be easily eliminated by an improved testing infrastructure. A well designed testing process ensures that the software program performs as per the requirement under all conditions. The process of testing a software system is a humongous task which is not only time consuming but also very costly. It was estimated that software testing and debugging alone consume almost 50% of software system development resources. The quality of software can be enhanced by rigorously testing the program at the various stages of its development.
The types of software testing has been evolving continuously to match with the rapidity of increase in the complexity of software design, frameworks and programming languages.
Software Testing Methods
Manual testing is the process of testing the application manually by a person for detecting the software. It requires a tester to play the role of an end user where they randomly check the application by keying in values and see if the application responds properly. To guarantee completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.
Automation testing is a technique that automates the test cases using specific tools which can detect the bugs in less time. In Automation testing, the tester writes scripts by own and uses suitable software to test the software. It is basically an automation of a manual process intended to reduce the time and cost overhead in repeated testing.
A comparison of three basic software testing techniques is done in (Kuhn et al, 2004). The techniques used are analysing the software structurally by complete statement coverage, testing the code line by line after abstracting it and equivalence partitioning and boundary value analysis, which are used as methods in functional testing. The fault detection capabilities of these three methods are analyzed in this paper. It was found that all three methods fared differently based on the professional capacity of the testers. Comparatively, code reading had better results when used by professional programmers, which indicates its slight efficacy over the other methods.
A neglected aspect of Software Testing is that as the use of the application increases, the resources which hosts the application can behave erroneously over time. This problem is analyzed and addressed in (Kuhn et al, 2009). The paper evaluates the aspect of Combinatorial Testing, where the different combinations of the resources that host the application are determined and verified up to a pre specified level. This can help in detecting the possible errors early at the testing phase, as it ensures there would not be a problem due to the different combinations of resources used.
Regression Testing becomes mandatory for an application that has to undergo frequent changes. These changes may be due to the new requirements or new modifications to the existing application. It becomes mandatory to check the whole application each time a modification is made to ensure that the current application is not affected adversely by new changes. (Wong et al, 1997) elucidates the different methods by which Regression Testing could be made simpler by reducing the number of test cases required to be run each time a modification happens. This is done by prioritizing and minimizing the test cases and using only a subset of them that recognizes the deviations specifically.
The significance of using Web Applications and methods to improve their quality is analyzed in (Ricca and Tonella, 2001). A UML representation of the web application is proposed first. The UML representation helps in deriving the static site structure and also other high level details of the web application. These details could be used to generate a framework for white box testing and automate the test cases. Testing by this method increases the accuracy and reduces the effort in testing a Web Application. The effectiveness of using automated testing tools and a comparative study of these tools like Selenium, QTP and Test Complete is made in (Kaur and Gupta, 2013). The study evaluates the performance of these tools specific to requirements such as Black Box Testing, Functional Testing and Regression Testing. Each tool has its advantages based on the requirements and the type of application.
In (Holmes and Kellog, 2006) explains the use of Selenium as a tool of choice for testing Web Applications. The paper elaborates certain features of Selenium. Selenium as a tool has the ability to do Testing initially before the design and can determine when to start or stop User Acceptance Testing. It also has the capacity to provide automated test suites for Regression Testing. An analysis is also made on the duration required to write the scripts in development environment and how far the functional requirements could be covered completely by the tool. Apart from its use as an automation tool for Web Applications, (Kongsli, 2007) discusses the use of Selenium in Security Testing. Unmasking the susceptibilities of an application by creating test scripts that simulate anomalous behaviour is called Security Testing, which is discussed in length in the paper. Test scenarios and Test scripts are designed meticulously which demonstrate the use of the tool in creating such test scripts.
The use of a Keyword Driven Framework is explained in (Singla and Kaur, 2014) to automate the test scripts. A testing tool called ‘Selenium Webdriver’ is used to abstract the detailed functions necessary to automate the testing process. These concise functions are then populated into an excel file. The description as to how the program should run is also given in the excel sheet and the automation process is carried out more efficiently.
The different methods by which Unit Tests can be written in an improved way is investigated in (Cheon and Leavens, 2002). The paper elucidates the methods by which a runtime assertion checker is used to determine the correctness of the test scripts. The use of specifications which abstract the elaborate test scripts is also considered here. Specifications are used instead of descriptive test code which makes the the testing process more concise and readable. This specification enabled Unit Testing with JML and JUnit is examined in depth here. How test case prioritization can increase the fault detection capability of a test suite is analyzed in (Rothermel and Kinneer, 2004). An experimental setup where unit test cases are executed after test case prioritization is implemented here. The paper reiterates the fact that unit testing using JUnit after test case prioritization proves to be more cost effective. Prioritization ensures that the Test Cases are executed in a particular order which could reduce the overhead in Regression Testing. By using JUnit for Java programs, the fault detection capabilities of the Unit tests significantly improve.
Why selenium and JUnit for testing web applications?
Selenium is an open source tool that gives a framework for testing Web Applications (Singh and Tarika, 2014). The tool can be easily accessed, downloaded and used. Essentially, the Selenium tool is an add-on feature for the Firefox browser, and can be used to automate the test scripts written for a Web Application. The sample program selected here is the Mercury Tours Web Application that is provided along with QTP installation to learn the initial concepts of Web Application Testing.
Since it is a Web Application that is being tested here, a cost effective, simple and compact technique for testing should be adopted. Selenium meets the criterion of being cost effective, since it is an open source tool. Selenium also supports a variety of platforms, languages and browsers which makes it very handy for the testers to work with. As with other modern tools, Selenium also has an active community based support for its users. It is a package of different tools intended to address different testing issues and hence referred to as a Test Suite. It is relatively easier to use and since the scripts run directly on the browsers, the results can be seen by the users (Bruns et al, 2009). The reporting feature of Selenium is also user friendly and allows the testers to analyze the results and take necessary steps to rectify them.
The basis of Selenium lies in a client – server architecture. The client component comprises of a WebDriver API and a Remote WebDriverClass. The WebDriver API is used to interact with the elements of the application while the Remote WebDriver class acts as an interface and corresponds with the Selenium server. Similarly the server comprises of a server component that receive requests, a WebDriver API that executes the test scripts on the server and a SeleniumGrid which ensures the cross browser and cross platform capabilities of the tool.
While testing the Mercury Tours Web Application, simple scripts can be used to run the tests directly within the browser. Thus the same test scripts could be used to test the same functionality in different browsers. For testing the login page of the sample application, the Selenium IDE provides a number of context menu’s and within the context the number of available commands could also been seen. There are certain assertion statements which ensure that the action performed is correct. The test scripts could be abstracted to HTML tables which make the testing process more concise and efficient. The dynamism of Selenium is illustrated by the Selenium IDE, also called the recorder which helps in the user to navigate and simulate the tests which may be done manually (Choudhary et al, 2010). The server in Selenium also gives the option to write the test with any program which enables different functionality like conditional logic to be integrated into the automated testing process (Gundecha, 2012).
There is a sequential order by which a test script is designed and executed in Selenium. First a WebDriver instance is created for the sample Mercury tours Web Application. Then an HTML element like an input box for the username is selected after navigating to the page. Once the HTML element is selected an action could be carried out on that particular element. Then the tests are run on the elements and the results are recorded. By creating test scripts for the sample application using Selenium, it can be determined that this tool is one of the best in the industry that can perform even miniscule user events. Because Selenium allows a great deal of freedom for the Tester to write their own logic, it provides the chance to do complex operations (Pajunen et al, 2011).
JUnit is a unit testing framework for Web Applications in Java. The concept of Test Driven Development is relevant while discussing JUnit in Unit Testing. Test Driven development is a Software Development process, where the requirements are converted into Test Cases and the code is rewritten to meet these requirements. As the name suggests, Test Driven Development lays emphasis on Testing early during the development phase, which thereby reduces the cost and effort in tracking down the bugs at a later stage.
Unit Testing is the process of verifying a single unit in an application to see if it is running without any hitches. Unit Testing could be termed as the initial step in testing a Web Application, by confirming that the independent units within the application works fine by themselves, before an integration of these units are made. By integrating these units, modules are made, which in turn when grouped together form the entire system. As testing these basic building blocks results in verifying the correctness of the program, it initiates the developer to scrutinize his code in a more meticulous way.
For a standalone application, it becomes mandatory to call the programs main() method to do any verification in the code level. Similarly in a Web Application, to ensure that the flow is working correctly, the program has to be run in the Server. Hence testing the Web Application becomes a gruesome task, as each time an error is uncovered, the Server has to be restarted, and the changes in the application has to get reflected in the Server. By using JUnit, these issues can be addressed. Practically, during web development, JUnit test cases are implemented in the DAO classes. The advantage of using these DAO classes is that it can be tested and verified without the help or presence of the Server. This makes JUnit a very handy tool for testing the Web Applications, where there is no need to keep verifying whether the changes has to be incorporated in the Server.
Web Testing is nothing but the process of checking the web application for errors before the application is delivered to the client or shifted to the production team for making it live. Since a variety of users will be accessing the Web Application it is absolutely mandatory to make sure that the Application reaches the end users in a standard and presentable manner. The Functional Testing of a Web Application ensures that the all the specifications and the functional requirements of the application is met correctly. Some of the basic functionalities that has to be checked and taken care of, while a Web Application is being released is that to see if all the web links like outgoing, mailto, anchor and internal links are working properly. Apart from this it also has to be taken care of that all the web forms are working as expected. A few things that are ensured during this process is that if all the mandatory fields are not populated it throws relevant exception messages. And also that the default values are populated and the values that are being entered into the forms are populated in the database as well.
Selenium proves to be the best among all the other tools that captures almost all the activities that a user performs in a browser. Since Selenium has the capability to add logic by using programming concepts, it is the tool of choice for many Testers in the industry. In addition to that Selenium and JUnit are open source tools which make it economical and viable for using them at will. Advantages like being portable and being cross browser and cross platform friendly makes it the tool of choice for automating the test scripts in Regression Testing. The fault detection capabilities of the tool is also analyzed and found to be better than most other tools in the industry now
- Bertolino, A. (2007). Software testing research: Achievements, challenges, dreams. In 2007 Future of Software Engineering(pp. 85-103).
- Kuhn, D. R., Wallace, D. R., & Gallo, A. M. (2004). Software fault interactions and implications for software testing. IEEE transactions on software engineering, 30(6), 418-421.
- Kuhn, R., Kacker, R., Lei, Y., & Hunter, J. (2009). Combinatorial software testing. Computer, 42(8).
- Wong, W. E., Horgan, J. R., London, S., & Agrawal, H. (1997). A study of effective regression testing in practice. In Software Reliability Engineering, 1997. Proceedings., The Eighth International Symposium on(pp. 264-274).
- Ricca, F., & Tonella, P. (2001). Analysis and testing of web applications. In Proceedings of the 23rd international conference on Software engineering(pp. 25-34).
- Kaur, H., & Gupta, G. (2013). Comparative study of automated testing tools: Selenium, quick test professional and testcomplete. International Journal of Engineering Research and Applications, 3(5), 1739-43.
- Holmes, A., & Kellogg, M. (2006). Automating functional tests using selenium. In Agile Conference, 2006(pp. 6-pp).
- Kongsli, V. (2007, October). Security testing with Selenium. In Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion(pp. 862-863). ACM.
- Singla, S., & Kaur, H. (2014). Selenium Keyword Driven Automation Testing Framework. IJARCSSE ISSN, 2277.
- Cheon, Y., & Leavens, G. T. (2002, June). A simple and practical approach to unit testing: The JML and JUnit way. In ECOOP(Vol. 2374, pp. 231-255).
- Do, H., Rothermel, G., & Kinneer, A. (2004). Empirical studies of test case prioritization in a JUnit testing environment. In Software Reliability Engineering, 2004. ISSRE 2004. 15th International Symposium on(pp. 113-124).
- Singh, I., & Tarika, B. (2014). Comparative analysis of open source automated software testing tools: Selenium, sikuli and watir. International Journal of Information & Computation Technology, 4(15), 1507-1518.
- Bruns, A., Kornstadt, A., & Wichmann, D. (2009). Web application tests with selenium. IEEE software, 26(5).
- Choudhary, S. R., Versee, H., & Orso, A. (2010, September). A cross-browser web application testing tool. In Software Maintenance (ICSM), 2010 IEEE International Conference on(pp. 1-6).
- Gundecha, U. (2012). Selenium Testing Tools Cookbook. Packt Publishing Ltd.
- Pajunen, T., Takala, T., & Katara, M. (2011, March). Model-based testing with a general purpose keyword-driven test automation framework. In Software Testing, Verification and Validation Workshops (ICSTW), 2011 IEEE Fourth International Conference on(pp. 242-251).