- Information
- AI Chat
Comparative Study of Software Automation
Software Engineering (CS530)
Visvesvaraya Technological University
Recommended for you
Preview text
ISSN : 2248-9622, Vol. 8, Issue 2, ( Part -3) February 2018, pp-
Comparative Study of Software Automation Testing Tools:
OpenScript and Selenium
Vaibhav Jain*, Dr. Kumar Rajnish**
*(M. in Computer Science (Research Scholar), Birla Institute of Technology, Mesra, Ranchi) ** (Associate Professor, Computer Science and Engineering Department, Birla Institute of Technology, Mesra, Ranchi)
ABSTRACT
Software Testing ensures delivery of good quality software to the customer. With the increase of product functionality, testing becomes challenging as it is time-consuming to perform manually, cost also increases as test suite size grows and human error can creep into a system which may lead to losses for the organization. Therefore, automation of software testing for the product is needed using appropriate Automation tool, which can enable developers and testers to easily automate the entire flow or process of automation testing. The objective of this paper is to conduct a comparative study of automated tools i. the OpenScript and Selenium- based on different criteria`s.
Keywords - Oracle OpenScript, OATS OpenScript, Selenium, Automation, Testing
----------------------------------------------------------------------------------------------------------------------------- ----------
Date of Submission: 28-02-2018 Date of acceptance: 13-03-
I. INTRODUCTION
Software testing is performed to make software defect free and to ensure that every effort used in development as per requirements works in a defined manner. It involves execution of a software component or system components to evaluate one or more properties and verify that it satisfies specified requirements or to identify differences between expected and actual results. Software Testing is of two types - manual and automation. Java-based application automation is been used for functional testing by using above mentioned two automated testing tools. Manual testing [1] is testing of the product by a human manually as per the procedures defined or documented to test the proper functioning of the software product. Manual Testers discover the imperfections manually which expects them to act as end client and utilize highlights of the application to guarantee its right and correct functioning. They follow a written composed test plan that leads them to an arrangement of imperative experiments. The issues with manual testing are, it is the extremely tedious process, not reusable, great human efforts required, and few mistakes may still remain unrevealed. Automation testing covers most of the issues of manual testing. Automation testing [2] automates the steps of manual testing using automation tools. Automating the test case execution for given
software is a need for software on time delivery as it improves reliability factor, saves time and human efforts increase productivity and also decreases cost in long run. It expands the test execution speed, make them more reliable, programmable, exhaustive, and reusable. Standard Practices [3] need to be followed for testing applications in correct manner.
II. PROBLEM STATEMENT
Various software testing automation tools are available currently in the market but, not every tool has the capabilities to perform all type of testing like Functional, Load, Performance, Unit etc. and lack capabilities to interact/detect most of software development frameworks. So, it is of utmost importance to understand the application to be automated and using appropriate software testing automation tool to simplify code and testing framework development. For this study, software testing automation tool are compared on different criteria`s which are discussed further [4] [5].
2 Literature Survey This section contains a brief mention of existing research papers which are used as references for this study. User Interface has changed a lot over recent years. The paper “Testing Tools (software)” by M. Lutz [8] surveys a set of tools that support the testing process in different ways. Some
RESEARCH ARTICLE OPEN ACCESS
ISSN : 2248-9622, Vol. 8, Issue 2, ( Part -3) February 2018, pp-
tools simulate the final execution environment as a way of expediting test execution, others automate the development of test plans, and still, others collect performance data during execution. In these tough economic times, more and better testing done faster. The automated testing tools facilitate higher quality and more productive testing, but acquiring such tools is often complicated. The paper “Evaluation and selecting testing tools” by Poston and Sexton [5] had proposed the evaluation criteria for selecting the testing tools that not only help the managers but technical team to select the appropriate automation tool. The paper “A survey on testing and reuse” by Torkar, Richard and Stefan [4] gives a survey which tries to give an account of what type of trends exist today in software reuse and testing. Book “Selenium Testing Tools Cookbook” by Unmesh Gundecha [6] and paper “Design and implementation in selenium ide with web driver” by Uppal, Nidhika and Vinay Chopra [9] gives a detailed in-depth about Selenium, its four components i. IDE, RC, Web Driver, and Grid. While official document of Oracle OpenScript [7] is programmers guide of OpenScript which describes primarily the technical implementations for automation engineer and about the tool. The focus was to try to find out how developers use different tools today and what is lacking, especially in the field of reuse and testing. The paper “Classification of Software testing Tools Based on the Software Testing Methods” by Khaled, Rafa and Mohammad [10] classify and distribute a set of testing tools over the types of testing (testing methods) for three types of software products (web application, application software, and network protocol). The paper “Logical capture/replay” by Silverstein [11] told us if there exists a reasonably well-structured system implementation, it is very easy to add in a mechanism to capture interactions with operations that system provides and to generate playback that are meaningful. Performing operation- centric capture/replay avoids many of the pitfalls of traditional GUI centric capture/replay. Paper titled “Comparative Study of Automated Testing Tools : Selenium, Quick Test Professional and Testcomplete” by H. Kaur and G. Gupta [12] has describes the comparative study between tools i. Selenium, HP Quick Test Professional and TestComplete functional testing tools on nine criteria`s.
III. EVALUATION STUDY
For this study OATS OpenScript 13.1.0 Build 363 and Selenium 3.4 versions were used. Comparison between these two tools is made on the basis of following parameters [12]:
- Recording Efficiency of Tool
- Capability of generation of scripts
- Hybrid-Driven Framework
- Test result report
- Reusability of code
- Execution speed of scripts
- Scripts Playback
- IDE features
- Licensing Cost
- Learning Ease
- Application Support
3 Recording Efficiency of Tool Both tools have capabilities of recording and playing back scripts [6] [7]. Using record feature, the tool will automatically insert the commands/code in the script. Both tools have easy access to controls while recording, Selenium recording engine icon present in the tool blinks which indicates it recording the user actions. Recording toolbar was also there which has all the controls. So, checkpoints can be easily applied, also add text and also see screen coordinates and window coordinates. Selenium provides different types of recording such as keyword, script, low level procedure based on screen or window coordinates, and HTTP task. Selenium and OpenScript both are good for web based applications/products automation only and are very stable. When the record button is pressed, application is started. It records all the actions as performed by user. Inserting checkpoint during recording in OpenScript is available.
3 Capability of generation of Scripts Selenium can generate various types of scripts i. C#, Java, Python, Ruby for Web Driver and Remote Control. But, OpenScript generates only Java code. Selenium do provide the capability for code conversion by Exporting Test Case or Test Suite to Ruby, Python 2, Java and C#.
3 Hybrid-Driven Framework Nowadays, Hybrid-Driven Framework becomes very important part of testing framework. Primarily, it involves use of combination of various other frameworks i. Functional Decomposition Framework, Data-Driven Framework, Keyword- Driven Framework and Page Object Model (POM). Combination of all these Frameworks/models are used these days and both tools support all these frameworks thereby supporting functional libraries to be part of Asset Scripts which will be used to call common methods. It is possible to make the scripts access the different sets of input data from external source line databases and CSV files etc. OpenScript [10] allows various types of Data Sources to be
ISSN : 2248-9622, Vol. 8, Issue 2, ( Part -3) February 2018, pp-
Figure 3 – Script Playback and Result`s Tab
3 Scripts Playback When script is played [11], it replays the user actions that were performed by the user during recording. If object is not recognized during replay, it gives the error message as object is not found. In our study, both tools played the scripts efficiently.
3 IDE feature OpenScript Integrated Development Environment (IDE) is having large feature sets which are similar to Eclipse IDEs. OpenScript [7] have two perspectives i. Tester Perspective and Developer Perspective. Selenium Suite has Selenium IDE which has limited features and WebDriver has extended features like that of OpenScript. OpenScript can interact with many browsers like Firefox, Internet Explorer and Chrome but Selenium WebDriver not only supports former mentioned browsers with support for Safari, Opera and Safari.
3 Licensing Cost Selenium [6] is Open Source software testing tool and hence no license fee is involved. But, OpenScript is part of Oracle Application Testing Suite (OATS) which has licensing cost involved and OpenScript is bundled with it.
3 Learning Ease Selenium is used for automating the web browser based applications and does not cater other desktop level application to a large extent and hence it can be learnt easily in small span of time but OpenScript has larger set of applications that can be automated and hence takes time to understand the tool capabilities and also it is in-house tool of Oracle and hence enhancements and support is available through organization portals only and new feature set will be released as per organization cycle. OpenScript has more features, and is more complicated than Selenium.
3 Application Support Selenium can automate browser based applications and has open community that supports it, due to support dependency on community future upgrades and patches will be limited and would be delayed. Browsers supported are Mozilla Firefox,
Internet Explorer, Safari, Opera and Chrome. Operating Systems supported are Microsoft Windows, Apple OS X and Linux. Programming Language and Frameworks are C#, Java, JavaScript, PHP, Python, R and Ruby. OpenScript supports application of both Desktop and Browser based applications. Its capability to work with Java Applications is highly stable and with Oracle Forms. But, with Application Development Frameworks (ADF) it is still in evolving state and is currently improving. Browsers supported are Internet Explorer, Firefox, Chrome (version 33 or higher), Safari and Microsoft Edge. On Linux platform only Firefox browser is supported. Microsoft Windows is the supported operating system. Programming Language supported is Java version 7 or higher.
IV. CONCLUSION
Automated software testing is beneficial for large organizations. Since, in long run functionality of software products scales and hence their complexity increases. Thus, both time and money saving are essential for fast delivery of product/application across patches. Selenium and OpenScript both are efficient tools for software testing automation with their own features. Selenium has easy to use UI and efficient playback and OpenScript do have rich UI but it do have larger applications that can be automated. Deciding which tool is better after taking into account various aspects i. technology stack used in Application Development. Selenium can be used for applications which are Browser based applications but OpenScript is best for automating Java based frameworks primarily with browser application as well.
ISSN : 2248-9622, Vol. 8, Issue 2, ( Part -3) February 2018, pp-
ijera DOI: 10/9622-0802032933 33 | P a g e
0
1
2
3
4
5
6
Recording Efficiency CapaďilitLJ of sĐript...
Hybrid-Driven Framework
Test result reportsReusability of Code EdžeĐution Speed of...
Scripts Playback
IDE Features Licensing CostLearning Ease
Application Support
Tool Comparison
OpenScript Selenium
REFERENCES
[1]. Craig, Rick David; Stefan P. Jaskiel (2002). Systematic Software Testing. Artech House. [2]. Dustin, Elfriede, Jeff Rashka, and John Paul. Automated software testing: introduction, management, and performance. Addison-Wesley Professional, 1999.
[3]. IEEE Standard for Software and System Test Documentation," in IEEE Std 829-2008 , vol., no., pp-150, July 18 2008 [4]. Torkar, Richard, and Stefan Mankefors. "A survey on testing and reuse." In Software: Science, Technology and Engineering, 2003. SwSTE'03. Proceedings. IEEE International Conference on, pp. 164-173. IEEE, 2003.
[5]. R. M. Poston and M. P. Sexton, "Evaluating and selecting testing tools," in IEEE Software, vol. 9, no. 3, pp. 33-42, May 1992 [6]. Selenium Testing Tools Cookbook, by Unmesh Gundecha, Packt Publishing, November 2012 [7]. Oracle® Functional Testing, OpenScript Programmer’s Reference Release 13.2.0, October 2017 [8]. M. Lutz et al., "Testing tools (software)," in IEEE Software, vol. 7, no. 3, pp. 53-57, May 1990. [9]. Uppal, Nidhika, and Vinay Chopra. "Design and implementation in selenium ide with web driver." International Journal of Computer Application 46 (2012): 8-11. [10]. Khaled Mustafa, Rafa E. Al-Qutaish, Mohammad I. Muhairat, "Classification of Software testing Tools Based on the Software Testing Methods", 2009 second International Conference on Computer and Electrical Engineering, 978-0-7695-3925-6, 2009 [11]. Silverstein, M. I. C. H. A. E. L. "Logical capture/replay." STQE Magazine 5, no. 6 (2003): 36-42. [12]. H. Kaur and G. Gupta, “Comparative Study of Automated Testing Tools : Selenium , Quick Test Professional and Testcomplete,” vol. 3, no. 5, pp. 1739–1743, 2013.
Vaibhav Jain " Comparative Study of Software Automation Testing Tools: OpenScript and
Selenium "International Journal of Engineering Research and Applications (IJERA), vol. 8, no.
2, 2018, pp. 29-33.
Comparative Study of Software Automation
Course: Software Engineering (CS530)
University: Visvesvaraya Technological University
