Software Test automation has been most controversial activity in any organization. In most of the organizations, it starts with lot of potential in reducing the test efforts drastically and magically, but ends up in spending unreasonably high amount of efforts in maintaining the developed automation suite. The ROI becomes non-achievable and finally management decides to switch back to manual testing. Some of the main reasons for failure of Software Test automation initiatives are: Continuous changes in requirements and UI object properties Requirement of highly skilled programmer (Java / C# / VB etc.) for developing automation scripts Less modularity in automated scripts Lack of co-ordination between manual testers and automation testers Higher development time required to write automation scripts Applications make use of mix types of objects (Web, Windows, CLI), which cannot be taken care by only one automation tool. We presented, an Integrated Hybrid Automation framework, built by Harbinger Systems, which addressed above mentioned challenges and expedites the automation process with least of programming skills to build a robust and modular automation test suite providing faster ROI.

1. © Harbinger Systems | www.harbinger-systems.com
Simplifying Test
Automation Using
Harbinger’s Automation
Framework
A Webinar by Harbinger Systems

2. © Harbinger Systems | www.harbinger-systems.com
Panelists
Mandar Kulkarni
Senior Manager
Marketing
Harbinger Systems
Avinash Magdum
General Manager
Quality
Harbinger Systems

3. © Harbinger Systems | www.harbinger-systems.com
Agenda
• Automation Drivers
• Why automation projects fail?
• HIHAF© Architecture
• HIHAF© Benefits
• Automation Life Cycle Using HIHAF©
• Creating Automation Script – Best Practices
• Case Study
• Sample Scenario
• Live Demo

4. © Harbinger Systems | www.harbinger-systems.com
Automation Drivers
Technology QA / TestingManagement Finance
• Increased user
and client
confidence in
application
• Reduction in
overall risk
• Aggressive
release
schedules
• Reduce time to
market
• Quick ROI
• Low running
costs
• Reduces money
spent on testing
• Quick Break-
Even point
• Increase
application
reliability
• Increase
software quality
• Higher test
coverage in less
time with few
resources
• Increase
productivity in
IT operation
• Increasing
complexities in
testing
environment
• Support for
multiple OS and
Browsers
• Increase in
supported
devices

5. © Harbinger Systems | www.harbinger-systems.com
Why Automation Projects Fail?
Continuous changes in requirements and UI object properties
Requirement of highly skilled programmer (Java / C# / VB etc.) for
developing automation scripts
Less modularity in automated scripts
Lack of co-ordination between manual testers and automation engineers
Higher development time required to write automation scripts
Applications make use of mix types of objects (Web, Windows, CLI), which
cannot be taken care by only one automation tool

6. © Harbinger Systems | www.harbinger-systems.com
HIHAF: Harbinger’s Integrated Hybrid
Automation Framework
Build automation scripts with no
programming knowledge and
skills
Improve automation script
development productivity by more
than 40%
Start automation script
development even before build
is available
Key Features

7. © Harbinger Systems | www.harbinger-systems.com
HIHAF© - Architecture
App -1
(Web)
App -2
(Win)
Web
OR
Win
OR
Test
Script
Test
Report
Mod-1
(CLI)
Selenium Web Driver Auto IT Driver
Harbinger Integrated
Test Driver

8. © Harbinger Systems | www.harbinger-systems.com
Benefits of HIHAF©
Writing automation scripts in plain English / Excel format
Allows to create a single of a scenario cutting across multiple
applications, having Web UI, Windows UI or Command line UI
Achieve highest level of reusability and minimize maintenance of
automation
Three levels of HTML reports – Summary, Detailed, Screen shot
Multi-browser support – Same scripts can be executed on any
browser – IE, FF, Chrome, Safari

9. © Harbinger Systems | www.harbinger-systems.com
Automation Life Cycle Using HIHAF©
Initiation Design Development Testing Execution Maintenance
Identifying Need
of Automation
Environment
Setup
Rationalization of
Test Suite
Test Data
Creation
Gathering UI and
Functional Change
Prioritization and
Batch Creation
Technical
Assessment
Object Repository
Creation
Test Script
Development
Single Mode
Execution Testing
Modifying, Adding
Libraries & Scripts
Test Data
Creation
Proof Of Concept
Identifying
Reusable
Components
Script Review
Batch Mode
Execution Testing
Testing Changes
Test Script
Execution
Scope Finalization
Library, Functions
Development
Implement
Review Changes
Fix Any Issues
Integrate to main
Test Suite
Report Analysis
Failure
Verification
Estimation and
Planning
Identifying Test
Data Requirement
Integrate to main
Test Suite
Defect Reporting
Deliverables
Automation
Assessment and
Coverage Report
Updated Library
document,
Data Creation
Document
Automation
Scripts Dictionary
Issue Log
Execution
Summary and
Detailed Report,
Defect Report
Updated
Documents

10. © Harbinger Systems | www.harbinger-systems.com
Creating Automation Script Using HIHAF©
Recommended workflow for script creation
• Get the detailed steps of the scenario to be automated
• Execute it manually at least once to ensure sequence and correctness of the steps
• Ensure that the start point and end point of every script should be same
– Launch the browser with main URL
– Import the appropriate data file
– Log in
– <Step 1> …. <Step n>
– Log out
– Close the browser
• Identify the variables
• Create appropriate data in the data file for each variable
• Use multiple rows of data, if the scenario is to be repeated for multiple sets of data
• For each step of scenario,
– Select appropriate keyword
– Specify appropriate object
– Specify the unique property, and its value from the Object Repository
– Provide data to the object by specifying the variable name (if required)
• Add validation / checks at appropriate places
• Take screen shots of the application at every new page

11. © Harbinger Systems | www.harbinger-systems.com
Case-Study
Accelerated Test Automation with Hybrid Framework

12. Company achieved enhanced customer satisfaction levels with the reduced number of leaked
defects and the best product quality.
 A US based high profile company in Network Security Domain
 They have a proprietary platform that has network, endpoint,
mobile and temporal event based malware detection and risk
modeling, as well as unparalleled threat feed data on malicious
IP addresses
Accelerated Test Automation with Hybrid Framework
 Within one week of customization, Herbinger’s Selenium based Hybrid
Automation Framework was deployed. This framework comprised of:
 The Keyword driven approach that simplified the scripting process (no
programming skills required)
 Data driven capability provided reuse of same scripts for multiple data
set
 Team was able to provide the script for every sprint during the agile
development
 Framework gives three levels of HTML reports- a test suite report for
management, detailed report of every test script in the suite for tester to help in
filing defects and screenshots to help the developer in defect fixing.
 Earlier the team was able to execute only 400 regression test cases (38%
coverage) within the stipulated timeframe and only on one browser. After
automation they were able to execute all 1500 test cases (100% coverage) in a
day’s time. It also allowed multiple browsers to execute automation scripts at
the same time.
www.harbinger-systems.com © Harbinger Systems rfi@harbingergroup.com
Calibri, 20, Bold
 Frequent patches and hot fixes due to insufficient test
coverage leading more defect leakage
 Company wanted Agile Testing Model to be implemented
requiring robust, efficient, and Frequent Testing
 Manual Testing and reporting incurred more time & effort
 Unable to cover all the scenarios for every maintenance
release
 Large number of permutations and combinations for Test data
 Support for multi-browser increased the testing scope further
Application Stack
 .NET 4.0 MVC4, AJAX, XML
 Web farming and Load balancer
 MySQL, Tomcat Apache
 Android for mobile platform
Automation Stack
 Selenium Web Driver, Java
 HHAF automation framework
 IE, FF and Chrome browsers
Automated Software Testing
Situation Harbinger Solution
Challenges
Tools & Technologies
 Increased the test coverage reducing defect leakage ratio. Thereby reducing the
overall cost of maintenance.
 Since the frequency of release cycle was one release every month, the above
benefits get multiplied twelve times a year and bring in higher cost savings.
 Improved the effectiveness of the testing cycle
 Provided high level and detailed level reports of all the tests, reducing test
management efforts and assisted in informed decision-making.
Benefits

13. © Harbinger Systems | www.harbinger-systems.com
Sample Scenario
Consider a sample scenario as below,
• Open the Offline Player Settings windows
• Verify whether the Enable Proxy Checkbox is checked or not
• If it is not checked then click on it to check
• Again check the value of the checkbox. If it is true, then send the proxy server and proxy port
values
• Change the Help URL
• Click on OK
• Access the Web URL of OLP
• Login to application and get the status of the Course
• Click on Progress link to check the progress
• Get the progress status and progress time which is displayed on UI
• Connect to database and get the status and progress time from DB
• Verify the progress time and status are matching with DB or not
• Click on Help link URL which is set to Harbinger-System URL
• Switch to help Window
• Close The Help window
• Again open the Windows application and click on Restore Default button
• Click on OK button

14. © Harbinger Systems | www.harbinger-systems.com
Live Demo

15. © Harbinger Systems | www.harbinger-systems.com
Q&A

16. © Harbinger Systems | www.harbinger-systems.com
Thank You!
Visit us at: www.harbinger-systems.com
Write to us at: hsplinfo@harbingergroup.com
Blog: blog.harbinger-systems.com
Twitter: twitter.com/HarbingerSys (@HarbingerSys)
Slideshare: slideshare.net/hsplmkting
Facebook: facebook.com/HarbingerSys
LinkedIn: linkedin.com/company/harbinger-systems
18