Watch a live presentation at http://offer.bitbar.com/parallel-test-runs-with-appium-on-real-mobile-devices
Appium is an open source test automation framework and currently one of the hottest framework for mobile app, game and web testing. In this webinar, we’ll discuss about the best practices, how to use Appium for different types of apps, games and web apps, and how to use all new features around it – the parallel test runs, image recognition and more!
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Testing tools and AI - ideas what to try with some tool examples
Parallel Test Runs with Appium on Real Mobile Devices – Hands-on Webinar
1. 12 May 2015
Ville-Veikko Helppi
Technical Product Manager
ville-veikko.helppi@bitbar.com
Parallel Test Runs with
Appium on Real Mobile
Devices
W E B I N A R
2. Agenda
• Testdroid Update
• Statistics from Q1'2015
• Testing and Mobile Test Automation
• Appium – Client vs. Server Side
Execution
• Demonstration
• Q&A
4. Public Device Cloud
on-demand devices
(multitenant)
Mobile app testing
on thousands of real
Android and iOS
devices hosted by
Bitbar
Private Device Cloud
Reserved Devices
Hosted by Bitbar in
the US and/or Europe
Devices chosen by
and reserved only for
the Customer
On-Premise
Device Cloud
Automated mobile
app testing devices
hosted by the
customer, usually
30-500 devices
1 Product – 3 Deployment Options
Testdroid Cloud Testdroid Enterprise Testdroid PrivateCloud
15. How Many Devices is Enough?
~90%
market coverage can
be achieved with
128
devices
~20%
market coverage can
be achieved with
12
devices
US Market
25 Android devices
= ~2/3 market
Global Market
60 Android devices
= ~1/2 market
18. Correct behaviour
across platforms and
browsers
Integration with web
back-ends
Typically need
to fully utilize
HW
(CPU+GPU)
Resource (e.g.
battery)
consumption
OpenGL ES 2/3
Functionality
and usability
Screen
orientations,
connectivity,
user profiles
Robustness
Robustness and
security!
Brand
Compliances,
verification with back-
ends and data
Different Mobile 'App Verticals'
21. Why Real Devices Are Must-to-Have
• Emulators cannot help you to test...
• User Experience
• Usability
• Hardware
• Software
• Infrastructure
0 % = the percentage of your app users
that use emulator to run your app!
23. Manual vs. Automation
Smaller coverage, More
money burnt & time
wasted, Error-prone
Manual Automation
Large
coverage,
quickly
completed,
Less money &
time wasted,
Exact results.
25. Family Tree of Android Test
Automation Frameworks
JUnit
Android Instrumentation Framework
Robotium
Espresso and
Espresso v2
uiautomator
Appium
ExtSolo
Calabash
26. What Framework Works You The Best?
Robotium uiautomator Espresso Appium Calabash
Android Yes Yes Yes Yes Yes
iOS No No No Yes Yes
Mobile web Yes
(Android)
Limited to x.y
clicks
No Yes
(Android &
iOS)
Yes
(Android)
Scripting
Language
Java Java Java Almost any Ruby
Test creation
tools
Testdroid
Recorder
UI Automator
viewer
Hierarchy
Viewer
Appium.app CLI
Supported
API levels
All 16 => 8, 10, 15- All All
Community Contributors Google Google Active Pretty quiet
33. Client Side Execution
Add Testdroid Desired
Caps to test script
{
“testdroid_username”: “user@domain.com”,
“testdroid_password”: “p4s$w0rd”,
“testdroid_project”: “My First Project”,
“testdroid_testrun”: “Test 1”,
“testdroid_device”: “iPad Mini 7.0.4 A1432”,
“testdroid_app”: “http://domain.com/app_v1.ipa”
.
.
“app”: “com.bitbar.testdroid.BitbarIOSSample”
}
Get a
Device Name
Go to
cloud.testdroid.com
34. Client Side Execution
driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps);
Point the Webdriver to
http://appium.testdroid.com/wd/hub
Add Testdroid Desired
Caps to test script
Get a
Device Name
Go to
cloud.testdroid.com
35. Client Side Execution
Run the Test ScriptGet Results from
Testdroid Cloud
Point the Webdriver to
http://appium.testdroid.com/wd/hub
Add Testdroid Desired
Caps to test script
Get a
Device Name
Go to
cloud.testdroid.com
36. Client Side Execution
Pull the Results from
the Result URL
driver = webdriver.Remote("http://appium.testdroid.com/wd/hub", desired_caps);
Run the Test ScriptGet Results from
Testdroid Cloud
Point the Webdriver to
http://appium.testdroid.com/wd/hub
Add Testdroid Desired
Caps to test script
Get a
Device Name
Go to
cloud.testdroid.com
42. Setup
• Using real Android devices at Testdroid Cloud
• Parallel test runs without a need to configure desired
capabilities
• Device groups (= set of devices used for runs) can be
manually created and configured
43. File Structure
• pom.xml (maven)
• testdroid.properties (overwritten after submitted to Cloud)
• run-test.sh (shell script for execution)
• image files