В докладе я расскажу, как был организован запуск автоматических тестов (appium/javascript) в gitlab CI для нативного Android приложения на каждый Merge Request. Опишу, как можно встроить автотесты в существующий процесс сборки, как правильно настроить запуск тестов в docker image (тесты бегут в TestObject облаке), как произошла интеграция с клаудом и какие результаты это принесло. Tech stack: Gitlab CI, kubernetes, android, appium, javascript, testobject.

1. Continuous test automation in
build pipeline

2. Hello!
I am Dmitry Lemeshko
Mobile QA Automation @ FrontierCarGroup, Berlin
@AutomateMeNow
Stackoverflow: dmle
2

3. Frontier Car Group
● 6 markets
● 3 web apps
● mobile apps
● ~20 microservices
3

4. Automation started
4

5. 5
1
Build APK
Build points to Staging
environment & uploaded
to Google Storage
Run tests
Specify branch, run tests
on emulators, analyse
results and report to
developers
2
Gitlab CI
Android
pipeline
Test
automation
pipeline

6. First result
● Android/iOS tests in same repo
● Regression testing on master
● Run on Staging env
● Use local emulators
6

7. Problems
● Hard to maintain tests
● No full control over test env
● Hard to scale
● Bugs are spotted lately
● Lack of interests from dev team
7

8. Lets think
What do we want to achieve?
8

9. Continuous
mobile test
automation
It sounds good, right?
9

10. New pipeline
10
1
Build APK
Build points to isolated
environment & uploaded
to Google Storage
Deploy services
Specified branch for each
service, services are up-
and-running
2
Run tests
Take recent APK and run
tests on random device
from market pool
Upload artifacts to gitlab
CI job
3
Attach artifacts
4

11. Steps to do
● Move tests to Android/iOS project repos
● Configure test environment from tests
● Configurable end-points in mobile apps
● Integrate device cloud service
● Build in android pipeline
11

12. Split & move:
12
Mobile tools iOS testsAndroid tests
● page objects
● tests & hooks
● wd.js wrapper
● setup, logging
● cloud utils

13. 13
Admin Web tests
New structure:
AUT
source
code
Dealer Web tests
Customer Web tests
Dealer iOS tests
Dealer Android tests
Web tools Tools (API/common) Mobile tools

14. Configure deployment from tests
● List services and branches
"services": [
{"name": "serviceDealer", "branch":
"master"},
{"name": "serviceAuction", "branch": "AUC-
270"}
]
● Specify env prefix
"prefix": "automation2.test14" 14

15. Custom mobile build for tests
That points to API with prefix configured before
15

16. Integrate cloud service
● Upload new builds from Google Storage
● Manage available devices
● Run tests in a single session*
● Handle cloud session time limit
● Set results and pull artifacts
16

17. Update pipeline
● Update docker image
● Add new job & gitlab CI runner
● Trigger for merge request:
○ Target branch is master
○ In sync with master
○ No WIP in title
○ Marked with `Reviewed` label
17

18. Update .gitlab-ci.yml:
deployAndUItests:
stage: build
script:
- export branch=$(echo $CI_BUILD_REF_NAME | tr
"[:upper:]" "[:lower:]" | sed "s/(da-[0-9]*).*/1/")
- bash /build.sh
...
- npm run deploy
- export appiumService=testobject env=automation
- npm run smoke
18

19. Update .gitlab-ci.yml:
artifacts:
name: "${CI_JOB_NAME}_${CI_COMMIT_REF_NAME}"
when: always
paths:
- tests/uiTests/screenshots
expire_in: 4 weeks
allow_failure: false
only:
- triggers
19

20. 20

21. Before
● Testing after MR
● Shared non-flexible test env
● Self support for appium &
devices
● Tests maintenance on QA
● Results reporting delay
What do we get?
Now
● Testing before MR
● Isolated configurable test env
● Cloud service
● Tests maintenance is shared
QA/Dev
● Results in build pipeline
21

22. Flexibility
Any environment, any version of services
Shared maintenance
Developers are fixing tests :)
Fast feedback
Developers get results faster & directly
22

23. 23
iOS project
coming next...

24. 24
Thanks!
Any questions?
You can find me at @AutomateMeNow