Testing is a part of software engineering process. You wouldn't promote a code you haven't tested to production, so why don't you use automated testing and actually reduce costs and increase the efficiency?
You can also learn how doing testing can put pressure on your code to improve your architecture.
10. public class SaiyanSagaTest {
public static final int GOKU_POWER_LEVEL = 10000;
public static final String EXPECTED_GOKU_OUTCOME ="It's over nine thousaaaaaaaaand!";
// public Saiyan(KiiSensingInterface sensingMethod)
Saiyan goku, vegeta, nappa;
@Before
public void setUp() {
// KiSense and Scouter classes implement KiSensingInterface
goku = new Saiyan(new KiSense());
nappa = new Saiyan(null);
vegeta = new Saiyan(new Scouter());
}
@Test
public void testVegetaReadsGokusPowerLevel() throws Exception {
goku.setPowerLevel(GOKU_POWER_LEVEL);
assertThat(EXPECTED_GOKU_OUTCOME, vegeta.scanPowerLevel(goku));
}
@After
public void tearDown() {
nappa.finish();
}
}
16. public class Tard {
// Initialize rest of Tard, it was awful
public String getResponse() {
return “nope”;
}
}
public class TardTest {
Tard tard = new Tard();
// Look ma’, I’m testing stuff
@Test
public void testGetTardResponse() throws Exception {
assertEquals(EXPECTED_NOPE, tard.getResponse());
}
}
17. public class Fry{
public void shutUpAndTakeMy(Money money) {
if (money != null)
shutUpAndTakeIt(money);
}
}
public class FuturamaTest {
@Mock Account fryAccount;
@Mock BankFactory bank;
@Spy Fry fry;
// I don’t want to live with this testing anymore
@Test
public void testIDontHaveAnyMoneyToTake() throws Exception {
when(fryAccount.getMoney()).thenReturn(null);
verify(fry, never()).shutUpAndTakeIt(fryAccount.getMoney());
}
}
18. public class Snape {
public int pointsToGryffindor(Student student) {
if (Student.DANIEL_RADCLIFFE.equalsTo(student))
return -10;
else if (Student.EMMA_WATSON.equalsTo(student))
return Integer.MAX_INT;
return 10;
}
}
public class HogwartsTest {
public static int MAX_POINTS = Integer.MAX_INT;
@Spy
Snape snape;
// Bullshit! 2,147,483,647 points to Gryffindor!
@Test
public void testSnapeGivingPointsToHermione() throws Exception {
when(snape.pointsToGryffindor(((Student) any()))).thenReturn(MAX_POINTS);
assertEquals(MAX_POINTS, snape.pointsToGryffindor(Student.EMMA_WATSON));
}
}
20. MOCKING & STUBBING UNIT & INTEGRATION ACCEPTANCE
Mockito JUnit Espresso
JMock Robolectric Robotium
PowerMock Calabash
Additionally JaCoCo can help you on keeping an eye on code coverage.
26. public class MarceusWallace {
public void initialization(){
// Lots of stuff concerning intialization
this.looksLikeABitch = !(Locale.WHAT.equalTo(this.brett.getLocale()) &&
Locale.ENGLISH.getDisplayLanguage().equalTo(
this.brett.getLocale().getDisplayLanguage()) &&
this.jules.describeMarceusWallace(this.brett) &&
this.brett.getWhatCounter() < 7);
}
}
public class MarceusWallaceTest {
@Test
public void testMrWallaceDoesNotLookLikeABitch() throws Exception {
// Testing all cases would lead to 24 cases plus every branch of the method
when(brett.getLocale()).thenReturn(whatLocale);
when(brett.getWhatCounter).thenReturn(5);
when(whatLocale.getDisplayLanguage())
.thenReturn(Locale.ENGLISH.getDisplayLanguage());
when(jules.describeMarceusWallace(brett)).thenReturn(Boolean.TRUE);
controller.initialize();
assertFalse(controller.doesLookLikeABitch());
}
}
27. public class MarceusWallace {
public void initialization(){
// Nice refactor!
this.looksLikeABitch = calculateIsLookingLikeABitch();
}
}
public class MarceusWallaceControllerTest {
@Test
public void testMrWallaceDoesNotLookLikeABitch() throws Exception {
// Much better! But what if Jules and Brett are final or cannot be mocked
// that would be still a lot of boilerplate!
when(controller.calculateIsLookingLikeABitch()
.thenReturn(Boolean.FALSE);
controller.initialize();
assertFalse(controller.doesLookLikeABitch());
}
}
28. public class MarceusWallace {
public void initialization(){
// Now I can test calculateIsLooking injecting its dependencies!
this.looksLikeABitch = calculateIsLookingLikeABitch(this.brett, this.jules);
}
}
public class MarceusWallaceControllerTest {
@Test
public void testMrWallaceDoesNotLookLikeABitch() throws Exception {
// Using a spy and injection I can narrow the SUT to the initialization
// method. We still have to do 24 cases for testCalculateIsLooking...
when(controller.calculateIsLookingLikeABitch(brett, jules))
.thenReturn(Boolean.FALSE);
controller.initialize();
assertFalse(controller.doesLookLikeABitch());
}
}
29.
30. public class GruePresenter {
public GruePresenter(Context context){
Crashlytics.start(context);
Log.d("GruePresenter", "Crashlytics Initialized");
// Even if you injected this. It would still be wrong
mixpanelAPI = MixpanelAPI.getInstance(context, MIXPANEL_TOKEN);
new YourFavoriteAsyncTask().execute(new DoSomethingNetworky());
throws new FacepalmException("You got eaten by a Grue!");
}
}
public class GruePresenterTest {
@Test(expects = FacepalmException.class)
public void testInitialization() throws Exception {
GruePresenter presenter = new GruePresenter(Robolectric.application);
// Your code is baaaaaaad and you should feel baaaaaaaad
}
}
31. public class GruePresenter {
// Remember, AsyncTask.execute is final!
public void doSomethingAsyncish(){
new YourFavoriteAsyncTask()
.execute(new
DoSomethingNetworky());
}
}
public class CrashlyticsAdapter {
public void start(Context context) {
Crashlytics.start(context);
}
}
public class Logger {
public void debug(String tag, String msg){
Log.d(tag, msg);
}
}
public class MixpanelAdapter {
// You can inject an instance of Mixpanel
public MixpanelWrapper(MixpanelAPI api) {
this.api = api;
}
}
32. public class GruePresenter {
public GruePresenter(CraslyticsWrapper crashlytics, Logger logger,
MixpanelWrapper mixpanelWapper){
crashlytics.start(context);
logger.debug("GruePresenter", "Crashlytics Initialized");
this.mixpanelWrapper = mixpanelWrapper;
// We had to move the doSomethingAsyncish out of here
}
}
public class GruePresenterTest {
@Test
public void testInitialization() throws Exception {
GruePresenter presenter = new GruePresenter(mockCrash, mockLogger,
mockMixpanel);
doNothing().when(presenter).doSomethingAsyncIsh()
// Alright, let’s dance!
}
}
34. 2.397 Tests
● 2.364 units
● 33 integration
● 1.942 Android - coupled
~2:30m full test suite
~70% Coverage
Maven 3.2.3
● JUnit
● Mockito
● Robolectric
59 users, 7 crashes
~35K lines of code
35. 1. We are paid to solve problems. Be efficient.
1. Tests test what your code does, not what it should do.
1. If it is not your code, there is no need to test it.
36.
37. Questions?
“Let me answer that question with a question: Who wants to make 60$?”
38. Bibliography
● xUnit Test Patterns: Refactoring Test Code, Gerard Meszaros
o http://xunitpatterns.com/
● 467 tests, 0 failures, 0 confidence, Katrina Owen
● Mocks aren’t stubs, Martin Fowler
● Eveyrthing you wanted to know about testing, but were afraid to ask, Jorge
Barroso
● Exploding Software-Engineering Myths, Janie Chang
First question we ask is: “Why do I have to test?” We keep daily struggling against PMs, Designers, Platforms, Bugs, Stakeholders… in the end of the day what we want to do is to finish what we’ve assigned to and go home and relax.
Let’s talk about philosophy. What does Testing provide you as a developer?
Testing is about economy. Investing early on testing during your development phase will help you identifying potential issues on your code earlier, saving time going back later to fix those errors.
Additionally the work is kept there forever, so you can ensure you keep it safe. If you refactor or change that code in the future, your test suite will help you find out if it keeps compatible.
Adding tests after a successful bugfix will also allow you to extend your suite further and avoid regressions.
Testing is about control.
Doing testing on your code as a part of the software development process is a good way to have control of that code. You know how the code is going to behave in several situations if you know can predict how your code behave both in input and output, you can be more efficient on knowing what’s failing and fix it faster than what you have.
Testing is about learning. Many of you will have heard the sentence: “You need no documentation, but a good naming and a test suite“.
A good test suite can help you understand faster what the underlying code does and how it does behave even if you have no idea on how it is done.
You can test whole blobs of software you don’t have the code of if you are able to set up testing for it. Also, automated testing puts pressure in your code in the first line of defense which is you. You understand perfectly the code underneath and if the test is complex, has lots of dependencies or basically smells, you can refactor it before it reaches QA, Production and the final user.
Let’s talk about the basics of testing.
Every test is divided in four phases:
on Setup test you prepare the fixture of the test, this is the conditions of the test. In order to perform a sucessful test we have to have a controlled start point which will leads to an expeced output. Setup stage can occur on a “global” test state or be particulary customized by every test. Not all tests are created equal.
In the excercise phase we actually execute the piece of code we are intending to test.
Then we verify the results. We can do it on several ways, but the important part is again that a controlled input must have a determnistic output, this is, that the test is repeteable for the same conditions with the same results.
And in the end we clean our stuff, if the tests requires to (eg. large information in memory,etc..)
When testing we should see the code we test no more than a box. This box can be composed of one or several players, and it is called System Under Test (SUT).
Our testing should be confined exclusively to this box, and box alone. This is one golden rule of testing.
Now we can apply two approaches to our test: Black box testing or White box testing.
Black box aim is to check the state of the SUT after the execution phase is testing. It is implementation-agnostic.
White box testing aim is to check that the right components were called. This makes these kind of tests more vulnerable to implementation changes and require more maintenance. However some code cannot be verified other way.
In general black box testing require less maintenance than white box ones and remember that test is software coupled to our code, and as such, it also evolves and needs maintenance when out code evolves.
Example of test.
A Saiyan requires to be built a KiSensingInterface.
In order to prepare our test, we create a Goku, Vegeta and Nappa instances of Saiyan. Goku senses the Ki through a learnt sensing system, but Vegeta uses a Scouter instead. Nappa is in the middle of the fight and is unable to sense.
The test will be about testing Vegetas’s correct response to scanning Goku’s power level and finding it over 9.000.
Individually we configure Goku’s power level to 10K.
We exercise the test and return the result.
Finally Nappa is finished after this test.
This is a black-box test. We don’t care what sensing interface Vegeta uses, nor how does the Scouter is implemented. If we switched for any other KiiSensingInterface or how Vegeta senses it, we would expect exactly the same result with the same conditions.
Fortunately you have some classes that provide help on managing your testing.
First you have the stub. Stubs are “fake” classes that provide canned responses. These will help you during set up and exercise phase, aka “fixture” to let the state of the SUT be in the proper one when you start testing.
Mock is used only for verification purposes.
Additonally you can have a “partial mock” or “spy” which is a real implementation of the object but can be switched down.
There are several levels you can test.
Unit tests individual classes.
Integration test several layers at the time.
Acceptance should test only particular cases of your user story.
Now we’re going to talk about efficiency. Or inefficiency. While testing we find we have to be effective on doing those test.
So, what actually qualifies a test as good?
Deterministic, AKA trustworthy. We cannot have tests in our suites which results vary depending on factors like order of execution, or because we have a race condition.
Test must be fast. The most valued resource of a developer is time. You don’t want to keep waiting for a 2h long suite to finish to go on.
Easy. Tests are programs meant to test other programs. The more code you write, the more bugs you will introduce, no matter how good you are.
But is there a way to measure how reliable our test code is?
NOOOOOOOOOOOOOPE!
Example 1:
Tard is a very lively cat which doesn’t like fun stuff. So, everytime he does something he always answers “nope”.
So we make a test to test that Tard what’s his response. His response is always nope. It’s a thrilling test. It actually passes, but really we lost like two minutes of our life setting up the test, but it does not contribute much to our test bench.
Example 2:
Fry wants to buy one of those cool eyePhones. But might happen that Fry does not have money to take. In that case, it would be stupid for him to yell at the cashier that he needs to take his money.
So we do a test to ensure what happens when Fry has no money to take. We set up the test and verify the results.
Yay! it passed! But, there isn’t anything strange there?
We forgot to actually excercise code. This test took also some time to prepare and it actually passes. But nothing is being executed, so actually has no point on our test suite.
Example 3:
Snape is about to give points to the students of Gryffindor. And he already has his score prepared whatever he does.
He hates Harry because his mother ended up with one of the guys he disliked the most, so he will remove 10 points from Gryffindor everytime Harry passes by.
However he’s heard that Hermione is some kind of sex symbol of something muggles calls “the interwebz” so, yeah, why not giving Hermione the best score possible?
We set up that text and actually we executes it. It passed again. But Snape is in dismay. He’s not being tested. His stub is being tested in his place. Damn you the-one-who-cannot-be-names!
We’ve talked about how to perform testing but what’s the tools we have at our disposal?
Mockito, JMock and PowerMock are tools we can use for mocking and stubbing.
JUnit is a classic Java framework for unit testing.
Robolectric is a Java VM implementation of the Android Framework. It helps you simulating you’re actually running code on the Dalvik VM with no emulator. Robolectric holds all the information regarding the enviroment (Context, Resources) and additionally provides shadow classes to expose some internal states some classes of Android have to allow better testing.
Espresso, Robotium and Calabash are acceptance frameworks. Espresso is the framework Google recommends. Robotium it’s been there for ages. And calabash is a scripting framework for doing the same stuff.
This section is called “F**k tips” and it is based on personal views and techniques which I believe might speed up the testing process.
Unit testing executes faster than any other kind of test, and the tests are easier because the SUT is smaller. around 90% of the issues you will find in your code can be detected early on testing phase.
Besides if you control all the input and the output of your classes, there will not be any surprises, and the sum of all your unit testing might go finer grained than a batch of integration.
A disclaimer here is Singletons. Singletons is something they sell you as a pattern, but it is actually an antipattern. It turns stateful classes into static ones. This means that once a class has been instantiated in the process, it won’t die unless you kill the process, and as the Tests live all around the execution it is impossible for you to return the Singleton to a clear state. You have then two solutions: create a singletonmanager which handles singleton-like instances which actually are not or have a reset method which is kind of spaghetti.
UI cannot be trivially tested, especially in how it is drawn. Of course you can know the sizes or the state of a View but, a visual inspection is required. Otherwise the amount of verification you’d have to do for UI would be crazy.
RC are hard to test by themselves and highly predictable.
Laplace, a precursor of the theory of games.
One of the biggest nuisances you’ll find during testing writing is to carry different state machines around your tests. The more complex your class is, the more state it will have. Additonally if you rely too much on integration tests you can find yourself in a combinatorial explosion.
But we have a solution...
Injection can ease your pain. And I mean injection at all levels.
BTW, when creating tests please use a semantic notation that allows you to understand at a glance what a code does like testCorrectLoginWithToken or testControllerCleanupOnCloseFlag instead of testLoginGood, test536 or testControllerAgain.
In this example, Jules and Brett are talking how about Marcelus Wallace looks like. There are four conditions that needs to be fulfilled to know if Marceus Wallace looks like a bitch.
If the controller needs to test several initializations, carrying all those stupid initializations back and forth. Let’s exercise a very easy refactoring there.
Injecting the inner dependencies on the Controller methods, I can excercise easier the calculateIsLookingLikeABitch even if I cannot access directly the instances.
That way, I can isolate methods of my class to test it without affecting the rest of the more general tests.
Do you know what a Grue is? In old Text Adventures a trick the developers had was to summon a Grue when the player went astray from the main playing area. He would be killed by a full of fang, claws and fur beast that were lukring in the dark. Call that a negative reinforcement! Nice lesson of the old masters: Do not go beyond your designated area.
What’s the matter with this code. Well, of course it could be improved. But the key here is that we’re going beyond our SUT. Our ultimate SUT is the boundaries of the application itself. Of course we have to put up with some issues like the need for Java code, but your job will never be to test Android network framework or the database.
In this case we’re accessing things outside our SUT like Crashlytics, Mixpanel or the Logger. Not like anything catastrophic is going to happen, but a rogue test suite can start sending mixpanel events and pervert the metrics or access the API and leave inconsistent data or even generate a DDoS on your own servers.
There are some techniques you could use for avoiding going beyond your SUT.
For instance, if your library forces you to use static methods or access the library directly, wrap it in a class to remove that staticity. Classic movement by using Adapter pattern. This class is under your control and can be mocked, stubbed and verified.
Use constructor for dependency satisfaction only. Especially in testing you cannot Spy an element before its instantiation. Extract a method to call the request and execute it inmediately after the constructor. This will make no difference on application flow, but might improve on Interface Segregation and testing enviroment.
This is the result using Method injection with mocks and removing the async task. This class is ready for testing.
So, I’ve been telling you a lot of stuff, but how do we actually work at Fever?