FitNesse is a wiki-based software testing tool that can be a powerful addition to your Continuous Integration Environments. Its greatest advantages include providing visibility into tests and results, and providing access to test-writing by non-technical team members. We will: * look at specific examples and code, * discuss the advantages and drawbacks of using FitNesse as a test framework * implement, deploy, and use a simple fixture in a fitnesse test * review different kinds of fixtures, including decision table, script, query, html, and selenium webtest fixtures * discuss some of the more interesting fixture extensions we've implemented, including JSON-based verification and the ability to pass in javascript code for dynamic verification * use Hudson/Jenkins to run your FitNesse tests as a step in your Continuous Integration/Deployment process

1. Fitnesse and Continuous Integration
Jennifer Wong | Staff SQE Engineer | twitter: @jenlwong
Proprietary and Confidential

2. Overview
►Intro
► What is FitNesse (Not an exercise program!)
► The Details
► Integration: putting the pieces together
► Lessons Learned
Proprietary and Confidential

3. Intro
• Who:
– Jennifer Wong
– Scrum Master for Tools Team, Staff SQE Engineer
• Where:
– Ingenuity Systems: A leading provider of information
and analytics solutions for life science researchers
• What:
– FitNesse as a key element of Continuous Integration
workflow
• Why:
– Follow up to last year’s presentation
(http://www.slideshare.net/jenlwong/ingenuity-svcc-ci-presentation-20111007 )
Proprietary and Confidential

4. FitNesse
(Not an exercise program!)
Proprietary and Confidential

5. FitNesse
• FitNesse is a wiki-based web server test tool
– Helps abstract test definition from technical implementation
– Provides visual reporting and result history tracking
Proprietary and Confidential

6. Test Types: FitNesse
• We use it for:
– Integration tests
– Acceptance and Functional tests
– UI Tests (com.jbergin.HtmlFixture, webtest)
– DB Tests (dbfit)
– Backward Compatibility tests
• What is it good for?
– Framework and visibility
– accessibility to non-technical people
• What is it bad for?
– Unit tests
– Complicated ui tests
– Extensive performance testing
Proprietary and Confidential

7. FitNesse: The Details
Proprietary and Confidential

8. Downloading and Installing FitNesse
• Get the jar file and run it
– http://fitnesse.org/FrontPage.FitNesseDevelopment.DownLoad
– java –jar fitnesse.jar
• Or use the demo package:
• Get the file: on the svcc web site, attached to the session, or
on Github
• unzip the file and cd to lib dir
• start the fitnesse server: java -jar fitnesse.jar -p 8080
• open web browser and access http://localhost:8080
• click the link at the top for "CodeCampDemoPage“
• click the test button
Proprietary and Confidential

9. Setting your classpath
• Classpath statements
– Fitnesse needs to know where to find your fixture code
• This kind of error :
• Means you need to add this kind of statement: !path
– You should usually have this kind of stuff:
!contents
!path fitnesse.jar
!path C:/eclipse/fit_demo/eclipse-bin
!define TEST_SYSTEM {slim}
Proprietary and Confidential

10. Some Basic Test Tables
• Script table
– Good for procedural/workflow tests
– Flexible
– Use syntax: check, reject, ensure, show
• Decision table
– Good for data-driven tests
– Specific workflow
• Input methods  execute()  output methods
– Special method name interpretation
– Automatically calls reset() and execute() methods
• Query table
– Good for validating lists or tables of data
Proprietary and Confidential

11. Variable Usage
• Defining a static variable
– !define ROOT_URL {http://myserver.com}
– !define TESTUSER {testuser1@something.com}
• Storing a value in a variable on the fly
– Store variable: $X=
– Use variable: $X
• Using a variable
– To use this variable, enclose the variable name in ${ }
– Example:
• this: ${URL_ROOT}/context/index.html
• Will resolve to this: http://myserver.com/context/index.html
Proprietary and Confidential

12. Naming and Parameter passing
(Methods)
• (Un) Graceful Naming
– Automatically concatenates space-separated
words
• isHalloween  isHalloween()
• Is Halloween  isHalloween()
• is halloween  isHalloween()
• Is halloWeen  error
– When using methods with multiple
parameters, tries to intersperse method
name and paremeters
Proprietary and Confidential

13. Parameter passing (cont)
• Multi-parameter methods: isHalloween(int, String)
– |ensure|is|31|Halloween|October|
– |ensure|is Halloween|31||October|
• Single parameter: setCostume(String)
– |set costume|Clark Kent|
– |set|Clark Kent|costume|
• Constructors with parameters
– This constructor:
• public Halloween(String month, int day, String costume)
– Translates to this usage in a fitnesse table:
|Halloween|October|31|Cat|
|isHalloween?|get surprise?|
Proprietary and Confidential

14. UI Test Fixtures
• com.jbergin.HtmlFixture
– an adapter between FitNesse and HtmlUnit for use in
testing web applications
– Need to use !define TEST_SYSTEM {fit}
– http://htmlfixture.sourceforge.net/
– http://uebuild5:8084/FrontPage.UmaFitNesse.IngsecuritySuite.ConcurrentUserS
essionTest.AcceptanceTests
• webtest selenium
– an extension to FIT/FitNesse that uses Selenium Remote
Control. WebTest runs inside FitNesse.
– http://www.fitnesse.info/webtest
– http://uebuild5:8084/FrontPage.ReportsFitNesse.IsoformView.IsoformVi
ewWebTestSuite.IsoformViewWebTests
Proprietary and Confidential

15. Real world usage is more complex
• What it looks like in the real (ie, complicated)world
– Session handling
• http://uebuild5.ingenuity.com:8084/FrontPage.UmaFitNesse.IngsecuritySuite.Concurren
tUserSessionTest.AcceptanceTests
– Static objects to provide data access
• http://uebuild5.ingenuity.com:8084/FrontPage.ContentserviceFitNesse.TestSuiteForCurrentContent.Ec
sMappingDataProviderTestSuite.P1Tests
– Complex checking of validity
• Unmarshaling JSON to check special conditions in a non
order dependent way
– http://uebuild5.ingenuity.com:8084/FrontPage.FaFitNesse.Test
SuiteForBaselineContentSpecific.FaProviderTestSuite.Execute
FaQuery.LfaQueryTestSuite.AcceptanceTests
Proprietary and Confidential

16. Fixture code can get complicated
very quickly
public static boolean matchGFAResult(JSONObject jsonResult, GFAResult actualResult, boolean allowSubset,
boolean allowPvalueVerification,boolean geneCountVerification ,boolean allowZscoreVerification, boolean allowGeneEffectVerification) throws
JSONException {
JSONArray jsonItems = jsonResult.getJSONArray("items");
logger.info("expected item size = " + jsonItems.length());
logger.info("actual item size = " + actualResult.getFAResultItems().size());
if (jsonItems.length() > actualResult.getFAResultItems().size()) {
return false;
}
Map<String, GFAResultItem> itemMap = buildGFAResult(jsonItems);
if (allowSubset) {
for (Map.Entry<String, GFAResultItem> entry : itemMap.entrySet()) {
logger.info("Look for " + entry.getKey() + " in actual result");
GFAResultItem item = entry.getValue();
if (!containsItem(item, actualResult.getFAResultItems(), allowSubset, allowPvalueVerification,
geneCountVerification,allowZscoreVerification, allowGeneEffectVerification)) {
logger.info(item.getId().getAsString() + " is expected but couldn't be found in actual result");
return false;
}
}
} else {
return equalGFAItems(itemMap, actualResult.getFAResultItems(), allowSubset,allowPvalueVerification,
geneCountVerification,allowZscoreVerification, allowGeneEffectVerification);
}
return true;
}
private static boolean equalGFAItems(Map<String, GFAResultItem> itemMap,
Collection<GFAResultItem> actualResultItems, boolean allowSubset,
boolean allowPvalueVerification,boolean geneCountVerification ,boolean allowZscoreVerification, boolean allowGeneEffectVerification) {
if (itemMap.size() != actualResultItems.size()) {
return false;
}
Proprietary and Confidential

17. Tips and tricks
• Search in your FitNesse wiki
• Use Includes
– Use includes as templates
– http://uebuild5.ingenuity.com:8084/FrontPage.IngtestFitNesse.StableSui
te.FaStableCompat
– http://uebuild5.ingenuity.com:8084/FrontPage.IngtestFitNesse.StableSui
te.ContentserviceStableCompat
• Comments
• Escaping special characters
– Start tables with ! to avoid unwanted interpretation of
graceful names, etc
– Surround special chars with !- -!
• Example: !-gobbledeygook ~!@#$%^&*(){}| as plain string-!
Proprietary and Confidential

18. Fancy fixtures and other nifty stuff
• JSON
– http://uebuild5:8084/FrontPage.MgFitNesse.TestSuiteForBaselineContentSpecific.GraphProviderTestSuite.GetNeighb
orhoodGraph.P1Tests
• Javascript validation
– http://localhost:8080/FrontPage.AutocompFitNesse.FunctionalTests.FitTests.GeneralTests.P1Tests
EVAL {
void execute(Parse row, JSONFixture fixture) {
Parse textCell = row.parts.more;// row.parts.more;
String evalText = textCell.text();
String text = fixture.page.getWebResponse().getContentAsString();
try {
jsEngine.eval("result = " + text + ";");
Object evaluationResult = jsEngine.eval(evalText);
if (evaluationResult instanceof Boolean) {
if ((Boolean)evaluationResult){ fixture.right(textCell);}
else {fixture.wrong(textCell); }
• Running tests based on tag
• Include: http://<host>:<port>/<suite path and test name>?responder=suite&suiteFilter=smoke,critical
• Exclude: http://<host>:<port>/<suite path and test name>?responder=suite&excludeSuiteFilter=NotRunningOnHudson
Proprietary and Confidential

19. Test Variations
• What we’ve done with it that is different
– Use as execution framework for more
complex tests
– Extension of fitnesse server for data-driven
tests
– json fixture – pass in javascript
– Execution of Selenium tests
– Backwards Compatibility tests
Proprietary and Confidential

20. Best practices
• Test robustness
• Test organization
• Test readability
• Fixture design
– Tradeoff between flexibility and readability,
usability
Proprietary and Confidential

21. Integration: putting the pieces together
Proprietary and Confidential

22. Our Environment
• Multiple products (3 external, plus internal tools)
• Services-based
• Builds:
– Produce multiple artifacts, including a fitnesse package
– Example: A build of contentservice produces:
• contentservice-1.2.179886.clover.tar.gz
• contentservice-1.2.179886.tar.gz
• contentservicedb-1.2.179886.tar.gz
• contentservice_fitnesse-1.2.179886.tar.gz
– Fitnesse package contains wiki page tests, libs, config files
Proprietary and Confidential

23. FitNesse as part of our
Continuous Integration Workflow
Application Deploy
Bundle Application
Run Fitnesse
Nightly Run Junit, Tests
Build Javascript (Nightly suite)
(Clover) Tests Fitnesse Deploy
Bundle Fitnesse
publish
publish
Hudson Dashboard Fitnesse Wiki
Link (Test history, Details,
(JUnit, Fitnesse summary, Code Coverage)
Test Case Management)
Commit
SVN (Test Cases)
Proprietary and Confidential

24. Integration with Hudson/Jenkins servers
• Fitnesse plugin:
Proprietary and Confidential

25. Lessons learned
• Adds a lot of value for our team
– Visibility into results and test history
– Accessible to non-technical people
– FitNesse is very good for visibility and straightforward
verification of data
• Not good for everything
– Easy to do it wrong
– Requires maintenance
– Not as flexible
• To do more, you have to get creative
• Fixture and test ownership needs to be a shared
responsibility
Proprietary and Confidential

26. • Demo files will be posted to github under jwong-github
• Slides are on slideshare
• Demo and slides are attached to session
• Q&A
Proprietary and Confidential

27. The “As Seen By” Matrix
Proprietary and Confidential