SlideShare une entreprise Scribd logo
1  sur  50
Télécharger pour lire hors ligne
School of Electronic Engineering
and Computer Science
Final Report
Supervisor: Stefan Poslad
Project Title:
Automatic Test Data
Generator For
Web Service Testing
Student Name: Muhammad Yahya
Date: 20/04/2015
Programme of study:
BSc Computer Science
Supervisor: Stefan Poslad
Supervisor: Dr Mustafa Bozkurt
Declaration of Authorship
I, Muhammad Yahya, declare that this thesis titled, ‘Automatic Test Data Generator For
Web Service Testing’ and the work presented in it are my own. I conform that:
 This work was done wholly in candidature for an undergraduate degree at this
University.
 Where any part of this thesis has been previously been submitted for a degree of
any other qualification at this University or any other institution, this has been
clearly attributed.
 Where I have consulted the published work of others, this has always been clearly
attributed.
 Where I have quoted from the work of others, the source has always been given.
With the exception of these such quotations, this thesis is entirely my own work.
 I have acknowledged all main sources of help.
Signed: Muhammad Yahya
Date: 20/04/2015
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 3 of 50
Abstract:
Web service testing is a process of finding bugs and errors in the web services. There are
different methods of testing web services. Web service testing is usually performed manually
which is very costly and time consuming. There are also number of web service testing tools
currently available in the market. Some of them are free open-source, such as SoupUI free
version, WebInject, Membrane, TestMaker and SOAPSonar and some of them can be bought
such as IBM® Rational® for SOA Quality (RTSQ) and SOAPUI NG Pro. To my
knowledge, most of these commercially or widely available web service testing tools does
not have automatic test values generator. SOAPSonar, RTSQ and few academic works does
provide automatic test values generator, but they are not very useful. I have created a web
service testing application which allow user to enter constrains and the test values are
generated based on those constrain. This testing tool is very flexible in terms of test values
generator. Everyone now a days is creating web services for their applications, this tool will
allow them to test their web services easily, thoroughly and quickly.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 4 of 50
Table of Contents
Introduction................................................................................................ 7
Objectives .....................................................................................................................7
Motivation......................................................................................................................8
Challenges....................................................................................................................8
Summary.....................................................................................................................10
Background ............................................................................................. 11
Web services ..............................................................................................................11
WSDL..........................................................................................................................13
SOAP request and response......................................................................................15
Regular Expression ....................................................................................................15
ASCII keys ..................................................................................................................16
XML Data Types .........................................................................................................16
Different types of testing .............................................................................................16
Summary.....................................................................................................................17
Critical review of the existing applications................................................ 18
Existing applications ...................................................................................................18
Advantages and limitation of FAWST.........................................................................20
Summary.....................................................................................................................21
Requirement specification........................................................................ 22
Primary functional requirements.................................................................................22
Secondary functional requirements ............................................................................23
Summary.....................................................................................................................24
Application Overview, Design and Application Walk Through .................. 25
Application Overviews ................................................................................................25
Design.........................................................................................................................26
Application Walk Through...........................................................................................28
Summary.....................................................................................................................36
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 5 of 50
Testing..................................................................................................... 37
Unit testing..................................................................................................................37
Boundary testing.........................................................................................................39
User Interface Testing ................................................................................................39
Acceptance Testing ....................................................................................................41
Platform Testing..........................................................................................................42
Summary.....................................................................................................................42
Conclusions and recommendations......................................................... 43
What I learned and achieved......................................................................................43
What would I add or do different.................................................................................43
Conclusion ..................................................................................................................45
Bibliography 46
Appendix 48
How FAWST stores test constrains in the Array Lists ................................................48
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 6 of 50
Figures and Tables
Figure 1.1 who uses website and web service [6]............................................................................ 11
Table 2.2.1 WSDL four major elements .......................................................................................... 13
Figure 1.2 Web Service Description Language [2].......................................................................... 14
Figure 1.3 Main structure of a WSDL document [3] ....................................................................... 14
Figure 2.3.1 SOAP request............................................................................................................... 15
Figure 2.3.2 SOAP response ............................................................................................................ 15
Table 2.6.1 commonly used XML data types .................................................................................. 16
Figure 5.1.1 Flow chart .................................................................................................................... 25
Figure 5.2.2.1 Model View Controller [10] ..................................................................................... 27
Figure 5.3.1 Initail window after FAWST is executed .................................................................... 28
Figure 5.3.2 Closing confirmation box ............................................................................................ 28
Figure 5.3.4 Window for String data type constrains....................................................................... 30
Figure 5.3.5 Window for Double data type constrains..................................................................... 31
Figure 5.3.6 Window for Enum data type constrains....................................................................... 31
Figure 5.3.7 Window for Char data type constrains......................................................................... 32
Figure 5.3.8 Window for Boolean data type constrains................................................................... 32
Figure 5.3.9 Window for Date data type constrains......................................................................... 33
Figure 5.3.10 Help Window for Date data type ............................................................................... 33
Figure 5.3.11 Custom Double window with error message ............................................................. 34
Figure 5.3.12 Custom Integer window............................................................................................. 35
Figure 5.3.13 Custom String window .............................................................................................. 35
Table 6.1.2. Unit testing................................................................................................................... 38
Figure 6.2.1 ...................................................................................................................................... 39
Figure 6.2.2 ...................................................................................................................................... 39
Table 6.3.1 Result from user interface testing.................................................................................. 41
Table 6.4.1 Acceptance testing summary......................................................................................... 42
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 7 of 50
Introduction
Web service testing is a process of finding bugs and errors in the web services. There are
different methods of testing web services and most of them are usually performed manually
which is very costly and time consuming. Now a days there are different machines and
software available to check and test the items we use in our daily life such as, cars, tyres,
and drugs etc. This manual process of testing is performed by the software testers, therefore,
the quality of the web service rely on the software testers’ ability and experience. Software
testers have to spend a huge amount of resources and time to make sure that the web service
is tested properly and rigorously.
To address the issue of manual web service testing I decided to create a tool which will test
web services automatically. This software is called Flexible Automatic Web Service Testing
(FAWST) and it will generate unique test cases based on user constrains.
Objectives
The main objectives of this project are following;
 Validate and parse WSDL
 Generate test data for any XML data type
 Generate test cases automatically with
o Any number of test cases
o Each test case should be unique
 Automatically send SOAP requests
 Analyse SOAP response
 Generate detailed test report
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 8 of 50
Motivation
There were number of different things that motivated me to do this project such as;
 Software Testing: Software Testing is my hobby, I test software for fun and I love
doing it. The reason why I love software testing is because it is a challenging task. I
have to look at software from a different angle and come up with all the different test
cases and scenarios. The joy and satisfaction of finding and fixing the bugs is also
very motivational for me.
 Software quality improvement: Poorly tested software can have a lot of
consequences such as security issue, loss of money and time, reliability and even
death or injury. With the help of FAWST, developers can develop better web services
which will contain minimum bugs. As a result, users will get more robust web service
to use.
 Knowledge enhancement: In this project most of the things were completely new
to me. Learning about new things and facing new challenges is very motivational for
me. It also gives me courage and self-esteem when I learn new things.
 Future Directions: I will be looking for job as a software tester. This project has
enhanced my chance of becoming a good software tester and it will open many
opportunities for me in the field of software testing.
Challenges
Testing web services is more challenging than testing traditional software due to the
complexity of Web service technologies and the limitations that are imposed by the Service-
Oriented Computing environment [1]. Also most of things in this project were completely
new to me because of this, I faced number of challenges right from the beginning.
1. Starting point – Starting of any project is difficult especially if something is new.
In my case I had the same issue because automatic test case generating is completely
new, I was not really sure how I should do this and I had nothing to compare it to.
2. API – Finding a right API for parsing WSDL and for SOAP request and response
was one of the biggest challenge I faced in this project. I spent almost six to seven
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 9 of 50
weeks on finding the correct API which can parse WSDL send SOAP request and
receive SOAP response. I went through all the major APIs such SOAP-UI, Selenium
and Wsdl4j. SoapUI and Selenium APIs are very complicated. In my point of view,
they are also very poorly documented. These APIs have their own forums and it is
almost impossible to find any information or get any help. Finding the right API
made the whole project very difficult because I spent a lot of time on it.
3. Complexity – By complexity I mean I had to use number of different java internal
and external libraries to achieve objectives of this project. I have also used NetBeans
GUI builder to create user interface. Understanding and changing NetBeans GUI
builder code is a very hard and complex task. Other libraries include XML parsers,
random number generator and date generator etc.
4. User Input – Finding a useful and simple way of getting test constrains from the
user is not easy task. I have to cover all the possible inputs. It was a difficult task but
most of the test constrains are covered. For example, for integer data type, number
can be a random, between two numbers, specific number or custom (combination of
specific, random and between two numbers). I had to get test constrains for all data
types.
5. Storing test input constrains – As FAWST allows the users to generate any test
inputs, this means user can do following:
1. Twenty characters long string
2. 1st
character is ‘a’
3. 2nd
character is between ‘h’ and ‘z’
4. 3rd
character is random
.
.
.
20. 20th
character is ‘4’.
It is just one example illustrated above; a string could be any characters long. I had
to store all this information because the test case data is generated based on this
information. Finding a good solution was also very hard. At the end, I used Array-
Lists and my own type of regular expressions to solve it.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 10 of 50
This project is like a star, looks small from far, but as you get closer and closer it gets bigger
and bigger. In the beginning of this project, it looked fairly simple and easy to do but as I
progressed through, it became very hard and very complex.
Summary
This chapter introduced FAWST and what are the main objects of the FAWST. I have also
talked about my motivations behind do this project and what challenges I had to faces while
working on FAWST.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 11 of 50
Background
This chapter explains in detail all the background research I have done for this project such
as web services, WSDL, XML data types, SOAP request and response, regular expressions,
ASCII keys and different types of testing.
Web services
Web service is a service which is available on web for application consumption/use and a
website is available on web for human consumption/use. In simple words web service
provides a way of communicating between software applications over the network, web
services can run on a variety of platforms and frameworks [25].
For example, on Twitter website we can post and read tweets but if a computer wants interact
with Twitter to post and read tweets, it has to use Twitter web service. We can write some
code which will allow the computer to access Twitter data though web service, this data then
can be analysed by computer without any human interaction. Similarly if we make weather
app, all we have to do is write some code which will automatically get data from met office
web service and then we can display it on our app however we like. We can see from the
above examples, that web services are very useful because they provide any easy way of
communication between two applications.
Figure 1.1 who uses website and web service [6]
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 12 of 50
There are two types of the web services, both do the same thing but they have their own
advantages and disadvantages. It is totally up to the programmer which technique suits
him/her most.
2.1.1 REST
REST stand for Representational State Transfer, it relies on stateless communication
protocol, typically HTTP. In the REST web services, data and functionality are considered
resources and they are accessed by using Uniform Resource Identifiers (URIs), typically
links on the Web [5]. This means that each unique URI is a representation of some object.
REST web service allows you to get and modify content though HTTP GET, POST, PUT or
DELETE methods. REST web services are lightweight, maintainable, and scalable [9]. A
web services based on REST are called RESTful services.
REST approach will work best in the following situations:
1. Limited bandwidth and resources - because the return structure can be in
any format but it is usually JSON which is very lightweight.
2. Totally stateless operations - it is good for stateless operations such as
Create, Read and Delete.
3. Caching situations - data can be cached because of the totally stateless
operation of the REST approach [12].
2.1.2 SOAP
SOAP stands for Simple Object Access Protocol. It is a communication protocol which
allows communication between applications. SOAP is platform and language independent
protocol and it is based on XML. It is simple and extensible and allows you the get around
fire walls, and it is also recommended by W3C [7].
As SOAP is platform and language independent, it provides a platform for communicate
between applications running on various different operating systems, with different
programming languages and technologies. SOAP web services needs to have WSDL [7].
SOAP approach will work best in the following situations:
1. Asynchronous processing and invocation: SOAP approach provides high
level of reliability and security
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 13 of 50
2. Formal contracts: “If both sides (provider and consumer) have to agree on
the exchange format then SOAP 1.2 gives the rigid specifications for this type
of interaction”[12]
3. Stateful operations: SOAP approach is good for stateful operations. Stateful
means the computer or program keeps track of the state of interaction [13]
WSDL
The Web Services Description Language (WSDL) is an XML-based language which is used
to describe the functionality offered by a web service. A real life example of a WSDL can
be a takeaway leaflet where we see all the different items that a particular takeaway offers.
Also Chinese and Indian takeaway leaflets will have different items and different phone
numbers to call them. Similarly WSDL tells us what each web service is offering and how
can we contact this web service. WSDL document can be divided into four major elements
shown below in table 2.2.1.
Element Description
<types> Data type of the parameter. Different data types are explained below
under heading 2.6 XML data types.
<message> Each message can consist of one or more parts. The parts are like
the parameters of a method in a traditional programming language
[3].
<portType> PortType is set of operation offered by a web service.
<binding> “A protocol and data format specification for a particular port type”
[3].
Table 2.2.1 WSDL four major elements
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 14 of 50
In the below Figure 1.2 we can see how each part of the WSDL are linked together and also
it show what the sub parts of the four main elements.
Figure 1.2 Web Service Description Language [2]
The main structure of a WSDL document looks like Figure 1.3. WSDL can also have other
elements such as extensions elements and service elements that allow us to group together
the definitions of several web services in one single WSDL document [3].
Figure 1.3 Main structure of a WSDL document [3]
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 15 of 50
SOAP request and response
1. SOAP Request is sent to invoke a method on the web service. Request call has to
have the parameters of the method. The request has to be in the XML format which
includes all the information about the request. See Example below in Figure 2.3.1.
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:example="http://www.webserviceX.NET/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<example:ChangeAngleUnit>
<example:AngleValue>10</example:AngleValue>
<example:fromAngleUnit>radians</example:fromAngleUnit>
<example:toAngleUnit>degrees</example:toAngleUnit>
</example:ChangeAngleUnit>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Figure 2.3.1 SOAP request
2. SOAP Response is sent back from the web service method, which includes return
value if the request was correct else error is sent back. A response is also sent back
in the XML format, which makes it easy to parse and use. See the Example below in
Figure 2.3.2.
<?xml version="1.0" encoding="UTF-16"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soap:Body>
<ChangeAngleUnitResponse xmlns="http://www.webserviceX.NET/">
<ChangeAngleUnitResult>572.95779513082323</ChangeAngleUnitResult>
</ChangeAngleUnitResponse>
</soap:Body>
</soap:Envelope>
Figure 2.3.2 SOAP response
Regular Expression
A regular expression is a string which describes a string pattern. Which is used to produce
new strings based on the pattern or validate existing strings such as “b [A-Z0-9._%+-
]+@[A-Z0-9.-]+.[A-Z]{2, 4}b” this regular expression checks if someone has entered the
correct format e-mail address.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 16 of 50
ASCII keys
ASCII stands for American Standard Code for Information Interchange. ASCII is an integer
representation of the characters for example alphabet “a” can be represented as number 97.
XML Data Types
WSDL uses XML define data types, in table 2.6.1 we can see all the main commonly used
XML data types but it is not a full list. There are many more XML data types which are
based on data types in table 2.6.1.
Data Type Description
String String data types are used for the values which contain sequence of
character. Such as, sSsd, hello-world and 354sdse%3 etc.
Integer Integer is sequence of number ranging from -2147483648 through
2147483647.
Double Double is sequence of number with decimal point. Double hold 64 bit
long numbers.
Float Float is similar to Double but it can only hold 32 bit long numbers.
Char Char is for single character. For example A, b, % and s etc.
Enum Enum data type contains a list of acceptable values.
Date and time Date and time data type are for the values that contains date and time.
Boolean Boolean data type is for true or false.
Table 2.6.1 commonly used XML data types
Different types of testing
1. Unit testing – Programmer test each individual software components and parts. This
is mostly done by the programmers themselves.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 17 of 50
2. Functional testing – “This type of testing ignores the internal parts and focus on
the output is as per requirement or not. Black-box type testing geared to
functional requirements of an application” [27].
3. End-to-end testing – It “involves testing of a complete application environment in a
situation that mimics real-world use, such as interacting with a database or using
network communications”.[27]
4. Sanity testing – “Testing to determine if a new software version is performing well
enough to accept it for a major testing effort. If application is crashing for initial use
then system is not stable enough for further testing and build or application is assigned
to fix”[27].
5. Load testing – “It is a performance testing to check system behaviour under load.
Testing an application under heavy loads, such as testing of a web site under a range
of loads to determine at what point the system’s response time degrades or fails”. [27]
6. Stress testing - “System is stressed beyond its specifications to check how and when
it fails. Performed under heavy load like putting large number beyond storage
capacity, complex database queries, continuous input to system or database load”.[27]
Summary
In this chapter, I have explained in detail what web service is, what are the different types of
web services and what is SOAP request and response is. I have also explained what WSDL
is and what data types WSDL accepts and I also talked a little bit about regular expressions,
ASCII keys and different types of software testing.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 18 of 50
Critical review of the existing
applications
This chapter presents a review the existing applications, both commercially and non-
commercially available. It also includes a review of FAWST and it shows in detail what
functionality FAWST does and does not provide.
Existing applications
There are number of web service testing tools currently available in the market. Some of
them are free open-source such as SoupUI free version, WebInject, Membrane 1
, TestMaker
and SOAPSonar and some of them can be bought such as IBM® Rational® for SOA Quality
(RTSQ), prices starts from $2995 and SOAP UI NG Pro €449 per year. To my knowledge,
most of these commercially or widely available web service testing tools do not have
automatic test case generator. SOAPSonar, RTSQ and few academic works such as WSTD-
Gen does provide automatic test case generator [21, 22, 23, 24].
Basically most of these widely available web service tools provide graphical user interface,
which allows the users to import and check WSDL, create and edit test cases manually, run
the test cases and see the result. I have personally used two of the web service testing tools
and their details are provided below;
1. SOAP UI
SoapUI is free and an open-source cross-platform functional testing tool for web
service, and is known as “Swiss-Army Knife of Testing”. SoupUI has well designed
and structured GUI. SoapUI allows the user to create and run automated functional,
regression, compliance, and load tests. It supports both SOAP and REST web service
approach. SoapUI also allows the user to run Functional/Load Tests and MockServices
from a task scheduler with a set of command-line tools. SoapUI received a number of
awards: ATI Automation Honors, 2009 , Jolt award in 2014 for the best testing tool
.InfoWorld Best of Open Source Software Award, 2008 and SOAWorld Readers'
Choice Award, 2007 [13] [16] [15].
1
www.service-repository.com
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 19 of 50
2. Service-repository.com
Service-repository.com is browser based web service testing tools. It provides very
basic web service testing functionality such as WSDL validation, send SOAP request
and display SOAP response. Users have to manually enter values for each test case one
by one. User can only test SOAP web service. This tool is only good for basic checks
and testing, it will be of no use if user wants to test their web service rigorously.
Some of these web service tools can generate test cases automatically. SOAPSonar utilizes
XSDMutation, they have submitted a patent for automation technique which is pending.
SOAPSonar generate a set of test cases, which are both negative and positive. “The test
mutations may occur at the data type, data value, message structure, or protocol binding
level. An example mutation is a buffer overflow type boundary conditions for an unbounded
string type field” [22].
SOAPSonar generates random test cases and they are ineffective in detecting defects.
SOAPSonar test case are based on the type information and data structure defined in the
WSDL. WSDL definition usually have basic information about the actual data type of
parameters. For example, the parameters of multiple method can be all of type String. If
generated values are strings such as are “ss” and “ds” the test case will fail because the
multiple method was expecting numeric values and generated values were string. These
randomly generated test cases will be ineffective in detecting defects as most of the test case
will fail because generated values were of different data type. [22]
RTSQ can generates random test cases based on the information taken from WSDL such
methods parameters data types, e.g. “1234” for integer data type. This type of testing is only
usefully from performance testing, they will of no help in actual errors and bugs detection.
WSTD-Gen is testing tool which provides a better way of generating test cases, this tool was
created for an academic work. Unfortunately I couldn’t find WSTD-Gen application on the
internet; I have reviewed this application based on a WSTD-Gen project report [22].
According to the report, users are allowed set constrains for each data-type as show in Figure
3.1.1.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 20 of 50
According to my understandings of this application users can only select from these
predefined contains shown in figure 3.1.1, which puts a limit on number of test cases this
application can produce.
All the applications I reviewed above have some type of a limit on values they can generate
and most of the times these values were randomly generated based WSDL. This type of
testing is not very useful for a web services tester because it randomly generated values
usually ineffective in detecting bugs and errors [21, 22, 23, 24].
Advantages and limitation of FAWST
In most the web service testing tools which I reviewed (sub heading 3.1), someone has to
generate all the test cases, this is the task which takes most of the user time and resources.
FAWST have all the main functionality which any standard web service testing application
does such as WSDL checking and parsing, GUI and automatic SOAP request and response
but it also have some unique features such as, automatic test data generator, result verifier
and detail test report which most of the other web services testing applications lack.
FAWST automatic test data generation is very flexible, users can put any constrains on any
character as they wish. For example if a tester wants FAWST to generate four characters
long string or a number, he/she can put constrains on each character, such as first character
can be specific, second can be random between two numbers or alphabets, third character
can be random and fourth can be random again. This flexibility allow the users to test their
web services with any possible values they can think of.
Figure 3.1.1 Constrain window of WSTD-Gen
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 21 of 50
Users can perform many different types of testing using FAWST such as Unit testing,
Functional testing, End-to-end testing and Sanity testing. These are some of the main
and common types of testing usually performed on software for quality insurance.
There are just two main limitations of my application. First one is that user can only test
SOAP web service and second limitation is not being able to parse all version of WSDL.
This is because the API I have used, the API is unable to parse some of the WSDL. I found
this problem at very late stage so I was unable change my API. I have not integrated the
REST web services in this version FAWST because of the limited time and main point of
this project was implement automatic test case generator which can generate any number of
different test cases.
There are some limitation in result verification, FAWST allow users to check multiple tags
and values but if there are two tags with the same name, users would only be able to check
the value of the first tag.
Summary
This chapter includes review of the well-known web service testing applications and shows
their limitations and advantages. I have reviewed FAWST and how it is unique from all other
web service testing applications available in the market and what are FAWST limitations.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 22 of 50
Requirement specification
This chapter describes the requirements identified for FAWST. Starting with functional
requirements, which are split into primary and secondary requirements.
Primary functional requirements
Primary functional requirements are the main requirements of FAWST. These requirement
were necessary to achieve the objectives to FAWST.
 Checking WSDL: SOAP web service require WSDL file for it to know what
functionality web services are offering. So checking if WSDL file or a link to WSDL
is correct is very important part of FAWST, because with the wrong WSDL file
nothing is going to work.
 Parsing and storing WSDL: Parsing WSDL means reading the WSDL files and
extracting all the key information, such as method names, parameters names and
data-type, complex data-type values and URL for SOAP requests. This information
has to be stored so it can be later displayed and used for SOAP requests. This
information can be stored on the RAM because this information is only required for
a small time.
 Collecting and storing test constrains: Collecting and storing test constrains enter
by a user as regular expressions is essential part of FAWST Based on the regular
expressions all the test case data will be generated. User test constrains are not stored
as normal regular expressions but it will be based on regular expressions concept.
For example 4-a-d means generate four character long string and the values can only
be between ‘a’ and ‘d’. Also these test constrains can be stored on RAM because
they will only be required for a small time period and accessing this data from RAM
is a lot faster than hard drive.
 Automatic test input data generator: This is the unique part of this project.
Automatic test inputs will only be generated once FAWST have collected and stored
test constrains in regular expressions. Based on the regular expressions FAWST will
generate all the test inputs. Without the user regular expressions FAWST will not be
able to generate any test inputs.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 23 of 50
 Automatic SOAP Request: Generating SOAP request and response is another main
functionality of FAWST. This functionality will allow the users to run as many test
cases as they want, without having to write a SOAP request for each test case.
 Automatic Result Verification: All the SOAP responses should be checked if they
passed or failed. The software should allow the user to type in expected values and
tag names and the values should be automatically checked against the SOAP
response.
 User Interface: User interface is necessary to have because it will allow the users to
interact with the FAWST easily. Without the user interface it would be incredibly
hard to navigate through the FAWST, because it is complex software and it requires
a fair amount user input at the start. Basic user interface will include following things;
 Home window: On this window user would be able to enter WSDL
link.
 Show method window: This window will display all the methods
names, parameter names and data types.
 Test data input windows: This is where user should easily be able to
enter constrains for test input generator. Test data input window
should be slightly different for each data type.
 Report: A report should be generated to display, what parameter values were
generated, SOAP request, SOAP response and result (pass/failed) of each test case.
This report should be in table so it can be easily checked by the user.
Secondary functional requirements
Secondary functional requirements are all the optional and additional features of FAWST.
These were implemented after the primary functional requirements.
 Checking for same test cases: Same test cases should be checked, because there are
no point testing web services with the same test cases again and again. It will only
waste time and consume unnecessary resources.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 24 of 50
 Help windows: Help windows should be added to help the users to use FAWST and
to warn if user if they enters wrong values or no values.
 Verification of user input: All the user input data should be checked and verified.
For example if a text field in FAWST is expecting an integer, it should only take an
integer for all the other value it should display an error message to inform users.
 Critical actions confirmation: Critical actions like closing the software or Back to
main menu should be caught by a confirmation window.
Summary
In this chapter I have described the identified requirements for FAWST. The requirements
were divided into primary and secondary requirements. Each of the requirements was
explained in detail for user understanding.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 25 of 50
Application Overview, Design and
Application Walk Through
This chapter explains in detail with flow chart how FAWST works. I will then reflect on the
design of the FAWST, which includes development environment and design patterns. I will
then explain in detail with screenshots how FAWST looks and works.
Application Overviews
Figure 5.1.1 Flow chart
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 26 of 50
In this topic, I will explain in detail how FAWST works and how it generates test inputs.
FAWST rely on some user input and based on that input it generates test case data. The
method is shown in Figure 5.1.1. Also this is just a basic overview of the FAWST main
functions and Figure 5.1.1 only shows main functions and flow.
Once FAWST is executed, user inputs a WSDL, which then checked to make user it is a
WSDL. Once the WSDL is parsed all the information is extracted such as methods name,
method input parameters name and data types, URL and complex data types etc. User selects
one of the method and enter an integer value for number test cases he/she wants FAWST to
generate. For each parameter, user have to enter test input constrains. There is almost no
limit on the test input constrains user can enter. There are number of basic options that user
can choose from such as:
1. Random value
2. Specific value
3. Between two values
4. Custom (combination of random, specific and between two values).
All the above options are explained in detail in application walk-through in sub section 5.3.
Once the user enters constrains for all parameters, FAWST generates test inputs based on
these constrains. Each test case is checked against all the pervious test cases to make sure
that there are no duplications. Once duplication is checked, SOAP request is sent. When a
SOAP response is received, user then enters tag names and expected values of each SOAP
response which is then checked and stored. User can only enter valid tag names otherwise
he/she will keep getting error (“Tag name not found”). Once all the test cases are generated,
a report is produced, which display all the generated parameter values, SOAP request and
response and result of each test case.
Design
5.2.1 Development Environment
I have used Java programming language to write code for FAWST because I have over two
years of experience of Java and I was more comfortable coding in Java. I have used NetBeans
IDE to help me manage and code FAWST because I have used NetBeans before and I knew
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 27 of 50
about the shoutcuts and help that I could get from using NetBeans. Most of the GUI
(graphical user interface) is created by NetBeans GUI builder. NetBeans GUI builder is very
helpful and it saves a lot of time once you know how to use it.
5.2.2 High level design
FAWST code design patteren is based on Model View Controller (MVC); MVC is design
pattern in which developers simply separate their code into three categories which are model,
view and controller. This allows developers to keep the code tidy and as result it makes it
easy to find and fix bugs, maintain code and enhance code usability. I have also divided the
code in to further sub folders to make it even simpler and tidier for example view package
has two sub-packages, MainGUI and ReportGUI.
Figure 5.2.2.1 Model View Controller [10]
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 28 of 50
Application Walk Through
Initail Window: The initial window the user will see is Figure 5.3.1. User can enter WSDL
URL or path to the WSDL store on the local machine. Once user enters WSDL, he/she has
to press the validate button. Once validate button is pressed WSDL URL or path is checked.
If correct, WSDL file is also checked to make sure it is correct. Once the WSDL is validated
“check methods” buttons is enabled, which opens new a window shown in Figure 5.3.3.
Check method button preform same checks as validate button, just to make sure user has not
changed WSDL. Clear button clears the text from the input WSDL test field. Exit button
closes the application but a confirmation dialog is display shown in Figure 5.3.2 to confirm
user intentions. Exit button does the exact same job throughout the application.
Methods Window: Method window is shown in Figure 5.3.3, The information extracted
from WSDL file is displayed here, such as, all the method names and paramter name and
data type. User can collapse or open each method to see input parameters name and data
Figure 5.3.1 Initail window after FAWST is executed
Figure 5.3.2 Closing confirmation box
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 29 of 50
type. User can select any method and click on the “Test” button to test that paticular method.
Once “Test” button is pressed, application checks if user selected a method, a parameter or
anything at all; based on result an error is displayed or if correct method was selected, a input
message dialog is displayed, where user is asked to enter number of test cases he/she wants
FAWST to generate, input message dialog only take integers, all ther other values are
rejected with an error message. “Back” button takes the user back to the initial window.
Different windows for different Data types: Once the user selects one of the methods and
clicks on “Test” Button in the method window and enters of number test he/she wants
FAWST to produce. Depending upon the method user wants to test, FAWST will open a
new a window for each parameter, where user can enters constrains. Different screens and
option are displayed for different data types. I will explain these different windows and
options in detail with screen shots below:
String: String window is shown in Figure 5.3.4. For string user have three basic options and
custom option which is explained in detail below under Custom heading. Any of these option
can be selected by clicking on their respective combo boxes;
1. Random String: User has to specify the length of the string.
2. Specific String: User can enter any string they want, it will stay the same though out
for every test case.
Figure 5.3.3 Method window
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 30 of 50
3. Random between two String: Users have to select alphabet one and two, alphabets
are between “a” and “z”. Users also have to enter length of the string.
Double: Double window is shown in Figure 5.3.5. It is mostly same as string window, the
only different is that user have to select decimal point position as well for random number,
decimal point position has to be less than random number length.
Specific and random between two numbers text field accept only numbers, which could be
either decimal or not decimal numbers.
Figure 5.3.4 Window for String data type constrains
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 31 of 50
Integer: Integer window is almost same as double window, the only difference is that all the
text fields only accepts integers and there is not decimal point position.
Enum: Enums are specific list of values which a method in the web service will accept,
FAWST extract these values from WSDL and display it to the user as shown in Figure 5.3.6.
Figure 5.3.6 Window for Enum data type constrains
Figure 5.3.5 Window for Double data type constrains
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 32 of 50
There are three options for a user, he/she can check the “Select Random one” check box or
chose one or multiple values from the list. If user selects multiple values, one of the value
will be randomly select for each test case.
Char: Char is similar to String window, the only difference is that there are no custom option
of char because char is a single character and there is no length option. Also user can only
enter single character in the specific option.
Boolean: Boolean window is very simple with just one options. User can either select true
or false.
Figure 5.3.7 Window for Char data type constrains
Figure 5.3.8 Window for Boolean data type constrains
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 33 of 50
Date: Date window simple and self-explanatory as shown in Figure 5.3.9. User have
following four options:
1. Random Date
2. Specific Date
3. Random between two dates
4. Today Date
For each option user have to select the date format so the generated date is in format that
user wants. Also there is a help window shown in Figure 5.3.10 for date format, this helps
the user to understand, what are difference between different date formats.
Figure 5.3.9 Window for Date data type constrains
Figure 5.3.10 Help Window for Date data type
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 34 of 50
Custom user constrain screens: Custom screens are for Integers, String and Double.
Custom screens allow the user to put constrains on each individual character, this means
FAWST will generate specific values for test cases based on user constrains. Each of custom
window is slightly different than the other, but they all have similar design and options. The
similarities and difference are following;
 Similarities
1. Length of characters
2. Option of specific value, random between two values and random value.
3. User can only select values from combo boxes.
4. Error messages if user forgets to enter a value, shown in Figure 5.3.11
 Differences:
1. Custom Integer window: Custom Integer window shown in Figure 5.3.12
only allow users to select numbers between 0 and 9.
Figure 5.3.11 Custom Double window with error message
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 35 of 50
2. Custom String window: Custom String window have one more option for
random value. User can have random character with or without punctuation.
Specific value can be any character and for between two values user can only
select alphabets.
Figure 5.3.12 Custom Integer window
Figure 5.3.13 Custom String window
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 36 of 50
3. Custom Double window: Custom Double window is very similar to custom
Integer window. The only difference is that user can have decimal point, only
one point can be selected, if more than one is selected user will get an error.
Summary
In this chapter, I have explained in how FAWST works. I have also reflected on the high
level design and design pattern of the FAWST code. I have also covered in detail how
FAWST looks and what are different options on the different screens with screenshots.
Figure 5.3.14 Custom Double window
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 37 of 50
Testing
This chapter will cover different types of testing performed on FAWST, this includes Unit
testing, Boundary testing, GUI testing, Acceptance testing and Platform testing, this is to
make sure that FAWST is bugs and error free.
Unit testing
Unit testing is a process in which the smallest testable parts of an application, called
units, are individually and independently scrutinized for proper operation [26]. I have
tested each and every class of this software individually. I wrote a main method of each class
and tested all the methods with different values.
The above Figure 6.1.1 is one of the example. I have three methods intGen, doubleGen and
charToInt in the CommonMethodsTwo class. All of these methods were tested with a lot of
different test cases such as, intGen takes two integers, so some of the test for this method
are as following;
1. intGen(1,4) // passsed
2. intGen(41,9) // failed, first value has to be smaller than second value
3. intGen(a,4) // failed, only take integers.
4. intGen(-10000,0) // passed
Table 6.1.2 shows a list of some of the methods that were individually tested. This table only
includes those methods whose test values were small and easy to understand.
Figure 6.1.1
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 38 of 50
NO# Method partial
header
Test values Expected
Result
Actual
Result
1 isDouble(String
parm)
2.3232 True passed
2 format(String xml) <?xml version="1.0"
encoding="utf-
8"?><soap:Envelope
xmlns:soap="http://schemas.xmlso
ap.org/soap/"
xmlns:xsi="http://www.w3.org/20
01"
xmlns:xsd="http://www.w3.org/20
01/XMLSchema"><soap:Body></
soap:Body></soap:Envelope>
Better
layout
Passed
3 smallerChar(Object
pramOne, Object
pramTwo)
C,K C Passed
4 biggerChar(Object
pramOne, Object
pramTwo)
C,K K Passed
5 checkConfirm(int
pram)
0 // this method is for check box,
this method true is check box is
checked
True Passed
6 isInt(String pram) 323232442424224 False Passed
7 intGen(int
rangeFrom, int
rangeTo)
1,4 2 or 3 Passed
8 doubleGen(double
rangeMin, double
rangeMax)
1.2,1.23 1.21 or
1.22
Passed
9 charToInt(String
pram)
abc 97 Passed
10 checkWSDLAvaila
ble(String wsdl)
http://www.webservicex.net/count
ry.asmx?WSDL
True Passed
11 checkPath(String
pram)
/homes/my301/year3/Project/other
/genericbarcode.wsdl
True Passed
Table 6.1.2. Unit testing
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 39 of 50
Boundary testing
Boundary testing is where test cases values are extreme values e.g. “maximum, minimum,
just inside/outside boundaries, typical values, and error values” [17]. I have checked all the
methods on which I could perform boundary testing e.g. intGen, doubleGen, intToChar,
isInt and charToInt etc. In Figure 6.2.1 shows one of the example.
In the above example (Figure 6.2.1) isInt method checks, if a given value is an int or not. As
we can see the four test cases, first one is a maximun value of an int, second value is minum
value of int, third testcase value is bigger than the int maxium value and fourth is less then
minmun value of the int. so the result should be true,true,false and false and we can see in
the Figure 6.2.2 it is correct.
User Interface Testing
The UI testing is process to make sure that all the functionality of the UI is working and it
does what it supposed to do. A thorough UI testing was performed on FAWST to make sure
that everything works. Table 6.3.1 shows all the main tests that were performed on the
FAWST UI.
Figure 6.2.1
Figure 6.2.2
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 40 of 50
No. Test Name Description Expected Result Actual
Result
1 Run
Application
Application was opened
in NetBeans and through
jar file execution.
Application first
window should open.
Passed
2 Clear button Click on the “Clear”
button.
Text field should get
clear.
Passed
3 Exit button
confirmation
box
Exit the application once
clicked but after the
confirmation.
Confirmation popup
box should appear.
Passed
4 Check
Methods
button
By clicking on check
methods button, it should
check the input WSDL. It
should call checkWSDL
method and if correct
open show method frame
Open show method
frame or display error
if WSDL is wrong.
Passed
5 Test button By clicking on test button,
it should display an input
dialog if a method is
selected else it should
display a message dialog.
Input dialog or
message dialog.
Passed
6 Main Menu
button with
confirmation
box
By clicking on the main
menu. Confirmation box
should appear for user
confirmation.
Confirmation box
should open.
Passed
7 Next button
with
message
dialog box
By clicking on the next
button. User constrains
should be validated.
Message box should
appear on the screen.
Passed
8 Done button
with
confirmation
box
By clicking on the done
button, confirmation box
should appear for user
confirmation.
Confirmation box
should appear
Passed
9 Click me
Jtable
column
By click on the click me
in report table screen
New frame should
open with extra
information
Passed
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 41 of 50
Acceptance Testing
The best way to determine whether a software had met all the user specifications and
requirements is to ask the user to check and test the software. Acceptance testing for FAWST
is performed by Dr Bozkurt. He tested the following WSDL which covers all the main data
types.
1. http://www.webservicex.net/geoipservice.asmx?WSDL for String
2. http://www.webservicex.net/ConverPower.asmx?WSDL for Double and Enum
3. http://www.webservicex.net/periodictable.asmx?WSDL for String
4. http://www.webservicex.net/MortgageIndex.asmx?WSDL for Int
5. http://www.webservicex.net/sunsetriseservice.asmx?WSDL for Int and Float
WSDL
Number
Issues Passed/Failed Comments Score 1-10
1 is low
1 N/A Passed N/A 8
2 N/A Passed Good example 9
3 Input didn’t
work
Passed/Failed Issue with one of the
method
7
4 N/A Failed Come up with some
sensible input.
5
5 N/A N/A N/A N/A
Overall Performance 5 4 3 2 1
10 Collapsible
table
On method screen, user
should be able to open
and collapse each method
and complex to see which
values are accepted by the
method.
Methods should open
and collapse
Passed.
Table 6.3.1 Result from user interface testing
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 42 of 50
Design 5 4 3 2 1
Met all user requirements
and specifications
5 4 3 2 1
Any comments Overall performance is okay. Design is good. FAWST have met
and exceeded all the user requirements and specifications.
Dr Bozkurt was happy with the FAWST and he said it met and exceeded all the user
requirements and specifications.
Platform Testing
It is very import for FAWST to run on all major operating system otherwise limited user
would be able to use FAWST. I have tested FAWST on the following platforms:
 Linux
o Fedora
 Windows
o Windows 8
o Windows 7
 Apple OS
o OSX Yosimite 10.10.2
FAWST worked on all the above operating systems and there were no issues.
Summary
In this chapter I have covered different types of testing that were performed on FAWST with
examples. The different types of testing performed on FAWST includes Unit testing,
Boundary testing, GUI testing, Acceptance testing and Platform testing to make sure that all
the bugs and errors are removed.
Table 6.4.1 Acceptance testing summary
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 43 of 50
Conclusions and recommendations
This Chapter describes what I have learned and achieved from this project. I would also
explain what I will do differently next time and what new things I would like to add to this
project.
What I learned and achieved
I am very pleased with the outcome of my project. For me, this project has opened a whole
new world of programming and testing which I had no clue about. This project was by far
the biggest project I have ever done and by doing so learned and achieved a lot.
Now I have a very good knowledge of web service and test data generation. Both of them
these topics were very new to me and I knew very little about then. This project has
broadened my programming skills, I have learnt and practice many different things such
working with external libraries, recursion, error handling and much more. I have used and
learnt more about the MVC design pattern, which will come very useful in my future
programming projects and as well as in my day to day life.
This project has also helped me learnt more about project management and time
management. These are very useful skills because project management skills reduces the
chance of a project failing, it increases the quality of the software, increase efficiency and
saves time. As we all know time is money, there should be no questions about how useful
time management skill is. The advantage of time management are, it reduces stress, free
more time, get more done in same time and we get more time where it needed most. These
two skills are not only good for projects but they are useful in all acpects of the life.
I am well aware of the limitation of FAWST but I am very proud of what I have achieved,
this project does what is supposed to do and it satisfies all the primary and secondary user
requirements. This by far the biggest project I have ever created and I have a lot from this
project.
What would I add or do different
The only thing that I would do differently is to make use of more object oriented
programming style. I haven’t used inheritance and subclasses, and after completing 80% of
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 44 of 50
the project I realized using inheritance and subclasses would have been very useful for this
project. FAWST code would have been more generalized, easy to maintain and expand.
Apart of this I am very happy with the rest of the project and the way I coded it. However, I
certainly would like to add more features if I had more time available, these features are not
part of functional or non-functional requirement but it will make this project much better.
 Printing and storing report: Printing and storing the report would be a very nice
feature; this will allow the users to analyse and compare the results.
 Enhance GUI: Current FAWST GUI is not very simple because it is a prototype.
By making the following changes to the GUI will enhance user experience of using
FAWST:
o Make the layout better
o Use colours effectively manage attention
o Improve navigation
 XPath for result verification: XPath is a language that describes a way to locate
and process items in XML [20]. XPath will allow FAWST to verify results more
accurately and also user would be able to check complex data types.
 Enhanced Settings: User preference settings should be added. For example, only
unique test case on and off, check multiple elements or single element, keep the
same value for tag or different value to each test case and stop is certain number of
test case fails.
 Increase testing capability: Currently user can only perform unit testing,
functional testing, end-to-end testing and sanity testing using FAWST. But I can
easily include the load and stress testing options. This will increase the testing
capability of FAWST and my application will be used by wider audience:
 Help: My application is very straight forward to use but adding more help screens
will make it even simpler for user to use.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 45 of 50
Conclusion
In conclusion to this report, I would simply say that I had great fun doing this project.
FAWST is an amazing application which can really help with the testing of web services. It
is also very unique and flexible and I am well aware that there is quite a few things needs to
be included in FAWST but, in my point of view creating FAWST was a great achievement
for me and as my supervisor said that FAWST had met and exceeded all the user
requirements and specifications. FAWST is also very thoroughly tested to make sure that it
is bugs and errors free. I would say again it was great fun doing this project and I learnt a lot
from this experience.
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 46 of 50
Bibliography
[1] M. Bozkurt, M. Harman and Youssef, “Testing & Verification In Service Oriented
Architecture :A Survey,” in Software Testing, Verification and Reliability, London, Wiley
InterScience, 2009, p. 259–350.
[2] A. Payong, “Examples of WSDL,” yurtopic, 27 03 2013. [Online]. Available:
http://www.yurtopic.com/tech/programming/wsdl-language.html . [Accessed 17 02 2015].
[3] “WSDL Documents,” w3schools, [Online]. Available:
http://www.w3schools.com/webservices/ws_WSDL_documents.as. [Accessed 17 02 2015].
[4] “WSDL,” webopedia, [Online]. Available:
http://www.webopedia.com/TERM/W/WSDL.html. [Accessed 16 02 2015].
[5] “What Are RESTful Web Services,” Oracle, [Online]. Available:
http://docs.oracle.com/javaee/6/tutorial/doc/gijqy.html. [Accessed 17 02 2015].
[6] Dr. P. Oliva, “ECS639U - Web Programming,” Queen Mary, [Online]. Available:
http://qmplus.qmul.ac.uk/course/view.php?id=4915. [Accessed 17 02 2015].
[7] “SOAP Introduction,” w3schools, [Online]. Available:
http://www.w3schools.com/webservices/ws_SOAP_intro.asp. [Accessed 17 02 2015].
[8] “SOAP Example,” w3schools, [Online]. Available:
http://www.w3schools.com/webservices/ws_SOAP_example.asp. [Accessed 17 02 2015].
[9] M. Vaqqas, “WEB DEVELOPMENT,” Dr.dobbs, 23 09 2014. [Online]. Available:
http://www.drdobbs.com/web-development/restful-web-services-a-tutorial/240169069.
[Accessed 17 02 2015].
[10] “MVC image,” wordpress, [Online]. Available:
https://imdjkoch.files.wordpress.com/2012/09/oaf-mvc-architecture.jpg. [Accessed 17 02
2015].
[11] “Software Testing Services in the US: Market Research Report,” IBISWorld, 2014.
[12] M. Rozlog, “REST and SOAP,” InfoQ, 04 01 2010. [Online]. Available:
http://www.infoq.com/articles/rest-soap-when-to-use-each. [Accessed 02 03 2015].
[13] F. M. Borger and J. , “stateless,” whatis, 01 04 2005. [Online]. Available:
http://whatis.techtarget.com/definition/stateless. [Accessed 02 03 2015].
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 47 of 50
[14] “What is SoapUI,” SoapUI, [Online]. Available: http://www.soapui.org/about-soapui/what-
is-soapui-.html. [Accessed 02 03 2015].
[15] “Award Winning Software,” SMARTBEAR, [Online]. Available:
http://smartbear.com/about-us/awards/ . [Accessed 02 03 2015].
[16] S. Hussain, Z. Wang and I. K. Tour, Web Service Testing Tools: A Comparative Study,
Beijing,.
[17] “Boundary testing,” Wikipedia, [Online]. Available:
http://en.wikipedia.org/wiki/Boundary_testing. [Accessed 04 03 2015].
[18] F. Usmani, “Product Scope vs Project Scope,” PM Study Circle, [Online]. Available:
http://pmstudycircle.com/2012/01/product-scope-vs-project-scope/. [Accessed 04 03 2014].
[19] A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Fifth Edition,
Pennsylvania: Project Management Institute, 2013.
[20] M. Rouse, “XPath,” Search SOA, [Online]. Available:
http://searchsoa.techtarget.com/definition/XPath. [Accessed 05 03 2015].
[21] J. Z. ,. L.-J. Z. a. N. M. M. Zhong Jie Li, Z. J. Li, J. Zhu, L.-J. Zhang and N. , Towards a
Practical and Effective Method for Web Services Test Case Generation, Vancouver: IEEE,
2009, pp. 106 - 114.
[22] X. J. Offutt and W. , Generating Test Cases for Web Services Using Data, Fairfax, 2009.
[23] “Home,” crosschecknet, [Online]. Available: http://www.crosschecknet.com/. [Accessed 09
03 2015].
[24] “Rational Service Tester for SOA Quality,” IBM, [Online]. Available: http://www-
03.ibm.com/software/products/en/servicetest. [Accessed 09 03 2015].
[25] “What Are Web Services,” Oracle, [Online]. Available:
http://docs.oracle.com/javaee/6/tutorial/doc/gijvh.html . [Accessed 14 04 2014].
[26] M. Rouse, “Unit Testing,” Search Software Quality, [Online]. Available:
http://searchsoftwarequality.techtarget.com/definition/unit-testing. [Accessed 14 04 2015].
M.
[27] “Types of software Testing,” Software Testing Help, [Online]. Available:
http://www.softwaretestinghelp.com/types-of-software-testing/. [Accessed 16 02 2015].
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 48 of 50
Appendix
How FAWST stores test constrains in the Array Lists
8.1.1 Custom String
------ Start for loop ------
User constrains Custom String
User constrains Specific
User constrains b
User constrains Random withOut
User constrains 1
User constrains Random with
User constrains 1
Length of Array List: 6
------ End for loop ------
8.1.2 String with random value
------ Start for loop ------
User constrains String
User constrains Random
User constrains 4
Length of Array List: 2
------ End for loop ------
8.1.3 String with between two values
------ Start for loop ------
User constrains String
User constrains BetweenTwoValue
User constrains 17-d-e
Length of Array List: 2
------ End for loop ------
8.1.4 String with specific value
------ Start for loop ------
User constrains String
User constrains Specific
User constrains abc
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 49 of 50
Length of Array List: 2
------ End for loop ------
8.1.5 Custom Int
------ Start for loop ------
User constrains Custom Int
User constrains Specific
User constrains 3
User constrains Random
User constrains 1
User constrains Specific
User constrains 4
Length of Array List: 6
------ End for loop ------
8.1.6 Int with specific value
------ Start for loop ------
User constrains Int
User constrains Specific
User constrains 3232
Length of Array List: 2
------ End for loop ------
8.1.7 Int with random value
------ Start for loop ------
User constrains Int
User constrains Random
User constrains 4
Length of Array List: 2
------ End for loop ------
8.1.8 Int with between two values value
------ Start for loop ------
User constrains Int
User constrains BetweenTwoValue
User constrains 2131-232314
Length of Array List: 2
------ End for loop ------
Automatic Test Data Generator For Web Service Testing Muhammad Yahya
Page 50 of 50
8.1.9 Boolean with specific value
------ Start for loop ------
User constrains Boolean
User constrains Specific
User constrains False
Length of Array List: 2
------ End for loop ------
8.1.10Enum values
------ Start for loop ------
User constrains Enum
User constrains None
User constrains Code
User constrains Type
User constrains Both
Length of Array List: 4
------ End for loop ------
8.1.11Enum values
------ Start for loop ------
User constrains Enum
User constrains None
User constrains Modulo10
Length of Array List: 2
------ End for loop ------

Contenu connexe

En vedette (15)

Comparison of town and region
Comparison of town and regionComparison of town and region
Comparison of town and region
 
AIChE2016 Chicago FINAL
AIChE2016 Chicago FINALAIChE2016 Chicago FINAL
AIChE2016 Chicago FINAL
 
elegance e-brochure
elegance e-brochureelegance e-brochure
elegance e-brochure
 
sohaib mohamed
sohaib mohamedsohaib mohamed
sohaib mohamed
 
Methods of cooking
Methods of cookingMethods of cooking
Methods of cooking
 
Askymo를 이용해야 하는 5가지 이유
Askymo를 이용해야 하는 5가지 이유Askymo를 이용해야 하는 5가지 이유
Askymo를 이용해야 하는 5가지 이유
 
USGBC2016 SJv1.0
USGBC2016 SJv1.0USGBC2016 SJv1.0
USGBC2016 SJv1.0
 
Research publications raji
Research publications rajiResearch publications raji
Research publications raji
 
ALAM RESUME
ALAM RESUMEALAM RESUME
ALAM RESUME
 
79 86
79 8679 86
79 86
 
Hokulea in South Africa
Hokulea in South AfricaHokulea in South Africa
Hokulea in South Africa
 
Online technology jizza
Online technology jizzaOnline technology jizza
Online technology jizza
 
5 cara menulis paragraf pertama
5 cara menulis paragraf pertama5 cara menulis paragraf pertama
5 cara menulis paragraf pertama
 
ICC iloilo-oltech
ICC iloilo-oltechICC iloilo-oltech
ICC iloilo-oltech
 
Pencarian ide
Pencarian idePencarian ide
Pencarian ide
 

Similaire à Final Report 1.3

Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...
Nóra Szepes
 
A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...
Vincenzo Ferme
 
S13CS61920410Presentation
S13CS61920410PresentationS13CS61920410Presentation
S13CS61920410Presentation
Abid Muslim
 

Similaire à Final Report 1.3 (20)

Lesson 5...Guide
Lesson 5...GuideLesson 5...Guide
Lesson 5...Guide
 
software_testing pdf.pdf
software_testing pdf.pdfsoftware_testing pdf.pdf
software_testing pdf.pdf
 
Testing Tutorial PDF
Testing Tutorial PDFTesting Tutorial PDF
Testing Tutorial PDF
 
software_testing pdf.pdf
software_testing pdf.pdfsoftware_testing pdf.pdf
software_testing pdf.pdf
 
software_testing pdf.pdf
software_testing pdf.pdfsoftware_testing pdf.pdf
software_testing pdf.pdf
 
software_testing pdf.pdf
software_testing pdf.pdfsoftware_testing pdf.pdf
software_testing pdf.pdf
 
software_testing pdf.pdf
software_testing pdf.pdfsoftware_testing pdf.pdf
software_testing pdf.pdf
 
Online Test
Online TestOnline Test
Online Test
 
Mobile d
Mobile dMobile d
Mobile d
 
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...
Thesis - Nora Szepes - Design and Implementation of an Educational Support Sy...
 
online examination management system
online examination management systemonline examination management system
online examination management system
 
BA_FCaballero
BA_FCaballeroBA_FCaballero
BA_FCaballero
 
A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...A Declarative Approach for Performance Tests Execution in Continuous Software...
A Declarative Approach for Performance Tests Execution in Continuous Software...
 
digiinfo website project report
digiinfo website project reportdigiinfo website project report
digiinfo website project report
 
ST_final (2).docx
ST_final (2).docxST_final (2).docx
ST_final (2).docx
 
A Machine Learning Approach To Predict Movie Box-Office Success
A Machine Learning Approach To Predict Movie Box-Office SuccessA Machine Learning Approach To Predict Movie Box-Office Success
A Machine Learning Approach To Predict Movie Box-Office Success
 
Undergrad Thesis | Information Science and Engineering
Undergrad Thesis | Information Science and EngineeringUndergrad Thesis | Information Science and Engineering
Undergrad Thesis | Information Science and Engineering
 
Online Examination System Project report
Online Examination System Project report Online Examination System Project report
Online Examination System Project report
 
Automation testing: how tools are important?
Automation testing: how tools are important?Automation testing: how tools are important?
Automation testing: how tools are important?
 
S13CS61920410Presentation
S13CS61920410PresentationS13CS61920410Presentation
S13CS61920410Presentation
 

Final Report 1.3

  • 1. School of Electronic Engineering and Computer Science Final Report Supervisor: Stefan Poslad Project Title: Automatic Test Data Generator For Web Service Testing Student Name: Muhammad Yahya Date: 20/04/2015 Programme of study: BSc Computer Science Supervisor: Stefan Poslad Supervisor: Dr Mustafa Bozkurt
  • 2. Declaration of Authorship I, Muhammad Yahya, declare that this thesis titled, ‘Automatic Test Data Generator For Web Service Testing’ and the work presented in it are my own. I conform that:  This work was done wholly in candidature for an undergraduate degree at this University.  Where any part of this thesis has been previously been submitted for a degree of any other qualification at this University or any other institution, this has been clearly attributed.  Where I have consulted the published work of others, this has always been clearly attributed.  Where I have quoted from the work of others, the source has always been given. With the exception of these such quotations, this thesis is entirely my own work.  I have acknowledged all main sources of help. Signed: Muhammad Yahya Date: 20/04/2015
  • 3. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 3 of 50 Abstract: Web service testing is a process of finding bugs and errors in the web services. There are different methods of testing web services. Web service testing is usually performed manually which is very costly and time consuming. There are also number of web service testing tools currently available in the market. Some of them are free open-source, such as SoupUI free version, WebInject, Membrane, TestMaker and SOAPSonar and some of them can be bought such as IBM® Rational® for SOA Quality (RTSQ) and SOAPUI NG Pro. To my knowledge, most of these commercially or widely available web service testing tools does not have automatic test values generator. SOAPSonar, RTSQ and few academic works does provide automatic test values generator, but they are not very useful. I have created a web service testing application which allow user to enter constrains and the test values are generated based on those constrain. This testing tool is very flexible in terms of test values generator. Everyone now a days is creating web services for their applications, this tool will allow them to test their web services easily, thoroughly and quickly.
  • 4. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 4 of 50 Table of Contents Introduction................................................................................................ 7 Objectives .....................................................................................................................7 Motivation......................................................................................................................8 Challenges....................................................................................................................8 Summary.....................................................................................................................10 Background ............................................................................................. 11 Web services ..............................................................................................................11 WSDL..........................................................................................................................13 SOAP request and response......................................................................................15 Regular Expression ....................................................................................................15 ASCII keys ..................................................................................................................16 XML Data Types .........................................................................................................16 Different types of testing .............................................................................................16 Summary.....................................................................................................................17 Critical review of the existing applications................................................ 18 Existing applications ...................................................................................................18 Advantages and limitation of FAWST.........................................................................20 Summary.....................................................................................................................21 Requirement specification........................................................................ 22 Primary functional requirements.................................................................................22 Secondary functional requirements ............................................................................23 Summary.....................................................................................................................24 Application Overview, Design and Application Walk Through .................. 25 Application Overviews ................................................................................................25 Design.........................................................................................................................26 Application Walk Through...........................................................................................28 Summary.....................................................................................................................36
  • 5. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 5 of 50 Testing..................................................................................................... 37 Unit testing..................................................................................................................37 Boundary testing.........................................................................................................39 User Interface Testing ................................................................................................39 Acceptance Testing ....................................................................................................41 Platform Testing..........................................................................................................42 Summary.....................................................................................................................42 Conclusions and recommendations......................................................... 43 What I learned and achieved......................................................................................43 What would I add or do different.................................................................................43 Conclusion ..................................................................................................................45 Bibliography 46 Appendix 48 How FAWST stores test constrains in the Array Lists ................................................48
  • 6. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 6 of 50 Figures and Tables Figure 1.1 who uses website and web service [6]............................................................................ 11 Table 2.2.1 WSDL four major elements .......................................................................................... 13 Figure 1.2 Web Service Description Language [2].......................................................................... 14 Figure 1.3 Main structure of a WSDL document [3] ....................................................................... 14 Figure 2.3.1 SOAP request............................................................................................................... 15 Figure 2.3.2 SOAP response ............................................................................................................ 15 Table 2.6.1 commonly used XML data types .................................................................................. 16 Figure 5.1.1 Flow chart .................................................................................................................... 25 Figure 5.2.2.1 Model View Controller [10] ..................................................................................... 27 Figure 5.3.1 Initail window after FAWST is executed .................................................................... 28 Figure 5.3.2 Closing confirmation box ............................................................................................ 28 Figure 5.3.4 Window for String data type constrains....................................................................... 30 Figure 5.3.5 Window for Double data type constrains..................................................................... 31 Figure 5.3.6 Window for Enum data type constrains....................................................................... 31 Figure 5.3.7 Window for Char data type constrains......................................................................... 32 Figure 5.3.8 Window for Boolean data type constrains................................................................... 32 Figure 5.3.9 Window for Date data type constrains......................................................................... 33 Figure 5.3.10 Help Window for Date data type ............................................................................... 33 Figure 5.3.11 Custom Double window with error message ............................................................. 34 Figure 5.3.12 Custom Integer window............................................................................................. 35 Figure 5.3.13 Custom String window .............................................................................................. 35 Table 6.1.2. Unit testing................................................................................................................... 38 Figure 6.2.1 ...................................................................................................................................... 39 Figure 6.2.2 ...................................................................................................................................... 39 Table 6.3.1 Result from user interface testing.................................................................................. 41 Table 6.4.1 Acceptance testing summary......................................................................................... 42
  • 7. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 7 of 50 Introduction Web service testing is a process of finding bugs and errors in the web services. There are different methods of testing web services and most of them are usually performed manually which is very costly and time consuming. Now a days there are different machines and software available to check and test the items we use in our daily life such as, cars, tyres, and drugs etc. This manual process of testing is performed by the software testers, therefore, the quality of the web service rely on the software testers’ ability and experience. Software testers have to spend a huge amount of resources and time to make sure that the web service is tested properly and rigorously. To address the issue of manual web service testing I decided to create a tool which will test web services automatically. This software is called Flexible Automatic Web Service Testing (FAWST) and it will generate unique test cases based on user constrains. Objectives The main objectives of this project are following;  Validate and parse WSDL  Generate test data for any XML data type  Generate test cases automatically with o Any number of test cases o Each test case should be unique  Automatically send SOAP requests  Analyse SOAP response  Generate detailed test report
  • 8. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 8 of 50 Motivation There were number of different things that motivated me to do this project such as;  Software Testing: Software Testing is my hobby, I test software for fun and I love doing it. The reason why I love software testing is because it is a challenging task. I have to look at software from a different angle and come up with all the different test cases and scenarios. The joy and satisfaction of finding and fixing the bugs is also very motivational for me.  Software quality improvement: Poorly tested software can have a lot of consequences such as security issue, loss of money and time, reliability and even death or injury. With the help of FAWST, developers can develop better web services which will contain minimum bugs. As a result, users will get more robust web service to use.  Knowledge enhancement: In this project most of the things were completely new to me. Learning about new things and facing new challenges is very motivational for me. It also gives me courage and self-esteem when I learn new things.  Future Directions: I will be looking for job as a software tester. This project has enhanced my chance of becoming a good software tester and it will open many opportunities for me in the field of software testing. Challenges Testing web services is more challenging than testing traditional software due to the complexity of Web service technologies and the limitations that are imposed by the Service- Oriented Computing environment [1]. Also most of things in this project were completely new to me because of this, I faced number of challenges right from the beginning. 1. Starting point – Starting of any project is difficult especially if something is new. In my case I had the same issue because automatic test case generating is completely new, I was not really sure how I should do this and I had nothing to compare it to. 2. API – Finding a right API for parsing WSDL and for SOAP request and response was one of the biggest challenge I faced in this project. I spent almost six to seven
  • 9. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 9 of 50 weeks on finding the correct API which can parse WSDL send SOAP request and receive SOAP response. I went through all the major APIs such SOAP-UI, Selenium and Wsdl4j. SoapUI and Selenium APIs are very complicated. In my point of view, they are also very poorly documented. These APIs have their own forums and it is almost impossible to find any information or get any help. Finding the right API made the whole project very difficult because I spent a lot of time on it. 3. Complexity – By complexity I mean I had to use number of different java internal and external libraries to achieve objectives of this project. I have also used NetBeans GUI builder to create user interface. Understanding and changing NetBeans GUI builder code is a very hard and complex task. Other libraries include XML parsers, random number generator and date generator etc. 4. User Input – Finding a useful and simple way of getting test constrains from the user is not easy task. I have to cover all the possible inputs. It was a difficult task but most of the test constrains are covered. For example, for integer data type, number can be a random, between two numbers, specific number or custom (combination of specific, random and between two numbers). I had to get test constrains for all data types. 5. Storing test input constrains – As FAWST allows the users to generate any test inputs, this means user can do following: 1. Twenty characters long string 2. 1st character is ‘a’ 3. 2nd character is between ‘h’ and ‘z’ 4. 3rd character is random . . . 20. 20th character is ‘4’. It is just one example illustrated above; a string could be any characters long. I had to store all this information because the test case data is generated based on this information. Finding a good solution was also very hard. At the end, I used Array- Lists and my own type of regular expressions to solve it.
  • 10. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 10 of 50 This project is like a star, looks small from far, but as you get closer and closer it gets bigger and bigger. In the beginning of this project, it looked fairly simple and easy to do but as I progressed through, it became very hard and very complex. Summary This chapter introduced FAWST and what are the main objects of the FAWST. I have also talked about my motivations behind do this project and what challenges I had to faces while working on FAWST.
  • 11. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 11 of 50 Background This chapter explains in detail all the background research I have done for this project such as web services, WSDL, XML data types, SOAP request and response, regular expressions, ASCII keys and different types of testing. Web services Web service is a service which is available on web for application consumption/use and a website is available on web for human consumption/use. In simple words web service provides a way of communicating between software applications over the network, web services can run on a variety of platforms and frameworks [25]. For example, on Twitter website we can post and read tweets but if a computer wants interact with Twitter to post and read tweets, it has to use Twitter web service. We can write some code which will allow the computer to access Twitter data though web service, this data then can be analysed by computer without any human interaction. Similarly if we make weather app, all we have to do is write some code which will automatically get data from met office web service and then we can display it on our app however we like. We can see from the above examples, that web services are very useful because they provide any easy way of communication between two applications. Figure 1.1 who uses website and web service [6]
  • 12. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 12 of 50 There are two types of the web services, both do the same thing but they have their own advantages and disadvantages. It is totally up to the programmer which technique suits him/her most. 2.1.1 REST REST stand for Representational State Transfer, it relies on stateless communication protocol, typically HTTP. In the REST web services, data and functionality are considered resources and they are accessed by using Uniform Resource Identifiers (URIs), typically links on the Web [5]. This means that each unique URI is a representation of some object. REST web service allows you to get and modify content though HTTP GET, POST, PUT or DELETE methods. REST web services are lightweight, maintainable, and scalable [9]. A web services based on REST are called RESTful services. REST approach will work best in the following situations: 1. Limited bandwidth and resources - because the return structure can be in any format but it is usually JSON which is very lightweight. 2. Totally stateless operations - it is good for stateless operations such as Create, Read and Delete. 3. Caching situations - data can be cached because of the totally stateless operation of the REST approach [12]. 2.1.2 SOAP SOAP stands for Simple Object Access Protocol. It is a communication protocol which allows communication between applications. SOAP is platform and language independent protocol and it is based on XML. It is simple and extensible and allows you the get around fire walls, and it is also recommended by W3C [7]. As SOAP is platform and language independent, it provides a platform for communicate between applications running on various different operating systems, with different programming languages and technologies. SOAP web services needs to have WSDL [7]. SOAP approach will work best in the following situations: 1. Asynchronous processing and invocation: SOAP approach provides high level of reliability and security
  • 13. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 13 of 50 2. Formal contracts: “If both sides (provider and consumer) have to agree on the exchange format then SOAP 1.2 gives the rigid specifications for this type of interaction”[12] 3. Stateful operations: SOAP approach is good for stateful operations. Stateful means the computer or program keeps track of the state of interaction [13] WSDL The Web Services Description Language (WSDL) is an XML-based language which is used to describe the functionality offered by a web service. A real life example of a WSDL can be a takeaway leaflet where we see all the different items that a particular takeaway offers. Also Chinese and Indian takeaway leaflets will have different items and different phone numbers to call them. Similarly WSDL tells us what each web service is offering and how can we contact this web service. WSDL document can be divided into four major elements shown below in table 2.2.1. Element Description <types> Data type of the parameter. Different data types are explained below under heading 2.6 XML data types. <message> Each message can consist of one or more parts. The parts are like the parameters of a method in a traditional programming language [3]. <portType> PortType is set of operation offered by a web service. <binding> “A protocol and data format specification for a particular port type” [3]. Table 2.2.1 WSDL four major elements
  • 14. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 14 of 50 In the below Figure 1.2 we can see how each part of the WSDL are linked together and also it show what the sub parts of the four main elements. Figure 1.2 Web Service Description Language [2] The main structure of a WSDL document looks like Figure 1.3. WSDL can also have other elements such as extensions elements and service elements that allow us to group together the definitions of several web services in one single WSDL document [3]. Figure 1.3 Main structure of a WSDL document [3]
  • 15. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 15 of 50 SOAP request and response 1. SOAP Request is sent to invoke a method on the web service. Request call has to have the parameters of the method. The request has to be in the XML format which includes all the information about the request. See Example below in Figure 2.3.1. <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:example="http://www.webserviceX.NET/"> <SOAP-ENV:Header/> <SOAP-ENV:Body> <example:ChangeAngleUnit> <example:AngleValue>10</example:AngleValue> <example:fromAngleUnit>radians</example:fromAngleUnit> <example:toAngleUnit>degrees</example:toAngleUnit> </example:ChangeAngleUnit> </SOAP-ENV:Body> </SOAP-ENV:Envelope> Figure 2.3.1 SOAP request 2. SOAP Response is sent back from the web service method, which includes return value if the request was correct else error is sent back. A response is also sent back in the XML format, which makes it easy to parse and use. See the Example below in Figure 2.3.2. <?xml version="1.0" encoding="UTF-16"?> <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soap:Body> <ChangeAngleUnitResponse xmlns="http://www.webserviceX.NET/"> <ChangeAngleUnitResult>572.95779513082323</ChangeAngleUnitResult> </ChangeAngleUnitResponse> </soap:Body> </soap:Envelope> Figure 2.3.2 SOAP response Regular Expression A regular expression is a string which describes a string pattern. Which is used to produce new strings based on the pattern or validate existing strings such as “b [A-Z0-9._%+- ]+@[A-Z0-9.-]+.[A-Z]{2, 4}b” this regular expression checks if someone has entered the correct format e-mail address.
  • 16. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 16 of 50 ASCII keys ASCII stands for American Standard Code for Information Interchange. ASCII is an integer representation of the characters for example alphabet “a” can be represented as number 97. XML Data Types WSDL uses XML define data types, in table 2.6.1 we can see all the main commonly used XML data types but it is not a full list. There are many more XML data types which are based on data types in table 2.6.1. Data Type Description String String data types are used for the values which contain sequence of character. Such as, sSsd, hello-world and 354sdse%3 etc. Integer Integer is sequence of number ranging from -2147483648 through 2147483647. Double Double is sequence of number with decimal point. Double hold 64 bit long numbers. Float Float is similar to Double but it can only hold 32 bit long numbers. Char Char is for single character. For example A, b, % and s etc. Enum Enum data type contains a list of acceptable values. Date and time Date and time data type are for the values that contains date and time. Boolean Boolean data type is for true or false. Table 2.6.1 commonly used XML data types Different types of testing 1. Unit testing – Programmer test each individual software components and parts. This is mostly done by the programmers themselves.
  • 17. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 17 of 50 2. Functional testing – “This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type testing geared to functional requirements of an application” [27]. 3. End-to-end testing – It “involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database or using network communications”.[27] 4. Sanity testing – “Testing to determine if a new software version is performing well enough to accept it for a major testing effort. If application is crashing for initial use then system is not stable enough for further testing and build or application is assigned to fix”[27]. 5. Load testing – “It is a performance testing to check system behaviour under load. Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails”. [27] 6. Stress testing - “System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load”.[27] Summary In this chapter, I have explained in detail what web service is, what are the different types of web services and what is SOAP request and response is. I have also explained what WSDL is and what data types WSDL accepts and I also talked a little bit about regular expressions, ASCII keys and different types of software testing.
  • 18. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 18 of 50 Critical review of the existing applications This chapter presents a review the existing applications, both commercially and non- commercially available. It also includes a review of FAWST and it shows in detail what functionality FAWST does and does not provide. Existing applications There are number of web service testing tools currently available in the market. Some of them are free open-source such as SoupUI free version, WebInject, Membrane 1 , TestMaker and SOAPSonar and some of them can be bought such as IBM® Rational® for SOA Quality (RTSQ), prices starts from $2995 and SOAP UI NG Pro €449 per year. To my knowledge, most of these commercially or widely available web service testing tools do not have automatic test case generator. SOAPSonar, RTSQ and few academic works such as WSTD- Gen does provide automatic test case generator [21, 22, 23, 24]. Basically most of these widely available web service tools provide graphical user interface, which allows the users to import and check WSDL, create and edit test cases manually, run the test cases and see the result. I have personally used two of the web service testing tools and their details are provided below; 1. SOAP UI SoapUI is free and an open-source cross-platform functional testing tool for web service, and is known as “Swiss-Army Knife of Testing”. SoupUI has well designed and structured GUI. SoapUI allows the user to create and run automated functional, regression, compliance, and load tests. It supports both SOAP and REST web service approach. SoapUI also allows the user to run Functional/Load Tests and MockServices from a task scheduler with a set of command-line tools. SoapUI received a number of awards: ATI Automation Honors, 2009 , Jolt award in 2014 for the best testing tool .InfoWorld Best of Open Source Software Award, 2008 and SOAWorld Readers' Choice Award, 2007 [13] [16] [15]. 1 www.service-repository.com
  • 19. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 19 of 50 2. Service-repository.com Service-repository.com is browser based web service testing tools. It provides very basic web service testing functionality such as WSDL validation, send SOAP request and display SOAP response. Users have to manually enter values for each test case one by one. User can only test SOAP web service. This tool is only good for basic checks and testing, it will be of no use if user wants to test their web service rigorously. Some of these web service tools can generate test cases automatically. SOAPSonar utilizes XSDMutation, they have submitted a patent for automation technique which is pending. SOAPSonar generate a set of test cases, which are both negative and positive. “The test mutations may occur at the data type, data value, message structure, or protocol binding level. An example mutation is a buffer overflow type boundary conditions for an unbounded string type field” [22]. SOAPSonar generates random test cases and they are ineffective in detecting defects. SOAPSonar test case are based on the type information and data structure defined in the WSDL. WSDL definition usually have basic information about the actual data type of parameters. For example, the parameters of multiple method can be all of type String. If generated values are strings such as are “ss” and “ds” the test case will fail because the multiple method was expecting numeric values and generated values were string. These randomly generated test cases will be ineffective in detecting defects as most of the test case will fail because generated values were of different data type. [22] RTSQ can generates random test cases based on the information taken from WSDL such methods parameters data types, e.g. “1234” for integer data type. This type of testing is only usefully from performance testing, they will of no help in actual errors and bugs detection. WSTD-Gen is testing tool which provides a better way of generating test cases, this tool was created for an academic work. Unfortunately I couldn’t find WSTD-Gen application on the internet; I have reviewed this application based on a WSTD-Gen project report [22]. According to the report, users are allowed set constrains for each data-type as show in Figure 3.1.1.
  • 20. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 20 of 50 According to my understandings of this application users can only select from these predefined contains shown in figure 3.1.1, which puts a limit on number of test cases this application can produce. All the applications I reviewed above have some type of a limit on values they can generate and most of the times these values were randomly generated based WSDL. This type of testing is not very useful for a web services tester because it randomly generated values usually ineffective in detecting bugs and errors [21, 22, 23, 24]. Advantages and limitation of FAWST In most the web service testing tools which I reviewed (sub heading 3.1), someone has to generate all the test cases, this is the task which takes most of the user time and resources. FAWST have all the main functionality which any standard web service testing application does such as WSDL checking and parsing, GUI and automatic SOAP request and response but it also have some unique features such as, automatic test data generator, result verifier and detail test report which most of the other web services testing applications lack. FAWST automatic test data generation is very flexible, users can put any constrains on any character as they wish. For example if a tester wants FAWST to generate four characters long string or a number, he/she can put constrains on each character, such as first character can be specific, second can be random between two numbers or alphabets, third character can be random and fourth can be random again. This flexibility allow the users to test their web services with any possible values they can think of. Figure 3.1.1 Constrain window of WSTD-Gen
  • 21. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 21 of 50 Users can perform many different types of testing using FAWST such as Unit testing, Functional testing, End-to-end testing and Sanity testing. These are some of the main and common types of testing usually performed on software for quality insurance. There are just two main limitations of my application. First one is that user can only test SOAP web service and second limitation is not being able to parse all version of WSDL. This is because the API I have used, the API is unable to parse some of the WSDL. I found this problem at very late stage so I was unable change my API. I have not integrated the REST web services in this version FAWST because of the limited time and main point of this project was implement automatic test case generator which can generate any number of different test cases. There are some limitation in result verification, FAWST allow users to check multiple tags and values but if there are two tags with the same name, users would only be able to check the value of the first tag. Summary This chapter includes review of the well-known web service testing applications and shows their limitations and advantages. I have reviewed FAWST and how it is unique from all other web service testing applications available in the market and what are FAWST limitations.
  • 22. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 22 of 50 Requirement specification This chapter describes the requirements identified for FAWST. Starting with functional requirements, which are split into primary and secondary requirements. Primary functional requirements Primary functional requirements are the main requirements of FAWST. These requirement were necessary to achieve the objectives to FAWST.  Checking WSDL: SOAP web service require WSDL file for it to know what functionality web services are offering. So checking if WSDL file or a link to WSDL is correct is very important part of FAWST, because with the wrong WSDL file nothing is going to work.  Parsing and storing WSDL: Parsing WSDL means reading the WSDL files and extracting all the key information, such as method names, parameters names and data-type, complex data-type values and URL for SOAP requests. This information has to be stored so it can be later displayed and used for SOAP requests. This information can be stored on the RAM because this information is only required for a small time.  Collecting and storing test constrains: Collecting and storing test constrains enter by a user as regular expressions is essential part of FAWST Based on the regular expressions all the test case data will be generated. User test constrains are not stored as normal regular expressions but it will be based on regular expressions concept. For example 4-a-d means generate four character long string and the values can only be between ‘a’ and ‘d’. Also these test constrains can be stored on RAM because they will only be required for a small time period and accessing this data from RAM is a lot faster than hard drive.  Automatic test input data generator: This is the unique part of this project. Automatic test inputs will only be generated once FAWST have collected and stored test constrains in regular expressions. Based on the regular expressions FAWST will generate all the test inputs. Without the user regular expressions FAWST will not be able to generate any test inputs.
  • 23. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 23 of 50  Automatic SOAP Request: Generating SOAP request and response is another main functionality of FAWST. This functionality will allow the users to run as many test cases as they want, without having to write a SOAP request for each test case.  Automatic Result Verification: All the SOAP responses should be checked if they passed or failed. The software should allow the user to type in expected values and tag names and the values should be automatically checked against the SOAP response.  User Interface: User interface is necessary to have because it will allow the users to interact with the FAWST easily. Without the user interface it would be incredibly hard to navigate through the FAWST, because it is complex software and it requires a fair amount user input at the start. Basic user interface will include following things;  Home window: On this window user would be able to enter WSDL link.  Show method window: This window will display all the methods names, parameter names and data types.  Test data input windows: This is where user should easily be able to enter constrains for test input generator. Test data input window should be slightly different for each data type.  Report: A report should be generated to display, what parameter values were generated, SOAP request, SOAP response and result (pass/failed) of each test case. This report should be in table so it can be easily checked by the user. Secondary functional requirements Secondary functional requirements are all the optional and additional features of FAWST. These were implemented after the primary functional requirements.  Checking for same test cases: Same test cases should be checked, because there are no point testing web services with the same test cases again and again. It will only waste time and consume unnecessary resources.
  • 24. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 24 of 50  Help windows: Help windows should be added to help the users to use FAWST and to warn if user if they enters wrong values or no values.  Verification of user input: All the user input data should be checked and verified. For example if a text field in FAWST is expecting an integer, it should only take an integer for all the other value it should display an error message to inform users.  Critical actions confirmation: Critical actions like closing the software or Back to main menu should be caught by a confirmation window. Summary In this chapter I have described the identified requirements for FAWST. The requirements were divided into primary and secondary requirements. Each of the requirements was explained in detail for user understanding.
  • 25. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 25 of 50 Application Overview, Design and Application Walk Through This chapter explains in detail with flow chart how FAWST works. I will then reflect on the design of the FAWST, which includes development environment and design patterns. I will then explain in detail with screenshots how FAWST looks and works. Application Overviews Figure 5.1.1 Flow chart
  • 26. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 26 of 50 In this topic, I will explain in detail how FAWST works and how it generates test inputs. FAWST rely on some user input and based on that input it generates test case data. The method is shown in Figure 5.1.1. Also this is just a basic overview of the FAWST main functions and Figure 5.1.1 only shows main functions and flow. Once FAWST is executed, user inputs a WSDL, which then checked to make user it is a WSDL. Once the WSDL is parsed all the information is extracted such as methods name, method input parameters name and data types, URL and complex data types etc. User selects one of the method and enter an integer value for number test cases he/she wants FAWST to generate. For each parameter, user have to enter test input constrains. There is almost no limit on the test input constrains user can enter. There are number of basic options that user can choose from such as: 1. Random value 2. Specific value 3. Between two values 4. Custom (combination of random, specific and between two values). All the above options are explained in detail in application walk-through in sub section 5.3. Once the user enters constrains for all parameters, FAWST generates test inputs based on these constrains. Each test case is checked against all the pervious test cases to make sure that there are no duplications. Once duplication is checked, SOAP request is sent. When a SOAP response is received, user then enters tag names and expected values of each SOAP response which is then checked and stored. User can only enter valid tag names otherwise he/she will keep getting error (“Tag name not found”). Once all the test cases are generated, a report is produced, which display all the generated parameter values, SOAP request and response and result of each test case. Design 5.2.1 Development Environment I have used Java programming language to write code for FAWST because I have over two years of experience of Java and I was more comfortable coding in Java. I have used NetBeans IDE to help me manage and code FAWST because I have used NetBeans before and I knew
  • 27. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 27 of 50 about the shoutcuts and help that I could get from using NetBeans. Most of the GUI (graphical user interface) is created by NetBeans GUI builder. NetBeans GUI builder is very helpful and it saves a lot of time once you know how to use it. 5.2.2 High level design FAWST code design patteren is based on Model View Controller (MVC); MVC is design pattern in which developers simply separate their code into three categories which are model, view and controller. This allows developers to keep the code tidy and as result it makes it easy to find and fix bugs, maintain code and enhance code usability. I have also divided the code in to further sub folders to make it even simpler and tidier for example view package has two sub-packages, MainGUI and ReportGUI. Figure 5.2.2.1 Model View Controller [10]
  • 28. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 28 of 50 Application Walk Through Initail Window: The initial window the user will see is Figure 5.3.1. User can enter WSDL URL or path to the WSDL store on the local machine. Once user enters WSDL, he/she has to press the validate button. Once validate button is pressed WSDL URL or path is checked. If correct, WSDL file is also checked to make sure it is correct. Once the WSDL is validated “check methods” buttons is enabled, which opens new a window shown in Figure 5.3.3. Check method button preform same checks as validate button, just to make sure user has not changed WSDL. Clear button clears the text from the input WSDL test field. Exit button closes the application but a confirmation dialog is display shown in Figure 5.3.2 to confirm user intentions. Exit button does the exact same job throughout the application. Methods Window: Method window is shown in Figure 5.3.3, The information extracted from WSDL file is displayed here, such as, all the method names and paramter name and data type. User can collapse or open each method to see input parameters name and data Figure 5.3.1 Initail window after FAWST is executed Figure 5.3.2 Closing confirmation box
  • 29. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 29 of 50 type. User can select any method and click on the “Test” button to test that paticular method. Once “Test” button is pressed, application checks if user selected a method, a parameter or anything at all; based on result an error is displayed or if correct method was selected, a input message dialog is displayed, where user is asked to enter number of test cases he/she wants FAWST to generate, input message dialog only take integers, all ther other values are rejected with an error message. “Back” button takes the user back to the initial window. Different windows for different Data types: Once the user selects one of the methods and clicks on “Test” Button in the method window and enters of number test he/she wants FAWST to produce. Depending upon the method user wants to test, FAWST will open a new a window for each parameter, where user can enters constrains. Different screens and option are displayed for different data types. I will explain these different windows and options in detail with screen shots below: String: String window is shown in Figure 5.3.4. For string user have three basic options and custom option which is explained in detail below under Custom heading. Any of these option can be selected by clicking on their respective combo boxes; 1. Random String: User has to specify the length of the string. 2. Specific String: User can enter any string they want, it will stay the same though out for every test case. Figure 5.3.3 Method window
  • 30. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 30 of 50 3. Random between two String: Users have to select alphabet one and two, alphabets are between “a” and “z”. Users also have to enter length of the string. Double: Double window is shown in Figure 5.3.5. It is mostly same as string window, the only different is that user have to select decimal point position as well for random number, decimal point position has to be less than random number length. Specific and random between two numbers text field accept only numbers, which could be either decimal or not decimal numbers. Figure 5.3.4 Window for String data type constrains
  • 31. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 31 of 50 Integer: Integer window is almost same as double window, the only difference is that all the text fields only accepts integers and there is not decimal point position. Enum: Enums are specific list of values which a method in the web service will accept, FAWST extract these values from WSDL and display it to the user as shown in Figure 5.3.6. Figure 5.3.6 Window for Enum data type constrains Figure 5.3.5 Window for Double data type constrains
  • 32. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 32 of 50 There are three options for a user, he/she can check the “Select Random one” check box or chose one or multiple values from the list. If user selects multiple values, one of the value will be randomly select for each test case. Char: Char is similar to String window, the only difference is that there are no custom option of char because char is a single character and there is no length option. Also user can only enter single character in the specific option. Boolean: Boolean window is very simple with just one options. User can either select true or false. Figure 5.3.7 Window for Char data type constrains Figure 5.3.8 Window for Boolean data type constrains
  • 33. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 33 of 50 Date: Date window simple and self-explanatory as shown in Figure 5.3.9. User have following four options: 1. Random Date 2. Specific Date 3. Random between two dates 4. Today Date For each option user have to select the date format so the generated date is in format that user wants. Also there is a help window shown in Figure 5.3.10 for date format, this helps the user to understand, what are difference between different date formats. Figure 5.3.9 Window for Date data type constrains Figure 5.3.10 Help Window for Date data type
  • 34. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 34 of 50 Custom user constrain screens: Custom screens are for Integers, String and Double. Custom screens allow the user to put constrains on each individual character, this means FAWST will generate specific values for test cases based on user constrains. Each of custom window is slightly different than the other, but they all have similar design and options. The similarities and difference are following;  Similarities 1. Length of characters 2. Option of specific value, random between two values and random value. 3. User can only select values from combo boxes. 4. Error messages if user forgets to enter a value, shown in Figure 5.3.11  Differences: 1. Custom Integer window: Custom Integer window shown in Figure 5.3.12 only allow users to select numbers between 0 and 9. Figure 5.3.11 Custom Double window with error message
  • 35. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 35 of 50 2. Custom String window: Custom String window have one more option for random value. User can have random character with or without punctuation. Specific value can be any character and for between two values user can only select alphabets. Figure 5.3.12 Custom Integer window Figure 5.3.13 Custom String window
  • 36. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 36 of 50 3. Custom Double window: Custom Double window is very similar to custom Integer window. The only difference is that user can have decimal point, only one point can be selected, if more than one is selected user will get an error. Summary In this chapter, I have explained in how FAWST works. I have also reflected on the high level design and design pattern of the FAWST code. I have also covered in detail how FAWST looks and what are different options on the different screens with screenshots. Figure 5.3.14 Custom Double window
  • 37. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 37 of 50 Testing This chapter will cover different types of testing performed on FAWST, this includes Unit testing, Boundary testing, GUI testing, Acceptance testing and Platform testing, this is to make sure that FAWST is bugs and error free. Unit testing Unit testing is a process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation [26]. I have tested each and every class of this software individually. I wrote a main method of each class and tested all the methods with different values. The above Figure 6.1.1 is one of the example. I have three methods intGen, doubleGen and charToInt in the CommonMethodsTwo class. All of these methods were tested with a lot of different test cases such as, intGen takes two integers, so some of the test for this method are as following; 1. intGen(1,4) // passsed 2. intGen(41,9) // failed, first value has to be smaller than second value 3. intGen(a,4) // failed, only take integers. 4. intGen(-10000,0) // passed Table 6.1.2 shows a list of some of the methods that were individually tested. This table only includes those methods whose test values were small and easy to understand. Figure 6.1.1
  • 38. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 38 of 50 NO# Method partial header Test values Expected Result Actual Result 1 isDouble(String parm) 2.3232 True passed 2 format(String xml) <?xml version="1.0" encoding="utf- 8"?><soap:Envelope xmlns:soap="http://schemas.xmlso ap.org/soap/" xmlns:xsi="http://www.w3.org/20 01" xmlns:xsd="http://www.w3.org/20 01/XMLSchema"><soap:Body></ soap:Body></soap:Envelope> Better layout Passed 3 smallerChar(Object pramOne, Object pramTwo) C,K C Passed 4 biggerChar(Object pramOne, Object pramTwo) C,K K Passed 5 checkConfirm(int pram) 0 // this method is for check box, this method true is check box is checked True Passed 6 isInt(String pram) 323232442424224 False Passed 7 intGen(int rangeFrom, int rangeTo) 1,4 2 or 3 Passed 8 doubleGen(double rangeMin, double rangeMax) 1.2,1.23 1.21 or 1.22 Passed 9 charToInt(String pram) abc 97 Passed 10 checkWSDLAvaila ble(String wsdl) http://www.webservicex.net/count ry.asmx?WSDL True Passed 11 checkPath(String pram) /homes/my301/year3/Project/other /genericbarcode.wsdl True Passed Table 6.1.2. Unit testing
  • 39. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 39 of 50 Boundary testing Boundary testing is where test cases values are extreme values e.g. “maximum, minimum, just inside/outside boundaries, typical values, and error values” [17]. I have checked all the methods on which I could perform boundary testing e.g. intGen, doubleGen, intToChar, isInt and charToInt etc. In Figure 6.2.1 shows one of the example. In the above example (Figure 6.2.1) isInt method checks, if a given value is an int or not. As we can see the four test cases, first one is a maximun value of an int, second value is minum value of int, third testcase value is bigger than the int maxium value and fourth is less then minmun value of the int. so the result should be true,true,false and false and we can see in the Figure 6.2.2 it is correct. User Interface Testing The UI testing is process to make sure that all the functionality of the UI is working and it does what it supposed to do. A thorough UI testing was performed on FAWST to make sure that everything works. Table 6.3.1 shows all the main tests that were performed on the FAWST UI. Figure 6.2.1 Figure 6.2.2
  • 40. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 40 of 50 No. Test Name Description Expected Result Actual Result 1 Run Application Application was opened in NetBeans and through jar file execution. Application first window should open. Passed 2 Clear button Click on the “Clear” button. Text field should get clear. Passed 3 Exit button confirmation box Exit the application once clicked but after the confirmation. Confirmation popup box should appear. Passed 4 Check Methods button By clicking on check methods button, it should check the input WSDL. It should call checkWSDL method and if correct open show method frame Open show method frame or display error if WSDL is wrong. Passed 5 Test button By clicking on test button, it should display an input dialog if a method is selected else it should display a message dialog. Input dialog or message dialog. Passed 6 Main Menu button with confirmation box By clicking on the main menu. Confirmation box should appear for user confirmation. Confirmation box should open. Passed 7 Next button with message dialog box By clicking on the next button. User constrains should be validated. Message box should appear on the screen. Passed 8 Done button with confirmation box By clicking on the done button, confirmation box should appear for user confirmation. Confirmation box should appear Passed 9 Click me Jtable column By click on the click me in report table screen New frame should open with extra information Passed
  • 41. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 41 of 50 Acceptance Testing The best way to determine whether a software had met all the user specifications and requirements is to ask the user to check and test the software. Acceptance testing for FAWST is performed by Dr Bozkurt. He tested the following WSDL which covers all the main data types. 1. http://www.webservicex.net/geoipservice.asmx?WSDL for String 2. http://www.webservicex.net/ConverPower.asmx?WSDL for Double and Enum 3. http://www.webservicex.net/periodictable.asmx?WSDL for String 4. http://www.webservicex.net/MortgageIndex.asmx?WSDL for Int 5. http://www.webservicex.net/sunsetriseservice.asmx?WSDL for Int and Float WSDL Number Issues Passed/Failed Comments Score 1-10 1 is low 1 N/A Passed N/A 8 2 N/A Passed Good example 9 3 Input didn’t work Passed/Failed Issue with one of the method 7 4 N/A Failed Come up with some sensible input. 5 5 N/A N/A N/A N/A Overall Performance 5 4 3 2 1 10 Collapsible table On method screen, user should be able to open and collapse each method and complex to see which values are accepted by the method. Methods should open and collapse Passed. Table 6.3.1 Result from user interface testing
  • 42. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 42 of 50 Design 5 4 3 2 1 Met all user requirements and specifications 5 4 3 2 1 Any comments Overall performance is okay. Design is good. FAWST have met and exceeded all the user requirements and specifications. Dr Bozkurt was happy with the FAWST and he said it met and exceeded all the user requirements and specifications. Platform Testing It is very import for FAWST to run on all major operating system otherwise limited user would be able to use FAWST. I have tested FAWST on the following platforms:  Linux o Fedora  Windows o Windows 8 o Windows 7  Apple OS o OSX Yosimite 10.10.2 FAWST worked on all the above operating systems and there were no issues. Summary In this chapter I have covered different types of testing that were performed on FAWST with examples. The different types of testing performed on FAWST includes Unit testing, Boundary testing, GUI testing, Acceptance testing and Platform testing to make sure that all the bugs and errors are removed. Table 6.4.1 Acceptance testing summary
  • 43. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 43 of 50 Conclusions and recommendations This Chapter describes what I have learned and achieved from this project. I would also explain what I will do differently next time and what new things I would like to add to this project. What I learned and achieved I am very pleased with the outcome of my project. For me, this project has opened a whole new world of programming and testing which I had no clue about. This project was by far the biggest project I have ever done and by doing so learned and achieved a lot. Now I have a very good knowledge of web service and test data generation. Both of them these topics were very new to me and I knew very little about then. This project has broadened my programming skills, I have learnt and practice many different things such working with external libraries, recursion, error handling and much more. I have used and learnt more about the MVC design pattern, which will come very useful in my future programming projects and as well as in my day to day life. This project has also helped me learnt more about project management and time management. These are very useful skills because project management skills reduces the chance of a project failing, it increases the quality of the software, increase efficiency and saves time. As we all know time is money, there should be no questions about how useful time management skill is. The advantage of time management are, it reduces stress, free more time, get more done in same time and we get more time where it needed most. These two skills are not only good for projects but they are useful in all acpects of the life. I am well aware of the limitation of FAWST but I am very proud of what I have achieved, this project does what is supposed to do and it satisfies all the primary and secondary user requirements. This by far the biggest project I have ever created and I have a lot from this project. What would I add or do different The only thing that I would do differently is to make use of more object oriented programming style. I haven’t used inheritance and subclasses, and after completing 80% of
  • 44. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 44 of 50 the project I realized using inheritance and subclasses would have been very useful for this project. FAWST code would have been more generalized, easy to maintain and expand. Apart of this I am very happy with the rest of the project and the way I coded it. However, I certainly would like to add more features if I had more time available, these features are not part of functional or non-functional requirement but it will make this project much better.  Printing and storing report: Printing and storing the report would be a very nice feature; this will allow the users to analyse and compare the results.  Enhance GUI: Current FAWST GUI is not very simple because it is a prototype. By making the following changes to the GUI will enhance user experience of using FAWST: o Make the layout better o Use colours effectively manage attention o Improve navigation  XPath for result verification: XPath is a language that describes a way to locate and process items in XML [20]. XPath will allow FAWST to verify results more accurately and also user would be able to check complex data types.  Enhanced Settings: User preference settings should be added. For example, only unique test case on and off, check multiple elements or single element, keep the same value for tag or different value to each test case and stop is certain number of test case fails.  Increase testing capability: Currently user can only perform unit testing, functional testing, end-to-end testing and sanity testing using FAWST. But I can easily include the load and stress testing options. This will increase the testing capability of FAWST and my application will be used by wider audience:  Help: My application is very straight forward to use but adding more help screens will make it even simpler for user to use.
  • 45. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 45 of 50 Conclusion In conclusion to this report, I would simply say that I had great fun doing this project. FAWST is an amazing application which can really help with the testing of web services. It is also very unique and flexible and I am well aware that there is quite a few things needs to be included in FAWST but, in my point of view creating FAWST was a great achievement for me and as my supervisor said that FAWST had met and exceeded all the user requirements and specifications. FAWST is also very thoroughly tested to make sure that it is bugs and errors free. I would say again it was great fun doing this project and I learnt a lot from this experience.
  • 46. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 46 of 50 Bibliography [1] M. Bozkurt, M. Harman and Youssef, “Testing & Verification In Service Oriented Architecture :A Survey,” in Software Testing, Verification and Reliability, London, Wiley InterScience, 2009, p. 259–350. [2] A. Payong, “Examples of WSDL,” yurtopic, 27 03 2013. [Online]. Available: http://www.yurtopic.com/tech/programming/wsdl-language.html . [Accessed 17 02 2015]. [3] “WSDL Documents,” w3schools, [Online]. Available: http://www.w3schools.com/webservices/ws_WSDL_documents.as. [Accessed 17 02 2015]. [4] “WSDL,” webopedia, [Online]. Available: http://www.webopedia.com/TERM/W/WSDL.html. [Accessed 16 02 2015]. [5] “What Are RESTful Web Services,” Oracle, [Online]. Available: http://docs.oracle.com/javaee/6/tutorial/doc/gijqy.html. [Accessed 17 02 2015]. [6] Dr. P. Oliva, “ECS639U - Web Programming,” Queen Mary, [Online]. Available: http://qmplus.qmul.ac.uk/course/view.php?id=4915. [Accessed 17 02 2015]. [7] “SOAP Introduction,” w3schools, [Online]. Available: http://www.w3schools.com/webservices/ws_SOAP_intro.asp. [Accessed 17 02 2015]. [8] “SOAP Example,” w3schools, [Online]. Available: http://www.w3schools.com/webservices/ws_SOAP_example.asp. [Accessed 17 02 2015]. [9] M. Vaqqas, “WEB DEVELOPMENT,” Dr.dobbs, 23 09 2014. [Online]. Available: http://www.drdobbs.com/web-development/restful-web-services-a-tutorial/240169069. [Accessed 17 02 2015]. [10] “MVC image,” wordpress, [Online]. Available: https://imdjkoch.files.wordpress.com/2012/09/oaf-mvc-architecture.jpg. [Accessed 17 02 2015]. [11] “Software Testing Services in the US: Market Research Report,” IBISWorld, 2014. [12] M. Rozlog, “REST and SOAP,” InfoQ, 04 01 2010. [Online]. Available: http://www.infoq.com/articles/rest-soap-when-to-use-each. [Accessed 02 03 2015]. [13] F. M. Borger and J. , “stateless,” whatis, 01 04 2005. [Online]. Available: http://whatis.techtarget.com/definition/stateless. [Accessed 02 03 2015].
  • 47. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 47 of 50 [14] “What is SoapUI,” SoapUI, [Online]. Available: http://www.soapui.org/about-soapui/what- is-soapui-.html. [Accessed 02 03 2015]. [15] “Award Winning Software,” SMARTBEAR, [Online]. Available: http://smartbear.com/about-us/awards/ . [Accessed 02 03 2015]. [16] S. Hussain, Z. Wang and I. K. Tour, Web Service Testing Tools: A Comparative Study, Beijing,. [17] “Boundary testing,” Wikipedia, [Online]. Available: http://en.wikipedia.org/wiki/Boundary_testing. [Accessed 04 03 2015]. [18] F. Usmani, “Product Scope vs Project Scope,” PM Study Circle, [Online]. Available: http://pmstudycircle.com/2012/01/product-scope-vs-project-scope/. [Accessed 04 03 2014]. [19] A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Fifth Edition, Pennsylvania: Project Management Institute, 2013. [20] M. Rouse, “XPath,” Search SOA, [Online]. Available: http://searchsoa.techtarget.com/definition/XPath. [Accessed 05 03 2015]. [21] J. Z. ,. L.-J. Z. a. N. M. M. Zhong Jie Li, Z. J. Li, J. Zhu, L.-J. Zhang and N. , Towards a Practical and Effective Method for Web Services Test Case Generation, Vancouver: IEEE, 2009, pp. 106 - 114. [22] X. J. Offutt and W. , Generating Test Cases for Web Services Using Data, Fairfax, 2009. [23] “Home,” crosschecknet, [Online]. Available: http://www.crosschecknet.com/. [Accessed 09 03 2015]. [24] “Rational Service Tester for SOA Quality,” IBM, [Online]. Available: http://www- 03.ibm.com/software/products/en/servicetest. [Accessed 09 03 2015]. [25] “What Are Web Services,” Oracle, [Online]. Available: http://docs.oracle.com/javaee/6/tutorial/doc/gijvh.html . [Accessed 14 04 2014]. [26] M. Rouse, “Unit Testing,” Search Software Quality, [Online]. Available: http://searchsoftwarequality.techtarget.com/definition/unit-testing. [Accessed 14 04 2015]. M. [27] “Types of software Testing,” Software Testing Help, [Online]. Available: http://www.softwaretestinghelp.com/types-of-software-testing/. [Accessed 16 02 2015].
  • 48. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 48 of 50 Appendix How FAWST stores test constrains in the Array Lists 8.1.1 Custom String ------ Start for loop ------ User constrains Custom String User constrains Specific User constrains b User constrains Random withOut User constrains 1 User constrains Random with User constrains 1 Length of Array List: 6 ------ End for loop ------ 8.1.2 String with random value ------ Start for loop ------ User constrains String User constrains Random User constrains 4 Length of Array List: 2 ------ End for loop ------ 8.1.3 String with between two values ------ Start for loop ------ User constrains String User constrains BetweenTwoValue User constrains 17-d-e Length of Array List: 2 ------ End for loop ------ 8.1.4 String with specific value ------ Start for loop ------ User constrains String User constrains Specific User constrains abc
  • 49. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 49 of 50 Length of Array List: 2 ------ End for loop ------ 8.1.5 Custom Int ------ Start for loop ------ User constrains Custom Int User constrains Specific User constrains 3 User constrains Random User constrains 1 User constrains Specific User constrains 4 Length of Array List: 6 ------ End for loop ------ 8.1.6 Int with specific value ------ Start for loop ------ User constrains Int User constrains Specific User constrains 3232 Length of Array List: 2 ------ End for loop ------ 8.1.7 Int with random value ------ Start for loop ------ User constrains Int User constrains Random User constrains 4 Length of Array List: 2 ------ End for loop ------ 8.1.8 Int with between two values value ------ Start for loop ------ User constrains Int User constrains BetweenTwoValue User constrains 2131-232314 Length of Array List: 2 ------ End for loop ------
  • 50. Automatic Test Data Generator For Web Service Testing Muhammad Yahya Page 50 of 50 8.1.9 Boolean with specific value ------ Start for loop ------ User constrains Boolean User constrains Specific User constrains False Length of Array List: 2 ------ End for loop ------ 8.1.10Enum values ------ Start for loop ------ User constrains Enum User constrains None User constrains Code User constrains Type User constrains Both Length of Array List: 4 ------ End for loop ------ 8.1.11Enum values ------ Start for loop ------ User constrains Enum User constrains None User constrains Modulo10 Length of Array List: 2 ------ End for loop ------