Paranoid Professor

At the same time you might wish to replace some of the hard coded values (ie usernames and passwords) with variables as well. It is possible to create lists of values which will be used by Gatling.

These feeder lists will be iterated through with each cycle of your test and the values from that line will be used everywhere that variable name is located.

Partial Soap message

    <soapenv:Body>
       <v1:getAccountData>
          <v1:contributor>
             <v1:contributorId>k.marx</v1:contributorId>             
          </v1:contributor>
          <v1:account>
             <v1:id>${ csvAcctId }</v1:id>
             <v1:idType>acctId</v1:idType>
          </v1:account>
          <v1:keyList>
            <v1:key>keu7</v1:key>
          </v1:keyList>
       </v1:getAccountData>
    </soapenv:Body>
 
 

In this case the “${csvAcctId}” variable is filled from the data file. What makes this really powerful is that this variable can be evaluated by Gatling as part of another variable (string)

Code sample

 val zipgetscenario = scenario("zip test get ")
 .repeat(iterations) {
 feed(csv(AcctFilename).circular)
 .exec( 
 AreaObj.getAccountData
 )
 } 

This one feature is so powerful that it alone could easily be the most important and flexible feature. With a single string body and a csv file you can pass in a lot of varied messages.

In one sense, Gatling, is a similar to LoadRunner for creating load and performance tests. It is similar as you need to write a test program which is run and the analysis of the run is graphed. One difference between LoadRunner and Gatling is that Gatling uses Scala as the scripting language.

There will be people who are either pro or anti Scala but from the standpoint of creating scripts in Gatling it is the language that you use – get over it.

The it doesn’t really matter which programming language or script you use. It may take a while to become familiar with the syntax but all programming languages are to a certain degree arbitrary.

Just like any other language it is possible to create a collection of routines that can be used again and again. Unsurprisingly this is true in Gatling as well.

In my next article I will discuss how this can be done so you can create some reusable building blocks to making the testing process easier.

Gatling and the Scala that it uses has quite a bit in common with Java – in this case it is the virtual machine. The Scala code is compiled to byte code and is run on the same JVM that your Java programs run on. Yet the scala script may look a bit odd even if you are a Java programmer. There are quite a few elements that should be fairly similar once you are familiar with the syntax.

Include files

import io.gatling.core.Predef._

The format is slightly different but this is almost identical to a Java import.

import java.io.*

Scala uses a underscore for the wild character and does not require a semicolon to finish the line.

Variables

var username = “chris”;
val password = “password”;

Scala doesn’t actually force you to use actual data types but rather the variable types can be determined by the types of data that is assigned to them. The only two key words you need to remember are var for variables and val for constants. Constants are equivalent to Java variables that are of type “final”.

Methods or Functions

// our own sample function
def myimporttext2(inputval : String, requestedIQ : Integer ) : Int = {
println(inputval)
return 125
}

Although my recorded script doesn’t explicitly have any functions defined it is possible to create your own functions which can be used to perform common tasks.

This Scala example is pretty non-sensical but you can see that it is possible to create a function with multiple inputs. It is also possible to return a value from the function which could be perhaps used in a calculation. It may not be obvious but it is trivial to do such functions with all strings some of the problems may occur when you try and use other data types. This example is returning the primitive Int because it is not possible to return the data type Integer.

Not all functions look alike. It is also possible to define a small chunk of code as seen below.

More methods or functions

val accountNotFoundCheck = regex("<returnCode>1</returnCode>").exists
val returnCodeZeroCheck = regex("<returnCode>0</returnCode>").exists
val bodyFetch = bodyString.saveAs("BODY")
 
This type of function can be used when performing a check after making a rest call.  The last of these was not to do a test but to save the body of a HTTP GET but to do so in an easy way.  Below is an example of actually using these functions.
 
 exec (
 http("update account")
 .post(maintainacctservice)
 //.body(RawFileBody("xsys\\2_XSYS-UpdateAccount.xml"))
 .body(StringBody(updateaccountdataxml))
 .check(returnCodeZeroCheck)
 .check(status.is(200))
 .check(bodyFetch)
 ) 

It might not be obvious in the example code from the recorder but it is possible to do multiple checks after performing a call. Simply add as many checks as is necessary to extract all the information or for verifying correctness.

More variables

Just like java it is possible to have variables and to have constants. Yet, test scripts would either be really long or impossible to manage if you had to create every variable individually as described above. It is possible to abstract out which values need to be changed and which may need to be grouped together. One fairly obvious example might be a users login credentials. We could run all of our tests with a single user but if the test is actually retrieving information from the database our tests might not be accurate as the database server may be caching that particular users information as it is requested so often.

In Gatling will simply group these values into a comma separated file. The file and a bit of syntactic sugar and voila we have converted those repetitive values into variables that our Gatling script can use.

File sample

first,last,id
paul,johnson,98
samual,smith,102
jack,jones,125

Each comma separated value from the first line of the file will be used as a variable. Gatling can use these csv files in several different ways. There are four different methods but in practice there are only really three.

Using one of these strategies is actually pretty easy.

val scn = scenario("RecordedSimulation")
 .feed(csv("users.csv").circular)
 .exec(http("search asrock")
 .get("/computers?f=asrock")
 .headers(headers_4)
 .check(status.is(200)) 
)

Simply add the csv file in at the top of the run and with each iteration it will take a new entry from the data file. In my example, I am not actually using these values but if we wanted we could change the strategy to random and then have lists of computers that will be found or that will not be found. This would give a certain amount of realism to the test and would prevent the database server from caching values to improve the response times in our test.

All of this is accurate but it is difficult to get a good overview of what the original script does. This is a bit sad considering that the original script is quite short. It is possible to convert this script which is essentially “flow of consciousness” into something that is a bit more organized. I have take recorded example and organized it by task. Once this is done, below, you can see this is much much easer to read and modify if necessary.

 import scala.concurrent.duration._
 
 
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import io.gatling.jdbc.Predef._
 
 
 class ModifyRecordedSimulation extends Simulation {
 
 
 var httpProtocol = http
 .baseUrl("http://computer-database.gatling.io")
 .inferHtmlResources(BlackList(""".*\.js""", """.*\.css""", """.*\.gif""", """.*\.jpeg""", """.*\.jpg""", """.*\.ico""", """.*\.woff""", """.*\.woff2""", """.*\.(t|o)tf""", """.*\.png""", """.*detectportal\.firefox\.com.*"""), WhiteList())
 .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3")
 .acceptEncodingHeader("gzip, deflate")
 .acceptLanguageHeader("de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7")
 .userAgentHeader("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36")
  
 httpProtocol = httpProtocol.proxy(Proxy("localhost", 31288).httpsPort(31288))
 
 val headers_4 = Map(
 "Proxy-Connection" -> "keep-alive",
 "Upgrade-Insecure-Requests" -> "1")
  
 object TestHelper {
  
 var searchFailing =
 exec(http("search asrock")
 .get("/computers?f=asrock")
 .headers(headers_4)
 .check(status.is(200))
 )
  
 var searchExisting =
 exec(http("search mac")
 .get("/computers?f=mac")
 .headers(headers_4)
 .check(status.is(200))
 )
  
 var loadExisting =
 exec(http("load mac")
 .get("/computers/19")
 .headers(headers_4)
 .check(status.is(200))
 )
 
 var updateExisting =
 exec(http("update computer")
 .post("/computers/19")
 .headers(headers_4)
 .formParam("name", "COSMAC VIP")
 .formParam("introduced", "1977-01-02")
 .formParam("discontinued", "")
 .formParam("company", "3")
 .check(status.is(200))
 )
 }
 
 val simpletest = scenario("scenario 1")
 .repeat(1) {
 feed(csv("users.csv").circular)
 .exec(
 TestHelper.searchFailing
 ,TestHelper.searchExisting
 ,TestHelper.loadExisting
 ,TestHelper.updateExisting 
 )
 }
  
 setUp(simpletest.inject(atOnceUsers(1))).protocols(httpProtocol)
 } 

The main thing that is really new is that we have created our own Scala object. This object, TestHelper in my case, can be used to group either a single step or multiple steps. This example is using TestHelper as a holder for all the individual tests but you might want to have different blocks of code that you can use in different tests. In that case you might wish to have login credentials in one block and searching queries or helpers in a second block.

The scenario definition is now very short and it is obvious what this test is doing by the names that have been selected. It is even possible to define many different scenarios and have them all run at the same time, thus, not just testing system load but perhaps also covering different functionality. Perhaps these many different tests provide full or nearly full system coverage.

   setUp(scenario1.inject(rampUsers(10) over (ramp seconds))
       .protocols(HttpConfig.value(baseURL)),
     scenario2.inject(rampUsers(66) over (ramp seconds))
       .protocols(HttpConfig.value(baseURL))
   ) 

If you create a test script using the recorder or even if you use your favorite editor the scala test code will need to be stored in the simulations directory (ie %gatling_home%\user-files\simulations). The gatling.bat script (ie %gatling_home%\bin) will look in this directory to compile all classes so you can run your test. If the gatling.bat script is run without any parameters then Gatling will display a list of tests that have been compiled and are available to be run.

 Example
 Choose a simulation number:
      [0] RecordedSimulation
      [1] gapiv5
      [2] bigfootv13
      [3] gpv10 

This is a list of the available “classes”, one for each test. Just like with Java each class name of each object must be unique. The class in the recorded example is called “RecordedSimulation”. This actually matches the name of the file that we saw in the recorder dialog. Simply press the number 0 (in this case) to run our recorded simulation.

Gatling test scripts are actually small command line programs that once compiled are run with all of the output going to the terminal screen. Below is the output from running our recorded simulation.

 Choose a simulation number:
      [0] RecordedSimulation
      [1] gapiv5
      [2] bigfootv13
      [3] gpv10
 0
 Select run description (optional)
 my first script
 Simulation RecordedSimulation started...
 
 
 =================================================================
 2019-11-07 14:30:27                            5s elapsed
 ---- Requests ---------------------------------------------------
 > Global                                    (OK=1      KO=0     )
 > search asrock                             (OK=1      KO=0     )
 
 
 ---- RecordedSimulation -----------------------------------------
 [---------------------------------------------------------------]  0%
           waiting: 0      / active: 1      / done: 0
 ==================================================================
 
==================================================================
 2019-11-07 14:30:33                           11s elapsed
 ---- Requests ---------------------------------------------------
 > Global                                    (OK=5      KO=0     )
 > search asrock                             (OK=1      KO=0     )
 > search mac                                (OK=1      KO=0     )
 > fetch computer                            (OK=1      KO=0     )
 > update computer                           (OK=1      KO=0     )
 > update computer Redirect 1                (OK=1      KO=0     )
 
 
 ---- RecordedSimulation -----------------------------------------
 [###########################################################]100%
           waiting: 0      / active: 0      / done: 1
 ==================================================================
 
 
 Simulation RecordedSimulation completed in 11 seconds
 Parsing log file(s)...
 Parsing log file(s) done
 Generating reports...
 
 
 ==================================================================
 ---- Global Information -----------------------------------------
 > request count                           5 (OK=5      KO=0     )
 > min response time                     176 (OK=176    KO=-     )
 > max response time                    3069 (OK=3069   KO=-     )
 > mean response time                    815 (OK=815    KO=-     )
 > std deviation                        1130 (OK=1130   KO=-     )
 > response time 50th percentile         240 (OK=240    KO=-     )
 > response time 75th percentile         400 (OK=400    KO=-     )
 > response time 95th percentile        2535 (OK=2535   KO=-     )
 > response time 99th percentile        2962 (OK=2962   KO=-     )
 > mean requests/sec                   0.417 (OK=0.417  KO=-     )
 ---- Response Time Distribution ---------------------------------
 > t < 800 ms                              4 ( 80%)
 > 800 ms < t < 1200 ms                    0 (  0%)
 > t > 1200 ms                             1 ( 20%)
 > failed                                  0 (  0%)
 ==================================================================

 Reports generated in 1s.
 Please open the following file: C:\Users\chris\gatling-charts-highcharts-bundle-3.3.0\results\recordedsimulation-20191107133020038\index.html 

This output actually does tell us some information about our test. Because some meaningful comments were inserted into our test script.

.exec(http(“search mac”)

We can see those meaningful names as the script progresses. At the end of the output we can see that there was four calls that were processed in less than 800 milliseconds while a fifth call took longer than 1200 milliseconds. We can also see that there were no failures.

Despite the fact that this output does indeed tell us about how the run went it is actually much easier to look at the html page that we created to see a graphical representation of our script. The last line of the run output gives us the path to the index.html file which has been generated. If we view that html file with our web browser we will see something similar to the following.

This image is just part of the web page that is created but it does contain all of the statistics. We can see which of our calls succeeded and how long it took (min, max and average). In this case all of our numbers are the same but if we did run this script multiple times we would be able to see how many were in the 50, 75^th and 95^th percentiles.

Each one of our steps only perform one task so there is little reason to look much deeper but if each of these performed more than one call or loaded some important resources we could get a better look at the times by examining the details of each one of these steps.

If we look in the upper right hand corner of the picture we can see the name of the class, the runtime and the description that we gave when running it (ie my first run script)

Our script was generated by the recorder as pure Scala code. In my next blog I will clarify a bit of the Scala syntax.

Of the three load and performance testing tools that I am going to describe Gatling is the new comer. While LoadRunner has been around since 1994 Gatling was first released in 2011 and since then has had 2 major releases with the most current version being 3.3.1. Gatling supports fewer protocols but can easily be used for testing HTTP, HTTPS, Web sockets and REST Api’s. In this sense it is more limited than LoadRunner yet as more and more applications are either browser based or implement a service that is available over the internet Gatling is certainly the right tool for the times. Gatling is an open source tool which uses Scala as the scripting language for writing tests. Why Scala?

• Scala can interact with Java
• It can use Java libraries
• Cleaner code
• Smaller code thus easier to have an overview

The only prerequisite for using Gatling is that you have a copy of the Java JDK installed on your machine. The Scala code is compiled into byte code which is then executed on the Java Virtual Machine of your Java installation. This has the pleasant side effect that you can use the native Java libraries to help in your testing scripts.

Installation

Gatling is distributed as a zip file that you can download from their homepage, https://gatling.io. Installing it is just a matter of finding a good location and unpacking the zip file.

Ie.

C:\users\myuser\testingapps\gatling-charts-highcharts-bundle-3.3.1

There is really nothing left to do if the JDK is installed. This directory where Gatling is installed should be stored in the environment variable GATLING_HOME. This is not necessary as this information can be determined by the start scripts but it is cleaner and this value can then be used in batch files or shell scripts. It is worth noting that a OpenJDK is also a JDK and worked just fine with Gatling for all of my tests.

Recorder

Gatling actually has a solution for people who are not familiar with Scala. Bundled with Gatling is a recorder which can be used for capturing an interaction between the user and a web site.. This recording tool is used as a proxy when surfing your web pages which allows it to capture all the URL’s and necessary headers. The actual output of this tool is a Scala script that will reproduce the same set of steps including all pauses made by the user. This recorder start script is started from the same directory as the gatling start script (%GATLING_HOME%\bin)

This may sound like very good news that web tests can be recorded except it is perhaps a bit simplistic. A long time ago it was pretty much determined from a web security standpoint that you cannot simply record a series of steps and then allow them to replay that exact set of steps without negative consequences. This type of action actually has its own name – a replay attack. A replay attack is somehow capturing a series of steps, which may also include the users credentials. Because of the inherent danger of allowing a strict copy and replay of a series of steps most if not all non-trivial web pages will have hidden variables that are set at each step but are unique each time you perform the same series of steps. The values form these variables must be used in subsequent steps and thus prevent you from creating a test script from a recording and using it without any changes.

These unique variables must be retrieved from the web pages and used for each test. So although Gatling recorder will capture all the hard coded unique values you will need to amend the test scripts. You will need go through the recorded script and make amendments to parse out these dynamic values and to use them at the correct locations while following the test steps.

If you are familiar with the website being tested you will find these recorded scripts to be fairly easy to read. Below is a small test along with the recorded test of these same steps. This example what you would see if you followed the Gatling quick start tutorial (https://gatling.io/docs/current/quickstart/)

This test will perform the following operations.

• Goto computer database gatling.io web site
• Search for computer ASRock
• Search for computer Mac
• click on computer (internal id #19)
• save computer id #19 with these details

This script almost exactly what is produced by the recorder. I have trimmed out a few unimportant steps and have added a few important changes (use my local proxy) and changed the comment values to be representative of what action they are performing.

 import scala.concurrent.duration._
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import io.gatling.jdbc.Predef._
 
 class RecordedSimulation extends Simulation {
 
 var httpProtocol = http
 .baseUrl("http://computer-database.gatling.io")
 .inferHtmlResources(BlackList(""".*\.js""", """.*\.css""", """.*\.gif""", """.*\.jpeg""", """.*\.jpg""", """.*\.ico""", """.*\.woff""", """.*\.woff2""", """.*\.(t|o)tf""", """.*\.png""", """.*detectportal\.firefox\.com.*"""), WhiteList())
 .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3")
 .acceptEncodingHeader("gzip, deflate")
 .acceptLanguageHeader("de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7")
 .userAgentHeader("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36")
 
 httpProtocol = httpProtocol.proxy(Proxy("localhost", 31288).httpsPort(31288))
 
 val headers_4 = Map(
 "Proxy-Connection" -> "keep-alive",
 "Upgrade-Insecure-Requests" -> "1")
 
 val headers_7 = Map(
 "Origin" -> "http://computer-database.gatling.io",
 "Proxy-Connection" -> "keep-alive",
 "Upgrade-Insecure-Requests" -> "1")
 
 val scn = scenario("RecordedSimulation")
 .exec(http("search asrock")
 .get("/computers?f=asrock")
 .headers(headers_4)
 .check(status.is(200))
 )
 .pause(3)
 .exec(http("search mac")
 .get("/computers?f=mac")
 .headers(headers_4)
 .check(status.is(200))
 )
 .pause(2)
 .exec(http("fetch computer")
 .get("/computers/19")
 .headers(headers_4)
 .check(status.is(200))
 )
 .pause(2)
 .exec(http("update computer")
 .post("/computers/19")
 .headers(headers_7)
 .formParam("name", "COSMAC VIP")
 .formParam("introduced", "1977-01-02")
 .formParam("discontinued", "")
 .formParam("company", "3")
 .check(status.is(200))
 )
 setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
 } 

What isn’t in the original recording is a status check that the HTTP call succeeded. I have also added a line that will amend the httpProtocol variable to also include my local proxy server. With just a small amount of imagination you can think about passing in a proxy port and host and if the port is not equal to zero then add the proxy to the httpProtocol as part of your test script.

One thing that might not be immediately obvious is that most of the script is not actually running anything but defining the variable scn. This scenario variable is what will be performed by Gatling. The last line of the scala code actually does the setup which will cause this variable to be used by Gatling to call our series of steps.

setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)

In this test script we are going to use only one user and that user should be created all at one time. This sounds pretty silly for a single user, however, if you are speaking about creating 10 or 100 users you can see that immediately quite a load will be created. It is possible that a system which might be able to support 500 concurrent users doesn’t work very well if all 500 start at once within 2 milliseconds. I will cover increasing the number of users and different ways of generating that load later.

My next post will cover how we actually run this script.