Building a personal computer – selecting a case

Picking a case is to a certain extent a decision based on your preferences. The CPU or GPU won’t go any faster by having a larger or smaller case.

The only limitations are that the case you purchase does support the size of your motherboard.

  • Extended ATX (E-ATX)
  • ATX
  • micro ATX (mATX)
  • Mini ITX

However, there is yet another couple of considerations when picking a case. The case has to be large enough to contain the motherboard but it must also support a graphics card if you choose a discrete graphics card. Simply put you cannot get a graphics card that is 12” long if there is only space for a 10” card which is the average size.

The available size for the graphic card depends on the case but if you have a tower case it also depends on any internal mounting hardware such as what might be used for hard disks or DVD’s.

One more consideration on the size of case is how large the CPU cooler is, if you choose an aftermarket air cooled fan for your CPU. Some CPU’s are sold with a cooler which typically appears to be a fan that is laying on top of the CPU. As you can imagine this not a very high cooler and would probably fit into all or almost all cases.

After market CPU coolers can be quite tall. The Noctua NH-D15 is actually 16.5cm tall. Thus if you are choosing such a CPU cooler you have to get case that will accommodate such a large cooler.

One of the least important attributes for the running of the computer is how the case looks but that is important for the owner as it displays a sense of style. The Final but probably most important consideration is airflow of the case. Both the GPU and CPU will generate heat when the computer is running. These pieces of equipment last longer and work better when cooled and to major (air cooled) or minor (water cooled) degree you should get a case that allows in cool air. How much air is required depends on the CPU and GPU but also how many intake and exhaust fans are running.

Case Dissassembly and review

Posted in DIY | Tagged | Comments Off on Building a personal computer – selecting a case

Building a personal computer – selecting a motherboard

Deciding on a motherboard is less of a religion than finding something that you can afford and that supports your processor.

The question of the motherboard is more complicated depending on how high end of a CPU you are planning on using. If you are just planning on a small 4 core/8 thread pc you could probably get away with any of the existing compatible motherboards. It would most certainly be overkill to get the brand new x570 AMD motherboards if you are planning on getting an Ryzen 3 with only has 4 cores.

With certainty you could find a motherboard from last years lineup and use it with even the most current Ryzen 3, well with a few exceptions. If You are planning on going to the other extreme and getting a high end processor you might then opt for a newer motherboard with all of the new functionality. It might not be necessary to have a PCI 4 bus for your new Ryzen 7 or 9 CPU but if you are getting a high end processor you might also be getting a high end graphics card and that would benefit from the faster PCI 4 bus.

In my case the decision was to go with one of the existing motherboards that supports the AM4 and 3rd gen ryzen or to get one of the new motherboards that is chock full of newer technology.

MotherboardMotherboard
x470x570
pci 3pci4
usb 3.1usb 3.2
8 pci 3 lanes20 pci 4 lanes
max 64gb rammax 128gb ram

Depending on the boards you might also be talking about better built in audio, more support m.2 cards, built in Wifi or even 2.5 gigabyte Ethernet port. Other things might also be availble such as newer easier to use versions of Bios or the ability to flash the firmware without even having a CPU.

Both motherboards have the AM4 socket which was released in 2017 and seems to be holding strong. The socket seems scheduled to be in use into 2020. How to actually select a motherboard is really difficult. I have done a lot of research and it is not to difficult to find negative reviews for all of the different motherboards.

My old computer is about 56 years old (in dog years) and I can only imagine that my new computer will also be kept for longer then the normal two or three years upgrade cycle. With this in mind I will be getting the current x570 which supports pci4 which will future proof things.

Wish list
2 x M.2 drive support
128gb ram support
USB 3.2
7 segment display for post codes

A lot of these options are actually available on a lot of x570 motherboards with the exception of the 7 segment display for post codes.

There is of course one additional consideration, the ability to install Linux. This actually falls under the heading of the UEFI bios and the ability to disable secure boot so I can install a non-microsoft signed operating system.

Note: make sure you get a motherboard with the same socket as your CPU.

Posted in DIY | Tagged | Comments Off on Building a personal computer – selecting a motherboard

Building a personal computer – selecting a CPU

Building a pc tends to be done by enthusiasts and so it is done by people who know the difference between ram and the hard disk. With that in mind selecting a cpu tends to become more of a question of which technical religion do you follow – Intel or AMD. Over the last few decades the undisputed champion of performance has been Intel, after all they did create the processor that made its way into the IBM personal computer from which most pc’s share their common ancestry.

Despite being in the AMD camp for a very long time I had also eyed the Intel I7 processors but never quite had the money for a processor which hasn’t changed dramatically over the last decade. It was only in 2017 when AMD changed the game. Since then I had been hoping to get the 1st or 2nd generation threadripper which would have more than doubled the processing power I currently have if my computer were twice as powerful as it is.

Late last year when I started to have some computer troubles I decided to get started. The AMD 3rd generation threadrippers were coming out but AMD had gone over the top. The new low end threadripper now has 24 cores and despite the relative cost performance to the Intel being quite good I seem to have been priced out of the threadripper market. It isn’t just the processor but the cost of an entire system which was getting a bit rich for my budget. Well not only that but 16 core/32 thread processor was probably more than I could fully utilize.

GenerationModelCoresCost
1Threadripper 1950x16617€
2Threadripper 2970wx241033€
2Threadripper 2990wx321979€
3Ryzen 9 3900x12517€
3Ryzen 9 3950x16979€
3Threadripper 3960x241400€
3Threadripper 3970x321999€
3Threadripper 3990x 643000+

This is a tremendous jump in capability over just a few years which in itself would be amazing but AMD’s entire line has taken a jump in abilities but they have also just released their own enthusiast chip Ryzen 9 3900x. This chip has 12 cores and 24 threads and the bigger brother 3950x has also been released. This chip will have 16 cores and 32 threads which is just as formidable as the first generation threadripper.

The Ryzen 9 3900x is an amazing chip which is quite muscular and should be relatively future proof for my particular uses.

The decision of the processor allows me to move on to the next decision which is which motherboard I should purchase.

Posted in DIY | Tagged | Comments Off on Building a personal computer – selecting a CPU

Building a personal computer

My experiences of building my own pc’s has been somewhat mixed. I had considerable help building the last two machines I have had by my friend Mikhail. When I say I had help building that means I created the requirements and he waded through the technical jungles to come up with a compatible parts list.

Oh, sure I helped to assemble them. My last one I assembled in its entirety including all the backwards fans. Once I was done my work was reviewed which is when some of my missteps were discovered and then corrected.

This time around I have to fly solo and do everything myself. I am both good with my hands and not afraid of electronics so how hard could it be? It did take a lot longer to come up to speed on all the changes in technology than I thought it would but I thought I would. Not only does the internet is full of all the information you need to know but it is also covered in a greater depth than I am prepared to go.

I know a couple of people who are considering building their own computer so I am pulling together a general list of the technology as well as a basic explanation of each. I am listing what I learned during my research. This will highlight a lot of the interesting or tricky steps as well as include links to a lot of supporting material.

Over the next few days I will be writing up my notes on the entire process and a quick summary after the process is over.

This series contains links to some of the resources I used as well as a few hardware vendors. I have not received anything from any of the vendors or websites. This information is included as it was either described better than I could do it or by someone who could be considered to be an expert.

Posted in DIY | Tagged | Comments Off on Building a personal computer

Artificially Intelligent or Creepy

I guess it happened a few years ago when I became a bit more cognizant that those helpful computers were a bit smarter than they first seemed. Perhaps smarter isn’t the right term, perhaps intrusive. The exact incident was when I was sending an email off and had forgotten the attachment. The email service that I was using had analyzed the text and pointed out that I had forgotten the attachment that I had mentioned. Indeed it was nice to get this oversight corrected but it was a bit disconcerting, what other keywords or phrases would my email provider be looking for. Would a question about insurance bring on an avalanche of insurance spam ? What about other topics of a more personal nature? Will I start to get mails about midwives and baby supplies? Perhaps mails about switching gynecologist.

I was actually relieved so far not as not to see anything like that, well based on my emails, I don’t think that this is the case based on my browsing history. It is difficult to have no browsing history but I have tried to no longer be logged into accounts and keep as few cookies as possible. Well, imagine my surprise when I logged into a seldom used email account to see this mail.

Not really remembering what a google map timeline was I took a look and it was stroll down memory lane. This thing has been on for some time and although it doesn’t have all of my movements I can see in a lot of instances which countries, cities, addresses or businesses I have visited. If you zoom in on a specific day you can see what time you started your journey and from which location. The addresses are amazingly specific and correct, which how long you spent at or near a location and how long it took to get to the next location. If this information would be available to a criminal it would be amazing.

You can probably guess that your “victim” worked at a certain location as they seem to be spending about 8 hours a day between 8 and 6 pm during the week at that location. You can see if that work location is close or far away. If you have somehow gotten this information you might already know that this victim is someone worth robbing but this information might also clarify that. Do they eat out a lot? Do they visit expensive stores or shops? What is the best time of the day or day of the month to “visit” their home. If you were a divorce lawyer this might also look pretty interesting. Your client’s husband claims to simply work late or be out of town on business when he is actually been tracked to a specific address across town during that same frame.

https://www.openstreetmap.org/

This information might also be quite interesting as a form of corporate espionage, where did the head of global acquisitions go? We know that she deals primarily with automotive industry.

https://www.openstreetmap.org/

You might be thinking that if I am dumb enough to leave my GPS on as well as enable this feature from Google I deserve all the tracking. This would be true, especially about the enable the feature from Google but it turns out that rarely have my GPS enabled, all of this information appears to have been gathered using either wifi or tracking via the various cell towers of the mobile network.

These markedup map examples are a markup using openstreetmap.org, the actual google maps are much scarier. The google map has wonderful detail listing ATM’s, coffee shops, barbershops, and even links to the various establishments. It is possible to get address, telephone number, opening hours and more.

I really like the openstreetmaps.org site and the information that it does provide but it does pale in comparison to Google Maps.

If this is the type of tracking that can be done with just some passive cooperation I now understand now how during large government investigations are able to have a detailed list of the “criminal” and all of his or her associates as well as detailed movement prior to the crime.

Getting clear of tracking

Well, there is probably no way to be truly clear of being tracked by your cell provider, the phone OS provider or the apps themselves but it is possible to turn off a few of the most direct tracking settings.

  1. Turn off location collection on your email account.
  2. Stop collection gps and wifi information
  3. Remove old history data

1. Gmail

Log into your email account and go to the settings.

Change the location (Standortverlauf) from On to Off.

This should stop that particular email account from wanting to track you.

2. turn off tracking on your phone

Turn off the collection of data on your smart phone (Auf meinen Standort zugreifen). This has the advantage of not tracking you and may also extend the life of your battery charge.

3. delete the history

I was quite surprised at this one. I was looking at my email account in an attempt to delete this information only to find out that does not seem to be possible. You can delete the history but to do that you need to do it on your cell phone. However, once you delete the history there your timeline as visible from your web browser no longer has any information on your previous travels.

Note: Openstreetmap.org simply provides the maps, they don’t track you and the example above was showing something similar to a Google map. Why not a google map? Well, there are other restrictions of displaying a Google map and I also do not wish to share my location data.

Posted in programming | Comments Off on Artificially Intelligent or Creepy

The great firewall of china

China apparently has a cyber wall since 1997. Who knew? Its role in internet censorship in China is to block access to selected foreign websites and to slow down cross-border internet traffic. The effect includes: limiting access to foreign information sources, blocking foreign internet tools (e.g.google search, facebook, twitter, Wikipedia and others) and mobile apps, and requiring foreign companies to adapt to domestic regulations. It also nurtures domestic companies. The great firewall was formerly operated by the SIIO since 2013 it was officially operated by the Cyberspace administration of china. Its job is to translate the Communist party of chinas will into technical specifications. The SARs* are not affected. The SARs are Hong Kong and Macau because they have their own governmental and legal systems. The word was firstly used by Geremie Barmé an Australien writer who officially wasn’t famous but many people think of him as famous.

Tell me in comments below what you think of the great firewall

Sources:https://en.wikipedia.org/wiki/Great_Firewall

*Special administrative regions

Posted in Soapbox | Tagged | Comments Off on The great firewall of china

Gatling and file variables

At the same time you might wish to replace some of the hard coded values (ie usernames and passwords) with variables as well. It is possible to create lists of values which will be used by Gatling.

These feeder lists will be iterated through with each cycle of your test and the values from that line will be used everywhere that variable name is located.

Partial Soap message

    <soapenv:Body>
       <v1:getAccountData>
          <v1:contributor>
             <v1:contributorId>k.marx</v1:contributorId>             
          </v1:contributor>
          <v1:account>
             <v1:id>${ csvAcctId }</v1:id>
             <v1:idType>acctId</v1:idType>
          </v1:account>
          <v1:keyList>
            <v1:key>keu7</v1:key>
          </v1:keyList>
       </v1:getAccountData>
    </soapenv:Body>
 
 

In this case the “${csvAcctId}” variable is filled from the data file. What makes this really powerful is that this variable can be evaluated by Gatling as part of another variable (string)

Code sample

 val zipgetscenario = scenario("zip test get ")
 .repeat(iterations) {
 feed(csv(AcctFilename).circular)
 .exec( 
 AreaObj.getAccountData
 )
 } 

This one feature is so powerful that it alone could easily be the most important and flexible feature. With a single string body and a csv file you can pass in a lot of varied messages.

In one sense, Gatling, is a similar to LoadRunner for creating load and performance tests. It is similar as you need to write a test program which is run and the analysis of the run is graphed. One difference between LoadRunner and Gatling is that Gatling uses Scala as the scripting language.

There will be people who are either pro or anti Scala but from the standpoint of creating scripts in Gatling it is the language that you use – get over it.

The it doesn’t really matter which programming language or script you use. It may take a while to become familiar with the syntax but all programming languages are to a certain degree arbitrary.

Just like any other language it is possible to create a collection of routines that can be used again and again. Unsurprisingly this is true in Gatling as well.

In my next article I will discuss how this can be done so you can create some reusable building blocks to making the testing process easier.

Posted in programming | Tagged | Comments Off on Gatling and file variables

Gatling and Scala

Gatling and the Scala that it uses has quite a bit in common with Java – in this case it is the virtual machine. The Scala code is compiled to byte code and is run on the same JVM that your Java programs run on. Yet the scala script may look a bit odd even if you are a Java programmer. There are quite a few elements that should be fairly similar once you are familiar with the syntax.

Include files

import io.gatling.core.Predef._

The format is slightly different but this is almost identical to a Java import.

import java.io.*

Scala uses a underscore for the wild character and does not require a semicolon to finish the line.

Variables

var username = “chris”;
val password = “password”;

Scala doesn’t actually force you to use actual data types but rather the variable types can be determined by the types of data that is assigned to them. The only two key words you need to remember are var for variables and val for constants. Constants are equivalent to Java variables that are of type “final”.

Methods or Functions

// our own sample function
def myimporttext2(inputval : String, requestedIQ : Integer ) : Int = {
println(inputval)
return 125
}

Although my recorded script doesn’t explicitly have any functions defined it is possible to create your own functions which can be used to perform common tasks.

This Scala example is pretty non-sensical but you can see that it is possible to create a function with multiple inputs. It is also possible to return a value from the function which could be perhaps used in a calculation. It may not be obvious but it is trivial to do such functions with all strings some of the problems may occur when you try and use other data types. This example is returning the primitive Int because it is not possible to return the data type Integer.

Not all functions look alike. It is also possible to define a small chunk of code as seen below.

More methods or functions

val accountNotFoundCheck = regex("<returnCode>1</returnCode>").exists
val returnCodeZeroCheck = regex("<returnCode>0</returnCode>").exists
val bodyFetch = bodyString.saveAs("BODY")
 
This type of function can be used when performing a check after making a rest call.  The last of these was not to do a test but to save the body of a HTTP GET but to do so in an easy way.  Below is an example of actually using these functions.
 
 exec (
 http("update account")
 .post(maintainacctservice)
 //.body(RawFileBody("xsys\\2_XSYS-UpdateAccount.xml"))
 .body(StringBody(updateaccountdataxml))
 .check(returnCodeZeroCheck)
 .check(status.is(200))
 .check(bodyFetch)
 ) 

It might not be obvious in the example code from the recorder but it is possible to do multiple checks after performing a call. Simply add as many checks as is necessary to extract all the information or for verifying correctness.

More variables

Just like java it is possible to have variables and to have constants. Yet, test scripts would either be really long or impossible to manage if you had to create every variable individually as described above. It is possible to abstract out which values need to be changed and which may need to be grouped together. One fairly obvious example might be a users login credentials. We could run all of our tests with a single user but if the test is actually retrieving information from the database our tests might not be accurate as the database server may be caching that particular users information as it is requested so often.

In Gatling will simply group these values into a comma separated file. The file and a bit of syntactic sugar and voila we have converted those repetitive values into variables that our Gatling script can use.

File sample

first,last,id
paul,johnson,98
samual,smith,102
jack,jones,125

Each comma separated value from the first line of the file will be used as a variable. Gatling can use these csv files in several different ways. There are four different methods but in practice there are only really three.

TypeDescription
QueueRead a value but generate an error if end of file is reached
RandomPick values from list at random
CircularRead items from list but go back to top if you reach end of file
ShuffleShuffle the entries and then treat list like a queue

Using one of these strategies is actually pretty easy.

val scn = scenario("RecordedSimulation")
 .feed(csv("users.csv").circular)
 .exec(http("search asrock")
 .get("/computers?f=asrock")
 .headers(headers_4)
 .check(status.is(200)) 
)

Simply add the csv file in at the top of the run and with each iteration it will take a new entry from the data file. In my example, I am not actually using these values but if we wanted we could change the strategy to random and then have lists of computers that will be found or that will not be found. This would give a certain amount of realism to the test and would prevent the database server from caching values to improve the response times in our test.

All of this is accurate but it is difficult to get a good overview of what the original script does. This is a bit sad considering that the original script is quite short. It is possible to convert this script which is essentially “flow of consciousness” into something that is a bit more organized. I have take recorded example and organized it by task. Once this is done, below, you can see this is much much easer to read and modify if necessary.

 import scala.concurrent.duration._
 
 
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import io.gatling.jdbc.Predef._
 
 
 class ModifyRecordedSimulation extends Simulation {
 
 
 var httpProtocol = http
 .baseUrl("http://computer-database.gatling.io")
 .inferHtmlResources(BlackList(""".*\.js""", """.*\.css""", """.*\.gif""", """.*\.jpeg""", """.*\.jpg""", """.*\.ico""", """.*\.woff""", """.*\.woff2""", """.*\.(t|o)tf""", """.*\.png""", """.*detectportal\.firefox\.com.*"""), WhiteList())
 .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3")
 .acceptEncodingHeader("gzip, deflate")
 .acceptLanguageHeader("de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7")
 .userAgentHeader("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36")
  
 httpProtocol = httpProtocol.proxy(Proxy("localhost", 31288).httpsPort(31288))
 
 val headers_4 = Map(
 "Proxy-Connection" -> "keep-alive",
 "Upgrade-Insecure-Requests" -> "1")
  
 object TestHelper {
  
 var searchFailing =
 exec(http("search asrock")
 .get("/computers?f=asrock")
 .headers(headers_4)
 .check(status.is(200))
 )
  
 var searchExisting =
 exec(http("search mac")
 .get("/computers?f=mac")
 .headers(headers_4)
 .check(status.is(200))
 )
  
 var loadExisting =
 exec(http("load mac")
 .get("/computers/19")
 .headers(headers_4)
 .check(status.is(200))
 )
 
 var updateExisting =
 exec(http("update computer")
 .post("/computers/19")
 .headers(headers_4)
 .formParam("name", "COSMAC VIP")
 .formParam("introduced", "1977-01-02")
 .formParam("discontinued", "")
 .formParam("company", "3")
 .check(status.is(200))
 )
 }
 
 val simpletest = scenario("scenario 1")
 .repeat(1) {
 feed(csv("users.csv").circular)
 .exec(
 TestHelper.searchFailing
 ,TestHelper.searchExisting
 ,TestHelper.loadExisting
 ,TestHelper.updateExisting 
 )
 }
  
 setUp(simpletest.inject(atOnceUsers(1))).protocols(httpProtocol)
 } 

The main thing that is really new is that we have created our own Scala object. This object, TestHelper in my case, can be used to group either a single step or multiple steps. This example is using TestHelper as a holder for all the individual tests but you might want to have different blocks of code that you can use in different tests. In that case you might wish to have login credentials in one block and searching queries or helpers in a second block.

The scenario definition is now very short and it is obvious what this test is doing by the names that have been selected. It is even possible to define many different scenarios and have them all run at the same time, thus, not just testing system load but perhaps also covering different functionality. Perhaps these many different tests provide full or nearly full system coverage.

   setUp(scenario1.inject(rampUsers(10) over (ramp seconds))
       .protocols(HttpConfig.value(baseURL)),
     scenario2.inject(rampUsers(66) over (ramp seconds))
       .protocols(HttpConfig.value(baseURL))
   ) 

Posted in programming | Tagged | Comments Off on Gatling and Scala

Running a Gatling test script

If you create a test script using the recorder or even if you use your favorite editor the scala test code will need to be stored in the simulations directory (ie %gatling_home%\user-files\simulations). The gatling.bat script (ie %gatling_home%\bin) will look in this directory to compile all classes so you can run your test. If the gatling.bat script is run without any parameters then Gatling will display a list of tests that have been compiled and are available to be run.

 Example
 Choose a simulation number:
      [0] RecordedSimulation
      [1] gapiv5
      [2] bigfootv13
      [3] gpv10 

This is a list of the available “classes”, one for each test. Just like with Java each class name of each object must be unique. The class in the recorded example is called “RecordedSimulation”. This actually matches the name of the file that we saw in the recorder dialog. Simply press the number 0 (in this case) to run our recorded simulation.

Gatling test scripts are actually small command line programs that once compiled are run with all of the output going to the terminal screen. Below is the output from running our recorded simulation.

 Choose a simulation number:
      [0] RecordedSimulation
      [1] gapiv5
      [2] bigfootv13
      [3] gpv10
 0
 Select run description (optional)
 my first script
 Simulation RecordedSimulation started...
 
 
 =================================================================
 2019-11-07 14:30:27                            5s elapsed
 ---- Requests ---------------------------------------------------
 > Global                                    (OK=1      KO=0     )
 > search asrock                             (OK=1      KO=0     )
 
 
 ---- RecordedSimulation -----------------------------------------
 [---------------------------------------------------------------]  0%
           waiting: 0      / active: 1      / done: 0
 ==================================================================
 
==================================================================
 2019-11-07 14:30:33                           11s elapsed
 ---- Requests ---------------------------------------------------
 > Global                                    (OK=5      KO=0     )
 > search asrock                             (OK=1      KO=0     )
 > search mac                                (OK=1      KO=0     )
 > fetch computer                            (OK=1      KO=0     )
 > update computer                           (OK=1      KO=0     )
 > update computer Redirect 1                (OK=1      KO=0     )
 
 
 ---- RecordedSimulation -----------------------------------------
 [###########################################################]100%
           waiting: 0      / active: 0      / done: 1
 ==================================================================
 
 
 Simulation RecordedSimulation completed in 11 seconds
 Parsing log file(s)...
 Parsing log file(s) done
 Generating reports...
 
 
 ==================================================================
 ---- Global Information -----------------------------------------
 > request count                           5 (OK=5      KO=0     )
 > min response time                     176 (OK=176    KO=-     )
 > max response time                    3069 (OK=3069   KO=-     )
 > mean response time                    815 (OK=815    KO=-     )
 > std deviation                        1130 (OK=1130   KO=-     )
 > response time 50th percentile         240 (OK=240    KO=-     )
 > response time 75th percentile         400 (OK=400    KO=-     )
 > response time 95th percentile        2535 (OK=2535   KO=-     )
 > response time 99th percentile        2962 (OK=2962   KO=-     )
 > mean requests/sec                   0.417 (OK=0.417  KO=-     )
 ---- Response Time Distribution ---------------------------------
 > t < 800 ms                              4 ( 80%)
 > 800 ms < t < 1200 ms                    0 (  0%)
 > t > 1200 ms                             1 ( 20%)
 > failed                                  0 (  0%)
 ==================================================================

 Reports generated in 1s.
 Please open the following file: C:\Users\chris\gatling-charts-highcharts-bundle-3.3.0\results\recordedsimulation-20191107133020038\index.html 

This output actually does tell us some information about our test. Because some meaningful comments were inserted into our test script.

.exec(http(“search mac”)

We can see those meaningful names as the script progresses. At the end of the output we can see that there was four calls that were processed in less than 800 milliseconds while a fifth call took longer than 1200 milliseconds. We can also see that there were no failures.

Despite the fact that this output does indeed tell us about how the run went it is actually much easier to look at the html page that we created to see a graphical representation of our script. The last line of the run output gives us the path to the index.html file which has been generated. If we view that html file with our web browser we will see something similar to the following.

Gatling generated report

This image is just part of the web page that is created but it does contain all of the statistics. We can see which of our calls succeeded and how long it took (min, max and average). In this case all of our numbers are the same but if we did run this script multiple times we would be able to see how many were in the 50, 75th and 95th percentiles.

Each one of our steps only perform one task so there is little reason to look much deeper but if each of these performed more than one call or loaded some important resources we could get a better look at the times by examining the details of each one of these steps.

If we look in the upper right hand corner of the picture we can see the name of the class, the runtime and the description that we gave when running it (ie my first run script)

Our script was generated by the recorder as pure Scala code. In my next blog I will clarify a bit of the Scala syntax.

Posted in programming | Tagged | Comments Off on Running a Gatling test script

Gatling

Of the three load and performance testing tools that I am going to describe Gatling is the new comer. While LoadRunner has been around since 1994 Gatling was first released in 2011 and since then has had 2 major releases with the most current version being 3.3.1. Gatling supports fewer protocols but can easily be used for testing HTTP, HTTPS, Web sockets and REST Api’s. In this sense it is more limited than LoadRunner yet as more and more applications are either browser based or implement a service that is available over the internet Gatling is certainly the right tool for the times. Gatling is an open source tool which uses Scala as the scripting language for writing tests. Why Scala?

  • Scala can interact with Java
  • It can use Java libraries
  • Cleaner code
  • Smaller code thus easier to have an overview

The only prerequisite for using Gatling is that you have a copy of the Java JDK installed on your machine. The Scala code is compiled into byte code which is then executed on the Java Virtual Machine of your Java installation. This has the pleasant side effect that you can use the native Java libraries to help in your testing scripts.

Installation

Gatling is distributed as a zip file that you can download from their homepage, https://gatling.io. Installing it is just a matter of finding a good location and unpacking the zip file.

Ie.

C:\users\myuser\testingapps\gatling-charts-highcharts-bundle-3.3.1

There is really nothing left to do if the JDK is installed. This directory where Gatling is installed should be stored in the environment variable GATLING_HOME. This is not necessary as this information can be determined by the start scripts but it is cleaner and this value can then be used in batch files or shell scripts. It is worth noting that a OpenJDK is also a JDK and worked just fine with Gatling for all of my tests.

Recorder

Gatling actually has a solution for people who are not familiar with Scala. Bundled with Gatling is a recorder which can be used for capturing an interaction between the user and a web site.. This recording tool is used as a proxy when surfing your web pages which allows it to capture all the URL’s and necessary headers. The actual output of this tool is a Scala script that will reproduce the same set of steps including all pauses made by the user. This recorder start script is started from the same directory as the gatling start script (%GATLING_HOME%\bin)

This may sound like very good news that web tests can be recorded except it is perhaps a bit simplistic. A long time ago it was pretty much determined from a web security standpoint that you cannot simply record a series of steps and then allow them to replay that exact set of steps without negative consequences. This type of action actually has its own name – a replay attack. A replay attack is somehow capturing a series of steps, which may also include the users credentials. Because of the inherent danger of allowing a strict copy and replay of a series of steps most if not all non-trivial web pages will have hidden variables that are set at each step but are unique each time you perform the same series of steps. The values form these variables must be used in subsequent steps and thus prevent you from creating a test script from a recording and using it without any changes.

These unique variables must be retrieved from the web pages and used for each test. So although Gatling recorder will capture all the hard coded unique values you will need to amend the test scripts. You will need go through the recorded script and make amendments to parse out these dynamic values and to use them at the correct locations while following the test steps.

If you are familiar with the website being tested you will find these recorded scripts to be fairly easy to read. Below is a small test along with the recorded test of these same steps. This example what you would see if you followed the Gatling quick start tutorial (https://gatling.io/docs/current/quickstart/)

This test will perform the following operations.

  • Goto computer database gatling.io web site
  • Search for computer ASRock
  • Search for computer Mac
  • click on computer (internal id #19)
  • save computer id #19 with these details

This script almost exactly what is produced by the recorder. I have trimmed out a few unimportant steps and have added a few important changes (use my local proxy) and changed the comment values to be representative of what action they are performing.

 import scala.concurrent.duration._
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import io.gatling.jdbc.Predef._
 
 class RecordedSimulation extends Simulation {
 
 var httpProtocol = http
 .baseUrl("http://computer-database.gatling.io")
 .inferHtmlResources(BlackList(""".*\.js""", """.*\.css""", """.*\.gif""", """.*\.jpeg""", """.*\.jpg""", """.*\.ico""", """.*\.woff""", """.*\.woff2""", """.*\.(t|o)tf""", """.*\.png""", """.*detectportal\.firefox\.com.*"""), WhiteList())
 .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3")
 .acceptEncodingHeader("gzip, deflate")
 .acceptLanguageHeader("de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7")
 .userAgentHeader("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36")
 
 httpProtocol = httpProtocol.proxy(Proxy("localhost", 31288).httpsPort(31288))
 
 val headers_4 = Map(
 "Proxy-Connection" -> "keep-alive",
 "Upgrade-Insecure-Requests" -> "1")
 
 val headers_7 = Map(
 "Origin" -> "http://computer-database.gatling.io",
 "Proxy-Connection" -> "keep-alive",
 "Upgrade-Insecure-Requests" -> "1")
 
 val scn = scenario("RecordedSimulation")
 .exec(http("search asrock")
 .get("/computers?f=asrock")
 .headers(headers_4)
 .check(status.is(200))
 )
 .pause(3)
 .exec(http("search mac")
 .get("/computers?f=mac")
 .headers(headers_4)
 .check(status.is(200))
 )
 .pause(2)
 .exec(http("fetch computer")
 .get("/computers/19")
 .headers(headers_4)
 .check(status.is(200))
 )
 .pause(2)
 .exec(http("update computer")
 .post("/computers/19")
 .headers(headers_7)
 .formParam("name", "COSMAC VIP")
 .formParam("introduced", "1977-01-02")
 .formParam("discontinued", "")
 .formParam("company", "3")
 .check(status.is(200))
 )
 setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
 } 

What isn’t in the original recording is a status check that the HTTP call succeeded. I have also added a line that will amend the httpProtocol variable to also include my local proxy server. With just a small amount of imagination you can think about passing in a proxy port and host and if the port is not equal to zero then add the proxy to the httpProtocol as part of your test script.

One thing that might not be immediately obvious is that most of the script is not actually running anything but defining the variable scn. This scenario variable is what will be performed by Gatling. The last line of the scala code actually does the setup which will cause this variable to be used by Gatling to call our series of steps.

setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)

In this test script we are going to use only one user and that user should be created all at one time. This sounds pretty silly for a single user, however, if you are speaking about creating 10 or 100 users you can see that immediately quite a load will be created. It is possible that a system which might be able to support 500 concurrent users doesn’t work very well if all 500 start at once within 2 milliseconds. I will cover increasing the number of users and different ways of generating that load later.

My next post will cover how we actually run this script.

Posted in programming | Tagged | Comments Off on Gatling