WebSizr (Web-Sizer) is used to size and benchmark Servers implementing the HyperText Transfer Protocol (HTTP). The WebSizr workload generator module used to simulate multiple Web Browser sessions using a single PC. WebSizr is capable of 'playing' a sequence of commands as defined in the script file that can be manually programmed or automatically "generated" using Technovations WebCorder. WebSizr produces concise and easily understandable results and reports, in addition to providing run-time analysis of test performance data.

WebSizr is intended to Load/Stress Test, Benchmark, Analyze, Tune and Optimize Web Applications and Servers using the HTTP protocol.

Common users of WebSizr can be of 3 types:

• System Architects/Engineers who are charged with design and implementation/deployment of Intranet/ Electronic Commerce applications. WebSizr can be used to conduct performance and stress tests against various configurations. In addition, it can also document results for such tests automatically and thus helps in the analysis to arrive at the most optimal configuration. By providing the ability to document the configuration files, WebSizr provides an ability to tune these system deployments. WebSizr can be used to measure the reliability, availability and security of applications under load.
• Web Application developers who develop/debug web applications where performance and scalability is important. WebSizr can analyze performance of CGI (or API based) server side programs, Java Applets or JavaScript or ActiveX based paradigms.
• System Administrators who need to upgrade to newer versions or point releases of Web Servers and more importantly, to newer versions and patches of Application programs which reside on these HTTP servers.

WebSizr Feature Summary - At a Glance

• Easy Setup - WebSizr configures your experiments by providing the user with point and click interfaces. WebSizr lets you define acceptance criteria, error handling, browser signatures, redirects, proxies, cookies and authentication handling, among other things.
• Modular construction (and licensing) - There are many components that make up a thorough or complete test system. This Sizer system is the most advanced test system in the industry, and constructed with modular components. The users of these systems have very diverse test interests and the advantage that modular construction offers is the ability to selectively deploy and license the components that are relevant. This ability translates into lower cost of ownership, lower learning curves, scalability to meet specific needs, and of course results in high usability of the product. This complete test system (collection of all components) is what we call the WebSizr Suite.
• Reproducibility of Results: WebSizr has been designed top down to the rigorous standards of a benchmarking tool. This results in more accurate correlation and reproducibility of results from different iterations of the same test.
• High efficiency and low cost of ownership: The workload generator module (Sizer) can simulate more than one thousand of users on a single PC, thus minimizing the need for expensive lab resources. The Sizer runs on the Microsoft Windows NT or Microsoft Windows 2000 operating systems and on Intel hardware, which results in lower test infrastructure costs. Even on a low end PC, the Sizer can create about 1500 users (PIII 400MHz 128MB), depending on workload. "Faster" PCs and sparser workloads would allow even higher number of simulated users. Using our LoadDirector technology, you can scale the user creation capabilies to create very a large collection of simultaneous users (We have run the workload generators to create up to 100k Users on a rack of PCs).
• Superior Test Management Capabilities. Ability to control experiments by using command verbs and run-time point-and-click interfaces which save valuable experiment time while providing relevant results.
• Run-Time Interactivity: The interactivity features allow a user to understand the characteristics of the System Under Test during experiment run-time, i.e.. well before test completion. This reduces testing time WebSizr provides the ability to truncate or cancel experiments which may not be relevant without the need to have them finish.
• Statistical Synopsis of experiments: WebSizr provides a statistical synopsis of the experiment by manipulating run-time data. This synopsis gives an in-depth understanding of the characteristics of the System Under Test during a certain experiment.
• Report Writer: WebSizr can document not only the results, but each and every setting and configuration parameter, in addition to CPU, IO, Disk and Controller Utilization, as well as initialization files for the server under test. The report writer also documents timing data for each and every operation that was performed on the system under test, as well as errors encountered during test. These reports are written into a HTML format which can be analyzed later or can be used to recreate tests and reproduce results.
• Multi-platform: WebSizr can run against any HTTP server or application, irrespective of Hardware platform or Operating System.

WebSizr Function Summary

WebSizr is different from most prevalent benchmarking vehicles for HTTP servers, and is meant for advanced users of Web applications. The differences between WebSizr and the competition are obvious. WebSizr is a configurable benchmark and picks up where others leave off to let the architect/developer do true Application benchmarking. Here are some unique abilities of WebSizr..

• Plays commands not only to static content but to CGI or other dynamic content generation programs
• Allows the user to define Test Acceptance Criteria before starting User Simulation.
• Gives logging facilities which allow developers to debug application programs.
• Gives user the ability to define Error Handling which can control the course of the experiment.
• Allows user to automatically desynchronize themselves for more realistic simulations.
• Allows users to define time-outs for user operations.
• Uses cookies, username/password and other credentials based authentication mechanisms
• Does different flavors of POST and GET commands based on need (and application usage).
• Allows user to collect abstract timing info for calls or can scope into socket level timing information
• Allows use of proxy servers to measure latencies outside your organization.
• Allows simulation of different browsers (supports about 25 such popular agent names).
• Logs status codes received from the Web Server for each HTTP operation to help scope problems even while the test is running.
• Allows capability to automatically field redirects (302) and authentication required (401) requests, etc.
• Graphically and Numerically Presents Run-Time performance and status views during test.
• Allows pages (from applications) during test to be logged to HTML files based on user's preference.
• Presents performance results in a easy to understand concise form.
• Allows simulation of multiple scripts in the context of a single test.
• Allows the test user to define custom collections to analyze test activity.
• Has constructs to create normal or bursty tests.
• Has an automatic Report Writer and Configuration Manager which produces a HTML report for each test..
WebSizr also has other features which can be used to mimic the activity of real users more realistically than other testing paradigms.

Platforms

WebSizr  is supported on Windows NT (4.0) and Windows 2000, and is available now.

Script File Parsing and Command Verbs

Please note that we recommend script file generation using Technovations WebCorder - which can latch on to most popular browsers and transcribe user activity, this enabling WebSizr to mimic it to intricate detail.

However, WebSizr supports a lot of scriptable command verbs which can be used in conjunction with scripts that are recorded. WebSizr has a built-in command checker which checks for validity of commands as they are supplied through a Script File. This validation mechanism mainly identifies syntactic errors and semantically malformed lines for some commands. !s and *s at the beginning of a new line are treated as comment characters. The commands are not case sensitive and can be abbreviated to the first 3 characters.

WebSizr natively supports SSL requests. Scripting for tests against SSL servers/applications could never have been as simple. All you need to do is to replace the "http://" string in the script file to the "https://" string !!! Please follow some of the examples shown below.

Here are the supported set of command verbs and the syntax for the commands.
 

• GET: This command targets a certain static page or dynamic program. It takes arguments which specify the number of such operations which need to be repeated back to back and the URL for the operation.

Basic Syntax : GET Number URL
Example HTTP GET requests:
GET 10 http://www.yourserver.com/index.html
GET 10 http://www.yourserver.com/test.cgi
The above command specifies to the WebSizr kernel that the user wants to do 10 GET requests to the specified URL.

Example SSL request - GET 10 https://www.yourserver.com/
 

• AGET: This command is the "Authenticated GET" command. When specified, this command has the same syntax as the GET command. The only difference between the GET command and this one is that the AGET command is executed with Username and Password authentication to the server. Username/Password pairs are supplied through a "Credentials file" during WebSizr setup.


Example authenticated HTTP GET request - AGET 10 http://www.yourserver.com/
Example authenticated SSL GET request - AGET 10 https://www.yourserver.com/

As a side note, the AGET command is automatically executed if the GET command results in a "Authentication Required - 401" request from the server.

The GET and AGET commands are typical occurrences when you do the following on the web .
• Click on highlighted URLs
• You specify a URL in the field supplied by your browser.
• You open up a bookmark
• You supply Username/Password to request access to protected content.

• POST: This command POSTs information to a server.


Syntax:

POST Number PostURL Payload

Examples:
 

POST 1 http://www.yourserver.com:9600/test.cgi product=wszr&platform=win95

In the above example, the POST request appends the payload formation for the test.cgi program to process. If you omit the payload field on a POST, a blank payload will be presented to the server. Note that you can compose URLs which target servers which can be resident on ports other than port 80. In the above example, the server is listening on port 9600.
 

Example SSL POST request - POST 10 https://www.yourserver.com/test.cgi product=wszr&platform=win95
 
• APOST: This command POSTs information to a server.


Syntax:

APOST Number PostURL Payload

Example:

APOST 1 http://www.yourserver.com/test.cgi product=wszr&platform=win95

In this example, you will notice that we have appended the same URL and payload. So what is different ? The only difference is that the APOST command supplied Username/Password authentication credentials while the POST command doesn't. The APOST command is automatically invoked on a Auth Request - 401 from the server if the user were to do a POST request without required credentials.

POST and APOST requests are typically what happen when you push Submit buttons on the web after filling out form info. The Technovations download pages are examples of such forms.

 
Example authenticated SSL POST request - APOST 10 https://www.yourserver.com/test.cgi product=wszr&platform=win95
 
• HEAD: This is one of the least used HTTP commands. It has the same syntax as a GET command. This command is used to request only server info and not content pointed by t he page.


Example: HEAD 1 http://www.yourserver.com/index.html

This returns server info back to the requester. This command does not require authentication and so there is no authenticated version of it.
 

• PAUSE : This command is used to create event lists using the script file. It takes at least one argument which specifies the number of seconds to pause. It could optionally be overloaded with a second argument which is used to create randomness in the PAUSE times for different threads and different instances of the PAUSE command within the same script.


Syntax : PAUSE WaitLowerLimit(seconds) [WaitUpperLimit(seconds)]

Examples :

PAUSE 3 //will make the execution thread pause for 3 seconds

PAUSE 5 10 // will cause a random wait anywhere between 5 and 10 seconds

Tip: The PAUSE command could optionally be used to customize de-synchronization of script activity.
 

• REWIND : This command allows a pattern in the script file to be iterated a number of times.

Syntax : REWIND #Iterations
Example : REWIND 300
This example specifies that all lines in the script file which occur after the REWIND command will be iterated 300 times.
 
• SEARCH : The SEARCH command is used to find a string from the content screen returned after a successful HTTP request. If the string is found, it results in a FOUND event for that specific thread. Else, the NOT_FOUND event is flagged.

Syntax : Search <search string>
Example : Search This is a test
In this example, the screen buffer will be tested for the presence of the string "This is a test". Remember that the search is case sensitive. If the string is present in the buffer, a FOUND event will result for the user. If not, a NOTFOUND event will result. These events can be used to force execution or ignore execution of a certain command or a set of commands. Most WebSizr commands' behavior can be altered based on these events.

Advanced Syntax using IFFOUND/IFNOTFOUND events

GET Number URL <IFFOUND|IFNOTFOUND>

AGET Number URL <IFFOUND|IFNOTFOUND>

POST Number PostURL Payload <IFFOUND|IFNOTFOUND>

APOST Number PostURL Payload <IFFOUND|IFNOTFOUND>

• EXIT : This command is an instruction for the test kernel to terminate activity for the User. This termination is usually instantiated using the FOUND/NOTFOUND events. Note that the EXIT command can also be invoked unconditionally.

Syntax: EXIT <IFFOUND|IFNOTFOUND>
Examples:
• EXIT IFFOUND
• EXIT IFNOTFOUND
• EXIT
Refer to the SEARCH command to understand more about how this command works.
 
• TERMINATE: This command is used to terminate the experiment, usually based on an event. The FOUND/NOTFOUND events are used to invoke this command.

Syntax: TERMINATE <IFFOUND|IFNOTFOUND>
Examples:
• TERMINATE IFFOUND
• TERMINATE IFNOTFOUND
• TERMINATE
Refer to the SEARCH command to understand more about how this command works.

• EXTRACT: This command is used to extract information from a screen buffer and store it in one of the named buffers available for each user. The goal of extraction is the ability to substitute stored values as part of other commands. First, we will describe the working of the EXTRACT command; and then, we will explain the application of the results of the EXTRACT command.


The WebSizr run-time system provides 10 named buffers (BUFFER1 through BUFFER10) for user manipulation. These buffers are used to dynamically store variable data that is extracted from screens which have been retrieved from previous WebSizr requests. So what is the point of having these buffers and how can you use them ? If you have to ask, maybe you don't need them. But here is a simple and practical example of why someone would need this capability.

Assume that a Web application has 3 steps. In the first step, the user goes to a web page and is asked to log in (GET request). The user logs in and a second page is delivered to the user with a certain AccountNumber(AGET request). The third step is a POST request which uses the AccountNumber embedded in the form that gets submitted as part of the POST request. Of course each user will have different AccountNumbers, etc.
In this above example, the simulation is imperfect if it could not simulate user activity as discrete steps. The Extract command is perfect for such a scenario. Let us first look at the syntax of the EXTRACT command.

Syntax: EXTRACT BUFFER# StartingDelimiter EndingDelimiter

The EXTRACT command will extract the string which lies between the Starting delimiter and the Ending Delimiter and puts it in the specified named buffer.

Now going back to the example application which has the above described 3 steps, how does this work ? In our example, we received a web page as a result of our second request which contains the AccountNumber field, which we need to extract. Here is the HTML that we get as a result of the second request.

<BODY>
....HTML....
<TR> <TD>Country:</TD>
<TD><INPUT NAME="AccountNumber" VALUE="2345DFJK"SIZE=45></TD> </TR>
...more HTML....
</BODY>

We will now formulate the following EXTRACT command to extract the AccountNumber information.

Example: EXTRACT BUFFER1 VALUE=" "SIZE
Note that the value that we want to extract is 2345DFJK, and we will be storing this value into BUFFER1.

As a result of this command, BUFFER1 will contain 2345DFJK.

Now that we have the value stored, how do we use it ? Remember that we need to complete step 3 of this application (POST request) by using the AccountNumber information that we just collected.

POST 1 http://www.test.com/step3.cgi name=joe&AccountNumber=#BUFFER1#

Note that this is the equivalent of submitting the request

POST 1 http://www.test.com/step3.cgi name=joe&AccountNumber=2345DFJK

However, by using the extract command, the script file will instruct the WebSizr run-time kernel to dynamically manipulate screen data using delimiter data and named buffers to construct the next URL or components of such a URL (as demonstrated above)

• RUNFOR: The RUNFOR command is used to specify the length of time for a test.

Syntax: RUNFOR <seconds>
Example: RUNFOR 3600
In this example, the test runs for a total period of 3600 seconds or 1 hour.

• RPAUSE : This command is used to do fixed pauses with millisecond granularity. Note the difference between this and the PAUSE command.

Syntax: RPAUSE <milliseconds>
Example: RPAUSE 234
The above example causes the user to pause for 234 milliseconds.

• CUT-COOKIE : This command is used to destroy cookie information during run-time. This feature can be used to simulate expiration of cookies, etc.

Syntax: CUT-COOKIE <IFFOUND|IFNOTFOUND>
Examples:
• CUT-COOKIE IFFOUND
• CUT-COOKIE IFNOTFOUND
• CUT-COOKIE
Refer to the SEARCH command to get more information about generating the IFFOUND and IFNOTFOUND events.
 
• RGET: This command is typically used as a command to handle recorded HTTP transactions. HTTP transactions are recorded using Technovations WebCorder. However, a special case of the RGET command is the ability to play random URLS picked up from a file. Here is how it works:


Syntax: RGET <portNumber> <hostName> [file or random data]

First we populate URLS into the URLFILE.TXT file which is in the WebSizr program directory. For the sake of simplicity, we will put 5 URLs in this file.

/index.html
/test.exe?test=yes&real=no
/todo.cgi?user=joe&account=open
/construct.gif
/press_relaease.htm

Now, here is a RGET command example

Example: RGET 80 test.technovations.com random

In this above example, a random URL from the above list will be pulled and played against the server test.technovations.com which lives on port 80.

• RPOST : This command is generated by Technovations WebCorder and does recorded POST commands.

 
• RAGET : This command is generated by Technovations WebCorder and is used to construct Recorded Authenticated GET commands with Authentication..

 
• RAPOST : This command is generated by Technovations WebCorder and is used to do recorded POST commands with Authentication.

• SOCKET: This is an advanced WebSizr command which can be used to send any TCP/IP command over a socket.


Syntax: SOCKET <portNumber> <hostName> <request>
Example: SOCKET 8080 test.technovations.com <file>sock_request0

In this above example, the TCP/IP request is contained in the file sock_request0. Generally, this is a command that is good not to use unless the user has a in-depth knowledge about TCP/IP. This command would be useful to prototype advanced communication prototypes using the TCP/IP protocol.
 

Using WebSizr - A Tutorial

This section provides a screen-by-screen illustration of using WebSizr. We will be conducting a simple demo test to show some capabilities of WebSizr.
 

Using the SETUP Applet

The SETUP applet allows you to set up and customize the test per your specifications. The SETUP applet has several tab panels that allow such customization and this subsection will explain these in more detail.
 

Control Panel

Let us start with the "Control Panel" view of the SETUP screen. Note that this applet, among other things, allows you to define the duration of your test. If you do not choose a test duration, the test will run to reproduce all the events defined in  the script file(s) used for the test. The duration of the test can be limited by using the "Run Test for" checkbox and the associated numeric value for the number of seconds.

Choosing a script file to use for test

This screen allows the user to specify the script file for this test. Remember that this script file can be either constructed/edited manually or can be generated automatically using Technovations WebCorder.
 

Choosing how many users to simulate during test

The total number of users (Simultaneous Web Browsers) that need to be simulated for the test that we are about to conduct can be specified by either entering this in the "Concurrent Users to Simulate" field or by using the drop down button to pick from preset choices.
 

Choosing how to handle errors

The SETUP screen allows the user to specify actions on errors generated during test.
By default, all errors are ignored (but reported) and tests continue in spite of them. This setting can be used if you can allow a margin for errors to hapen during test.
The test can be configured where test users are dropped as they encounter errors. This setting may be useful to figure out the optimal number of users that can be fielded by the server or application.
The test can be configured to terminate the test when any user encounters errors. This setting is frequently used during reliability and regression testing of applications.
There is also a mode where you could choose to retry URLs whenever errors occur. This setting mimics the way some real applications work.
 

Assigning credentials to simulated users

For each user to execute the assigned script, the user may need to log in to the server. These User names and passwords are input through the credentials file associated with WebSizr. This can be initiated using the "Edit Credentials" button. Picture below shows the format of the credentials file, which is a plain text file that can be edited using your favorite editor.

The first field in the credentials file is the User number. This is the index which helps WebSizr identify and assign usernames and passwords to various user sessions being simulated.

The second field is the name of the user and the third field is the password associated with the username. In this case, we have simplified setup by simulating the same user multiple times. Note that many applications may not allow the same user to be logged in twice. If this is the case, then the test user needs to register unique user credentials and supply them in this file.

User credentials can also be entered/edited/imported using the SIZR.CRD file as opposed to using the above Graphical interface. The SIZR.CRD is located in the WebSizr program directory.

The credentials file is also used to store cookies associated with a certain user. In cases where cookies are encountered by WebSizr. the value of the cookie is written back into this file at the end of the test. If you would like to conduct tests by using pre generated cookies, you can input these values into the credentials file.
 

The Screens and Result Logs Panel

Now back to the details of the SETUP screen. The SETUP screen allows the test user to start a new test. Before we start a new test, let us look at some of the other settings that can be manipulated as part of test SETUP.

The "Screen/Result Logging" screen is used to

• Specify the log file for the test.
• Specify the level of instrumentation needed from WebSizr.
• Specify if data should be read back from the System Under Test and how it should be presented

Result Logs: Specifying the Log file for test.

Time:5818, USER#:5, RT:1636ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:6353, USER#:4, RT:1671ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:7396, USER#:3, RT:1629ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:7863, USER#:2, RT:1618ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:8907, USER#:1, RT:1757ms, OP:GET, URL=http://test:80/wsstat.jpg

The Time: field shows the relative time in Milliseconds from test start time. The User#: field is the identity of the simulated user for the experiment. The RT: field shows the Round-Trip Response Time for this operation, and lastly, the URL: field shows the URL that was hit during the test.
 

Instrumenting additional detail from WebSizr Runs.

Let us see what would happen if we rerun the experiment with Result Logging to get "Detailed(Socket Level Timing)" info.

Time:9950, USER#:5, OP:GET, ToCreateSocket:32ms, ToConnect:5ms, ToSendCmd:0ms, ToReadFirstByte:11ms, ToReadRestOfBytes:1598ms
Time:9950, USER#:5, RT:1666ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:11551, USER#:4, OP:GET, ToCreateSocket:52ms, ToConnect:4ms, ToSendCmd:0ms, ToReadFirstByte:11ms, ToReadRestOfBytes:1606ms
Time:11551, USER#:4, RT:1674ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:12208, USER#:3, OP:GET, ToCreateSocket:28ms, ToConnect:5ms, ToSendCmd:0ms, ToReadFirstByte:11ms, ToReadRestOfBytes:1539ms,
Time:12208, USER#:3, RT:1583ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:13946, USER#:2, OP:GET, ToCreateSocket:5ms, ToConnect:4ms, ToSendCmd:0ms, ToReadFirstByte:11ms, ToReadRestOfBytes:1560ms
Time:13946, USER#:2, RT:1581ms, OP:GET, URL=http://test:80/wsstat.jpg
Time:14599, USER#:1, OP:GET, ToCreateSocket:5ms, ToConnect:4ms, ToSendCmd:0ms, ToReadFirstByte:11ms, ToReadRestOfBytes:1565ms
Time:14599, USER#:1, RT:1585ms, OP:GET, URL=http://test:80/wsstat.jpg

In addition to the abstract log information, the detailed timing information about when certain events happened for these requests is presented.

• ToCreateSocket: is the time required to resolve the Server name and create a socket to communicate with it.
• ToConnect: is the time to connect to the bound port.
• ToSendCommand: is the time to send the HTTP request to the server
• ToReadFirstByte: is the time taken until the first byte of data is read from the socket.
• ToReadRestOfBytes: is the time to read all the remaining bytes on the socket (for this request).

Screen Logging - Specifying recording of data read back from the System Under Test.

Screen logging is the log that keeps track of the data read by the "test browser". You can optionally decide not to log this data to avoid additional IO on the load generator machine, but this could come in handy for debugging applications under load.
If you conducted a certain test where there were failures only with a load of 200 simultaneous users. How would you debug this problem ? You would enable screen logging to understand what happened during test execution and analyze the screen logs to understand the nature of the problem. what would make it more convenient is to have the ability to log screens for each simulated user into a separate file. This would make it easier to read the screen logs. Also, logging into separate files provides the ability to compare screen output between different users.

The Screen page size is the size of the buffer that is used to collect data that is read from the server. This should be larger than the largest page that will be read through this experiment, but also should not be too big because this consume valuable run-time memory resources.
 

The Miscellaneous Settings Panel

The "Miscellaneous Settings" panel allows you to set specific characteristics, most of which define how this experiment is run.

 

Specifying Network Buffer Size

The Network Buffer size is the chunk of data that is read over the socket at a time. If this is of interest to you, you can vary this setting to see performance changes, but most commercial applications typically use a buffer size of 8K (default).
 

Specifying Time-outs for Operations.

The Operation Time-out parameter defines the length of time that a request will wait to be serviced when inactivity occurs. If the request takes longer than this specified time, the request will terminate and an error will result. This error can be caught and thrown if desired or can terminate the user or experiment, as defined in the "Error Handling" panel. Note that if you anticipate the operation to take a long period of time, you would need to set this parameter to be high enough value. Note that you can set the timeout value to "0" and override this function.
 

Customizing the creation of users for test

It may be important in certain test scenarios to replicate the pattern in which users get created during test. WebSizr gives you the ability to ramp up users by allowing you to specify the Minimum frequency and Maximum frequency of creation.

For example, if you wanted users to be created between 5 seconds and 20 seconds, you would specify 5 seconds in the "Minimum Frequency Text box" and 20 Seconds in the "Maximum Frequency Text box". Note that the users will still be created at random.

If you want to create users at a constant rate (rather than being random as in the above example), you can do this by keeping the values in the  "Minimum Frequency Text box" and " "Maximum Frequency Text box" the same.
 

User De-synchronization:

This panel also allows user to be automatically desynchronized so that not all users are generating requests at the same time. This is recommended to mimic real world behavior. A user can also defeat automatic de-synchronization and put de-synchronization commands manually in the script. file. This is done by having PAUSE commands at the top of the script file. As an example a "PAUSE 0 300" command at the top of the script file starts user activity randomly between 0 and 5 minutes.

Automatic de-synchronization can be disabled to enable very quick user RampUp. This can be useful to try quick tests with large number of users without necessarily waiting for WebSizr to RampUp over an extended period of time.
 
 

Specifying redirection handling

This panel also presents a setting which allows users to automatically follow redirects. Redirects are HTTP constructs which allow users who request one page to be redirected to another page. Most popular browsers interpret this action so that the user automatically sees the page to which she is redirected. By setting the "Automatically Follow Redirects" check box, WebSizr will mimic the nature of these popular browsers.
 
 

Ignoring Pause directives in script:

This is a shortcut to quickly run tests without pauses. This could be desirable in many scenarios, especially during debug of scripts or application. In such cases the Sizr is usually in "single user mode" - number of users simulated = 1.
There are other cases where this is used to simulate very large loads by ignoring all the "Pause" events in the script.
 

Defining when to start simulating test activity:

By default, each user after becoming active will start simulating test activity as defined by the basic test script or the test script assigned to that user. This is the natural and the best way of conducting tests because this is exactly the pattern that happens in real life.

However, there are sometimes specific needs to understand the impact of a large number of users hitting a server at the same time. Such tests can be used to determine the health of the server under extremely bursty workloads. WebSizr can enable these tests by proving the test user the ability to "start simulating user activity only after all users are ramped up". In this case, WebSizr will wait for all users to become active and then will hit the server with an active workload.

This "Automatic De-synchronization" feature can also be used creatively in cases where such bursty tests are being designed. Since the bursty tests by definition defeat user de-synchronization, the "Automatic de-synchronization" option can be turned off for such tests to enable users to become active as fast as the workload generator can ramp up the users.

Stats Collection Definitions

The above tab panel allows the test user to customize the collection and marshaling of test data to produce ready to interpret statistics.

By default, the collections are started when the test starts - user hits the "start test" button - and ends when test activity ends.

Collections can also be started "after user RampUp". This method analyzes the data that is collected only after all users have been activated. The time between the first user becoming active to the last user becoming active is the total RampUp time.

In some cases, you may want to enable collections after a specific period of time. You can do this by using the third option and by specifying the time. Note that the time is specified in Milliseconds.

As in the case where the test user can define the start of collections, the time when the collections can be ended can also be customized using this panel. Collections can be started "before user Ramp Down". This method analyzes the data that is collected until the time when the first user exits from the experiment. The time between when the first user completes test activity to the time when the last user completes such activity is defined as Ramp Down time.

In some cases, you may want to enable collections until a specific period of time. You can do this by using the third option and by specifying the time when you want to end collections. Note that the time is specified in Milliseconds.

Irrespective of how you define the collection/marshaling of test statistics, the test harness will still collect the data for all operations and record them in the log. So, the customization of test statistics can be used as a way to use WebSizr to "pre analyze test data" without sacrificing the ability to do post-test analysis.

Once you have completed the required setups in this screen, you can proceed to execute the test by depressing the Start Test button (in the Vital Settings/Controls panel). This will start the experiment and pop the display to monitor run-time events.

Cookies/Proxies Panel

 

Mimicking different browsers.

The Agent Name drop down box allows you to mimic many popular browsers out there. This can be useful if your application is screening for such requests to generate dynamic content. If you are developing a client side application and wanted to mimic its signatures, you can simply do that by entering it in the Agent Name text field.
 

Routing test requests through a proxy server.

Another use of the "/Cookies/Proxies/Browsers" tab is to be able to define a proxy server configuration for tests. The user can either use or not use the Proxy server for the tests. By default, no proxy server will be used for requests.
 

Using Cookies during test and Saving Cookies after test

The user can also define if cookies should be used for the test. If this optioned is checked, the WebSizr run-time system will check for existence of a cookie for this user and this host name. If one is present, the cookie will be presented to the server as part of the HTTP request. The current implementation of WebSizr can support 5 cookies per simulated user.

There are cases when cookies need to be stored at the end of the test. This storage of cookie info allows the user to start the next next test by presenting a cookie upfront (like most popular browsers do) as opposed to generating a new one.

Defining Test Acceptance Criteria

This panel allows the user to define acceptance criteria for tests. The terms and effects are self explanatory. This panel is modeled after most popular RFP (request for proposals) requirements. The test user could translate the requirements from such RFPs into test configuration details using this setup panel.

Once you have completed the required setups in this screen, you can proceed to execute the test by depressing the Start Test button (in the Vital Settings/Controls panel). This will start the experiment and pop the display to monitor run-time events.

 

Advanced Test Settings


 





 

Assigning individual scripts to users.

During some tests, there comes a need to simulate users who do very different things. For example, when you are trying to mimic some users whose jobs are technical in nature and others whose jobs are administrative in nature, etc., it can be quite painful to come up with a single script which models the collective behavior of these users. One solution is to run different test instances for each user type and then merge the result sets. But even this approach can be quite tedious, error prone and requires excessive manual coordination. Technovations has invented a new and simple paradigm so you do not have to endure such pains any more.

WebSizr gives you the ability to simulate such users by eliminating the need for modeling/result set merging. This is enabled by simulating multiple scripts in the same run. There are 3 approaches to accomplishing this:

Assigning by Pattern:

Here is how this is done:

• First, the test user checks the box that indicates that the user intends to "Assign individual scripts to test users".
• Second, defining a pattern (in the "By Pattern Text box") for the files that contain the directives that need to be simulated for the users. For example, if you have a pattern like "mult_" as shown above, this pattern will be used by WebSizr to read individual files. If there is a file called mult_1.scr, this will be assigned to User#1. If there is a file called mult_37.scr, this script file will be assigned for User#37, and so on. So, in a 1000 user simulation scenario, it is possible to simulate the 1000 users each with its own script.
Oftentimes, there is a need to simulate about 2 flavors of users. How can you do this without having to manage a large number of files? Let us illustrate with an example.

Let us assume that we need to simulate 25 users of a certain type and 975 users of a certain different type. In this case, we create 25 files as described above (say mult_1.scr through mult_25.scr with directives for User#1 through User#25 respectively). The basic script file that will be used for the test will consist of directive for the rest of the 1000 users (975).

WebSizr will specifically search for the mult_*.scr pattern while assigning the individual scripts to users. When it cannot find a file that matches the specific pattern, it will assign the basic script pointed to by the "script file" field to that user. For example, if mult_26.scr does not exist (according to our example, it will not be), then the basic script is assigned to User#26.

Assigning by File

It is also possible to assign explicity using a text file. This is accomplished by specifying the bonding between a script and a simulated user. Picture below shows a file which defines such bonding.



Note that if a bond has not explicitly been defined, the test will automatically allocate the "basic test script" (defined in the Control Panel Applet of Setup) for this particular user.

Assigning Scripts Randomly

In some cases, it may be desirable to allocate script users randomly as the user is being created. This can be accomplished by defining a text file that contains a set of scripts that will be picked up randomly by each user on creation. Here is a illustration of such a file:



Note that the probability of choosing a specific script is defined implicitly by the number of occurrences of such a script in the container file.
 
 

Test Stop Modes

There may be several reasons why you may want to stop a test, ranging from "Satisfactory collection of data" to "Test needs more refinements". In any of these cases, there are 2 ways to stop the tests. The "Soft Termination" mode issues a signal to all users to terminate tests after completing the current task at hand. The "Hard Termination" option specifies that the test should be terminated immediately and all activity should be truncated to instantly "obey" the termination directive.
 
 

Playing URLs from a file

In some cases, it may be more appropriate to have the workload generator hit an application with pre-defined URLs played randomly rather than being restricted to a pattern imposed by the script file. In such cases, these URLs are first registered into a user supplied text file which will be pointed to by the user and can be played back either in sequence or in random. Picture below shows the format of the URLFILE.TXT file used in the above example.
 
 







 
 
 
 
 
 

Loading Forms data from a file

As in the case of URLs, in some other cases, it may be desirable to test web applications with a variety of inputs to understand its characteristics, for example, generating stimulus for a search engine. In this case, you can have all the form data resident in a file that can be played back either in sequence or in random. Picture below shows the format of the PAYLOAD.TXT file used to accomplish this.



Note that there may be other types of data that may be required to be imported and played against the server. WebSizr supports a wide variety of such options as well. This information is taught as part of training sessions options that come with WebSizr.
 

These panels are advanced options to pre detect errors on servers and manifest them into appropriate actions on the run-time test kernel. These settings and their respective script file semantics are part of training curriculum options that come with WebSizr.
 
 

Creating very large Collection of Users (Tens of Thousands of Users)






Would you like to create more than 2000 users ?
Would you like to create users from multiple distributed test locations ?
Would you like to manage tests easily (from a setup and go perspective) ?
Would you like result sets merged automatically into one comprehensive report ?

If you answered yes to one or more of these questions, you need the power of Technovations Load Director and Load Module logic to do this. Use of Load Director and Load Modules are part of WebSizr enterprise edition. The use of these are also part of the WebSizr training sessions.
 
 

Monitoring Run-Time activity

The next screen shot shows the WebSizr run-time monitor. On the left-hand side of the screen, you see a graphical picture of the throughput through the experiment (shown in solid blue) and an average of the throughput (in dotted blue). Throughput is expressed in URLs/Second and is graphed over time (Abscissa or axis). At the bottom of the monitor screen is a test state indicator and a counter to show the number of Socket and Connection errors. These are errors that happen over the socket and can happen due to the server's inability to field requests or because of the inability of the workload generator machine to generate enough socket connections. In a Microsoft Windows 95 system, there are restrictions on the number of concurrent socket connections that can be active on the system. This is why we recommend use of a Windows NT workstation or server as the workload generator.

Picture below shows the "Run Time Control Panel" of WebSizr.



The run-time monitor also gives the user controls which can be used to PAUSE TEST or STOP TESTs. If a test is PAUSEd while executing, it sends a signal to all simulated users to stop after completing their current request. Just remember that if you have a lot of simulated users and you have heavy load on the server, the PAUSE may not happen right away.

The STOP TEST command button stops the test in the middle of test execution. Even if you decide to stop tests by using the STOP TEST button, you will still be able to get detailed statistics up until the time that you stopped the test. There are 2 STOP TEST modes.

• Soft Termination : This is a signal which instructs all simulated users to stop after the current command. Again, Just remember that if you have a lot of simulated users and you have heavy load on the server, the STOP may not happen right away.
• Hard Termination : This is a hard stop at the time the user decides to terminate the test. The simulated users are all terminated immediately.
Hard Terminate is enabled by default.

During the test, you can also view the content retrieved as part of the tests if "Screen Logging" was enabled during setup.

The run-time screen has updates the display every 5 seconds. This display interval can be changed for your convenience.
 

Using the Run-Time Monitor Applet

This section will demonstrate the use of the Run-Time Monitor and the facilities that it provides for analysis and control.
 

Vital settings Panel

The "Vital Statistics" view of the run-time monitor shows performance statistics through the test as the test progresses. Latency (Response Time) and Throughput (URLs/Minute) are displayed at the top of the viewing panel.

Test Warm-up Time is the time to setup the experiment. This starts from the time the user hit the "Start Test" button to the time when the first user is generated.
Test RampUp Time is the time to start up all the users for the test. This is measured from the time when the first user starts up to the time when the last user is started up.
Test Time is the time from when the last user is started up to the time when the first user completes or drops out of the test.
Ramp down Time is the exact opposite to RampUp Time. This is the time from when the first user drops off or terminates to the time when the last user drops off or terminates.

In the viewing panel, there is an indicator which shows the total number of Active users Vs. Specified Users. This shows you the progress through test RampUp, but there is a more important use for this. Consider the following example: You want to find out how many users you can put on your system without generating errors, etc. Let us for the moment think that this magic number is 632. Rather than conducting many tests to figure out this optimal number of users for your system, WebSizr allows you to come up with the answer with a single experiment. How ? Just specify the number of user in the SETUP screen to a high number. In our case, let us set it to 800. Set the "Error Handling" in the SETUP screen to "Terminate User on Errors". Start Test. As the test progresses, users who encounter errors are dropped off automatically. 632 users will be running on the system without errors and will be visible through the Run-time screen. And this is how you will know that your system can support 632 users.
 

The HTTP Status Code Display Panel

The second view of the Run-time screen is the View which show the different status codes during test. Please refer to the HTTP specification for meanings of these access codes. For more information on what these status codes mean please refer to the HTTP specification.
 

The Test Progress Panel

The "Test Progress" view of the Run-Time screen shows the Number of requests completed by category. It covers the 3 basic types of HTTP operations, GETs, POSTs and HEADs. In addition, it categorizes the commands which involve authentications separately. The GETs with Authentication and the POSTS with Authentication complete the picture. Notice that the right hand columns show the percent of total operations completed so far.
 

The Response Times Panel

Remember the Response Time Criteria that were input to the program in the SETUP screen. In the "Response Times" view, you can find out how you are doing with respect to those settings. There are 5 Response Time buckets and this view shows you how many of the requests fall into the separate categories.

When the test is complete, we will see that the number of active users will be 0. All the other Test stat values will be filled up in the "Vital Settings" view. Notice that there is a new command button on the landscape which prompts the user to "View Test Results".
 
 
 
 

Reading Test results and statistics - The Test Results Applet

The "Test Results" screen shows different statistical categories for data that was obtained during the test.

The stats show the total number of each type of operation played against the system under test. The "Average Rate" is the Throughput metric which presents the total number of operations completed per unit time on the system under test. "Average Latency" is the metric which reveals Response Time details for different operations conducted against the System Under Test. This is an average number (in milliseconds). Other Numbers presented on this screen are the Standard deviation of response time about the average Response Time, which shows the variability of individual response times from the average. There are 3 discrete points on the Response Time graph that are captured and presented on this screen - The MIN, MAX. and LAST values. These, along with the Standard Deviation can be used to get a quick synopsis of the test before going into more details.

The "ALL" category folds the metrics for the individual operations into an overall test average. It is possible that you may need more detailed statistical information than what is presented in this screen. For this exact reason, a complete audit trail of Response times and timestamps are maintained for each individual operation conducted through the test. This log is in Comma Separated Variable (spreadsheet friendly CSV) format which can be manipulated further if necessary.

From the Statistics screen, you will be given the option of documenting this test or to start a new one. If you choose to Start Another Test, you will be redirected to the Basic Setup screen. If you decide to document results, you will be directed to the screen which helps you define the elements in your test report and generate your test report.
 

Using the Document Results Applet

As you can see, we have setup the test report to write to a HTML file called demoreport.html. You can decide to log your server's configuration files, and any other test files (shell scripts, other programs used for the experiment), etc. if you want to capture a complete snapshot in time of all your test setup.

Note that you also have the flexibility to also log the content generated by the server (during the test). You need to be aware that this can log a lot of data into the test report that may not exactly be relevant to analyze the report.

After you "Commit Results", you can "View Test Results". Here is the HTML file which contains the full report of the experiment that we have been discussing all along. This is the file that you will see in your browser when you click on the "View Test Results" button.

WebSizr provides a mechanism to automatically document all test parameters and values of importance. This is useful for analysis at a later point of time.
 

Summary

WebSizr presents a powerful paradigm to setup, run, view and develop reports for experiments which simulate hundreds of users. It very easily adapts to your test plans to deliver data and results from your experiments which can be used for Capacity Planning, Performance Analysis and Tuning, Regression Testing and Patch Management. Also, you can debug applications against realistic user Workloads and configurations for Reliability and Availability.

If you are developing a mission critical application, you can performance engineer it using WebSizr. Without it, you can only guess how good your application could be under load.
If you are deploying a mission critical application and tuning it to meet ever changing user workload needs, WebSizr can not only make your deployment more robust, but also optimize your deployment cycle by providing you with the right testing technology as you incorporate new patches and versions of Application software.