Third in a series on how to effectively use Nouvola DiveCloud
Earlier this year, we talked about the best way to calculate concurrent users when using Nouvola DiveCloud, and touched on the importance of performance tests to let you know how much traffic your software can handle with the right speed. We also defined Virtual users, sessions and concurrent virtual users.
Now, let’s take a look at all the timing options in Nouvola DiveCloud.
Test Plan Duration
The test plan duration is the duration of the test execution. That is the time during which the virtual users are executing sessions and generating load against the application under test. If the test duration is set for 10 minutes, some additional time will be required to that at the beginning and the end for test queuing and results processing. The total duration of the test from beginning to end will be longer than the test duration specified in the test plan.
The test duration in DiveCloud is divided into 10 intervals of equal length. If the test duration is 10 minutes, each interval will be one minute long. The concept of interval is then used when visualizing results and for the linear ramp traffic pattern.
Within a single interval, the number of users does not change. When the interval starts, the users will execute a session concurrently (within the same second) until it is completed. Then they all wait until all users are ready to start a new cycle. These cycles are called waves. There are typically multiple waves in an interval.
Let’s see how this all works. Let’s assume the test duration is set for 10 minutes (test duration). In that case, each interval will be one minute long. Let’s assume we want to test a user session made of 5 requests and each of them takes 600 milliseconds to complete. In the case, the total length of the wave will be 3000 milliseconds, or three seconds, and in one minute there will be 20 waves, in average.
DiveCloud also offers a feature to define a think time, which essentially inserts a break between requests to better mimic the way actual users would use the application under test. The think time can be fixed at 300, 600 or 1,000 milliseconds, or it can be assigned a random value between 1 and 1,000 milliseconds. After each request, the user will wait the assigned think time before executing the next request.
Some requests may take a very long time to complete, and may even go beyond the length of the interval. And even if a single request is long, this will delay the start of the next wave, and reduce the number of requests sent to the system.
To avoid this disruption, Nouvola DiveCloud lets you set a timeout, so that if a request takes too long it is stopped and marked as timed out. This error will be logged, the specific request terminated and the session execution will continue with the next step. The test results will contain specific information about the timeout, like which request caused it.
We recommend that users set a reasonable timeout that would only go into effect with a real outlier, and not a request of typical length.