In the previous blog post, I showed how easy it is to initiate a load test from Azure by just providing a URL of the web page you want to load test, specify the number of users and the duration. This is great for websites where requests are GET requests or you are not trying to test a user scenario that consists of multiple steps. In this post, I will show how to use Azure Cloud Load Testing in a more advanced scenario that consists of multiple steps.
In order to record a performance test which will be used for load testing, you need to have Visual Studio Enterprise or Ultimate Edition. You can download a 30 day trial version
Recording a Scenario
- Open Visual Studio
- Create a new Web Performance and Load Test project
- Click the record button on your web test
- Go through the scenario you want to record. I uploaded the scenario I recorded at https://youtu.be/6-ZiIL7yPpA
- I recorded a test to go from the home page to the search page and then perform a search
You can test your script by clicking the run test button
Create a Load Test
A load test can consist of one or more test scenario. You can also define the duration of your test and the number of users to simulate running the test. Let’s create and configure a load test which will run in Azure.
- Right click on your project
- Add a load test by right clicking on the project and add a new itm
- On the wizard, I am selecting a Cloud-based Load Test and click next
Select the location where the load will be generated from and click next
On the Run Settings page, you select the duration of the run and then click next
On the scenario page, name the scenario and select the Think time profile, think time is the duration between your test steps
On the Load pattern page, you can select a constant load of users or if you want to start with a smaller number of users and ramp up the user users while your test is running. I am using a constant load but you may want to use the second option if that what you want to simulate.
On the test mix page, you can select one or more tests. in my case I am only selecting one web test. You can also selected MSTest unit tests as well.
Add the browsers you want your users to use during the test
Click the Run load test button
- Click on the Performance or Throughput buttons to see the graphs. The application button with capture the Application Insights telemetry
In this post, I showed how to create a performance and load test using Visual Studio 2015 Enterprise Edition and leverage Azure Cloud Load Testing to execute the recorded test. I didn’t have to create any Virtual Machines or install test agents to run my load test. I had to push a button and monitor the test run. Cloud load testing would save you a lot of money in infrastructure purchases and labour costs when you want to run load tests to similar millions of users and you want to only run the test scenarios for few weeks a year. The first 20,000 Virtual User Minutes (VUMs) a month are free. Give it a try today.
Load testing is a very import milestone in your development lifecycle that is usually ignored. Load testing allows you to discover how your system behaves while x number of users are using your application. It may uncover performance issues and hence you can optimize the code or beef up your hardware.
On Premises Load Testing
It is fairly easy to build a test lab to host your Visual Studio Test Controller and Visual Studio Test Agents. Microsoft MSDN documentation recommends one dual core CPU and 2GB for each Test Agent to support 1000 virtual users. In other words, you will need 20 core with at least 20GB of ram to create 10 VM’s to simulate 10,000 users or 200 cores and 200GB of ram to create VM’s to simulate 100,000 users and so on. As you can see it may get so expensive to purchase all that hardware, hire people to build and maintain your servers and add to that the hydro bill.
Azure Cloud Load Testing (CLT)
Microsoft Azure allows you with clicking of few buttons to leverage cloud agents to perform your load test. Current, you can only use 25,000 VUsers (100 cores) for up to an hour per test. If you want to increase those limits you can either run up to 10 test in parallel to use up to 1000 cores or 250,000 users or contact Microsoft at email@example.com. As of today, Each Visual Studio Online account gets a free 25,000 Virtual User Minutes (VUMin)
Getting Started with Cloud Load Testing
To create a load test from the Azure website you need
- Azure account
- VSO account that is linked to your Azure account. (as of today, your VSO account needs to be created in Azure)
- Azure Web App (Your websites can be hosted out of Azure)
Microsoft made it really easy to load test a single URL. The following is a step by step instructions on how get started with Azure Cloud Load Testing:
- Go to https://portal.azure.com/
- Click on Web Apps
- If you are hosting your site in Azure, select it, otherwise create a new one
- Click on Tools
- On the Tools panel, clicks Performance Test
- Click on Set Account
- Click on Account to select an existing VSO account. If “Existing accounts” list is empty, follow the steps in the Create New Account section
- Click OK to go back to the Performance tests panel
- Click on New and fill the test info. The generate load from field allows you to select the geolocation where the test will be generated from.
- If you are using a free Service Plan tier, you are limited to 40 user load for 1 Minute Duration. To change your Service Plan tier, follow the steps in the “Change Service Plan Tier” section
- Click on Run test
- You can run multiple tests from the same or different geographical location in parallel
Azure Cloud Load Testing allows you to hit the road and start load testing your application without waiting for an infrastructure to be built. In a future post, I will show how to use Visual Studio to build a more complex load test scenario while using Cloud Load Testing.
Create New VSO Account from Azure
- While you are on the Account Settings panel, click “or create new account” hyper link
- Type the name of the account and select the subscription
- Click OK
Change Service Plan Tier
Click on Service Plan in the main Azure menu
- Select the Service Plan you need to upgrade
- Click on the Pricing tier tile
- Select the pricing tier you want to upgrade your service plan to
As you may already know, some MSDN Subscriptions come with Windows Azure Credit that can be used for Dev and Test usage. As of October 24, 2013, you can get upto $150 to spend on your Windows Azure account. For more detail visit
In this post we will provision a new VM on Windows Azure with Visual Studio 2013 Ultimate. All steps and screen shots were done and taken on October 24, 2013.
Activate Your MSDN Windows Azure
Login to MSDN
Click on MSDN Subscriptions
Click on the Access benefits button
Click the Activate Windows Azure link
Windows Azure Portal
The previous steps should activate your Windows Azure account and take you the Windows Azure Portal. If you have already activated it before, you can go directly to www.WindowsAzure.com and sign in with your account
Select your subscription
Click on the Portal button
Click Virtual Machines link on the left and then click new on the bottom
From the Menu, click on the From Gallery menu item
I am selecting Visual Studio 2013 Ultimate on WS 2012 but feel free to select another VM image and click on the arrow at bottom right corner
Give the VM a name
Select the VM Size
Create a username and password (we will use that username to login using remote desktop)
Select a Cloud Service if you have one or choose Create a new cloud service
Give it a name
Select a location
I am leaving all fields as default value
Click the check mark at the bottom right
The VM now is being created, it took around 5 minutes to complete the provisioning of the VM
When provisioning is completed, you will get notified at the bottom of the screen
Click on the connect button
Double click the downloaded rdp file or open it from your browser
Enter your username and password and you