Costing a system for running on Windows Azure seems complex at times and sometimes it is easy to calculate and other times it is not. However there have been a number of times in recent months where I have experienced higher than expected costs for Windows Azure. The first thing to check is the actual bill. The bills are pretty comprehensive and will itemise the usage pretty well. You should easily be able to identify overspend. The following areas are where we have seen unexpected expenses appear.
Compute
The compute prices are straight forward in Windows Azure and you pay for each instance hour you use. The minimum duration is 1 hour and is metered in blocks of hours that means if you have an instance running for 1 minute or 59 minutes you will be charged for 1 hour, similarly if you had 2 instances running at the start of the hour and then turn one off you would still be charged for 2 hours. It may also get complicated if you then put the second instance back up within the same hour. You will probably be charged for 3 hours as the second instance reappearing would be treated as a new deployment and would most likely be on a different machine (although I’ve never checked this) thus being seen as different to the first time the instance was active at the beginning of the hour. This in reality won’t cost you too much as these examples are not likely to happen. What is likely to happen and could cost a significant amount are the following
· Staging & Production
· Instance count
When you upgrade your system using the VIP swap mechanism, you will deploy your new software to the Staging area, do your testing and then VIP swap Production for Staging. During the period where you have something deployed in Staging you will be charged for the number of instances you have running in Production plus the number of instances you have running in Staging. For example I have deployed 2 instances to Production and want to upgrade my software using VIP swap. I then deploy my software to Staging configured to have 2 instances running and carry out the tests. This process takes 30 minutes. I then VIP swap Staging to Production. I will be charged for 4 instances for that hour. If I then do not undeploy the software that is currently in Staging (i.e. the old software that was previously in Production) I will continue to be charged for 4 instances. You may want to leave the old software there so that you can swap back easily if there is an issue, but remember you are being charged twice still.
Another area where you could be getting billed is if you don’t bring the instance count back down after your peak loading has subsided. An example of this is that you increase your instance count on your website just before a big marketing campaign kicks in. Once the campaign is complete it is likely that the load of your website will drop. At this point it would make sense to reduce the instance count of your website otherwise you are paying more that you need to for your site.
Data
With Windows Azure storage you get charged for the space needed to store the data, the cost to transfer the data out of the data centre and the cost of transactions on the data storage API. Just to clarify a storage transaction is effectively a call to the Storage API, whether that is to retrieve data, query queues, delete data etc. Each is at least one transaction (Some data retrievals might be multiple SDK calls). Data storage costs are straightforward, but bandwidth and transaction costs could take you by surprise if you are not careful.
Bandwidth charges cover the cost to extract data from the data centre, that could be the presentation of a web page to a user, extraction of a blob extracting data from a queue outside of Windows Azure etc. unless you are regularly moving data outside of Windows azure the bandwidth costs are relatively small (when compared to the compute costs) e.g. $0.12 per GB. Another thing to bear in mind is that data transfer within the data centre is free and also data transferred into the data centre is also currently free. This means you can put a lot of data into a data centre, process it and then store it without incurring any bandwidth charges. You will only be charged for bandwidth when you pull the data out of the data centre. It is therefore important to make sure that if you are transferring data around your system that all your compute and data storage (Storage and SQL Azure) are all in the same data centre. If you don’t then you will be charged for bandwidth to get the data from one data centre to another. For example, in the diagram below we put our web site in Dublin and SQL Azure in Amsterdam.
In the simple scenario of putting some data into our database from a website and then doing a single retrieve you can see that the data will incur 3 bandwidth charges when it should only have 1. With a lot of data transfers the costs will soon mount up.
Transactions charges also appear to be relatively low when compared to compute costs and most of the time they are, but as you are charged each time a method on the Storage API is called you could be getting charged even when there is no data being processed. This would happen when you are processing a queue. Each time you call the SDK to see if there is something to process you would be registering a transaction, therefore if you had this code in a loop that regularly checked the queue you could start to rack up hidden costs. For example a queue is polled once per second this would cost around $300 per year, however if you then reduced the polling time to check it every 100ms then the cost would be around $3000 per year, now multiply that by the number of instance you have running and the costs significantly rise. Do you really need to poll every 100ms, can you redesign to an event driven system or poll less often. Polling the queue every minute for example reduces the costs from $3000 to $5 per year
The charging concept needs to be understood by your developers so that they you don’t end up with surprise bills. You also need to be careful not to engineer a solution that is overly complex just to save transaction costs when the costs could be negligible in the scheme of things. Sensible calculations up front can help you to balance the costs effectively but watch out for the charges mentioned above as these are often done by mistake.