So many options, which one is best?

One of my current projects requires me to use an EC2 for some beefier computation, but I actually don’t know too much about EC2 apart from using the most basic t2 instances. Nothing I’ve worked on has really needed a server with 60gb of RAM. Until now that is.

How much memory do you need?

One of the first options that will limit you is memory for your usage. There are so many different instance types depending on how fast you want your CPU(s), but largely the difference in price is from storage. You can sacrifice a couple ghz in speed for a couple GB in RAM, and vice versa.

Working in R, my dataset has 44,000,000 rows and 17 columns. Approximating all the data to be 10 bytes, this gives us:

    bytes
    >>> 44000000*17*10
    7480000000

Which is rounded up to be 7.5GB of data. This sets a baseline of how much RAM you will need at a minimum for your instance type. You will also need to account for OS overhead, which will be something like ~.5GB

What does your program do?

In my case, I’m building a tree based model, which I know will be memory intensive. Apart from doing that, even setting up the data will require more memory overhead for manipulation of data. This is why when dealing with large datasets, using an appropriate memory-efficient data structure is very important. For R, this means using data.table, which will let you perform in place calculations without having to duplicate your data. Duplicating data will come at a very high memory cost when you have large amounts of data.

This is where memory estimation gets a bit tricky. Depending on the algorithm used for your model, this will determine how much memory is required. Essentially, this is the space complexity of an algorithm, remember all that stuff you learned from algorithms class but forgot? Yeah that stuff.

Each algorithm will be different, for my case I’m looking at gradient boosted regression trees (xgboost). The algorithm will have built in measures to optimise for memory, such as using histograms to optimise the growing of the trees based on differences between parent and sibling.

With that said, knowing the baseline requirement, I would suggest running the script for a baseline version of N trees and M depth, and then scale it appropriate and approximate up. My parameters for the model are:

    ntrees = 500
    max_depth = 10

With this, I know that 7.5GB baseline will probably require more than 30GB RAM just from thinking about the number of trees built. Of course this has a lot to do with how the algorithm will work, but it’s always better to have too much RAM than too little.

CPU requirements

Something that is also important is how fast you need your CPU to be. Obviously there will be a point in which paying for an hour of a slower CPU will be the better option if you need to satisfy a memory requirement.

For me, I went with AWS R4 instance type which is a memory optimised instance that offers the best value in terms of decent CPU speed and large amounts of RAM. The r4.2xlarge has 61GB of RAM and 8 vCPUs for $0.4 an hour.

That said, I’m still very new to all the offerings of AWS and the different compute and memory types. I’ve tried using c4 instance types in the past and didn’t feel like they were the right choice for my application because of how little RAM they have.