I've been talking a lot recently with colleagues (and anyone foolish enough to ask) about the benefits of Elixir, based on the performance of the underlying Erlang Virtual Machine (BEAM).

One of the things I'm most impressed with is the ability to utilize as many processor as you can throw at it. The other thing that makes it compelling is the extremely small memory requirements.

I decided to see how much performance I could squeeze out of a Raspberry Pi 2.

Why a Pi?

A Raspberry Pi 2 in Australia will cost around $48, and has a 4 core ARM processor and 1GB of RAM. As a hobby device it's pretty well priced.

I'm currently in the process of building a house and because of the location, I will be getting access to Fiber-To-The-Premises broadband. Like any good nerd, I'm beside myself with excitement about that extra bandwidth (though it's really latency that kills web browsing experience).

I currently host a bunch of pet project on rented servers 'in the cloud'. I want to know if a Raspberry Pi 2 connected to my NBN connection would be able to replace some / all of these.

Setting up

Installing Elixir on a Raspberry Pi requires a bit of perseverance and time.

I'm not going to go into too much detail, but at this point in time, it requires compiling Erlang from source (if you want the latest version), this took about an hour.

The Phoenix application I have installed on the Pi is a side project. The details aren't that important, but I'll be testing two different parts of the application.

Firstly, I'll look at performance service a single 'index.html' endpoint. This receives a request and returns a static file, no additional work is required.

The second will receive a request for a collection of records. These will be retrieved from a Postgres database and the converted into JSON using JA_Serializer. JA_Serializer is a library for outputting JSON API compatible responses - it's a great library, but slightly slower than straight Poison.encode! as there's a bit more work to do.

I'll be using a tool called wrk for benchmarking. It's a Lua based project which allows you to hit a specified endpoint, measuring things like throughput and latency.

I'll also use the Erlang observer and htop.

THE BIG CAVEAT

These are benchmarks, run under nearly ideal conditions. They aren't going to provide a definitive answer to 'how much can I pump out of a Raspberry Pi?'. The results will give some indication of what's possible though.

A smaller one

The Raspberry Pi is connected to my router with a wired connection, the laptop driving the tests is connected via WiFi, so there's likely some impact on that as a result of latency and WiFi performance. I don't have any other data to compare, but it's worth keeping in mind.

Tests

1 thread, 1 connection, static endpoint

➜  mfeckie.github.io git:(master) wrk -d 30 -c 1 -t 1 http://192.168.1.12:4001/
Running 30s test @ http://192.168.1.12:4001/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.59ms   47.72ms 600.50ms   98.23%
    Req/Sec   268.20     22.84   292.00     90.14%
  7884 requests in 30.02s, 34.62MB read
Requests/sec:    262.61
Transfer/sec:      1.15MB

Looking at htop, we can see that we didn't even max out one of the cores

This is pretty decent performance, but using only 1 connection isn't really going to stress out the BEAM.

1 thread, 2 connections, static endpoint

➜  mfeckie.github.io git:(master) wrk -d 30 -c 2 -t 1 http://192.168.1.12:4001/
Running 30s test @ http://192.168.1.12:4001/
  1 threads and 2 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.99ms  773.42us  33.51ms   92.87%
    Req/Sec   401.21     16.94   460.00     75.67%
  12001 requests in 30.05s, 52.70MB read
Requests/sec:    399.31
Transfer/sec:      1.75MB

Looking at htop, we can see we've maxed out one core, but it looks like there's a lot more capacity.

8 threads, 8 connections, static endpoint

➜  mfeckie.github.io git:(master) ✗ wrk -d 30 -c 8 -t 8 http://192.168.1.12:4001/
Running 30s test @ http://192.168.1.12:4001/
  8 threads and 8 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.96ms    2.27ms  48.09ms   79.99%
    Req/Sec   126.02     13.26   170.00     57.08%
  30217 requests in 30.10s, 132.68MB read
Requests/sec:   1003.90
Transfer/sec:      4.41MB

Now we're talking, using much more of the available processing power.

16 threads, 16 connections, static endpoint

➜  /  wrk -d 30 -c 16 -t 16 http://192.168.1.12:4001
Running 30s test @ http://192.168.1.12:4001
  16 threads and 16 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    22.68ms   72.52ms 809.18ms   97.75%
    Req/Sec    80.82     16.45   121.00     68.26%
  37831 requests in 30.05s, 166.12MB read
Requests/sec:   1259.11
Transfer/sec:      5.53MB

Now this time we're really close to maxing out the cores, but when I cranked it up to 32 connections and threads I notice no real improvement. Then it dawned on me that 6MB/sec is about the limit of my WiFi router.

At this point I tried to run wrk on the Pi, but ran into some compatibility errors, so I gave up. Then I remembered I had a Thunderbolt ethernet adapter, so I gave that a try. Turns out it wasn't the network limiting it, this was basically the best performance I could squeeze out of it.

8 threads, 8 connections with database and JSON rendering

➜  mfeckie.github.io git:(master) ✗ wrk -d 30 -c 8 -t 8 http://192.168.1.12:4001/api/standards
Running 30s test @ http://192.168.1.12:4001/api/standards
  8 threads and 8 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    37.43ms   10.05ms 252.77ms   87.10%
    Req/Sec    26.83      5.35    40.00     66.72%
  6469 requests in 30.10s, 27.96MB read
Requests/sec:    214.93
Transfer/sec:      0.93MB

Once we throw the database and JSON into the mix, we're asking it to do a LOT more work.

Enter the Erlang observer

I've posted previously about connecting the Erlang observer over SSH, so I'll use the techniques I outlined there to connect.

Once connected I ran a few of the tests again and made this video.

Here you'll get a feel for what's going on 'under the hood' when we try and stress the VM.

It's really nice to see how little memory is used by the application, peaking at around 80MB.

Without wish to be another language troll, this kind of performance with low resource requirements is unimaginable with many other web frameworks.