Thanks to Wendell from Level1Techs for his all of his help on this project and pointing us in the right direction!
A few years ago, we were fortunate enough to get a fiber internet connection installed at the PC Perspective office. Capable of 1Gbps download speeds and about 250Mbps upload, we were excited at the possibilities that laid ahead.
However, when you have access to a very fast internet connection, you begin to notice that the bottleneck has shifted from your connection to the servers on the other side of the content delivery networks (CDNs) that power the internet. While these CDNs have very fast links to the internet, they generally limit bandwidth so that there is more speed to go around to multiple people at the same time.
DL from Steam at 101 MB/s! So fast that the SSD is having trouble keeping up. Able to grab 30+ GB in under 5 mins. pic.twitter.com/ptHUyQVHJr
— Ryan Shrout (@ryanshrout) September 3, 2014
A look back at what once was
One of the services that we found would max out our connection was Steam. Since we download a lot of PC games at the office, it was a nice benefit to have an internet connection as fast as our NICs could handle, and that the Steam CDNs would serve us at our maximum potential. In fact, the bottleneck shifted over to storage performance, as the random writing nature of Steam thrashed our SSDs at the time.
By no stretch of the imagination is 60MB/s slow.. but what happened to our 100MB/s!
Unfortunately, this has ceased to remain the case. At some point, Steam downloads started getting slower on our same internet connection. Not only did storage utilization during a Steam download start to increase, but also CPU usage, pointing to a potential change in how Steam distributed their data. While downloads on our high-end systems fell to around 50-60MB/s, systems with less CPU horsepower started to see speeds fall to 20-30MB/s. All hope was lost for fast game downloads.. or was it?
Recently, Wendell from Level1Techs mentioned on Twitter that they were running a local Steam caching server on their network with great success. After some guidance from Wendell, we decided to tackle this project and see if it would help our specific scenario.