Self-Hosted Bliss
When I originally founded Custom PC Review back in 2012, it was nothing more than a small personal blog running off some run of the mill shared hosting. Since then the site has grown quite a bit, and along with the type of infrastructure hosting the site as well.
The first type of hosting infrastructure for CPCR was shared hosting. While shared hosting was a good, cheap option to get started, the site faced constant crashes as soon as any surge of traffic rolled in. Several months after starting the site I was facing more downtime than uptime, so I decided it was time to upgrade to a VPS. Unlike shared hosting where tons of sites are all hosted on the same server sharing the same resources, a VPS gives you control of a certain specified amount of server resources which are dedicated to you. While this was a decent option during our growth phase, like the shared server, the VPS failed to keep up. With some of our most popular articles drawing 150 or more simultaneous users on the site, the VPS simply couldn’t handle the traffic when it mattered the most. While I could’ve spent hours and hours optimizing the VPS to handle the traffic, I decided it was time to step up the infrastructure.
From there, I then upgraded CPCR to a bare metal dedicated server, which is what we’ve been running on for almost the past two years. While a dedicated server has been significantly more expensive than our VPS, it was not only better able to handle higher traffic loads, it was also significantly faster as well. Our uptime over the past year has been 99.96% with only 3 hours and 50 minutes worth of downtime. For that, I’d like to publicly thank our previous host, WiredTree. They’ve been a fantastic hosting company with top notch uptime and top notch customer service.
While I’ve been very happy with my dedicated box, it’s about that time to take it to the next level. That next level would be self-hosted.
For my purposes, the self hosted server isn’t going to be much different than the previous dedicated server. While I’ll be going the virtualized route so I can run a pfSense firewall, other than that it’ll be a single dedicated server running both the main CPCR website along with the forums as well. In the future, I also plan on bringing online a warm backup server probably in another state far away from earthquake prone California, but that’ll be another topic for another time. To be honest, for me it still makes a lot of sense to go with a managed hosting provider, but having a self-hosted box does allow me the ability to spec the server the way I want rather than the way a hosting provider wants. This also lets me put server hardware to the test in a live, production environment – something that I wasn’t able to do with a managed host.
The Hardware
Lenovo ThinkServer RD540
When it came time to decide which server I wanted to bring online, the biggest concern was physical footprint, power consumption, and price. Physical footprint because most colocations charge by the amount of physical space that you use, price because well… price, and power consumption because most colocations charge by the amount of power used.
Because of these requirements, I decided to pick up several last generation Lenovo ThinkServer RD540’s. With a 1U physical footprint, energy efficient yet powerful Ivy Bridge based server processor, and a street price of under $700 depending on the configuration, it was an easy choice. While the Lenovo ThinkServer RS140 was also considered as an excellent candidate, redundant power supplies, >32GB memory support, and dual CPU support ultimately led me to the ThinkServer RD540. Lenovo is also releasing a new ThinkServer RD550 which is already available, but given the difference in price and our applications, we just didn’t see the necessity of purchasing ThinkServer RD550’s at 3x the cost of the ThinkServer RD540.
Of course, there’s quite a bit more to this beast than meets the eye, so we’ll be doing another article on that down the line. If you’re interested, be sure to keep an eye out for that.
Intel Xeon E5-2609v2
Our previous dedicated server for the past few years was running on a single Intel Xeon E3-1230 processor, which is a 4-core, 8-thread server processor based on Intel’s Sandy Bridge architecture. It was perfectly fine for our webserver and unlike a desktop environment where high CPU performance is king, I’ve come to learn over the years that practically any quad core Sandy Bridge processor offers enough processing power for all but the largest websites. Looking at server metrics, when it comes to serving static content such as webpages on CPCR, lots of memory for caching and fast database access are the most important.
Because there’s little reason to overpay for a processor that would only see minimal utilization in my applications, I decided to equip the ThinkServer RD540 with an Intel Xeon E5-2609V2 which is a 4-core, 4-thread server processor based on Intel’s 22nm Ivy Bridge architecture. While the Intel Xeon E5-2609V2 is inferior in performance compared to the Intel Xeon E3-1230, the ability to add in insanely large amounts of memory and the relatively low cost of ThinkServer RD540’s equipped with the Xeon E5-2609v2 ultimately helped seal the deal.
Kingston DDR3L-1600 ECC RDIMM (KTL-TS316LV/16G)
The ThinkServer RD540 supports up to 320GB of ECC DDR3L RDIMMs, but considering the previous dedicated server was running fine on just 4GB of ECC DDR3 memory, I didn’t really feel need to max out the memory no matter how cool it may be to do so. However, I did want to make sure that the server was as future proofed as possible.
Since we already use Kingston memory on all of our production systems and test benches here at CPCR, Kingston was the first choice as the memory provider for our server. They were generous enough to provide us with 32GB of DDR3L-1600 ECC RDIMMs (2x Kingston KTL-TS316LV/16G). The great thing about these specific Kingston modules is that they’re 1.35v modules which consume less power and generate less heat than comparable 1.5v or 1.65v modules. While it’s a negligible difference in power consumption for our specific purposes, even the relatively small amounts of power consumed by memory tends to add up in larger datacenter environments.
Prior to putting these modules in service, I burned these modules in for a modest 24 hours. Kingston also does burn-in testing for their server memory modules, so I’m extremely confident in its reliability.
Samsung 845DC PRO 400GB
Several years ago when I first switched to a dedicated server, I had already made the decision to go all flash. At the time, the dominant force in the consumer SSD space was the Intel 520 and considering anything else would’ve been unnecessarily expensive on a managed host, I ended up going with a single Intel 520. 24 months later, the server is still running like a champ with zero drive issues, so I’d have to say the drive served its purpose very well.
That said, since we’d been running on a single Intel 520 for the past few years, the fear of a drive failure bringing the site down was a constant concern. Although the server was running nightly, weekly, and monthly backups into a 1TB WD Black HDD and an offsite NAS, a drive failure would still cause substantial downtime.
For our new server, I wanted to minimize the risk of any downtime so I decided to reach out to Samsung for a couple of their Samsung 845DC PRO SSDs. We decided to go with two Samsung 845DC PRO’s in a RAID 1 array. This would minimize the risk of downtime due to a drive failure regardless of how small the chance of failure may be. When we reviewed the Samsung 845DC PRO late last year, we were extremely impressed with its performance so it was the absolute top choice for our new server’s storage. Admittedly, the Samsung 845DC PRO’s MLC 3D-VNAND is a bit overkill for our purposes, but we really wanted to put 3D-VNAND to the test in a production environment.
Down the line I’m looking into throwing in an additional drive as a hot-spare along with a couple more drives for backup purposes.
LSI MegaRAID 9260-8i (ThinkServer RAID 700)
For storage redundancy, our Lenovo ThinkServer RD540 comes equipped with a LSI MegaRAID 9260-8i, which Lenovo refers to as the ThinkServer RAID 700. The ThinkServer RAID 700 is equipped with a full 512MB cache, supports RAID 0/1/5/10/6/50/60, and can optionally be equipped with a battery to protect data in the onboard cache in the event of sudden power loss.
Obviously, this is far and beyond overkill for a simple RAID 1 setup and we would’ve preferred not using dedicated hardware RAID to minimize points of failure, but the ThinkServer RD540’s motherboard only features just a single SATA connector which is designed for an optical drive in models that support an internal optical drive. Given the option of running a single drive and using the ThinkServer RAID 700, we naturally went with the RAID card.
Final Thoughts
Over the next few months, I’ll be closely monitoring and optimizing CPCR’s server performance. It’s been a while since I’ve overhauled the server software behind the web applications and the database, so I’ll use this as an opportunity to do so. I’ll also be monitoring the hardware very closely as well.
The nice thing about self-hosting is that I can get better insights into hardware performance – something I couldn’t do with a managed host. I’m also using hardware that I’m recommending in my own production environment. Since I’m drinking my own Kool-Aid, any component failures will be made public. As a result, I think it really speaks volumes about the confidence our sponsors have in the reliability and the performance of the components they’ve provided.
We’ll be sharing information about our colocation and more in-depth information about our server next, so stay tuned!
Update 02/19/2015: Colocation article live! Click here!
Update 02/20/2015: Lenovo ThinkServer RD540 in-depth overview live! Click here!
