As I blogged about a few days ago I’ve fallen out with my web host. This tends to happen every few years; the standard of “cheap but reliable” hosting has taken a dive. Especially in the area of customer service (I don’t ask much, just that you read my support requests!).
This is a pretty lengthy post, so bear with it!
And, so, I decided to switch out to another provider. Which is where it got interesting. For a few years I have run a dedicated machine next to my shared hosting – for testing, game servers and, occasionally, web hosting. It was the latter that got me thinking. I use the dedicated box to host sites with, say, background processes or limitations that mean a shared host is no use.
Why couldn’t I switch to hosting all of my own sites?
So I’ve given it a go.
The Dispute
For those who like that kind of thing (so, most of you) this is how my old host (Servage) pissed me off: bad customer service.
A few weeks back a programming mistake (and, I argue, a configuration mistake on their part) locked up the shared server – so they suspended my account. Which consisted of returning a Servage suspension page and a 404 header. Oh, and they never told me (apart from a cryptic copy of a “top” output) what the issue was.
They then proceeded to not read my support requests and act like I was a criminal 
so screw ‘em.
The Solution
Hosting your own sites is actually pretty easy; VPS’s are easy and cheap to purchase nowadays and the tools are few clicks a way. Here is my ingedient list:
- VPS’s (I use Gandi; so far they have been excellent)
- DNS (DynDNS provide for my main site, more on that below, but other sites are self hosted using PowerDNS)
- A Web Server (no contest: Nginx)
Once you have the pieces it is fairly simple to slot together. I recommend you have two (or more) smaller servers rather than one massive server. This means you can keep production/development separate and have fail over DNS. So far I have successfully put together hosting and DNS provision for about 50% of my active sites and plan to pull the others over before Servage asks for renewal.
The most important thing is to PLAN your approach to this; I made several mistakes with the first few sites transferred leading to annoying down time.
Hosting
Hosting is the easiest part of the puzzle (and will be specific to your language choices/needs).
Most of my code is in PHP so I was after an LNMP (Linux, Nginx, MySQL, PHP) stack – guides to which are only a Google away. There are a couple of ways you can pipe Nginx into PHP but I chose the Fast CGI route in the end which is proving stable (go with what you prefer). After a little deliberation I decided to install pretty much all of
One good idea I had was to set up a sub-domain for every VPS; that way you can host Mysql and other services under that name (i.e. no need to make them available under your hosted domains).
Iit is worth setting up some form of caching with PHP (I recommend Xcache). While most of my PHP code ran fine WordPress slowed to an absolute crawl. Xcache brought what can only be described as “astonishing” speed improvements.
Oh and if you do use Gandi you have to install sendmail manually to get mail() working (I was stumped by this for ages thinking it was a problem with PHP)
DNS
DNS was the most difficult part of the setup. In my previous setup most of my domains were registered with Fast Hosts (they still are as it happens) and Servage provided my DNS.
One major problem with self-hosted DNS is that fact that you need to register “glue” subdomains with the central repositories (this is so that DNS lookups don’t go into a recursive loop of doom). After an initial play, and fail, around with Bind 9 I cast about for hosted DNS. In the end I went with DynDNS for my personal domain – it was pretty cheap ($29.99 for a year) and seems a stable service. Doing this made me a lot happier about attempting to self-host DNS for the other sites.
Bind is a pig; there I said it. Luckily I came across PowerDNS, which for me, had several advantages:
- Installation was a breeze, just a few seconds
- It is MySQL based (which fits in with some future plans)
- There is Power Admin, a drop in PHP configuration manager
Power Admin really is the killer feature; it is by far the easiest piece of DNS setup I have used. Because of the MySQL based configuration you can provide two DNS servers (which is recommended) by installing Power DNS on another VPS and setting up MySQL replication (I cannot stress how beautifully easy this was to do).
Once happy I added my DNS zones, asked Fast Hosts to register ns1/ns2.errant.me.uk as Glue records and changed all the DNS records. Phew.
Side note: I wish I had explored Power DNS before buying DynDNS! It could easily have convinced me to risk the full self-host.
Cost
I suppose cost is important. For me costs went down (as I will explain) but for those with only shared hosting you may end up paying more. It’s up to you but I much prefer the versatility!
My previous setup was:
- Shared Hosting with Servage: £75/year
- Dedicated Server with Fast Hosts: £960/year
So around £1000/year.
Currently I have two shares with Gandi priced at £240/year. Which is a pretty substantial saving overall – though a big leap up from shared hosting costs.
On the other hand there are a lot of advantages that may make it worth it; not least that you have full access to the server to do whatever the hell you like (site deployment using Mercurial has become a dream!)
Next?
- Domains: My next plan is to administer my own domains directly. At the moment they are spread about amongst Fast Hosts, Gandi and Servage and it would be handy to consolidate those into one place. From what I can gather this is looking to be rather expensive, so perhaps for a later date. It’s something I would like, though, just to be entirely self sufficient.
- DNS: soon I plan to ditch DynDNS and bring all my DNS service “in-house”. That’s a pretty major leap so I plan to be very ready before trying
- Control Panel; hand configuration of Nginx is pretty trivial if you don’t have many domains (a lot of the cfgs can be reused). Plus I have Power Admin and PHPMyAdmin for DNS and MySQL maintenance. But an “all in one” method would be quite nice (particularly one that could handle domains/DNS together). None of the current control panels support Nginx very well – so it is easier to hand configure.
- Mongrel 2; an interesting project which is configured using SQLite. As soon as it becomes stable I will be testing it to sit in front of Nginx (or even directly in front of PHP) – because of the SQLite approach it should be easy to hack together a Power DNS/Mongrel based control panel.
Final Thoughts
Self hosting has been on my mind for a while, but I have always shied away from the difficulty. So I am actually pleased Servage gave me a push.
One major concern was that this makes me responsible for server maintenance – I’m not a bad sysadmin but you could hardly call me great. The positive thing is that this is proving to be less of a worry that I thought. The only serious stumbling block so far has been the caching/slowness issue with PHP; but I managed to figure that out in just a few hours.
Best of all I am learning a lot of useful stuff in the process.
Security is another major problem. I think everything is safe but I do need to remember to update software regularly (this is becoming an advantage because I can update much faster than most hosts!).
If you want to do this then I highly recommend it! But I also recommend you take it slow and do your research. I rushed the initial process (due to being mad at Servage) and made some mistakes.
By the way, yes, the blog you are reading is self hosting. I hope it coped!
Hosting dns yourself is dumb. There are millions of free dns-hosting companies, also most registrars offer free dns pointing. Whats the idea, their uptime will most likely always be better than yours.
I’ve been using a VPS (Slicehost) for my own site for over a year now and it does involve a LOT of work up-front. Once I got everything running though, it’s just chugged along on its own.
As far as security and notifications, I’ve got iptables set up with as few ports open as possible, logcheck scanning all my system and application logs for apparent errors, and fail2ban monitoring for any break-in attempts. I’ve also set up a (free!) account with Cloudkick that regularly monitors the uptime of my server and emails me when it goes down.
Also note that most VPS’ offer some kind of backup solution (Slicehost for example offers scheduled snapshots for a nominal fee, so I have a daily and weekly backup automatically for I think $10/mo extra).
The most important part of deciding to self-host is to pick a reputable and reliable VPS provider.
Good post with only one comment. I hosted my own public DNS for many years and eventually decided to move it over to GoDaddy. One tiny wrong turn with your own DNS easily causes all of the upstream servers to barf on your domain and render your site inaccessible for weeks at a time. And I mean one TINY error, such as missing a single period indicating the root. DNS is also hackable, with pollution being a common enemy, so it’s an additional security concern for you to constantly worry about.
For the cost of having your glue record pointing to you, you can host several domains with GoDaddy and their tools are very easy to work with. It’s very fun and geeky to host your own public DNS but that’s the one component that I decided to outsource.
@dfg; it may be slightly dumb. But it seems pretty per formant for the moment. I have failover and Gandi’s uptime are very very good. I can’t see other people’s uptime being any better.
The only advantages of hosted DNS are faster connections (which doesn’t worry me too much – everything is performant at the moment) and Geographically distributed DNS. If the latter becomes a big advantage it is not too expensive to put it together myself (in fact I would call it relatively trivial).
@Evan: I’m still fine tuning my setup but am now able to transfer about 3 sites a week (as opposed to 1 site a fortnight a month ago
). It’s been really useful experience.
“The most important part of deciding to self-host is to pick a reputable and reliable VPS provider.”
I forgot to mention in the post but I am working on ways to fail-safe this. I just bought a few Slicehost VPS (for the US) and would up some Amazon E2 instances.
Eventually my aim is to use S3 for static content (with an alternative backup, perhaps Google) and then use Round Robin/GeoIP DNS to serve sites from multiple locations.
But I want to put together the sync infastructure at the back end first.
(this is, of course, complete overkill but a) I want to be completely host agnostic and b) it’s become an interesting exercise
)
Chris; thanks for the pointers!
You are right; and I may well leave my main domain with DynDNS as a failsafe (I wouldn’t go with Go Daddy for personal preference
).
One advantage is that Power Admin is as good as any hosting provider DNS control panel – so I am only lightly worried about DNS mistakes.
As others have pointed out there are lots of good reasons not to host your own DNS. For example you have to monitor DNS and with PowerDNS in particular you need to make sure that everything is getting replicated to all servers.
We’ve built DeadSimpleDNS.com not only to take care of DNS for you but we’re also adding handy features like global search and replace, one click gmail hosted MX records and a lot more.
With that being said I’ve hosted my own DNS for years at various companies and PowerDNS is a breath of fresh air compared to BIND.
Sam, you are a saviour – I spent about an hour trying to remember who you guys were earlier today (after a HN link last week I think).
It;s a very tempting service – the only reason I didn’t blog it is because I couldn’t find you
Intersting article – I’ve always wanted to do the self hosting and at one point I went down the VPS root and thoughly got on with it until the host almost doubled it’s price.
For me, with only the one website (well arguably 2 – my website and blog), the price of a VPS doesn’t make it worthwhile when you consider Just Host supply me with all I need for ~£50 for two years!
I guess with the VPS you’re paying for the convince of running what you want. I briefly considered running one as a proxy/SSH server to encrypt all my traffic from my house (if I’ve been accepted for a sub warden at university) but decided against it when I didn’t get the place.
Out of interest, is there a benefit of nginix over lighttpd (my personal favourite server)?
Hi Errant,
Sorry that we’ve been so tough to find. We’re working on that.
Give me a shout if you have any ideas on how we can make it better. DNS has been boring for too long and we’re looking to shake things up a bit.
Sam