I recently made the switch from Hostgator Shared hosting to a Linode instance, and WOW. It's amazing what you can do with a single (relatively small) box dedicated to serving up a single site. Without any further ado, I want to share some tips I gathered from setting the server up, and some benchmark results from along the way.
Tip 1: Use Percona DB. I don't know enough about database technology to say conclusively that it's better than MySQL, but it's a performance oriented fork, and installing it is as easy as adding the Percona Repository to your sources.list file and apt-get installing it. It's a drop-in replacement, so you'll never know it's not regular MySQL.
Tip 2: APC is amazing. It will more than double the speed your PHP scripts run. It's dead simple to install, and there's no reason not to.
Tip 3: Use Varnish if you can. If you can handle an apt-get install of varnish and following these instructions, do it. For Drupal 7, the excellent Varnish module is a drop in replacement for the regular page cache.
Tip 4: As long as it's a single server install, skip Memcache. Even though it's hailed as being a performance super-enhancer for Drupal, that only applies if you are in an environment where you need to access the same cache from multiple servers. Instead of memcache, use the APC module for caches that won't change much, like
cache_bootstrap and 'cache_menu'. It's way faster than memcache within the same server, and there's nothing extra to install.
|Stock Drupal + Percona
|+ cache bootstrap & cache menu in APC
|+Drupal page cache
|+Varnish again, more requests
|+page cache invoke hooks = FALSE & page cache without database = TRUE in settings.php
These tests were performed using apachebench (ab). As you can see, I had some trouble actually stressing Varnish; so much trouble I had to spin up a large EC2 instance to deliver the testing traffic. While these results aren't quite up to Project Mercury standards, for a 2 hour install on a $20/month server, I am extremely impressed. Here's the obligatory linode referral link.