Shattered Symmetry - Home

Using Capistrano To Control Xen Domains

2008-12-23T02:57:00Z

Since virtualizing CentOS 5.2, I've been running 4 Ubuntu Xen domains:

sudo xm list

Name                       ID Mem(MiB) VCPUs State   Time(s)
Domain-0                    0      793     4 r-----    779.1
u810s1                      1     1031     2 -b----   3023.7
u810s2                      2     2055     2 -b----   7071.0
u810s3                      3     2055     2 -b----   6805.7
u810s4                      4     2055     2 -b----   7103.9

dom0 is the base system (Centos 5.2); u810s1 is an Ubuntu 8.10 Server instance with 1Gb of RAM; u810s2-4 are the same but with 2Gb each. Each machine has a common user account and password, and that user has the following entry at the bottom of /etc/sudoers on each machine:

justin ALL=NOPASSWD: ALL

Also in each user's ~/.ssh/authorized_keys file is the public key from the user justin on dom0, so passwordless SSH can happen.

I installed Ruby Enterprise Edition on dom0, as well as RubyGems and Capistrano. I know the IP addresses of the u810s instances. So where does that leave me?...

Able to control them all in unison. :-)

capfile

role :app_servers, "192.168.1.201", "192.168.1.202", 
                        "192.168.1.203", "192.168.1.204"

# `cap update_ubuntu`
task :update_ubuntu, :roles => :app_servers do
  run "sudo apt-get update"
  run "sudo apt-get upgrade -y"
end

# `cap shutdown_all`
task :shutdown_all, :roles => :app_servers do
  run "sudo shutdown -P now"
end

# e.g. `cap -s pkg=denyhosts install`
task :install, :roles => :app_servers do
  run "sudo apt-get install #{pkg} -y"
end

Virtualizing CentOS 5.2

2008-12-07T02:40:00Z

I recently switched my primary computer from a Linux desktop to a Macbook Pro. Holy wars aside, having a Q9450 with 2TB of disk space and 8GB of DDR2 sitting in the corner collecting dust was getting on my nerves. I set it up as a file and print server for the house, but the CPU was sitting idle most of the time. Today I decided to install Xen on it so I could create disposable development environments. I've been experimenting with different Rails and PHP architectures, so being able to create virtual environments to test things out seemed like the way to go.

Seeing as I've been spending way too much time with Ubuntu as of late, I decided to jump back into the CentOS world for a spell. Getting Xen up and running was surprising straight-forward on CentOS 5.2 (x64, "Server - GUI" install option):

# install the Xen packages
yum install kernel-xen xen

# make the Xen kernel the default
vi /boot/grub/menu.lst
change default=1 to default=0

# restart to use the new kernel
reboot -t now

After you've restarted, you're ready to get some virtual machines going:

# create a new virtual machine
virt-install

What is the name of your virtual machine? u810d
How much RAM should be allocated (in megabytes)? 2048
What would you like to use as the disk (file path)? /vm/u810d.img
How large would you like the disk (/vm/vm01.img) to be (in gigabytes)? 4
Would you like to enable graphics support? (yes or no) yes
What is the install location? /media/ubuntu-8.10-desktop-i386.iso

(Astute readers will notice the first thing I installed was Ubuntu...)

The xm command is used to control your Xen VMs at this point. Here are a few of the commands that you'll use often (as root or with sudo):

xm create u810d - start the virtual machine
xm shutdown u810d - stop the virtual machine
xm destroy u810d - forcefully stop the virtual machine
virt-viewer u810d - get console access to the virtual machine via X11
xm list - list all running virtual machines

And since my machine is sitting behind a hefty firewall, I enabled remote desktop in Ubuntu and use Apple's Remote Desktop as my VNC client for any XFree86 access... I use SSH (with X11 forwarding as needed) for most everything else. Get a base instance you like up and running, and then just copy the image and configuration files to a new location to create clones.

Upgrading MySQL From 4.x to 5.0.x On FreeBSD 6

2008-11-29T23:48:00Z

I had to update a MySQL installation for a client this weekend who runs FreeBSD. I've never done a MySQL 4.x to 5.0.x upgrade before, so I thought I'd share my notes from the process. It's relatively straight-forward:

# make a SQL backup in case something explodes
mysqldump --all-databases > mysql_backup.sql

# stop MySQL
/usr/local/etc/rc.d/mysql-server stop

# remove the existing MySQL client and server packages
# (run `mysql -V` to get your MySQL version number)
pkg_delete mysql-server-4.1.21
pkg_delete mysql-client-4.1.21

# add the LinuxThreads package, which we'll use to get better threading
pkg_add -r linuxthreads

# build the new version of MySQL from source, using LinuxThreads
cd /usr/ports/databases/mysql50-server/
make WITH_LINUXTHREADS=yes BUILD_OPTIMIZED=yes BUILD_STATIC=yes
make install

# start up MySQL 5
/usr/local/etc/rc.d/mysql-server start

# upgrade the MySQL data files from 4.x to 5.0.x
/usr/local/bin/mysql_upgrade -u root -p --datadir=/var/db/mysql

The mysql_upgrade script gave me some issues, as it would report that tables needed to be repaired but wouldn't actually repair them. I used the following PHP script to repair all the tables prior to running mysql_upgrade:

<?php
### Enter your username and password into the connection string: ###
$dbLink = mysql_connect("localhost", "user", "pass") or die("Unable to connect.");

# hack to get a list of database names;
# the actual query should be "SHOW DATABASES", but breaks for info schema
$sql = "SELECT 'mydatabasename' as 'Database' UNION ALL SELECT 'myseconddatabase' as 'Database'";
$query = mysql_query($sql) or die("error fetching database names");

while ($rs=mysql_fetch_array($query)) {

mysql_select_db($rs['Database'],$dbLink) or die("Unable to select database: " . $rs['Database']);

$sql = "SHOW TABLES";
$query2 = mysql_query($sql) or die("error fetching table names");

while ($rs2=mysql_fetch_array($query2)) {
  $key = "Tables_in_" . $rs['Database'];
  $sql = "REPAIR TABLE " . $rs2[$key];
  $query3 = mysql_query($sql) or die("Error repairing a table - $sql");
  echo "Repaired " . $rs2[$key] . "\n";
}
}

echo "Finished!\n";
?>

You're now running MySQL 5.0.x! The majority of your queries should work without any changes; the main issue I came across was with implicit INNER ("comma") JOINs. As always, test in dev before touching production. :-)

Why You Need DenyHosts

2008-11-25T04:16:00Z

Since installing DenyHosts four days ago, this is what my /etc/hosts.deny file looks like:

sshd: 189.56.25.146
sshd: 211.245.106.195
sshd: 189.19.43.55
sshd: 58.120.97.64
sshd: 61.129.64.137
sshd: 190.90.3.171
sshd: 125.69.132.102
sshd: 208.116.36.234
sshd: 75.40.150.27
sshd: 220.162.243.52
sshd: 81.218.0.18
sshd: 219.238.236.3
sshd: 200.33.10.41
sshd: 202.70.193.25
sshd: 222.83.251.84
sshd: 117.36.192.75

Brazil, Korea, China, Columbia, Israel, Mexico, India and friggin' New Jersey. One of these things does not belong.

Update: more additions to the wall of shame...

sshd: 92.50.193.94
sshd: 210.211.187.31
sshd: 12.45.22.213
sshd: 78.40.226.156
sshd: 218.16.101.70
sshd: 65.44.85.155
sshd: 189.10.197.107
sshd: 201.49.29.169
sshd: 212.150.149.164
sshd: 61.148.212.34
sshd: 222.221.12.13
sshd: 24.159.194.218
sshd: 207.226.88.28
sshd: 219.137.24.12
sshd: 202.10.65.81
sshd: 81.208.51.90
sshd: 74.208.99.46
sshd: 24.128.78.196

Install Xdebug On Mac OS X Leopard 10.5.5

2008-11-23T00:55:00Z

Here's a quick how-to explaining how to get xdebug working on Leopard. To get started, download the xdebug source from http://xdebug.org/. Untar the source and cd into the directory. Then:

#prepare the build environment
phpize

# configure xdebug for Leopard 10.5.5
MACOSX_DEPLOYMENT_TARGET=10.5 CFLAGS="-arch ppc -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe -no-cpp-precomp" CCFLAGS="-arch ppc -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe" CXXFLAGS="-arch ppc -arch ppc64 -arch i386 -arch x86_64 -g -Os -pipe" LDFLAGS="-arch ppc -arch ppc64 -arch i386 -arch x86_64 -bind_at_load" ./configure --enable-xdebug

# compile xdebug
make

# install it to the php extensions folder
sudo cp modules/xdebug.so /usr/lib/php/extensions/no-debug-non-zts-20060613/

# configure your php.ini to load xdebug
sudo vi /etc/php.ini
add:
[xdebug]
; load xdebug
zend_extension="xdebug.so"
# finally, restart Apache
sudo apachectl restart

If you do it any other way, you'll get a non-64-bit extension that you'll be able to see with 'php -m' but won't be active in Apache. It took me a bloody hour to figure this out.

Speed Up Mephisto By Enabling Caching

2008-11-23T00:29:00Z

One of the things that wasn't obvious about Mephisto when I set it up was its oh-so awesome caching support. Behind the scenes, it does full-page caching for all site content and stores it within the public/cache directory. Unfortunately, it doesn't use the cache unless you configure your web server to do so, and there's no mention of this in the documentation. :-)

So how does one enable caching in Mephisto? Well, I'm using Apache, so I can just enable mod_rewrite and redirect you the reader to the cache file for a page if it exists. The good part about this is that the request avoids Rails altogether, so it's especially speedy and gentle on your server. (If you're using a different web server, the process will be similar, but you'll obviously have to adapt the syntax to your specific needs.)

Just put the following RewriteRules in your httpd.conf file:

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}/cache/%{HTTP_HOST}/$1.html -f
RewriteRule ^/([^.]+)$ /cache/%{HTTP_HOST}/$1.html [QSA,L]

Or, if like me, you've edited config/initializers/custom.rb in Mephisto to disable multi-site support, your rewrite rules are even simpler:

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}/$1.html -f
RewriteRule ^/([^.]+)$ /$1.html [QSA,L]

Restart Apache and enjoy the speed boost!

Secure Your Ubuntu 8.10 Linode With Iptables

2008-11-21T05:05:00Z

Setting up iptables is essential to having a secure Linux installation; leaving ports open when no service is running on them sounds like a rational idea, but it misses the point: actively blocking/dropping connections is much better than doing nothing at all. Showing the script kiddies that you know what you're doing is a powerful deterrent in and of itself, and iptables is the first step towards that.

The first thing I did after getting my Linode up and running was setup iptables to get a firewall protecting my server. Here's how you can, too:

# change to the root user
sudo -i

# install iptables
apt-get install iptables

# create your iptables configuration
vi /etc/iptables.up.rules

add:
*filter

-A INPUT -i lo -j ACCEPT
-A INPUT -i ! lo -d 127.0.0.0/8 -j REJECT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A OUTPUT -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT
-A INPUT -m limit --limit 5/min -j LOG --log-prefix \
      "iptables denied: " --log-level 7
-A INPUT -j REJECT
-A FORWARD -j REJECT
COMMIT
# apply these rules to the server
iptables-restore < /etc/iptables.up.rules

# then, setup your interfaces so the rules are reapplied at startup
vi /etc/network/interfaces

add:
auto lo
iface lo inet loopback
pre-up iptables-restore < /etc/iptables.up.rules

This will leave you with ports 80, 443 and 22 open, and everything else closed. For good measure, you should also install DenyHosts, which will automatically blacklist IP addresses that have multiple failed SSH login attempts:

apt-get install denyhosts

Finally, update your SSH config to not allow remote root logins:

vi /etc/ssh/sshd_config

change:
PermitRootLogin yes
to
PermitRootLogin no

# reload SSH
/etc/init.d/ssh reload

That's it! You now have a more secure Ubuntu 8.10 Linode.

Installing Mephisto on Ubuntu 8.10

2008-11-20T02:54:00Z

Note: this tutorial assumes you already have a working Ruby on Rails installation (installed via apt-get/aptitude with rubygems built from source). Bonus points will be awarded if you already have Ruby Enterprise Edition installed.

When it comes to Rails blog engines, there aren't too many choices; Typo and Mephisto are the two most popular packages, and both are rather stagnant in terms of development. Seeing as the Ruby world moves fast and the Rails world even faster, this can make it difficult to get either of these packages working with the current versions of Ruby and/or Rails. Ubuntu 8.10, the Intrepid Ibex from hell, has this problem. It took a few tries, but this blog is using Mephisto now, and I'd like to share the process that worked for me so you can get it running for your blog, too.

Ubuntu 8.10 comes with Ruby 1.8.7 and Rails 2.1.2. This is problematic for Mephisto in two ways: Mephisto only works with Rails <= 2.0.2, and Ruby 1.8.7 removes some Enumerate functionality that Mephisto relies on. But with a little maneuvering, we can get Mephisto working just fine. The process looks like this:

# install the required gem dependency:
sudo gem install tzinfo

# get Mephisto from GitHub
wget http://github.com/technoweenie/mephisto/tarball/master.tar.gz

# untar the file and cd into the directory
tar zxvf technoweenie-mephisto-*.tar.gz
cd technoweenie-mephisto-*

# put the 2.0.2 version of Rails on ice
rake rails:freeze:edge RELEASE=2.0.2

# use the 2.0.2 boot file in Mephisto
cp vendor/rails/railties/environments/boot.rb config/boot.rb

# create your production database
mysqladmin create mephisto_production

-- do the usual config/database.yml username/password/database setup--

# hack Mephisto to work with Rails 2.1.2 and Ruby 1.8.7
vi config/boot.rb and add the following lines at the top:
RAILS_GEM_VERSION = '2.0.2' unless defined? RAILS_GEM_VERSION

unless '1.9'.respond_to?(:force_encoding)
String.class_eval do
    begin
      remove_method :chars
    rescue NameError
      # OK
    end
  end
end
# bootstrap the production database
rake db:bootstrap RAILS_ENV=production

# restart your webserver for good measure
sudo /etc/init.d/apache2 restart

Provided you did this in a web accessible location, you should be able to log into Mephisto now. The magic is freezing Rails to 2.0.2, updating the boot.rb file to force 2.0.2 to be used and overriding Ruby 1.8.7's Enumerate weirdness. If you're using Ruby Enterprise Edition, you can ignore the 'unless' section above since REE is still 1.8.6. Make sure to install tzinfo with REE's gem binary, though, or mod_rails will give you a nice FAIL message when you try to access Mephisto.

Installing Ruby Enterprise Edition on Ubuntu 8.10

2008-11-19T06:20:00Z

So Ruby Enterprise Edition doesn't compile on Ubuntu 8.10. It appears as though the version of GCC that comes bundled in Intrepid's build-essential (4.3.2-1ubuntu11) is too awesome for REE to handle. Actually, it's due to some path hacking that REE does to work on systems like FreeBSD, but I digress...

So you've all but given up hope of getting REE to work on your Intrepid install. But fear not; the change is actually really simple! In order to make it compile properly, you just need to apply the following patch to the REE source:

--- /tmp/ruby-enterprise-1.8.6-20080810/installer.rb
+++ ruby-enterprise-1.8.6-20080810/installer.rb
@@ -159,9 +159,9 @@
@destdir += "/"
end

- ENV['C_INCLUDE_PATH'] = "#{@destdir}#{@prefix}/include:/usr/include:/usr/local/include:#{ENV['C_INCLUDE_PATH']}"
- ENV['CPLUS_INCLUDE_PATH'] = "#{@destdir}#{@prefix}/include:/usr/include:/usr/local/include:#{ENV['CPLUS_INCLUDE_PATH']}"
- ENV['LD_LIBRARY_PATH'] = "#{@destdir}#{@prefix}/lib:#{ENV['LD_LIBRARY_PATH']}"
+ #ENV['C_INCLUDE_PATH'] = "#{@destdir}#{@prefix}/include:/usr/include:/usr/local/include:#{ENV['C_INCLUDE_PATH']}"
+ #ENV['CPLUS_INCLUDE_PATH'] = "#{@destdir}#{@prefix}/include:/usr/include:/usr/local/include:#{ENV['CPLUS_INCLUDE_PATH']}"
+ #ENV['LD_LIBRARY_PATH'] = "#{@destdir}#{@prefix}/lib:#{ENV['LD_LIBRARY_PATH']}"

--- /tmp/ruby-enterprise-1.8.6-20080810/source/vendor/google-perftools-0.98/src/base/linuxthreads.c
+++ ruby-enterprise-1.8.6-20080810/source/vendor/google-perftools-0.98/src/base/linuxthreads.c
@@ -49,7 +49,7 @@
#include <asm/fcntl.h>
#include <asm/posix_types.h>
#include <asm/types.h>
-#include <linux/dirent.h>
+#include <dirent.h>

For those not well-versed in the intricacies of diff, you need to comment out the three ENV lines in installer.rb starting at line 161, and you need to change the <linux/dirent.h> include directive to point to plain ol' <dirent.h> in source/vendor/google-perftools-0.98/src/base/linuxthreads.c. The rest of the Ruby Enterprise Edition install works the same as in the documentation.

That's all there is to it. :-D

If you found this post at all useful, I would encourage you to buy an Enterprise License for Ruby Enterprise Edition to support Phusion. The gods of Karma will thank you.

Rebooted

2008-11-19T02:59:00Z

Welcome to my new home. All hail Mephisto!