atom.xml

<?xml version="1.0" encoding="utf-8" ?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Joseph Ruscio</title>
  <link href="http://joseph.ruscio.org/atom.xml" rel="self" />
  <link href="http://joseph.ruscio.org" />
  <updated>2010-08-23T06:20:06-07:00</updated>
  <id>http://joseph.ruscio.org</id>
  <author>
    <name>Joseph Ruscio</name>
    <email>joe@ruscio.org</email>
  </author>
  <entry>
    <title>Setting up an APT Repository</title>
    <link href="http://joseph.ruscio.org/blog/2010/08/19/setting-up-an-apt-repository" />
    <updated>2010-08-19T00:00:00-07:00</updated>
    <id>http://joseph.ruscio.org/blog/2010/08/19/setting-up-an-apt-repository</id>
    <content type="html">
      &lt;p&gt;I recently set out to create an APT repository to host publicly available
      packages for &lt;a href=&quot;http://librato.com&quot;&gt;Librato&#8217;s Silverline&lt;/a&gt;. Our users install
      a tiny agent on their servers and a hosted APT repository provides those
      using Debian/Ubuntu a configuration management solution superior to
      manual downloads.&lt;/p&gt;
      
      &lt;p&gt;Our APT repository must support multiple distributions of both Debian
      (e.g. Etch, Lenny) and it&#8217;s popular derivative Ubuntu (e.g. Karmic, Lucid).
      The repository must support both the i386 and x86_64 architectures across
      all such distributions. A short google search reveals that &lt;code&gt;reprepro&lt;/code&gt; is
      a tool designed for just such a purpose. There are also
      &lt;a href=&quot;http://www.jejik.com/articles/2006/09/setting_up_and_managing_an_apt_repository_with_reprepro/&quot;&gt;informative&lt;/a&gt;
      &lt;a href=&quot;http://www.danielbond.org/archives/114&quot;&gt;reprepro&lt;/a&gt; &lt;a href=&quot;http://davehall.com.au/blog/dave/2010/02/06/howto-setup-private-package-repository-reprepro-nginx&quot;&gt;walkthroughs&lt;/a&gt;
      already online, but they all gloss over some subtle yet extremely
      important details.&lt;/p&gt;
      
      &lt;p&gt;So what follows is my attempt to minimally but comprehensively document the
      procedure assuming a familiarity with shell basics, but little or no experience
      with Debian packging past &lt;code&gt;apt-get update/upgrade/install&lt;/code&gt;.&lt;/p&gt;
      
      &lt;h3&gt;Host Environment&lt;/h3&gt;
      
      &lt;p&gt;Our APT repository is hosted on a fresh installation of Ubuntu 10.04 Lucid, but
      the procedure should work on any recent version of Debian/Ubuntu.&lt;/p&gt;
      
      &lt;h3&gt;Package Configuration&lt;/h3&gt;
      
      &lt;p&gt;The most important detail that AFAIK isn&#8217;t covered in any of the tutorials
      had to do with package naming conventions. The naive assumption (at least
      on my part) is that you&#8217;ll have a different build of your package for
      each distro/arch combination, and import them into your repository as such.
      In other words &lt;code&gt;reprepro&lt;/code&gt; should track the distro/arch of each import.
      In actuality, each build&#8217;s &lt;code&gt;&lt;PACKAGE&gt;_&lt;VERSION&gt;_&lt;ARCH&gt;&lt;/code&gt; must be unique, even though
      you specify the distro during the &lt;code&gt;includedeb&lt;/code&gt; operation.&lt;/p&gt;
      
      &lt;p&gt;To address this requirement there&#8217;s a common practice of appending the
      distro to the package version e.g. &lt;code&gt;1.0.7&lt;/code&gt; becomes &lt;code&gt;1.0.7~lenny1&lt;/code&gt; or &lt;code&gt;1.0.7+etch4&lt;/code&gt;.
      (There doesn&#8217;t appear to be a consensus on the joining character.) Adding
      the second version number after the distro string versions the
      package itself, enabling updates that only contain changes to the packaging itself.&lt;/p&gt;
      
      &lt;p&gt;If you do not build your packages to include the distro as part of the version, you
      will not be able to import a package for more than one distro. You can verify your
      package was built correctly by inspecting the &lt;code&gt;Version&lt;/code&gt; field listed by &lt;code&gt;dpkg -I&lt;/code&gt;:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~/lenny$ dpkg -I librato_silverline_debian_lenny_5.0.3.x86_64.deb
       new debian package, version 2.0.
       size 306916 bytes: control archive= 834 bytes.
       255 bytes,     7 lines      conffiles            
       277 bytes,    10 lines      control              
       991 bytes,    26 lines   *  postinst             #!/bin/sh
       Package: librato-silverline
       Version: 2.0.7~lenny
      ...
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;h3&gt;GnuPG&lt;/h3&gt;
      
      &lt;p&gt;Assuming you want sign your debian packages to create a secure APT repository,
      you need a GPG key. If you already have a GPG key you can skip to the next
      section. Otherwise first ensure that GPG is installed:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~$ sudo apt-get install gnupg
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Create a key with the &lt;code&gt;--gen-key&lt;/code&gt; command. Select the defaults and be sure to
      record the key&#8217;s passphrase, you&#8217;ll need it each time you import a package into
      the repository:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~$ gpg --gen-key
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;h3&gt;Install and Configure reprepro&lt;/h3&gt;
      
      &lt;p&gt;Armed with properly configured packages and a GPG key with which to sign them,
      we&#8217;re now ready to start constructing the repository itself. Start by installing
      &lt;code&gt;reprepro&lt;/code&gt;:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:-$ sudo apt-get install reprepro
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Create a directory to serve as the &lt;em&gt;base&lt;/em&gt; of the Debian repository. In our setup
      &lt;code&gt;/var/packages/debian&lt;/code&gt; serves as the Debian repository base. In the future
      &lt;code&gt;/var/packages/ubuntu&lt;/code&gt; will house the Ubuntu repository base. We&#8217;ll change ownership
      of these directories to the &lt;code&gt;ubuntu&lt;/code&gt; user that owns the signing key.&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~$ sudo mkdir /var/packages
      ubuntu:~$ sudo chown ubuntu:ubuntu /var/packages
      ubuntu:~$ mkdir /var/packages/debian
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Each repository needs a top-level &lt;code&gt;conf&lt;/code&gt; directory:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~$ mkdir /var/packages/debian/conf
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Create a file named &lt;code&gt;distributions&lt;/code&gt; in the &lt;code&gt;conf&lt;/code&gt; directory. You&#8217;ll add a set
      of newline separated configuration blocks (one for each supported distro) to
      this file. In this example the repository supports &lt;code&gt;etch&lt;/code&gt; and &lt;code&gt;lenny&lt;/code&gt;. In the future
      &lt;code&gt;squeeze&lt;/code&gt; support could be added with a third block:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;Origin: Librato, Inc.
      Label: Librato, Inc.
      Codename: etch
      Architectures: i386 amd64
      Components: non-free
      Description: Librato APT Repository
      SignWith: yes
      DebOverride: override.etch
      DscOverride: override.etch
      
      Origin: Librato, Inc.
      Label: Librato, Inc.
      Codename: lenny
      Architectures: i386 amd64
      Components: non-free
      Description: Librato APT Repository
      SignWith: yes
      DebOverride: override.lenny
      DscOverride: override.lenny
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Fill in the &lt;code&gt;Origin&lt;/code&gt;, &lt;code&gt;Label&lt;/code&gt;, and &lt;code&gt;Description&lt;/code&gt; as you see fit. &lt;code&gt;Codename&lt;/code&gt; identifies the
      distro the block describes and &lt;code&gt;Architectures&lt;/code&gt; is self-explanatory (note that &lt;code&gt;sources&lt;/code&gt; is a valid arch).
      &lt;code&gt;Components&lt;/code&gt; lists the component of the packages in the repository e.g. for
      Debian &lt;code&gt;main&lt;/code&gt;, &lt;code&gt;contrib&lt;/code&gt;, or &lt;code&gt;non-free&lt;/code&gt;. &lt;code&gt;DebOverride&lt;/code&gt; and &lt;code&gt;DscOverride&lt;/code&gt; exceed the scope of
      this posting, the reader may either research them on their own or set them as shown.&lt;/p&gt;
      
      &lt;p&gt;&lt;code&gt;SignWith&lt;/code&gt; instructs &lt;code&gt;reprepro&lt;/code&gt; that these packages should be signed. The &lt;code&gt;yes&lt;/code&gt;
      is sufficient as the &lt;code&gt;ubuntu&lt;/code&gt; user only has the one key generated above. If
      you have more than one key you can specify the ID of the signing key with this
      field.&lt;/p&gt;
      
      &lt;p&gt;Create empty override files:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~$ touch /var/packages/conf/override.etch
      ubuntu:~$ touch /var/packages/conf/override.lenny
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Create a file named &lt;code&gt;options&lt;/code&gt; in the &lt;code&gt;conf&lt;/code&gt; directory and fill it with the
      following content. This file will store a
      set of options for &lt;code&gt;reprepro&lt;/code&gt; to always run. We only use a few but there are
      more available on the &lt;code&gt;reprepro&lt;/code&gt; manpage.&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;verbose
      ask-passphrase
      basedir .
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;The &lt;code&gt;verbose&lt;/code&gt; option is self-explanatory. The ask-passphrase option instructs
      &lt;code&gt;reprepro&lt;/code&gt; to ask us for a GPG key passphrase during the import (otherwise
      it&#8217;ll fail to sign the packages). The &lt;code&gt;basedir .&lt;/code&gt; implies that we&#8217;ll be
      running all of our &lt;code&gt;reprepro&lt;/code&gt; commands with the repository base as our
      working directory:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:~$ cd /var/packages/debian
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;h3&gt;Import Packages&lt;/h3&gt;
      
      &lt;p&gt;The repository base is completely configured and ready for package importing. Just
      use the &lt;code&gt;includedeb&lt;/code&gt; command and specify the correct distro. In our case:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:/var/packages/debian$ reprepro includedeb etch ~/librato_silverline_debian_etch_4.0r8.x86_64.deb
      ...
      ubuntu:/var/packages/debian$ reprepro includedeb lenny ~/ubuntu/librato_silverline_debian_lenny_5.0.3.x86_64.deb
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Each &lt;code&gt;reprepro includedeb&lt;/code&gt; operation should prompt for your GPG key passphrase and import the package.&lt;/p&gt;
      
      &lt;h3&gt;Repository Access&lt;/h3&gt;
      
      &lt;p&gt;We now have a proper Debian repository and all that remains is making it
      accessible for installations over the WAN. We just need to serve static
      files so we&#8217;ll use &lt;code&gt;nginx&lt;/code&gt; to serve the packages over HTTP.&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:/var/packages/debian$ sudo apt-get install nginx
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Configure the APT server in &lt;code&gt;/etc/nginx/sites-available/vhost-packages.conf&lt;/code&gt;:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;server {
        listen 80;
        server_name apt.librato.com;
      
        access_log /var/log/nginx/packages-error.log;
        error_log /var/log/nginx/packages-error.log;
      
        location / {
      root /var/packages;
      index index.html;
        }
      
        location ~ /(.*)/conf {
      deny all;
        }
      
        location ~ /(.*)/db {
      deny all;
        }
      }
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Configure the hash bucket size by creating the file
      &lt;code&gt;/etc/nginx/conf.d/server_names_hash_bucket_size.conf&lt;/code&gt;&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;server_names_hash_bucket_size 64;
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Enable the APT server:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;ubuntu:/var/packages/debian$ cd /etc/nginx/sites-enabled
      ubuntu:/etc/nginx/sites-enabled$ sudo ln -s ../sites-available/vhosts-packages.conf .
      ubuntu:/etc/nginx/sites-enabled$ sudo service nginx start
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;h3&gt;Enable Public Key Access&lt;/h3&gt;
      
      &lt;p&gt;Our users will need our public key to validate the signed packages we&#8217;re
      providing. Placing it in the root of our &lt;code&gt;nginx&lt;/code&gt; configuration above
      makes it accessible with a simple &lt;code&gt;curl&lt;/code&gt; invocation:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;gpg --armor --output /var/packages/packages.librato.key --export apt@librato.com
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;h3&gt;Installation Test&lt;/h3&gt;
      
      &lt;p&gt;Now we can test our fully operational APT repository. On some
      candidate machine log in as root and add our repository to
      &lt;code&gt;/etc/apt/sources.list&lt;/code&gt;:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;deb http://apt.librato.com/debian/ lenny non-free
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Import the repository&#8217;s public key:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;lenny:# curl http://apt.librato.com/packages.librato.key | apt-key add -
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Fetch the list of packages available at the new source:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;lenny:# apt-get update
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Install the hosted package!&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;lenny:# apt-get install librato-silverline
      &lt;/code&gt;&lt;/pre&gt;
    </content>
  </entry>
  <entry>
    <title>Increasing hard disk size with LVM on VMWare Fusion Linux guest</title>
    <link href="http://joseph.ruscio.org/blog/2009/12/08/increasing-hard-disk-size-with-lvm-on-vmware-fusion-linux-guest" />
    <updated>2009-12-08T00:00:00-08:00</updated>
    <id>http://joseph.ruscio.org/blog/2009/12/08/increasing-hard-disk-size-with-lvm-on-vmware-fusion-linux-guest</id>
    <content type="html">
      &lt;p&gt;I currently use VMware Fusion to maintain and run a Linux guest on my Macbook.
      When I originally installed the guest, I assumed that a 20 GB virtual disk
      would be more than enough space for my Linux hacking.&lt;/p&gt;
      
      &lt;p&gt;As you can probably infer from the title of this post, some recent developments
      invalidated that assumption and necessitated more disk space. What follows is a
      procedure I cobbled together from a bunch of disparate sources that works for my
      particular setup (VMware Fusion 2.0.4, Fedora 10 guest w/LVM and an ext3
      filesystem). These instructions involve modifying physical partitions, logical
      volumes, and your filesystem so I advise you to read through them to the end
      before getting started.&lt;/p&gt;
      
      &lt;p&gt;The very first step is to use Fusion to increase the &#8220;physical size&#8221; of the
      virtual disk. This functionality is accessible through the &lt;em&gt;Virtual Machine
      -&gt; Hard Disk -&gt; Hard Disk Settings&lt;/em&gt; menu item. The resulting dialog contains a
      &lt;em&gt;Disk size&lt;/em&gt; slider. Note that this slider is not accessible unless the VM in
      question is shut down and (more annoyingly) all pre-existing snapshots are deleted.
      I just had one snapshot to delete, but it took a while, so be patient. After
      you&#8217;ve increased the disk size, probably wouldn&#8217;t hurt to take a new snapshot,
      just in case something goes wrong with the procedure below.&lt;/p&gt;
      
      &lt;p&gt;Now that we&#8217;ve increased the disk size (I increased mine from 20GB -&gt; 50GB), let&#8217;s
      boot the VM and log into our guest. We&#8217;ll use the &lt;em&gt;df&lt;/em&gt; command to examine our
      mounted file filesystems:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# df -h
      Filesystem            Size  Used Avail Use% Mounted on
      /dev/mapper/VolGroup00-LogVol00
                 18G   15G  1.9G  89% /
      /dev/sda1             190M  163M   18M  91% /boot
      tmpfs                 502M   80K  502M   1% /dev/shm
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Still shows 20 GB of total space and a filesystem device &lt;em&gt;/dev/mapper/VolGroup00-LogVol00&lt;/em&gt;
      that&#8217;s indicative of LVM, the default for Fedora. Use the venerable &lt;em&gt;fdisk&lt;/em&gt; to examine
      the &#8220;physical&#8221; disk:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# fdisk -l
      Disk /dev/sda: 52.6 GB, 52613349376 bytes
      255 heads, 63 sectors/track, 6396 cylinders
      Units = cylinders of 16065 * 512 = 8225280 bytes
      Disk identifier: 0x000f2b12
      Device    Boot      Start         End      Blocks   Id  System
      /dev/sda1   *           1          25      200781   83  Linux
      /dev/sda2              26        2610    20764012+  8e  Linux LVM
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;The physical disk has the capacity, but its in the form of unpartitioned/unformatted space.
      Before we can do anything, We need to create a new partition containing this space.
      First we&#8217;ll drop into &lt;em&gt;parted&lt;/em&gt; and examine the existing partition table:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;(parted) print
      Model: VMware, VMware Virtual S (scsi)
      Disk /dev/sda: 52.6GB
      Sector size (logical/physical): 512B/512B
      Partition Table: msdos
      Number  Start   End     Size    Type     File system  Flags
       1      32.3kB  206MB   206MB   primary  ext3         boot 
       2      206MB   21.5GB  21.3GB  primary               lvm  
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Create a new partition that uses up all of the new space on the physical device:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;(parted) mkpart primary 21.4GB -1s 
      (parted) print
      Model: VMware, VMware Virtual S (scsi)
      Disk /dev/sda: 52.6GB
      Sector size (logical/physical): 512B/512B
      Partition Table: msdos
      Number  Start   End     Size    Type     File system  Flags
       1      32.3kB  206MB   206MB   primary  ext3         boot 
       2      206MB   21.5GB  21.3GB  primary               lvm  
       3      21.5GB  52.6GB  31.1GB  primary                    
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Use &lt;em&gt;partprobe&lt;/em&gt; to load the table:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;(parted) quit
      [root@fedora ~]# partprobe
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;At this point we&#8217;ve got a new partition, &lt;em&gt;/dev/sda3&lt;/em&gt;, that you could simply
      format with &lt;em&gt;mke2fs&lt;/em&gt; and mount as its own filesystem e.g. &lt;em&gt;/mnt/data&lt;/em&gt;. I
      personally chose to leverage LVM&#8217;s capabilities and incorporate the new
      partition into the existing logical volume mounted as /. Start by examining
      the set of existing physical volumes:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# lvm pvs
        PV         VG         Fmt  Attr PSize  PFree 
        /dev/sda2  VolGroup00 lvm2 a-   19.78G 32.00M
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Create a new physical volume with the new partition:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# lvm pvcreate /dev/sda3
        Physical volume &quot;/dev/sda3&quot; successfully created
      [root@fedora ~]# lvm pvs
        PV         VG         Fmt  Attr PSize  PFree 
        /dev/sda2  VolGroup00 lvm2 a-   19.78G 32.00M
        /dev/sda3             lvm2 --   29.01G 29.01G
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Add new physical volume to the volume group. If you recall from the initial
      &lt;em&gt;df&lt;/em&gt; output above, the volume group in question is VolGroup0:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# lvm vgextend VolGroup00 /dev/sda3
        Volume group &quot;VolGroup00&quot; successfully extended
      [root@fedora ~]# lvm pvs
        PV         VG         Fmt  Attr PSize  PFree 
        /dev/sda2  VolGroup00 lvm2 a-   19.78G 32.00M
        /dev/sda3  VolGroup00 lvm2 a-   29.00G 29.00G
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Now extend the logical volume to include the physical volume we just added to
      the group. Use &lt;em&gt;lvm vgdisplay&lt;/em&gt; to examine the group:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# lvm vgdisplay VolGroup00
        --- Volume group ---
        VG Name               VolGroup00
        System ID
        Format                lvm2
        Metadata Areas        2
        Metadata Sequence No  4
        VG Access             read/write
        VG Status             resizable
        MAX LV                0
        Cur LV                2
        Open LV               2
        Max PV                0
        Cur PV                2
        Act PV                2
        VG Size               48.78 GB
        PE Size               32.00 MB
        Total PE              1561
        Alloc PE / Size       632 / 19.75 GB
        Free  PE / Size       929 / 29.03 GB
        VG UUID               8w2Wi9-T2lV-IKCV-fLRP-yTJl-teJM-LuGoNb
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;The &lt;em&gt;Free PE/Size&lt;/em&gt; field shows the new physical volume as 929 free extents.
      Extend the logical volume to include all free extents:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# lvm lvextend -l+929 /dev/VolGroup00/LogVol00
        Extending logical volume LogVol00 to 46.81 GB
        Logical volume LogVol00 successfully resized
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Use &lt;em&gt;resize2fs&lt;/em&gt; to extend our ext3 filesystem online. Note that many LVM
      tutorials online refer to the now deprecated &lt;em&gt;ext2online&lt;/em&gt; command for this
      step, &lt;em&gt;resize2fs&lt;/em&gt; now provides this functionality:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;root@fedora ~]# resize2fs /dev/VolGroup00/LogVol00 
      resize2fs 1.41.4 (27-Jan-2009)
      Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required
      old desc_blocks = 2, new_desc_blocks = 3
      Performing an on-line resize of /dev/VolGroup00/LogVol00 to 12271616 (4k) blocks.
      The filesystem on /dev/VolGroup00/LogVol00 is now 12271616 blocks long.
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Finally, use &lt;em&gt;df&lt;/em&gt; again to verify that we have lots of free space:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@fedora ~]# df -h
      Filesystem            Size  Used Avail Use% Mounted on
      /dev/mapper/VolGroup00-LogVol00
                 47G   15G   29G  34% /
      /dev/sda1             190M  163M   18M  91% /boot
      tmpfs                 502M   80K  502M   1% /dev/shm
      &lt;/code&gt;&lt;/pre&gt;
    </content>
  </entry>
  <entry>
    <title>Aggregate statistics gem released!</title>
    <link href="http://joseph.ruscio.org/blog/2009/09/13/aggregate-statistics-gem-released" />
    <updated>2009-09-13T00:00:00-07:00</updated>
    <id>http://joseph.ruscio.org/blog/2009/09/13/aggregate-statistics-gem-released</id>
    <content type="html">
      &lt;p&gt;Finally got around to putting together a decent README (and a few last bugfixes)
      for my Aggregate statistics gem. With that out of the way, its ready for
      public consumption. Aggregate is an easy-to-use ruby implementation of a
      statistics aggregator that includes configurable histogram (binary or linear)
      support. Stats are aggregated in a &lt;em&gt;streaming&lt;/em&gt; fashion that doesn&#8217;t record/store
      any of the individual sample values, enabling statistics tracking across a
      practically unlimited number of samples without any impact on your application&#8217;s
      performance or memory footprint.&lt;/p&gt;
      
      &lt;p&gt;Perhaps more importantly than the basic aggregate statistics (e.g. mean,
      standard deviation, max, min, etc) Aggregate also maintains a histogram of the
      sample distribution. For anything other than normally distributed data averages
      are insufficient at best and often downright misleading. 37Signals recently
      posted a
      &lt;a href=&quot;http://37signals.com/svn/posts/1836-the-problem-with-averages&quot;&gt;terse but effective explanation&lt;/a&gt;
      on the importance of histograms. Here&#8217;s a second more
      &lt;a href=&quot;http://www.zedshaw.com/essays/programmer_stats.html&quot;&gt;detailed posting&lt;/a&gt; by Zed
      Shaw authored in his inimitable and entertaining style.&lt;/p&gt;
      
      &lt;p&gt;Full source code and a detailed README can be found on the
      &lt;a href=&quot;http://github.com/josephruscio/aggregate&quot;&gt;project homepage&lt;/a&gt; and the gem is
      available on &lt;a href=&quot;http://www.gemcutter.org/gems/aggregate&quot;&gt;gemcutter.org&lt;/a&gt;.
      But just to whet your appetite, here&#8217;s a quick example:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;require 'rubygems'
      require 'aggregate'
      
      # Create an Aggregate instance
      binary_aggregate = Aggregate.new
      linear_aggregate = Aggregate.new(0, 65536, 8192)
      
      65536.times do
        x = rand(65536)
        binary_aggregate &gt;&gt; x
        linear_aggregate &gt;&gt; x
      end
      
      puts &quot;\n** Binary Histogram **\n%s&quot; % [binary_aggregate.to_s]
      puts &quot;\n** Linear Histogram **\n%s&quot; % [linear_aggregate.to_s]
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;And the resulting output:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;macbook:aggregate(master) jruscio$ ruby ./aggregate_example.rb 
      
      ** Binary Histogram **
      value |------------------------------------------------------------------| count
          1 |                                                                  |     1
          2 |                                                                  |     1
          4 |                                                                  |     2
          8 |                                                                  |    10
         16 |                                                                  |    24
         32 |                                                                  |    33
         64 |                                                                  |    71
        128 |                                                                  |   121
        256 |                                                                  |   258
        512 |@                                                                 |   497
       1024 |@@                                                                |  1061
       2048 |@@@@                                                              |  2004
       4096 |@@@@@@@@                                                          |  4186
       8192 |@@@@@@@@@@@@@@@@                                                  |  8257
      16384 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@                                 | 16345
      32768 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| 32665
        ~
      Total |------------------------------------------------------------------| 65536
      
      ** Linear Histogram **
      value |------------------------------------------------------------------| count
          0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |  8269
       8192 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |  8257
      16384 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |  8302
      24576 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@   |  8043
      32768 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |  8273
      40960 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@   |  8052
      49152 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|  8315
      57344 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@   |  8025
      Total |------------------------------------------------------------------| 65536
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Feedback welcome and enjoy!&lt;/p&gt;
    </content>
  </entry>
  <entry>
    <title>Installing git on CentOS 5.2</title>
    <link href="http://joseph.ruscio.org/blog/2009/07/29/installing-git-on-centos-5-2" />
    <updated>2009-07-29T00:00:00-07:00</updated>
    <id>http://joseph.ruscio.org/blog/2009/07/29/installing-git-on-centos-5-2</id>
    <content type="html">
      &lt;p&gt;&lt;a href=&quot;http://www.centos.org/&quot;&gt;CentOS&lt;/a&gt; for all intents and purposes is a community
      supported clone of Red Hat Enterprise Linux (RHEL). If you want to run RHEL
      without paying for support, CentOS is as good as it gets. One of the hallmarks
      of Enterprise Linux distributions is their rock-steady stability i.e. they only
      run really boring, old software. This risk-aversion has a downside in restricted
      access to the latest and greatest packages and tools. Case in point, since Red
      Hat released RHEL 5.0 in March 2007, Git has really matured from its origins as
      a custom kernel development tool into a version control world-beater. Unfortunately
      the base CentOS 5.2 repositories don&#8217;t appear to contain rpms for git. Or at
      least the CentOS 5.2 AMI I&#8217;m using (ami-e300e68a) doesn&#8217;t:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;-bash-3.2# yum install git
      Loading &quot;fastestmirror&quot; plugin
      Loading mirror speeds from cached hostfile
      ... [snip] ...
      No package git available.
      Nothing to do
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Not to fear, &lt;a href=&quot;https://rpmrepo.org/RPMforge&quot; title=&quot;&quot;&gt;RPMforge&lt;/a&gt; maintains an RPM
      repository for CentOS full of goodies, including git (1.5.x as of this writing).
      The CentOS Wiki provides
      &lt;a href=&quot;http://wiki.centos.org/AdditionalResources/Repositories/RPMForge?action=show&amp;amp;redirect=Repositories%2FRPMForge#head-b06dd43af4eb366c28879a551701b1b5e4aefccd&quot; title=&quot;&quot;&gt;detailed instructions&lt;/a&gt; on configuring RPMforge access, but here&#8217;s the rapid-fire how-to.&lt;/p&gt;
      
      &lt;p&gt;Install &lt;em&gt;priorities&lt;/em&gt; to avoid conflicts between packages in RPMforge and the
      Base CentOS repositories:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;-bash-3.2# yum install yum-priorities
      ... [snip] ...
      Installed: yum-priorities.noarch 0:1.1.16-13.el5.centos
      Complete!
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;To configure priorities first edit &lt;em&gt;/etc/yum.repos.d/CentOS-Base.repo&lt;/em&gt; and add
      a &lt;em&gt;priority=N&lt;/em&gt; line to each entry. For [base], [updates], [addons], and
      [extras] set &lt;em&gt;N&lt;/em&gt; to 1. For [centosplus] set &lt;em&gt;N&lt;/em&gt; to 2. Also edit
      &lt;em&gt;/etc/yum.repos.d/rpmforge.repo&lt;/em&gt; and for [rpmforge] set &lt;em&gt;N&lt;/em&gt; &gt; 2.
      (I chose 4 because its the next power of 2, and that&#8217;s how I roll).&lt;/p&gt;
      
      &lt;p&gt;Verify our access to RPMforge with a check-update:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;-bash-3.2# yum check-update |grep forge
      * rpmforge: ftp-stud.fht-esslingen.de
      ... [snip] ...
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Finally install git:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;-bash-3.2# yum install git
      ... [snip] ...
      Complete!
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Enjoy the tasty DVCS goodness!&lt;/p&gt;
    </content>
  </entry>
  <entry>
    <title>KVM on Fedora 11 QuickStart Guide</title>
    <link href="http://joseph.ruscio.org/blog/2009/07/28/kvm-on-fedora-11-quickstart-guide" />
    <updated>2009-07-28T00:00:00-07:00</updated>
    <id>http://joseph.ruscio.org/blog/2009/07/28/kvm-on-fedora-11-quickstart-guide</id>
    <content type="html">
      &lt;p&gt;With the upcoming release of RHEL 5.4, Red Hat officially enters the
      virtualization fray with a KVM-based solution. Although KVM is exciting for
      several reasons, e.g. it&#8217;s free, fully supported by the Linux kernel community,
      doesn&#8217;t require any special hardware past the increasingly common Intel-VT or
      AMD-V extensions, etc, its also represents the next stage of virtualization evolution.&lt;/p&gt;
      
      &lt;p&gt;I&#8217;ll expound in a future post about how and why (admittedly with the benefit of
      hindsight) KVM represents a far more elegant model for virtualization, but
      after the jump today you&#8217;ll find a minimal set of instructions to rapidly get a
      Debian Lenny virtual guest up and running on a fresh out-of-the-box Fedora 11
      server installation.&lt;/p&gt;
      
      &lt;p&gt;Lengthy instructions on Fedora virtualization suitable for KVM and/or Xen can
      be found at
      &lt;a href=&quot;http://fedoraproject.org/wiki/Virtualization_Quick_Start#Using_virtualization_on_fedora&quot;&gt;the Fedora wiki&lt;/a&gt;
      The following lists the minimal set of steps to get a guest running focusing
      soley on KVM to the exclusion of Xen.&lt;/p&gt;
      
      &lt;p&gt;Our setup comprises a freshly installed remote Fedora 11 server serving as the
      KVM target (bash prompt &lt;em&gt;server&lt;/em&gt;) and a local Fedora 11 laptop (bash prompt &lt;em&gt;laptop&lt;/em&gt;).&lt;/p&gt;
      
      &lt;p&gt;First we&#8217;ll shell into the remote server to install and configure all the
      necessary KVM software. Verify we&#8217;re using Fedora 11 with a Intel-VT or AMD-V
      processor:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@server ~]# uname -a &amp;&amp; egrep '(vmx|svm)' /proc/cpuinfo | wc -l
      Linux server 2.6.29.4-167.fc11.x86_64 #1 SMP Wed May 27 17:27:08 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
      4
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;This server is running the modified 2.6.29 Linux kernel that comes stock with
      Fedora Core 11. It possesses 4 processing cores enabled with Intel-VT. Installing
      the basic tools required for KVM is our first task, luckily Fedora provides a
      convenient meta-package:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@server ~]# yum groupinstall 'Virtualization'
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Alongside kvm/qemu, the Virtualization group also includes the
      &lt;a href=&quot;http://libvirt.org/&quot;&gt;libvirt&lt;/a&gt; toolset. Libvirt provides a common abstraction
      across several types of hypervisors. This abstraction includes simplified
      installation and remote graphical console access a la $VMW&#8217;s VirtualCenter. To
      access these features we must start the libvirtd daemon.&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@server ~]# /etc/init.d/libvirtd start
      Starting libvirtd daemon:                                  [  OK  ]
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Using the &lt;em&gt;virsh&lt;/em&gt; command line tool, verify the currently guestless system is
      running and ready to accept commands:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@server ~]# virsh -c qemu:///system list
      Id Name                 State
      ----------------------------------
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Now we&#8217;re ready to install our guest. We&#8217;ll need to download an ISO image to
      install our virtual machine. To keep things simple we&#8217;ll go with Debian Stable
      aka &lt;em&gt;Lenny&lt;/em&gt; (as of this writing). We&#8217;ll create a top-level directory to store
      both the ISO image and the virtual machine disk image:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;[root@server ~]# mkdir /kvm &amp;&amp; cd /kvm
      [root@server kvm]# wget http://cdimage.debian.org/debian-cd/5.0.2/amd64/iso-cd/debian-502-amd64-netinst.iso
      ...
      [snip]
      ...
      Saving to: `debian-502-amd64-netinst.iso.1'
      100%[==================================================================================================&gt;] 137,713,664 1.49M/s in 93s
      2009-07-27 19:10:24 (1.56 MB/s) - `debian-502-amd64-netinst.iso' saved [137713664/137713664]
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;Now armed with the KVM/libvirt tools and an ISO image for installation, we issue
      a single &lt;em&gt;virt-install&lt;/em&gt; command to a) create a backing disk image and b) boot
      a newly created virtual guest off of the ISO:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;root@server kvm]# virt-install --connect qemu:///system -n lenny0 \
      &gt; -r 512 --disk path=/kvm/lenny0.qcow2,size=16 \
      &gt; -c /kvm/debian-502-amd64-netinst.iso --noautoconsole --os-type linux \
      &gt; --os-variant debianlenny --accelerate --hvm
      
      Starting install...
      Creating storage file...  |  16 GB     00:00
      Creating domain...        |    0 B     00:01
      Domain installation still in progress. You can reconnect to the console to complete the installation process.
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;The &#8220;domain&#8221; referred to in virt-install&#8217;s output is our new guest. We&#8217;ve
      created a single-core virtual server named lenny0 with 512 MB of RAM and a 16GB
      virtual disk housed on server at /kvm/lenny0.qcow2. Furthermore, lenny0 has been
      booted off of our Debian net-install ISO image.&lt;/p&gt;
      
      &lt;p&gt;Now we can use the graphical &lt;em&gt;virt-manager&lt;/em&gt; tool locally from our laptop to
      connect to the virtual server&#8217;s graphical console (basically VNC) and kick off
      the Debian net-install process. In theory we should be able to run virt-manager
      locally on our laptop and connect over ssh to the libvirtd daemon on the remote
      server. In practice it&#8217;s still pretty shaky and has some onerous requirements.
      Notably that there&#8217;s no progress indicator informing how much longer remains
      before the connection initializes (and it takes a long time). So if you want
      to try it out, &lt;a href=&quot;http://virt-manager.org/page/RemoteSSH&quot; title=&quot;&quot;&gt;good luck&lt;/a&gt;, YMMV
      (hopefully someone at $RHAT puts some polish on this aspect soon).&lt;/p&gt;
      
      &lt;p&gt;I&#8217;ve personally chosen to avoid the whole debacle and just run virt-manager on
      the remote server and use X Forwarding over ssh to interact with it from the
      laptop. It&#8217;s slow, but sufficient for Debian&#8217;s curses-based installer
      (I actually did this across the continental US).&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;fedora:lm jruscio$ ssh -X root@server 'virt-manager' &amp;amp;
      [1] 20038
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;The virt-manager tool should show a single VM, lenny0, running on server.
      Double-clicking on the lenny0 entry brings up the VNC console, that should show
      the Debian net-install splash screen. Start the installation and for the most
      part select all the defaults (I removed &lt;em&gt;Desktop&lt;/em&gt; from the additional packages).&lt;/p&gt;
      
      &lt;p&gt;After the installer finishes it leaves the machine in a halted state. Boot the
      machine with the &lt;em&gt;Run&lt;/em&gt; button above the VNC console and log in as root. As a
      final step install ssh and use ifconfig to determine the IP address DHCP granted
      the virtual server:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;lenny0:~# apt-get install ssh
      [snip]
      lenny0:~# ifconfig eth0 |grep inet
            inet addr:192.168.122.99  Bcast:192.168.122.255  Mask:255.255.255.0
            inet6 addr: fe80::5652:ff:fe46:6f93/64 Scope:Link
      &lt;/code&gt;&lt;/pre&gt;
      
      &lt;p&gt;That was pretty easy and we&#8217;re now able to shell into our virtual guest from
      the physical server hosting it!:&lt;/p&gt;
      
      &lt;pre&gt;&lt;code&gt;server:~ jruscio$ ssh 192.168.122.99
      jruscio@192.168.122.99's password:
      Linux lenny0 2.6.26-2-amd64 #1 SMP Sun Jun 21 04:47:08 UTC 2009 x86_64
      &lt;/code&gt;&lt;/pre&gt;
    </content>
  </entry>
  <entry>
    <title>Securing your online identity</title>
    <link href="http://joseph.ruscio.org/blog/2009/07/16/securing-your-online-identity" />
    <updated>2009-07-16T00:00:00-07:00</updated>
    <id>http://joseph.ruscio.org/blog/2009/07/16/securing-your-online-identity</id>
    <content type="html">
      &lt;p&gt;This week a
      &lt;a href=&quot;http://www.mercurynews.com/topstories/ci_12844562&quot;&gt;widely publicized security breach&lt;/a&gt;
      occurred at media darling Twitter. The cracker made off with a slew of internal
      company documents, most of which are now easily located at your technology news
      site of choice. It&#8217;s somewhat of a tempest in a teapot, given that most of the
      secret documents reveal such salacious details such as Twitter&#8217;s desire to make
      money and grow. Shocking, I know.&lt;/p&gt;
      
      &lt;p&gt;What should should concern the average net denizen (you, me, etc), is the
      attack vector used to compromise Twitter. In a similar fashion to what
      &lt;a href=&quot;http://www.wired.com/threatlevel/2008/09/palin-e-mail-ha/&quot;&gt;befell Sarah Palin&lt;/a&gt;
      during the last election, the cracker managed to compromise a personal
      web-based email account of a handful of Twitter employees or their family (Gmail
      in this case, Yahoo for Sarah Palin). After gaining access to Gmail accounts,
      the cracker was able to both access other online accounts, any attached documents,
      or Google Docs repositories.&lt;/p&gt;
      
      &lt;p&gt;You see, almost any website you log into will gladly email you everything you
      need to access your account through a simple click of the &#8220;Forgot my password&#8221;
      button. So even assuming you have a unique, strong password set on every account
      (and lets be honest, you don&#8217;t), once a cracker gains access to your favorite
      email account, its game over.&lt;/p&gt;
      
      &lt;p&gt;And what industrial strength security mechanism do email providers use to
      protect your online achilles heel? Some simple question that&#8217;s ridiculously
      susceptible to attacks. (What&#8217;s your mother&#8217;s maiden name? What town did you go
      to high school in? What kind of car do you drive?) So while IANASE (I Am Not A
      Security Expert), and you need to take responsibility for securing your
      identity online, there are a few simple things you can do to drastically
      increase your safety.&lt;/p&gt;
      
      &lt;p&gt;Even if you&#8217;re not going to use a unique, random password for each account
      (and you should), at least secure your online email accounts or any Single
      Sign-on (e.g. OpenID) with a unique, random password. Furthermore, if you have
      the opportunity to select your own security question (Gmail at least permits
      this), you should use a question along the lines of &#8220;What is the answer to my
      security question?&#8221; and supply as that answer a highly-randomized, secure
      password that you record and store somewhere safe. FYI, a post-it note affixed
      to your monitor is not such a safe location. Alternatively you could just
      answer one of the silly questions with your random password.&lt;/p&gt;
      
      &lt;p&gt;(For those looking for more ubiquitous protection,
      &lt;a href=&quot;http://agilewebsolutions.com/products/a/1Password&quot;&gt;1Password&lt;/a&gt; is a highly-touted,
      secure password manager that seamlessly integrates with your web-browser to
      maintain a full set of unique, bulletproof passwords.)&lt;/p&gt;
    </content>
  </entry>
  <entry>
    <title>printf(&#8220;Hello World!\n&#8221;);</title>
    <link href="http://joseph.ruscio.org/blog/2009/07/15/hello-world" />
    <updated>2009-07-15T00:00:00-07:00</updated>
    <id>http://joseph.ruscio.org/blog/2009/07/15/hello-world</id>
    <content type="html">
      &lt;p&gt;Welcome to my blog. I&#8217;m only about 10 years behind the times, but better late
      than never I suppose. In my defense, the need for a place for musings and
      information too long for Twitter (&lt;a href=&quot;http://twitter.com/josephruscio&quot;&gt;@josephruscio&lt;/a&gt;)
      served as the main motivator.
      Look in this space for intermittent postings on software development, computer
      science, startupping, marathon training, food/drink, or whatever other shiny
      object has my attention.&lt;/p&gt;
    </content>
  </entry>
</feed>