Performance Tuning EXT4 in Ubuntu for an HDD

iozone_baselineDisk I/O performance is one of the most common performance bottlenecks, particularly on midrange and/or slightly older hardware running traditional platter-based hard disk drives. Ubuntu 12.04 and up is generally very good for getting maximum performance out of this type of hardware out of the box but there are a couple of tricks and tweaks which can yield substantial performance gains if implemented.

EXT4 Tuning

The first step is to make a couple of adjustments to any EXT4 partitions. To begin with, run the following command on each partition (XY replaced by partition identifier such as /dev/sda1):

sudo tune2fs -o journal_data_writeback /dev/sdXY

Then run the following to edit your fstab file:

sudo nano /etc/fstab

Identify any mount points with EXT4 partitions which you have modified with the tune2fs command like so:

UUID=95441b58-45a7-4e84-85f7-7859d51d22fb    /   ext4     errors=remount-ro    0    1

…and add the following into the Options column:

noatime,data=writeback, that each of the lines in /etc/fstab which mounts an EXT4 like so:

UUID=95441b58-45a7-4e84-85f7-7859d51d22fb    /   ext4     noatime,data=writeback,errors=remount-ro    0    1

Hit CTRL+O to save, CTRL+X to exit and reboot


Ureadahead is the default readahead application for Ubuntu and it works very well without requiring any specific configuration or user intervention. E4rat, on the other hand, requires a bit of work to get it functioning but provides a number of advantages including the following:

  • default run time for the analysis phase is 120 seconds for E4rat while ureadahead only tracks for 60 seconds by default. This means that E4rat accelerates the login process as well as the boot process.
  • E4rat reorganized the hard drive based on it’s analysis which cuts down on access time latency

Keep in mind that E4rat is built with HDDs in mind and will not help an SSD in any way. If you have an SSD, stick to ureadahead.

To begin with, remove ureadahead:

sudo apt-get remove ureadahead

Next, download the appropriate DEB from Sourceforge at and install using the Software Center (double-click) or from the terminal:

sudo dpkg -i e4rat_0.2.3_amd64.deb

Now we need to run the e4rat collector by rebooting the computer, doing a one-time edit of the grub menu and adding the collector. To do this, reboot and hold down on the shift key starting at the BIOS screen to get a grub menu. Hit the e key on the first line, scroll down to the line that starts with linux /boot/vmlinuz… (usually the second from the bottom), cursor to the end of the line and add the text init=/sbin/e4rat-collect. Your line should look something like this when you are done:

linux /boot/vmlinuz-3.8.0-30-generic root=UUID-224049dd-f84e-41cd-623e56d497eo ro quiet splash $vt_handoff init=/sbin/e4rat-collect

On some systems I have found that the init=/sbin/e4rat-collect (or any other init line for that matter) needs to come before the $vt_handoff part. Once your line is set, hit the F10 key, boot up and log in. Once you log in, make sure to execute any programs that you might want to run immediately after startup such as an email client, web browser or chat client as the collector will record all activity in the first 120 seconds of startup. e4rat-collect will make a log of all files that are loaded in a log located at /var/lib/e4rat/startup.log. To confirm that the collector worked properly, run the following command:

tail /var/lib/e4rat/startup.log

The output should show the last ten files that the collector observed the system accessing before the 120 seconds were up. Now reboot the system into single user mode by rebooting, holding the shift key at bootup to get the grub menu and pressing e on the first option. Key down to the line that starts with linux /boot/vmlinuz… like before but this time add the word single to the end (or before the $vt_handoff if necessary) and hitting F10. If your system ends up at a blank screen with a blinking cursor in the top left, hit CTRL+ALT+F7 to get a terminal. Once at a root single user command prompt, enter the following command to reallocate the necessary files:

/sbin/e4rat-realloc /var/lib/e4rat/startup.log

Once the reallocation process is complete, reboot the system and log back in. The final step is to set the pre-allocation binary to run on startup every time. To do this, pull up a terminal enter the following:

sudo nano /etc/default/grub

Find the GRUB_CMDLINE_LINUX_DEFAULT option and add init=/sbin/e4rat-prealloc inside the quotes after “quiet spash” so the entire line looks something like this:

GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash init=/sbin/e4rat-preload”

Hit CTRL+SHIFT+O to write out and CTRL+SHIFT+X to exit, then type the following command to rebuild Grub:

sudo update-grub

Reboot your system and hang on to your hat!


File System Tuning

The following is a comparison of results before and after tuning taken using iozone. This test was limited to a single run on a 100MB file with a 4k record size and not done under the most scientifically stringent conditions so they should be considered a rough estimation.


Getting Synapse working on Mint 17 MATE (and probably other Ubuntu 14.04 distros as well)

Synapse is my absolute favorite keyboard launcher but for some reason, it is not included in Ubuntu 14.04 and I have not found any decent directions on how to get it working. I have since switched from Ubuntu to Mint 17 MATE and devised a way to get it functional. I’m not sure how much if any of these directions apply to Cinnamon or Ubuntu itself but I’m sure that it will at least point you in the right direction.

  1. Install the Synapse PPA: sudo add-apt-repository ppa:synapse-core/ppa && sudo apt-get -y update
  2. Install Synapse and Zeitgeist. For some reason the dependency resolution for Zeitgeist doesn’t work:  sudo apt-get install synapse zeitgeist
  3. Get compositing working. In MATE, this is done by going to Menu – Preferences – Desktop Settings – Windows and tick the Use compositing box. If you don’t do this (or whatever it takes to get compositing working on your distro) Synapse will look boxy and messed up.



Tasque + Dropbox = centralized task management for Linux and Windows

Wunderlist has always been the go-to application for cross-platform synchronized task management. Unfortunately, it’s a heavy system and version 2 withdrew support for Linux, apparently never to return. Tasque is a great fantastic simple task manager which runs well on both Linux and Windows but it lacks any sort of synchronization capabilities. Technically it is supposed to support Remember The Milk as a backend but it does not work well on Linux and doesn’t seem to work at all on Windows.

Fortunately, the local storage option for Tasque is simply an SQLite database and a config file which are stored in the local users profile which can easily be redirected to a synced folder such as Dropbox. For this setup, we are going to use hard links and junctions rather than actually syncing the file using something like rsync. The sync method is probably a bit safer and more accurate, particularly if there is a chance that the database might be open on multiple systems at the same time, but for the sake of simplicity, we are going to skip this.

For starters, choose one installation of Tasque that has your tasks in it that you want to keep and back that installation up. The folder is located at ~/.config/tasque on Linux and %APPDATA%\tasque on Windows. If you don’t have tasks that you want to save, you can skip this. If you have tasks on multiple systems that need to be saved, see the last section regarding editing the database.

For either system, the first step is to create a folder called tasque in your Dropbox folder. The name is case sensitive and it is probably a good idea to bury it at least a little bit so that it doesn’t accidentally get deleted.

For Windows, open Windows Explorer, enter %APPDATA% in the address bar and then delete the tasque folder.  Then open a command prompt and enter:

mklink -j tasque path_to_tasque_folder_in_dropbox

In Linux, enter:

rm -rf ~/.config/tasque
ln -h ~/Dropbox/tasque ~/.config/tasque

Since this setup is directly accessing the same sqlite file, it’s probably a good idea to close Tasque after use and don’t let multiple instances on different machines try to access the file at the same time. There is undoubtedly a cleaner way to do this which involves rsync and cron but in the interests of brevity, this solution works.

Bonus: Edit the categories

Tasque doesn’t offer a way to change the categories from the defaults which it ships with which is kindof weak. Fortunately, adding, removing and editing these categories is just a database edit away. Install the open source SQLite Database Browser from here or use apt-get to install sqlitebrowser on Ubuntu. Spin it up, open the file sqlitedatabase.db that should now be appearing in Dropbox for your, switch to the Browse tab and edit the Categories table. Save your work when complete and you are ready to go!


Shrew Soft VPN on Ubuntu

The Shrew Soft vpn client is far and away the best IPSEC vpn client available and it is fantastic for creating mobile vpn connections to firewalls like pfSense. Unfortunately, the version in the 12.04 repository is years out of date and uses qt3 which has also been superseded at this point by qt4. To get a modern, fully functional version of the client working on Ubuntu 12.04 (also tested on 13.04), follow these steps:

  1. Download the client from
  2. Open a terminal window and enter the folllowing to install dependancies:
    sudo apt-get install cmake libqt4-core libqt4-dev libqt4-gui libedit-dev libssl-dev checkinstall flex bison
  3. Then enter the following to decompress it:
    cd ~/Downloads && tar zxvf ike-2.2.1-release.tgz  && cd ike
  4. Enter the following to prep the installation:
  5. And enter the following to compile, create a .deb and install:
    sudo checkinstall -y
  6. Now download this archive and extract it: ike_additions.tar
  7. Install the iked daemon with these commands:
    cd ~/Downloads && sudo cp iked /etc/init.d && sudo chmod +x /etc/init.d/iked && sudo update-rc.d iked defaults
  8. Copy iked.conf from it’s included sample file
    sudo cp /etc/iked.conf.sample /etc/iked.conf
  9. Start the iked service
    sudo service iked start
  10. Finally, copy the .desktop file included in the iked_additions.tar.gz archive and copy it to /usr/share/applications for a menu item.
    sudo cp Shrew\ Soft\ VPN.desktop /usr/share/applications/

MySQL Backup Cron Script

Do you have a MySQL server that needs a basic backup mechanism with compression and retention? Try this script on for size and see if it fits your needs.

  • Create the backup script:
  • Add the following in to the script:
# Variables, adjust as necessary
date=$(date +%Y-%m-%d)
ret="5" # retention in number of days
pass="password" # mysql root password
# Workin' section
find $dest -type f -mtime +$ret -exec rm {} ;
mysqldump -uroot -p$pass --all-databases |gzip -9 > $dest/backup_db_$date.sql.gz
exit 0
  • Adjust any of the variables for your own environment, particularly the $pass variable
  • Make it executable, chown it to root and lock down the permissions because it has your MySQL root password stored in plain text:
chmod +x
chmod root:root
chmod 600
  • Move the script to cron.daily so it will run once a day:
mv /etc/cron.daily
  • Make sure that /var/backup (or whatever destination you specified in the script variables) exists and only root can access is since it will be storing all of your databases:
mkdir /var/backup
chown root:root /var/backup
chmod 600 /var/backup

This backup method doesn’t support error handling, notification, logging or differential backups so it probably isn’t appropriate a production server unless it is low-priority or there is also another backup system in place.


PS3 Media Server Headless on Ubuntu 10.04 Server

I decided to get the PS3 Media Server running on my Ubuntu 10.04 server and found the documentation available online to be fragmented and contradictory in some cases, so once I got my system up and running, I figured I would document what I did for others. All of the extra repositories that I added support most if not all modern versions of Ubuntu so I am pretty sure that this process will work for anything up to 11.11.

For this setup, I wanted to use repositories as much as possible rather than compiling so that everything can stay up to date easily. That said, I didn’t want to use the Passaid repository for PS3MS because I have seen it go down frequently and it offers up an outdated version.

PS3MS on Linux uses native versions of mplayer, mencoder, ffmpeg and mediainfo along with requiring Java jre rather than packaging them with the installer. We need to install these dependencies ahead of time using the newest possible versions.


sudo add-apt-repository ppa:shiki/mediainfo


sudo add-apt-repository ppa:jon-severinsson/ffmpeg

Medibuntu for a newer version of mplayer and mencoder

sudo wget --output-document=/etc/apt/sources.list.d/medibuntu.list$(lsb_release -cs).list && sudo apt-get --quiet update && sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring && sudo apt-get --quiet update

Install the dependancies

sudo apt-get install mplayer mencoder mediainfo ffmpeg openjdk-6-jre-headless

Download the Linux version of PS3MS from using wget or downloading to your desktop and running it over to the server using scp.  Once you have PS3MS, extract it to /opt and rename it:

sudo tar zxvf pms-generic-linux-unix-1.50.0.tgz -C /opt/

sudo mv /opt/pms-linux-1.50.0/ /opt/pms

Finally, get the init script from here: Unzip it and scp it over to /etc/init.d on your server. Then

sudo nano /etc/init.d/PS3MediaServer

and put your unprivileged user and group for PMS_USER and PMS_GROUP. Then make it executable with

sudo chmod +x /etc/init.d/PS3MediaServer

Finally, install the service so that it starts and stops with the server

sudo update-rc.d PS3MediaServer defaults 90

Now the somewhat tricky part: getting the configuration file set up. The conf file specified in the init script is /opt/pms/PMS.conf but it does not exist by default and needs to be automatically created by PS3MS because the first line is a UUID. First try simply running PS3MS from the command line:


and then stopping the process with CTRL+C. If that did not create a PMS.conf file, try

/etc/init.d/PS3MediaServer start

/etc/init.d/PS3MediaServer stop

The file should now contain a single line with a UUID; add the following lines to get everything working:

thumbnails = true
thumbnail_seek_pos = 300
mencoder_ass = true
folders = /mnt/BigDiskWithMusic,/mnt/BigDiskWithVideos
hide_videosettings = true
hide_enginenames = true
audiochannels = 2

Obviously you want to change the folders line to reflect the actual location of your media. Do NOT leave a space between folders, PS3MS will not like it and generate an error.

If you are feeling adventurous, edit /opt/pms/WEB.conf and set it up with your favorite podcasts, RSS feeds, radio and video streams.




Create and deploy workstation images

Want to create and deploy drive images over your network but don’t feel like getting jacked for proprietary software? Not surprisingly, there is an easy way to do it with open source software, but it is slightly more involved.  This how-to is written with the following assumptions that should be checked before starting:

  • The workstation and the server are on the same network.
  • Both are using a Windows operating system such as 2000, XP, 2003,Vista or 7
  • Neither device has any firewall or security impeding communications between the two.
  • The server has enough drive space to hold the image.  If you do everything right, the size of the image will be about 40-50% of the space used on the storage device in the workstation being imaged.

Getting Started

Download and install Filezilla FTP Server from the Filezilla website  Create a folder to store the images and open the Filezilla Server interface. Go to Edit – Users and click on the Add button. Create a user named install, assign a password and add the folder that you created as the home shared folder.

Workstation Prep

On the workstation that you are going to image, make sure that all of the software is up to date and properly configured. It is a very good idea to remove the workstation from your Windows domain or you will risk problems with duplicate hostnames and SIDs.  When everything is ready to go, defrag the hard drive and download this app to your root drive:  Nullfile will write a zero in every sector on the drive where it is located which is not marked as Deleted.  If you do not do this, g4u will see sectors on the hard drive with deleted data and copy them to your image, dramatically increasing the size of your image. Nullfile works by creating a temporary file with all zeros that fills up all space on the storage device and then deleting the file. If you run Nullfile and kill it before it completes, be sure to delete the temporary file that it created.

Image Creation

Download the g4u ISO here:, burn the ISO to a CD and boot the workstation to it. When you get to the main menu, create the image by using the following command:

uploaddisk server-IP name.gz

Once complete, you should see a file in the directory you created on your FTP server called name.gz.

Image Deployment

Important deployment notes:

  • This process WILL overwrite everything on your hard drive, it will do it in a way that cannot be undone and it will do it without prompting.
  • This method will not make any changes to the drivers on the image, so the image will probably only work on the make and model of workstation that you used to create it. If you want to create an image that could work on a wider range of hardware, you could theoretically install other drivers before creating the image, but I have not tested this.

To deploy your image to another workstation, boot the workstation to the g4u disc and enter the following command at the main menu:

slurpdisk server-IP name.gz

Once the process is complete, you can reboot the workstation with the reboot command and boot to the hard drive. It is worth noting that imaged systems are all identical, including the hostname, so one of the first things that you should do with a newly-imaged workstation is change the hostname to something else.

Maintaining and Updating Images

To the best of my knowledge, there is no easy way to “start” an image in a virtualized environment.  In my experience, the best way to update or make changes to an image is to use Microsoft Virtual PC or VMware along with the ISO of g4u and “deploy” the image to a virtual machine. Once complete, you can make any necessary changes to the image and then re-image it back to the server.


g4u –

FileZilla –

Nvidia GEForce 8500GT on Ubuntu 9.04

A quick how-to on getting 3d graphics working in Ubuntu with an Nvidia 8500GT.  This process has been tested on both x86 and x64 distros and has also been tested with an Nvidia 8400GS.  I have also successfully used a similar process on Fedora 11 x64, but the step-by-step was slightly different. It appears that all Nvidia 8000 and 9000 series graphics cards use the same set of drivers, so I would hazard a guess that this process would work on any of these, but I don’t have the cash to find out.

  1. Make sure that all updates are installed
  2. Go to System – Administration – Hardware Drivers and check X. Make sure that the installation disc is in the optical drive, or that the CDROM drive has been disabled in Software Sources. Otherwise, the installer will just sit there at the Downloading phase without asking for the disc.
  3. Once the installation is complete, restart the computer and log back in.

At this point, 3d acceleration should be enabled and applications like Compiz can now be used. However, Xorg and/or the Nvidia driver may not be able to correctly detect your monitor which will lead to a max resolution of 800×600, as was the case with my Dell SE197FP.  To fix this, you need to find the vertical and horizontal refresh rates for your monitor and enter them manually.  This information is usually located on the manufacturers web site, in your monitor documentation if you still have it, or sometimes it is listed on a sticker on the back.  Once you have this information, open up the Nvidia Control Panel with sudo as follows:

sudo nvidia-settings

Note: Going to System – Preferences – Nvidia also opens the control panel but it does not use sudo or gksu so it cannot edit xorg.conf or make any permanant changes. Also, the built-in Detect feature in the Nvidia Control Panel ignores and values from xorg.conf so don’t bother doing this step before you install the Nvidia driver and don’t use Detect afterward because it will overwrite your work without asking.

Click on the Save settings to xorg.conf button and then click on the Preview button on the next screen. Scroll down to SubSection Monitor and replace the existing hsync and vsync values with the ones for your monitor and save it. Click the Apply button back on the Nvidia Control Panel and a bunch of new values should be available on the Resolution dropdown.  Grab the one that you want (your monitor documentation should indicate which resolution looks the best), click Apply and then save xorg.conf again.

You should now be able to enable Desktop Effects, turn them all the way up and turn on Water Effects and other fun plugins at full resolution!

LSI MegaRaid 8204XLP

I recently had a need for a SATA RAID card that worked with Linux, so I opted for the LSI MegaRaid 8204XLP based on their advertisement of Linux support including SLES 9 and 10 along with RHEL 4 and 5. Actually, I originally went with an Adaptec 1420SA which was  an unmitigated disaster, but that is the topic of a different discussion. My first step with the LSI card was CentOS 5.2 and Anaconda was able to successfully load the drivers and recognize the array, but that is where the fun stopped. On restart, initrd crashed when trying to load the kernel and hung. I contacted LSI support with the issue and they were diligent but it became immediately clear that their level 2 support techs new far less about this problem than I already did. I tested the issue with a number of other RHEL kernels including a 4.7 kernel and all had the same issue. I also discovered that the issue only occurred if the drives were plugged into the card and when unplugged, the server would boot correctly (aside from fstab errors associated with a missing volume. Eventually I gave up on Red Hat/CentOS altogether and gave it a shot with SuSe Enterprise Server 10.2 with the SLES drivers downloaded from LSI and it worked like a charm.

Overall, I was eventually able to make the 8204XLP work and there are apparently a significant number of people out there who have had success with RHEL. Once working, the performance seems quite solid although I have not benchmarked it. However, the Linux support from LSI is less than inspirational, both in terms of drivers and technical support and I would recommend caution when buying this card for use with Linux.