Improving Apache performance

Remove modules
Apache include many modules that are enabled, and you may not be needing all of them, disable the ones you do not need. (By default in CentOS when you install Apache, over 50 modules are loaded.) This will help speed up Apache. The modules that are loading are listed in /etc/httpd/conf/httpd.conf and the line begins with the word LoadModule, as in:

LoadModule auth_basic_module modules/

For instance if you don’t use LDAP to authenticate with Apache, you can disable the authnz_ldap_module module.
If you are unsure about disabling an Apache module whose name is perhaps not self explanatory, you can take a look here for a more detailed description.

DNS tunning
Each DNS lookup takes up time, so make sure that Apache is not doing hostname lookups, you can enable this feature with the following directive ‘HostnameLookups Off’. This is normally off by default.

Don’t use htaccess if there is no need to
Use ‘AllowOverride None’, since allowing override will force Apache to look for .htaccess file, which you may not be using. This will speed up Apache, since it’s one less thing that Apache has to do before serving content.

Avoid content negotiation
When you access the root directory of a web server, Apache usually looks for an index file which is basically the ‘home-page’ of a web server. This file can have various names such as index, index.html, etc. You can specify the exact filename so that Apache does not have to look for different files. So replace ‘DirectoryIndex index’ with ‘DirectoryIndex index.cgi index.shtml index.html’

How do you improve Apache’s performance? Share your comments in the blog.


Understanding rsyslog.conf

rsyslog is the logging daemon used by CentOS and RedHat. A number of Linux applications use rsyslog to send logging output to, including the Linux kernel. Rsyslog runs as /sbin/rsyslogd and it’s configuration file is /etc/rsyslog.conf.
Rsyslog is a full replacement of syslog and is more fully featured.

rsyslog has a modular design which supports over a dozen modules, the two most common ones are specified in /etc/rsyslog.conf as:

#UDP logging
$ModLoad imudp 
$UDPServerRun 514

#TCP logging
$ModLoad imtcp
$InputTCPServerRun 514

Lines starting with ‘#’ are ignored in /etc/rsyslog.conf.
Global directives start with $ on their own line.
Templates allow you to specify the format of the logged message. By default rsyslog logs output in the standard syslog format. To change the format use the template directive as in ‘$template RFC3164fmt,”%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%”‘. This will output syslog messages in the format specified in RFC3164. The RFC3164ftm is the name given to this template, although you can call it anything else you want, what matters is the actual format in double quotes.

Rules are specified on what action to take with a selector and an action in rsyslog.conf. A selector is a combination of facility and priority.

Facility can be any of the following: auth, authpriv, cron, daemon, kern, lpr, mail, mark, news, syslog, user, uucp and local0 through local7. Facility is the subsystem that produced the log, for instance kern is the kernel produced log messages.

Priority in ascending order can be: debug, info, notice, warning, warn (same as warning), err, error (same as err), crit, alert, emerg, panic (same as emerg). Severity of the message is defined with priority.

Action is what to do with the message, for instance to output to a log file. An example of selector and action would be ‘kern.* /dev/console’ which means send all kernel messages with any priority to /dev/console.

I have only covered some of the options of rsyslog, for more information you many want to run ‘man rsyslog.conf’.

Have you done anything fancy with rsyslog or do you use the stock config? Share your comments in this blog.

Using RPM

Q. How do you view files in a given package?
A. $ rpm -q –filesbypkg

Ever wonder which package a file is part of? Try this:

/bin/rpm -qf
$ /bin/rpm -qf /usr/bin/whois

How do you extract files in an RPM without installing the RPM? The below will extract in your ‘pwd’ the files of the RPM.

rpm2cpio rpmname.rpm | cpio -dimv

How do you install Extra Packages for Enterprise Linux (or EPEL) on CentOS 6.x?

sudo wget
sudo wget
sudo rpm -Uvh remi-release-6*.rpm epel-release-6*.rpm
sudo vim /etc/yum.repos.d/remi.repo 
Enable the remi repo by changing the 0 to 1 for enabled in remi.

For further reference visit

Installing additional software in Fedora

Fedora includes the following yum repos by default:
1) Fedora
2) Fedora-updates
3) Fedora-updates-testing
However, if you need to install additional software that is missing in the above repos, I would recommend adding RPM Fusion repo, which will allow you to install packages such as ‘VLC’. To install the additional repo, and installing VLC, try the below:

$ sudo rpm -Uvh
$ sudo yum install vlc -y

How do you install additional repos in Fedora? Share your comments in this blog.

XFS vs Ext4 performance

I wanted to test XFS vs Ext4 performance, so I created two partitions /dev/mapper/vg_hv1-lv_vm1 which is xfs based and /dev/mapper/vg_hv1-lv_vm2 which is ext4 based. Both partitions are on a single RAID-1 disk.

[hv ~]$ sudo hdparm -Tt /dev/mapper/vg_hv1-lv_vm1

Timing cached reads: 24786 MB in 2.00 seconds = 12413.11 MB/sec
Timing buffered disk reads: 370 MB in 3.01 seconds = 123.01 MB/sec
[hv ~]$ sudo hdparm -Tt /dev/mapper/vg_hv1-lv_vm1

Timing cached reads: 24602 MB in 2.00 seconds = 12320.66 MB/sec
Timing buffered disk reads: 366 MB in 3.00 seconds = 121.80 MB/sec
[hv ~]$ sudo hdparm -Tt /dev/mapper/vg_hv1-lv_vm1

Timing cached reads: 24300 MB in 2.00 seconds = 12169.27 MB/sec
Timing buffered disk reads: 374 MB in 3.01 seconds = 124.37 MB/sec
[hv ~]$ sudo hdparm -Tt /dev/mapper/vg_hv1-lv_vm2

Timing cached reads: 24566 MB in 2.00 seconds = 12302.76 MB/sec
Timing buffered disk reads: 392 MB in 3.01 seconds = 130.37 MB/sec
[hv ~]$ sudo hdparm -Tt /dev/mapper/vg_hv1-lv_vm2

Timing cached reads: 24576 MB in 2.00 seconds = 12307.80 MB/sec
Timing buffered disk reads: 366 MB in 3.01 seconds = 121.42 MB/sec
[hv ~]$ sudo hdparm -Tt /dev/mapper/vg_hv1-lv_vm2

Timing cached reads: 24322 MB in 2.00 seconds = 12180.78 MB/sec
Timing buffered disk reads: 396 MB in 3.01 seconds = 131.41 MB/sec

I would expect the timing cached reads results to be very similar, since timining cached reads is a measure of processor, cache and memory. Timing cached reads basically reads from the Linux buffer cache without disk access.

The timing buffered disk read on the other hand flushes the Linux buffer caches and reads through it directly from the disk. These numbers were also very similar.

I was hoping that dd numbers would be significantly different for XFS and Ext4, but as you can see below there is minimal difference in the write operations:
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm1/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.12586 s, 119 MB/s
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm1/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.12256 s, 120 MB/s
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm1/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.15067 s, 117 MB/s
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm1/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.13103 s, 119 MB/s
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm2/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.18037 s, 114 MB/s
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm2/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.09832 s, 122 MB/s
[hv ~]$ sudo dd bs=1M count=128 if=/dev/zero of=/vm2/test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.10921 s, 121 MB/s

The fdatasync option to dd tells dd to physically write output file data before finishing, which is a more realistic approach for testing. What are your thoughts on the above results?

Qcow2 vs Raw Image Performance

I wanted to test our qcow2 versus raw image format performance, so I created two virtual machines on my hypervisor, below are the results I encountered.

Hypervisor: CentOS 6.4 with KVM
Processor:Intel Xeon E3-1230v2 3.30 GHz, 8M Cache, Turbo, Quad Core/8T (69W)
Memory:16GB Memory (4x4GB), 1600Mhz, Dual Ranked, Low Volt UDIMM (speed is CPU dependent)
RAID Controller:PERC H200 Adapter RAID Controller
Disk:2x1TB 7.2K RPM SATA 3Gbps 3.5in Cabled Hard Drive
Raw-VM: 10GB RAW disk format
Qcow2-VM:10GB Qcow2 format
Raw-VM and Qcow2-VM Filesystem type: ext4
Operating system: Raw-VM is Ubuntu 12.04 LTS and Qcow2 VM is CentOS 6.4
VM Memory and VCPU: Both VM’s have 2GB RAM and 1 VCPU of the same speed

Both VM’s are on a XFS based filesystem on the hypervisor.

I used hdparm and ran the following:
sudo hdparm -Tt

My results are as follows:


Timing cached reads: 19414 MB in 2.00 seconds = 9715.65 MB/sec
Timing buffered disk reads: 350 MB in 3.02 seconds = 116.09 MB/sec

Timing cached reads: 19428 MB in 2.00 seconds = 9722.02 MB/sec
Timing buffered disk reads: 614 MB in 3.01 seconds = 204.17 MB/sec

Timing cached reads: 19900 MB in 2.00 seconds = 9958.25 MB/sec
Timing buffered disk reads: 896 MB in 3.01 seconds = 297.31 MB/sec


Timing cached reads: 20594 MB in 2.00 seconds = 10311.02 MB/sec
Timing buffered disk reads: 396 MB in 3.02 seconds = 131.07 MB/sec

Timing cached reads: 19916 MB in 2.00 seconds = 9972.19 MB/sec
Timing buffered disk reads: 408 MB in 3.02 seconds = 134.96 MB/sec

Timing cached reads: 19386 MB in 2.00 seconds = 9704.97 MB/sec
Timing buffered disk reads: 406 MB in 3.02 seconds = 134.40 MB/sec

Based on hdparm man pages, the timing cached reads (-t) are :

This displays the speed of reading through the buffer cache to the disk without any prior caching of data. This measurement is an indication of how fast the drive can sustain sequential data reads under Linux, with‐out any filesystem overhead.

And Timing buffered disk reads (-T) are:

This displays the speed of reading directly from the Linux buffer cache without disk access. This measurement is essentially an indication of the throughput of the processor, cache, and memory of the system under test.

I was under the impression that RAW disk format is a lot faster than QCow2, however in my results above both performed similar with respect to timing cached reads, which is a measure of the disk speed. On the other hand, the timing buffered disk read  was much faster on the Ubuntu raw VM, than the CentOS Qcow2 VM, which does not make much sense, this the timing buffered disk read is a measure of  processor, cache and memory throughput and I would think it should be the same on both?

So I ran dd to check my numbers:


# dd bs=1M count=128 if=/dev/zero of=test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 2.43185 s, 55.2 MB/s

# dd bs=1M count=128 if=/dev/zero of=test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 2.17012 s, 61.8 MB/s

# dd bs=1M count=128 if=/dev/zero of=test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.93576 s, 69.3 MB/s


$ dd bs=1M count=128 if=/dev/zero of=test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.37035 s, 97.9 MB/s

$ dd bs=1M count=128 if=/dev/zero of=test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.24225 s, 108 MB/s

$ dd bs=1M count=128 if=/dev/zero of=test conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.21815 s, 110 MB/s

Looks like Raw-VM performed much better than Qcow-2 in dd write operations.

What has your experience been like? Share your comments below.

Installing KVM on CentOS 6.4

Install CentOS 6.4

CentOS installation is the same as RedHat installation. See my previous blog on installing CentOS using  a USB stick here

Install KVM

Installing KVM is fairly easy, on CentOS you can use the yum groupinstall command to get all the goodies, such as below:

sudo yum groupinstall virtualization

Start libvirt with :

sudo service libvirtd start

Create network bridge if you want to use external IP and not NAT

By default KVM creates a bridge called virbr0, for NAT access from VM’s and add’s the appropriate IPtables rules in both the filter and the NAT tables. I created another bridge for public IP guests. Below are the steps you can follow to create the bridge which I named br0. Keep in mind that you can use bridge ctl utilities, however I did this manually by editing the interface config files.

$ cat /etc/sysconfig/network-scripts/ifcfg-br0

$ cat /etc/sysconfig/network-scripts/ifcfg-em1

The above will give br0 the IP of em1. Em1 is basically the BIOS name of eth0.

Enable IP forwarding

$ cat /proc/sys/net/ipv4/ip_forward
$ grep -A1 -i forwarding /etc/sysctl.conf
# Controls IP packet forwarding
net.ipv4.ip_forward = 1

Download ISO image for Guest VM


Use virt-install or Virtual Machine Manager to create guest VM

sudo virt-install --connect qemu:///system --name \
--ram 1024 --vcpus 1 --disk path=/vm1/  \
--network=bridge:virbr0 --os-type=linux --os-variant=rhel6 \
--cdrom /vm1/iso/CentOS-6.4-x86_64-bin-DVD1.iso \
--graphics spice,password=mypassword --autostart

You can use virt-manager gui or virt-install which is command line based to install a VM. In terms of the network, I am using the default bridge virbr0 since the VM will have a private RFC 1918 IP, which in KVM defaults to 192.168.122.X/24 network. Specifying os-type and also os-variant allows KVM to optmize for that particular OS. Using the cdrom method I am able to specify the ISO image to install from. For connecting to the VM console, I am using spice, with the password specified on the command line. (Not secure, but there is a bug with KVM that does not allow spice connections if you specify a default spice password in /etc/libvirt/qemu.conf. The autostart option causes KVM to restart the domain when the host (hypervisor) restarts.

Use Spice or VNC to connect to console of VM and complete install

virt-viewer --connect qemu:///system

External Links


Apache directory access from outside of DocumentRoot

Apache 2.2 has a document root that by default is /var/www/html. If you want to share directory not in document root, one way to do it is using the <Directory> directive. For instance I wanted to share a CentOS 6.4 install DVD which I had mounted using loop back. The way to mount an ISO file via loopback is:

mount -o loop /vm1/iso/CentOS-6.4-x86_64-minimal.iso \

To share the above mentioned directory via apache, in /etc/httpd/conf.d/ create a file with a .conf extension, the filename can be anything you want, in this case, let’s say the file is called local.conf. In the file place the following:

Alias /centos64 /mnt/CentOS6.4
<Directory /mnt/CentOS6.4>
Order deny,allow
 Deny from all
 Allow from
 Allow from localhost
 Allow from
 Options +Indexes

The ‘Alias’ part lets you access the URL with instead of having to type in the full directory path.
The Deny/Allow control who can access the file and the +Indexes allows directory browsing.
Now you are ready to use the URL for network installs for CentOS or whatever else you need it for!
How do you share directories via Apache? Leave your comments below.

Installing CentOS 6.4 Using USB

One would think that installing CentOS using USB is fairly easy, however Linux often requires additional steps that go beyond ‘it just works’. I purchased a Dell PowerEdge T110 II server and decided on to run KVM on it. CentOS is built from RedHat sources so if you don’t want to pay RedHat, then CentOS is a good option.

1. Download CentOS from I used the 6.4 server image which is two DVD images, one is 4.1GB and the other is  1.4GB. For me Stanford mirror was the closest I used wget and also tried curl. After downloading for a few minutes, both would just stop downloading.  I used the verbose option to both to see if I could see some error messages, however that did not help.  Wget supports –continue option which will continue to download from where it left off if you kill wget or if the previous wget hangs, I found this option useful to download the file.

2. Next step was to create a USB bootable stick, I downloaded which allows you to create bootable USB disks. I pointed Unetbootin to the ISO using my 8GB memory stick, and within 15 minutes I had a bootable USB drive.

3. I booted my Dell server and from the boot menu picked USB, which started CentOS. When CentOS came to the disk partition page it asked me for the CentOS ISO. Of course I did not have the ISO on the same USB stick, so I removed the stick and tried to copy the ISO on it, which did not work, since the file is 4.1GB and the USB was formatted with vfat which does not support files larger than 4GB. I then reformmated the USB with ext2, ran Unetbootin, copied the ISO, and tried to boot, but my Dell server would not boot from the ext2 partition of the USB. So I went back and got a 16GB USB stick, and made 2 partitions, one vfat size 4GB and another ext2 size 8GB. On the 8GB I copied the ISO image, and the 4GB vfat I made a bootable partition using Unetbootin. Using this I was able to boot the Dell and start the install. After that you should use Ctrl-Alt-F2 to switch to a shell prompt, and then try the following: ‘umount /mnt/isodir’, followed by ‘mount -t ext2 /dev/sda2 /mnt/isodir’.  Partition 2 of my USB had the ISO and it was /dev/sda2.  Then I switched back to the installer using Ctrl-Alt-F6 and choose Retry which caused the installed to continue.

4. CentOS default partition is not ideal, since it does not separate /var or /tmp. I created my own partition with /tmp, /var, swap, /boot, /boot/efi, /, /vm1, /vm2, /home. I used LVM for all of them except for /boot and /boot/efi. Since I had set my server to be in UEFI boot mode instead of BIOS, CentOS created /boot/efi partition as well.

5. A minimal server installation is what I picked when asked during the install, since I wanted to complete the installation quickly. I had used a USB 2.0 stick, my previous USB stick was 1.0 which was slow. It would have been nice to see USB 3.0 support on the Dell.

6. Once the installation completed, I was able to login with the account I had created earlier during the install.

A few links that you may find useful are:


How was your experience with installing CentOS using USB? Do share your comments below.

Installing an Ubuntu Minecraft Server

Minecraft has a lot of servers which you can connect to and play online with others. I wanted to setup a private server for my son so that I have some control over who he gets to play.  I hope you find my experience on setting up a server useful.

  1. All the steps below assume Ubuntu 12.04 LTS, you can probably use similar versions of Ubuntu as well.
  2. The authoritative server  twiki is here, so do read that, however you will find the below instructions easier than the twiki since I have condensed them down for Ubuntu based server. If you need to install Minecraft server for different platforms use the steps here
  3. To start off look at the bare minimum server requirements here My server is running as a VM guest with 2GB RAM, and 1 VCPU. The VCPU is on a host/hypervisor that has a single Quad Core Xeon processor which is multi-threaded. I also allocated 10GB disk space for my VM. You can run the server on your laptop if it meets the minimum requirements, as long as you open up the Minecraft port through the firewall. The only problem with running it on your laptop is that the server will be down when your laptop is turned off so you may not get many friends who want to join your server. My server is hosted online with a co-location provider. I would suggest looking at Amazon EC2 if you want to get your own server here
  4. You need this package in order to run apt-add-repository later: sudo apt-get install software-properties-common -y
  5. sudo apt-get install python-software-properties -y
  6. sudo apt-add-repository ppa:webupd8team/java
  7. sudo apt-get update
  8. sudo apt-get install oracle-java7-installer -y
  9. java -version
  10. I am running my server as user ‘minecraft’ which has no login shell for security. sudo adduser –system –no-create-home –home /home/minecraft-server minecraft
  11. sudo mkdir /home/minecraft-server
  12. sudo chown -R minecraft minecraft-server/
  13. Next is to add user ‘minecraft’ to ‘games’ group, so edit the group file by running  ‘sudo vi /etc/group’ and in the games line add user ‘minecraft’ or you can use the usermod command as well
  14. sudo chgrp -R games minecraft-server/
  15. I want all files in the minecraft directory to belong to the games group so run: sudo chmod g+s minecraft-server/
  16. It is a good idea to have start and stop scripts that start/stop minecraft when the server reboots, so in the next step I created a file which allowed me to specify that. In terms of the Xms sizes, adjust the size based on how much RAM you have, I decided to give the JVM 1.5GB RAM.
  17. $cat /etc/init/minecraft-server.conf
     chdir /home/minecraft-server
     exec su -s /bin/sh -c 'exec "$0" "$@"' minecraft -- \
     /usr/bin/java -Xms1536M -Xmx1536M -jar \
     minecraft_server.jar nogui > /dev/null</pre>
     start on runlevel [2345]
     stop on runlevel [^2345]
  18. Now download the server itself : cd /home/minecraft-server; sudo wget
  19. sudo mv  minecraft_server.13w18c.jar minecraft-server.jar
  20. sudo vi and look here to see which entries to modify
  21. Test starting the server usingjava -Xms1G -Xmx1G -jar minecraft_server.jar nogui
  22. Once the server is up and running view the server commands which can be run by joining the server here
  23. sudo stop minecraft-server && sudo start minecraft-server && ps axu | grep -i mine 
  24. That’s it, you are not ready to use this server. Start a minecraft client, and type in this server name in multiplayer mode. The server name is the hostname of the server.

How has your experience been with setting up Minecraft? Share your comments below.

Installing a Minecraft Fedora 17 Linux client

I am writing my experience on setting up an Fedora minecraft client in the hopes that the reader finds it useful.  Minecraft is a hugely popular game which can be downloaded from As of the writing of this blog there are Windows, Mac and Linux versions of the game available. To setup the client follow the below instructions:

  1. Visit and create a free login account by clicking on Register at the top right
  2. Once you create an account, and login to the site, your login by the way is your email address, you can purchase the game by clicking on Store, then Buy Minecraft for this account. The price is $26.95 as of the writing of this blog
  3. Now you are ready to download the game. I have a Fedora laptop, so I download the Linux version here
  4. You will also need Java. On my Fedora box I already had Java installed, if you need Java installed try the following steps which are pretty good Fedora ships with OpenJDK which has issues with Minecraft and that is why we need Sun/Oracle Java installed.
  5. Now that you have a acount, Java installed on your desktop and also downloaded Minecraft Jar file, you are ready to start Minecraft, which you can do by running the following command ‘java -Xmx1024M -Xms512M -cp minecraft.jar net.minecraft.LauncherFrame,’ The command assumes you are in the same directory as minecraft.jar which you had downloaded above.
  6. Once you run the above command Minecraft should start and ask for a login/password, enter the information from the purchase time, and you are ready to play! For further help on how to play watch howto videos here

How has your experience been with installing a Minecraft client on Linux or other platform? Do share your comments!

%d bloggers like this: