Linux System Admins Blog

The Planet and Softlayer to merge

andrew — Thu, 05 Aug 2010 15:18:02 +0000

Looks like consolidation in the hosting space continues as Softlayer is going to be under the same ownership as “The Planet”.

For those who are not aware, GI also owns a large stake in The Planet, and I now have the pleasure of telling you that we are in discussions to merge The Planet with SoftLayer. The goal is for the transaction to be complete in the fourth quarter of this year.

I know you have a lot of questions. To be perfectly honest, we are at the beginning of this and don’t have all the answers. Teams at both companies have just begun discussing how to integrate our organizations, and their work will yield the details we don’t yet know.

Drupal Performance talk video from Drupal Con 2010

andrew — Fri, 30 Jul 2010 04:01:17 +0000

I have already mentioned that google will start penalizing your site in search if it is not fast enough. I have recently given a presentation at Drupal Con about Drupal performance and page speed. Below is a video of our session – its currently the fifth most watched session from the conference according to archive.org. Alternatively, there are several different formats there as well.

Debian FTP archive for Etch – archive.debian.org

marius — Sat, 10 Jul 2010 00:49:17 +0000

Debian Etch has been discontinued for a while now, and in an ideal world everyone has upgraded to lenny a long time ago. Still, this is not always possible and there are some systems out there that are still running etch, some for a good reason, some just because their admins are lazy . Recently we worked on a project with such a system that was still running etch, and the devs on the team told us that all is working perfect but they are no longer able to install new packages using apt tools. Hmm… let’s see.

And indeed running apt update was giving a 404 error, as the etch files are no longer in the main ftp archive (on ftp.debian.org):

apt-get update Ign http://ftp.debian.org etch Release.gpg Ign http://ftp.debian.org etch Release Ign http://ftp.debian.org etch/main Packages Ign http://ftp.debian.org etch/non-free Packages Ign http://ftp.debian.org etch/contrib Packages Err http://ftp.debian.org etch/main Packages 404 Not Found [IP: 130.89.149.226 80] Err http://ftp.debian.org etch/non-free Packages 404 Not Found [IP: 130.89.149.226 80] Err http://ftp.debian.org etch/contrib Packages 404 Not Found [IP: 130.89.149.226 80] Fetched 4B in 1s (3B/s) Reading package lists... Done W: Couldn't stat source package list http://ftp.debian.org etch/main Packages (/var/lib/apt/lists/ftp.debian.org_debian_dists_etch_main_binary-i386_Packages) - stat (2 No such file or directory) W: Couldn't stat source package list http://ftp.debian.org etch/non-free Packages (/var/lib/apt/lists/ftp.debian.org_debian_dists_etch_non-free_binary-i386_Packages) - stat (2 No such file or directory) W: Couldn't stat source package list http://ftp.debian.org etch/contrib Packages (/var/lib/apt/lists/ftp.debian.org_debian_dists_etch_contrib_binary-i386_Packages) - stat (2 No such file or directory) W: You may want to run apt-get update to correct these problems

and the apt sources line causing this error was (from /etc/apt/sources.list):
deb http://ftp.debian.org/debian/ etch main non-free contrib

We could not upgrade the machine because of internal constrains, and this was not even the scope of our project, but we needed to install some new debian packages we had to point the apt sources to a new place and this is to the archive.debian.org that continues (and will continue) to have the etch files. Basically our new apt sources became:
deb http://archive.debian.org/debian/ etch main non-free contrib
and this made it possible to complete our project and install the needed libraries. A few weeks after we finished the project we were hired for a new project to perform the upgrade to lenny, but this is a different storry.

I hope you found this post useful, in case for some reason you are still running etch and need to find a proper etch mirror to install new softwares as needed. Of course I would urge you to upgrade to lenny, or even to squeeze if possible, as etch is no longer supported, you have no longer security patches, etc.

HowTo remove a list of files

marius — Fri, 09 Jul 2010 21:33:28 +0000

Here is a quick tip on how to remove a list of files. Let’s say you have the list of files inside a file called files_to_remove. Usually I would do something like this:
LIST=`cat files_to_remove`
and then
ls -al $LIST
just to check what is in the list and if it looks good.

And finally:
rm -vf $LIST

svnadmin: can’t open file ‘svn/db/fsfs.conf’: No such file or directory

marius — Sun, 13 Jun 2010 05:37:48 +0000

While working on setting up a backup script for a subversion repository I encountred an interesting problem. I’ve done this before many times, on different repos, and haven’t seen any issues, but in this case the backup command that is using the built-in svnadmin hotcopy command was failing with this error:
svnadmin hotcopy --clean-logs /svn/repo/ /backup/repo/ svnadmin: Can't open file '/svn/repo/db/fsfs.conf': No such file or directory

Hmm… looking at the respective path I can see that the command is not lying and that file fsfs.conf is indeed not present. I could find a file fs-type but not fsfs.conf. So my only assumption was that this is an older repository created with an older svn version than the one we were running currently. Checking the existing svn version I got:
svn --version svn, version 1.6.11 (r934486) compiled Apr 20 2010, 00:24:22
and the fact that this repo was very old (~2009) made my assumption sound correct. Ok, now what? Well in this situation my first thought was to use the svnadmin upgrade command; from the manual it looked like this is what I needed to fix this issue:

“svnadmin help upgrade
upgrade: usage: svnadmin upgrade REPOS_PATH

Upgrade the repository located at REPOS_PATH to the latest supported
schema version.

This functionality is provided as a convenience for repository
administrators who wish to make use of new Subversion functionality
without having to undertake a potentially costly full repository dump
and load operation. As such, the upgrade performs only the minimum
amount of work needed to accomplish this while still maintaining the
integrity of the repository. It does not guarantee the most optimized
repository state as a dump and subsequent load would.”

After I’ve made a manual backup archive of the repo (a simple tar.gz of the repo folder) I ran the upgrade command sure this is going to fix my issue:
svnadmin upgrade /svn/repo/
and after it completed, I verified that svn was still working as expected and checked for the fsfs.conf file. But that was not created… Hmm… Let’s try the hotcopy command anyway:
svnadmin hotcopy --clean-logs /svn/repo/ /tmp/repo/ svnadmin: Can't open file '/svn/repo/db/fsfs.conf': No such file or directory
the exact same error.

Trying to understand what the fsfs.conf file contains I just created a new repository to see if it gets created. Indeed my v1.6.11 of svn created the file for a new repo, and after copying it to the location of my existing repository (as it was basically just an empty file) my issue was fixed and the hotcopy command started working. Here is the content of the file as created by my svn version, that I copied in the older repo to fix this problem:

cat fsfs.conf
### This file controls the configuration of the FSFS filesystem.

[memcached-servers]
### These options name memcached servers used to cache internal FSFS
### data.  See http://www.danga.com/memcached/ for more information on
### memcached.  To use memcached with FSFS, run one or more memcached
### servers, and specify each of them as an option like so:
# first-server = 127.0.0.1:11211
# remote-memcached = mymemcached.corp.example.com:11212
### The option name is ignored; the value is of the form HOST:PORT.
### memcached servers can be shared between multiple repositories;
### however, if you do this, you *must* ensure that repositories have
### distinct UUIDs and paths, or else cached data from one repository
### might be used by another accidentally.  Note also that memcached has
### no authentication for reads or writes, so you must ensure that your
### memcached servers are only accessible by trusted users.

[caches]
### When a cache-related error occurs, normally Subversion ignores it
### and continues, logging an error if the server is appropriately
### configured (and ignoring it with file:// access).  To make
### Subversion never ignore cache errors, uncomment this line.
# fail-stop = true

[rep-sharing]
### To conserve space, the filesystem can optionally avoid storing
### duplicate representations.  This comes at a slight cost in performace,
### as maintaining a database of shared representations can increase
### commit times.  The space savings are dependent upon the size of the
### repository, the number of objects it contains and the amount of
### duplication between them, usually a function of the branching and
### merging process.
###
### The following parameter enables rep-sharing in the repository.  It can
### be switched on and off at will, but for best space-saving results
### should be enabled consistently over the life of the repository.
# enable-rep-sharing = false

Hopefully this will help others seeing the same issue I was experiencing.

Google 500 Error

andrew — Wed, 02 Jun 2010 15:01:14 +0000

Funny, here is what you get to those who have not received this yet. I got this while on YouTube unlink accounts:

500 Internal Server Error

Sorry, something went wrong.

A team of highly trained monkeys has been dispatched to deal with this situation.

If you see them, show them this information: 9tE7ZMgxBSriv_77QysO1ltAPvPBKMi_nAq0HsNCM4reIvdRU5qCmvvn-uG9 x4XTgNjGQxteXotgBV7IeeugS_eQq17zaz5OoKdyub6gP7ilQgI3bUWbZQQ7 VX6fbW3FBjOrkkOmvC1ROJHEZkysx_o6a4Y8n_vXzFL3ymU_i87bXN8RBzuk l2JNYFAGdoNaLq_Q-bPXixNlQPoLwIAKECq7ZHGQPCDMe8tdkZCu6QE3Lz1v dF5jgpnm-qmwZTPj695svH5zsCh9QkmD9SMsUVj10fETMVqgHwV6HL1XXtfS w0uSnunW5hdVOMZ85VlQmH8w0lemC2k9C37y3yX69IyK_2NSeu1FE3F6nB90 XLHxG9BkSASeeo9z2BNKuDWoED_p9SJubi-xJ4vl1cPO5yXcwhoDWJ8icF-W 89IBGxsXYOoBshN-kULVBl0qCLx-OAL9b7cwIOpAi10NahEZzhjqsCLAYbh_ 55mSawO0JyeQOMGvDH1smY-rm7uyCwoW5gjGf04C_s0pi-gHLduRf1DSlHMj o11Lxme77acBzZG-gbW_Yvm9TClYQjHbbMEore2qmcRsGc7-PpcjG8I4fmjc UwXd1jc87ELIafKFhbEfKKSOAEAzu5OjTPLgsexW0oL9LuxEcxAYu6rugY6Q rlawjjaOloYkQmiKCmIKjKVa8TB3GLn1Q6U00GCpJ0U53s8YnFWFAK9y8XJy uuPQzDzxpMWKa11FYzxgiSKr_JyDRSDh1Dg65hXyRGcf-Hs0mcdckvOFOWwD K0dFbrcgxPqXVgsVpEjhxjVJnAftt3AKePsaPcKzW8k3BxlBGsyqWisk_gQ9 DuysZLb9jAI5zZVIGvDbN0uGM9pc6ARffEn3MkVHqdAnnzZ7WusTZ0bMT-vf dTGlcycgcjx1StBKEcNhTYc1F_1FCf_ONmrMQHWv91y3HIV4b1vf3mZSIsVO wm2uf0WjBFWr2uLWLuD2f504OptdEe4qxHhtVPGnbZeVRTJ39JX4nMNScDg1 K2hkc9-YOiAhjgImpMhh2WdZvlOMA3t_NsiUDNHbOW5NogoxxMUZDmG0mTtz oFvH8lVWv9BuHqsMHurzuo-yDFTj_90Z1ephEDor3sFPqX41u3mfRXxUJxCx kS6cwQ5HwS1xnojbAGMu5O5ES0tHzIdUcyZs73GW-6V21fHI2vSkyyrcrVf6 WYOBeI4EBMmYZhuh9vRwzorKmj3Bim2vf-GRhseqY2Blkbt6Wo2A0Un0Q7Pc Rev33zueEQz8IIVG6xVG8SMkyZSV1737Afc3amMSLtzTN1-i3rrGhLXk6rF6 p7PWdC_6UqNvFjenQ69EkE9iFYboyRMisoy2eE0_gfbSAlFnjqp4oaLwYO5y St1SKE_v8DHXcCmVQDEzubtmrJ9M6ARwTxvQFT_hJ7rXNsapPg9j_dnzHcT0 -5xpiJV9tfUDgq1iBndScPLluK0IYfwtfOtH5BLCiD_foBFBNq9b2vPO4_Fs GZKEVq4DCupWYCby3VIZAP1GYeixzpHtmBp1P7g5vTTfaq0ENfslcCiI4QQg 4AamEM14L6YMQbUr_C7hC375hzbyrQJ1ZXi7nCazu_USD5CnxdVaEb3NKKTl K-3CfrsO6QpwerA-LmYim0ys1fdG3xrSRy1ewyYXgBGcS4o5nyTceLae2_4c ool3SlAKOJl68q09ziQSubSjGpLaOj8Pxuo8QAwhl9dvLzJKQ6HneQiG7OdU mRZ_ONm9_7pH3iJdulfWR5sa-2uytUmzce4fNTKDU_JWHhUIZbsndxU5_NO3 jFtTJK_eysUBnx2WY7nSQlbfD6FkVpznQ0ZesbaZHK2gs903Bak_8y9jic01 gX1dhfkRuFgoB6mq6w4vPDGUNFj99AAWl1HiSQBtv75tONpwn8EF_qvy2417 sC_nzYgPuZzDz_phKWS-HO3Zw-DNrt9Eo8-RJ3b1bzFfU0PVp3vEAPpFHsWS G3k2m6KhrD0BDbPMWyYwTsW0F_Np1qFG6ulr48HqkgexnXKg_MrG2rLHd_FL -hwtu-0p-nMOSSPx1kdzve-rqiXVBbUqaj5QR2tSq24Flf0HkpDJ-tfGeVKc rtJ-U1eYWZJho9L0MIaqBlOXgCtqB8lVrHlq8_LuCKMZGKsbn5HowL58Ug-a zF-EAd7eMuZxjdoZqqiMOXJMK8A56MJFq5GQuvhw_P9tstCczq9688Qh6vUN OYBiZ39m7L_5FoeN_u3A1NX2FUrb5ocXh8DWC21qTHaL3i0yEKWxZN3RMEY7 BMwO4I9pqYmNVTNHceRw9kjoXyp6qh-Yrf_8yRrb-bf3buk7uldHQjOa4qDw yMFLTjejxiVpOJWYlymy5bGCGdCJDgf4_F6fYshPVUi6Rai7lF3DDuwJNquu OwhO6q3b0pubxrw7w64Z9eazfybD2ZrvyVZJnowRHV7O2Ixb0fK016BW2cXq _7V_fnM55BH5sr97xdJDSRX27WmduvDVqPnRqj2nicvWgaGnIvbsGAKJJXn2 7rICyih2gzF-U-d9YCBktgoCpTk9up_aXqT_oOqkTHwKnt4ksaDPkcwh3laj 9E-9rDEgBZ6Fbn8UpHJrxkxEyta1T1Mf115nJd3YjstAAtpdsZ2afGbyFzvS dUMkySkLUSDVHJuKnQCaqOZRSQ96zVBc0m8SD5QNKUh0gj0voIMriUAuz_yh Xl7KadRx4g8PiL8z94CSeBhsa8P_zwHZvAhWpnXqT1YDndz1HCmYy94gFD0F xG2SmJD7GwGp_8wBAsYMCvQOH-n8keovGuFulosU2UivnVKnmL51mI7XMrpY F7n6pOactiFAilO27zqXI9iIKYA8ELwbdRyhD3yWXedqSOArEIcSc7HakyNc

Joomla site error: “mark( ‘afterLoad’ ) : null; /** * CREATE THE APPLICATION…”

gerold — Wed, 19 May 2010 16:39:40 +0000

I happened to setup a Joomla site from other hosting (zipped Joomla files and mysqldump), and after the setup on our server I got the error (below) – not exactly an error but a contents of “index.php” file. I found few Joomla related discussion regarding this issue but there’s no solution.
mark( 'afterLoad' ) : null; /** * CREATE THE APPLICATION * * NOTE : */ $mainframe =& JFactory::getApplication('site'); /** * INITIALISE THE APPLICATION * * NOTE : */ // set the
......
JDEBUG ? $_PROFILER->mark('afterRender') : null; $mainframe->triggerEvent('onAfterRender'); /** * RETURN THE RESPONSE */ echo JResponse::toString($mainframe->getCfg('gzip'));

I found the problem on “.htaccess” file as it contain a line “AddType x-mapp-php5 .php”. I commented out the line because I think it is related to parsing PHP files and this solved my problem. I searched for this code and it is related to servers with PHP 4 and 5 installed – details below from 1and1 hosting.

By default Apache uses PHP 4 for .php extension. If you don’t want to rename all your
scripts to .php5 you can do the following:
Create a .htaccess file and place the following line AddType x-mapp-php5 .php in it.

This will tell Apache to use PHP 5 instead of PHP 4 for the extension .php in the
directory the .htaccess is placed and all sub-directories under it.

You can use AddHandler x-mapp-php5 .php as an alternative for or instead of
AddType x-mapp-php5 .php

SSL Certificate registration now includes “www” subdomain.

gerold — Wed, 19 May 2010 10:37:46 +0000

Good News!

I recently purchased SSL certificate (GeoTrust QuickSSL Premium) for our client domain (example.com) and I was surprised because it also registered the “www” sub-domain (www.example.com). Before, you need to purchase both SSL or configure your website to redirect to either www or non-www.

I am not sure if this is available on other SSL type from GeoTrust and other SSL providers.

Free alternative to InnoDB Hot Backup

Pim — Thu, 29 Apr 2010 13:28:22 +0000

I recently found out that there is a free alternative to InnoDB Hot Backup. For those of you using MySQL with the InnoDB plugin you probably know that MySQL does not provide a tool for making online non-blocking backups. InnoBase Oy, the makers of InnoDB, do provide a tool but it’s not free. In fact they charge around $600 per year per server.

The tool that I’m talking about is XtraBackup by Percona. This tool is originally meant to accompany the XtraDB storage engine which in itself is a patched version of InnoDB. XtraBackup will create online non-blocking backups for both XtraDB and InnoDB databases and best of all, it’s free.

For those of you who are not that familiar with MySQL backups, the standard way of doing backups is with mysqldump. This can be done with the database online but it blocks the tables it’s backing up which is not acceptable for production environments. It also takes a good amount of time to restore a mysqldump since it writes out everything as SQL statements which then have to be processed again. A binary copy is much faster to restore but commonly requires the server to be stopped. The best alternative is to create an LVM snapshot of the binary files but this requires LVM to be set up and enough disk space to perform the LVM snapshot. All in all it’s nice to have a free alternative although I have to add the footnote that I haven’t tested it on any decently sized database to check what the performance impact is.

Drupalcon Security Session Notes

andrew — Mon, 19 Apr 2010 22:27:19 +0000

Some of us are attending the San Francisco Drupalcon 2010 this week. I am in a session which some of you may find interesting. The concepts are far from revelational but they are relevant.

Most security breaches are done still XSS
Some things to do to increase security
1. Do not allow full HTML as input to anyone – filtered HTML only
2. ONly give Super admin privileges to trusted users
Some best practices
1. Change passwords
3. do not use ftp
4. Keep your site up to date
Online resources
1. drupal.org/sercurity
2. drupal.org/security/contrib
3. Security Review Module