Sometimes, the code looks good, but it doesn't mean that the whole project in itself is good.

First, a project should be simple to install. The install documentation should exist and always be up to date. I encounter too many projects where "inside knowledge" is required to install the application, and you don't know if it doesn't work because you didn't install it right or it just doesn't work. There should be a single command (makefiles or shell scripts are here for that) for a single task. It should not require weird system settings (this is very annoying with PHP when a lot of the language is customizable through php.ini).

On a similar note, the architecture should be simple to comprehend. When it feels too much like "magic", when a simple action has too many unsaid consequences, something is wrong. There should not be hundreds of classes and inheritances when an option would be enough (this is an idea found at least in Python and Symfony 1.4 that I quite like). Names should be readable and short, not everyone has or likes using auto-completion.

I'm not sure what creates this kind of WTF. Inexperienced but talented programmers? Programmers who think too high of themselves? Programmers that are not lazy enough?

Well, chmod 666 is the true evil obviously, but people who use the chmod 777 trick really don't want to bother with the different signification of x for files and folders, so they mark all files as executables. This makes ls in my terminal quite ugly, and is what motivated me to write yet another rant!

Traditionally, the web server runs with a special, underprivileged user. Now, this is totally fine — I'll get back to that later. However, when a developer starts a project, he naturally does it with his own account. This is fine, too. When he wants to test it, he installs a web server (let's say Apache and mod_php¹), and tells the web server to use the project directory. However, the web server's user can't read, or at least can't write in the directory. And here, our web developers asking for help gets recommended to "chmod 777". Symfony even has a command for doing it, and it is a real shame.

It will work. It will also make an ugly git commit, and an ugly ls. Il will create files owned by the web server, and the developer is likely to use sudo before every command to work around the problems that will ensue, and it just gets insane.

Separation of privileges is what makes UNIX systems great. Let's understand them and use them, please.

The obvious solution here is to run your development server under the development user. There is no need to separate when it's for your own usage. There is a even better solution, embraced by almost every language but PHP: a way to start a web server on demand. I believe it was started by Ruby on Rails with WEBrick, and now every non-PHP framework has it. Python, by using WSGI makes it very easy. I simulated this feature for Symfony by writing symfttpd. It's actually simpler for the developer as there is no configuration or installation at all.

Enough about developers; it's not their job to setup daemons and manage UNIX systems. Let's talk about the real accomplices of the Devil: system administrators.

Yes, there are system administrators that don't use permissions properly. And they are legion. I've seen horrors, up to "sudo svn up" on the production server, because half the files ended being owned by root. And then "chmod 777" on millions of files.

There are many solutions there; group inheritance with the setgid bit, forcing the users to su as the web server user, or a deployment script (I've used the three of them for different situations).

Why is this important? Because it is often useful to separate users (one should not have access to the other's projects in reading or writing), or to separate projects for security (one hacked project should not give access to the others).

I've seen it… done wrong:

• safe_mode for PHP. It doesn't work and will disappear in newer versions anyway.
• Only allowing FTP access to users, who can still upload a PHP script which will have access to everything (if run through the web server). Oh, and FTP sucks. Same issue with SSH and chroot.
• Add the users to the group the web server is running as. Allows SSH access. Nice, but the PHP script trick will, again, defeat it.

There is only one solution: use the "group" solution, but run a different PHP instance for each user. It is quite rarely used because the convenience of Apache and mod_php. But running PHP in the same process as the web server feels quite dangerous for me too. I think mod_php is an abomination.

I've done it for years with Lighttpd, PHP and FastCGI with a few alterations to Gentoo's spawn-fgci init script (which is now able to handle multiple configurations without any alteration since a few months). My setup is very similar to that one.

Note that while I mention PHP, this issue is not strictly related to PHP, yet seems widespread in PHP communities.

1. I don't like much both of them, but more on that later

I see on a lot of projects stuff like this:

 
function getStuff()
{
  try
  {
    return $this->retrieveStuff()
  }
  catch (Exception $e)
  {
    return null;
  }
}
 

The issue, in this example, is not that we silence an error. It should be logged, but there could be a legitimate use for this. For instance, it requires a connection to a remote server we have no control over, and we still want to serve the rest of the content to the user, as the information is not vital.

No, my issue it that we catch all errors, when the aim was to catch only one error (for instance "unable to connect to that remote server"). Imagine now there is a real bug in retrieveStuff(), it would still be interpreted by developers as a connection errors.

The solution is to have one exception type per error type (it sounds obvious said that way, doesn't it?), and only catch the specific error, for instance ConnectionError.

PHP's error handling is historically bad, and even though new features use exceptions, the legacy ones don't. Or sometimes functions should trigger a fatal error but it is only a warning, and the code continues, which can trigger unwanted and hard-to-debug results. There are amazing plugins for Symfony that convert errors to exceptions (and warnings seem to be handled properly, i.e. they are logged and don't cause the page to fail). It's a start.

However, even when you get exceptions, you can't always filter them easily. For instance, in Symfony, there are many different errors that would throw the same class. Symfony has a handful of exceptions, Doctrine has a bit more (though they can be quite meaningless!). Overall, there are defined "by module" than "by error".

Comparatively, Python's handling of exceptions is exceptional¹. There are exceptions for everything, and most of the time errors are handled by only checking the class and not the contents of the exception. It is probably the feature I love the most about Python. With the else, finally, and with statements, it is almost a pleasure to handle errors.

1. sorry

Symfony is pretty good for a PHP framework — it's probably still one of the better ones. While Symfony is open source, its development clearly isn't open.

Nowadays, it is mainly unmaintained. Well, it's maintained by Sensio, which is very closed towards outsiders, and doesn't apply fixes for versions below 1.3 even when patches are handed ready to them. Which means for instance that the official 1.0 doesn't work with PHP 5.3 even though the fixes are available and straightforward.

I will pass on the worrisome attitude towards security of the most used plugin as I've talked about it extensively already. Constant reminders don't even work.

There is much more activity towards Symfony2, which might explain the lack of updates to the 1.x branches. Symfony2 introduces more decoupling, which should avoid many bugs reported but ignored by the development team, probably because they are boring to fix and require very deep knowledge in Symfony's architecture — I would have tried to fix them myself if I had the time and motivation; I'm afraid I usually take the easy route and work around them.

But how does Symfony2 looks for a developer? What I've seen is fairly standard and boring, and sometimes looks like sad attempts of reimplementing Python features in PHP. Why not use Python then?

Doctrine2 is even worse — it's the same broken, undocumented API. Propel 1.5, with a minimum set of changes, became much more powerful and easy to use for a developer. At least there's hope on this side.

Overall, it seems the Symfony lead developers don't take much input from the community (like the decision to follow Zend conventions), and focus on making something fun to write, instead of fun to use. I have the choice between an undermaintened version and a future version which will break compatibility without giving me much advantages; moreover some of the stuff that was shown in conferences isn't event implemented yet. I'm not used to see that in the free software word.

Time to switch?

Something that always annoyed me is how tedious it is to install a Symfony project on a machine. Since I frequently need to intervene quickly on a project for work, and I was getting a brand-new machine, I really didn't want to create an apache vhost (let alone install Apache: it's painfully slow and its configuration is obscure and hard to debug), edit my /etc/hosts file, etc. for each project.

Moreover, developers are not system administrators and should not have to do complicated setups, especially when it turns out to be badly set up, and problems are "resolved" by "chmod 777" or "chmod 666" (which is indeed evil), a very sad but true practice promoted even by the developers of Symfony since there is a symfony fix-perms command that basically does that. It should have been named symfony break-perms or symfony please-make-things-insecure-i-want-to-go-back-to-windows or never been made.

On most non-PHP frameworks, there is a small embedded webserver that you can run on-demand. No configuration needed. Moreover, no special rights needed: a simple user can start it.

Since there was no such webserver written in PHP that was able to run Symfony properly, I chose to auto-configure lighttpd with a simple tool called symfttpd. The genconf tool was born.

However, I encountered another issue I didn't think of before: for each project, I had to create symbolic links to the Symfony source code (another practice of the Symfony community, and there is no proper alternative in the PHP world). Hence, I created mksymlinks. For the developer, it is very simple to use: configure once on the machine, once per project, and that's it.

genconf only generates a configuration, which is still very practical for a system administrator; moreover it is flexible and well-tested. But it still required the develop to configure something, and there still was the rights problem.

Hence I created spawn which handles starting and stopping the webserver, just like non-PHP frameworks do. As a nice addition, it keeps server and PHP logs in the logfolder of the project.

symfttpd has even more uses; one I didn't think of at first was that it can automate the installation of a project on a continuous integration platform, and can start a webserver for functional testing (both are used daily at work).

One of the most important aspects of symfttpd is that all tools are independent: you can use only mksymlinks or genconf (though spawn more or less requires the use of both, it isn't set in stone). A system administrator will find use in mksymlinks and genconf, and a developer more in mksymlinks and spawn.

You'll find extensive documentation on the project page; what will follow is a quick tutorial for developers.

Install the necessary packages:

 
# Debian/Ubuntu
aptitude install php5-cgi php5-cli lighttpd
 
# Gentoo
emerge php # with USE="cli cgi"
emerge lighttpd # with USE="fastcgi"
 
# Macports
port install php5 +fastcgi && port install lighttpd
 
# Windows
Nice try.
 

Get the symfttpd source code:

 
cd && git clone git://github.com/laurentb/symfttpd.git
 

There are also archives you can download here if you want to avoid git or bleeding-edge changes.

Basic configuration:

 
# notice the dot before symfftpd.conf.php
$EDITOR ~/.symfftpd.conf.php
 

Enter something like that:

 
<?php
$options['sf_path']['1.0'] = '/home/myuser/symfony/1.0';
$options['sf_path']['1.4'] = '/home/myuser/symfony/1.4';
 

Of course, you have to have to adapt it to the Symfony versions you have installed and where you put them.

Configure the project:
If the project is using Symfony 1.4 in the lib/vendor/symfony, you don't need to do anything. In case it is different, or to be on the safe side, create the file config/symfttpd.conf.php in your project. After, add the file to your project's version control repository. If you're lucky, someone already did it for you.

 
cd ~/myproject
# this time, no dot
$EDITOR config/symfttpd.conf.php
 

 
<?php
$options['want'] = '1.3'; // The version of Symfony used by your project
$options['lib_symlink'] = 'lib/vendor/symfony'; // lib/vendor/symfony will lead to the "lib" directory of Symfony
 

 
~/symfttpd/mksymlinks
 

You're done.
It will create symbolic links for plugins too, even if the version of Symfony (1.0 for instance) doesn't handle them!

To start the server:

 
~/symfttpd/spawn
 

It will then tell you how to access it. It's time to stop fighting with old, unpredictable software like Apache and start developing again!

What's coming in future releases:

• Colors
• Interactive configuration
• Server/PHP logs displayed in the terminal
• Handling "sample" files
• Custom configuration support on various places

Contributors are welcome.

Update: You can safely ignore this angry rant as the issues have been fixed.

I am speechless. While doAuthPlugin looks interesting (especially because it uses inheritance and not some silly secondary Profile table), on the topic of security it is worse than sfDoctrineGuardPlugin.

Let's have a quick look at doAuthTools.

  public static function rememberHash(User $user) {
    return md5($user->getId().'-'.$user->getUsername().substr($user->getPassword(),0,5));
  }

There's the use of md5, which doesn't doesn't resist long to rainbow tables attacks nowadays, but the worst is what is used. No random data at all, no salt.
The user id is most of the time public, the username almost always is, and for absolutely no reason only the first five characters of the password are used.

 
  public static function activationCode(User $user) {
    return md5($user->getCreatedAt().time().$user->getUsername().substr($user->getUsername(),0,5));
  }

created_at is easy to guess, time() is easy to guess and the rest is public. Again, what's with the use of substr()? I would also guess the author wanted to use the password and not the username.

 
  public static function generatePassword() {
    return substr(md5(rand(1000,9999).time()),0,8);
  }
 

Now this is just sad. Sure, even with the use of a-f0-9 there is still 16^8 possible passwords, but why not spend a little time and try to use all the available letters? It also means that if you know the user id and username, you know you only have 16^5 rememberHashes to brute-force, which is easily doable.

We've already seen that using rand() is bad, but restricting it is even worse. Why on earth? To sum up, the password is constructed from a poor random algorithm with under 9000 possible values, and an easily predictable timestamp.

Frightening.

Joel Spolsky on Twitter:

Although I appreciate that many people find Twitter to be valuable, I find it a truly awful way to exchange thoughts and ideas. It creates a mentally stunted world in which the most complicated thought you can think is one sentence long. It’s a cacophony of people shouting their thoughts into the abyss without listening to what anyone else is saying. Logging on gives you a page full of little hand grenades: impossible-to-understand, context-free sentences that take five minutes of research to unravel and which then turn out to be stupid, irrelevant, or pertaining to the television series Battlestar Galactica. I would write an essay describing why Twitter gives me a headache and makes me fear for the future of humanity, but it doesn’t deserve more than 140 characters of explanation, and I’ve already spent 820.

I couldn't agree more.

I will also be far from being the first to say it, but Twitter is just a poor reimplementation of IRC over HTTP, which is completely inappropriate. Twitter is a constant failure with bad uptime, scalability issues and over-centralization. Twitter is one of the worst things that happened to the Web; it also promotes URL shorteners, which have security issues and double the possible unavailabilities.

Yet another case of trying to be too clever with randomness!

Update: After a year, both plugins are finally updated with a better random key generator.

Security is not easy. Programmers should leave things like random number and identifier generation to a library (or at least research the best way to do it). A lot of projects learned it the hard way.

Let's talk for instance of a function I encountered about six months ago:

function generateRandomKey($len = 20)
{
  $string = '';
  $pool   = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
  for ($i = 1; $i < = $len; $i++)
  {
    $string .= substr($pool, rand(0, 61), 1);
  }
 
  return md5($string);
}
 

It is a real gem as it seems it was meant to concentrate everything a programmer should not do.

Flaw 1: Using rand() instead of mt_rand(). Any PHP programmer should know this.

Flaw 2: Using md5() instead of sha1(). Any PHP programmer should know this too.

Flaw 3: Generating a 20 character random string and then hashing it to a 32 character string looks a bit silly. Granted, the hash has less different characters, but still the input can have 7.04423e+35 different values while the output can have 3.40282e+38.

Flaw 4: Calling rand() multiple times. It denotes a strong ignorance on how pseudo-random number generators work. Since the seed doesn't change, it can be easy to guess what will be the next number from the previous ones. It isn't "more random" to call it multiple times.

Flaw 5: Using a very limited range for picking a random number. Also, why choose alphanumeric characters if you are not using them afterwards since they are passed to a hash function? This seems to be very badly designed.

Flaw 6: Feeding md5 with alphanumeric characters, present in all the rainbow tables in the world.

Flaw 7: Ever heard of chr()?

Flaw 8: Why reinvent the wheel? At least if you have a reason, leave a comment (no comments at all on this function).

Without even testing if the results are good or not, any programmer should know this function should be completely rewritten.

Let's see how it should have been done.

 
function generateRandomKey()
{
 
  return base_convert(sha1(uniqid(mt_rand(), true)), 16, 36);
}
 

How did I come up with this? Mostly be reading the PHP manual and not creating a random number generator, but merely using one.

The use of base_convert() is not really important, it is mostly because the database field was too short (nothing is lost from the sha1()). We actually would be better off without sha1() though, it is there mostly for the eye-candy.

uniqid() will take the result of mt_rand() and append it an unique value (but in a predictable range, hence the use of both functions). This is recommended by the PHP manual and is more than enough.

Now you're probably asking yourself where did I find this piece of code.

It is in sfGuardPlugin and sfDoctrineGuardPlugin, for the symfony framework. It's here almost from the start, though it was even wronger. Those are by far the most used plugins, and are endorsed by the official documentation.

What does this mean? On some systems, it will be very easy to brute-force the cookie to log in as any user. Or an user could accidentally become logged as another one, since collisions are highly likely.

Come on, if this is real, someone would have spotted it! I is probably not exploitable.
Actually, it is. I have been able to reproduce it on an older Debian server, and on this server was a website where users reported multiple times getting identified as another user (hopefully this server is retired and the project now runs on the fixed function).
I also tried to reproduce it on a Windows XP virtual machine (since it was the easiest way to get a PHP with a flawed random number generator) with success. Since on these two machines I got a lot of collisions, even on better systems the flaw should be exploitable with some effort.

Why am I doing this?
Because I reported it six month ago, and it received very little attention.
I think symfony is great, but the handling of security isn't (though my other criticisms are more on standard programming or hosting practices encouraged by symfony than on the code; more on the latter in a coming article, be sure to subscribe).

 
    /**
     * returns an array of modified fields and associated values
     * @return array
     * @todo What about a better name? getModifiedFields?
     */
    public function getModified()
    {
        $a = array();
 
        foreach ($this->_modified as $k => $v) {
            $a[$v] = $this->_data[$v];
        }
        return $a;
    }
 
    /**
     * REDUNDANT?
     */
    public function modifiedFields()
    {
        $a = array();
 
        foreach ($this->_modified as $k => $v) {
            $a[$v] = $this->_data[$v];
        }
        return $a;
    }