Enforcing target descriptions within build files with a Git hook

2011-04-22T17:44:00.001+02:00

When automating mundane tasks of a project or development environment with a build tool like Phing or Ant, the driving build file will naturally accumulate several targets and tasks over time. To ease the build file acceptance within a team and at a later stage also the contribution rate by team members, it's crucial that all build targets have a description attribute to provide at least a rough outline of the build features at hand. When these attributes are in place the (potential) build file user will get such an outline by executing the build tool's list command (phing -l or ant -p). To get a better picture of the problem at hand imagine a project poorly covered with tests and your personal attitude towards extending it or just take a peek at the screenshot below showing a very poorly documented build file.

To overcome this accumulation of some sort of technical debt (i.e. poorly documented targets) there are various options at hand. The first one, not covered in this blog post, would be to add a pursuant test which verifies the existence of a description for every target/task of the build file under test. As it's very uncommon, at least from what I've heard, to have your build files covered by tests; the next thinkable approach would be to use a Git pre-commit hook to guard your repository/ies against the creeping in of such poorly documented build files.

The next listing shows such a Git hook (also available via GitHub) scribbled away in PHP, which detects any build file(s) following a common build file naming schema (i.e. build.xml|build.xml.dist|personal-build.xml|…) , prior to the actual commit. For every target element in the detected build file(s) it's then verified that it has a description attribute and that it's actual content is long enough to carry some meaning. If one of those two requirements aren't met, the commit is rejected while revealing the build file smells to the committer, so she can fix it, as shown in the outro screenshot. Happy build file sniffing.

#!/usr/bin/php
<?php
define('DEPENDENT_EXTENSION', 'SimpleXML');

if (!extension_loaded(DEPENDENT_EXTENSION)) {
    $consoleMessage = sprintf(
        "Skipping build file checks as the '%s' extension isn't available.", 
        DEPENDENT_EXTENSION
    );
    echo $consoleMessage . PHP_EOL;
    exit(0);
}

define('MIN_TARGET_DESCRIPTION_LENGTH', 10);
define('TARGET_DESCRIPTION_ATTRIBUTE', 'description');
define('TARGET_NAME_ATTRIBUTE', 'name');
define('CHECK_DESCRIPTION_LENGTH', true);

$possibleBuildFileNames = array(
    'build.xml.dist',
    'build.xml-dist',
    'build-dist.xml',
    'build.xml',
    'personal-build.xml'
);

$violations = getAllBuildFileViolationsOfCommit($possibleBuildFileNames);
fireBackPossibleViolationsAndExitAccordingly($violations);

function getAllBuildFileViolationsOfCommit(array $possibleBuildFileNames)
{
    $filesOfCommit = array();
    $gitCommand = 'git diff --cached --name-only';
    
    exec($gitCommand, $filesOfCommit, $commandReturnCode);
    
    $allViolations = array();
    foreach ($filesOfCommit as $file) {
      if (in_array(basename($file), $possibleBuildFileNames)) {
          $violations = checkBuildFileForViolations($file);
          if (count($violations) > 0) {
            $allViolations[$file] = $violations;
          }
      }
    }

    return $allViolations;
}

/**
 *  @param  array $allViolations
 *  @return void
 */
function fireBackPossibleViolationsAndExitAccordingly(array $allViolations)
{
    if (count($allViolations) > 0) {
        foreach ($allViolations as $buildFile => $violations) {

            $buildFileConsoleMessageHeader = sprintf("Build file '%s':", $buildFile);
            echo $buildFileConsoleMessageHeader . PHP_EOL;

            foreach ($violations as $violationMessage) {
                $buildFileConsoleMessageLine = sprintf(" + %s", $violationMessage);
                echo $buildFileConsoleMessageLine . PHP_EOL;
            }
        }
        if (count($allViolations) > 1) {
            $rejectCommitConsoleMessage = sprintf(
                "Therefore rejecting the commit of build files [ %s ].", 
                implode(', ', array_keys($allViolations))
            );
        } else {
            $rejectCommitConsoleMessage = sprintf(
                "Therefore rejecting the commit of build file [ %s ].", 
                implode(', ', array_keys($allViolations))
            );
        }

        echo $rejectCommitConsoleMessage . PHP_EOL;
        exit(1);
    }
    exit(0);
}
/**
 *  @param  string $buildfile
 *  @return array
 */
function checkBuildFileForViolations($buildFile) {
    if (!file_exists($buildFile)) {
        return array();
    }

    $buildfileXml = file_get_contents($buildFile);
    $buildXml = new SimpleXMLElement($buildfileXml);
    $allBuildTargets = $buildXml->xpath("//target");
    $violations = array();

    if (count($allBuildTargets) > 0) {

        $targetsWithNoDescription = $targetsWithTooShortDescription = array();

        foreach ($allBuildTargets as $buildTarget) {

            $actualTragetAttributes = $buildTarget->attributes();
            $allUsedTragetAttributes = array();
            $actualTargetName = null;

            foreach ($actualTragetAttributes as $attribute => $value) {
                $allUsedTragetAttributes[] = $attribute;

                if ($attribute === TARGET_NAME_ATTRIBUTE) {
                    $actualTargetName = $value;
                }

                if (CHECK_DESCRIPTION_LENGTH === true && 
                    $attribute === TARGET_DESCRIPTION_ATTRIBUTE && 
                    strlen($value) < MIN_TARGET_DESCRIPTION_LENGTH) 
                {
                    $targetsWithTooShortDescription[] = $actualTargetName;
                }
            }   

            if (!in_array(TARGET_DESCRIPTION_ATTRIBUTE, $allUsedTragetAttributes)) {
                $targetsWithNoDescription[] = $actualTargetName;
            }
        }
        if (count($targetsWithNoDescription) > 0) {
            if (count($targetsWithNoDescription) > 1) {
                $violations[] = sprintf(
                    "Build targets [ %s ] don't have mandatory descriptions.", 
                    implode(', ', $targetsWithNoDescription)
                );
            } else {
                $violations[] = sprintf(
                    "Build target [ %s ] doesn't have a mandatory description.", 
                    implode(', ', $targetsWithNoDescription)
                );
            }
        }

        if (count($targetsWithTooShortDescription) > 0) {
            if (count($targetsWithTooShortDescription) > 1) {
                $violations[] = sprintf(
                    "Build targets [ %s ] don't have an adequate target description length.", 
                    implode(', ', $targetsWithTooShortDescription),
                    MIN_TARGET_DESCRIPTION_LENGTH
                );
            } else {
                $violations[] = sprintf(
                    "Build target [ %s ] doesn't have an adequate target description length.", 
                    implode(', ', $targetsWithTooShortDescription),
                    MIN_TARGET_DESCRIPTION_LENGTH
                );
            }
        }
    }
    return $violations;
}

Measuring & displaying Phing build times with buildhawk

2010-11-20T16:30:00.003+01:00

Recently I installed a Ruby gem called buildhawk which allows to measure and display the build times of Rake driven builds. As I like the idea behind this tool a lot but mostly use Phing for build orchestration, it was time to explore the possibility to interconnect them both. In this blog post I'll show an implementation of an apposite Phing Logger gathering the buildhawk compatible build times via git note(s) and how to put the interplay between those two tools to work.

Logging on

As mentioned above the build time of each build is stored as a git note and associated to the repository's HEAD, reflecting the current state of the system under build (SUB), which assumes that the SUB is versioned via Git. The next shown Phing Logger (i.e. BuildhawkLogger) grabs the overall build time by hooking into the buildFinished method of the extended DefaultLogger class, transforms it into a buildhawk specific format and finally adds it as a git note.

<?php

require_once 'phing/listener/DefaultLogger.php';

/**
 *  Writes a build event to the console and store the build time as a git notes in the   
 *  project's repository HEAD.
 *
 *  @author    Raphael Stolt <raphael.stolt@gmail.com>
 *  @see       BuildEvent
 *  @link      https://github.com/xaviershay/buildhawk Buildhawk on GitHub
 *  @package   phing.listener
 */
class BuildhawkLogger extends DefaultLogger {
    
    /**
     *  @var string
     */
    private $_gitNotesCommandResponse = null;

    /**
     *  Behaves like the original DefaultLogger, plus adds the total build time 
     *  as a git note to current repository HEAD.
     *
     *  @param  BuildEvent $event
     *  @see    BuildEvent::getException()
     *  @see    DefaultLogger::buildFinished
     *  @link   http://www.kernel.org/pub/software/scm/git/docs/git-notes.html
     */
    public function buildFinished(BuildEvent $event) {
        parent::buildFinished($event);
        if ($this->_isProjectGitDriven($event)) {
            $error = $event->getException();
            if ($error === null) {
                $buildtimeForBuildhawk = $this->_formatBuildhawkTime(
                    Phing::currentTimeMillis() - $this->startTime
                );
                if (!$this->_addBuildTimeAsGitNote($buildtimeForBuildhawk)) {
                    $message = sprintf(
                        "Failed to add git note due to '%s'",
                        $this->_gitNotesCommandResponse
                    );
                    $this->printMessage($message, $this->err, Project::MSG_ERR);
                }
            }
        }
    }
    
    /**
     * Checks (rudimentary) if the project is Git driven
     *
     *  @param  BuildEvent $event
     *  @return boolean
     */
    private function _isProjectGitDriven(BuildEvent $event)
    {
        $project = $event->getProject();
        $projectRelativeGitDir = sprintf(
            '%s/.git', $project->getBasedir()->getPath()
        );
        return file_exists($projectRelativeGitDir) && is_dir($projectRelativeGitDir);
    }
    
    /**
     *  Formats a time micro integer to buildhawk readable format.
     *
     *  @param  integer The time stamp
     */
    private function _formatBuildhawkTime($micros) {
        return sprintf("%0.3f", $micros);
    }
    
    /**
     *  Adds the build time as a git note to the current repository HEAD
     *
     *  @param  string  $buildTime The build time of the build
     *  @return mixed   True on sucess otherwise the command failure response
     */
    private function _addBuildTimeAsGitNote($buildTime) {
        $gitNotesCommand = sprintf(
            "git notes --ref=buildtime add -f -m '%s' HEAD 2>&1",
            $buildTime
        );
        $gitNotesCommandResponse = exec($gitNotesCommand, $output, $return);
        if ($return !== 0) {
            $this->_gitNotesCommandResponse = $gitNotesCommandResponse;
            return false;
        }
        return true;
    }
}

Putting the Logger to work

As the buildhawk logger is available via GitHub you can easily grab it by issuing sudo curl -s http://gist.github.com/raw/707868/BuildhawkLogger.php -o $PHING_HOME/listener/BuildhawkLogger.php. The next step, making the build times loggable, is achieved by using the -logger command line argument of the Phing Cli and specifying the buildhawk logger name or the path to it. In case you want the buildhawk logger to be used per default (it behaves like the default logger if the SUB isn't Git driven/managed) you can also add it to the Phing shell script.

The next console command issued in the directory of the SUB shows a Phing call utilizing the BuildhawkLogger, assumed it has been installed at $PHING_HOME/listener/BuildhawkLogger.php and not been made the default logger.

phing -logger phing.listener.BuildhawkLogger

Looking at them Phing build times

Now it's time to switch to buildhawk and let it finally perform it's designated task, rendering an with the commit SHAs, commit messages, and build times fed Erb template into an informative, viewable HTML page. To install it you simply have to run sudo gem install buildhawk and you're good to go.

The next console command shows the buildhawk call issued in the SUB's directory to produce it's build time report page.

buildhawk --title 'Examplr' > examplr-build-times.html

The outro screenshot below gives you a peek at a rendered build time report.

Growling PHPUnit's test status

2010-06-03T05:05:00.005+02:00

Two years ago I blogged about a Xinc (R.I.P?) plugin that growls each build status for any via Xinc continuously integrated project. Since I'm using PHPUnit more and more lately, especially in continuous testing sessions (sprints without hitting the continuous integration server), my dependence on a fast and more visual feedback loop rose. In this post I'll provide an easy solution that meets these requirements by utilizing PHPUnit's test listener feature.

What's the motivation, yo?

While doing story or feature sprints embedded in a continuous testing approach I first used a combination of stakeout.rb and PHPUnit's --colors option to radiate the tests status, but soon wasn't that satisfied with the chosen route as it happened that the console window got superimposed with other opened windows (e.g. API Browser, TextMate etc.) especially on my 13,3" MacBook.

To overcome this misery I decided to utilize PHPUnit's ability to write custom test listeners and to implement one that radiates the test status in a more prominent and sticky spot via Growl.

Implementing the Growl test listener

Similar to the ticket listener plugin mechanism I blogged about earlier PHPUnit also provides one for test listeners. This extension mechanism allows to bend the test result formatting and output to the given needs and scenarios a developer might face and therefore is a perfect match.

To customize the test feedback and visualization the test listener has to implement the provided PHPUnit_Framework_Testlistener interface. A few keystrokes later I ended up with the next shown implementation, which is also available via a GitHub gist, supporting the previous stated requirements.

<?php

class PHPUnit_Extensions_TestListener_GrowlTestListener 
    implements PHPUnit_Framework_Testlistener
{
    const TEST_RESULT_COLOR_RED = 'red';
    const TEST_RESULT_COLOR_YELLOW = 'yellow';
    const TEST_RESULT_COLOR_GREEN = 'green';
    
    private $_errors = array();
    private $_failures = array();
    private $_incompletes = array();
    private $_skips = array();
    private $_tests = array();
    private $_suites = array();
    private $_endedSuites = 0;
    private $_assertionCount = 0;
    private $_startTime = 0;

    private $_successPicturePath = null;
    private $_incompletePicturePath = null;
    private $_failurePicturePath = null;

    /**
     * @param string $successPicturePath
     * @param string $incompletePicturePath
     * @param string $failurePicturePath
     */
    public function __construct($successPicturePath, $incompletePicturePath, 
        $failurePicturePath)
    {
        $this->_successPicturePath = $successPicturePath;
        $this->_incompletePicturePath = $incompletePicturePath;
        $this->_failurePicturePath = $failurePicturePath;
    }

    public function addError(PHPUnit_Framework_Test $test, Exception $e, $time)
    {
        $this->_errors[] = $test->getName();
    }
    
    public function addFailure(PHPUnit_Framework_Test $test, 
        PHPUnit_Framework_AssertionFailedError $e, $time) 
    {     
        $this->_failures[] = $test->getName();
    }
    
    public function addIncompleteTest(PHPUnit_Framework_Test $test, 
        Exception $e, $time)
    {
        $this->_incompletes[] = $test->getName();
    }
    
    public function addSkippedTest(PHPUnit_Framework_Test $test, 
        Exception $e, $time) 
    {
        $this->_skips[] = $test->getName();
    }
    
    public function startTest(PHPUnit_Framework_Test $test)
    {
    
    }
    
    public function endTest(PHPUnit_Framework_Test $test, $time) 
    { 
        $this->_tests[] = array('name' => $test->getName(), 
            'assertions' => $test->getNumAssertions()
        );
        $this->_assertionCount+= $test->getNumAssertions();
    }
    
    public function startTestSuite(PHPUnit_Framework_TestSuite $suite)
    {
        if (count($this->_suites) === 0) {
            PHP_Timer::start();
        }
        $this->_suites[] = $suite->getName();
    }
    
    public function endTestSuite(PHPUnit_Framework_TestSuite $suite)
    {
        $this->_endedSuites++;
        
        if (count($this->_suites) <= $this->_endedSuites)
        {
            $testTime = PHP_Timer::secondsToTimeString(
                PHP_Timer::stop());

            if ($this->_isGreenTestResult()) {
                $resultColor = self::TEST_RESULT_COLOR_GREEN;
            }
            if ($this->_isRedTestResult()) {
                $resultColor = self::TEST_RESULT_COLOR_RED;
            }
            if ($this->_isYellowTestResult()) {
                $resultColor = self::TEST_RESULT_COLOR_YELLOW;
            }

            $suiteCount = count($this->_suites);
            $testCount = count($this->_tests);
            $failureCount = count($this->_failures);
            $errorCount = count($this->_errors);
            $incompleteCount = count($this->_incompletes);
            $skipCount = count($this->_skips);

            $resultMessage = '';

            if ($suiteCount > 1) {
                $resultMessage.= "Suites: {$suiteCount}, ";
            }
            $resultMessage.= "Tests: {$testCount}, ";
            $resultMessage.= "Assertions: {$this->_assertionCount}";

            if ($failureCount > 0) {
                $resultMessage.= ", Failures: {$failureCount}";
            } 

            if ($errorCount > 0) {
                $resultMessage.= ", Errors: {$errorCount}";
            }

            if ($incompleteCount > 0) {
                $resultMessage.= ", Incompletes: {$incompleteCount}";
            }

            if ($skipCount > 0) {
                $resultMessage.= ", Skips: {$skipCount}";
            }
            $resultMessage.= " in {$testTime}.";
            $this->_growlnotify($resultColor, $resultMessage);
        }
    }

    /**
     * @param string $resultColor
     * @param string $message
     * @param string $sender The name of the application that sends the notification
     * @throws RuntimeException When growlnotify is not available
     */
    private function _growlnotify($resultColor, $message = null, $sender = 'PHPUnit')
    {
        if ($this->_isGrowlnotifyAvailable() === false) {
            throw new RuntimeException('The growlnotify tool is not available');
        }
        $notificationImage = $this->_getNotificationImageByResultColor(
            $resultColor);
        $command = "growlnotify -w -s -m '{$message}' "
                 . "-n '{$sender}' "
                 . "-p 2 --image {$notificationImage}";
        exec($command, $response, $return);
    }

    /**
     * @return boolean
     */
    private function _isGrowlnotifyAvailable()
    {
        exec('growlnotify -v', $reponse, $status);
        return ($status === 0);
    }

    /**
     * @param string $color 
     * @return string
     */
    private function _getNotificationImageByResultColor($color)
    {
        switch ($color) {
            case self::TEST_RESULT_COLOR_RED:
                return $this->_failurePicturePath;
                break;
            case self::TEST_RESULT_COLOR_GREEN:
                return $this->_successPicturePath;
                break;
            default:
                return $this->_incompletePicturePath;
        }
    }

    /**
     * @return boolean
     */
    private function _isGreenTestResult()
    {
        return count($this->_errors) === 0 && 
               count($this->_failures) === 0 &&
               count($this->_incompletes) === 0 &&
               count($this->_skips) === 0;
    }

    /**
     * @return boolean
     */
    private function _isRedTestResult()
    {
        return count($this->_errors) > 0 ||
               count($this->_failures) > 0;
    }

    /**
     * @return boolean
     */
    private function _isYellowTestResult()
    {
        return count($this->_errors) === 0 &&
               count($this->_failures) === 0 &&
               (count($this->_incompletes) > 0 ||
                count($this->_skips) > 0);
    }
}

Hooking the Growl test listener into the PHPUnit ecosystem

To make use of the just outlined test listener it's necessary to add an entry to PHPUnit's XML configuration file telling PHPUnit which test listener class to utilize and where it's located in the file system. In a next step the images for the three possible Growl notifications have to be added to the local file system, and as the Growl test listener constructor takes these as arguments they have also to be injected in the PHPUnit XML configuration file (i.e. phpunit-offline.xml). Take a peek yourself how this is done in the next listing.

<phpunit backupGlobals="false"
         backupStaticAttributes="true"
         bootstrap="bootstrap.php"
         colors="true"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
         processIsolation="true"
         stopOnFailure="true"
         syntaxCheck="true"
         testSuiteLoaderClass="PHPUnit_Runner_StandardTestSuiteLoader">
  <testsuites> 
    <testsuite name="Zend_Service_GitHub Offline Testsuite">
      <directory>Zend/Service/GitHub</directory>
      <directory>Zend/Service</directory>
    </testsuite>
  </testsuites>
  <groups>
    <include>
      <group>offline</group>
    </include>
  </groups>
  <listeners>
    <listener class="PHPUnit_Extensions_TestListener_GrowlTestListener" 
              file="/Users/stolt/Work/GrowlTestListener.php">
     <arguments>
       <string>$HOME/Pictures/pass.png</string>
       <string>$HOME/Pictures/pending.png</string>
       <string>$HOME/Pictures/fail.png</string>
     </arguments>
    </listener>
  </listeners>
</phpunit>

Putting the Growl test listener to work

Attention shameless plug! As an example application for a continuous testing session I chose a Zend Framework Service component I'm currently working on. To set up the continuously testing workflow, stakeout.rb is still my #1 choice, but in a recent blog post Andy Stanberry shows another tool dubbed Kicker which seems to be coequal. The following console snippet shows in a concrete scenario how to utilize stakeout.rb to watch for any changes on the Zend_Service_GitHub component or it's backing tests which immediately trigger the test suite execution if one is detected.

stakeout.rb 'phpunit --configuration phpunit-offline.xml' **/*.{php} ../Zend/**/*.php ../Zend/Service/**/*.php

In the classic TDD cycle we start with a failing test. Creating the test, adding the assertions that the system under test (SUT) has to fulfill and saving the according test class automatically triggers the test suite execution which ends up in the next shown Growl notification.

Nest a très important client call comes in and since we are clever, a quick TextMate shortcut marks the currently worked on test as incomplete. This step might be a bit controversy as it's also suggested to leave the last worked on test broken, but I got to show you the pending/incomplete Growl notification ;D

After finishing the 'interruptive' client call aka context switch we can continue to work on the feature of the SUT until it fulfills the expected behavior which will be radiated via the next shown Growl notification. Happy Growl flavored testing!

* As you might notice in the shown Growl notification images there's a test suite count of 9 while we are only operating on a single one, this seems to be a possible PHPUnit bug, or just a misconfiguration of my testing environment.

In case you got a solution for this problem feel free to add an illuminating comment.

Installing the PHP redis extension on Mac OS X

2010-05-15T07:53:00.002+02:00

Recently I took a look at Redis, a popular and advanced key-value store. Peeking at the supported languages section of the project's website you'll notice a lot of client libraries available for PHP. Two out of them caught my particular attention: Rediska due to it's impressive Zend Framework integration and phpredis as it's a native PHP extension written in C and therefore supposed to be blazingly faster than vanilla PHP client libraries. The following blog post will show how to install and configure the aforementioned, native PHP extension on a Mac OS X system.

The next steps assume that you've installed redis on your machine. In case you are using MacPorts and haven't installed the key-value store yet, all it takes are the following two commands and you're good to go. In case you prefer Homebrew for managing your package/software installations, there's also a Formula for redis available that allows you to install it via brew install redis.

sudo port install redis
sudo launchctl load -w /Library/LaunchDaemons/org.macports.redis.plist

The very first step for building the native PHP redis extension is to get the source code by cloning the GitHub repository of the extension without it's history revisions.

mkdir phpredis-build
cd phpredis-build
git clone --depth 1 git://github.com/nicolasff/phpredis.git
cd phpredis

The next task is to compile the extension with the following batch of commands.

phpize
./configure
make
sudo make install

The next to last step is to alternate your php.ini, use php --ini | grep 'Loaded' to get the location of it on your system, so that the redis module/extension is available to your PHP ecosystem. Therefor simply add extension=redis.so in the Dynamic Extensions section of your php.ini. Afterwards you can verify that the redis module is loaded and available via one of the following commands.

php -m | grep redis
php -i | grep 'Redis Support'

To make the extension also available to the running Apache PHP module you'll need to restart the Apache server. Looking at phpinfo()'s output in a browser you should see the entry shown in the next image.

For testing the communication between the just installed redis extension and the running Redis server, I further created a simple test script called redis-glue-test.php you can fetch from GitHub and run via the next commands.

curl -s http://gist.github.com/raw/402018/redis-glue-test.php -o redis-glue-test.php
php redis-glue-test.php

When you see the following shown console output you're good to go. Happy Redising!

Using MongoHq in Zend Framework based applications

2010-03-16T22:32:00.009+01:00

As the name slightly foreshadows MongoHq is a currently bit pricey cloud-based hosting solution for MongoDb databases provided by CommonThread. Since they went live a few weeks ago I signed up for the small plan and started to successfully re-thinker with it in an exploratory Zend Framework based application.

Therefore the following post will show how to bootstrap such an instance into a Zend Framework based application and how to use it from there in some simple scenarios like storing data coming from a Zend_Form into a designated collection and vice versa fetching it from there.

Bootstrapping a MongoHq enabled connection

To establish and make the MongoDb connection application-wide available the almighty Zend_Application component came to the rescue again. After reading Matthew Weier O'Phinney's enlightening blog post about creating re-usable Zend_Application resource plugins and deciding to use MongoDb in some more exploratory projects, I figured it would be best to create such a plugin and ditch the also possible resource method approach.

The next code listing shows a possible implementation of the MongoDb resource plugin initializing a Mongo instance for the given APPLICATION_ENV (i.e. production) mode.

For the other application environment modes (development | testing | staging) it's currently assumed that no database authentication is enabled, which is also the default when using MongoDb, so you might need to adapt the plugin to your differing needs; and since I'm currently only rolling on the small plan the support for multiple databases is also not accounted for.

library/Recordshelf/Resource/MongoDb.php

<?php

class Recordshelf_Resource_MongoDb 
    extends Zend_Application_Resource_ResourceAbstract
{
    /**
     * Definable Mongo options.
     *
     * @var array
     */
    protected $_options = array(
        'hostname'     => '127.0.0.1',
        'port'         => '27017',
        'username'     => null,
        'password'     => null,
        'databasename' => null,
        'connect'      => true
    );
    /**
     * Initalizes a Mongo instance.
     *
     * @return Mongo
     * @throws Zend_Exception
     */
    public function init()
    {
        $options = $this->getOptions();

        if (null !== $options['username'] && 
            null !== $options['password'] &&
            null !== $options['databasename'] &&
            'production' === APPLICATION_ENV) {
            // Database Dns with MongoHq credentials
            $mongoDns = sprintf('mongodb://%s:%s@%s:%s/%s',
                $options['username'],
                $options['password'],
                $options['hostname'],
                $options['port'],
                $options['databasename']
            );
        } elseif ('production' !== APPLICATION_ENV) {
            $mongoDns = sprintf('mongodb://%s:%s/%s',
                $options['hostname'],
                $options['port'],
                $options['databasename']
            );
        } else {
            $exceptionMessage = sprintf(
                'Recource %s is not configured correctly',
                __CLASS__
            );
            throw new Zend_Exception($exceptionMessage);
        }
        try {
            return new Mongo($mongoDns, array('connect' => $options['connect']));
        } catch (MongoConnectionException $e) {
            throw new Zend_Exception($e->getMessage());
        }
    }
}

With the MongoDb resource plugin in the place to be, it's time to make it known to the boostrapping mechanism which is done by registering the resource plugin in the application.ini.

Further the MongoHq credentials, which are available in the MongoHq > My Database section, and the main database name are added to the configuration file which will be used to set the definable resource plugin ($_)options and to connect to the hosted database.

application/configs/application.ini


[production]
pluginPaths.Recordshelf_Resource = "Recordshelf/Resource"
resources.mongodb.username = __MONGOHQ_USERNAME__
resources.mongodb.password = __MONGOHQ_PASSWORD__
resources.mongodb.hostname = __MONGOHQ_HOSTNAME__
resources.mongodb.port = __MONGOHQ_PORT__
resources.mongodb.databasename = __MONGOHQ_DATABASENAME__

...

Cloudifying documents into collections

Having the MongoHq enabled connection in the bootstrapping mechanism it can now be picked up from there and used in any Zend Framework application context.

The example action method (i.e. proposeAction) assumes data (i.e. a tech talk proposal to revive the example domain from my last blog post) coming from a Zend_Form which will be stored in a collection named proposals, a table in old relational database think.

The next code listings states the action method innards to do so by injecting the valid form values into a model class which provides accessors and mutators for the domain model's properties and can transform them into a proposal document aka an array structure.

application/controllers/ProposalController.php

<?php

class ProposalController extends Zend_Controller_Action
{
    public function indexAction()
    {
        $this->view->form = new Recordshelf_Form_Proposal();
    }
    public function thanksAction()
    {
    }
    public function proposeAction()
    {
        $this->_helper->viewRenderer->setNoRender();
        $form = new Recordshelf_Form_Proposal();
        
        $request = $this->getRequest();
        
        if ($this->getRequest()->isPost()) {
            if ($form->isValid($request->getPost())) {
                $model = new Recordshelf_Model_Proposal($form->getValues());
                $mapper = new Recordshelf_Model_ProposalMapper();
                if ($mapper->insert($model)) {
                    return $this->_helper->redirector('thanks');
                }
                $this->view->form = $form;
                return $this->render('index');
            } else {
                $this->view->form = $form;
                return $this->render('index');
            }
        }
    }
}

Next the model/data mappper is initialized, which triggers the picking of the MongoHq enabled Mongo connection instance and the auto-determination of the collection name to use based on the mapper's class name. Subsequently the populated model instance is passed into the mappper's insert method which is pulling the document (array structure) and doing the actual insert into the proposals collection.

To give you an idea of the actual document structure it's shown in the next listing, followed by the model/data mapper implementation.

Array
(
    [state] => new
    [created] => MongoDate Object
        (
            [sec] => 1268774242
            [usec] => 360831
        )

    [submitee] => Array
        (
            [title] => Mr
            [firstname] => John
            [familyname] => Doe
            [email] => john.doe@gmail.com
            [twitter] => johndoe
        )

    [title] => How to get a real name
    [description] => Some descriptive text...
    [topictags] => Array
        (
            [0] => John
            [1] => Doe
            [2] => Anonymous
        )

)

application/models/ProposalMapper.php

<?php

class Recordshelf_Model_ProposalMapper
{
    private $_mongo;
    private $_collection;
    private $_databaseName;
    private $_collectionName;

    public function __construct()
    {
        $frontController = Zend_Controller_Front::getInstance();
        $this->_mongo = $frontController->getParam('bootstrap')
                                        ->getResource('mongoDb');
        $config       = $frontController->getParam('bootstrap')
                                        ->getResource('config');
        
        $this->_databaseName   = $config->resources->mongodb->get('databasename');
        
        $replaceableClassNameparts = array(
            'recordshelf_model_', 
            'mapper'
        ); 
        $this->_collectionName = str_replace($replaceableClassNameparts, '',
            strtolower(__CLASS__) . 's');

        $this->_collection = $this->_mongo->selectCollection(
            $this->_databaseName, 
            $this->_collectionName
        );
    }
    /**
     * Inserts a proposal document/model into the proposals collection.
     *
     * @param  Recordshelf_Model_Proposal $proposal The proposal document/model.
     * @return MongoId
     * @throws Zend_Exception
     */
    public function insert(Recordshelf_Model_Proposal $proposal)
    {
        $proposalDocument = $proposal->getValues();
        try {
            if ($this->_collection->insert($proposalDocument, true)) {
                return $proposalDocument['_id'];
            }
        } catch (MongoCursorException $mce) {
            throw new Zend_Exception($mce->getMessage());
        }
    }
}

Querying and retrieving the cloudified data

As what comes in must come out, the next interaction with the Document Database Management System (DocDBMS) is about retrieving some afore-stored talk proposal documents from the collection so they can be rendered to the application's user. This isn't really MongoHq specific anymore, like most of the previous model parts, and is just here to round up this blog post and use some more of that MongoDb goodness. Looks like I have to look for an anonymous self-help group that stuff is highly addictive.

Anyway the next listing shows the action method fetching all stored documents available in the proposals collection. To save some CO2 on this blog post all documents are fetched, which ends up in the most trivial query but as you can figure the example domain provides a bunch of query examples like only proposals for a given topic tag, specific talk title or a given proposal state which can be easily created via passed-through Http request parameters.

application/controllers/ProposalController.php

<?php

class ProposalController extends Zend_Controller_Action
{
    ...
    
    public function listAction()
    {
        $mapper = new Recordshelf_Model_ProposalMapper();
        $proposals = $mapper->fetchAll();
        // For iterating the Recordshelf_Model_Proposal's in the view
        $this->view->proposals = $proposals;
    }
}

The last code listing shows the above used fetchAll method of the data mapper class returning an array of stored proposal documents mapped to their domain model (i.e. Recordshelf_Model_Proposal) in the application.

application/models/ProposalMapper.php

<?php

class Recordshelf_Model_ProposalMapper
{
    ...
    
    /**
     * Fetches all stored talk proposals.
     *
     * @return array
     */
    public function fetchAll()
    {
        $cursor = $this->_collection->find();
        $proposals = array();
        
        foreach ($cursor as $documents) {
            $proposal = new Recordshelf_Model_Proposal();
            foreach ($documents as $property => $value) {
                if ('submitee' === $property) {
                    $proposal->submitee = new Recordshelf_Model_Submitee($value);
                } else {
                    $proposal->$property = $value;
                }
            }
            $proposals[] = $proposal;
        }
        return $proposals;
    }
}

Utilizing Twitter lists with Zend_Service_Twitter

2010-02-05T02:05:00.002+01:00

Several months ago Twitter added the list feature to it's public API. While debating some use cases for an event registration application I stumbled upon an interesting feature, which adds participants automatically to a Twitter list upon registration. This way registered and interested users can discover like-minded individuals and get in touch prior to any pre-social event activities. This post will show how this feature can be implemented by utilizing the Zend_Service_Twitter component, and how it then can be used in a Zend Framework based application.

Implementing the common list features

Looking at the three relevant parts of the Twitter list API some common features emerged and had to be supported to get the feature out of the door. These are namely the creation, deletion of new lists and the addition, removal of list members (i.e. event participants). Since the current Twitter component doesn't support these list operations out of the box it was time to put that develeoper hat on and get loose; which was actually a joy due to the elegance of the extended Zend_Service_Twitter component laying all the groundwork.

A non-feature-complete implementation is shown in the next code listing and can alternatively be pulled from GitHub. Currently it only supports the above stated common operations plus the ability to get the lists of a Twitter account and it's associated members; but feel free to fork it or even turn it into an official proposal.

<?php

require_once 'Zend/Service/Twitter.php';
require_once 'Zend/Service/Twitter/Exception.php';

class Recordshelf_Service_Twitter_List extends Zend_Service_Twitter
{
    const LIST_MEMBER_LIMIT = 500;
    const MAX_LIST_NAME_LENGTH = 25;
    const MAX_LIST_DESCRIPTION_LENGTH = 100;
    
    /**
     * Initializes the service and adds the list to the method types 
     * of the parent service class.
     *
     * @param string $username The Twitter account name.
     * @param string $password The Twitter account password.
     * @see Zend_Service_Twitter::_methodTypes
     */
    public function __construct($username = null, $password = null)
    {
        parent::__construct($username, $password);
        $this->_methodTypes[] = 'list';
    }
    /**
     * Creates a list associated to the current user.
     *
     * @param string $listname The listname to create.
     * @param array $options The options to set whilst creating the list. 
     * Allows to set the list creation mode (public|private) 
     * and the list description.
     * @return Zend_Rest_Client_Result
     * @throws Zend_Service_Twitter_Exception
     */
    public function create($listname, array $options = array())
    {
        $this->_init();
        
        if ($this->_existsListAlready($listname)) {
            $exceptionMessage = 'List with name %s exists already';
            $exceptionMessage = sprintf($exceptionMessage, $listname);
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        
        $_options = array('name' => $this->_validListname($listname));
        foreach ($options as $key => $value) {
            switch (strtolower($key)) {
                case 'mode':
                    $_options['mode'] = $this->_validMode($value);
                    break;
                case 'description':
                    $_options['description'] = $this->_validDescription($value);
                    break;
                default:
                    break;
            }
        }
        $path = '/1/%s/lists.xml';
        $path = sprintf($path, $this->getUsername());
        
        $response = $this->_post($path, $_options);
        return new Zend_Rest_Client_Result($response->getBody());
    }
    /**
     * Deletes an owned list of the current user.
     *
     * @param string $listname The listname to delete.
     * @return Zend_Rest_Client_Result
     * @throws Zend_Service_Twitter_Exception
     */
    public function delete($listname)
    {
        $this->_init();
        
        if (!$this->_isListAssociatedWithUser($listname)) {
            $exceptionMessage = 'List %s is not associate with user %s ';
            $exceptionMessage = sprintf($exceptionMessage, 
                $listname, 
                $this->getUsername()
            );
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        $_options['_method'] = 'DELETE';
        $path = '/1/%s/lists/%s.xml';
        $path = sprintf($path, 
            $this->getUsername(), 
            $this->_validListname($listname)
        );
        $response = $this->_post($path, $_options);
        return new Zend_Rest_Client_Result($response->getBody());
    }
    /**
     * Adds a member to a list of the current user.
     *
     * @param integer $userId The numeric user id of the member to add.
     * @param string $listname The listname to add the member to.
     * @return Zend_Rest_Client_Result
     * @throws Zend_Service_Twitter_Exception
     */
    public function addMember($userId, $listname)
    {
        $this->_init();
        
        if (!$this->_isListAssociatedWithUser($listname)) {
            $exceptionMessage = 'List %s is not associate with user %s ';
            $exceptionMessage = sprintf($exceptionMessage, 
                $listname, 
                $this->getUsername()
            );
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        
        $_options['id'] = $this->_validInteger($userId); 
        $path = '/1/%s/%s/members.xml';
        $path = sprintf($path, 
            $this->getUsername(), 
            $this->_validListname($listname)
        );
        
        if ($this->_isListMemberLimitReached($listname)) {
            $exceptionMessage = 'List can contain no more than %d members';
            $exceptionMessage = sprintf($exceptionMessage, 
                self::LIST_MEMBER_LIMIT
            );
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        
        $response = $this->_post($path, $_options);
        return new Zend_Rest_Client_Result($response->getBody());
    }
    /**
     * Removes a member from a list of the current user.
     *
     * @param integer $userId The numeric user id of the member to remove.
     * @param string $listname The listname to remove the member from.
     * @return Zend_Rest_Client_Result
     * @throws Zend_Service_Twitter_Exception
     */
    public function removeMember($userId, $listname)
    {
        $this->_init();
        
        if (!$this->_isListAssociatedWithUser($listname)) {
            $exceptionMessage = 'List %s is not associate with user %s ';
            $exceptionMessage = sprintf($exceptionMessage, 
                $listname, 
                $this->getUsername()
            );
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        
        $_options['_method'] = 'DELETE';
        $_options['id'] = $this->_validInteger($userId);        
        $path = '/1/%s/%s/members.xml';
        $path = sprintf($path, 
            $this->getUsername(), 
            $this->_validListname($listname)
        );
        $response = $this->_post($path, $_options);
        return new Zend_Rest_Client_Result($response->getBody());
    }
    /**
     * Fetches the list members of the current user.
     *
     * @param string $listname The listname to fetch members from.
     * @return Zend_Rest_Client_Result
     * @throws Zend_Service_Twitter_Exception
     */
    public function getMembers($listname) {
        $this->_init();
        $path = '/1/%s/%s/members.xml';
        $path = sprintf($path, 
            $this->getUsername(), 
            $this->_validListname($listname)
        );
        $response = $this->_get($path);
        return new Zend_Rest_Client_Result($response->getBody());
    }
    /**
     * Fetches the list of the current user or any given user.
     *
     * @param string $username The username of the list owner.
     * @return Zend_Rest_Client_Result
     */
    public function getLists($username = null)
    {
        $this->_init();
        $path = '/1/%s/lists.xml';
        if (is_null($username)) {
            $path = sprintf($path, $this->getUsername());
        } else {
            $path = sprintf($path, $username);
        }
        $response = $this->_get($path);
        return new Zend_Rest_Client_Result($response->getBody());
    }
    /**
     * Checks if the list exists already to avoid number 
     * indexed recreations.
     *
     * @param string $listname The list name.
     * @return boolean
     * @throws Zend_Service_Twitter_Exception
     */
    private function _existsListAlready($listname)
    {
        $_listname = $this->_validListname($listname);
        $lists = $this->getLists();
        $_lists = $lists->lists;
        foreach ($_lists->list as $list) {
            if ($list->name == $_listname) {
                return true;
            }
        }
        return false;
    }
    /**
     * Checks if the list is associated with the current user.
     *
     * @param string $listname The list name.
     * @return boolean
     */
    private function _isListAssociatedWithUser($listname) 
    {
        return $this->_existsListAlready($listname);
    }
    /**
     * Checks if the list member limit is reached.
     *
     * @param string $listname The list name.
     * @return boolean
     */
    private function _isListMemberLimitReached($listname)
    {
        $members = $this->getMembers($listname);
        return self::LIST_MEMBER_LIMIT < count($members->users->user);
    }
    /**
     * Returns the list creation mode or returns the private mode when invalid.
     * Valid values are private or public.
     *
     * @param string $creationMode The list creation mode.
     * @return string
     */
    private function _validMode($creationMode)
    {
        if (in_array($creationMode, array('private', 'public'))) {
            return $creationMode;
        }
        return 'private';
    }
    /**
     * Returns the list name or throws an Exception when invalid.
     *
     * @param string $listname The list name.
     * @return string
     * @throws Zend_Service_Twitter_Exception
     */
    private function _validListname($listname)
    {
        $len = iconv_strlen(trim($listname), 'UTF-8');
        if (0 == $len) {
            $exceptionMessage = 'List name must contain at least one character';
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        } elseif (self::MAX_LIST_NAME_LENGTH < $len) {
            $exceptionMessage = 'List name must contain no more than %d characters';
            $exceptionMessage = sprintf($exceptionMessage, 
                self::MAX_LIST_NAME_LENGTH
            );
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        return trim($listname);
    }
    /**
     * Returns the list description or throws an Exception when invalid.
     *
     * @param string $description The list description.
     * @return string
     * @throws Zend_Service_Twitter_Exception
     */
    private function _validDescription($description)
    {
        $len = iconv_strlen(trim($description), 'UTF-8');
        if (0 == $len) {
            return '';
        } elseif (self::MAX_LIST_DESCRIPTION_LENGTH < $len) {
            $exceptionMessage = 'List description must contain no more than %d characters';
            $exceptionMessage = sprintf($exceptionMessage, 
                self::MAX_LIST_DESCRIPTION_LENGTH
            );
            throw new Zend_Service_Twitter_Exception($exceptionMessage);
        }
        return trim(strip_tags($description));
    }
}

Adding the 'auto list' feature

For using the above implemented add member feature it's assumed that a participant has provided a valid and existing Twitter username, his approval of being added to the event list (i.e. zfweekend) and that he further has been registered effectively. To have the name of the Twitter list to act on and the account credentials available corresponding configuration entries are set as shown next.

application/configs/application.ini


[production]
twitter.username = __USERNAME__
twitter.password = __PASSWORD__
twitter.auto.listname = zfweekend

With the Twitter credentials and the list name available it's now possible to pull this feature into the register method of the register Action Controller, where it's applied as shown in the outro listing. As you will see, besides some bad practices due to demonstration purposes, the register Form makes use of a custom TwitterScreenName validator and filter which are also available via GitHub. Happy Twitter listing!

<?php

class RegisterController extends Zend_Controller_Action
{   
    /**
     * @badpractice Push this into a specific Form class.
     * @return Zend_Form
     */
    private function _getForm()
    {
        $form = new Zend_Form();
        $form->setAction('/register/register')
             ->setMethod('post');
        $twitterScreenName = $form->createElement('text', 'twitter_screen_name', 
            array('label' => 'Twittername: ')
        );
        $twitterScreenName->addValidator(new Recordshelf_Validate_TwitterScreenName())
            ->setRequired(true)
            ->setAllowEmpty(false) 
            ->addFilter(new Recordshelf_Filter_TwitterScreenName());

        $autoListApproval = $form->createElement('checkbox', 'auto_list_approval', 
            array('label' => 'I approved to be added to the event Twitter list: ')
        );
        
        $form->addElement($twitterScreenName)
             ->addElement($autoListApproval)
             ->addElement('submit', 'register', array('label' => ' Register '));
        
        return $form;
    }
    public function indexAction()
    {
        $this->view->form = $this->_getForm();
    } 
    public function thanksAction()
    {
    }
    /**
     * @badpractice Handle possible Exception of
     * Recordshelf_Service_Twitter_List::addMember.
     * @return Recordshelf_Service_Twitter_List
     */
    public function registerAction()
    {
        $this->_helper->viewRenderer->setNoRender();
        $form = $this->_getForm();
        $request = $this->getRequest();
                
        if ($this->getRequest()->isPost()) {
            if ($form->isValid($request->getPost())) {
                
                $model = new Recordshelf_Model_Participant($form->getValues());
                $model->save();

                if ($form->getElement('auto_list_approval')->isChecked()) {
                    $twitterScreenName = $form->getValue('twitter_screen_name');
                    $twitter = $this->_getTwitterListService();
                    $response = $twitter->user->show($twitterScreenName);
                    $userId = (string) $response->id;
                    $response = $twitter->list->addMember($userId, 
                        $this->_getTwitterListName());
                    
                    $model->hasBeenAddedToTwitterList(true);
                    $model->update();

                    return $this->_helper->redirector('thanks');
                }
            } else {
                return $this->_helper->redirector('index');
            }
        }
    }
    /**
     * @badpractice Push this into a dedicated Helper or something similar.
     * @return Recordshelf_Service_Twitter_List
     */
    private function _getTwitterListService()
    {
        $config = Zend_Registry::get('config');
        return new Recordshelf_Service_Twitter_List(
            $config->twitter->get('username'),
            $config->twitter->get('password')
        );
    }
    /**
     * @badpractice Push this into a dedicated Helper or something similar.
     * @return string
     */
    private function _getTwitterListName()
    {
        $config = Zend_Registry::get('config');
        return $config->twitter->auto->get('listname');
    }
}

Closing and reopening GitHub issues via PHPUnit tests

2010-01-19T22:41:00.003+01:00

Since PHPUnit 3.4.0 a new extension point for interacting with issue tracking systems (TTS) based on the test results has been added to PHP's first choice xUnit framework. The extension point has been introduced by an abstract PHPUnit_Extensions_TicketListener class, which allows developer to add tailor-made ticket listeners supporting their favoured TTS. Currently PHPUnit ships with a single ticket listener for Trac as it's still the used TTS for the framework itself. As I start to become more and more accustomed to use GitHub for some of my exploratory projects and hacks, the following blog post will contain a GitHub_TicketListener implementation and a showcase of it's usage.

Annotating tests with ticket meta data

As you might know, it's considered to be a best practice to write a test for each new ticket representing a bug and drive the system under test (SUT) till the issue is resolved. This extension of test-driven development is also known as test-driven bug fixing. To create a relation between these tests and their associated tickets, PHPUnit provides a new @ticket annotation which will be analyzed before each test is run. The following code listing shows such an annotated test.

<?php
require_once 'PHPUnit/Framework.php';
 
class ExampleTest extends PHPUnit_Framework_TestCase
{
    ....

    /**
     * @ticket 2
     * @test
     */
    public function shouldGuarantyThatTheSutHandlesTheIssueCorrectly()
    {
        // test code
    }
    ....

Peeking at the GitHub_TicketListener implementation

The current version (3.4.6) of PHPUnit has a pending issue regarding the abstract TicketListener class, so the first step is to apply an 'exploratory' patch, which might break the functionality of the shipped Trac ticket listener but will enable the use of the one for GitHub's TTS.

The next step en route to a working GitHub_TicketListener is to extend the patched abstract PHPUnit_Extensions_TicketListener class. This abstract class contains two abstract methods named getTicketInfo and updateTicket which have to be implemented by the specific ticket listener class, and will be responsible for the interaction with the TTS.

The implementation of the getTicketInfo method retrieves the ticket status for the annotated ticket, while the updateTicket method is responsible for changing the ticket status based on the test result and the former ticket state. Both implementations make use of the relevant TTS part of the GitHub API by utilizing PHP's curl extension as shown in the next code listing which alternatively is available via this gist.

<?php
require_once('PHPUnit/Extensions/TicketListener.php');
PHPUnit_Util_Filter::addFileToFilter(__FILE__, 'PHPUNIT');

/**
 * A ticket listener that interacts with the GitHub issue API.
 */
class PHPUnit_Extensions_TicketListener_GitHub extends 
    PHPUnit_Extensions_TicketListener
{
    const STATUS_CLOSE = 'closed';
    const STATUS_REOPEN = 'reopened';
    
    private $_username = null;
    private $_apiToken = null;
    private $_repository = null;
    private $_apiPath = null;
    private $_printTicketStateChanges = false;
    
    /**
     * @param string $username   The username associated with the GitHub account.
     * @param string $apiToken   The API token associated with the GitHub account.
     * @param string $repository The repository of the system under test (SUT) on GitHub.
     * @param string $printTicketChanges Boolean flag to print the ticket state 
     * changes in the test result.
     * @throws RuntimeException
     */
    public function __construct($username, $apiToken, $repository, 
        $printTicketStateChanges = false)
    {
        if ($this->_isCurlAvailable() === false) {
            throw new RuntimeException('The dependent curl extension is not available');
        }
        if ($this->_isJsonAvailable() === false) {
            throw new RuntimeException('The dependent json extension is not available');
        }
        $this->_username = $username;
        $this->_apiToken = $apiToken;
        $this->_repository = $repository;
        $this->_apiPath = 'http://github.com/api/v2/json/issues';
        $this->_printTicketStateChanges = $printTicketStateChanges;
    }
    
    /**
     * @param  integer $ticketId 
     * @return string
     * @throws PHPUnit_Framework_Exception
     */
    public function getTicketInfo($ticketId = null) 
    {
        if (!ctype_digit($ticketId)) {
            return $ticketInfo = array('status' => 'invalid_ticket_id');
        }                
        $ticketInfo = array();
        
        $apiEndpoint = "{$this->_apiPath}/show/{$this->_username}/"
            . "{$this->_repository}/{$ticketId}";
            
        $issueProperties = $this->_callGitHubIssueApiWithEndpoint($apiEndpoint, true);

        if ($issueProperties['state'] === 'open') {
            return $ticketInfo = array('status' => 'new');
        } elseif ($issueProperties['state'] === 'closed') {
            return $ticketInfo = array('status' => 'closed');
        } elseif ($issueProperties['state'] === 'unknown_ticket') {
            return $ticketInfo = array('status' => $issueProperties['state']);
        }
    }

    /**
     * @param string $ticketId   The ticket number of the ticket under test (TUT).
     * @param string $statusToBe The status of the TUT after running the associated test.
     * @param string $message    The additional message for the TUT.
     * @param string $resolution The resolution for the TUT.
     * @throws PHPUnit_Framework_Exception
     */
    protected function updateTicket($ticketId, $statusToBe, $message, $resolution)
    {
        $apiEndpoint = null;
        $acceptedResponseIssueStates = array('open', 'closed');
        
        if ($statusToBe === self::STATUS_CLOSE) {
            $apiEndpoint = "{$this->_apiPath}/close/{$this->_username}/"
                . "{$this->_repository}/{$ticketId}";
        } elseif ($statusToBe === self::STATUS_REOPEN) {
            $apiEndpoint = "{$this->_apiPath}/reopen/{$this->_username}/"
                . "{$this->_repository}/{$ticketId}";
        }
        if (!is_null($apiEndpoint)) {
            $issueProperties = $this->_callGitHubIssueApiWithEndpoint($apiEndpoint);
            if (!in_array($issueProperties['state'], $acceptedResponseIssueStates)) {
                throw new PHPUnit_Framework_Exception(
                    'Recieved an unaccepted issue state from the GitHub Api');
            }
            if ($this->_printTicketStateChanges) {
                printf("\nUpdating GitHub issue #%d, status: %s\n", $ticketId, 
                    $statusToBe);
            }
        }
    }

    /**
     * @return boolean 
     */
    private function _isCurlAvailable()
    {
        return extension_loaded('curl');
    }

    /**
     * @return boolean 
     */
    private function _isJsonAvailable()
    {
        return extension_loaded('json');
    }

    /**
     * @param string  $apiEndpoint API endpoint to call against the GitHub issue API.
     * @param boolean $isShowMethodCall Show method of the GitHub issue API is called? 
     * @return array
     * @throws PHPUnit_Framework_Exception
     */
    private function _callGitHubIssueApiWithEndpoint($apiEndpoint, 
        $isShowMethodCall = false) 
    {
            $curlHandle = curl_init();

            curl_setopt($curlHandle, CURLOPT_URL, $apiEndpoint);
            curl_setopt($curlHandle, CURLOPT_FOLLOWLOCATION, true);
            curl_setopt($curlHandle, CURLOPT_FAILONERROR, true);
            curl_setopt($curlHandle, CURLOPT_FRESH_CONNECT, true);
            curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, true);
            curl_setopt($curlHandle, CURLOPT_HTTPPROXYTUNNEL, true);
            curl_setopt($curlHandle, CURLOPT_USERAGENT, __CLASS__);  
            curl_setopt($curlHandle, CURLOPT_POSTFIELDS,
                "login={$this->_username}&token={$this->_apiToken}");

            $response = curl_exec($curlHandle);
            
            // Unknown tickets throw a 403 error
            if (!$response && $isGetTicketInfoCall) {
                return array('state' => 'unknown_ticket');
            }

            if (!$response) {
                $curlErrorMessage = curl_error($curlHandle);
                $exceptionMessage = "A failure occured while talking to the "
                    . "GitHub issue Api. {$curlErrorMessage}.";
                throw new PHPUnit_Framework_Exception($exceptionMessage);
            }
            $issue = (array) json_decode($response);
            $issueProperties = (array) $issue['issue'];
            curl_close($curlHandle);
            return $issueProperties;
    }
}

Plugging the GitHub_TicketListener into the PHPUnit test environment

To hook the GitHub ticket listener into the test runtime environment PHPUnit provides several approaches to do so. The chosen approach makes use of a XML configuration file which allows an injection of the ticket listener in a declarative manner. As you will see in the configuration file snippet, the GitHub ticket listener is initialized with four parameters: The first one is the GitHub username, followed by the GitHub API token, the associated GitHub project, and a boolean flag for displaying the ticket status changes in the test result.

<phpunit>
  <listeners>
    <listener class="PHPUnit_Extensions_TicketListener_GitHub" 
              file="/path/to/GitHubTicketListener.php">
      <arguments>
        <string>raphaelstolt</string>
        <string>API_TOKEN</string>
        <string>PROJECT_NAME</string>
        <boolean>true</boolean>
      </arguments>
    </listener>
  </listeners>
</phpunit>

To run the tests against a SUT and see the PHPUnit GitHub TTS interaction at work, all it takes is the forthcoming PHPUnit Cli call.

phpunit --configuration github-ticketlistener.xml ExampleTest.php

The outro screenshot shows the test result for an example SUT along with a GitHub TTS interaction due to a passing test which is associated with a open ticket in the TTS.

A final note: As the interaction with an TTS adds some overhead to the test execution and thereby might cause Slow Tests, ticket listener should only be considered in non time-critical test scenarios (e.g. nightly builds).

Zend Framework 1.8 Web Application Development book review

2009-10-14T23:22:00.005+02:00

As the days are rapidly getting shorter, my reading appetite grows potentially and this evening I finished the 'Zend Framework 1.8 Web Application Development' book written by Keith Pope. While Keith worked on the book, I peeked several times at it's tutorial application, dubbed the Storefront, to get me going with the new Zend_Application component. Looking at it's code made me feel certain to get another great digest of the new features and components of version 1.8, and also a different practical perspective on web application development with the Zend Framework, once the book has been published. Therefor I got in touch with the publisher Packt and fortunately got a copy of which I'd like to share a personal review in this blog post.

What's in it?

The book opens with a quick run-through of the Model-View-Controller (MVC) architecture by creating a project structure via Zend_Tool and building a first very basic web application. While this introduction intentionally skips over a lot of details, the following chapter provides very detailed insights into the Zend Framework's MVC components by explaining the surrounded objects, the Design Patterns they are based upon and their interactions.

After laying out that hefty block of theory the aforementioned tutorial application is introduced and built incrementally over several chapters; each one going into more detail for the specific application aspect. The highlight content of these chapters reach from introducing the Fat Model Skinny Controller concept, thoughts on Model design strategies which are reflected in a custom Storefront Model design, to developing application specific Front Controller Plugins, Action-Helpers, and View-Helpers. The application walk-through is completed by looking at general techniques to optimize the Storefront application and by building an automated PHPUnit Test Suite of functional tests utilizing Zend_Test to keep the Zend Framework based application self-reliant and refactorable.

Conclusion

The book by Keith Pope provides any interested PHP developer, who's not already sold on a specific framework, a thorough introduction to the vivid Zend Framework and it's use in a MVC based web application development context. The content of the book is delivered in a fluent, very enthusiastic and 'knowledge-pillowed' writing tone. By implementing or working through the Storefront application seasoned web developers using older versions of the Framework will get a good blue sheet on new components like Zend_Application and it's implication in the bootstrapping process; while new developers tending towards picking up the Zend Framework will get a current and well compiled guide, which might first start off with a steep learning-curve but will turn into profund knowledge once hanging in there.

The only thing that seemed a bit odd to me, was the utilization of Ant instead of Phing as the build tool for the Storefront application to set the application environment, to remove all require_once statements from the framework library and to run the PHPUnit Test Suite; but this might also be inflicted by my Phing nuttiness.

Logging to MongoDb and accessing log collections with Zend_Tool

2009-09-19T21:00:00.018+02:00

Influenced by a recent blog post of a colleague of mine and by being kind of broke on a Saturday night; I tinkered with the just recently discovered MongoDb and hooked it into the Zend_Log environment by creating a dedicated Zend_Log_Writer. The following post will therefore present a peek at a prototypesque implementation of this writer and show how the afterwards accumulated log entries can be accessed and filtered with a custom Zend_Tool project provider.

Logging to a MongoDb database

The following steps assume that an instance of a MongoDb server is running and that the required PHP MongoDb module is also installed and loaded. To by-pass log entries to a MongoDb database there is a need to craft a proper Zend_Log_Writer. This can be achieved by extending the Zend_Log_Writer_Abstract class, injecting a Mongo connection instance and implementing the actual write functionality as shown in the next listing.

<?php
require_once 'Zend/Log/Writer/Abstract.php';

class Recordshelf_Log_Writer_MongoDb extends Zend_Log_Writer_Abstract
{
    private $_db;
    private $_connection;

   /**
    * @param Mongo $connection The MongoDb database connection
    * @param string $db The MongoDb database name
    * @param string $collection The collection name string the log entries 
    */
    public function __construct(Mongo $connection, $db, $collection)
    {
        $this->_connection = $connection;
        $this->_db = $this->_connection->selectDB($db)->createCollection(
            $collection
        );
    }
    public function setFormatter($formatter)
    {
        require_once 'Zend/Log/Exception.php';
        throw new Zend_Log_Exception(get_class() . ' does not support formatting');
    }
    public function shutdown()
    {
        $this->_db = null;
        $this->_connection->close();
    }
    protected function _write($event)
    {
        $this->_db->insert($event);
    }
   /**
    * Create a new instance of Recordshelf_Log_Writer_MongoDb
    * 
    * @param  array|Zen_Config $config
    * @return Recordshelf_Log_Writer_MongoDb
    * @throws Zend_Log_Exception
    * @since  Factory Interface available since release 1.10.0
    */
    static public function factory($config) 
    {
        $exceptionMessage = 'Recordshelf_Log_Writer_MongoDb does not currently '
            . 'implement a factory';
        throw new Zend_Exception($exceptionMessage);
    }
}

With the MongoDb writer available and added to the library directory of the application it's now possible to utilize this new storage backend as usual with the known Zend_Log component. The Mongo connection injected into the writer is configured via Zend_Config and initialized via the Zend_Application bootstrapping facility as shown in the listings below.

application/configs/application.ini

[production]
app.name = recordshelf

....

log.mongodb.db = zf_mongo
log.mongodb.collection = recordshelf_log
log.mongodb.server = localhost
log.priority = Zend_Log::CRIT

....

application/Bootstrap.php

<?php

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    protected $_logger;

    protected function _initConfig()
    {
        Zend_Registry::set('config', new Zend_Config($this->getOptions()));
    }

    protected function _initLogger()
    {
        $this->bootstrap(array('frontController', 'config'));
        $config = Zend_Registry::get('config');     

        $applicationName = $config->app->get('name', 'recordshelf');
        $mongoDbServer = $config->log->mongodb->get('server', '127.0.0.1');
        $mongoDbName = $config->log->mongodb->get('db', "{$applicationName}_logs");
        $mongoDbCollection = $config->log->mongodb->get('collection', 'entries');

        $logger = new Zend_Log();
        $writer = new Recordshelf_Log_Writer_MongoDb(new Mongo($mongoDbServer), 
        $mongoDbName, $mongoDbCollection);

        if ('production' === $this->getEnvironment()) {
            $priority = constant($config->log->get('priority', Zend_Log::CRIT));
            $filter = new Zend_Log_Filter_Priority($priority);
            $logger->addFilter($filter);
        }
        $logger->addWriter($writer);
        $this->_logger = $logger;
        Zend_Registry::set('log', $logger);
    }
}

controllers/ExampleController.php

<?php

class ExampleController extends Zend_Controller_Action
{
    private $_logger = null;

    public function init()
    {
        $this->_logger = Zend_Registry::get('log');
    }

    public function fooAction()
    {
        $this->_logger->log('A debug log message from within action ' . 
            $this->getRequest()->getActionName(), Zend_Log::DEBUG);
    }

    public function barAction()
    {
        $this->_logger->log('A debug log message from within ' . 
            __METHOD__, Zend_Log::DEBUG);
    }
}

Accessing the log database with a Zend_Tool project provider

After handling the application-wide logging with the MongoDb writer sooner or later the issue to access the gathered log entries will rise. For this mundane and recurring use case the ProjectProvider provider of the Zend_Tool framework is an acceptable candidate to hook a custom action into the Zend_Tool environment of a given project. Therefor a new Zend_Tool_Project Project provider is first scaffolded via the forthcoming command.

sudo zf create project-provider mongodb-logs filter

Second the generated provider skeleton its filter action is enliven with the logic to query the MongoDb database and the stored log collection. The action to come accepts three arguments to filter the stored log entry results by a specific date in the format of 'YYYY-MM-DD' and a given Zend_Log priority (currently limited to the constants defined in Zend_Log) in a specific application environment. The next listing shows the implementation of the import action of the MongodbLogsProvider project provider; which is clearly, as it's length indicates, in need for a clean-up task.

providers/Mongodb-logsProvider.php

<?php
require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';
require_once 'Zend/Date.php';
require_once 'Zend/Validate/Date.php';
require_once 'Zend/Log.php';
require_once 'Zend/Config/Ini.php';

class MongodbLogsProvider extends Zend_Tool_Project_Provider_Abstract
{

    public function filter($date = null, $logPriority = null, 
        $env = 'development')
    {
        $ref = new Zend_Reflection_Class('Zend_Log');
        $logPriorities = $ref->getConstants();

        if (in_array(strtoupper($date), array_keys($logPriorities)) || 
            in_array(strtoupper($date), array_values($logPriorities))) {
            $logPriority = $date;
            $date = null;
        }
        if (!is_null($date)) {
            $validator = new Zend_Validate_Date();
            if (!$validator->isValid($date)) {
                $exceptionMessage = "Given date '{$date}' is not a valid date.";
                throw new Zend_Tool_Project_Provider_Exception($exceptionMessage);
            }
            $dateArray = array();
            list($dateArray['year'], $dateArray['month'], $dateArray['day']) = 
                explode('-', $date);
            $date = new Zend_Date($dateArray);
        } else {
            $date = new Zend_Date();
        }
        $date = $date->toString('Y-MM-dd');

        if (!is_null($logPriority)) {
            if (!is_numeric($logPriority)) {
                $logPriority = strtoupper($logPriority);
                if (!in_array($logPriority, array_keys($logPriorities))) {
                    $exceptionMessage = "Given priority '{$logPriority}' is not defined.";
                    throw new Zend_Tool_Project_Provider_Exception($exceptionMessage);
                } else {
                    $logPriority = $logPriorities[$logPriority];
                }
            }
        if (!in_array($logPriority, array_values($logPriorities))) {
            $exceptionMessage = "Given priority '{$logPriority}' is not defined.";
            throw new Zend_Tool_Project_Provider_Exception();
        }
            $priorities = array_flip($logPriorities);
            $priorityName = $priorities[$logPriority];
        }

        if ($env !== 'development' && $env !== 'production') {
            $exceptionMessage = "Unsupported environment '{$env}' provided.";
            throw new Zend_Tool_Project_Provider_Exception();
        }
        $config = new Zend_Config_Ini('./application/configs/application.ini', 
            $env);

        $applicationName = $config->app->get('name', 'recordshelf');
        $mongoDbServer = $config->log->mongodb->get('server', '127.0.0.1');
        $mongoDbName = $config->log->mongodb->get('db', "{$applicationName}_logs");
        $mongoDbCollection = $config->log->mongodb->get('collection', 'entries');

        try {
            $connection = new Mongo($mongoDbServer);
            $db = $connection->selectDB($mongoDbName)->createCollection(
            $mongoDbCollection);
        } catch (MongoConnectionException $e) {
            throw new Zend_Tool_Project_Provider_Exception($e->getMessage());
        }
        $dateRegex = new MongoRegex("/$date.*/i");

        if (is_null($logPriority)) {
            $query = array('timestamp' => $dateRegex);
            $appendContentForResults = "Found #amountOfEntries# log entrie(s) "
                . "on {$date}";
            $appendContentForNoResults = "Found no log entries on {$date}";
        } else {            
            $query = array('priority' => (int) $logPriority, 
                           'timestamp' => $dateRegex
                     );
            $appendContentForResults = "Found #amountOfEntries# log entrie(s) "
                . "for priority {$priorityName} on {$date}";
            $appendContentForNoResults = "Found no log entries for priority "
                . "{$priorityName} on {$date}";
        }

        $cursor = $db->find($query);
        $amountOfEntries = $cursor->count();

        if ($amountOfEntries > 0) {
            $content = str_replace('#amountOfEntries#', $amountOfEntries, 
                $appendContentForResults);
            $this->_registry->getResponse()->appendContent($content);
            foreach ($cursor as $id => $value) {
                $content = "{$id}: {$value['timestamp']} > ";
                if (is_null($logPriority)) {
                    $content.= "[{$value['priorityName']}] ";
                }
                $content.= "{$value['message']}";
                $this->_registry->getResponse()->appendContent($content);
            }
        } else {
            $content = $appendContentForNoResults;
            $this->_registry->getResponse()->appendContent($content);
        }
        $connection->close();
    }
}

The coming outro screenshots show two use cases for the filter action of the MongodbLogsProvider issued against the zf command line client. The first screenshot shows the use case where all log entries for the current day are queried, while the second one shows the use case where all log entries for a specific date and log priority are queried and fed back to the user.

Kicking off custom Phing task development with TextMate

2009-08-22T15:08:00.004+02:00

As a reader of this blog you migth have noticed that from time to time I like to utilize Phing's ability to write custom tasks. Though that's not an everyday routine for me and therefor I might, depending on my form of the day, end up with some real smelly code where for example the task's properties validation is handled in the task's main worker method. This is actually a bad habit/practice I'm aware of and to improve my future endeavours in custom Phing task development, I bended TextMate's snippet feature to my needs.

Snippets in TextMate are a very powerful feature that can be used to insert code that you do not want to type again and again, or like in my case might have forgotten over a certain time.

The next code listing shows the snippet providing a basic custom Phing task class skeleton which can be utilized over and over at the beginning of the implementation activities.

<?php
require_once 'phing/Task.php';

class ${1:CustomName}Task extends Task
{
    private \$_${2:property} = null;
    
    /**
     * @param string \$${2:property} ${3:description}
     */
    public function set${2/./\u$0/}(\$${2:property})
    {
        \$this->_${2:property} = trim(\$${2:property});
    }
    /**
     * Initializes the task environment if necessary
     */
    public function init()
    {
    }
    /**
     * Does the task main work or delegates it
     * @throws BuildException
     */
    public function main()
    {
        \$this->_validateProperties();
    }
    /**
     * Validates the task properties
     * @throws BuildException
     */
    private function _validateProperties()
    {
        if (is_null(\$this->_${2:property})) {
            throw new BuildException('${4:message}.');
        }$0
    }
}

To apply the snippet, after installing it, on a PHP source file it can either be selected from the Bundles menue or more comfortable via the assigned tab trigger i.e. ctask. After triggering the snippet it's possible to properly name the task under development and dynamically set it's first property, which is also treated as a mandatory property in the extracted _validateProperties method.

The outro image shows the above stated snippet in the TextMate Bundle Editor and it's configuration.

Scaffolding, implementing and using project specific Zend_Tool_Project_Providers

2009-07-04T16:41:00.011+02:00

Working on a project involving several legacy data migration tasks, I got curious what the Zend_Tool_Project component of the Zend Framework offers to create project specific providers for the above mentioned tasks or ones of similar nature. Therefore the following post will try to show how these providers can be developed in an iterative manner by scaffolding them via the capabilities of the Zend_Tool_Project ProjectProvider provider, enlived with action/task logic, and be used in the project scope.

Scaffolding project specific providers

All following steps assume there is a project available i.e. recordshelf initially created with the Zend_Tool_Project Project provider and that the forthcoming commands are issued from the project root directory against the zf command line client. The scaffolding of a project specific provider can be triggered via the create action of the ProjectProvider provider by passing in the name of the provider i.e. csv and it's intended actions. As the next console snippet shows it's
possible to specify several actions as a comma separated list.

sudo zf create project-provider csv importSpecials,importSummersale

After running the command the project's profile .zfproject.xml has been modified and a new providers directory exists in the project root directory containing the scaffolded Csv provider. The next code snippet shows the initial Csv provider class skeleton and its two empty action methods named importSpecials and importSummersale. At the point of this writing, using the Zend Framework 1.8.4 and PHP 5.2.10 on a Mac OS X system the generated Csv provider code or the mapping in the .zfproject.xml is incorrect, but can be fixed by renaming the class from CsvProvider to Csv.

<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class CsvProvider extends Zend_Tool_Project_Provider_Abstract
{

    public function importSpecials()
    {
        /** @todo Implementation */
    }

    public function importSummersale()
    {
        /** @todo Implementation */
    }


}

Implementing the action logic

Having the project provider class skeleton ready to get going, it's time to enliven the actions with their intended features by using either other components of the Zend Framework, any suitable third party library or plain-vanilla PHP. For the sake of brevity I decided to implement only the importSpecials action which transforms the data of a known CSV file structure into a relevant database table. The CSV parsing steps shown next might not be that sophisticated, as their sole purpose is to illustrate an exemplary implementation of a project specific provider action.

<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class Csv extends Zend_Tool_Project_Provider_Abstract
{
    private function _isProjectProviderSupportedInProject(Zend_Tool_Project_Profile $profile, 
        $projectProviderName)
    {
        $projectProviderResource = $this->_getProjectProfileResource($profile, 
            $projectProviderName);
        return $projectProviderResource instanceof Zend_Tool_Project_Profile_Resource;
    }
    
    private function _isActionSupportedByProjectProvider(Zend_Tool_Project_Profile $profile, 
        $projectProviderName, $actionName)
    {
        $projectProviderResource = $this->_getProjectProfileResource($profile, 
            $projectProviderName);
        $projectProviderAttributes = $projectProviderResource->getContext()
                                                             ->getPersistentAttributes();
        return in_array($actionName, explode(',', $projectProviderAttributes['actionNames']));
    }
    
    private function _getProjectProfileResource(Zend_Tool_Project_Profile $profile, 
        $projectProviderName)
    {
        $profileSearchParams[] = 'ProjectProvidersDirectory';
        $profileSearchParams['ProjectProviderFile'] = 
            array('projectProviderName' => strtolower($projectProviderName));
        return $profile->search($profileSearchParams);  
    }

    public function importSpecials($csvFile, $env = 'development')
    {
        $relatedTablename = 'specials';
        
        if (!$this->_isProjectProviderSupportedInProject($profile, __CLASS__)) {
            throw new Exception("ProjectProvider Csv is not supported in this project.");
        }
        if (!$this->_isActionSupportedByProjectProvider($profile, __CLASS__, __FUNCTION__)) {
            $exceptionMessage = "Action 'importSpecials' is not supported by "
                . "the Csv ProjectProvider in this project.";
            throw new Exception($exceptionMessage);
        }

        if (!file_exists($csvFile)) {
            throw new Exception("Given csv-file '{$csvFile}' doesn't exist.");
        }
        
        $importEnvironment = trim($env);
        if ($importEnvironment !== 'development' && $importEnvironment !== 'production') {
            throw new Exception("Unsupported environment '{$importEnvironment}' provided.");
        }
        
        $csvHandle = fopen($csvFile, "r");
        
        if (!$csvHandle) {
            throw new Exception("Unable to open given csv-file '{$csvFile}'.");
        }
        
        $config = new Zend_Config_Ini('./application/configs/application.ini', 
            $importEnvironment);
        $db = Zend_Db::factory($config->database);

        $db->query("TRUNCATE TABLE {$relatedTablename}");
        echo "Truncated the project '{$relatedTablename}' database table." . PHP_EOL;
        
        $rowCount = $insertCount = 0;
        
        while (($csvLine = fgetcsv($csvHandle)) !== false) {
            if ($rowCount > 0) {
                $insertRow = array(
                    'product_name' => $csvLine[0],
                    'product_image_path' => $csvLine[1],
                    'price' => $csvLine[2],
                    'special_until' => $csvLine[3]
                );
                $db->insert($relatedTablename, $insertRow);
                ++$insertCount;
            }
            ++$rowCount;
        }
        fclose($csvHandle);
        $importMessage = "Imported {$insertCount} rows into the project "
            . "'{$relatedTablename}' database table.";
        echo $importMessage;
    }
    ...
}

Making providers and actions pretendable

To make project specific providers its actions pretendable and thereby providing some kind of user documentation the provider classes have to implement a marker interface called Zend_Tool_Framework_Provider_Pretendable. For making a action of a provider pretendable and giving some feedback to the user, the request is checked if the action has been issued in the pretend mode; which is possible by adding -p option to the issued zf command line client command. The next code snippet shows how the above stated Csv provider and its importSpecials action is made pretendable.

<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class Csv extends Zend_Tool_Project_Provider_Abstract implements 
    Zend_Tool_Framework_Provider_Pretendable
{

    public function importSpecials($csvFile, $env = 'development')
    {
        ...
        
        if ($this->_registry->getRequest()->isPretend()) {
            $pretendMessage = "I would import the specials data provided in {$csvFile} "
                . "into the project '{$relatedTablename}' database table.";
            echo $pretendMessage;
        } else {
            ...   
        }
    }
    ...
}

Using project specific providers

To use the bundled up capabilities of project specific providers, these have to made accessable to the zf command line client by putting them in the include_path. Currently I discovered no best practice for doing so only for single project scopes and simply added the path to the project to my php.ini and thereby global include_path; another approach might be to add the project name as a prefix to the Provider. After doing so it's possible to get an overview of all with the Zend_Tool_Project shipped providers plus the project specific providers and their offered actions by issuing the zf --help command as shown in the next screenshot. To ensure that project specific providers and its actions are only runnable in projects which support them, it is necessary to check if these and the offered action exists as resources in the project its profile .zfproject.xml file as shown in the implementation of the importSpecials action in one of above code snippets.

As shown in the previous screenshot the first character of the project specific providers are omitted, this is another minor bug which might be fixed in one of the forthcoming Zend Framework releases. The current workaround for this issue is simply to type the command exactly as shown in the help. The outro screenshot shows how the import-specials action of the project specific Csv provider is issued against the zf command line client and its provided user feedback after an successfull import against the projects development database.

Testing Phing buildfiles with PHPUnit

2009-05-10T16:15:00.017+02:00

While transforming some of the Ant buildfile refactorings described in Julian Simpson's seminal essay into a Phing context, it felt plainly wrong that I didn't have any tests for the buildfile to back me up on obtaining the pristine behaviour throughout the process. While Ant users can rely on an Apache project called AntUnit there are currently no tailor-made tools available for testing or verifying Phing buildfiles. Therefor I took a weekend off, locked myself in the stuffy lab, and explored the abilities to test Phing buildfiles respectively their included properties, targets and tasks with the PHPUnit testing framework. In case you'd like to take a peek at the emerged lab jottings, keep on scanning.

Introducing the buildfile under test

The buildfile that will be used as an example is kept simple, and contains several targets ranging from common ones like initializing the build environment by creating the necessary directories to more specific ones like pulling an external artifact from GitHub. To get an overview of the buildfile under test have a look at the following listing.

<?xml version="1.0" encoding="UTF-8"?>
<project name="test-example" default="build" basedir=".">

  <property name="project.basedir" value="." override="true" />
  <property name="github.repos.dir" value="${project.basedir}/build/github-repos" override="true" />

  <target name="clean" depends="clean-github-repos" description="Removes runtime build artifacts">
    <delete dir="${project.basedir}/build" includeemptydirs="true" verbose="false" failonerror="true" />
    <delete dir="${project.basedir}/build/reports" includeemptydirs="true" verbose="false" failonerror="true" />
  </target>

  <target name="clean-github-repos" description="Removes runtime build artifacts">
    <delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
  </target>

  <target name="-log-build" description="A private target which should only be invoked internally">
    <!-- omitted -->
  </target>

  <target name="build" depends="clean" description="Builds the distributable product">
    <!-- omitted -->
  </target>

  <target name="database-setup" description="Sets up the database structure">
    <!-- omitted -->
  </target>

  <target name="init" description="Initalizes the build by creating directories etc">
    <mkdir dir="${project.basedir}/build/logs/performance/" />
    <mkdir dir="${project.basedir}/build/doc" />
    <mkdir dir="${project.basedir}/build/reports/phploc" />
  </target>

  <target name="init-ad-hoc-tasks" 
          description="Initalizes the ad hoc tasks for reusability in multiple targets">      
    <adhoc-task name="github-clone"><![CDATA[
  class Github_Clone extends Task {
    private $repository = null;
    private $destDirectory = null;

    function setRepos($repository) {
      $this->repository = $repository;
    }
    function setDest($destDirectory) {
      $this->destDirectory = $destDirectory;
    }
    function main() {
      // Get project name from repos Uri
      $projectName = str_replace('.git', '',
        substr(strrchr($this->repository, '/'), 1));

      $gitCommand = 'git clone ' . $this->repository . ' ' .
      $this->destDirectory . '/' . $projectName;

      exec(escapeshellcmd($gitCommand), $output, $return);

      if ($return !== 0) {
        throw new BuildException('Git clone failed');
      }
      $logMessage = 'Cloned Git repository ' . $this->repository .
        ' into ' . $this->destDirectory . '/' . $projectName;
      $this->log($logMessage);
    }
  }
]]></adhoc-task>

    <echo message="Intialized github-clone ad hoc task." />
  </target>

  <target name="github" depends="init-ad-hoc-tasks, clean-github-repos" 
          description="Clones given repositories from GitHub">
    <github-clone repos="git://github.com/raphaelstolt/phploc-phing.git" dest="${github.repos.dir}" />
  </target>
</project>

Testing the buildfile

All tests for the buildfile under test will be bundled, like 'normal' tests, in a class i.e. BuildfileTest extending the PHPUnit_Framework_TestCase class. When testing buildfiles it's possible to build some tests around the actual buildfile XML structure, by utilizing the xpath method of PHP's SimpleXMLElement class and asserting against the XPath query results, or around the dispatching of specific targets and asserting against the expected build artifacts. Furthermore these two identified groups, structure and artifact, can be used to organize the accumulating tests via PHPUnit's @group annotation.

To be able to dispatch specific build targets and feed them with properties if necessary I additionally developed a very basic build runner shown in the next code listing.

<?php
class Phing_Buildfile_Runner {

    private $_buildfilePath = null;

    public function __construct($buildfilePath) {
        if (!file_exists($buildfilePath)) {
            throw new Exception("Buildfile '{$buildfilePath}' doesn't exist");
        }
        $this->buildfilePath = realpath($buildfilePath); 
    }
    public function runTarget($targets = array(), $properties = array()) {
        $runTargetCommand = "phing " . "-f {$this->buildfilePath} ";
        if (count($targets) > 0) {
            foreach ($targets as $target) {
                $runTargetCommand.= $target . " ";
            }
        }
        if (count($properties) > 0) {
            foreach ($properties as $property => $value) {
                $runTargetCommand.= "-D{$property}={$value} ";
            }       
        }
        exec(escapeshellcmd($runTargetCommand), $output, $return);
        return array('output' => $output, 'return' => $return);
    }
}

Out of the box PHPUnit's assertion pool provides all the utilities to test buildfiles; although it would be cleaner to create domain specfic assertions for this testing domain this technique will be ignored for the sake of brevity.

After an initial 1000ft view on how to test buildfiles let's jump into the actual testing of a structural aspect of the buildfile under test. The test to come shows how to verify that a clean target is defined for playing along in the build orchestra by querying a XPath expression against the buildfile XML and asserting that a result is available.

/**
 * @test
 * @group structure
 */
public function buildfileShouldContainACleanTarget() {
    $xml = new SimpleXMLElement($this->_buildfileXml);
    $cleanElement = $xml->xpath("//target[@name='clean']");
    $this->assertTrue(count($cleanElement) > 0, "Buildfile doesn't contain a clean target");
}

The next artifactual test raises the bar an inch, by verifying that the defined init target of the build does initialize the build environment correctly, or to pick up the orchestra metaphor again that the specific instrument plays along and holds the directed tone. Therefor the build runner executes the target and afterwards asserts a list of expected artifacts against the current state of the build process.

/**
 * @test
 * @group artifact
 */
public function initTargetShouldCreateInitialBuildArtifacts() {
    $this->_isTearDownNecessary = true;
    $this->_buildfileRunner->runTarget(array('init'));
    $expectedInitArtifacts = array(
        "{$this->_buildfileBasedir}/build",
        "{$this->_buildfileBasedir}/build/logs/performance/",
        "{$this->_buildfileBasedir}/build/doc",
        "{$this->_buildfileBasedir}/build/reports"
    );

    foreach ($expectedInitArtifacts as $artifact) {
        $this->assertFileExists($artifact, "Expected file '{$artifact}' doesn't exist");
    }
}

The next code listing shows the whole picture of the BuildfileTest class containing additional test methods verifying different aspects of the buildfile under test and also the innards of the setup and teardown method.

The main assignment of the setup method is to load the XML of the buildfile under test and to intialize the build runner so an instance is available for an use in artifactual tests. The teardown method its sole responsibility is to reset the build state by running the clean target of the buildfile.

<?php
require_once 'PHPUnit/Framework.php';
require_once 'Phing/Buildfile/Runner.php';

class ExampleBuildfileTest extends PHPUnit_Framework_TestCase {

    protected $_buildfileXml = null;
    protected $_buildfileName = null;
    protected $_buildfileBasedir = null;
    protected $_buildfileRunner = null;
    protected $_isTearDownNecessary = false;

    protected function setUp() {
        $this->_buildfileName = realpath('../../build.xml');        
        $this->_buildfileBasedir = dirname($this->_buildfileName);
        $this->_buildfileXml = file_get_contents($this->_buildfileName);
        $this->_buildfileRunner = new Phing_Buildfile_Runner(
        $this->_buildfileName);
    }

    protected function tearDown() {
        if ($this->_isTearDownNecessary) {
            $this->_buildfileRunner->runTarget(array('clean'));
        }
    }

   /**
    * @test
    * @group structure
    */
    public function targetBuildShouldBeTheDefaultTarget() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//@default";
        $defaultElement = $xml->xpath($xpath);
        $this->assertSame('build', trim($defaultElement[0]->default), 
            "Buildfile doesn't have a default target named 'build'"
        );
    }
   /**
    * @test
    * @group structure
    */
    public function propertyGithubReposDirShouldBeSet() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//property[@name='github.repos.dir']/@value";
        $valueElement = $xml->xpath($xpath);
        $this->assertTrue($valueElement[0] instanceof SimpleXMLElement, 
            "Buildfile doesn't contain a 'github.repos.dir' property"
        );
        $this->assertGreaterThan(1, strlen($valueElement[0]->value));
    }
   /**
    * @test
    * @group structure
    */
    public function buildfileShouldContainACleanTarget() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $cleanElement = $xml->xpath("//target[@name='clean']");
        $this->assertTrue(count($cleanElement) > 0, 
            "Buildfile doesn't contain a clean target"
        );
    }
   /**
    * @test
    * @group structure
    */
    public function targetLogBuildShouldBeAPrivateOne() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $nameElement = $xml->xpath("//target[@name='-log-build']");
        $this->assertTrue(count($nameElement) > 0, 
            'Log build target is not a private target'
        );
    }
    /**
     * @test
     * @group structure
     */
    public function targetBuildShouldDependOnCleanTarget() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//target[@name='build']/@depends";
        $dependElement = $xml->xpath($xpath);
        $this->assertTrue(count($dependElement) > 0, 
            'Target build contains no depends attribute'
        );
        $dependantTasks = array_filter(explode(' ',
            trim($dependElement[0]->depends))
        );
        $this->assertContains('clean', $dependantTasks, "Target build doesn't 
            depend on the clean target"
        );
    }
    /**
     * @test
     * @group structure
     */
    public function allDefinedTargetsShouldHaveADescriptionAttribute() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//target";
        $targetElements = $xml->xpath($xpath);
        $describedTargetElements = array();
        foreach ($targetElements as $index => $targetElement) {
            $targetDescription = trim($targetElement->attributes()->description); 
            if ($targetDescription !== '') {
                $describedTargetElements[] = $targetDescription;
            }
        }
        $this->assertEquals(count($targetElements),
            count($describedTargetElements), 
            'Description not for all targets set'
        );
    }
    /**
     * @test
     * @group structure
     */
    public function githubCloneAdhocTaskShouldBeDefined() {
        $xml = new SimpleXMLElement($this->_buildfileXml);
        $xpath = "//target[@name='init-ad-hoc-tasks']/adhoc-task";
        $adhocElement = $xml->xpath($xpath);
        $this->assertSame('github-clone',
            trim($adhocElement[0]->attributes()->name), 
            "Ad hoc task 'github-clone' isn't defined"
        );
    }
    /**
    * @test 
    * @group artifact
    */
    public function initTargetShouldCreateInitialBuildArtifacts() {
        $this->_isTearDownNecessary = true;
        $this->_buildfileRunner->runTarget(array('init'));

        $expectedInitArtifacts = array(
            "{$this->_buildfileBasedir}/build", 
            "{$this->_buildfileBasedir}/build/logs/performance/", 
            "{$this->_buildfileBasedir}/build/doc",
            "{$this->_buildfileBasedir}/build/reports"
        );

        foreach ($expectedInitArtifacts as $artifact) {
            $this->assertFileExists($artifact, 
                "Expected file '{$artifact}' doesn't exist"
            );
        }
    }
    /**
     * @test
     * @group artifact
     */
    public function sqlFilesForDatabaseSetupTargetShouldBeAvailable() {
        $expectedSqlFiles = array(
            "{$this->_buildfileBasedir}/sqlfiles", 
            "{$this->_buildfileBasedir}/sqlfiles/session-storage.sql", 
            "{$this->_buildfileBasedir}/sqlfiles/acl.sql", 
            "{$this->_buildfileBasedir}/sqlfiles/log.sql"
        );

        foreach ($expectedSqlFiles as $sqlFile) {
            $this->assertFileExists($sqlFile, 
                "SQL file '{$sqlFile}' doesn't exist"
            );
        }
    }
    /**
     * @test
     * @group artifact
     */
    public function githubTargetShouldFetchExpectedRepository() {
        $this->_isTearDownNecessary = true;
        $this->_buildfileRunner->runTarget(array('github'));
        $expectedGitRepository = "{$this->_buildfileBasedir}/build/"
            . "github-repos/phploc-phing/.git";
        $this->assertFileExists($expectedGitRepository, 
            "Github target doesn't fetch the expected 'phploc-phing' repository"
        );
    }
}

The outro screenshot shows the above stated test class run against the example buildfile on a Mac OS X system utilizing the --colors option; which by the way comes in really handy in combination with Stakeout.rb during the process of refactoring or extending/creating buildfiles the test-driven way.

Creating and using Phing ad hoc tasks

2009-04-18T02:48:00.006+02:00

Sometimes there are build scenarios where you'll badly need a functionality, like adding a MD5 checksum file to a given project, that isn't provided neither by the available Phing core nor the optional tasks. Phing supports developers with two ways for extending the useable task pool: by writing 'outline' tasks that will end up in a directory of the Phing installation or by utilizing the AdhocTaskdefTask, which allows to define custom tasks in the buildfile itself. The following post will try to outline how to define and use these inline tasks, by sketching an ad hoc task that enables the build orchestra to clone Git repositories from GitHub during a hypothetical workbench setup.

Creating the inline/ad hoc task

The AdhocTaskdefTask expects a name attribute i.e. github-clone for the XML element which will later referr to the ad hoc task and a CDATA section hosting the task implementation. Similar to 'outline' tasks the ad hoc task extends Phing's Task class, configures the task via attributes and holds the logic to perform. Unfortunately inline task implementations don't allow to require or include external classes available in the include_path, like Zend_Http_Client which I initially tried to use for an example task fetching short Urls from is.gd. This limits the available functions and classes to craft the task from to the ones built into PHP. The following buildfile snippet shows the implementation of the github-clone ad hoc task which is wrapped by a private target to encourage reusability and limit it's callability.

<target name="-init-ad-hoc-tasks" 
        description="Initializes the ad hoc task(s)">
  <adhoc-task name="github-clone"><![CDATA[
      class Github_Clone extends Task {

          private $repository = null;
          private $destDirectory = null;

          function setRepos($repository) {
              $this->repository = $repository;
          } 
          function setDest($destDirectory) {
              $this->destDirectory = $destDirectory;
          }
          function main() {
              // Get project name from repos Uri
              $projectName = str_replace('.git', '', 
                  substr(strrchr($this->repository, '/'), 1));

              $gitCommand = 'git clone ' . $this->repository . ' ' . 
                  $this->destDirectory . '/' . $projectName;

              exec(escapeshellcmd($gitCommand), $output, $return);

              if ($return !== 0) {
                  throw new BuildException('Git clone failed');
              }
              $logMessage = 'Cloned Git repository ' . $this->repository . 
                  ' into ' . $this->destDirectory . '/' . $projectName;
              $this->log($logMessage);
          }
      }
  ]]></adhoc-task>
  <echo message="Initialized github-clone ad hoc task." />
</target>

Using the ad hoc task

With the ad hoc task in the place to be, it's provided functionality can now be used from any target using the tasks XML element according to the given name i.e. github-clone in the AdhocTaskdefTask element earlier and by feeding it with the required attributes i.e. repos and dest. The next snippet allows you to take a peek at the complete buildfile with the ad hoc task in action.

<?xml version="1.0" encoding="UTF-8"?>
<project name="recordshelf" default="init-work-bench" basedir=".">
  
  <property name="github.repos.dir" value="./github-repos" override="true" />

  <target name="init-work-bench" 
          depends="-init-ad-hoc-tasks, -clone-git-repos" 
          description="Initializes the hypothetical workbench">
    <echo message="Initialized workbench." />
  </target>

  <target name="-clean-git-repos" 
          description="Removes old repositories before initializing a new workbench">
    <delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
  </target>

  <target name="-init-ad-hoc-tasks" 
          description="Initializes the ad hoc task(s)">
    <adhoc-task name="github-clone"><![CDATA[
        class Github_Clone extends Task {

            private $repository = null;
            private $destDirectory = null;

            function setRepos($repository) {
                $this->repository = $repository;
            } 
            function setDest($destDirectory) {
                $this->destDirectory = $destDirectory;
            }
            function main() {
                // Get project name from repos Uri
                $projectName = str_replace('.git', '', 
                    substr(strrchr($this->repository, '/'), 1));

                $gitCommand = 'git clone ' . $this->repository . ' ' . 
                    $this->destDirectory . '/' . $projectName;

                exec(escapeshellcmd($gitCommand), $output, $return);

                if ($return !== 0) {
                    throw new BuildException('Git clone failed');
                }
                $logMessage = 'Cloned Git repository ' . $this->repository . 
                    ' into ' . $this->destDirectory . '/' . $projectName;
                $this->log($logMessage);
            }
        }
    ]]></adhoc-task>
    <echo message="Initialized github-clone ad hoc task." />
  </target>
  
  <target name="-clone-git-repos" depends="-clean-git-repos"
          description="Clones the needed Git repositories from GitHub">
    <github-clone repos="git://github.com/abc/abc.git" 
                  dest="${github.repos.dir}" />
    <github-clone repos="git://github.com/xyz/xyz.git" 
                  dest="${github.repos.dir}" />
  </target>
    
</project>

Favouring inline over 'outline' tasks?

The one big advantage of using inline tasks over 'outline' tasks is that they are distributed with the buildfile and are instantly available without the need to modify the Phing installation. Some severe disadvantages of inline tasks are the limitation to use only the core PHP functions and classes for the implementation, the introduction of an additional hurdle to verify the task behaviour via PHPUnit as it's located in a CDATA section of the buildfile and the fact that the use of several inline tasks will blow up the buildfile, and thereby obfuscate the build flow.

Regrettably Phing doesn't provide an import task like Ant which might enable a refactoring to pull the ad hoc task definitions into a seperate XML file and include them at buildtime; in case you might have some expertise or ideas for a suitable workaround hit me with a comment. So far I tried to get it working, with no success, by utilizing Phing's PhingTask and XML's external entities declaration.

Using Haml & Sass from a Rake task

2009-03-31T16:02:00.061+02:00

Some time ago I had the 'lightning' idea to implement another Rake automation to support my current blogging workflow, which at the moment consists of finding a sparkling idea to blog about, write it out in WriteRoom and refine the post in TextMate before publishing. As this process was a recurring and copy & paste driven event, I strove for an automation supporting this workflow. So unsurprisingly the post will show my current solution to achieve this goal by utilizing Rake, Haml and Sass.

So what's that Haml and Sass thingy?

Haml (HTML Abstraction Markup Language) is a templating language/engine with the primary goal to make Markup DRY, beautiful and readable again. It has a very shallow learning curve and therefor is perfectly suited for programmers and designers alike. Haml is primarily targeted at making the views of Ruby on Rails, Merb or Sinatra web applications leaner, but as you will see later the Ruby implementation also can be used framework independently.

Sass (Syntactically Awesome StyleSheets) is a module which comes bundled with Haml providing a meta-language/abstraction on top of CSS sharing the same goals and advantages as Haml.

Gluing Haml and Sass into a Rake task

To get going you first have to install Haml and Sass by running the gem command shown next.

sudo gem install haml

With Haml and Sass available it's about time to identify and outline the parts you want to automate, in my case it's the creation of a WriteRoom and/or a XHTML draft document for initial editings. So the parameters to pass into the task to come are the targeted editor(s), the title of the blog post to draft and a list of associated and whitespace separated category tags.

The XHTML document skeleton content and it's inline CSS are defined each in a separate Haml and Sass template file and will be rendered into the outcoming document along with the content passed into the Rake task. While the document skeleton for the WriteRoom draft document, due to it's brevity, is defined inside of the task itself. The following snippets are showing the mentioned Haml and Sass templates for the XHTML draft output file, which are located in the same directory as the Rake file.

Haml

!!! 1.1
%html
  %head 
    %title= "#{title} - Draft"
    %style{ :type => 'text/css' }= inline_css
  %body
    %h3= title
    %h4.custom sub headline 
    %pre.consoleOutput console command
    %pre.codeSnippet code snippet
    %br/
    = "Tags: #{tags.join ', '}"

Sass

body
  :margin 5
  :line-height 1.5em
  :font small Trebuchet MS, Verdana, Arial, Sans-serif
  :color #000000
h4
  :margin-bottom 0.3em
.consoleOutput
  :padding 6px 
  :background-color #000 
  :color rgb(20, 218, 62)
  :font-size 12px
  :font-weight bolder
.codeSnippet
  :padding 3px
  :background-color rgb(243, 243, 243)
  :color rgb(93, 91, 91)
  :font-size small
  :border 1px solid #6A6565

To inject the dynamic content into the Haml template and have it rendered into the outcoming document, the values i.e. draft_title, draft_tags and draft_inline_css have to be made available to the template engine by passing them in a bundling Hash into the to_html alias method of the Haml Engine object like shown in the next Rake task.

task :default do
  Rake::Task['blog_utils:create_draft_doc'].invoke
end

namespace :blog_utils do
  
  desc 'Create a new draft document for a given title, category tags and editor'
  task :create_draft_doc, [:title, :tags, :editor] do |t, args|
    draft_title = args.title
    draft_tags = args.tags.split(' ')
    draft_target_editor = args.editor
    
    raise_message = 'No title for draft provided'
    raise raise_message if draft_title.nil?
    
    raise_message = 'No tags for draft provided'
    raise raise_message if draft_tags.nil?
    
    draft_target_editor = '*' if draft_target_editor.nil?
    
    raise_message = 'Unsupported target editor provided' 
    raise raise_message unless draft_target_editor == 'Textmate' || 
      draft_target_editor == 'Writeroom' || draft_target_editor == '*'
    
    if draft_target_editor == 'Writeroom' || draft_target_editor == '*'
      draft_output_file = draft_title.gsub(' ', '_') + '.txt'
      
      File.open(draft_output_file, 'w') do |draft_file_txt|
        draft_file_txt.puts draft_title
        draft_file_txt.puts
        draft_file_txt.puts "Tags: #{draft_tags.join ', '}"
      end
    end
    
    if draft_target_editor == 'Textmate' || draft_target_editor == '*'
      
      template_sass_content, template_haml_content = ''
      
      ['haml', 'sass'].each do |template_type|
        template = File.dirname(__FILE__) + "/draft_template.#{template_type}"
        raise_message = "#{template_type.capitalize} template '#{template}' not found"
        raise raise_message if !File.exists?(template)
        
        template_sass_content = File.read(template) if template_type === 'sass'
        template_haml_content = File.read(template) if template_type === 'haml'
      end
      
      require 'sass'
      require 'haml'

      draft_inline_css = Sass::Engine.new(template_sass_content).to_css
      draft_document_content = Haml::Engine.new(template_haml_content).to_html(
        Object.new, { :title => draft_title , :tags => draft_tags ,
          :inline_css => draft_inline_css } )
      
      draft_output_file = draft_title.gsub(' ', '_') + '.html'      
      File.open(draft_output_file, 'w') do |draft_file_html|
        draft_file_html.puts(draft_document_content)
      end
    end
    
  end
end

Easing invocation pain with alias

Now as the Rake task is implemented and waiting for demands it can be invoked by calling the task as shown in the next console snippet.

sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:create_draft_doc['Title','Tag1 TagN','Editor']

As I'm not even close to being a console ninja and probably will have forgotten the task call structure before initiating the next blog post, I decided to add an easing and more memorizable alias to $HOME/.profile as shown next.

alias createdraft='sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:create_draft_doc[$title,$tags,$editor]'

The created alias now allows to invoke the Rake task in a nice and easy way as shown in the next console command.

createdraft title='Using Haml & Sass from a Rake task' tags='Rake Ruby' editor='Textmate'

Taking a peek at the generated draft document

After running the described Rake task I end up with the XHTML document shown in the outro code snippet, which then can be used for the further editing process. Of course I could have setup a TextMate Snippet to get me going, but that way I would have missed the opportunity to mess around with another amazing Ruby tool.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html>
  <head>
    <title>Using Haml & Sass from a Rake task - Draft</title>
    <style type='text/css'>
      body {
        margin: 5;
        line-height: 1.5em;
        font: small Trebuchet MS, Verdana, Arial, Sans-serif;
        color: #000000; }

      h4 {
        margin-bottom: 0.3em; }

      .consoleOutput {
        padding: 6px;
        background-color: #000;
        color: rgb(20, 218, 62);
        font-size: 12px;
        font-weight: bolder; }
      
      .codeSnippet {
        padding: 3px;
        background-color: rgb(243, 243, 243);
        color: rgb(93, 91, 91);
        font-size: small;
        border: 1px solid #6A6565; }
    </style>
  </head>
  <body>
    <h3>Using Haml & Sass from a Rake task</h3>
    <h4>sub headline</h4>
    <pre class='consoleOutput'>console command</pre>
    <pre class='codeSnippet'>code snippet</pre>
    <br />
    Tags: Rake, Ruby
  </body>
</html>

Phplocing your projects with Phing

2009-02-22T14:35:00.007+01:00

When I started to play around with Ruby on Rails, my attention got somehow soon drawn to it's Rake stats task, which provides developers or more likely project managers with an overview of the actual project size. Exactly one month ago Sebastian Bergmann, of PHPUnit fame, started to implement a similar tool dubbed phploc which can give you an overview of the size for any given PHP project. As I wanted to automate the invocation of this handy tool and collect it's report output out of a Phing buildfile, I invested some time to develop a custom Phing task doing so. Thereby the following post will show you a possible implementation of this task and it's use in a buildfile.

Installing phploc

To setup phploc on your system simply install the phploc PEAR package available from the pear.phpunit.de channel as shown in the next commands. In case you already have installed PHPUnit via PEAR you can omit the channel-discover command.

sudo pear channel-discover pear.phpunit.de
sudo pear install phpunit/phploc

Implementing the phploc task

As I already blogged about developing custom Phing task I'm only going to show the actual implementation and not dive into any details; alternatively you can also grab it from this public GitHub repository.

<?php
require_once 'phing/Task.php';
require_once 'phing/BuildException.php';
require_once 'PHPLOC/Analyser.php';
require_once 'PHPLOC/Util/FilterIterator.php';
require_once 'PHPLOC/TextUI/ResultPrinter.php';

class PHPLocTask extends Task
{
    protected $suffixesToCheck = null;
    protected $acceptedReportTypes = null;
    protected $reportDirectory = null;
    protected $reportType = null;
    protected $fileToCheck = null;
    protected $filesToCheck = null;
    protected $reportFileName = null;
    protected $fileSets = null;
    
    public function init() {
        $this->suffixesToCheck = array('php');
        $this->acceptedReportTypes = array('cli', 'txt', 'xml');
        $this->reportType = 'cli';
        $this->reportFileName = 'phploc-report';
        $this->fileSets = array();
        $this->filesToCheck = array();
    }
    public function setSuffixes($suffixListOrSingleSuffix) {
        if (stripos($suffixListOrSingleSuffix, ',')) {
            $suffixes = explode(',', $suffixListOrSingleSuffix);
            $this->suffixesToCheck = array_map('trim', $suffixes);
        } else {
            array_push($this->suffixesToCheck, trim($suffixListOrSingleSuffix));
        }
    }
    public function setFile(PhingFile $file) {
        $this->fileToCheck = trim($file);
    }
    public function createFileSet() {
        $num = array_push($this->fileSets, new FileSet());
        return $this->fileSets[$num - 1];
    }
    public function setReportType($type) {
        $this->reportType = trim($type);
    }
    public function setReportName($name) {
        $this->reportFileName = trim($name);
    }
    public function setReportDirectory($directory) {
        $this->reportDirectory = trim($directory);
    }
    public function main() {       
        if (!isset($this->fileToCheck) && count($this->fileSets) === 0) {
            $exceptionMessage = "Missing either a nested fileset or the "
                . "attribute 'file' set.";
            throw new BuildException($exceptionMessage);
        }
        if (count($this->suffixesToCheck) === 0) {
            throw new BuildException("No file suffix defined.");
        }
        if (is_null($this->reportType)) {
            throw new BuildException("No report type defined.");
        }
        if (!is_null($this->reportType) && 
            !in_array($this->reportType, $this->acceptedReportTypes)) {
            throw new BuildException("Unaccepted report type defined.");
        }
        if (!is_null($this->fileToCheck) && !file_exists($this->fileToCheck)) {
            throw new BuildException("File to check doesn't exist.");
        }
        if ($this->reportType !== 'cli' && is_null($this->reportDirectory)) {
            throw new BuildException("No report output directory defined.");
        }
        if (count($this->fileSets) > 0 && !is_null($this->fileToCheck)) {
            $exceptionMessage = "Either use a nested fileset or 'file' " 
                . "attribute; not both.";
            throw new BuildException($exceptionMessage);
        }
        if (!is_null($this->reportDirectory) && !is_dir($this->reportDirectory)) {
            $reportOutputDir = new PhingFile($this->reportDirectory);
            $logMessage = "Report output directory does't exist, creating: " 
                . $reportOutputDir->getAbsolutePath() . '.';
            $this->log($logMessage);
            $reportOutputDir->mkdirs();
        }
        if ($this->reportType !== 'cli') {
            $this->reportFileName.= '.' . trim($this->reportType);
        }
        if (count($this->fileSets) > 0) {
            $project = $this->getProject();
            foreach ($this->fileSets as $fileSet) {
                $directoryScanner = $fileSet->getDirectoryScanner($project);
                $files = $directoryScanner->getIncludedFiles();
                $directory = $fileSet->getDir($this->project)->getPath();
                foreach ($files as $file) {
                    if ($this->isFileSuffixSet($file)) {
                        $this->filesToCheck[] = $directory . DIRECTORY_SEPARATOR 
                            . $file;
                    }
                }
            }
            $this->filesToCheck = array_unique($this->filesToCheck);
        }
        if (!is_null($this->fileToCheck)) {
            if (!$this->isFileSuffixSet($file)) {
                $exceptionMessage = "Suffix of file to check is not defined in"
                    . " 'suffixes' attribute.";
                throw new BuildException($exceptionMessage);
            }
        }
        $this->runPhpLocCheck();
    }
    protected function isFileSuffixSet($filename) {
        $pathinfo = pathinfo($filename);
        $fileSuffix = $pathinfo['extension'];
        return in_array($fileSuffix, $this->suffixesToCheck);
    }
    protected function runPhpLocCheck() {
        $files = $this->getFilesToCheck();
        $result = $this->getCountForFiles($files); 
        
        if ($this->reportType === 'cli' || $this->reportType === 'txt') {
            $printer = new PHPLOC_TextUI_ResultPrinter;
            if ($this->reportType === 'txt') {
                ob_start();
                $printer->printResult($result);
                file_put_contents($this->reportDirectory 
                    . DIRECTORY_SEPARATOR . $this->reportFileName, 
                        ob_get_contents());
                ob_end_clean();
                $reportDir = new PhingFile($this->reportDirectory);
                $logMessage = "Writing report to: " 
                    . $reportDir->getAbsolutePath() . DIRECTORY_SEPARATOR 
                        . $this->reportFileName;
                $this->log($logMessage);
            } else {
                $printer->printResult($result);
            }
        } elseif ($this->reportType === 'xml') {
            $xml = $this->getResultAsXml($result);
            $reportDir = new PhingFile($this->reportDirectory);
            $logMessage = "Writing report to: " . $reportDir->getAbsolutePath()
                . DIRECTORY_SEPARATOR . $this->reportFileName;
            $this->log($logMessage);
            file_put_contents($this->reportDirectory . DIRECTORY_SEPARATOR
                . $this->reportFileName, $xml);
        }
    }
    protected function getFilesToCheck() {
        if (count($this->filesToCheck) > 0) {
            $files = array();
            foreach ($this->filesToCheck as $file) {
                $files[] = new SPLFileInfo($file);
            }
        } elseif (!is_null($this->fileToCheck)) {
            $files = array(new SPLFileInfo($this->fileToCheck));
        }
        return $files;
    }
    protected function getCountForFiles($files) {
        $count = array('files' => 0, 'loc' => 0, 'cloc' => 0, 'ncloc' => 0,
            'eloc' => 0, 'interfaces' => 0, 'classes' => 0, 'functions' => 0);
        $directories = array();

        foreach ($files as $file) {
            $directory = $file->getPath();
            if (!isset($directories[$directory])) {
                $directories[$directory] = TRUE;
            }          
            PHPLOC_Analyser::countFile($file->getPathName(), $count);
        }
        
        if (!function_exists('parsekit_compile_file')) {
            unset($count['eloc']);
        }
        $count['directories'] = count($directories) - 1;
        return $count;
    }
    protected function getResultAsXml($result) {        
        $newline = "\n";
        $newlineWithSpaces = sprintf("\n%4s",'');
        $xml = '<?xml version="1.0" encoding="UTF-8"?>';
        $xml.= $newline . '<phploc>'; 
        
        if ($result['directories'] > 0) {
            $xml.= $newlineWithSpaces . '<directories>' . $result['directories'] . '</directories>';
            $xml.= $newlineWithSpaces . '<files>' . $result['files'] . '</files>';
        }
        $xml.= $newlineWithSpaces . '<loc>' . $result['loc'] . '</loc>';
        
        if (isset($result['eloc'])) {
            $xml.= $newlineWithSpaces . '<eloc>' . $result['eloc'] . '</eloc>';
        }
        $xml.= $newlineWithSpaces . '<cloc>' . $result['cloc'] . '</cloc>';
        $xml.= $newlineWithSpaces . '<ncloc>' . $result['ncloc'] . '</ncloc>';
        $xml.= $newlineWithSpaces . '<interfaces>' . $result['interfaces'] . '</interfaces>';
        $xml.= $newlineWithSpaces . '<classes>' . $result['classes'] . '</classes>';
        $xml.= $newlineWithSpaces . '<methods>' . $result['functions'] . '</methods>' . $newline;
        $xml.= '</phploc>';
        return $xml;
    }
}

Hooking the phploc task into Phing

To use the task in your Phing builds simply copy it into the phing/tasks/my directory and make it available via the taskdef task. The next table shows the available task attributes and the values they can take to configure it's behaviour and output. As you will see it also provides the ability to generate reports in a XML format; I chose to implement this feature to have the possibilty to transform the report results into HTML documents by applying for example a XSLT stylesheet. This way they can provide more value to non-technical project members or can be made accessible in a CI system dashboard if desired.

Name	Type	Description	Default	Required
reportType	string	The type of the report. Available types are cli\|txt\|xml.	cli	No
reportName	string	The name of the report type without a file extension.	phploc-report	No
reportDirectory	string	The directory to write the report file to.	false	Yes, when report type txt or xml is defined.
file	string	The name of the file to check.	n/a	Yes, when no nested fileset is defined.
suffixes	string	A comma-separated list of file suffixes to check.	php	No

Supported Nested Tags:

fileset

The closing buildfile extract shows an example phploc task configuration and is also available at the public GitHub repository. Happy phplocing!

<?xml version="1.0"?>
<project name="example" default="phploc" basedir=".">
    <taskdef name="phploc" classname="phing.tasks.my.PHPLocTask" />
    <target name="phploc">
      <tstamp>
        <format property="check.date.time" pattern="%Y%m%d-%H%M%S" locale="en_US"/>
      </tstamp>
      <phploc reportType="txt" reportName="${check.date.time}-report"
              reportDirectory="phploc-reports">
        <fileset dir=".">
          <include name="**/*.php" />
          <include name="*.php" />
        </fileset>
      </phploc>
    </target>
</project>

Broadcasting blog post notifications to Twitter with Ruby and Rake

2009-01-24T06:08:00.032+01:00

During my latest blogging absence I had some time to tinker around with Ruby. For an introductory challenge I chose to implement a real life feature which currently isn't supported by Blogger.com and screams siren-like for an one-button automation: Broadcasting the latest blog entry to my Twitter account. As I didn't want to sign up for a Twitterfeed account and couldn't resort to the Twitter Tools plugin like WordPress users, I had to perform these broadcasting steps manually, until now. To see how this repetitive and time-stealing process was transformed into a semi-automated one by utilizing Ruby, a splash of Hpricot, Ruby's excellent Twitter Api wrapper and Rake, read on my dear.

Installing the required RubyGems

Prior to diving into the implementation details of the given scenario I had to install the required RubyGems like shown in the next console snippet. The installation of the twitter gem might take a while due to it's dependency on several other gems.

sudo gem install hpricot rake twitter

Scraping the latest blog post details with Hpricot

The initial implementation step was to gather relevant metadata (Url, title and used tags) of the latest blog post. I first took the route to get it by grabbing the blog's RSS feed and extracting the metadata from there, but soon stumbled into problems getting an outdated feed from Feedburner. The next alternative was to scrape the needed metadata directly from the blog landing page. As I went this route before with the Zend_Dom_Query component of the Zend Framework I decided to use something similar from the Ruby toolbox. Some Google hops later I was sold to Hpricot, a HTML Parser for Ruby and as you can see in the first code snippet, showing an extract of the Rake file to come, this is done in just 13 lines of code.

doc = Hpricot(open(blog_landing_page, scrape_options))
latest_post_url = doc.at('h3.post-title > a')['href']
latest_post_title = doc.at('h3.post-title > a').inner_html
label_doc = Hpricot(doc.search('span.post-labels').first.to_s)
label_links = label_doc.search('span.post-labels > a').each do |label_link|
  label = label_link.inner_html.gsub(' ', '').downcase
  if label.include?('/')
    labels = label.split('/')
    labels.each { |label| last_post_labels.push(label) }
  else
    last_post_labels.push(label)
  end
end

Outstanding tasks

With the metadata available the oustanding tasks to implement were:

to get a short Url for the actual blog post by utilzing a public API of an Url shortening service i.e. is.gd
to build the tweet to broadcast by injecting the available metadata into a tweet template
to broadcast the notification tweet to the given Twitter account
to log the broadcasted blog title to prevent spamming or duplication scenarios

As a guy sold to build tools and eager to learn something new I subverted Rake, Ruby's number one build language, to glue the above mentioned tasks and their implementation together, to manage their sequential dependencies and to have a comfortable invocation interface. The nice thing about Rake is that it allows you to implement each tasks unit of work by using the Ruby language; and there is no need to follow a given structure to implement custom tasks like it's the case for custom Phing tasks. As you will see in the forthcoming complete Rakefile some of the tasks are getting quite long and complex; therefor some of them are pending candidates for Refactoring activities like for example extract task units of work into helper/worker classes.

  require 'rubygems'
  require 'hpricot'
  require 'open-uri'
  require 'twitter'

  task :default do
    Rake::Task['blog_utils:broadcast_notification'].invoke
  end

  namespace :blog_utils do

    scrape_options = { 'UserAgent' => "Ruby/#{RUBY_VERSION}" }
    blog_landing_page = 'http://raphaelstolt.blogspot.com'
    latest_post_short_url, latest_post_url, latest_post_title = nil
    notification_tweet = nil
    last_post_labels = []
    broadcast_log_file = File.dirname(__FILE__) + '/broadcasted_posts.log'
    twitter_credentials = { :user => 'raphaelstolt', :pwd => 'thatsasecret'}

    desc 'Scrape metadata of latest blog post from landing page'
    task :scrape_actual_post_metadata do
      doc = Hpricot(open(blog_landing_page, scrape_options))
      latest_post_url = doc.at('h3.post-title > a')['href']
      latest_post_title = doc.at('h3.post-title > a').inner_html
      label_doc = Hpricot(doc.search('span.post-labels').first.to_s)
      label_links = label_doc.search('span.post-labels > a').each do |label_link|
        label = label_link.inner_html.gsub(' ', '').downcase
        if label.include?('/')
          labels = label.split('/')
          labels.each { |label| last_post_labels.push(label) }
        else
          last_post_labels.push(label)
        end
      end
    end

    desc 'Shorten the Url of the latest blog post'
    task :shorten_post_url => [:scrape_actual_post_metadata] do
      raise_message = 'No Url for latest blog post available'
      raise raise_message if latest_post_url.nil?
      url_shorten_service_call = "http://is.gd/api.php?longurl=#{latest_post_url}"
      latest_post_short_url = open(url_shorten_service_call, scrape_options).read
    end

    desc 'Check if generate shorten Url references the latest blog post url'
    task :check_shorten_url_references_latest do
      url_referenced_by_short_url = nil
      open(latest_post_short_url, scrape_options) do |f|
        url_referenced_by_short_url = f.base_uri.to_s
      end 
      raise_message = "Generated short Url '#{latest_post_short_url}' does not"
      raise_message << " reference actual blog post url '#{latest_post_url}'"
      raise raise_message unless url_referenced_by_short_url.eql?(latest_post_url)
    end

    desc 'Check if latest blog post has already been broadcasted'
    task :check_logged_broadcasts do
      logged_broadcasts = []
      if  File.exist?(broadcast_log_file)
        File.open(broadcast_log_file, 'r') do |f|
          logged_broadcasts = f.readlines.collect { |line| line.chomp }
        end
      end
      raise_message = "Blog post '#{latest_post_title}' has already been "
      raise_message << "broadcasted"
      raise raise_message if logged_broadcasts.include?(latest_post_title)
    end

    desc 'Build notification tweet by injecting scraped metadata into template'
    task :build_notification_tweet => [:shorten_post_url, 
      :check_shorten_url_references_latest] do
      raise_message = 'Required metadata to build tweet is not available'
      raise raise_message if latest_post_title.nil? || latest_post_short_url.nil? 
      raise raise_message if last_post_labels.nil?

      notification_tweet = "Published a new blog post '#{latest_post_title}' "
      notification_tweet << "available at #{latest_post_short_url}."

      raise_message = 'Broadcast for latest blog post exceeds 140 characters'
      raise raise_message if notification_tweet.length > 140

      last_post_labels.each do |tag|
        notification_tweet << " ##{tag}" unless notification_tweet.length + 
          " ##{tag}".length > 140
      end
    end

    desc 'Broadcast latest blog post notification to twitter'
    task :broadcast_notification_to_twitter => [:build_notification_tweet, 
      :check_logged_broadcasts] do
      raise_message = "Notification tweet to broadcast is not available"
      raise raise_message if notification_tweet.nil?
      puts "Broadcasting '#{notification_tweet}'"
      http_auth = Twitter::HTTPAuth.new(twitter_credentials[:user], twitter_credentials[:pwd])
      Twitter::Base.new(http_auth).update(notification_tweet)
      #Twitter::Base.new(twitter_credentials[:user], twitter_credentials[:pwd]).post(notification_tweet)
      Rake::Task['blog_utils:log_broadcast_title'].invoke
    end

    desc 'Log broadcasted blog post title'
    task :log_broadcast_title do
      puts "Logging latest post title to #{broadcast_log_file}"
      File.open(broadcast_log_file, 'a') do |f|
        f.puts latest_post_title
      end
    end

  end

Putting the Rake task(s) to work

The next step was to put the Rakefile into my $HOME directory; and after publishing a new blog post I'm now able to broadcast an automated notification by firing up the console and calling the Rake task like shown next.

sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:broadcast_notification

And as I'm too lazy to type this lengthy command everytime I further added an alias to the $HOME/.profile file which allows me to call the task via the associated alias i.e. blogger2twitter shown in the .profile excerpt.

alias blogger2twitter='sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:broadcast_notification'

After running the Rake task against this blog post the notification gets added to the given Twitter timeline like shown in the outro image.

Installing Zend_Tool on Mac OS X

2009-01-23T14:19:00.032+01:00

Yesterday I decided to tiptoe into the development of custom Zend_Tool Providers as the introductional article series by Ralph Schindler motivated me to learn more about it and I already have some useful use cases on my mind. Therefor I prior had to install the Zend_Tool component and it's driving CLI scripts on my MacBook. The following brief instruction describes a possible approach that got me running in no time on a Mac OS X system. Once the Zend Framework has an official PEAR channel most of the forthcoming steps should be obsolete and entirely performed by the PEAR package installer command.

Fetching and installing the Zend_Tool component

First I tried to install the 1.8.0(devel) version of the Zend Framework via the pear.zfcampus.org PEAR channel but it currently only delivers the 1.7.3PL1(stable) package; even after switching the stability state of the PEAR config. To dodge the include_path setting hassle and for a further use when customizing other tools like Phing tasks I decided to keep the installed package.

sudo pear channel-discover pear.zfcampus.org
sudo pear install zfcampus/zf-devel

The next commands are showing the footwork I had to do to get the Zend_Tool component into the PEAR Zend Framework package installed in /opt/local/lib/php/Zend.

sudo svn co http://framework.zend.com/svn/framework/standard/incubator/library/Zend/Tool/ $HOME/Cos/Zend/Tool
sudo rsync -r --exclude=.svn $HOME/Cos/Zend/Tool /opt/local/lib/php/Zend

Putting the Zend_Tool CLI scripts to work

The next steps were to fetch the CLI scripts from the public Subversion repository and to link them into the system path /opt/local/bin as shown in the next commands.

sudo svn co http://framework.zend.com/svn/framework/standard/incubator/bin $HOME/Cos/Zend/bin
sudo ln $HOME/Cos/Zend/bin/zf.sh /opt/local/bin/zf
sudo ln $HOME/Cos/Zend/bin/zf.php /opt/local/bin/zf.php

Checking the installation

With everything hopefully in place it was time to verify the success of the installation via the below stated provider action call; and as I got the version of the installed Zend Framework as a response of the executed action/command I'm good to go.

zf show version

Rails for PHP Developers book review

2008-11-13T16:21:00.007+01:00

The e-book version of the Pragmatic Programmers release Rails for PHP Developers written by Derek DeVries and Mike Naberezny occupies now some of my scarce hard drive space for several months, and today I managed to hit the last page of it. In case you're interested in knowing if it's worthy to sacrifice some rare hard drive or bookshelf space for this book read on.

What's in it?

The book consists of three main parts which are addressing open-minded developers with a PHP background tempted to add the Ruby language and the thereupon built Rails 2.0 framework to their toolset.

The first part introduces the classic and nowadays omnipresent MVC pattern, the concepts and conventions of Rails by converting a simple PHP newsletter application into a Rails based one. The follow-up chapters of the first part are covering the basics of the Ruby language by looking at known PHP language features and constructs, and how they translate to their Ruby counterparts. Reading these chapters you will get a thorough understanding of the Ruby language and be able to apply unique features like blocks or the reopening of existing classes. The communicated knowledge builds the foundation to accelerate the use and understanding of the Rails framework which is covered in-depth through-out the book's second part.

While teaming up with their imaginary buddy Joe the authors walk you through building a Rails user group application. The chapters of the second part are covering a lot of ground reaching from domain modeling, putting the particular MVC parts to work, ensuring quality by utilizing the Test::Unit library to finally deploying the application into a ~~productive~~ production environment.

The first two chapters of the final and reference part cover the differences and similarities between PHP and Ruby data structures, operations and language constructs. The final chapter of the book closes with a web development specific comparision of PHP constructs and approaches to the ones used by the Rails framework. The book is accompanied by a dedicated blog and a PHP to Rails online reference to satisfy severe thirst for more knowledge.

Conclusion

The book provides interested PHP developers a thorough introduction to the Ruby language and the Rails framework in a fluent and enjoyable writing tone. By implementing the example application of the second book part any decent PHP developer will derive a solid understanding of the Rails framework, he can build upon and that puts him in the position to make reasonable judgments for using/flaming it or not. IMHO this book is so far one of the best PHP related book releases of the out fading year 2008, and can be a real motivator to extend the just gained knowledge by diving deeper into the Ruby/Rails ocean.

So be prepared to see one or another Ruby related post popping up in the future timeline of this blog; I just added another costly addiction to the medicine cupboard. Word!

Tinyizing URLs with Zend_Http_Client

2008-10-31T22:44:00.002+01:00

While doing some initial research for a blog related automation task to implement I learned some more about services which transform long URLs into short ones. The well-knownst of these services, due to the Twitter hype, is probably TinyURL which can be accessed via a classic webinterface or by calling a public API. In a recent blog post Dave Marshall outlined a quick workaround for tweeting via the Zend_Http_Client component which is a reasonable approach for calling services that aren't in the Zend Framework core yet like Zend_Service_Twitter or are not supported out of the box. Therefore this post will try to describe a Zend Framework way of creating tinyized URLs.

Getting tiny tiny y'all

According to Wikipedia there are numerous services available e.g. RubyUrl providing the same feature as TinyURL, so to be prepared for the future and thereby maybe violating the YAGNI principle I decided to declare a very basic interface first in case of switching the service provider someday.

<?php
/** 
 * 'Interface-level' PHPDoc Block   
 */
interface Recordshelf_Service_UrlShortener_Interface
{
    public function __construct($serviceEndpoint = '');
    public function shortenize($url);
}

The next code snippet shows the implementation for the TinyURL service programmed against the interface and hosting an additional alias method called tinyize which is simply wrapping the actual worker method. The service utilizes Zend_Http_Client by setting the endpoint of the service, transmitting a GET request parameterized with the URL to shorten against it and returning the response containing the tinyized URL.

<?php
require_once('Zend/Http/Client.php');
require_once('Recordshelf/Service/UrlShortener/Interface.php');
/** 
 * 'Class-level' PHPDoc Block   
 */
class Recordshelf_Service_TinyUrl implements
    Recordshelf_Service_UrlShortener_Interface
{
    /**
     * The service endpoint
     *
     * @var string
     */
    private $_serviceEndpoint = null;
    /**
     * Recordshelf service tinyURL constructor
     *
     * @param string $serviceEndpoint
     */
    public function __construct(
        $serviceEndpoint = 'http://tinyurl.com/api-create.php')
    {
        $this->_serviceEndpoint = $serviceEndpoint;
    }
    /**
     * Shortenizes a given Url
     *
     * @param string $url
     * @return string
     * @throws Exception
     */
    public function shortenize($url) {
        if (is_null($this->_serviceEndpoint)) {
            throw new Exception('No service endpoint set');
        }
        $client = new Zend_Http_Client($this->_serviceEndpoint);
        $client->setParameterGet('url', $url)
               ->setMethod(Zend_Http_Client::GET);
        try {
            $response = $client->request();
        } catch (Exception $e) {
            throw $e;
        }

        if (200 === $response->getStatus()) {
            return $response->getBody();
        } else {
            throw new Exception($response->getStatus() . ": " .
                $response->getMessage());
        }
    }
    /**
     * Alias method for the shortenize method
     *
     * @param string $url
     * @throws Exception
     * @see shortenize
     */
    public function tinyize($url)
    {
        return $this->shortenize($url);
    }
}

Now with everything hopefully operating smoothly it's time for a test-drive, yeah I'm lazy and cut that development approach called TDD, by creating a service instance and requesting a TinyURL for the Zend Framework website as shown in the outro listing.

<?php
$service = new Recordshelf_Service_TinyUrl();
$service->tinyize('http://framework.zend.com');
// => http://tinyurl.com/nf8kf

In case off considering or favouring a more framework independent approach there are also other blends available like one via file_get_contents or via curl. Happy tinyizing!

Getting a visualization of a Phing buildfile

2008-10-26T20:28:00.004+01:00

Today I spent some time to get a tool running to visualize Phing buildfiles as this can come in handy for maintaing, refactoring or extending large buildfiles. Out of the box the Phing -l option can be used to get a first overview of all available targets in a given buildfile but it doesn't untangle the target dependencies and sometimes a picture is still worth a thousand words. Luckily the Ant community already provides several tools to accomplish the visualization of Ant buildfiles, reaching from solutions that apply a Xslt stylesheet upon a given buildfile e.g. ant2dot to those ones that take a programmatically approach e.g. Grand. All these solutions utilize Graphiz to generate a graphic from a DOT file representing the buildfile structure, it's targets and their dependencies. As Phing is a very close descendant of Ant the Xslt approach was best suited and the one with the least effort because their buildfile markup is very similar. The following post will walk you through on how to get a simple Phing buildfile visualization tool running in just a few minutes.

Grabbing the Xslt file

The first step is to get the ant2dot Xslt stylesheet and put it into the same directory as the visualization buildfile and target to come. Due to the aforementioned Phing and Ant buildfile markup similarities it can be used without any modfications.

Setting up the buildfile visualization target

The next step is to create a Phing target that utilizes the Xslt task to transfrom the fed buildfile into a DOT file which gets passed further to a platform dependent Exec task handling the final transformation into a PNG image. To make the visualization target independent from the buildfile to visualize it's hosted in an own buildfile and the target accepts the buildfile to be transformed as a property passed to the Phing Cli or if none given uses the default build.xml. Further the Xslt stylesheet accepts several parameters to add extended data to the resulting DOT file/PNG image which can be set in the <param> tags of the Xslt task. For a list of possible parameters have a look at the options section of ant2dot. The following codesnippet shows the visualization buildfile and the visualize target doing the Whodini like magic.

<?xml version="1.0"?>

<project name="buildfile-visualizer" default="visualize" basedir=".">    

  <target name="visualize" 
          description="Generates a visualization(PNG image) of a given buildfile">
    <property name="buildfile" value="build.xml" />
    <property name="phing2dot.xsl" value="${project.basedir}/ant2dot.xsl" />
    <property name="dot.file" value="${buildfile}.dot" />
    <property name="png.file" value="${buildfile}.png" />
    <property name="dot.command.win" value="dot.exe -Tpng ${dot.file} -o ${png.file}" />
    <property name="dot.command.mac" value="dot -Tpng ${dot.file} -o ${png.file}" />
    <!-- Transform buildfile into DOT file -->
    <xslt file="${buildfile}" tofile="${project.basedir}/${dot.file}" 
          style="${phing2dot.xsl}" overwrite="true"> 
      <param name="graph.label" expression="${buildfile}" />
      <param name="use.target.description" expression="true" />
    </xslt>
    <!-- Generate image from DOT file -->
    <exec command="${dot.command.win}" 
          dir="${project.basedir}" os="WINNT" />
    <exec command="${dot.command.mac}" 
          dir="${project.basedir}" os="Darwin" />
    
    <delete file="${project.basedir}/${dot.file}" />
  </target>
    
</project>

Running the buildfile visualization target

Now as mostly all necessary pieces are available it's time to check if the DOT command is available on the targeted platform by running a dot(.exe) -V on the console. If it isn't available it has to be installed, this might take several minutes depending on the given platform. Finally with everything in place the visualization process/target can be kicked off by calling Phing the
following way.

triton:tmp stolt$ phing -f buildfile-visualizer.xml [-Dbuildfile=<targeted-buildfile.xml>]

The last picture shows the visualization of the simple buildfile described in the Phing Userguide but it's also possible to get a meaningful visualization of larger buildfiles like the one I currently use for setting up Zend Framework based projects.

Scraping websites with Zend_Dom_Query

2008-10-16T20:49:00.007+02:00

Today I stumbled upon an interesting and reportable scenario where I had to extract information of the weekly published Drum and Bass charts provided by BBC 1Xtra. As this information currently isn't available in any consumer friendly format like for example a RSS feed, I had to go that scraping route but didn't want to hustle with a regex approach. Since version 1.6.0 the Zend_Dom_Query component has been added to the framework mainly to support functional testing of MVC applications, but it also can be used for rolling custom website scrapers in a snap. Woot, perfect match!

The following code snippets are showing the Bbc_DnbCharts_Scraper class I came up with and an example of its usage. The class utilizes curl to read the website holding the desired data, which will be passed to Zend_Dom_Query to execute queries upon it. For querying the former loaded XHTML Document Object Model it's possible to either utilize XPath or CSS selectors. So I had to pick my poison, and decided to go with the CSS selectors as them were best suited for the document to query and will be more familiar to most jQuery or Prototype users. The query returns a result set of all matching DOMElements which are further unpuzzled via a private helper method returning just the desired charts data as shown in the closing listing. As you can see the implementation of the scraping can be done with a minimum of effort and these are exactly the moments I love the Zend Framework for.

<?php
require_once('Zend/Dom/Query.php');
/** 
 * 'Class-level' PHPDoc Block   
 */
class Bbc_DnbCharts_Scraper
{
    private $_url = null;
    private $_xhtml = null;

    /**
     * @param string $url
     */
    public function __construct($url)
    {
        $this->_url = $url;
    }
    /**
     * Scrapes off the drum and bass charts content from the BBC 1Xtra website.     
     *
     * @return array
     * @throws Exception
     */
    public function scrape()
    {
        try {
            $dom = new Zend_Dom_Query($this->_getXhtml());
        } catch (Exception $e) {
            throw $e;        
        }
        $results = $dom->query('div.chart div');
        $chartDetails = array();
        foreach ($results as $index => $result) {
            /* @var $result DOMElement */
            if ($result->nodeValue !== '') { //filter out <br /> element
                $chartDetails[] = $result->nodeValue;
            }
        }
        return $this->_unpuzzleChartDetails($chartDetails, true);
    }
    /**
     * Unpuzzles the chart details and groups them by their chart position, 
     * if desired with associative keys.
     *  
     * @param array $details
     * @param boolean $associative
     * @return array
     */
    private function _unpuzzleChartDetails(array $details, $associative = false)
    {
        if (0 === count($details)) {
            return array();
        } else {  
            $nextChartRank = 2;
            $charts = array();
            $groupedChartDetails = array();
            
            foreach ($details as $index => $chartDetail)
            { 
                if ($index <= $nextChartRank) {
                    $groupedChartDetails[] = $chartDetail;
                }
                if ($index == $nextChartRank) {
                    $nextChartRank+=3;
                    $charts[] = $groupedChartDetails;
                    unset($groupedChartDetails);
                }
            }
            if ($associative) {
                $associatives = array('artist', 'tune', 'label');
                foreach ($charts as $chartsIndex => $chart) {
                    unset($charts[$chartsIndex]);
                    foreach ($chart as $chartIndex => $chartDetails) {
                        $charts[$chartsIndex][$associatives[$chartIndex]] = 
                            $chartDetails;
                    }
                }
            }
            return $charts;
        }
    }
    /**
     * Gets the XHTML document via curl 
     * 
     * @return string
     * @throws Exception
     */
    private function _getXhtml()
    {
        $curl = curl_init();
        if (!$curl) {
            throw new Exception('Unable to init curl. ' . curl_error($curl));
        }
        curl_setopt($curl, CURLOPT_URL, $this->_url);
        curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
        // Faking user agent
        curl_setopt($curl, CURLOPT_USERAGENT, 
            'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');
        $xhtml = curl_exec($curl);
        if (!$xhtml) {
            throw new Exception('Unable to read XHTML. ' . curl_error($curl));
        }
        curl_close($curl);
        return $xhtml; 
    }
}

// Usage demo
$scraper = new Bbc_DnbCharts_Scraper('http://www.bbc.co.uk/1xtra/drumbass/chart/');
$charts = $scraper->scrape();

The closing code snippet shows an extract of the Drum and Bass charts from BBC 1Xtra scraped off around the 16th October 2008.

Array
(
    [0] => Array
        (
            [artist] => Chase & Status Ft Plan B
            [tune] => Pieces
            [label] => Ram Records
        )

    ...
    
    [9] => Array
        (
            [artist] => Zen
            [tune] => Full Effect
            [label] => Flipmode Audio
        )

)

Creating custom PHPUnit assertions

2008-07-28T18:38:00.006+02:00

While developing PHP applications and applying developer testing the applications safety net will grow along the timeline, and as normal code, test code should be in a fresh, odour free state too. A common test code smell, amongst others, is the duplication of assertion logic which can reduce reusability, readability and thereby obscure the specific verification intention of tests. To subdue this special smell several patterns and refactorings are available to acquaint the test code with the DRY principle. So in this blog post I'd like to set the focus on some of the aspects of the Custom Assertion pattern, by showing how to create custom PHPUnit assertions, which attacks the above mentioned smell and its retroactive effects with a huge antiperspirant flagon, while also providing the chance to build a customer friendly and domain related test vocabulary.

The first introductive code snippet shows an example of unwanted code duplications and smells in a PHPUnit(i.e. version: 3.2.21) test case class spreading over several test methods. The first test code duplications smell is present when verifying that a given bag is having an expected item count and an explicit 'intent obscuration' smell can be spotted when verifying the bags stock id against an assumed convention via a 'distracting' regular expression.

<?php
require_once 'PHPUnit/Framework.php';
require_once 'Record/Bag.php';
require_once 'Record/Item.php';

class Record_Bag_Test extends PHPUnit_Framework_TestCase 
{
    private $_bag = null;
    
    protected function setUp() 
    {
        /**
         * Creates a new named bag with an unique stock id
         * in a format of AAA-NN-AA-NNNNNNNN e.g. XDS-76-YS-00000124, 
         * where A stands for an alphanumeric and N for a numeric character.
         */
        $this->_bag = new Record_Bag('Test_Bag');
    }
    
    /**
     * @test
     */
    public function bagShouldNotContainDuplicateItems() 
    {
        $this->assertTrue(0 === $this->_bag->getItemQuantity(), 
            'New bag not empty on creation.');
            
        $this->_bag->addItem(new Record_Item('Dubplate 1'));
        $this->_bag->addItem(new Record_Item('Dubplate 2'));
        $this->_bag->addItem(new Record_Item('Dubplate 2'));
        
        $this->assertTrue(2 === $this->_bag->getItemQuantity(), 
            'Bag does contain duplicated items.');
    }
    
    /**
     * @test
     */
    public function bagShouldBeReducedByOneItemAfterRemoval()
    {
        $this->assertTrue(0 === $this->_bag->getItemQuantity(), 
            'New bag not empty on creation.');

        $this->_bag->addItem(new Record_Item('Dubplate 1'));
        $this->_bag->addItem(new Record_Item('Dubplate 2'));
        $this->_bag->addItem(new Record_Item('Dubplate 3'));

        $this->_bag->removeItem('Dubplate 2');

        $this->assertTrue(2 === $this->_bag->getItemQuantity(), 
            'Former three items bag does not contain two items after removal of one.');  
    }
    
    /**
     * @test 
     */
    public function bagStockIdShouldFollowAgreedConvention()     
    {    
        $stockIdPattern = '/^[A-Z]{3}-\d{2}-[A-Z]{2}-\d{8}/U';
        $this->assertRegExp($stockIdPattern, $this->_bag->getStockId(), 
            'Stock id <string:' . $this->_bag->getStockId() 
                . '> does not follow the agreed convention.'); 
    }
    
    ....
    
    protected function tearDown() 
    {
        unset($this->_bag);
    }
}

Mechanics of rolling custom assertions

The 'Custom Assertion' pattern can be applied from the very first test code creation activities, in sense of a prefactoring, or refactored towards by extracting the assert duplications and verification intention obscurations into tailormade and intent revealing named assertions.

To define custom assertions are several approaches available, the first and easiest one is to define custom assertions merely for a single test class and make them private (inline) methods of this specific class by facading/wrapping the standard PHPUnit assertion. This approach is outlined in code snippet a).

Another approach is to create/introduce an Assert Class to promote a cleaner reusability in other test case classes or scenarios, this approach might get chosen to collect and organize the evolving domain specific assertions. You will see a code sketch of this approach in code listing b).

Other and more complex approaches would be the utilisation of PHPUnits' Constraint feature which is available since release 3.0.0 or in the near future the use of the Hamcrest test matcher features which might be available from PHPUnit 4.0.0.

Either way the purpose specific assertions should further provide a default and convention conform assertion message, which will be raised on a verification failure, and also the feature to feed in a custom one to avoid gambling annoying Assertion Roulette rounds.

a) Assert definition inside Test Case:

<?php
require_once 'PHPUnit/Framework.php';
require_once 'Record/Bag.php';
require_once 'Record/Item.php';

class Record_Bag_Test extends PHPUnit_Framework_TestCase 
{
    private $_bag = null;
    
    protected function setUp() 
    {
        $this->_bag = new Record_Bag('Test_Bag');
    }
    
    ....
    
    private function assertBagItemCount($expectedCount, $bag, 
        $message = 'Expected bag item count <integer:#1> does not match actual count of <integer:#2> items.')
    {
        if (strpos($message, '#1')) {
            $message = str_replace('#1', $expectedCount, $message);        
        }
        if (strpos($message, '#2')) {
            $message = str_replace('#2', $bag->getItemQuantity(), $message);        
        }
        $this->assertTrue($expectedCount === $bag->getItemQuantity(), $message);
    }    
}

b) Assert definition in Assert Class:

<?php
require_once 'PHPUnit/Framework/Assert.php';

class Record_Bag_Assert extends PHPUnit_Framework_Assert 
{
    /**
     * Verifies that a given bag stock id follows the agreed convention.
     *
     * @param string $stockId
     * @param string $message
     * @see Record_Bag::createUniqueStockId()
     */
    public function assertStockIdFollowsConvention($stockId, 
        $message = 'Stock id <string:##> does not follow the agreed convention.') 
    {
        if (strpos($message, '##')) {
            $message = str_replace('##', $stockId, $message);        
        }
        $stockIdPattern = '/^[A-Z]{3}-\d{2}-[A-Z]{2}-\d{8}/U';
        $this->assertRegExp($stockIdPattern, $stockId, $message);        
    }
    
    ....
    
}

Putting the custom assertions to work

Now as the tailormade assertions are available all verification work can be delegated to them the same way it would be done with any of the standard PHPUnit assertions. In case the custom assertions are hosted in an 'Assert Class' they can be required_once or loaded via __autoload() in the test setup, otherwise if they are defined inside the test case class itself they can be used like regular private methods of that class. The last code extract illustrates the use of the prior outlined 'Assert Class' assertion for verifing the stock id format alongside the use of the 'inline' custom assertion for verifying the amount of items in a given bag.

<?php
require_once 'PHPUnit/Framework.php';
require_once 'Record/Bag/Assert.php';

class Record_Bag_Test extends PHPUnit_Framework_TestCase 
{
    
    ....
    
    /**
     * @test
     */
    public function bagShouldNotContainDuplicateItems() 
    {
        $this->assertBagItemCount(0, $bag,  'New bag not empty on creation.');
            
        $this->_bag->addItem(new Record_Item('Dubplate 1'));
        $this->_bag->addItem(new Record_Item('Dubplate 2'));
        $this->_bag->addItem(new Record_Item('Dubplate 2'));
        
        $this->assertBagItemCount(2, $bag, 'Bag does contain duplicate items.');
    }
    
    /**
     * @test
     */
    public function bagShouldBeReducedByOneItemAfterRemoval()
    {
        $this->assertBagItemCount(0, $bag, 'New bag not empty on creation.');

        $this->_bag->addItem(new Record_Item('Dubplate 1'));
        $this->_bag->addItem(new Record_Item('Dubplate 2'));
        $this->_bag->addItem(new Record_Item('Dubplate 3'));

        $this->_bag->removeItem('Dubplate 2');

        $this->assertBagItemCount(2, $bag, 
            'Former three items bag does not contain two items after removal of one.');   
    }
    
    /**
     * @test 
     */
    public function bagStockIdShouldFollowAgreedConvention()     
    {    
        Record_Bag_Assert::assertStockIdFollowsConvention(
            $this->_bag->getStockId());
    }
    
    ....
    
}

Adding value through a domain specific test vocabulary

As you might have noticed the readability and intention communication of the test code has been improved significantly from the introductive code snippet towards the last one. Furthermore by distilling a ubiquitous(see Domain Driven Design book by Eric Evans) test language domain experts, which are mostly not fluent in the targeted programming language i.e. PHP, are enabled to read test code and provide valuable feedback or contribute changes to test scenarios affecting their domain.

Another common area where domain specific test vocabularies are used, by providing their own assertion and constraint sets, test case classes and additional test helpers, are extensions to xUnit frameworks e.g. DBUnit for PHPUnit or the Zend Framework MVC testing scaffold.

Six valuable Phing build file refactorings

2008-07-04T01:16:00.008+02:00

Some ~~weeks~~ months ago I finally got my hands on the ThoughtWorks Anthology and got immediately hooked on one of the featured essays called 'Refactoring Ant Build Files' contributed by Julian Simpson aka the build doctor. After absorbing and studying the provided catalogue of overall 24 refactorings, I spent some time to transform a few health-promoting ones to the Phing universe. So the following post will outline six ~~five~~ basic, but valuable Phing build file refactorings by showing the smelly example first, followed by the scentless one and a closing refactoring description.

Making build files 'first-class' codebase citizens

You might ask yourself if there even is any need to refactor and care about mostly poorly treated project artifacts like build files. Well according to the book market there are growing needs and catalogues for refactoring non-sourcecode matters like databases and nowadays even (X)HTML, and as build tools are often used to automate the whole build and delivery process of complete projects, their feed build files should be readable and clear, painless maintainable and easily customizable as the controlled project takes new directions and hurdles.

Today build files are also often used to drive the Continuous Integration(CI) process and are heavily used to run local development builds prior to commiting the finished development tasks, therefor messy and clotty build files will have a counterproductive impact on the build management lifecycle and on making required changes to it. So when striving for codebase citizen equality agree upon a build file coding standard (e.g. 5. Introduce distinct target naming or a common element indentation), reside all build files of a project in a SCM system like Subversion or Git and try not to neglect the build files constantly just because they aren't actual business logic.

Trapeze balancing without a safety net

While developing an applications business logic 'ideally' automated behaviour verifying tests are written, whether in a test-first or test-last approach, and these are building the implicit and required safety net for all follow-up refactorings. For build files there are currently no tailored testing tools/frameworks available to warrant their external behaviour during and after a refactoring, though it might be possible to create a basic safety net by using for example a combination of PHPUnit's assertFileExists and assertContains assertions. And even if there were such tools available, it's rather questionable that these would be applied to test-drive mostly simple starting and incremental evolving build files. So currently this flavour of refactoring needs to be applied with much more descipline and caution than classic sourcecode refactorings, even 'old-school' manual tests have to be run frequently and only with a proceeding practice more and more agressive refactorings will become an everyday tool. After this short note of caution, let us jump right into the refactoring catalogue extract.

1. Extract target

Take parts of a large target, declare them as independent targets and preserve dependencies by using the
target depends attribute.

Before

<target name="what-a-clew">
  
  <phplint>
    <fileset dir="${build.src}">
      <include name="**/*.php"/>
    </fileset>
  </phplint>
  
  <phpcodesniffer standard="PEAR" format="summary">
    <fileset dir="${build.src}">
      <include name="**/*.php"/>
    </fileset>
  </phpcodesniffer>
  
  <phpunit>
    <formatter todir="reports" type="xml"/>
    <batchtest>
      <fileset dir="="${build.tests}">
        <include name="**/*Test*.php"/>
      </fileset>
    </batchtest>
  </phpunit>

</target>

After:

<target name="phplint-report">
  <phplint>
    ....
  </phplint>
</target>

<target name="sniff-report" depends="phplint-report">
  <phpcodesniffer standard="PEAR" format="summary">
    ....
  </phpcodesniffer>
</target>

<target name="test-report" depends="sniff-report">
  <phpunit>
    ....
  </phpunit>
</target>

The Extract target refactoring is similar to the well-known Extract method refactoring catalogued by Martin Fowler and should be applied to unclutter long targets, which can become hard to understand and troubleshoot while maintaining or extending a build file. This refactoring is achieved by taken each atomic task (e.g. phplint) of the cluttered target and provide them each a own target (e.g. phplint-report) while the former tasks execution sequence can be obtained by utilizing the target depends attribute. You can compare this refactoring to the technique of tackling a method that's to large and infringes upon the single responsibility principle.

While twiddling with this refactoring I came up with a follow-up and hand in hand refactoring that might go by the name of Introduce facade target, which simply orchestrates the target execution sequence so you can remove the depends attribute of all orchestrated targets and thereby use them separately if needed and advisable. The following build file extract shows the result of this refactoring in action.

1.1 Introduce facade target

Provide a facade target to obtain the task execution sequence and to make each involved target a single callable unit.

After:

<target name="phplint-report">
  <phplint>
    ....
  </phplint>
</target>

<target name="sniff-report" depends="phplint-report">
  <phpcodesniffer standard="PEAR" format="summary">
    ....
  </phpcodesniffer>
</target>

<target name="test-report" depends="sniff-report">
  <phpunit>
    ....
  </phpunit>
</target>

<target name="quality-report" depends="lint-report, sniff-report, test-report" 
        description="Generates the overall projects quality report" />

2. Introduce property file

Move infrequently changing properties from the build file body to a flat file.

Before:

<property name="db.port" value="3306" />
<property name="db.name" value="example" />
<property name="db.user" value="funkdoc" />
....
<property name="runtime.property.x" value="default" />
....

After:


build.poperties file
[example properties]
db.port = 3306
db.name = example
db.user = funkdoc
....

build file
<property file="build.properties" />
<property name="runtime.property.x" value="default" />
....

The Introduce property file refactoring can be applied for moving infrequently changing or static properties out of the main build file body to raise the overall legibility and keep them distinct from runtime properties. The downside of this refactoring is a lost of property visibility and breaking up the former single build file into multiple units, which is contradictory to the third ANT best practice named 'Prefer a Single Buildfile' of an older best practice catalogue compiled by Eric M. Burke. So in this case, like in any case, you have to make your own choice based on your needs and requirements.

3. Replace comment with description

Annotate elements(targets) with the description attribute instead of XML comments.

Before:

<!-- This target runs the PHP_Codesniffer task and reports coding standard violations --!>
<target name="sniff-report">
  ....
</target>

After:

<target name="sniff-report" description="Runs the PHP_Codesniffer task and reports coding standard violations">
  ....
</target>

Often build files are accentuated with plain XML comments to retain the mechanics and purpose of build file elements(i.e. targets) and can become a diversionary/obscuring source while maintaining or extending a build file. By using the available description attribute of the target element to annotate its purpose it's possible to reduce that kind of noise and even better, if used constantly, they can provide valuable information about all accumulated targets of a build file when phing is called with the -l(ist) option. As you can see the Replace comment with description refactoring requires a minimum of effort/investment to achieve a very valuable impact.

4. Reuse elements by id

Declare an instance e.g. a fileset once and make references to it elsewhere to reduce duplication and increase clarity.

Before:

<target name="phplint-report">
  <phplint>
    <fileset dir="${build.src}">
      <include name="**/*.php"/>
    </fileset>
  </phplint>
</target> 

<target name="sniff-report" depends="phplint-report">
  <phpcodesniffer standard="PEAR" format="summary">
    <fileset dir="${build.src}">
      <include name="**/*.php"/>
    </fileset>
  </phpcodesniffer>
</target>

After:

<fileset id="src_artifacts" dir="${build.src}">
  <include name="**/*.php"/>
</fileset>

<target name="phplint-report">
  <phplint>
    <fileset refid="src_artifacts" />
  </phplint>
</target> 

<target name="sniff-report" depends="phplint-report">
  <phpcodesniffer standard="PEAR" format="summary">
    <fileset refid="src_artifacts" />
  </phpcodesniffer>
</target>

The Reuse elements by id refactoring is, as the short description states, tailor-made to increase clarity while reducing code duplication, which is when present a risk that an alternation made to one element will be skipped for the other duplicates, by declaring top-level elements once by assigning an id attribute to it and then referring to it thoughout the rest of the build file. This refactoring is best compared to the classic sourcecode refactoring called Pull Up Method also catalogued by Martin Fowler and moreover it enforces the compliance with the DRY principle by providing a single point of change for futurities alternations.

5. Introduce distinct target naming

Use a different punctuation for targets and properties to enhance readability.

Before:

<property name="example.property1" value="abc" />
<property name="example_property2" value="def" />
<property name="example-Property3"  value="ghi" />

<target name="example.target1">
   <echo msg="${example.property1}" />
   ....
</target>

<target name="example-target2">
   <echo msg="${example-Property3}" />
   ....
</target>

After:

<property name="example.property1" value="abc" />
<property name="example.property2" value="def" />
<property name="example.property3" value="ghi" />

<target name="example-target1">
   <echo msg="${example.property1}" />
   ....
</target>

<target name="example-target2">
   <echo msg="${example.property3}" />
   ....
</target>

The Introduce distinct target naming refactoring once again tackles the improvement of readability in a build file by applying a constant and different punctuation on the common elements: targets and properties. The appliance of this refactoring leaves you and coworkers with an immediate reply whether you're looking at a property value or a target and can lead towards an agreed upon in-house/project build file coding standard. For target names underscores and dashes are suitable, although dashes are preferred by me as a hypen is used when enforcing internal targets, while for the build properties dots should be considered to build namespaces and their names should 'always' be lowercased except for environment variables/properties.

In case this blog post whetted your appetite for more, heavier and here uncovered build file refactorings, you might consider picking up a copy of the ThoughtWorks Anthology book. Last but not least a shout out goes to the build doctor for the remix permission and until the next post I'm ghost like dog.

Hooking a Growl publisher plugin into Xinc

2008-04-17T02:37:00.009+02:00

This week I finally had the time to setup Xinc, PHP's shiny new Continuous Integration(CI) server, on my MacBook and started to fiddle with it's publisher plugins and plugin architecture. While still down with 'tool envy' influenza I recently came across some nice and inspiring blog posts from the Ruby/Rails camp, showing how to employ Growl as an useful and fun feedback radiator for test/spec results. Since then the idea of building a Growl publisher plugin for Xinc was travelling my mind repeatedly, so the following post will break this circle and show a possible approach to build such a plugin, which can be used to notify the build result for continuously integrated projects and thereby provide an on-point/immediate feedback.

Building the Growl publisher plugin

Xinc comes with an elegant plugin system, allowing you to easily role your own one. The following code shows the plugin class and is the place where the actual Growl notifications are executed/raised by calling the growlnotify CLI. The images in the public growl() method, used to accent the two main build statuses(failure and success) in the Growl notification, are borrowed from this blog post and should be located in the $HOME/Pictures directory.

<?php
/**
 * A Growl publisher plugin for Xinc
 *
 * @package Xinc.Plugin
 * @author Raphael Stolt
 * @version 2.0
 * @copyright 2008 Raphael Stolt, Constance
 * @license  http://www.gnu.org/copyleft/lgpl.html GNU/LGPL, see license.php
 */
require_once 'Xinc/Plugin/Base.php';
require_once 'Xinc/Plugin/Repos/Publisher/Growl/Task.php';

class Xinc_Plugin_Repos_Publisher_Growl extends Xinc_Plugin_Base
{
    public function validate()
    {
        return true;
    }
    public function getTaskDefinitions()
    {
        return array(new Xinc_Plugin_Repos_Publisher_Growl_Task($this));
    }
    private function _sendGrowlNotification($message, $image, $name)
    {
        $command = "growlnotify -w -m '{$message}' "
                 . "-n '{$name}' "
                 . "-p 2 --image {$image}";

        Xinc_Logger::getInstance()->info('Executed growlnotify command: ' . $command);

        exec($command, $response, $return);

        if ($return === 0) {
            return true;
        }
        return false;
    }
    public function growl(Xinc_Project &$project, $message, $buildstatus, $name = 'Xinc')
    {
        if ($buildstatus === Xinc_Build_Interface::PASSED) {
            $image = '$HOME/Pictures/pass.png';
            $buildstatus = 'PASSED';
        } elseif ($buildstatus === Xinc_Build_Interface::FAILED) {
            $image = '$HOME/Pictures/fail.png';
            $buildstatus = 'FAILED';
        }

        $project->info('Executing Growl publisher with content '
                      ."\nMessage: " . $message
                      ."\nBuildstatus: " . $buildstatus
                      ."\nImage: " . $image);

        return $this->_sendGrowlNotification($message, $image, $name);
    }
}

The next big code chunk shows the task definition of the plugin and is the place where the publisher plugin name and is acceptable and required attributes are programmaticly defined, as you can see the Growl task currently only takes a required message attribute which can be set for the Growl publisher within the Xinc project configuration file.

<?php
/**
 * A Growl publisher plugin task for Xinc
 *
 * @package Xinc.Plugin
 * @author Raphael Stolt
 * @version 2.0
 * @copyright 2008 Raphael Stolt, Constance
 * @license  http://www.gnu.org/copyleft/lgpl.html GNU/LGPL, see license.php
 */
require_once 'Xinc/Plugin/Repos/Publisher/AbstractTask.php';

class Xinc_Plugin_Repos_Publisher_Growl_Task extends Xinc_Plugin_Repos_Publisher_AbstractTask
{
    private $_message;

    public function setMessage($message)
    {
        $this->_message = $message;
    }
    public function getName()
    {
        return 'growl';
    }
    private function _isGrowlnotifyAvailable()
    {
        exec('growlnotify -v', $reponse, $return);

        if ($return === 0) {
            return true;
        }
        return false;
    }
    public function validateTask()
    {
        if (!$this->_isGrowlnotifyAvailable()) {
            $message = 'The growlnotify command seems to be not available '
                     . 'on this system';
            Xinc_Logger::getInstance()->error($message);
            throw new RuntimeException($message);
        }

        if (!isset($this->_message)) {
              $message = 'Element publisher/growl - required attribute '
                       . '\'message\' is not set';
              throw new Xinc_Exception_MalformedConfig($message);
        }
        return true;
    }
    public function publish(Xinc_Build_Interface &$build)
    {
        $statusBefore = $build->getStatus();
        $res = $this->_plugin->growl($build->getProject(), $this->_message, $build->getStatus());
        if (!$res && $statusBefore == Xinc_Build_Interface::PASSED ) {
            /**
             * Status was PASSED, but now the publish process made it fail
             */
            $build->setStatus(Xinc_Build_Interface::FAILED);
        }
    }
}

Hooking the Growl publisher plugin into the build system

To make the just crafted Growl publisher plugin available to the Xinc build system you have to add the plugin file and class name/path to the /etc/xinc/system.xml file. After restarting the Xinc server the Growl publisher should be available to use and also be listed in the section of the /var/log/xinc.log file stating all registered plugins.

The next listing shows the Growl publisher plugin added to the aforementioned XML file.

<?xml version="1.0" encoding="UTF-8"?>
<xinc>
  <configuration>
    <setting name="loglevel" value="2"/>
    <setting name="timezone" value="Europe/Berlin"/>
  </configuration>
  <plugins>
    <plugin filename="Xinc/Plugin/Repos/ModificationSet.php" classname="Xinc_Plugin_Repos_ModificationSet"/>
    ...
    <plugin filename="Xinc/Plugin/Repos/Publisher/Growl.php" classname="Xinc_Plugin_Repos_Publisher_Growl"/>
    ...
    <plugin filename="Xinc/Contrib/Warko/Plugin/ModificationSet/SvnTag.php"
            classname="Xinc_Contrib_Warko_Plugin_ModificationSet_SvnTag"/>
  </plugins>

  <engines>
    <engine classname="Xinc_Engine_Sunrise" filename="Xinc/Engine/Sunrise.php" default="default"/>
  </engines>
</xinc>

Putting the Xinc Growl publisher plugin to action

Now that the Growl publisher plugin is available it can be used in the build cycle. The following Xinc project configuration file shows the utilization of the Growl publisher plugin in both of the main publisher realms onsuccess and onfailure.

<?xml version="1.0" encoding="UTF-8"?>
<xinc engine="Sunrise">
  <project name="recordshelf">
    <configuration>
      <setting name="loglevel" value="2"/>
    </configuration>
    <property name="dir" value="${projectdir}/${project.name}" />
    <property name="build.failure.message" value="Build for project ${project.name} failed!"/>
    <schedule interval="180" />
    <modificationset>
      <svn directory="${dir}" update="true" />
    </modificationset>
    <builders>
      <phingbuilder buildfile="${dir}/build.xml" target="main"/>
    </builders>
    <publishers>
      <onfailure>
        <email to="example@example.com" subject="[${project.name}] Build failure"
               message="${build.failure.message}" />
        <growl message="${build.failure.message}" />
      </onfailure>
      <onsuccess>
        <email to="example@example.com" subject="[${project.name}] Build success"
               message="Build for project was successful" />
        <growl message="Build for project ${project.name} was successful." />
      </onsuccess>
    </publishers>
  </project>
</xinc>

The next two images are finally showing the Growl notifications for a successful and a failed build. To minimize the notification noise it's a good and common practice to only 'ring the alarm' for failed builds, which can be achieved by using the Growl publisher only in the onfailure publisher realm of the Xinc project configuration file. Nuff talk, happy Xincing!

Getting an overview of all targets accumulated in a Phing build file

2008-03-29T19:04:00.006+01:00

Looking for an equivalent to Ant's -p(rojecthelp) command-line option in Phing, I dug up that you can use Phing's -l(ist) option to get a quick overview of all targets hosted in a given build file. This comes in handy when you are maintaining build files or have to get a raw picture of the provided targets of a project specific build file. The following console output shows the targets hosted in a example build file. As you will see, by using the -l command-line option you get an overview of the default target, the main targets and further subtargets.

triton:work stolt$ phing -l
Buildfile: /Volumes/USB DISK/work/build.xml
Build file for recordshelf project
Default target:
-------------------------------------------------------------------------------
 help                Displays the help for this build file

Main targets:
-------------------------------------------------------------------------------
 build               Builds the project
 clean               Removes data left over by former build
 cs-coding-standard  Runs the coding standard code inspection
 help                Display the help for this build file
 init                Initializes the build process
 layout              Creates the project layout for the build
 lint-project        Lints all code artifacts
 test-components     Runs the automated component tests
 test-integration    Runs the automated integration tests
 test-units          Runs the automated unit tests

Subtargets:
-------------------------------------------------------------------------------
 build-initial-model

A target becomes a main target when it has an non-empty description attribute, otherwise it will be a subtarget. This can be used to define the visibility and thereby 'callability' of targets, similar as you would do when defining access specifiers for class members of a PHP class. Main targets would become the public targets of your build file and all subtargets could be considered as private targets, which shouldn't be called directly as they might/should be used to provide specific helper functionality to public main targets or target wrappers. To definitively veto the call to a target, which never should be run directly via the Phing CLI(e.g. phing tragetname), just define an invalid target name(i.e. -build-initial-model instead of build-initial-model) so the Phing CLI will respond with an error and thereby not run the build file. The next console output shows the Phing CLI response when calling a private target.

triton:work stolt$ phing -build-initial-model
Unknown argument: -build-initial-model
phing [options] [target [target2 [target3] ...]]
Options:
  -h -help               print this message
  -l -list               list available targets in this project
  -v -version            print the version information and exit
  -q -quiet              be extra quiet
  -verbose               be extra verbose
  -debug                 print debugging information
  -logfile <file>        use given file for log
  -logger <classname>    the class which is to perform logging
  -f -buildfile <file>   use given buildfile
  -D<property>=<value>   use value for given property
  -find <file>           search for buildfile towards the root of the
                         filesystem and use it
  -inputhandler <file>   the class to use to handle user input

Report bugs to <dev@phing.tigris.org>

This post is somehow a partial and PHP flavoured remix of a post by Julian Simpson, who runs the build doctor blog and is also the author of the 'Refactoring Ant Build Files' contribution to the upcoming ThoughtWorks Anthology book.