#! code blogs

PHPNW13 Call For Papers And Blind Bird Ending Soon!

philipnorton42 — Tue, 18 Jun 2013 21:15:56 +0000

The 5th annual PHPNW conference is set for October 5th and 6th this year and preparations for the Manchester conference are in full swing. If you want to talk at the conference then the call for papers is set to close on the 23rd June. The speaker package includes accommodation for the Friday and Saturday night as well as a speakers dinner on the Friday night. If you want some help in submitting a talk then take a look at Lorna Mitchell's blog post on submitting a talk proposal. The call for papers page can be found on the PHPNW13 website.

Once the call for paper submissions are closed it will only be a few days before the talks start being announced, at which point the blind bird ticket price of £95 will increase to the early bird ticket prices. The ticket gets you two full days of excellent talks, free tea and coffee, two full meals and a free bar (whilst it lasts) on the Saturday night. As there are only a few days left before the blind bird tickets are stopped then you should head over to the PHPNW13 conference site ticket page and get them.

The conference has got better year on year and this years event is set to be just as good as ever. It really is one of the best development conferences in the UK and absolutely worth the cost of a ticket. If you are using PHP at all and can visit Manchester in October then you absolutely must attend. The talks are of the highest calibre and I always learn a lot each year.

If you are interested in seeing what has happened in previous years PHPNW conference then take a look at our previous reviews.

I'll be there to represent #! code and I hope you will attend to!

Category:

PHP

Tags:

phpnw

PHPNW13

Copying Files With Secure Copy

philipnorton42 — Sat, 15 Jun 2013 17:52:19 +0000

The secure copy command (run using scp) is a Linux command that allows the transfer of files between two computers. This can be locally to a remote server, from a remote server to a local location, or even between two remote servers.

When copying to or from a remote host scp uses ssh for the data transfer. This means that authentication is required, but the files are copied in a secure fashion. When starting a scp request the command first sets up an ssh connection to the remote location, which is then used for the rest of the transfer.

It is also possible to copy the files on a local hard drive, but in this case you should probably use the standard cp command.

Secure copy is an easy command to use and has a pretty straightforward syntax. I tend to use scp more than other remote file copying tools (like rsync) as I can easily remember the flags and syntax for the command. It is also much more secure than other transfer mechanisms like the insecure FTP.

scp has the syntax of the 'source' followed by the 'destination'. Either the target or the destination can be written as a remote reference. For example, to copy a local file to a remote server your would use the following.

sci source.txt user@host:/destination/directory/target.txt

Using the above will also change the name of the file from source.txt to target.txt so care must be taken if you want the file to have the same name. An alternative to the above mechanism is to leave out the target filename, this will then use the original filename specified in the copu.

sci source.txt user@host:/destination/directory

When copying to a remote server it is also a good idea to use the full directory of the destination. The default directory used is the ssh user's home directory. A full path is needed to set the destination to be elsewhere.

If you want to copy a file from a remote server to a local directory you just need to put the remote part first.

scp user@host:/source/directory/source.txt target.txt

To copy the contents of an entire directory use the -r flag. This recursively copies the directory and any sub-directories. This will copy the source directory into the destination directory and so you don't need to add the directory name here.

scp -r source/directory user@host:/destination

If your ssh server is using a non-standard port then you can use the -P flag to let scp know what port to use for the remote connection. The following uses the port 6001 to connect to the remote host.

scp -P 6001 source.txt user@host:/destination/source.txt

Note that the port flag is a capital 'P' because the lower case -p flag is used to preserve the modification times, access times and modes from the original file.

When you run a scp command each file that is copied will be given a progress meeter showing the time left to complete the transfer. This is useful for larger files but is pointless when copying lots of smaller files as each files progress meeter will only be seen for an instant. To turn off the progress meeter use the -q flag.

If you want to save on bandwidth you have the option of limiting the bandwidth to a citation threshold with the -l flag (specified in Kbit/s). Alternatively, you can try to allow compression to be used on the files before they are transmitted with the -C flag, although not all versions of ssh support this.

These are the basic controls for using the scp command. There are a few more flags available that control how the ssh connection is used and are only really useful if you have a special ssh setup.

Category:

Linux/Unix

Tags:

linux

SSH

Control Structures In Phing

philipnorton42 — Sun, 02 Jun 2013 10:52:15 +0000

Phing has a few different tasks and elements that allow you to select paths of code execution depending on what you need to happen in a build file. These are limited to loops and if statements, but a lot of functionality can be covered with just a couple of lines of XML.

Foreach Task

The foreach task iterates over a list of items, and/or fieldsets. For every item found in the loop a phing target will be called with the item in the list passed as a parameter to the target. The foreach target requires the following two attributes.

target : This is the phing target to call on each parameter found in the param attribute.
param : This is the parameter that will be passed to the target found in the target attribute.

The param attribute can contain either a property or a placeholder for a fileset. The property can be split in the same way as the explode() PHP function to split the string into an array. You can use the delimiter attribute to alter the separator, which defaults to a comma (,), but you must also include a list attribute to let the foreach target know what property to use.

Here is a simple example of a foreach task that loops through a series of numbers and prints them out in turn.

<?xml version="1.0"?>
<project name="foreachtest" default="foreachtest"> 
 
  <target name="foreachtest">
    <property name="list" value="1,2,3" />
    <foreach list="${list}" param="item" target="itemtarget" delimiter="," /> 
  </target>
 
  <target name="itemtarget">
    <echo>${item}</echo>
  </target>
 
</project>

This produces the following output.

foreachtest > foreachtest:
 
  [foreach] Calling Buildfile '/foreach.xml' with target 'itemtarget'
 
foreachtest > itemtarget:
 
     [echo] 1
  [foreach] Calling Buildfile '/foreach.xml' with target 'itemtarget'
 
foreachtest > itemtarget:
 
     [echo] 2
  [foreach] Calling Buildfile '/foreach.xml' with target 'itemtarget'
 
foreachtest > itemtarget:
 
     [echo] 3
 
BUILD FINISHED
 
Total time: 0.0632 seconds

The foreach target can also include a fileset or a mapper element as a nested argument. If this is the case then the parameter attribute will be used as a placeholder for the output of the nested elements. You can either create a new fileset within the foreach task, or just add a reference to one. The following Phing project creates a fileset and passes this to a foreach target, which prints each filename in turn. The directory I am using here is a common sounds directory found on most Debian/Ubuntu systems and so should produce some sort of result.

<project name="foreachtest" default="foreachtest"> 
 
 <target name="foreachtest"> 
   <fileset dir="/usr/share/sounds/alsa/" id="thefiles">
     <include name="**" />
   </fileset>
 
   <foreach param="filename" target="foreachfile">
     <fileset refid="thefiles" />
   </foreach>
 </target>
 
 <target name="foreachfile">
   <echo>${filename}</echo>
 </target>
</project>

The idea here is that you would do more than just print out the name of the file. You could change the permissions of the file or perhaps something more involved. The benefits of using this approach mean that you have a target that does a single action on a file that can be used anywhere in your project, not just as the target of the foreach loop.

Conditions

Conditions are nested elements that are used in condition and if tasks to evaluate expressions and need to be understood before those elements can be used. Essentially, a conditional will evaluate something and return true or false. They can be split into comparators that compare two values and modifiers that change the output of a comparison in some way.

None of the comparators work on their own. Instead they are used in the condition and if tasks in order to drive those conditional statements. It is essential to understand these elements before looking at the condition and if elements.

Comparators

This consists of a series of comparison operators that compare one or more values and return a boolean result. The tables in this section were taken directly from the official Phing support documentation on conditions.

equals

This is the basic comparator and essentially compares two strings (provided by the arg1 and arg2 parameters) to one another. If the two strings are identical then the element returns true. The casesensitive parameter can be used to force the comparison to be case or non case sensitive. The trim parameter can also be used to trim whitespace from the arguments before they are compared. The two arguments can also accept properties as arguments. All of the following examples will return true.

<equals arg1="text" arg2="text" />
<equals arg1="TEXT" arg2="text" casesensitive="false" />
<equals arg1="TEXT" arg2=" text " trim="true" casesensitive="false" />
 
<property name="property1" value="TExt" />
<equals arg1="${property1}" arg2="text " trim="true" casesensitive="false" />

Attribute	Description	Required
arg1	First string to test.	Yes
arg2	Second string to test.	Yes
casesensitive	Perform a case sensitive comparison. Default is true.	No
trim	Trim whitespace from arguments before comparing them. Default is false.	No

isset

The isset element returns true if a property has been set within the project scope. This is useful for checking to see if certain properties have been assigned before trying to use them. If the property hasn't been set then a property prompt or even a failure event can be used to act accordingly.

1 2	<property name="property1" value="text" /> <isset property="property1" />

Attribute	Description	Required
property	The name of the property to test.	Yes

contains

This element tests to see if one string contains another string. The string attribute is the string to search in and the substring attribute is the string to search for. If the string attribute contains substring then the condition will return true. The casesensitive attribute is a boolean value that can be used to turn on or off case sensitivity, the default being on. All of the following examples return true.

<contains string="some string" substring="me str" />
<contains string="SOME string" substring="me str" casesensitive="false" />
<contains string="abcdef" substring="a" />
<contains string="aBcDe" substring="aB" />
<contains string="aBcDe" substring="ab" casesensitive="false" />

Attribute	Description	Required
string	The string to search in.	Yes
substring	The string to search for.	Yes
casesensitive	Perform a case sensitive comparision. Default is true.	No

istrue

This tests to see if a string evaluates to true. This uses standard PHP boolean string casting so any non-empty string will equate to true, whereas values like 'false', 0 or empty strings equate to false. All of the following examples will return true.

<istrue value="1" />
<istrue value="true" />
<istrue value="yes" />
<istrue value=" " />

Attribute	Description	Required
value	value to test	Yes

isfalse

This is essentially the negative of istrue, but will return true if the value is false. All of the following examples will return true.

1
2
3

<isfalse value="0" />
<isfalse value="false" />
<isfalse value="" />

Attribute	Description	Required
value	value to test	Yes

os

This tests to see if an operating system is of a given type. The element takes one attribute of family which can have one of three values. For Microsoft Windows based systems this is 'windows', for Apple Macintosh systems this is 'mac' and for Unix-like operating systems this is 'unix'. For example, if the family attribute is 'unix' and the script was being run on Ubuntu then this would return true. This is a pretty good way of running different functionality on different platforms and the options for each system are pretty simple.

1	<os family="unix" />

Attribute	Description	Required
family	The name of the operating system family to expect.	Yes

referenceexists

The referenceexists condition tests to see if a specified reference exists. A reference is a Phing type that references files, which includes filset, filelist, filterchains, chains, and file mappers.

The following will return true because the filset reference 'someid' exists.

<fileset id="someid">
  <include name="*.php" />
</fileset>
 
<referenceexists ref="someid" />

Attribute	Description	Required
ref	reference to test for	Yes

available

The available comparison is used to test to see if a file or directory exists. The file attribute details the file or directory name to be looked for. This can be swapped for the resource attribute to look for a specific path or the extension attribute to find a specific extension. The type attribute is used to differentiate between files and directories and the filepath attribute dictates which path should be used to look for the file.

This comparator works in the same way as the available task with the exception that the property and value attributes do nothing and will be ignored. It is a good way of double checking that things have worked correctly during the build. If the primary function of the build file is to create a compressed file then a check should be done to make sure the compressed file exists.

Using the following should always return true as this will find the build file.

<available extension="xml" />

The following will detect the /home directory on a Unix based system.

<available file="/home" type="dir" />

The following will return true if the the file index.php is within the directory /some/path.

<available file="index.php" type="file" filepath="/some/path/" />

Name	Type	Description	Default	Required
file	String	File/directory to check existence.	n/a	Yes (or resource or extension)
resource	String	Path of the resource to look for.	n/a	Yes (or file or extension)
extension	String	Name of the extension to look for.	n/a	Yes (or file or resource)
type	String (file\|dir)	Determines if AvailableTask should look for a file or a directory at the position set by file. If empty, it checks for either file or directory.	n/a	No
filepath	String	The path to use when looking up file.	n/a	No
followSymlinks	Boolean	Whether to dereference symbolic links when looking up file.	false	No

Modifiers

Modifiers are used to modify the result of a comparison in some way. They do not compare things themsevles but they can be used to group together multiple comparisons or to negate a comparison. None of the comparison modifiers have any arguments.

not

This is the simplest form of modifier and will just negate the result of the comparison. This is a good way of testing to see if a property is set and acting accordingly using the isset comparison. The not comparison modifier cannot accept more than one condition, but it is possible to nest multiple comparisons within another comparison modifier and negate that. The following example will negate the result of the istrue comparison of false, which essentially returns true.

1
2
3

<not>
  <istrue value="false" />
</not>

The following is an example of nesting multiple comparisons into an and comparison modifier and then negating the result using the not modifier. In this case the result of the and comparison is false, but this negates and so the overall result is true.

<not>
  <and>
    <istrue value="false" />
    <istrue value="0" />
  </and>                        
</not>

and

The and comparison modifier compares the result of multiple nested comparisons. The result is true if all of the inner comparisons are equal to true. The conditions will be evaluated in the order they appear in the build file. As soon as a condition is evaluated to false then no other conditions are evaluated in the rest of the and element. The following is an example of an and element comparing multiple nested comparison elements to return true.

<property name="property1" value="text" />
 
<and>
  <equals arg1="text" arg2="text" />
  <isset property="property1" />
  <contains string="some string" substring="me str" />
  <istrue value="1" />
  <isfalse value="0" />
</and>

or

This comparison modifier will evaluate multiple nested comparisons and return true if at least one of those conditions is true. The comparisons will be evaluated in the order they are found in the build file. When a true value is found the element will return true immediately and no other comparisons will be evaluated. The following example will evaluate the nested conditions and return true because at least one of them returns a true value.

<or>
  <equals arg1="text" arg2="TEXT" />
  <contains string="some string" substring="me str" />
</or>

Condition Task

The condition task is used to set a property if a condition is true. By default this property is set to be a boolean (i.e. the result of the comparison) but this can be changed by using the value attribute. The condition task must have exactly one nested comparison, although comparison modifiers (such as and) can be used when multiple comparisons need to be done. If the condition is not met then the property is not set. Here are a few examples of the condition task in action.

The following example sets the property 'textisthesame' to be true, based on the output of the equals comparison.

1
2
3

<condition property="textisthesame">
  <equals arg1="text" arg2="text" />
</condition>

The following example sets the property 'sametextandmac' to be true, based on the output of both the equals comparison and the build being run on a Mac computer.

<condition property="sametextandmac">
  <and>
    <equals arg1="text" arg2="text" />
    <os family="mac" />
  </and>
</condition>

The following example sets the property 'textisthesame' to be the string 'text' due to the value attribute being used in the condition task.

1
2
3

<condition property="textisthesame" value="text">
  <equals arg1="text" arg2="text" />
</condition>

The following example will not set the property because the result of the equals comparison is false. If you try to use this property elsewhere in the build file you will just print the name of the property.

1
2
3

<condition property="textisthesame">
  <equals arg1="text" arg2="notthesame" />
</condition>

Name	Type	Description	Default	Required
property	String	The name of the property to set.	n/a	Yes
value	String	The value to set the property to. Defaults to "true".	true	No

If Task

http://www.phing.info/docs/guide/stable/chapters/appendixes/AppendixB-Co...

The if task is a bit more complex than the condition task in that it is able to direct the flow of the execution of the build file. It works in much the same way that an if statement is used in normal programming, although it can be a little difficult to understand at first due to the nested XML syntax. The if task has no attributes and on it's own doesn't really do all that much, the real work is done by a comparison and the then, else, and elseif child elements. These child elements direct the flow of the build file depending on the outcome of the if comparison. The then and else child elements must be used only once within an if statement.

The elseif element acts like a sub if task and so must also have it's own comparison along with an then or elseif elements. The only exception here is that the elseif element must not contain an else element. It is also possible to nest if tasks to create complex comparisons. Here are a few examples of the if statement in action.

The following if task compares two properties and prints a message if they are the same.

<property name="text1" value="text" />
<property name="text2" value="text" />
 
<if>
  <equals arg1="${text1}" arg2="${text2}" />
  <then>
    <echo message="The two strings are the same." />
  </then>
</if>

The following is an example of using the and condition modifier in an if task. This just tests to see if two values are false and prints a message if they are.

<if>
  <and>
    <isfalse value="false" />
    <isfalse value="0" />
  </and>
  <then>
    <echo message="The two values are false." />
  </then>
</if>

The following if task will print a message depending on whether or not it is run on Windows.

<if>
 <os family="windows" />
 <then>
   <echo>You are running this build file on Windows.</echo>
 </then>
 <else>
   <echo>You are not running this build file on Windows</echo>
 </else>
</if>

The following if task will detect what operating system the build file is being run on and print a message. Notice that the elseif element here contains it's own comparison and then child element in order to compare and act on the comparison.

<if>
  <os family="windows" />
  <then>
    <echo message="This build file is being run on Windows.'" />
  </then>
 
  <elseif>
    <os family="mac" />
    <then>
      <echo message="This build file is being run on OSX.'" />
    </then>
  </elseif>
 
  <else>
    <echo message="This build file is being run on Linux.'" />
  </else>
</if>

The following if statement sets a property with a default value if the property doesn't exist. This is a good way of allowing properties to be passed to build files but to also have them available if the property isn't present.

<if>
  <not>
    <isset property="docs_path" />
  </not>
  <then>
    <resolvepath propertyName="docs_path" file="/some/static/path" />     
  </then>
</if>

The following is an example taken from a Drupal build file that puts the apc.php file into the correct place for it to be used by the system. It is an example of using an if statement with a then, elseif, and else element.

On some platforms the acp.php file is kept as a compressed 'gz' file that must be extracted. This file exists then the file is copied, to the destination directory before being extracted. The copied compressed file is then deleted as it is not needed. If the apc.php file is present then it is just copied into the correct place. Finally, the build file will announce if it is unable to find the file on the current system.

<if>
  <available file="/usr/share/doc/php-apc/apc.php.gz" />
  <then>
    <echo>Compressed apc file found, extracting.</echo>
    <mkdir dir="/docroot/sites/all/libraries/APC/" />
    <copy file="/usr/share/doc/php-apc/apc.php.gz" tofile="/docroot/sites/all/libraries/APC/apc.php.gz" overwrite="true" />
    <exec command="gunzip apc.php.gz" dir="/docroot/sites/all/libraries/APC/" />
    <copy todir="${make_path}/docroot/sites/all/libraries/APC/">
      <mapper type="glob" from="apc.php" to="apc.php.inc"/>
      <fileset dir="/docroot/sites/all/libraries/APC/">
        <include name="apc.php" />
      </fileset>
    </copy>
    <delete file="/docroot/sites/all/libraries/APC/apc.php" />
  </then>
 
  <elseif>
    <available file="/usr/share/doc/php-apc/apc.php" />
    <then>
      <echo>Uncompressed apc file found, copying</echo>
      <mkdir dir="/docroot/sites/all/libraries/APC/" />
      <copy file="/usr/share/doc/php-apc/apc.php" tofile="/docroot/sites/all/libraries/APC/apc.php.inc" overwrite="true" />
    </then>
  </elseif>
 
  <else>
    <echo>APC not installed on this system, skipping APC setup</echo>
  </else>
</if>

The following is a silly example that shows the syntax used when nesting if tasks but it useful to see how nesting works. This example will print 'True' because each if comparison will be true.

<if>
  <istrue value="true" />
  <then>
    <if>
      <istrue value="true" />
        <then>
          <if>
            <istrue value="true" />
            <then>
              <echo>True</echo>
            </then>
          </if>
        </then>
    </if>
  </then>
</if>

Category:

Tags:

Monitoring Performance With Munin

philipnorton42 — Mon, 20 May 2013 18:42:24 +0000

I have been searching for a good server monitoring solution for a while so that I can keep an eye on some of the servers that I run. Tools like Smokeping, Cacti and Nagios seemed promising at the outset, but they are more concerned with bandwidth and server status, not how the server is running. What I really needed was a way to find out how much memory a server was using, how many Apache requests were being made, what the average load of the server was, and also some way of letting me know when things were under strain.

I did find some tools like New Relic and AppNeta that seemed to fit the bill, which have large feature sets, and nice graphing and reporting solutions. The only major hurdle for me was that they are expensive, especially when I was considering setting this monitoring system up on my home server for my own projects.

After not finding anything I decided to create a system that ran the uptime command on a remote server and recorded the output. uptime is a Linux command that has varied output between systems but will report on how long the server had been running, how many users are logged in and what the load average of the server is. Here is an example output from my local machine (which was doing a lot of code updates and database processing on a development site at the time).

10:42:44 up 1 day, 20:36, 4 users, load average: 0.97, 1.07, 0.90

I used Phing to allow me to run the uptime command on a remote server and capture the result. The ssh task can be used to send any command to a remote system and capture the output in a property. So I used the ssh task to get the output of the uptime command and store it in a property called monitor.uptime.

<ssh username="${monitor.username}" password="${monitor.password}" host="${monitor.host}" command="uptime" property="monitor.uptime" />

With the output of this command in a property I then used some other processing to extract the data needed and add this to a log file, which was updated thanks to a cron task that just ran the Phing file every 2 minutes. With this log file I created a web page that then graphed the output using Flot charts. This took me an hour to put together and all worked nicely, but the major flaw was that in order to add more options I would need to use a separate ssh task, and each task then added another ssh call to the server. This was aside from the massive security issues surrounding automated ssh logins using passwords stored in plain text.

I ran the Phing script for a day, but I was still looking for a better solution to monitoring server performance. After posting about the Phing script (and how mad it was) on Facebook Jeremy Coates posted a quick message about a system called Munin. Thanks Jeremy!

Munin is a system that uses a set of Perl scripts to poll external systems to bring down information about server performance statistics. The data found is then written into a series of graphs that represent the data by day, week, month, and year. By default you get access to things like CPU usage, memory usage, system load, disk access times, processes, uptime, and network traffic. The system is modular so it can be extended to include Apache load, Varnish statistics, MySQL load, and even the number of emails sent.

After using Munin for a couple of weeks I am really impressed by how much data I can view, and it has even allowed me to track down a performance issue on a server. It's pretty easy to install, but I thought that other people might benefit from me writing down some notes on getting up and running with Munin.

Install Munin Server

The first step in getting up an running with Munin is to install a Munin server. This is the primary hub of the Munin reporting tool and where you can view the data from each of your Munin nodes. Setting up Munin on most systems is pretty easy, just use the following apt-get command on Debian/Ubuntu setups.

sudo apt-get install munin munin-node

If you are on Centos/Redhat then you can use the yum package manager in the same way.

sudo yum install munin munin-node

In order to view the Munin graphs via the web interface you might also need to install the libwww-perl package. This is required by Apache to run Perl scripts and is necessary in order to allow the server to show the Munin graphs.

Additionally, you might also want to install the munin-plugins-extra package. This package contains a bunch of extra modules that allow you to capture results for things like Apache, MySQL or Varnish that are on top of the normal CPU and memory usage statistics. You can do this as a separate step or as part of the installation of the Munin server. You might also be wondering why we are installing a Munin node on the server. This isn't essential and it will be used to report on the status of the server itself, but it's a good test to see if things are working correctly and lets you keep an eye on how things are running locally. My Munin server is hosted on a machine that does a bunch of other things so it makes sense to install the node along side this in order to make sure the rest of the services are able to run.

The Munin graphs are shown in a web interface (i.e. through a web browser), but you'll need to properly set things up so that you can view the graphs. After installing Munin server on a few different systems as a test I have found that some systems (e.g. Ubuntu) will set everything up, whilst others tend to leave the finer detail to the administrators. On Ubuntu you can edit the file /etc/apache2/conf.d/munin in order to configure the location of the Apache webroot. By default this will only allow access from the localhost IP address and so you might want to add a couple of Allow rules. To allow everyone to access the Munin web interface use the following.

Allow from 0.0.0.0

To allow everyone from your local network use something like the following.

192.168.1.0/24

Be careful though, if you use port forwarding between your router and the Munin server you will essentially be allowing access from the outside world to your Munin interface. This is because internally the request appears to be coming from the internal network. This data is not the sort of thing you want everyone to see!

The main Munin server configuration file is located at /etc/munin/munin.conf. There are a few configuration options to see here as well as the setup of where the Munin nodes are located. The default Munin configuration options are usually fine, unless you need to alter the directories Munin uses to write out the graphs. For example, if you changed the Apache webroot then you will need to alter the config option htmldir to point to the same place.

Now you have Munin installed you should be able to view the web interface via the address http://localhost/munin. By default you will see the statistics for localhost, as long as you installed the local Munin node. On installation a cron job is also set up that automatically updates the Munin data every 5 minutes and so it will take a little time for the graphs to start filling up.

Setting up a node for Munin is probably your primary reason for editing the munin.conf file. The minimal node configuration you need is the display name of the node (for the web interface) and the physical address of the node.

1 2	[www.hashbangcode.com] address 10.0.0.1

You can also group the hosts together, which helps when viewing the results as they will be grouped in the same way. Grouping has the following format:

[MyHost;]
 
[MyHost;www.hashbangcode.com]
    address 10.0.0.1
 
[MyHost;www.example.com]
    address 10.0.0.2

The address config option is the minimum needed to get a Munin node working.

To control the updating of a Munin node use the update node level directive. This defaults to 'yes', but you can set it to 'no' to turn off the updates but retain the data and show the data on the graph.

1
2
3

[www.hashbangcode.com]
  address 10.0.0.1
  update no

To change the port that Munin uses as a default (4949) use the port node level directive. This is useful if you have to use a different port.

1
2
3

[www.hashbangcode.com]
  address 10.0.0.1
  port 12345

The use_node_name directive determines whether the IP address or the node name (i.e. the name between []) is to be used to contact the node. This defaults to 'no', but can be useful to set to 'yes' when the IP address isn't translated into the correct server address. This might be the case if you are using Munin in an internal network and have a strict firewall policy, but for the most part you can leave this alone.

1
2
3

[www.hashbangcode.com]
  address 10.0.0.1
  use_node_name yes

There are a lot more options than I have shown here. For more information about using the config file take a look at the Munin documentation on config files.

Installing Munin Node

Munin is a useful tool when viewing localhost statistics, but creating nodes is really where the power lies. As I have said previously, Munin works by having a central server polling remote nodes to draw down the data. So when we install Munin as a node we are only installing a small component that reacts to incoming Munin requests. This is actually an important concept with Munin, if your server doesn't ask the nodes for results (e.g. because of a network failure) then the data is not be generated and your Munin graphs will have holes in them. Your Munin nodes will not keep hold of data until the next time it can give them to the server, they don't even generate it.

To install a Munin node on a Debian/Ubuntu server use the following command.

sudo apt-get install munin-node

To install a Munin node on a Centos/Redhat server use the following. If this fails or is not found then you might need to have the EPEL repository installed as a prerequisite.

sudo yum install munin-node

Now that the Munin node is installed you'll want to look at the config file, which can be found at /etc/munin/munin-node.conf. The following is a typical Munin node configuration file.

log_level 4
log_file /var/log/munin-node/munin-node.log
pid_file /var/run/munin/munin-node.pid
 
background 1m
setsid 1
 
user root
group root
 
# Regexps for files to ignore
ignore_file [\#~]$m
ignore_file DEADJOE$
ignore_file \.bak$
ignore_file %$
ignore_file \.dpkg-(tmp|new|old|dist)$
ignore_file \.rpm(save|new)$
ignore_file \.pod$
 
#host_name localhost.localdomain
 
allow ^127\.0\.0\.1$
allow ^::1$
 
# Which address to bind to;
host *
 
# And which port
port 4949

By default this node only allows access to the Munin data from the local machine. Before you can use the Munin node you need to add your IP address as an allow rule in this config file.

Once that is done, it's probably a good idea to make sure that the server the node is installed on allows external connections on the Munin port (4949). For this you will need to check your iptables configuration. To view the current iptables configurations you have use the following command (the sudo is important).

sudo iptables -L -n

If no reference to the Munin server IP address on port 4949 exists then you will need to create one. Here is an example of adding a iptables rule for a Munin server.

sudo iptables -A INPUT -p tcp --dport 4949 -s 10.0.0.1 -j ACCEPT

When you run the configuration lookup command (sudo iptables -L -n) a second time you should see the following.

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:80
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:8080
ACCEPT     tcp  --  10.0.0.1       0.0.0.0/0            tcp dpt:4949

Everything is now set up, but you might notice that your Munin server doesn't connect. The reason is that when you change any of the rules in your Munin node you will need to restart the node for the options to be loaded. To restart a Munin node use the following.

sudo service munin-node restart

Or, if service isn't available then you can use this command instead.

sudo /etc/init.d/munin-node restart

When you install a Munin node for the first time you need to make sure that the service is running if the server is rebooted. To make sure the Munin node starts if the server restarts with the server you can use chkconfig.

sudo chkconfig --add munin-node

If that fails then try the alternative of.

sudo /sbin/chkconfig munin-node on

One final thing to remember is that if your host is running a hardware firewall then you'll need to allow connections to pass through that as well.

Troubleshooting

Getting started with Munin can sometimes be a little bit painful. For the most part everything installs and runs fine. My main problem was when trying to hookup a new node. There tends to be a lot of stuff in-between that can cause the Munin server to fail it's data polling. If you have any problems then remember that Munin logs everything. If you edit your server configuration file to add a new node and nothing appears to be showing in the web interface then have a look at the munin-update.log. It will usually tell you if there is a connection problem or if something wrong happened during the update process, which happens every 5 minutes. To view the last 50 log entries use the following.

sudo tail -n 50 /var/log/munin/munin-update.log

The Munin server uses a cron job set to run every 5 minutes in order to poll the Munin nodes. You need to make sure that the user who runs the cron is munin-cron. If nothing is being written to the logs then it's generally because the cron job isn't running. To view the cron jobs that have been setup for the munin-cron user use the following command.

sudo -u munin-cron crontab -e

What is good about Munin is that it uses a standard interface and so even if you install Munin 2.1 on your server and Munin 1.4 on your node the two will happily talk to each other so that tends not to be a problem.

Understanding The Output

Munin produces a lot of graphs and other data, and I could write an entire book about understanding it all. My first few days with Munin was trying to get to grips exactly what everything meant and if a graph with a certain value was a bad thing or not. Munin reports are spit into sections, which makes pinpointing issues really handy. Here is a breakdown of a few of the graphs in different sections.

Disk

Disk throughput per device - This shows the amount of data being written or read from a disk. A disk will always be slower than memory when retrieving files, but a lot of disk reads can be a problem. For example, it could indicate that your database server doesn't have enough RAM and is therefore storing tables on the disk rather than in memory.

Inode usage in percent - Every file system has an index showing where every file is stored, what it's name is, and what the permissions of the file are. This has a physical limit, and when it gets full it means you can't create any more files, even if there is enough space on the disk. Keeping an eye on this value is therefore very important as it will quickly cripple or bring down your website.

Network

Firewall Throughput - This is the amount of traffic going through the iptables firewall, which is generally all traffic on most servers. This is only important if you are running the Munin node on a router, firewall or gateway system.

eth0 Traffic - The amount of traffic passing through the eth0 interface. Useful for keeping an eye on the available bandwidth on a web host.

Processes

Fork rate - Processes are created by forking existing processed into two new processes. This indicates the rate at which new processes are created and can be an indication of a bottleneck if lots of processes are being created.

Number of threads - This is the total number of processes running on a system.

CPU Usage - This indicates the usage of the CPU and is divided into multiple parts. For example, iowait shows the amount of time spent by processes for IO (e.g. disk access).

System

Available entropy - The entropy is the measure of the random numbers available from /dev/urandom. These random numbers are needed to create SSL connections. If you create a large number of SSL connections this randomness pool could possibly run out of real random numbers.

Inode table usage - This shows the total number of files that are open on a system. If this number increases suddenly then it might mean that there is a program on the system that is not closing files correctly.

Load average - This is a useful number that shows how much is trying to be run by the system. Numbers under 1.0 mean that there is enough resources to cope with the amount of work to be done. Numbers over 1.0 mean that the system is under high load and there are not enough system resources. Understanding Linux CPU Load - when should you be worried? is perhaps the best explanation I have seen for load average.

All of the above screenshots were taken from the Munin setup I have for #! code. Those of you who are familiar with the statistics Munin produces will see that the server isn't really under strain all that much. This is only a small sample of the statistics available from Munin, especially when you start to use plugins were you can view statistics for Apache, Varnish, MySQL, sendmail and a variety of other things. Setting up these extra plugins is usually quite simple, and quite often is done for you anyway.

Notifications

Looking at graphs is fine, but the real power of Munin is that it can tell you when things start going wrong. Built into Munin is a notification system that will highlight problems within the web interface but also contact you (via email as a default) when things reach certain limits. These limits are numbers that you need to set yourself, but doing so isn't too difficult. There are two levels of issues in Munin, warning is for when things might go wrong and critical is for when things really need attention. As these values are set arbitrarily it is up to you to decide what is a warning and what is a critical error.

The first step in setting up Munin to contact you when a problem arrises is to create a contact. To do this you just need to add something like the following to your servers munin.conf file. This tells Munin to always email the 'user' email address when a warning or critical issue has been found.

1
2
3

# 'user' can be any user you want it to be
contact.user.command mail -s "Munin notification" email@example.com
contact.user.always_send warning critical

The next step is to set up some warnings for your nodes. In the following example I am setting the load average warning threshold to be 15 and the critical threshold to be 20. This means that if the system load passes any of these two values Munin will let me know about it.

[www.example.com]
    address 10.0.0.1
    load.load.warning 10
    load.load.critical 20

A good tip here is that when you are trying out these values for the first time set the threshold to an intentionally low value so that they will be triggered. This ensures that the correct values are being watched in Munin and that the alerts will work correctly. Increase these values to sensible levels when you are happy that things will work when they are needed.

Category:

Tags:

Source Controlled Git Hooks With Phing

philipnorton42 — Fri, 10 May 2013 23:15:51 +0000

The other day I was experimenting with Git hooks. These are scripts that you can execute before certain actions are run in Git. For example, you might want to ensure that forced updates are not run, ensuring respository files have the correct permissions after merging, or that the files have ASCII standard names before being committed.

To use a hook in Git you just need to add them to the .git/hooks directory in your respository and to change the mode of the file so that it is executable. A new Git repository will create several sample hook files that can be used by removing the '.sample' from the end and making them executable. For more information on Git hooks and how to use them see the Git hooks manual page in the Git documentation.

My primary reason for creating a Git hook was to ensure that there were no syntax errors on my code before it was even committed. I had previously been either checking manually or using Jenkins to download the repository and check the code for errors, but then I realised that it would be far easier to stop them at the source.

To this end I created a hook file called pre-commit and added this to the hooks directory. As you can probably guess this hook is run before the commit action is executed in Git. I have talked previously about using Phing to syntax check all PHP and JavaScript files in a repository so I just used the same Phing target here. The only addition was that I also needed the commit action to stop if the Phing syntax check failed, which could be done in the hook file itself. There would be enough information about what failed in the Phing output to show why the commit had failed so I didn't need to print anything else out here. This is the hook file in full.

#!/bin/sh
 
echo "Parsing PHP and JavaScript files for lint errors."
 
phing -f scripts/build.xml syntaxcheck || exit 1

This worked as expected and I was unable to commit and files that contained syntax errors to the repository. The only problem was that if I checked the repository out again in a different location I would lose this hook as the files in the .git directory (and therefore .git/hooks) are not source controlled. This also meant that I couldn't get other people working on the project to follow the same practice as they would be missing the hook on their repositories as well. After a bit of searching for a solution to this I found the best course of action was to include the hook file in the respository and then write a script to automate the inclusion of the hook into the .git/hooks directory.

I created a directory in my respository called githooks and moved the hook I had created into it so that it could be source controlled. Rather than create a script from scratch I decided to add a githook target to the already existing Phing build file in my repository. This target has the following actions:

Define a couple of properties that point to the .git/hooks directory and the githooks directory in my repository.
Delete the existing .git/hooks direcory.
Symlink the githooks directory to .git/hooks, thereby replacing the original hooks directory.
Create a fileset that contains all of the possible hooks files in the githooks directory.
Loop through the files in the fileset and ensure that they are all set to be executable. This is tricky and requires the use of a secondary target called chmodsetexecute that has the single action of setting the filename passed to it to be executable.

Here are the two Phing targets used to set up the repository with custom Git hooks.

 <target name="githook">
   <resolvepath propertyName="git_hooks_path" file="${project.basedir}/../.git/hooks"/>
   <resolvepath propertyName="repo_hooks_path" file="${project.basedir}/../githooks"/>
 
   <!-- Delete existing .git/hooks directory -->
   <delete dir="${git_hooks_path}" quiet="true" failonerror="false" />
   
   <!-- link the git hooks directories together -->
   <symlink target="${repo_hooks_path}" link="${git_hooks_path}" />
   
   <fileset dir="${repo_hooks_path}" id="hookfiles">
     <include name="applypatch-msg" />
     <include name="post-commit" />
     <include name="post-update" />
     <include name="pre-commit" />
     <include name="update" />
     <include name="commit-msg" />
     <include name="post-receive" />
     <include name="pre-applypatch" />
     <include name="pre-rebase" />
   </fileset>
   
   <foreach param="hookfilename" target="chmodsetexecute">
     <fileset refid="hookfiles" />
   </foreach>
 </target>
 
 <target name="chmodsetexecute">
   <exec command="chmod +x ${repo_hooks_path}/${hookfilename}" />
 </target>

With all this in place it is just a case of making sure that this target is ran every time a new checkout is done and that existing repositories are updated with this change. This sets up the correct Git hooks in place and makes sure that no one can commit code that has syntax errors.

Category:

Phing

Tags:

git

Phing

Detecting The Sudo User In Phing

philipnorton42 — Wed, 10 Apr 2013 19:50:18 +0000

I use Phing for a lot of different tasks, it helps me to automate things that I would otherwise mess up if left to my own devices. Prime candidates for Phing scripts are things that I don't do that much and forget how to do them, or that have a number of complex steps. The only problem I have found is that because many of the Phing scripts I create rely on system changes (eg, configuring an Apache server) they therefore require system changing privileges. Normally I would just prefix the Phing command with sudo, but every now and then I forget all about that step and the build fails. This can be dangerous as I am then left with a build that failed, which might leave a system partly configured or even take a server offline.

This got me to thinking about what Phing could do to remind me if I forget to use sudo on my sudo requiring scripts. There must a better way of reminding myself that I should be running the script as sudo rather than just having the build fail on me. After looking through the Phing documentation and a bit of searching online I found nothing that does this.

The @phingofficial channel on Twitter (who are awesome by the way) suggested that I could use the output of 'whoami' or 'id -u' to test if a user was running as sudo. This is quite easy to do using the <exec> task, which I can use to set a property. Here are a couple of examples.

The following sets the property 'currentuser' to be the output of the 'whoami' command. If the script is being run as sudo then this will be 'root' otherwise it will be the username of the current user.

<exec command="whoami" outputProperty="currentuser" />

The following sets the property 'currentuser' to be the output of the 'id -u' command. If the script is being run as sudo then this will be '0', otherwise it will be the ID of the current user (1000 on my localhost).

<exec command="id -u" outputProperty="currentuser" />

If you are on Windows then you might be able to use the following command, which will produce an access denied message if the user is not an administrator. If you have any other suggestions on how to detect the administrator user on Windows via the command line (or even through PHP) then I'd be happy to hear from you.

<exec command="net user "%username%"" outputProperty="currentuser" />

With that in place I can use the property to create a boolean value that I can use later in the script using the <condition> task. Assuming that I use the 'whoami' command from the above examples I can do the following.

<!-- Create the currentuser property -->
<exec command="whoami" outputProperty="currentuser" />
 
<!-- Set the 'issudo' property to be true if the user is root -->
<condition property="issudo">
  <equals arg1="${currentuser}" arg2="root" />
</condition>
 
<!-- Fail the build if the user is not root -->
<if>
  <not>
    <equals arg1="${issudo}" arg2="true" />
  </not>
  <then>
    <fail message="You must run this build as sudo!" />
  </then>
</if>

Alternatively, I could just a single <if> task to do the same sort of check.

<!-- Fail the build if the user is not root -->
<if>
  <not>
    <equals arg1="${currentuser}" arg2="root" />
  </not>
  <then>
    <fail message="You must run this build as sudo!" />
  </then>
</if>

The <fail> task here will just stop all execution of the script and show that the build has failed to the user, along with the message text.

I could have left if there, but I wanted to take it further by creating a custom Phing task that will then create a property called 'issudo' that will tell me if the user has used sudo to run the build script. This involves creating a PHP class that checks the sudo status using PHP, which should therefore work on any platform. To integrate with Phing you just need to create a class that extends Task and implements a method called main(). The main() method is executed when the object is used so it is within this method that the work needs to be done to check the user and create the property.

To find out if a user has run a PHP script using sudo is quite easy on Linux hosts. The $_SERVER array will contain a few elements that are to do with who the origin user was and what command was run when used on the command line. If we look for the presence of SUDO_COMMAND within the $_SERVER array then we can be pretty sure that the user is running the script as a sudo user. Again, if you have more information on how this works on Windows I'd be happy to hear from you.

To set a property that can be used in the Phing build file the $this->project->setProperty() method needs to be called. This will create a property with a given name and takes two parameters.

$name : The name of property to set.
$value : The value of the property. If the property name already exists then this will overrite the previous value.

The following class was placed into the same directory as my build script with the name of IsSudo.php. The call to log() just prints out a message to the terminal so that we can see that the class has been run.

/**
 * Sets a phing property called 'issudo' that can be used to see if the user has root privilages.
 *
 */
class IsSudo extends Task {
 
  /**
   * Set the property.
   */
  public function main() {
    $issudo = false;
    
    $this->log('Determining sudo.');
 
    if (isset($_SERVER['SUDO_COMMAND'])) {
      $issudo = true; 
    }
    
    $this->project->setProperty('issudo', $issudo);
  }
}

The final step here is to tell Phing that we have a class that we want to be used. This is done using the <taskdef> task. In it's simplest terms this task takes two parameters of the name of the task we want to create and the name of the class that contains the code behind the task. The following code creates a task called <issudo>, which can be found in the class IsSudo.

<taskdef name="issudo" classname="IsSudo" />

This allows us to use the custom <issudo> task that creates a property called ${issudo}, which can then be used to see if the user has run the script as sudo.

<!-- Create issudo property -->
<issudo />
 
<!-- Check issudo -->
<if>
  <not>
    <istrue value="${issudo}" />
  </not>
  <then>
    <fail message="You must run this build as sudo!" />
  </then>
</if>

This works as intended and will stop the execution of the build file if I've forgotten to run it via sudo.

I did think that it might be possible to integrate with the comparison operators within Phing and create an element that works in much the same way as the <istrue> or <isset> tasks. After some investigation I don't think there is a way of doing this without altering the Phing source code.

Update : @Beryllium9 on twitter suggested that a good way of forcing a sudo check is to create a single sudo target and then require that as a prerequisite to running any sudo requiring targets. The way to do this is to use the depends property of the target element. Adding a depends attribute to the target element will make Phing run the named target first before running the current target. This is the case even if the default target is being called.

As an example the following build file will only run the main target if it has been run using sudo.

<?xml version="1.0"?>
<project name="sudotest" default="main">
 
  <target name="sudotest">
    <!-- Create the currentuser property -->
    <exec command="whoami" outputProperty="currentuser" />
 
    <!-- Set the 'issudo' property to be true if the user is root -->
    <condition property="issudo">
      <equals arg1="${currentuser}" arg2="root" />
    </condition>
 
    <!-- Fail the build if the user is not root -->
    <if>
      <not>
        <equals arg1="${issudo}" arg2="true" />
      </not>
      <then>
        <fail message="You must run this build as sudo!" />
      </then>
    </if>
  </target>
 
  <target name="main" depends="sudotest">
    <echo>Running main target</echo>
  </target>
</project>

Category:

Tags:

DrupalCampNW 2012 Videos

philipnorton42 — Sun, 31 Mar 2013 12:42:02 +0000

During DrupalCampNW 2012 last November the company we hired to record the sessions also spent some time interviewing the people who attended. From these interviews we were able to create three videos, two from the main event and one about what Drupal means to different people. These videos are now complete so I'm posting them here so everyone can see. I think they came out really well.

I would also like to thank Drupal Association, who very generously financed the creation and production of these videos.

DrupalCamp North West Business Day

DrupalCamp North West Business Day from DrupalCamp NW on Vimeo.

DrupalCamp North West 2012

DrupalCamp North West 2012 from DrupalCamp NW on Vimeo.

What Does Drupal Mean To Me?

What does Drupal mean to me? from DrupalCamp NW on Vimeo.

To me, Drupal means more than just a CMS or a framework. The core of Drupal is more like a starting block to a whole world of modules, themes, technologies, practices, services and people that make Drupal more than the sum of its parts. The community of people behind Drupal are amazing and some incredible work has been done to make Drupal the enterprise class software it is today. The future of Drupal looks good, especially with the amazing work going into the next version of Drupal.

When I first started using Drupal I had just started a new job and was dropped into the deep end of a half finished Drupal 6 project that needed to be completed. After working through the project for a couple of weeks I found out about the local user Drupal group in Manchester (NWDUG). After just a few minutes of talking through the issues I was having I went away with some ideas about how to solve them and was able to complete the project. I now help to run NWDUG and try to do the same thing for other people either by directly helping them or by pointing them to the correct resource. If you have a local user group then I urge you to get involved, you will learn so much just by being there that it will make it worth the trip. If you don't have a local user group then maybe you can start one? All you need is a venue and a couple of other people and you can at least get together once a month and talk about your Drupal problems.

What does Drupal mean to you? Post a comment below and tell us.

Category:

Drupal

Tags:

DrupalCampNW

Drupal 7: Setting Default Value For A Field

philipnorton42 — Sat, 30 Mar 2013 11:48:31 +0000

I was recently working on a module that contained a content type as a feature. When the module was enabled the hook_install() hook set up a collection of taxonomy terms which were used within the content type as a field. Whilst testing this out I realised that although the terms were installed correctly the default value of the field changed depending on which system the module was installed on. The reason for this was that the term ID was being used to pull out the default term from the database, which is the normal behaviour in Drupal. The problem here was that if a term the ID of the term was different (because one had been added) then this had a knock-on effect of changing the default value of the field.

To get around this I needed a way of setting the default value of the field based on a given taxonomy term. After some research and reading of the Drupal source code I found a way to programatically set the default value of a taxonomy field.

Before getting into that there is a little bit of setup to do beforehand. As I said before, the module I was working on contained a feature that held a content type and a few fields, one of which was a taxonomy reference. In order to set a default value on a taxonomy field on a content type I needed to ensure that all of the components are correctly assembled first. In the hook_install() hook of the module I added a call to features_revert() so that I could be sure that the rest of the actions I was about to complete would work correctly. The features_revert() function works like clicking the 'Revert Features' button in the features module administration section and will force the given arguments to be reverted. In order for this to work during the install hook the correct feature files also need to be included and so the start of the hook includes the features files involved with nodes, fields and taxonomies.

function my_module_install() {
  // Ensure the correct features files are included
  module_load_include('inc', 'my_module', 'my_module.features');
  module_load_include('inc', 'my_module', 'my_module.features.field');
  module_load_include('inc', 'my_module', 'my_module.features.taxonomy');
 
  // Ensure that the content type, fields and taxonomy vocabularies are actually there.
  features_revert(
    array(
      'my_module' => array(
        'taxonomy',
        'node',
        'field'
      )
    )
  );
 
}

As a side note it may have been better to run this code in a different hook (e.g., hook_enable() or hook_modules_enabled()) so that the features_revert() function isn't needed. I have found, however, that you can't rely on features being fully implemented in any of the installing and enabling hooks. Forcing just the components you need to use in a feature is a good way of ensuring that everything you need is ready to go.

The next step was to load the vocabulary and add the default terms to it. The following code adds six terms to the vocabulary called 'task_status', which is also included in the install hook.

// Set up 'Status' vocabulary terms.
$task_status_vocab = taxonomy_vocabulary_machine_name_load('task_status');
$statuses = array(
  'New',
  'Being Worked On',
  'Postponed',
  'Failed QA/Testing',
  'Cancelled',
  'Ready For QA/Testing'
);
foreach ($statuses as $status) {
  $term = new stdClass();
  $term->name = $status;
  $term->vid = $task_status_vocab->vid;
  taxonomy_term_save($term);
}

Now that everything is set up the default field value can be set. What we need to do first is to run a database query to pick out the correct term so that we can set the taxonomy field to the correct value.

// Get the tid for the default status
$term = db_select('taxonomy_term_data', 'td')
  ->fields('td', array('tid', 'name'))
  ->condition('name', 'New')
  ->condition('vid', $task_status_vocab->vid)
  ->execute()
  ->fetchObject();

Now we need to take the term ID we just found and set it to be the default for the field, which involves three steps. The first step is to get an instance of the field using the function field_read_instance() that we can modify. The second step is to then modify the field with the new default value of the taxonomy term ID (referred to as tid). The final step is to update the field instance with the new field information using the field_update_instance() function.

// Set the default taxonomy term for the taxonomy field.
 
// Get an instance of the field that we can modify
$instance = field_read_instance('node', 'field_task_status', 'task');
 
// Update the default value of the field
$instance['default_value'] = array(array('tid' => $term->tid));
 
// Save the field back to the database.
field_update_instance($instance);

The only problem here is that once the default value has been updated the Feature will report itself as being overridden (which it is). This is unavoidable, but it's not something that will cause many problems. The one main problem is that if you ever need to revert the fields components of the feature again in the future then you'll need a way of updating the field with the correct default value. Even without Features this is a good trick to know for any fields.

Category:

Tags:

Drupal Camp London 2013

PaulWoodhead — Tue, 26 Mar 2013 22:04:13 +0000

I'm just letting all my neurons cool down after a fantastic Drupal Camp at City University, London. These Drupal camps are getting bigger, better, and much more frequent in a really, feely, organic way. They never feel like there is a sponsor driven sales push to use this service or that, moreover, it genuinely does feel like a community of like minded people, groups of people, willingly sharing their knowledge, ideas, and enthusiasm in what it is they are doing, using, or believe in.

The keynote was by David Axmark, co-founder of MySQL AB, and one of the real players in the shaping of how the open source World evolved. It was a very engaging talk covering the inception of MySQL, to it's $1 Billion sale to Sun Micro-systems and subsequent sale to Oracle. He is a man who is obviously still passionate about licence issues, the tyranny of software patents, and the rise of nosql databases. He is still very active with MariaDB and put forward some good arguments using Maria against NoSQL. There is also a Drupal Camp London 2013 Interview available on YouTube.

For me, every camp has a different theme, flavour, running through it. I'm not sure if that's because of the tracks I choose, or, whether that is the way the tonal centre of the camp evolves. The fantastic Manchester Camp, at Media City in November, had a themer theme running through it. Whereas the London Camp, had more of an emphasis on developers, development, development environments. My first session was Drush Make Driven Development with Steven Jones. As someone reasonably new to Drush and Git (About 12 Months) I was a little concerned that I was going to be left cross eyed and bewildered, but Steven's skills as not just a developer, but a speaker, kept me totally engaged and with it, knowing full well that he would be uploading the slides to refresh my informationally bombarded brain.

Over the entire weekend I had a bit of a Vagrant fest. My next session was what sparked that off. I've dabbled with Vagrant before, but not knowing to what ends. My Vagrant Love: A Developer’s Kiss-and-Tell by Joe Baker gave me a lot more to go at, and a better understanding of what I could do with it when I got there. I've used "vagrant up" numerous times since this talk.

Wake up and smell the CoffeeScript with Rob Knight was also an engaging talk. He very eloquently explained and live demo'd using coffee script. As someone who's a regular semi-colon excluder, and lover of DRY, I can see the benefits. It reminds me of SASS for Javascript.

After lunch it was Manage your media assets with the Scald Module by Sylvain Moreau and Olivier Friesse. I'm still an IMCE kinda guy myself, but this module does have lots of bells and whistles. Which is probably why I'm still an IMCE kinda guy ;) It was a good presentation though, with lots of live demos (very brave).

More vagrant! Make your cheap VM fly with Greg Harvey was very exciting. I feel quite antiquated using LAMP stack now. This was all about serving drupal on nginx, with varnish, percona, and all the trimmings. It was a really good talk for addressing some of the criticisms Drupal hosting sometimes unfairly receives.

Next up was Drupal Commerce and SagePay - the reboot (with added doughnuts) by Richard Jones. It was really good and showed how Commerce is now more than a one trick pony with payment gateways. Commerce is awesome, but the lack of payment gateways has been holding it's World domination back.

Yea, more vagrant, but I'd not had enough. Vagrant: A crash course by Marcus Deglos was entertaining and educating. This talk gave me a lot more practical uses for rapidly setting up development environments, and what to do with them when I've got them. I was wondering on the bus on the way home whether you could use Jenkins to move those config's about? But then came the social sponsored by Agile Collective. It's like a dream night having a pub full of Drupalistas, and talking Drupal progressivly louder and more slurred as the evening went on, and good laughing at some people who developed jelly legs.

The Sunday was a little more subdued to with a few self pitying groans from here and there. The keynote was Step 4: Distributions could help, if we do them right by Robert Douglass He started off by talking about what's needed in Drupal in terms of distributions and modules. However, he was actually reading from a 2006 blog. Highlighting that not enough has changed and more can be done to compete against Wordpress's theme forrest, and Google Apps. There is a lack of Drupal Apps in the App server. It did have me thinking about the UI in all what I develop.

Next was Drupal 8 Configuration system for Coders by Alex Pott which offered an insight into settings, state and config in Drupal 8. It was a good talk, and heads up for developers to look out for en route to Drupal 8. I was feeling a bit jaded, but Alex kept me engaged.

Death of a Themer by James Panton was a really interesting presentation which highlighted a cool way of tidying up your markup. This was also a great tool provider. In essence it was about stripping bare the tpl.php and using Display Suite, Panels and Panels Everywhere to render the page. I've not put it into practice yet. I can see some pitfalls, like slideshows etc, but we are definitely looking at it as a way to dramatically reduce the <div> count.

My final talk was by Steve Cowie from Code Enigma explaining Distributed working - how to work from anywhere It was an honest insight into distributed working which highlighted some factors I'd never thought about: like different country employment laws, and how to use Agile and Scrum with a distributed team. Definitely looking at our organisation becoming distributed using a lot of the structures Steve highlighted.

All in all, it was a fantastic weekend, and I only wish I could have been in loads of places at once. I wanted to do the BOFF sessions and I definitely would love to have joined in with some of the sprints, but having too much choice is a good thing right? I can't wait for the next Drupal camp. Drupal rocks my disco, from our local user groups to the Drupal camps, and this year I'm hoping to to go to the European Drupal conference in Prague.

Thanks go to Pedro Lozano for allowing me to use some of his amazing pictures from the event in this post.

Category:

Drupal

Tags:

DrupalCamp

Finding The First And Last Items In An Array In PHP

philipnorton42 — Sun, 24 Mar 2013 12:41:37 +0000

Getting the first or last item from an array in PHP is usually quite easy. If you create an array and then add a bunch of values to it then the array index will increment by 1 for every element you add. This means that in order to get the first element from an array you just reference the default stating position (0). To get the last item in the array the index key will be the length of the array, minus 1. Here is some example code showing this.

<?php
$array = array();
$array[] = 1;
$array[] = 2;
 
// get the first item in the array
print $array[0]; // prints 1
 
// get the last item in the array
print $array[count($array) - 1]; // prints 2

Things become slightly more complicated when the array has non standard key values. Take the following array for example in which the array count it started at 1.

1 2	<?php $array = array(1 => 'one', 2 => 'two', 3 => 'three');

It isn't possible to reference the first item of the array as 0 here because that array key doesn't exist (we would get an 'undefined offset' error in this case). Also, if we try to access the last item in the array in the same way as before then we will return the value 'two' as the length of the array (3) minus 1 will point to the key in the middle of this array.

PHP comes with a couple of functions that can help when pulling out items in an array. One way of doing this is to use the array_values() function, which will return an array with all of the key values reset. This means that the first item in the array will have a key of 0 and the last item will have a key of the length of the array, minus 1. Therefore, you can just use the original method to find the first and last items.

<?php
$array = array(1 => 'one', 2 => 'two', 3 => 'three');
 
$array_values = array_values($array);
 
// get the first item in the array
print $array_values[0]; // prints 'one'
 
// get the last item in the array
print $array_values[count($array_values) - 1]; // prints 'three'

As the array_values() function creates returns a copy of the array we can use the a couple of functions to snip off the beginning and end of the array without altering the original array. The array_shift() function takes an array as it's only parameter and will cut off and return the first item found in that array. The array_pop() function takes an array as it's only parameter and will cut off and return the last item found in the array. This provides us with an easy mechanism to complete our original task without these functions altering the original array.

<?php
$array = array(1 => 'one', 2 => 'two', 3 => 'three');
 
$array_values = array_values($array);
 
// get the first item in the array
print array_shift($array_values); // prints 'one'
 
// get the last item in the array
print array_pop($array_values); // prints 'three'

Another way that is similar to this would to use the array_keys() function to return the first and last keys of the array using array_shift() and array_pop(), which would then be referenced directly within the original array.

<?php
$array = array(1 => 'one', 2 => 'two', 3 => 'three');
 
$array_keys = array_keys($array);
 
// get the first item in the array
print $array[array_shift($array_keys)]; // prints 'one'
 
// get the last item in the array
print $array[array_pop($array_keys)]; // prints 'three'

There are clearly a few ways of doing this simple task in PHP. However, the best way to extract the first and last items from an array is by using the internal PHP array pointer functions. Every array has an internal pointer that points to the 'current' element. When the array is created this pointer is initialised to be the first element in the array. There are a few PHP functions that deal with moving this pointer around an array, but for the purposes of finding the first and last element we only need to look at current(), reset(), and end().

The end() function will move the pointer to the last element in the array and return the value of it. The reset() function will move the pointer to the start of the array and return the first element. The current() function will return the current item in the array, which should be the first element if the array is newly initialised. Using the current() function can be unpredictable unless you know exactly where the array pointer is, in which case the reset() function might be a better choice.

<?php
$array = array(1 => 'one', 2 => 'two', 3 => 'three');
 
// get the first item in the array (probably)
print current($array); // prints 'one'
 
// get the first item in the array (definitely)
print reset($array); // prints 'one'
 
// get the last item in the array
print end($array); // prints 'three'

Using the internal PHP array pointer like this is many times more performant than using a combination of array_values(), array_pop() or count(). This is an important consideration, especially when dealing with multiple arrays in large applications.

Category:

Tags: