You know that really popular design patterns book that most software developers (especially in the Java world) claim to have read? You know the one I'm talking about. The one written by the "Gang of Four". The one you were supposed to have read during your junior or senior year of college. Yeah, this one.

I've never read it.

Sure, I've owned it. I have even thumbed through it a few times. But I've never sat down with any real intention of absorbing the information within its covers. The problem for me personally was that when I first opened Design Patterns I was just starting my software development career. I was too young and hadn't seen enough code to really comprehend the authors' message or understand the need for such a book.

A few years have passed since then and while it's still early in my career I now have a much better idea of how to develop software and have a much better understanding of Object Oriented Design. Even though I spend a significant amount of my free time developing my skills and reading about software development I have yet to read the seminal Design Patterns book.

Design patterns have left a bad taste in my mouth. Over the years they have been abused and misunderstood. They seem to have become caught up in the all too familiar silver bullet buzzword vernacular of the IT industry (SOA anyone?). So I simply avoided them. Unless I was about to interview for a new job, design patterns never so much as crossed my mind. Repeat after me: Singleton, Factory, MVC. You're hired!

But things have changed. Over the past year and a half or so I've really started to focus on my career as a software developer. I've learned new operating systems and languages, created this blog, and started attending local developer meetings on a regular basis. While I still have my favorite technologies and interests, I've been doing a much better job of keeping an open mind about trying out new development tools and methodologies.

Enter Russ Olsen. In addition to being a great developer and all around good guy, Russ is someone who I had the honor of working with at the beginning of my career. He was very much a mentor and I credit him with teaching me many of the applied fundamentals of software development. So what does Russ have to do with design patterns? Well it just so happens that a few years after I left the company where Russ and I worked together he wrote a book called Design Patterns in Ruby. Because of my respect for Russ and my fondness of Ruby I decided I'd give the book a shot. While I was certainly expecting to learn some new things, I was surprised (although I shouldn't have been) by how much great information I was able to extract from this book.

In their most minimal form, design patterns are simply reusable solutions that have been created to solve common coding problems. While this may seem intuitive, it was actually somewhat of a revelation to me – a person who has ignored the direct usage of design patterns for all these years. I see now that given an appropriate situation design patterns can be used to create consistent, maintainable code. The important thing to remember is that each pattern has a specific use and should only be used when applicable.

Although the title of the book is Design Patterns in Ruby, a majority of the principles and patterns apply to many other object oriented languages as well. In fact, while I was reading Russ's book I was in the middle of working on a small Java project. This project had to keep track of rapid state changes on a particular website. I needed to keep track of a countdown timer, a list of participants, and an aggregate score. Each of these pieces of information was updated independently, but I needed to make decisions in my code based on the combination of this information at any given time. After reading the chapter on the Observer pattern, I realized it would be the perfect fit. Prior to using the pattern my code was a huge pile of tangled spaghetti. Afterwards, it was clean and easy to read.

I really liked the way the chapters were organized in Design Patterns in Ruby. Russ started each chapter by posing a common programming problem. Then he would explain how a specific design pattern could be used to solve that problem. As the chapter progressed Russ would work through the problem using examples that were very easy to understand. Each chapter ended with real world examples of the patterns as they have been implemented in the Ruby core libraries or various open source projects. I found the real world code samples to be especially useful as compliments to the examples given in the book.

Not only did I enjoy reading this book because it was written by Russ, but I genuinely feel like it has helped to make me a better programmer. In addition, it has helped increase my Ruby specific programming skills. Learning to program in Ruby is easy, but learning to program the "Ruby Way" is a much greater challenge. Ruby is full of powerful idioms that are not necessarily obvious without a deep examination of the language. Russ's book helps enlighten would be Ruby programmers to many of Ruby's powerful language features.

For example, compare an implementation of the Proxy pattern written in Java and Ruby:

Java Version

MathService.java

public interface MathService {
    public void add(int x, int y);
    public void subtract(int x, int y);
}

MathServiceImpl.java

public class MathServiceImpl implements MathService {
    public void add(int x, int y) {
        System.out.println("Result ("+x+ " + "+y+"): " + (x + y));
    }
	
    public void subtract(int x, int y) {
        System.out.println("Result ("+x+ " - "+y+"): " + (x - y));
    }	
}

MathServiceProxy.java

public class MathServiceProxy implements MathService {
    private MathService delegate;
	
    public MathServiceProxy(MathService ms){
        this.delegate = ms; 
    }

    public void add(int x, int y) {
        System.out.println("Call made to the add method");
        delegate.add(x, y);
    }
	
    public void subtract(int x, int y) {
        System.out.println("Call made to the subtract method");
        delegate.subtract(x, y);
    }	
}

MathServiceDriver.java

public class MathServiceDriver {
    public static void main(String[] args) {
        MathService ms = new MathServiceProxy(new MathServiceImpl());		
        ms.add(1, 2);
        ms.subtract(4, 3);
    }
}

Ruby Version

math_service.rb

class MathService
  def add(x, y)
    puts "Result (#{x} + #{y}): #{x + y}"
  end

  def subtract(x, y)
    puts "Result (#{x} - #{y}): #{x - y}"
  end
end

class MathServiceProxy
  def initialize(math_service)
    @delegate = math_service
  end

  def method_missing(name, *args)
    puts "Call made to the #{name} method"
    @delegate.send(name, *args)
  end
end

ms = MathServiceProxy.new(MathService.new)
ms.add(1,2)
ms.subtract(4,3)

Result of Running Either Program

Call made to the add method
Result (1 + 2): 3
Call made to the subtract method
Result (4 - 3): 1

In this example we use the Proxy pattern to print out a message before each method call. If we decided to add any methods to our MathService interface we would need to update our MathServiceImpl and MathServiceProxy Java classes as well. However, by using Ruby's built in method_missing method we have the ability to wrap each MathService method call dynamically without having to write a new method in our MathServiceProxy class. Ruby removes the tedium of implementing the Proxy pattern.

This is just one small example of the flexibility and power provided by the Ruby language. Before reading Russ's book I didn't necessarily "get" many of Ruby's idioms. Reading Design Patterns in Ruby has certainly helped solidify my understanding and is great for programmers who would like to take their Ruby skills to the next level.

Code Samples

All code samples from this blog entry can be downloaded from GitHub by clicking here

Since most web pages are simply made of HTML it is easy for a computer to parse and store the information contained within these documents. Each programming language commonly has a host of libraries to assist in the screen scraping/parsing process and Ruby is no exception. To create simple screen scrapers in Ruby I have been using a library called scRUBYt!. scRUBYt! provides methods to access a given website and scrape its content. All the programmer needs to do is provide the XPath string to the desired information.

Using the scRUBYt! library has allowed me to write a small screen scraper script to access the FranklyMLS.com website, check for new listings, and then report back with the results. This has saved me a lot of time and effort. Let's dive into some code to see how this is done.

First, we'll need to create a simple class to store the information that we scrape from the FranklyMLS.com website. The Property class will hold various property related information (price, MLS number, square footage, etc.):

class Property
  attr_accessor :mls, :list_price, :dom, :address, 
                :city, :zip, :bed, :bath, :sqft, :built

  def initialize(property)
    @mls = (property/:mls).inner_html
    @list_price = (property/:list_price).inner_html
    @dom = (property/:dom).inner_html
    @address = (property/:address).inner_html
    @city = (property/:city).inner_html
    @zip = (property/:zip).inner_html
    @bed = (property/:bed).inner_html
    @bath = (property/:bath).inner_html
    @sqft = (property/:sqft).inner_html
    @built = (property/:built).inner_html
  end
end

Next we'll need to make sure that scRUBYt! is installed. If you don't already have Github set up as one of your gem repositories do so now by executing the following command:

gem sources -a http://gems.github.com

Then install the scRUBYt! gem:

gem install jspradlin-scrubyt

Side note: I've built some functionality into the scRUBYt! library so you will need to grab the gem from my Github repository (i.e. jspradlin-scrubyt). I've spoken with the lead developer on the scRUBYt! project and it looks like my changes might make it into a future version of the official gem.

At this point we need to give scRUBYt! the URL of a website that we wish to scrape. The FranklyMLS.com website has its own special URL query syntax which displays only properties that meet our specific criteria. For example, if we only wanted to find active listings in the following zip codes - 22201, 22202, 22203 - the FranklyMLS.com URL would be:

http://franklymls.com/default.aspx?m=R&s=(22201,22202,22203)+active

We can dynamically generate a URL with our specific housing criteria by including the following code in our script:

# Generate the URL for FranklyMLS.com given 
# the following criteria:
zips = [22201,22202,22203,22209]
beds = [2, 3].collect{ |bed| bed.to_s+'bdr'}
min_price = 150 #in thousands
max_price = 350 #in thousands
exclusions = ['JEFFERSON'].collect{ |exclude| "+-#{exclude}"}

fmls_url = "http://franklymls.com/default.aspx?"
fmls_url += "m=R&l=#{min_price}K&h=#{max_price}K"
fmls_url += "&s=(#{zips.join(',')})+active"
fmls_url += "+(#{beds.join(',')})"
fmls_url += exclusions.to_s

Now we're ready to scrape some housing data. Once the FranklyMLS.com property page loads we are presented with a table that contains information about the listings that meet our criteria (image modified to save space):

The HTML that generates this table would appear like this (modified to save space):

<table id="dgRealtorStyle">
  ...
  <tr style="display:visible">
    <td><a>...</a><a>AR6552162</a></td><!-- td[1]-->
    <td>$256,000</td><!-- td[2]-->
    <td>$339,000</td>
    <td>$</td>
    <td> </td>
    <td> </td>
    <td>562</td>
    <td>1931 CLEVELAND #313</td><!-- td[8]-->
    ...
    <td>ARLINGTON</td>
    <td>22201</td>
    <td> </td>
    <td>CLEVLAND HO</td>
    <td>3/1</td>
    <td>860</td>
    <td>1960</td>
    <td>...</td>
    <td>1</td>
    <td>160</td>
    <td><a>x</a></td> 
  </tr>
  ...
</table>

Finally, the Ruby code to scrape this data using scRUBYt!:

#Scrape the FranklyMLS.com website using scRUBYt!
property_data = Scrubyt::Extractor.define do
  fetch fmls_url

  properties '//table[@id="dgRealtorStyle"]' do
    property "//tr" do
      mls "/td[1]//a[2]"
      list_price "/td[2]"
      dom "/td[7]"
      address "/td[8]"
      city "/td[9]"
      zip "/td[10]"
      bed "/td[13]", 
        :format_output => lambda {|bed_bath| bed_bath.split('/')[0]}
      bath "/td[13]", 
        :format_output => lambda {|bed_bath| bed_bath.split('/')[1]}
      sqft "/td[14]"
      built "/td[15]"			
    end
  end  
end

If you look at the table, the HTML code, and the Ruby code you'll see that I've color coordinated each separate piece of information to illustrate how it is parsed and then stored. The scRUBYt! library will "fetch" the given URL, locate the HTML elements by the given XPath, and then store the data.

Once we have all of the data collected we may want to do something useful with the information such as convert it into an RSS feed. We can accomplish this by using the Hpricot and Builder libraries (which should be installed as dependencies of scRUBYt!). The code for the RSS conversion would look like this:

# Read in the XML generated by scRUBYt! then 
# convert the data into Property objects
# and store them in the property_hash.
property_hash = {}

hp = Hpricot.XML(property_data.to_xml)
(hp/:property).each do |property|
  property_hash[(property/:mls).inner_html] = Property.new(property)
end

# Using the Builder library, iterate through 
# the property_hash to generate an RSS feed.
xml = Builder::XmlMarkup.new(:target => $stdout, :indent => 2 )
xml.instruct! :xml, :version => "1.0"
xml.rss :version => "2.0" do
  xml.channel do
    xml.title "Property Feed"
    xml.link "<--YOUR URL-->"
    xml.description "This is my property feed"

    property_hash.each do | key, property |
      pub_date = (Time.now - property.dom.to_i*60*60*24)
      pub_date = pub_date.strftime("%a, %d %b %Y %I:%M:%S")
		
      xml.item do 
        xml.title property.address
        xml.link "http://franklymls.com/#{property.mls}"
        xml.pubDate "#{pub_date} EST"
        xml.description "City: #{property.city}
          Address: #{property.address}
          Price: #{property.list_price}
          Bed: #{property.bed}
          Bath: #{property.bath}
          Sqft: #{property.sqft}
          Built: #{property.built}"
      end
    end
  end
end

To make sure I get routine updates, I run this Ruby code on my server every hour using a cron job and pipe its output to an RSS feed. I am subscribed to the generated RSS feed so now I know exactly when a new property becomes available in my area!

Overall scRUBYt! is very easy to use and for simple screen scraping tasks it should work fine. However, I have found that it can run into some problems when the HTML gets complex. In these cases I would recommend using Hpricot for fine-level scraping.

To view the source code for this entry and to view other screen scraping code that I have written check out my Github page.

If you'd like to look at another example of scRUBYt! in action, feel free to read the post I wrote for my company's blog by clicking here.

In order to stay up to speed in an industry that changes by the day developers must constantly be learning and acquiring new skills. As technology fads come and go it is important for developers to isolate and keep pace with the technologies that are in high demand. Because technologies change so rapidly it is important that we are able to learn as quickly and efficiently as possible. While there is certainly no shortage of information available on the latest technologies, making sense of this information, figuring out what is important, and putting this information to use can be a very challenging task. Therefore it is important that developers learn how to learn.

I recently finished the book Pragmatic Thinking and Learning: Refactor Your Wetware by Andy Hunt, co-author of the seminal book The Pragmatic Programmer. In this book Hunt discusses how people (specifically software developers) learn and offers tips on how we can set out to learn more effectively. Here are some useful tips I stumbled upon while reading Pragmatic Thinking and Learning:

SQ3R

It is unfortunate that the most common way that many people attempt to teach themselves new skills is by reading. A lot of people tend to learn better through experimentation and observation, but in many cases these are not practical first steps to take when starting out with a given subject. On the other hand, reading is an important skill and reading material is abundant so it's important that we are able to get the most out of the things we read.

The main reason reading is not a great way to learn is because it is often viewed as a passive activity. As Hunt describes in his book however humans don't do a great job of simply ingesting material. Instead, Hunt suggests that we turn reading into a more active endeavor and describes using a technique called SQ3R. SQ3R is a simple strategy that can be used to actively engage yourself in whatever it is you are reading. The SQ3R process is as follows:

Survey - Scan the chapter headings and summaries for an overview.
Question - Note any questions you have.
Read - Read in its entirety.
Recite - Summarize, take notes, and put in your own words.
Review - Reread, expand notes, and discuss with colleagues.

There is plenty of information available on SQ3R on the web and of course there are many other techniques that can be used as well. What is important however is that instead of passively trying to absorb what we read we are deliberate in the way we go about consuming information.

Mind Maps

Generating ideas, especially on structured topics can oftentimes be a challenge. Traditional brainstorming techniques such as creating outlines are a great start, but are often limited by their structure and by the restriction of simply using words to express ideas. An alternative approach which Hunt presents in this book is to create a Mind Map.

A Mind Map is a free form diagram that focuses on a central idea and illustrates the relationships among topics associated with that idea. The advantage of using a Mind Map is that you are not limited to simply using words. You can sketch pictures, draw lines between ideas, and use color to group similar thoughts.

As an example, I decided to create a Mind Map for a presentation I will be giving in a few weeks on VirtualBox. To be fair this might not have been the greatest example of the power of using a Mind Map because I have already done quite a bit of research on the topic, but it was a good place for me to get started with Mind Maps and definitely helped me solidify some ideas for my presentation. Here's what I came up with (click to enlarge):

One thing I found interesting while creating this Mind Map was that I actually needed to create multiple drafts. My first draft of this very same image was pretty scatter-brained. But that is the point of creating a Mind Map. It helps you take the ideas from your head and organize them into a tangible format that can then be translated into actual work (i.e. a presentation, an essay, code, etc.).

Harvesting Techniques

Have you ever struggled with a tough coding problem? Do you think as hard as you can about the problem, but are still unable come up with the answer? Then has the solution ever popped in your head upon waking the next day? Did the answer you were looking for suddenly appear during your daily commute when you were least expecting it? This happens all the time and the reason it occurs is because in many case we already know the answers to our problems.

Our brains have an amazing capacity for storing and retrieving all sorts of relevant (and irrelevant) information. As an oversimplification we can think of our brains as being split into two distinct hemispheres. The left hemisphere (L-mode) is analytic, logical, temporal, linear, and rational. It is responsible for processing words and symbols. The right hemisphere (R-mode) is more spatial and is good at processing patterns and relationships. The right side of our brain is intuitive and makes decisions based on hunches, feelings, and images.

As developers we tend to heavily favor using the left side of our brain to solve problems in our daily work, but in order to learn and work as efficiently as possible we must use both sides of our brain. It is very difficult to force R-mode thinking, but you can help coax R-mode ideas out of your brain by using what Hunt calls harvesting techniques. Harvesting techniques are ways to relax your brain's L-mode processing in order to allow your R-mode thoughts to surface. Here are a few examples of harvesting techniques:

Free Form Journaling - Writing is a great way to relax your mind and allow your R-mode thoughts and ideas to escape your brain and present themselves onto paper. When ideas pop into your head, write them down, and then elaborate on those ideas. Simple brainstorming on paper can give you the opportunity to clarify your thoughts.

Walking - Sometimes the best way to come up with ideas is to simply step away from your desk, relax your mind, and go for a walk. While you're walking though, try not to think about anything, especially the problem you are trying to solve. The goal is to silence your L-mode and give your R-mode the chance to do some work.

Image Streaming - This is the process of deliberately observing images in your mind and paying close attention to them. First, pose a problem to yourself or ask yourself a question. Then shut your eyes and relax. As images start to cross your mind describe them out loud. Try to describe as many details as you can using all five senses. This type of thinking can help you discover fresh insights to the problem or question you presented yourself.

At first some of these ideas may seem a bit silly, but keep and open mind and give them a chance. You may be pleasantly surprised with the results.

Conclusion

Pragmatic Thinking and Learning is very accessible and easy to read. Hunt writes the book using simple terms and concepts and illustrates his points with many examples. I've really only begun to scratch the surface of the material covered with this blog entry. Hunt goes into much further detail in his book and offers many additional techniques to help developers discover how they can best approach learning a new subject.

The book describes many different aspects of human physiology and psychology to explain why certain techniques work and others do not. While Hunt does not claim to be an expert in either of these fields he cites a multitude of references and clearly shows that he has put in the thorough research required to write such a book. Overall this book was a pleasure to read and I would certainly recommend it to anyone interested in improving their ability to learn.

Overcoming these issues was always a bit of a challenge for me until about six months ago when I stumbled upon VirtualBox. VirtualBox is Sun's "family of powerful x86 virtualization products." VirtualBox allows you to install and run "guest" operating systems in their own virtual environment on top of a "host" operating system. I know that virtualization has been around for a while, but until finding VirtualBox, I've never had much success running virtualization software. VirtualBox is different. It's intuitive and easy to use. Here are the top 5 reasons why I love VirtualBox:

1. Cheap Machines for Easy Experimentation
2. No More Dual Booting
3. Networking Works
4. Virtual Machine Cloning
5. Guest Additions Make for Seamless Work

1. Cheap Machines for Easy Experimentation

If you've ever wanted to try out a particular OS, but didn't feel like dealing with the hassles mentioned above then VirtualBox may be exactly what you are looking for. VirtualBox allows you to install guest operating systems in their own virtual environment which means there are no side effects on the current state of your machine (other than taking up some hard drive space of course). If you mess something up simply discard your virtual machine (VM) and start over.

The net effect is that you save a lot of time. No need for additional machines, formatting, or partitioning. Simply create a new VM, mount the CD drive (or an .iso image of the OS you want to install), and fire up the virtual machine.

VirtualBox allows you to install a wide range of OSes including, but not limited to Linux (Ubuntu, etc.), BSD, Solaris, Windows (including 3.1). For a full list of guest operating systems click here.

2. No More Dual Booting

When I first started dual booting years ago I appreciated the fact that it was possible to run two (or more) operating systems on the same machine. This proved to be very useful when I wanted to play video games, write, or work with multimedia and graphics tools using Windows, but wanted to develop software using the tools available under various flavors of Linux. There was a problem however. Sometimes I would want to do work that required tools available in one of the OSes but not the other. Other times I would simply prefer to work in a way that favored a particular operating system. In either case I would often have to reboot my machine, select a particular OS, do my work in that OS, and then reboot back to my main operating system. As you can imagine this was quite painful.

With VirtualBox you can have the best of all worlds. You can work with as many OSes as you want at the same time. I do this a lot particularly when developing websites. I can run a full LAMP stack on an Ubuntu Linux guest operating system while I use photo editing software on my Windows host machine. As a bonus you can place the guest OS into "Full Screen" mode and be totally ignorant of the fact that you are running a virtualized operating system. Best of all: No reboots required.

VirtualBox running inside of Windows.

VirtualBox running in full screen mode.

3. Networking Works

One of the most exciting aspects about VirtualBox for me is that networking works between guest and host operating systems, guest and other guest operating systems, and guest operating systems and external networks (i.e. the Internet). This has a lot of interesting implications. One thing that I've been able to do is set up an old Dell computer in my closet running two VirtualBox VMs. I have Ubuntu Linux installed on both VM instances. I use one VM to serve as a centralized Git source code repository (you are using Git, right?). The other VM hosts a LAMP stack for serving a simple Wiki. On my network these two VMs act as individual machines. I can restart, shutdown, or in any other way modify one VM without disrupting the other.

With previous versions of VirtualBox networking could be slightly cumbersome to set up, but version 2.1 makes setup a breeze. If you do try VirtualBox and get caught up along the way there is plenty of documentation and a strong development community to help you out along the way.

VirtualBox networking allows for communication between guests, hosts, and outside networks.

4. Virtual Machine Cloning

I once spent a lot of time setting up one of my virtual machine guest environments to the exact specifications that I needed. I downloaded tons of software, added bash aliases, and tweaked various configurations to suit my needs. Then an interesting thing happened. I ended up needing another VM with the exact same specifications. Taking the time to manually configure another VM wasn't really an option. Fortunately VirtualBox has built in VM cloning* support. All I had to do was run the following command:^**

VBoxManage clonehd </path/source.vdi> </path/target.vdi> -format VDI

This saves a lot of time by reducing the effort needed to reproduce the guest virtual machines that you have previously set up using VirtualBox. This also allows you to share your environment with other people by simply giving them access to your .vdi file (VirtualBox VM instance) and configuration settings. In addition, once you set up a pristine environment you can clone it and always have it stored as a backup incase you mess something up with the VM you are currently working with.

^*There is a bit of a bug in VirtualBox 2.1. The clonehd command seems to be generating corrupt .vdi files. In the meantime as a work around you can simply copy the .vdi file, paste it into a target directory, and then run VBoxManage internalcommands sethduuid <target.vdi> command (thanks to The Bonobo Journal for the tip).

^**Note: On Windows don't forget to make sure you add the VBoxManage application to your path so that you can run it from the command line.

5. Guest Additions Make for Seamless Work

If you decide to install VirtualBox you will definitely want to install the "Guest Additions" package that is available for most guest operating systems. Guest Additions allow for better interoperability between your installed guest VMs and your host system. A few of the best features included with Guest Additions are:

Shared Clipboard - gives you the ability to bi-directionally copy and paste text between guest and host operating systems.

Independent Mouse Control - permits you to move your mouse between guest and host systems. Without Guest Additions installed your mouse is "captured" by the guest virtual machine until the "Host Key" is pressed (usually the right ctrl key).

Better Video Support - allows you to resize your VM window while also dynamically resizing the screen resolution of your guest operating system.

With Guest Additions installed VirtualBox screens can dynamically be resized to any screen resolution.

Conclusion

Besides simply saving time, space, and energy there are a lot of real world applications for VirtualBox. I really like to setup multiple VMs on my host machine to serve as development and staging environments for the various web applications that I work on. Also I like to test my web applications using a combination of different operating systems and browsers (especially various Windows/Internet Explorer combinations). With VirtualBox this is all very straight forward. In the future I would like to use VirtualBox to experiment with running multiple VMs over a network to learn more about setting up clustered, load balanced environments.

There are other virtualization and emulation tools on the market (VMWare, Xen, Wine), but I find that VirtualBox has the right mix of performance and ease of use to suit all of my development needs.

Author: Neal Ford
Rating: 3.5/5

As a developer I am constantly seeking ways in which I can improve my ability to quickly generate high quality code so I was excited to pick up Neal Ford's latest book, The Productive Programmer. This book offers advice on how to accelerate the production and quality of code by exploring many tools and practices that developers can use on a daily basis. In the book, Ford introduces several productivity patterns that can be used immediately, but more importantly he defines a nomenclature that allows developers to construct additional productivity patterns on their own.

The Productive Programmer is broken up into two parts:

Part I. Mechanics

This section discusses tips, tricks, and tools that developers can use while physically engaged in the act of writing code. This section is laid out in a recipe style format and is the part of the book which I found to be the most useful.

Ford breaks Part 1 down into four chapters which he calls the "Principles of Programmer Productivity". These principles are:

Acceleration

This chapter discusses tips developers can use to speed up their development efforts. Ford lists many tips on using launchers, clipboards, and keyboard shortcuts for a variety of popular development tools and operating systems.

Focus

Ford uses this chapter to introduce guidelines on how to limit distractions, context switching, and pesky officemates so that developers can stay in the zone and focus on what's really important - their work!

Automation

Common tasks that are repeated over and over again can kill a developer's productivity. This chapter lists strategies for automating such tasks to help save precious time throughout the workday.

Canocality (i.e. D.R.Y)

In this chapter Ford discusses the importance of version control, continuous build machines, and syncing development tools. This helps to ensure the entire development team is on the same playing field so that precious time is not lost in dealing with inconsistencies between individual developer environments.

Part II. Practice

In Part 2, Ford takes aim at practices that developers can use to improve the quality of their code. By using the techniques and philosophies discussed in this section, Ford illustrates how code can be broken down into small, readable pieces that are easy to maintain. Practices covered include Test Driven Development, Static Analysis, and Meta-Programming. Also discussed are a few development philosophies that can guide developers when making decisions about how they should go about writing their code. These philosophies include: YAGNI (You Ain't Gonna Need It), SLAP (Single Level of Abstraction Principle), and my favorite - Questioning Authority, an excerpt:

This is a story I first heard from Dave Thomas during a keynote address he delivered called Angry Monkeys and Cargo Cults. I don't know if it's true (despite researching it quite a bit), but it doesn't matter-it illustrates a point beautifully.

Back in the 1960s (when scientists were allowed to do all kinds of crazy things), behavioral scientists conducted an experiment where they placed five monkeys in a room with a stepladder and a bunch of bananas hanging from the ceiling. The monkeys quickly figured out that they could climb the ladder and eat the bananas, but every time the monkeys got near the stepladder, the scientists would douse the entire room in ice cold water. You can guess what that generated: angry monkeys. Soon, none of the monkeys would go near the ladder.

Then, the scientists replaced one of the monkeys with a new monkey, who had not been subject to the blasts of water. The first thing he did was make a beeline for the ladder, and all the other monkeys beat him up. He didn't know why they were beating him up, but he quickly learned: don't go near the ladder. Gradually, the scientists replaced the original monkeys with new monkeys until they had a group of monkeys who had never been doused with cold water, yet they would still attack any monkey that approached the ladder.

The point is that just because things have always been done a certain way does not mean that we should blindly accept the status quo in favor of doing things that might actually make more sense or improve our code.

Thoughts

The Productive Programmer was an enjoyable read. At 200 pages the book reads very quickly - perhaps a bit too quickly in fact. My one criticism of the book is that it would have helped to have some additional examples to solidify some of the concepts in the second part of the book (I would guess that is probably my one criticism of most books). I did pick up a lot of small tips however and have been using them in my day to day work. I've noticed some small speed and productivity improvements so in my opinion this book was totally worth the Amazon sticker price.

GitHub is Sexy

GitHub is an online Git repository hosting service that is freely available to anyone who wishes to share their Git managed projects. The service is very similar to SourceForge and Google Code Hosting, but because GitHub uses Git as its source control management system it offers a variety of advantages over other code hosting services.

To begin with, by using GitHub the barrier for working on and contributing to open source projects is significantly reduced. In traditional centralized models only certain developers are given commit rights to any given project repository. This creates a problem however for those developers who checkout the project and make changes. Those developers have no way of keeping track of their local changes and history unless they import the project into their own centralized repository. Doing this however means that the developer will lose the ability to easily synchronize their local project with the original project.

This problem goes away when using Git because all commits are local. You can checkout a project from GitHub, makes changes as you wish, and commit them to your local repository. This allows you to keep track of all of your changes and their history. If you want to synchronize with the original project's repository you simply need to issue a "pull" command to get the latest updates. As a bonus, if you come up with some feature that you think is particularly useful to the original project, you can ask the owner of that project to issue a "pull" request from your repository so your changes can be merged back into the master project.

Setting up your own remote Git repository is not that difficult, but GitHub adds a great web interface on top of Git that feels very much like a social networking site and serves as a platform for developers to share and contribute to each other's projects. A lot of big name open source projects (especially in the Ruby community) are hosted on GitHub including Rails, Merb, and Script.aculo.us so developers can feel confident that the guys at GitHub have their act together. GitHub also offers a variety of other features including private repositories (for a fee), a developer API, and integration with other services including Basecamp, Lighthouse, and Twitter.

Git in Practice

It's highly unlikely that a developer will be able to get their current project team to switch version control systems overnight, but with Git you don't have to. Git has powerful migration tools that can be used in conjunction with other version control systems, especially Subversion. Current Subversion users can use Git to manage the source code on their local machine while still staying in sync with their team's Subversion repository using a built in Git command called "git-svn". The git-svn command allows developers to "push" and "pull" commits to and from a centralized Subversion repository. This gives developers the power of local branching/merging capabilities using Git without forcing the technology on their team members. In fact, no one even needs to know that Git is being used.

Git, it's not for everyone

Git is a pretty incredible tool, but it is likely not going to be championed by all developers. To begin with, Git's learning curve can be a bit steep especially when compared with other version control tools. Developers who desire to give Git a shot should be very comfortable working with the command line because Git's visualization support leaves something to be desired.

Git was created by Linux kernel developer Linus Torvalds so it may not come as a huge surprise that Git support on Windows is a bit lackluster. I would guess that a majority of developers use Windows in some capacity so having a tool with weak Windows support is likely to hinder the adoption of that tool. Fortunately, there is a project called MSysgit which allows developers to use Git on their Windows machines. I've personally used this tool on my Windows XP laptop and find that it works pretty well.

Conclusion

Minor drawbacks aside, Git is a pretty amazing tool for source control management and will be the one that I use for my future projects. Its distributed nature, speed, and branching/merging capabilities make it a great tool for managing source code changes and revision history.

Distributed Source Code Management

Git is a distributed source code management tool which means that there is no centralized source code repository. This is a fundamental shift in the way source control is handled by systems like Subversion, CVS, and Perforce and is probably the most challenging concept to grasp (it certainly was for me) when first learning about Git. Version control using a centralized model is useful for things such as sharing code among developers, providing a history of changes to the code, and backing up data in a single location. However, since Git does not use a centralized repository, a few obvious questions are raised about the Git's workflow model. Questions like:

Where are code commits stored?

All code commits are stored on each developer's local machine. Everyone has a copy of the entire repository and all of its history.

How do developers share code?

Developers share code with Git by executing commands called "push" and "pull". Issuing a push command will push changes made in your repository to another developer's repository. Conversely, issuing a pull command will pull changes from another developer's repository into your repository. This may seem strange at first, but it actually ends up providing developers with a lot of collaborative power. For example, instead of checking code changes into a central repository for everyone to see, two or more developers can share code with one another directly in a controlled fashion by pushing and pulling changes to and from each of their repositories respectively.

How are backups handled in Git?

If all of my code commits are stored on my local machine and my hard drive fails, wouldn't I be in a lot of trouble? Not with Git. If you are working by yourself, regularly backing up your repository using rsync, Mozy, Time Capsule or some other backup system will allow you to completely restore all of your work. This is because Git stores all code revisions and history within each repository.

If you are working with other developers you simply need to clone their copy of the repository. In this sense, Git is actually incredibly fault-tolerant in that it provides geographically redundant archives of a repository because each repository is essentially a mirror of the entire project.

Finally, you could use one of the many publicly available Git repositories including GitHub and repo.or.cz.

Git is Blazing Fast

Since all commits, branching, and merging operations are local, Git is fast. Real fast. Unless you are pushing or pulling code there is no network to worry about. Increased speed means developers can be more productive because they don't have to spend time waiting for costly repository operations to take place across a network.

Git Makes Branching and Merging Easy

In a centralized version control system branching and merging can often times be a source of frustration for developers. All branches must live in the centralized repository and are likely viewable by other developers on the team. If enough developers decide to create branches on the centralized server the repository can quickly become bloated.

Git takes a different approach. Branching and merging in Git occur locally. This dramatically increases the speed in which branching and merging can take place. In addition, a single developer can create and destroy as many branches as they desire without having to worry about polluting a centralized repository. Experimental features can quickly and easily be tested by creating a branch from the current working copy's "master" branch. If the code produced in the branch proves worthy, it can be merged back into the "master" branch. If the code from the branch ends up being useless, the branch can simply be deleted and no one ever needs to know that it existed.

Part 1 Wrap Up

Sunday I'll post Part 2 of my "Getting it on with Git" series where I'll discuss GitHub, Git in practice, and why Git may not be for everyone. Be sure to check back then!

Useful Links

Official Git Site

Git Cheat sheet

Google Tech Talk on Git

Linus Torvalds on Git

GitMagic Book

Domain Specific Languages are typically small, simple languages with a highly expressive syntax and grammar. In computing, DSLs are used to provide domain experts (who may or may not have prior programming experience) with programming capabilities limited to their specific area of expertise.

There are two types of DSLs: external and internal. External DSLs allow you to define a new language using any grammar or syntax you wish. External DSLs must then be parsed and executed by the programming language of your choice. Internal DSLs also define a new language, but are limited to the constructs made available by the implementing language. Because of their dynamic features, languages like Groovy and Ruby make writing internal DSLs relatively easy.

A Stock Trading DSL

A few weeks ago I wrote about trading stocks using Google's Finance API. Sticking with that theme, I thought it would be cool to write a Domain Specific Language that could also be used to mimic stock trading activities.

I'm guessing that most Wall Street traders don't know much about programming, but I'm sure most traders could decipher the actions that would take place after executing the following code:

buy 500, "AAPL", 179.30
buy(250, "AAPL", 179.30) //parenthesis are optional
buy 500, "SUNW", 10.14
sell 350, "AAPL", 179.30

show_transactions "AAPL"

print_portfolio_value

The code above is actual Groovy code. It tells the application to buy 750 shares of Apple stock at $179.30 in two separate transactions, buy 500 shares of Sun Microsystems stock at $10.14, and then sell 350 shares of Apple stock at $179.30. After all the transactions take place the code shows the transaction history for the Apple trades and then prints the total value of the portfolio.

The Groovy code to drive the functionality behind this simple DSL looks like this:

portfolio = [:] //Holds all of the stock transactions

/**
 *A simple object to hold the data for each individual transaction
 */
class StockTransaction {
    def tickerSymbol
    def numberOfShares
    def sharePrice
    def type
    def transactionDate
}

def buy(numberOfShares, symbol, sharePrice){
    transaction('Buy', numberOfShares, symbol, sharePrice)    
}

def sell(numberOfShares, symbol, sharePrice){
    transaction('Sell', numberOfShares, symbol, sharePrice)        
}

def transaction(transactionType, numberOfShares, symbol, sharePrice){
    def transaction = new StockTransaction(tickerSymbol:symbol, 
        numberOfShares:numberOfShares, sharePrice:sharePrice, 
        type:transactionType, transactionDate:new Date())
        
    println "${transaction.type}ing ${transaction.numberOfShares} shares" +
        " of ${transaction.tickerSymbol} at ${transaction.sharePrice}"
    
    //If no transactions exist for a particular stock, check to see that the
    //transaction type is a 'Buy' and if so create a new list to hold 
    //transactions for a particular stock.
    if(portfolio[transaction.tickerSymbol] == null){
        if(transactionType == 'Buy')
            portfolio[transaction.tickerSymbol] = [transaction]
        else
            println "You can't sell a stock you don't own."
    } else{
        portfolio[transaction.tickerSymbol] << transaction
    }     
}

def show_transactions(tickerSymbol){
    portfolio[tickerSymbol].each { transaction ->
        println "${transaction.transactionDate}: ${transaction.type} " +
          "${transaction.numberOfShares} shares of ${tickerSymbol} at " +
          "${transaction.sharePrice}"
    }    
}

def getPrint_portfolio_value(){
    def totalValue = 0
        
    //Add all of the 'Buy' transactions and subtract all of the 'Sell' 
    //transactions from the total portfolio value.
    portfolio.each { tickerSymbol, transactionList ->        
        transactionList.each { transaction ->
            if(transaction.type == 'Buy')
                totalValue += 
                    (transaction.numberOfShares * transaction.sharePrice)
            else
                totalValue -= 
                    (transaction.numberOfShares * transaction.sharePrice)                            
        }
    }
    
    println "Your Total Portfolio Value is: ${totalValue}"
}

The only thing the code really does is define methods for the different types of stock trading activities. I use a HashMap called portfolio to hold each of the individual stock transaction's attributes stored in a simple StockTransaction object. Each time a buy or sell action takes place the portfolio HashMap is updated with the corresponding StockTransaction object.

The readability of the DSL code is greatly improved because Groovy allows for optional parenthesis on methods that take one or more parameters. Notice that the second call to purchase Apple stock uses the parenthesis to illustrate this point. It is important to keep things as simple as possible when creating a DSL. Programmers should try to avoid bogging the domain expert down by forcing him or her to use unnecessary constructs of a language such as parenthesis.

If you look closely at the code that defines the DSL you will notice that in order to make the parenthesis optional for calls to the print_portfolio_value method I had to get a little hacky and define the method as: getPrint_portfolio_value(). This is because Groovy thinks that print_portfolio_value is a call to a property. By adding the "get" in front of the method call, we can "trick" Groovy into calling the method as desired.

After executing the DSL code the output is as follows:

Buying 500 shares of AAPL at 179.30
Buying 250 shares of AAPL at 179.30
Buying 500 shares of SUNW at 10.14
Selling 350 shares of AAPL at 179.30
Sun Aug 17 19:15:05 EDT 2008: Buy 500 shares of AAPL at 179.30
Sun Aug 17 19:15:05 EDT 2008: Buy 250 shares of AAPL at 179.30
Sun Aug 17 19:15:05 EDT 2008: Sell 350 shares of AAPL at 179.30
Your Total Portfolio Value is: 76790.00

Fluency

In the context of trading stocks, the DSL we created above is pretty straightforward, but it does suffer a bit from a problem with fluency. It might be nice if we could make the DSL a bit more readable. For example, what if the DSL was changed to read like this:

buy 500.shares.of("AAPL").at(179.30)
buy 250.shares.of("AAPL").at(179.30)
buy 500.shares.of("SUNW").at(10.14)
sell 350.shares.of("AAPL").at(179.30)

show_transactions "AAPL"

print_portfolio_value

Using categories it is possible to allow for a fluent DSL like the one above. Take a look at the code behind this DSL:

/**
 * The class used for the category
 */
class StockHelper {
    static portfolio = [:]
    
    static buy(self, stockTransaction) { 
        stockTransaction.type = 'Buy'    
        transaction(stockTransaction)
    }
    
    static sell(self, stockTransaction) { 
        stockTransaction.type = 'Sell'    
        transaction(stockTransaction)    
    }
    
    static transaction(stockTransaction){
        println "${stockTransaction.type}ing " +
        "${stockTransaction.numberOfShares}" +
        " shares of ${stockTransaction.tickerSymbol} at " +
        "${stockTransaction.sharePrice}"
        
        if(portfolio[stockTransaction.tickerSymbol] == null){
            if(stockTransaction.type == 'Buy')
                portfolio[stockTransaction.tickerSymbol] = [stockTransaction]
            else
                println "You can't sell a stock you don't own."
        } else{
            portfolio[stockTransaction.tickerSymbol] << stockTransaction
        }                 
    }    
    
    static getShares(self) { 
        def startTransaction = new StockTransaction(numberOfShares:self, 
            transactionDate:new Date())
        return startTransaction
    }
    
    static of(self, tickerSymbol) { 
        self.tickerSymbol = tickerSymbol         
        return self
    }    
    static at(self, sharePrice) { 
        self.sharePrice = sharePrice 
        return self
    }    
    
    static show_transactions(self, tickerSymbol){
        portfolio[tickerSymbol].each { transaction ->
            println "${transaction.transactionDate}: ${transaction.type} " +
              "${transaction.numberOfShares} shares of ${tickerSymbol} at " +
              "${transaction.sharePrice}"
        }    
    }
    
    static getPrint_portfolio_value(self){
        def totalValue = 0
            
        portfolio.each { tickerSymbol, transactionList ->
            
            transactionList.each { transaction ->
                if(transaction.type == 'Buy')
                    totalValue += 
                        (transaction.numberOfShares * transaction.sharePrice)
                else
                    totalValue -= 
                        (transaction.numberOfShares * transaction.sharePrice)                            
            }
        }
        
        println "Your Total Portfolio Value is: ${totalValue}"
    }    
}

By chaining methods calls together we can make our code very fluent. We build a StockTransaction object by calling 3 methods: getShares(), of(), and at(). Each of these calls is responsible for building a specific piece of the StockTransaction object and passing it along to the next call. The self keyword is used to reference the target instance (i.e. the StockTransaction object being created). After the StockTransaction is created, the buy or sell methods can be called to update the portfolio HashMap just as in the first example.

Even though parenthesis and the dot notation (for method access) must be used in order to execute this DSL code, I personally think this code is easier to read and understand as compared with the previous example.

Conclusion

The code in this blog entry is pretty brittle, but it illustrates how easily a DSL can be created using the Groovy programming language. In a real life scenario programmers must be diligent about the design and usage of the DSLs they create.

By creating an internal DSL domain experts with some programming knowledge will also be able to exploit other constructs found in the implementing language such as loops and conditional statements. For example, code could be written to do things such as:

if(currentStockPriceVariable < SOME_SHARE_PRICE_CONSTANT){
    buy 500.shares.of("AAPL").at(currentStockPriceVariable)
}

or perhaps something like this:

3.times {
    sell 500.shares.of("AAPL").at(179.30)
}

Over the past few years DSLs have gained popularity along with the rise of dynamic languages. For example, GANT is a popular DSL for the Groovy programming language that can be used to script Ant style builds. It will be interesting to see what other types of DSLs become available in the future. Be sure to look out for Martin Fowler's upcoming book on the subject.

Code

Click here to download the code examples from this post.

Disclaimer

I am in no way endorsing any of the companies mentioned in this blog entry and all company references are for example purposes only.

Opening Keynote - Rich Kilmer

Rich Kilmer's opening keynote was by far the best presentation of the day. Rich is an incredibly experienced and skilled Ruby developer who was one of the earlier adopters of the language. During his keynote, Rich walked through his experiences with Ruby dating all the way back to 2001. He has certainly worked on some interesting projects for impressive government agencies including DARPA and the United State Air Force. Rich also talked about projects he has created for the Ruby community including FreeRIDE - a Ruby IDE and RubyForge - a code repository for Ruby projects.

One particularly interesting thing Rich mentioned was that Ruby is becoming a mainstream language. I personally find this both exciting and scary. It is exciting that the community is growing at that there will likely be more opportunities to work on Ruby projects professionally, but it is a bit worrisome that the community could start to be overpopulated with run-of-the-mill developers transitioning from other dying mainstream languages.

Ruby Puzzlers - Mike Furr

Mike Furr, a doctoral student at UMCP, gave a presentation on edge case scenarios that occur in the Ruby language. Since I'm pretty new to Ruby I struggled to follow along, but Mike did do a good job of explaining how Ruby code reacted under various situations. Even though it can be rare to deal with edge cases in a language, it is nice to know how to avoid them or work your way out of them if they do arise.

Practical JRuby - David Keener

A Java(ish) presentation at a Ruby conference? It may seem a bit strange at first, but it actually makes a lot of sense. Java as a platform has some pretty rock solid features including performance, garbage collection, and concurrency. A lot time and thought has also been invested into Java libraries and corporate infrastructure so in many ways it makes sense to take advantage of what is already available. David Keener hit on all of these points in his presentation and gave an example of how to integrate JRuby with the JFreeChart Java library.

Tools for Your Ruby Toolbox - Dave Bock

Dave Bock is another FGM employee that I had the pleasure of working with a few years back. Dave has since moved on and started his own shop called Codesherpas. In his presentation, Dave talked about 3 lightweight tools that can be used to quickly accomplish tasks in Ruby. Dave first talked about StaticMatic, a framework for developing simple static websites using templates, but without using all of the built in functionality of Rails. Another web framework Dave talked about was Sinatra. Again, this framework allows developers to create simple web applications without the full Rails stack, but it also includes some RESTful functionality. Finally, Dave talked about GServer, a generic server library that allows developers to easily create server applications of their own. During his presentation Dave demoed a quick "Knock Knock" client he created using the GServer library.

Lightning Talks

There were five lightening talks given yesterday, but again Bryan Liles stole the show with his presentation on Project Management. The basic takeaway: meet with your reports regularly, project confidence, and set up processes that are easy to follow. Sounds so simple doesn't it?

Closing Keynote: Bad Ruby - Stuart Halloway

Stuart Holloway closed RubyNation with a talk on Bad Ruby. Playing off of a point made earlier in Rich Kilmer's talk about Ruby becoming mainstream, Stuart noted that now was the time for Ruby developers to begin to establish best practices for the language so that common pitfalls can be avoided when mainstream developers start to migrate to Ruby. (Assuming of course "mainstream" developers actually migrate to Ruby).

Conference Wrap Up

Overall the conference was pretty great. Day two was a little less energetic than day one, but that may not be too surprising seeing as how it was on a Saturday. I was a little disappointed that Bruce Tate wasn't able to make the conference because I was really looking forward to hearing him speak, but I was pleasantly surprised at the quality of the other speakers. I know that a lot of people put a lot of hard work into the conference so I'd personally like to thank everyone responsible for putting on this years conference and I can't wait to attend again next year.

In case you weren't able to make it, here's a quick rundown of today's events:

Opening Keynote: Ceremony vs. Essence - Neal Ford

Neal Ford is a seasoned consultant and speaker that I've seen present on more than one occasion. I've always enjoyed his presentations, but his talk today really hit home with me. Neal's talk basically boiled down to the fact that we developers need to Keep It Simple (KISS) and avoid the "accidental complexity" that is often unnecessarily added to the problems we are trying to solve. I've listen rhetoric like this a 1,000 times, but I've never heard it articulated so elegantly. It was literally like he was looking inside of my head and expressing the way I (and likely a lot of developers) feel about the current state of the industry (especially if you are a Java programmer like me).

DSLs and Ruby: Blurring the Lines Between Programs And Data - Russ Olsen

I used to work with Russ at a FGM so I was really looking forward to hearing him speak. Russ has become quite a leader in the Ruby community especially after publishing his first book: Design Patterns in Ruby. Russ's talk centered around Domain Specific Languages (DSLs) and how relatively easy it is to implement one using the Ruby language. During his presentation Russ put together a simple Internal DSL for example purposes and also discussed use cases for when it is appropriate to create a DSL to solve a problem.

The Culture of Innovation in Ruby - Glenn Vanderburg

Innovation in testing frameworks in Ruby was the topic of Glenn Vanderburg's talk. Glenn discussed some of the interesting testings frameworks available in the Ruby community (Rspec, Mocha, Flex Mock, Shoulda) and compared them to some of the stagnant frameworks in the Java community (JUnit, TestNG). Glenn also pointed out how Ruby's testing frameworks tend to make it much easier to mock objects and create more thorough tests with much less effort than the equivalent tests in the Java world.

Living on the Edge - Yehuda Katz

Yehuda Katz of EngineYard talked about some of the new(er) technologies that are emerging in the Ruby community including Merb, DataMapper, Sake, Thor, YARDoc, and GitHub. DataMapper looks like an especially interesting alternative to the incumbent ORM ActiveRecord. It promises thread safety and increased performance which are obviously two great things. GitHub looks pretty cool too. I'm not quite ready to abandon Subversion just yet, but I will definitely be checking out Git in the near future.

Lightning Talks

Chris Bucchere, David Jones, Simon Kaczor, and Bryan Liles all gave lightening talks today on a variety of topics including social networking, state machines, command line libraries, and testing respectively. All of the talks presented useful information, but Bryan Liles was without a doubt the most entertaining speaker of the day. He made it through about 65 slides in 15 minutes and was still able to convey a very important message about how often developers should be testing their code. Check out his blog at: http://smartic.us/

Archaeopteryx: A Ruby MIDI Generator - Giles Bowkett

I actually wasn't able to attend Giles Bowkett's session because I had a previous engagement to attend, but I did get a chance to view some screencasts that show off his Ruby MIDI Generator. Pretty cool stuff.

Tomorrow

Tomorrow looks to be a pretty exciting day as well. I'm especially looking forward to talks from my former co-worker Dave Bock and author Stuart Holloway.