Justin Spradlin

Book Review: Eloquent Ruby

Justin Spradlin — Sun, 06 Mar 2011 06:16:23 +0000

I can’t even begin to imagine how difficult it would be to write a book covering such an in-depth topic as the Ruby Programming language. Add to that a target audience with a widely diverse range of skills and the tasks seems to be out right impossible. Fortunately for us mere mortals Russ Olsen has taken on this challenge and surpassed my high expectations in his second book, Eloquent Ruby.

The first part of the book is for the Ruby newbies, but takes an interesting deviation from most programming books. Instead of focusing simply on the syntax and language libraries, Eloquent Ruby focuses on the community aspect of the Ruby programming language. Each programming community has its own style and norms and without a lot of direction and practice, these style and norms can be difficult to learn. The Ruby community is heavily opinionated and Russ’s book does a great job of explaining these opinions and their manifestation in many Ruby codebases.

In Part Two of Eloquent Ruby, Olsen dives into the core concepts and building blocks of the Ruby programming language. This section of the book covers the everyday usage patterns and common best practices when working with Ruby’s classes, iterators, blocks, and modules.

The power and flexibility of Ruby is put on full display in Part Three of Russ’s book. Newcomers to Ruby are often mystified by Ruby code that appears to be doing magical things like dynamically adding methods to a class. In this section, Russ pulls back the curtain and explains the magic that is metaprogramming. Those unfamiliar with the concept may find the topic confusing at first, but Olsen breaks it down in a way that is easy to understand. This section thoroughly covers Ruby’s method_missing hook and how to open classes to add your own customizations.

In the last section Russ ties a lot of the concepts of the book together through a discussion on how Ruby can be used to create Domain Specific Languages (DSLs). As a bonus Russ finishes the book with a chapter on how to package your code into a gem (Ruby library) and a chapter on the different Ruby Implementations.

At almost 400 pages I feared Eloquent Ruby would take a while to read, but I was pleasantly surprised at the pace at which I made it through the book. Even though the book was a quick read it still felt as if I had gained a lot of new knowledge after its completion. I credit the speed reading and depth of understanding to Olsen’s writing style. The book is broken up into 30 concise chapters that each focus on a specific topic. This makes the book’s information very easy to digest.

Following a similar pattern from his first book, Design Patterns in Ruby, each chapter of Eloquent Ruby describes code examples “In the Wild” where Olsen discusses code snippets from open source projects using the topics covered within that chapter. In my opinion giving “real world” code examples was a great way to prove the relevance of the information covered in the book. Overall, Eloquent Ruby was a great read and I’d highly recommend it to Rubyist at all skill levels.

Introducing GMoney – A RubyGem for Interacting with the Google Finance API

Justin Spradlin — Mon, 01 Feb 2010 06:00:30 +0000

Update March 03, 2010:

Due to changes in the Google Finance API only GMoney gem versions >= 0.4.3 are working. If you run gem update gmoney to get the latest version or install the gem from scratch you will be fine.

Let’s pretend the year is 2007 and I have a close “friend” who is really excited about the upcoming 2010 World Cup. My friend is not too naive and realizes that airfare to Africa, lodging, tickets to the games, and Safaris are all very expensive. My friend has a little extra cash to save each month and with a three year time horizon he figures he can stow away a little money in a moderately aggressive mutual fund in order to take advantage of some growth opportunities.

Unfortunately for my friend the next two years would be some of the worst the world economy has ever seen. Like many people, my friend took a bath on his investments and his hopes and dreams of going to the World Cup went down the drain along with his money.

It’s too bad my friend didn’t keep better track of his investments. Perhaps if he had he could have pulled his money out of the stock market and salvaged his vacation. Since my friend is an avid Ruby lover I figured I’d help make sure he doesn’t end up in this situation ever again. That’s why I wrote GMoney, a RubyGem for interacting with the Google Finance API.

Background

The Google Finance API allows users to interact with Google Finance and can help developers programmatically “request a list of a user’s portfolios, retrieve performance and return statistics on an existing portfolio, query the positions and transactions in a portfolio, and create new portfolios and transactions.”

GMoney is a wrapper for this API and allows developers to take advantage of the API’s functionality using the Ruby programming language. To minimize the learning curve I tried to give GMoney an interface similar to that of ActiveRecord so that developers would have a familiar starting point.

Google Finance Model

The Google Finance model is pretty simple. It is made up of the following three entities:

Portfolios

A portfolio is a collection of positions that the user holds in various securities, plus metadata. Each portfolio entry contains the portfolio’s title along with data such as currency and total market value. Each portfolio entry also contains a link to the portfolio’s position feed (i.e. a Portfolio “has_many” Positions).

Positions

A position is a collection of information about a security that the user holds. Each position entry contains the ticker exchange and symbol for a stock, mutual fund, or other security, along with the number of units of that security that the user holds. Each position entry also contains a link to the position’s transaction feed (i.e. a Position “has_many” Transactions).

You can’t directly create or update position entries; positions are derived from transactions.

Transactions

A transaction is a collection of information about an instance of buying or selling a particular security. Each transaction entry contains a transaction type (such as buy or sell), a number of units, the price, and so on.

Installation

To install GMoney simply run the following command:

gem install gmoney

Login

You will need a Google account in order to use the Google Finance API. Once you have obtained an account you can login via GMoney using the following code:

require 'rubygems'
require 'gmoney'

GMoney::GFSession.login('','')

Create a portfolio

If you were to login to Google Finance for the first time using a browser you would see the following screen showing the default Google Finance Portfolio called “My Portfolio”:

To create your own Portfolio using GMoney use the following code:

portfolio = GMoney::Portfolio.new
portfolio.title = "My New Ruby Portfolio"
portfolio.save

After executing the code and refreshing the browser you should see that the “My New Ruby Portfolio” has been created.

Create a transaction(s)

As mentioned above transactions are used to buy and sell stocks within a given portfolio. The positions within your portfolio will be based on the transaction information. Running the following code will buy shares of Apple, Red Hat, Google, and Microsoft and place them into your portfolio:

stocks = {"NASDAQ:GOOG" => 529.00, 
          "NASDAQ:AAPL" => 192.00, 
          "NYSE:RHT" => 27.00, 
          "NASDAQ:MSFT" => 28.00}
 
stocks.each do |symbol, price| 
  transaction = GMoney::Transaction.new
  #pid is the human readable id of the portfolio created above
  transaction.portfolio = portfolio.pid
  transaction.type = GMoney::BUY
  transaction.shares = 50
  transaction.ticker = symbol
  transaction.price = price  
  transaction.save
end

The result of running the code is as follows:

Update Transaction

If you actually meant to buy 100 shares of Goolge instead of 50 you can update the transaction using the following code:

#I know the transaction id is '4'
transaction = GMoney::Transaction.find "#{portfolio.pid}/NASDAQ:GOOG/4"
transaction.shares = 100
transaction.save

The result:

Delete Transaction

We can delete a transaction like so:

GMoney::Position.find("#{portfolio.pid}/NASDAQ:MSFT").delete

Now we have no more MSFT stock:

Reading all the Positions in a Porfolio

If we’d like to see all of the positions within our portfolio we can execute this code:

portfolio.positions.each do |position| 
  puts position.title
end

Output:

Red Hat, Inc.
Apple Inc.
Google Inc.

Gotcha

There is a bit of a “gotcha” right now with the Google Finance API. For whatever reason, stock returns information is not being sent by Google. I figured it was something I was doing wrong in my code, but then I used Google’s official API client and was still unable to retrieve the returns data. I’ll be sure to file a bug report with Google over the next few days.

Conclusion

GMoney offers a lot more features than what has been written about here. Be sure to check out the documentation and the source code on Github. The full script of the code used for this blog entry can also be found on Github.

Overall building GMoney was an incredible learning experience. It was my first significant Ruby project and also my first Open Source project. As of today GMoney has been downloaded almost 200 times. I’ve tested GMoney on Windows and Linux using Ruby versions 1.8.6, 1.8.7, and 1.9.1 (via RVM). Feedback and bug reports are more than welcome.

With GMoney in place my friend will be sure to keep much better track of his investments and is looking forward to the 2014 World Cup in Brazil.

Disclaimer: I am in no way endorsing any of the companies mentioned in this blog entry and all company references are for example purposes only.

Create a Photostream Using jQuery and the Flickr API

Justin Spradlin — Wed, 13 Jan 2010 02:18:04 +0000

Recently I added a Flickr photostream to the footer of the home page of this blog and I thought it would be cool to post an entry on how you can use jQuery and the Flickr API to do the same. Even if you’re not familiar with jQuery or the Flickr API it’s actually quite simple to accomplish this task.

Prerequisites

Download jQuery

jQuery is a fast and concise JavaScript Library that simplifies HTML document traversing, event handling, animation, and Ajax interactions for rapid web development.

Download jQuery lightBox

jQuery lightBox is a jQuery plugin that can be used to display a gallery of images on the current page using the power and flexibility of jQuery’s selectors.

Create a Flickr Account and API Key if you don’t already have them

A Flickr API key is necessary for accessing additional service features offered by Flickr.

Code Time

Let’s assume that our HTML page is as follows:

 

 
  JQuery/Flickr API Code

There are only a few important things to be aware of with this code. First we need to make sure that we include all of our style sheets and JavaScript files in the head section of our HTML page. In addition to including the jQuery and jQuery lightBox files notice that I have also created a style sheet called “style.css” and a JavaScript file called “custom.js“. We will use these files to store our custom code.

Flickr has a very extensive API that allows developers to grab images using many different techniques. For this example I have chosen to pull back images using the flickr.photos.search method. This method – among other things – allows users to grab images that have been labeled with a specific tag.

In the custom.js file we will include the following code:

$(function() {
    var apiKey = '';
    var userId = '';
    var tag = '';
    var perPage = '25';
    var showOnPage = '6';
    
    $.getJSON('http://api.flickr.com/services/rest/?format=json&method='+
        'flickr.photos.search&api_key=' + apiKey + '&user_id=' + userId + 
        '&tags=' + tag + '&per_page=' + perPage + '&jsoncallback=?', 
    function(data){
        var classShown = 'class="lightbox"';
        var classHidden = 'class="lightbox hidden"';
        
        $.each(data.photos.photo, function(i, rPhoto){
          var basePhotoURL = 'http://farm' + rPhoto.farm + '.static.flickr.com/'
            + rPhoto.server + '/' + rPhoto.id + '_' + rPhoto.secret;            
            
            var thumbPhotoURL = basePhotoURL + '_s.jpg';
            var mediumPhotoURL = basePhotoURL + '.jpg';
            
            var photoStringStart = ';'                
            var photoString = (i < showOnPage) ? 
                photoStringStart + classShown + photoStringEnd : 
                photoStringStart + classHidden + photoStringEnd;
                                        
            $(photoString).appendTo("#flickr");
        });
        $("a.lightbox").lightBox();
    });
});

Let’s examine the variables that are declared at the top of this JavaScript file:

apiKey – Your Flickr API key
userId – User ID of the user whose pictures you wish to grab
tag – A comma separated list of tag values
perPage – The number of images you wish to pull back (maximum: 500)
showOnPage – The number of images you wish to display on the page

Next we call jQuery’s $.getJson method passing it a custom URL and a callback method. The URL is dynamically created based on the values set in the variables mentioned above. The callback method takes the JSON object returned by Flickr and parses it to grab the relevant photo data. The JSON returned by Flickr might look something like this:

jsonFlickrApi({"photos":{"page":1, "pages":1, "perpage":25, "total":"17",
"photo":[{"id":"42626017633f", "owner":"705311402@N00", 
"secret":"2casd5d2079e", "server":"4010", "farm":5, "title":"Some Photo Title",
"ispublic":1, "isfriend":0,"isfamily":0}, {"id":"42626017633f", 
"owner":"705311402@N00", "secret":"2casd5d2079e", "server":"4008", 
"farm":5, "title":"Some Other Photo", "ispublic":1, "isfriend":0, 
"isfamily":0}, ...]}, "stat":"ok"})

Using the jQuery $.each method we can easily iterate through all of the photo elements and create the dynamic HTML needed to add the photostream to our HTML page. For each photo, we grab its relevant information (id, server location, title, etc.) and build image links with the following structure:


    _s.jpg"/>

These images will be appended to the div with the ID of “flickr” on our HTML page. The jQuery $.append method is used to accomplish this task. Notice that we end the “thumbnail” photo URLs with “_s“. This will request a small square shaped version of our image from Flickr which is perfect for display in a photostream.

Each image will have a CSS class of “lightbox” so that we can properly select all of the images to be used by the jQuery lightBox library. Notice that the callback function passed into the $.each method stores a loop counter variable called “i“. Since we only want 6 images displayed on our page at once we can use this counter variable to check the current loop position and add an additional CSS class called “hidden“” to the images not within the first 6 photos returned. These photos will be added to the page, but will not be visible to the user.

After we add all of the images to the page we call $("a.lightbox").lightBox(); to create a Lightbox for all of the images (including the hidden ones) in our photostream.

When we view the page we will see the first 6 images in our photostream.

Clicking on any of the images will launch a Lightbox and allow us to scroll through the remaining photos.

Source Code

The complete source code example for this blog entry can be downloaded by clicking here.

Hello World: 2.0

Justin Spradlin — Sat, 02 Jan 2010 01:30:30 +0000

Hello and welcome to JustinSpradlin.com (formerly Fiascode.com). This is the personal technology blog and resume site for Justin Spradlin.

The last few weeks of 2009 were spent rewriting this site from the ground up. I hope you like the new design. Come back often for updates or subscribe to the feed to have the latest posts appear directly in your favorite RSS reader.

Happy New Year!

Giving Design Patterns a Second Chance: Ruby Edition

Justin Spradlin — Thu, 02 Jul 2009 00:00:28 +0000

I have a confession to make.

You know that really popular design patterns book that most software developers (especially in the Java world) claim to have read? You know the one I’m talking about. The one written by the “Gang of Four”. The one you were supposed to have read during your junior or senior year of college. Yeah, this one.

I’ve never read it.

Sure, I’ve owned it. I have even thumbed through it a few times. But I’ve never sat down with any real intention of absorbing the information within its covers. The problem for me personally was that when I first opened Design Patterns I was just starting my software development career. I was too young and hadn’t seen enough code to really comprehend the authors’ message or understand the need for such a book.

A few years have passed since then and while it’s still early in my career I now have a much better idea of how to develop software and have a much better understanding of Object Oriented Design. Even though I spend a significant amount of my free time developing my skills and reading about software development I have yet to read the seminal Design Patterns book.

Design patterns have left a bad taste in my mouth. Over the years they have been abused and misunderstood. They seem to have become caught up in the all too familiar silver bullet buzzword vernacular of the IT industry (SOA anyone?). So I simply avoided them. Unless I was about to interview for a new job, design patterns never so much as crossed my mind. Repeat after me: Singleton, Factory, MVC. You’re hired!

But things have changed. Over the past year and a half or so I’ve really started to focus on my career as a software developer. I’ve learned new operating systems and languages, created this blog, and started attending local developer meetings on a regular basis. While I still have my favorite technologies and interests, I’ve been doing a much better job of keeping an open mind about trying out new development tools and methodologies.

Enter Russ Olsen. In addition to being a great developer and all around good guy, Russ is someone who I had the honor of working with at the beginning of my career. He was very much a mentor and I credit him with teaching me many of the applied fundamentals of software development. So what does Russ have to do with design patterns? Well it just so happens that a few years after I left the company where Russ and I worked together he wrote a book called Design Patterns in Ruby. Because of my respect for Russ and my fondness of Ruby I decided I’d give the book a shot. While I was certainly expecting to learn some new things, I was surprised (although I shouldn’t have been) by how much great information I was able to extract from this book.

In their most minimal form, design patterns are simply reusable solutions that have been created to solve common coding problems. While this may seem intuitive, it was actually somewhat of a revelation to me – a person who has ignored the direct usage of design patterns for all these years. I see now that given an appropriate situation design patterns can be used to create consistent, maintainable code. The important thing to remember is that each pattern has a specific use and should only be used when applicable.

Although the title of the book is Design Patterns in Ruby, a majority of the principles and patterns apply to many other object oriented languages as well. In fact, while I was reading Russ’s book I was in the middle of working on a small Java project. This project had to keep track of rapid state changes on a particular website. I needed to keep track of a countdown timer, a list of participants, and an aggregate score. Each of these pieces of information was updated independently, but I needed to make decisions in my code based on the combination of this information at any given time. After reading the chapter on the Observer pattern, I realized it would be the perfect fit. Prior to using the pattern my code was a huge pile of tangled spaghetti. Afterwards, it was clean and easy to read.

I really liked the way the chapters were organized in Design Patterns in Ruby. Russ started each chapter by posing a common programming problem. Then he would explain how a specific design pattern could be used to solve that problem. As the chapter progressed Russ would work through the problem using examples that were very easy to understand. Each chapter ended with real world examples of the patterns as they have been implemented in the Ruby core libraries or various open source projects. I found the real world code samples to be especially useful as compliments to the examples given in the book.

Not only did I enjoy reading this book because it was written by Russ, but I genuinely feel like it has helped to make me a better programmer. In addition, it has helped increase my Ruby specific programming skills. Learning to program in Ruby is easy, but learning to program the “Ruby Way” is a much greater challenge. Ruby is full of powerful idioms that are not necessarily obvious without a deep examination of the language. Russ’s book helps enlighten would be Ruby programmers to many of Ruby’s powerful language features.

For example, compare an implementation of the Proxy pattern written in Java and Ruby:

Java Version

MathService.java

public interface MathService {
    public void add(int x, int y);
    public void subtract(int x, int y);
}

MathServiceImpl.java

public class MathServiceImpl implements MathService {
    public void add(int x, int y) {
        System.out.println("Result ("+x+ " + "+y+"): " + (x + y));
    }

    public void subtract(int x, int y) {
        System.out.println("Result ("+x+ " - "+y+"): " + (x - y));
    }
}

MathServiceProxy.java

public class MathServiceProxy implements MathService {
    private MathService delegate;

    public MathServiceProxy(MathService ms){
        this.delegate = ms;
    }

    public void add(int x, int y) {
        System.out.println("Call made to the add method");
        delegate.add(x, y);
    }

    public void subtract(int x, int y) {
        System.out.println("Call made to the subtract method");
        delegate.subtract(x, y);
    }
}

MathServiceDriver.java

public class MathServiceDriver {
    public static void main(String[] args) {
        MathService ms = new MathServiceProxy(new MathServiceImpl());
        ms.add(1, 2);
        ms.subtract(4, 3);
    }
}

Ruby Version

math_service.rb

class MathService
  def add(x, y)
    puts "Result (#{x} + #{y}): #{x + y}"
  end

  def subtract(x, y)
    puts "Result (#{x} - #{y}): #{x - y}"
  end
end

class MathServiceProxy
  def initialize(math_service)
    @delegate = math_service
  end

  def method_missing(name, *args)
    puts "Call made to the #{name} method"
    @delegate.send(name, *args)
  end
end

ms = MathServiceProxy.new(MathService.new)
ms.add(1,2)
ms.subtract(4,3)

Result of Running Either Program

Call made to the add method
Result (1 + 2): 3
Call made to the subtract method
Result (4 - 3): 1

In this example we use the Proxy pattern to print out a message before each method call. If we decided to add any methods to our MathService interface we would need to update our MathServiceImpl and MathServiceProxy Java classes as well. However, by using Ruby’s built in method_missing method we have the ability to wrap each MathService method call dynamically without having to write a new method in our MathServiceProxy class. Ruby removes the tedium of implementing the Proxy pattern.

This is just one small example of the flexibility and power provided by the Ruby language. Before reading Russ’s book I didn’t necessarily “get” many of Ruby’s idioms. Reading Design Patterns in Ruby has certainly helped solidify my understanding and is great for programmers who would like to take their Ruby skills to the next level.

Code Samples

All code samples from this blog entry can be downloaded from GitHub by clicking here

Ruby Screen Scraping with scRUBYt!

Justin Spradlin — Mon, 27 Apr 2009 10:00:21 +0000

I am currently in the market to buy my first home so I've been spending a lot of time on various real estate websites searching through listings trying to find the perfect property. I live in a competitive housing market so it is important that I am informed whenever a new property becomes available. Logging onto any number of real estate websites to check for new listings each day is very repetitive and time consuming. Fortunately, it is possible to easily gather this information automatically using a technique called screen scraping.

Since most web pages are simply made of HTML it is easy for a computer to parse and store the information contained within these documents. Each programming language commonly has a host of libraries to assist in the screen scraping/parsing process and Ruby is no exception. To create simple screen scrapers in Ruby I have been using a library called scRUBYt!. scRUBYt! provides methods to access a given website and scrape its content. All the programmer needs to do is provide the XPath string to the desired information.

Using the scRUBYt! library has allowed me to write a small screen scraper script to access the FranklyMLS.com website, check for new listings, and then report back with the results. This has saved me a lot of time and effort. Let's dive into some code to see how this is done.

First, we'll need to create a simple class to store the information that we scrape from the FranklyMLS.com website. The Property class will hold various property related information (price, MLS number, square footage, etc.):

class Property
  attr_accessor :mls, :list_price, :dom, :address,
                :city, :zip, :bed, :bath, :sqft, :built

  def initialize(property)
    @mls = (property/:mls).inner_html
    @list_price = (property/:list_price).inner_html
    @dom = (property/:dom).inner_html
    @address = (property/:address).inner_html
    @city = (property/:city).inner_html
    @zip = (property/:zip).inner_html
    @bed = (property/:bed).inner_html
    @bath = (property/:bath).inner_html
    @sqft = (property/:sqft).inner_html
    @built = (property/:built).inner_html
  end
end

Next we'll need to make sure that scRUBYt! is installed. If you don't already have Github set up as one of your gem repositories do so now by executing the following command:

gem sources -a http://gems.github.com

Then install the scRUBYt! gem:

gem install jspradlin-scrubyt

Side note: I've built some functionality into the scRUBYt! library so you will need to grab the gem from my Github repository (i.e. jspradlin-scrubyt). I've spoken with the lead developer on the scRUBYt! project and it looks like my changes might make it into a future version of the official gem.

At this point we need to give scRUBYt! the URL of a website that we wish to scrape. The FranklyMLS.com website has its own special URL query syntax which displays only properties that meet our specific criteria. For example, if we only wanted to find active listings in the following zip codes - 22201, 22202, 22203 - the FranklyMLS.com URL would be:

http://franklymls.com/default.aspx?m=R&s=(22201,22202,22203)+active

We can dynamically generate a URL with our specific housing criteria by including the following code in our script:

# Generate the URL for FranklyMLS.com given
# the following criteria:
zips = [22201,22202,22203,22209]
beds = [2, 3].collect{ |bed| bed.to_s+'bdr'}
min_price = 150 #in thousands
max_price = 350 #in thousands
exclusions = ['JEFFERSON'].collect{ |exclude| "+-#{exclude}"}

fmls_url = "http://franklymls.com/default.aspx?"
fmls_url += "m=R&l=#{min_price}K&h=#{max_price}K"
fmls_url += "&s=(#{zips.join(',')})+active"
fmls_url += "+(#{beds.join(',')})"
fmls_url += exclusions.to_s

Now we're ready to scrape some housing data. Once the FranklyMLS.com property page loads we are presented with a table that contains information about the listings that meet our criteria (image modified to save space):

The HTML that generates this table would appear like this (modified to save space):


  ...
  
    ...
    
  ...

    ...AR6552162 
    $256,000 
    $339,000
    $
     
     
    562
    1931 CLEVELAND #313 ARLINGTON
    22201
     
    CLEVLAND HO
    3/1
    860
    1960
    ...
    1
    160
    x

Finally, the Ruby code to scrape this data using scRUBYt!:

#Scrape the FranklyMLS.com website using scRUBYt!
property_data = Scrubyt::Extractor.define do
  fetch fmls_url

  properties '//table[@id="dgRealtorStyle"]' do
    property "//tr" do
      mls "/td[1]//a[2]"
      list_price "/td[2]"
      dom "/td[7]"
      address "/td[8]"
      city "/td[9]"
      zip "/td[10]"
      bed "/td[13]",
        :format_output => lambda {|bed_bath| bed_bath.split('/')[0]}
      bath "/td[13]",
        :format_output => lambda {|bed_bath| bed_bath.split('/')[1]}
      sqft "/td[14]"
      built "/td[15]"
    end
  end
end

If you look at the table, the HTML code, and the Ruby code you'll see that I've color coordinated each separate piece of information to illustrate how it is parsed and then stored. The scRUBYt! library will "fetch" the given URL, locate the HTML elements by the given XPath, and then store the data.

Once we have all of the data collected we may want to do something useful with the information such as convert it into an RSS feed. We can accomplish this by using the Hpricot and Builder libraries (which should be installed as dependencies of scRUBYt!). The code for the RSS conversion would look like this:

# Read in the XML generated by scRUBYt! then
# convert the data into Property objects
# and store them in the property_hash.
property_hash = {}

hp = Hpricot.XML(property_data.to_xml)
(hp/:property).each do |property|
  property_hash[(property/:mls).inner_html] = Property.new(property)
end

# Using the Builder library, iterate through
# the property_hash to generate an RSS feed.
xml = Builder::XmlMarkup.new(:target => $stdout, :indent => 2 )
xml.instruct! :xml, :version => "1.0"
xml.rss :version => "2.0" do
  xml.channel do
    xml.title "Property Feed"
    xml.link "<--YOUR URL-->"
    xml.description "This is my property feed"

    property_hash.each do | key, property |
      pub_date = (Time.now - property.dom.to_i*60*60*24)
      pub_date = pub_date.strftime("%a, %d %b %Y %I:%M:%S")

      xml.item do
        xml.title property.address
        xml.link "http://franklymls.com/#{property.mls}"
        xml.pubDate "#{pub_date} EST"
        xml.description "City: #{property.city}
          Address: #{property.address}
          Price: #{property.list_price}
          Bed: #{property.bed}
          Bath: #{property.bath}
          Sqft: #{property.sqft}
          Built: #{property.built}"
      end
    end
  end
end

To make sure I get routine updates, I run this Ruby code on my server every hour using a cron job and pipe its output to an RSS feed. I am subscribed to the generated RSS feed so now I know exactly when a new property becomes available in my area!

Overall scRUBYt! is very easy to use and for simple screen scraping tasks it should work fine. However, I have found that it can run into some problems when the HTML gets complex. In these cases I would recommend using Hpricot for fine-level scraping.

To view the source code for this entry and to view other screen scraping code that I have written check out my Github page.

If you'd like to look at another example of scRUBYt! in action, feel free to read the post I wrote for my company's blog by clicking here.

Helpful Tips from Pragmatic Thinking and Learning

Justin Spradlin — Mon, 02 Feb 2009 02:00:29 +0000

In order to stay up to speed in an industry that changes by the day developers must constantly be learning and acquiring new skills. As technology fads come and go it is important for developers to isolate and keep pace with the technologies that are in high demand. Because technologies change so rapidly it is important that we are able to learn as quickly and efficiently as possible. While there is certainly no shortage of information available on the latest technologies, making sense of this information, figuring out what is important, and putting this information to use can be a very challenging task. Therefore it is important that developers learn how to learn.

I recently finished the book Pragmatic Thinking and Learning: Refactor Your Wetware by Andy Hunt, co-author of the seminal book The Pragmatic Programmer. In this book Hunt discusses how people (specifically software developers) learn and offers tips on how we can set out to learn more effectively. Here are some useful tips I stumbled upon while reading Pragmatic Thinking and Learning:

SQ3R

It is unfortunate that the most common way that many people attempt to teach themselves new skills is by reading. A lot of people tend to learn better through experimentation and observation, but in many cases these are not practical first steps to take when starting out with a given subject. On the other hand, reading is an important skill and reading material is abundant so it’s important that we are able to get the most out of the things we read.

The main reason reading is not a great way to learn is because it is often viewed as a passive activity. As Hunt describes in his book however humans don’t do a great job of simply ingesting material. Instead, Hunt suggests that we turn reading into a more active endeavor and describes using a technique called SQ3R. SQ3R is a simple strategy that can be used to actively engage yourself in whatever it is you are reading. The SQ3R process is as follows:

Survey – Scan the chapter headings and summaries for an overview.
Question – Note any questions you have.
Read – Read in its entirety.
Recite – Summarize, take notes, and put in your own words.
Review – Reread, expand notes, and discuss with colleagues.

There is plenty of information available on SQ3R on the web and of course there are many other techniques that can be used as well. What is important however is that instead of passively trying to absorb what we read we are deliberate in the way we go about consuming information.

Mind Maps

Generating ideas, especially on structured topics can oftentimes be a challenge. Traditional brainstorming techniques such as creating outlines are a great start, but are often limited by their structure and by the restriction of simply using words to express ideas. An alternative approach which Hunt presents in this book is to create a Mind Map.

A Mind Map is a free form diagram that focuses on a central idea and illustrates the relationships among topics associated with that idea. The advantage of using a Mind Map is that you are not limited to simply using words. You can sketch pictures, draw lines between ideas, and use color to group similar thoughts.

As an example, I decided to create a Mind Map for a presentation I will be giving in a few weeks on VirtualBox. To be fair this might not have been the greatest example of the power of using a Mind Map because I have already done quite a bit of research on the topic, but it was a good place for me to get started with Mind Maps and definitely helped me solidify some ideas for my presentation. Here’s what I came up with (click to enlarge):

One thing I found interesting while creating this Mind Map was that I actually needed to create multiple drafts. My first draft of this very same image was pretty scatter-brained. But that is the point of creating a Mind Map. It helps you take the ideas from your head and organize them into a tangible format that can then be translated into actual work (i.e. a presentation, an essay, code, etc.).

Harvesting Techniques

Have you ever struggled with a tough coding problem? Do you think as hard as you can about the problem, but are still unable come up with the answer? Then has the solution ever popped in your head upon waking the next day? Did the answer you were looking for suddenly appear during your daily commute when you were least expecting it? This happens all the time and the reason it occurs is because in many case we already know the answers to our problems.

Our brains have an amazing capacity for storing and retrieving all sorts of relevant (and irrelevant) information. As an oversimplification we can think of our brains as being split into two distinct hemispheres. The left hemisphere (L-mode) is analytic, logical, temporal, linear, and rational. It is responsible for processing words and symbols. The right hemisphere (R-mode) is more spatial and is good at processing patterns and relationships. The right side of our brain is intuitive and makes decisions based on hunches, feelings, and images.

As developers we tend to heavily favor using the left side of our brain to solve problems in our daily work, but in order to learn and work as efficiently as possible we must use both sides of our brain. It is very difficult to force R-mode thinking, but you can help coax R-mode ideas out of your brain by using what Hunt calls harvesting techniques. Harvesting techniques are ways to relax your brain’s L-mode processing in order to allow your R-mode thoughts to surface. Here are a few examples of harvesting techniques:

Free Form Journaling – Writing is a great way to relax your mind and allow your R-mode thoughts and ideas to escape your brain and present themselves onto paper. When ideas pop into your head, write them down, and then elaborate on those ideas. Simple brainstorming on paper can give you the opportunity to clarify your thoughts.

Walking – Sometimes the best way to come up with ideas is to simply step away from your desk, relax your mind, and go for a walk. While you’re walking though, try not to think about anything, especially the problem you are trying to solve. The goal is to silence your L-mode and give your R-mode the chance to do some work.

Image Streaming – This is the process of deliberately observing images in your mind and paying close attention to them. First, pose a problem to yourself or ask yourself a question. Then shut your eyes and relax. As images start to cross your mind describe them out loud. Try to describe as many details as you can using all five senses. This type of thinking can help you discover fresh insights to the problem or question you presented yourself.

At first some of these ideas may seem a bit silly, but keep and open mind and give them a chance. You may be pleasantly surprised with the results.

Conclusion

Pragmatic Thinking and Learning is very accessible and easy to read. Hunt writes the book using simple terms and concepts and illustrates his points with many examples. I’ve really only begun to scratch the surface of the material covered with this blog entry. Hunt goes into much further detail in his book and offers many additional techniques to help developers discover how they can best approach learning a new subject.

The book describes many different aspects of human physiology and psychology to explain why certain techniques work and others do not. While Hunt does not claim to be an expert in either of these fields he cites a multitude of references and clearly shows that he has put in the thorough research required to write such a book. Overall this book was a pleasure to read and I would certainly recommend it to anyone interested in improving their ability to learn.

5 Reasons VirtualBox Rocks My Socks

Justin Spradlin — Mon, 12 Jan 2009 01:30:37 +0000

Lately I’ve been interested in learning about a variety of the Operating Systems that are available today. I’m a big advocate of using the right tool for the job so experimenting with the latest OSes is a must in order to figure out the strengths and weaknesses of each. While experimentation is a great way to learn it can be a bit of a pain to find an old machine, format the hard drive, and install an OS. Dual booting is an alternative option, but there is the inherent risk of messing up the current state of your machine.

Overcoming these issues was always a bit of a challenge for me until about six months ago when I stumbled upon VirtualBox. VirtualBox is Sun’s “family of powerful x86 virtualization products.” VirtualBox allows you to install and run “guest” operating systems in their own virtual environment on top of a “host” operating system. I know that virtualization has been around for a while, but until finding VirtualBox, I’ve never had much success running virtualization software. VirtualBox is different. It’s intuitive and easy to use. Here are the top 5 reasons why I love VirtualBox:

Cheap Machines for Easy Experimentation
No More Dual Booting
Networking Works
Virtual Machine Cloning
Guest Additions Make for Seamless Work

1. Cheap Machines for Easy Experimentation

If you’ve ever wanted to try out a particular OS, but didn’t feel like dealing with the hassles mentioned above then VirtualBox may be exactly what you are looking for. VirtualBox allows you to install guest operating systems in their own virtual environment which means there are no side effects on the current state of your machine (other than taking up some hard drive space of course). If you mess something up simply discard your virtual machine (VM) and start over.

The net effect is that you save a lot of time. No need for additional machines, formatting, or partitioning. Simply create a new VM, mount the CD drive (or an .iso image of the OS you want to install), and fire up the virtual machine.

VirtualBox allows you to install a wide range of OSes including, but not limited to Linux (Ubuntu, etc.), BSD, Solaris, Windows (including 3.1). For a full list of guest operating systems click here.

2. No More Dual Booting

When I first started dual booting years ago I appreciated the fact that it was possible to run two (or more) operating systems on the same machine. This proved to be very useful when I wanted to play video games, write, or work with multimedia and graphics tools using Windows, but wanted to develop software using the tools available under various flavors of Linux. There was a problem however. Sometimes I would want to do work that required tools available in one of the OSes but not the other. Other times I would simply prefer to work in a way that favored a particular operating system. In either case I would often have to reboot my machine, select a particular OS, do my work in that OS, and then reboot back to my main operating system. As you can imagine this was quite painful.

With VirtualBox you can have the best of all worlds. You can work with as many OSes as you want at the same time. I do this a lot particularly when developing websites. I can run a full LAMP stack on an Ubuntu Linux guest operating system while I use photo editing software on my Windows host machine. As a bonus you can place the guest OS into “Full Screen” mode and be totally ignorant of the fact that you are running a virtualized operating system. Best of all: No reboots required.

VirtualBox running inside of Windows.

VirtualBox running in full screen mode.

3. Networking Works

One of the most exciting aspects about VirtualBox for me is that networking works between guest and host operating systems, guest and other guest operating systems, and guest operating systems and external networks (i.e. the Internet). This has a lot of interesting implications. One thing that I’ve been able to do is set up an old Dell computer in my closet running two VirtualBox VMs. I have Ubuntu Linux installed on both VM instances. I use one VM to serve as a centralized Git source code repository (you are using Git, right?). The other VM hosts a LAMP stack for serving a simple Wiki. On my network these two VMs act as individual machines. I can restart, shutdown, or in any other way modify one VM without disrupting the other.

With previous versions of VirtualBox networking could be slightly cumbersome to set up, but version 2.1 makes setup a breeze. If you do try VirtualBox and get caught up along the way there is plenty of documentation and a strong development community to help you out along the way.

VirtualBox networking allows for communication between guests, hosts, and outside networks.

4. Virtual Machine Cloning

I once spent a lot of time setting up one of my virtual machine guest environments to the exact specifications that I needed. I downloaded tons of software, added bash aliases, and tweaked various configurations to suit my needs. Then an interesting thing happened. I ended up needing another VM with the exact same specifications. Taking the time to manually configure another VM wasn’t really an option. Fortunately VirtualBox has built in VM cloning* support. All I had to do was run the following command:^**

VBoxManage clonehd   -format VDI

This saves a lot of time by reducing the effort needed to reproduce the guest virtual machines that you have previously set up using VirtualBox. This also allows you to share your environment with other people by simply giving them access to your .vdi file (VirtualBox VM instance) and configuration settings. In addition, once you set up a pristine environment you can clone it and always have it stored as a backup incase you mess something up with the VM you are currently working with.

^*There is a bit of a bug in VirtualBox 2.1. The clonehd command seems to be generating corrupt .vdi files. In the meantime as a work around you can simply copy the .vdi file, paste it into a target directory, and then run VBoxManage internalcommands sethduuid command (thanks to The Bonobo Journal for the tip).

^**Note: On Windows don’t forget to make sure you add the VBoxManage application to your path so that you can run it from the command line.

5. Guest Additions Make for Seamless Work

If you decide to install VirtualBox you will definitely want to install the “Guest Additions” package that is available for most guest operating systems. Guest Additions allow for better interoperability between your installed guest VMs and your host system. A few of the best features included with Guest Additions are:

Shared Clipboard – gives you the ability to bi-directionally copy and paste text between guest and host operating systems.

Independent Mouse Control – permits you to move your mouse between guest and host systems. Without Guest Additions installed your mouse is “captured” by the guest virtual machine until the “Host Key” is pressed (usually the right ctrl key).

Better Video Support – allows you to resize your VM window while also dynamically resizing the screen resolution of your guest operating system.

With Guest Additions installed VirtualBox screens can dynamically be resized to any screen resolution.

Conclusion

Besides simply saving time, space, and energy there are a lot of real world applications for VirtualBox. I really like to setup multiple VMs on my host machine to serve as development and staging environments for the various web applications that I work on. Also I like to test my web applications using a combination of different operating systems and browsers (especially various Windows/Internet Explorer combinations). With VirtualBox this is all very straight forward. In the future I would like to use VirtualBox to experiment with running multiple VMs over a network to learn more about setting up clustered, load balanced environments.

There are other virtualization and emulation tools on the market (VMWare, Xen, Wine), but I find that VirtualBox has the right mix of performance and ease of use to suit all of my development needs.

Book Review: The Productive Programmer

Justin Spradlin — Fri, 28 Nov 2008 17:13:10 +0000

Author: Neal Ford
Rating: 3.5/5

As a developer I am constantly seeking ways in which I can improve my ability to quickly generate high quality code so I was excited to pick up Neal Ford’s latest book, The Productive Programmer. This book offers advice on how to accelerate the production and quality of code by exploring many tools and practices that developers can use on a daily basis. In the book, Ford introduces several productivity patterns that can be used immediately, but more importantly he defines a nomenclature that allows developers to construct additional productivity patterns on their own.

The Productive Programmer is broken up into two parts:

Part I. Mechanics

This section discusses tips, tricks, and tools that developers can use while physically engaged in the act of writing code. This section is laid out in a recipe style format and is the part of the book which I found to be the most useful.

Ford breaks Part 1 down into four chapters which he calls the “Principles of Programmer Productivity”. These principles are:

Acceleration

This chapter discusses tips developers can use to speed up their development efforts. Ford lists many tips on using launchers, clipboards, and keyboard shortcuts for a variety of popular development tools and operating systems.

Focus

Ford uses this chapter to introduce guidelines on how to limit distractions, context switching, and pesky officemates so that developers can stay in the zone and focus on what’s really important – their work!

Automation

Common tasks that are repeated over and over again can kill a developer’s productivity. This chapter lists strategies for automating such tasks to help save precious time throughout the workday.

Canocality (i.e. D.R.Y)

In this chapter Ford discusses the importance of version control, continuous build machines, and syncing development tools. This helps to ensure the entire development team is on the same playing field so that precious time is not lost in dealing with inconsistencies between individual developer environments.

Part II. Practice

In Part 2, Ford takes aim at practices that developers can use to improve the quality of their code. By using the techniques and philosophies discussed in this section, Ford illustrates how code can be broken down into small, readable pieces that are easy to maintain. Practices covered include Test Driven Development, Static Analysis, and Meta-Programming. Also discussed are a few development philosophies that can guide developers when making decisions about how they should go about writing their code. These philosophies include: YAGNI (You Ain’t Gonna Need It), SLAP (Single Level of Abstraction Principle), and my favorite – Questioning Authority, an excerpt:

This is a story I first heard from Dave Thomas during a keynote address he delivered called Angry Monkeys and Cargo Cults. I don’t know if it’s true (despite researching it quite a bit), but it doesn’t matter-it illustrates a point beautifully.

Back in the 1960s (when scientists were allowed to do all kinds of crazy things), behavioral scientists conducted an experiment where they placed five monkeys in a room with a stepladder and a bunch of bananas hanging from the ceiling. The monkeys quickly figured out that they could climb the ladder and eat the bananas, but every time the monkeys got near the stepladder, the scientists would douse the entire room in ice cold water. You can guess what that generated: angry monkeys. Soon, none of the monkeys would go near the ladder.

Then, the scientists replaced one of the monkeys with a new monkey, who had not been subject to the blasts of water. The first thing he did was make a beeline for the ladder, and all the other monkeys beat him up. He didn’t know why they were beating him up, but he quickly learned: don’t go near the ladder. Gradually, the scientists replaced the original monkeys with new monkeys until they had a group of monkeys who had never been doused with cold water, yet they would still attack any monkey that approached the ladder.

The point is that just because things have always been done a certain way does not mean that we should blindly accept the status quo in favor of doing things that might actually make more sense or improve our code.

Thoughts

The Productive Programmer was an enjoyable read. At 200 pages the book reads very quickly – perhaps a bit too quickly in fact. My one criticism of the book is that it would have helped to have some additional examples to solidify some of the concepts in the second part of the book (I would guess that is probably my one criticism of most books). I did pick up a lot of small tips however and have been using them in my day to day work. I’ve noticed some small speed and productivity improvements so in my opinion this book was totally worth the Amazon sticker price.

Getting it on with Git – Part 2

Justin Spradlin — Sun, 05 Oct 2008 21:20:09 +0000

In my previous post I introduced Git and discussed its distributed nature, speed, and powerful branching/merging capabilities. Today I'm going to continue the discussion on Git by writing about GitHub, Git in practice, and why Git may not be for everyone.

GitHub is Sexy

GitHub is an online Git repository hosting service that is freely available to anyone who wishes to share their Git managed projects. The service is very similar to SourceForge and Google Code Hosting, but because GitHub uses Git as its source control management system it offers a variety of advantages over other code hosting services.

To begin with, by using GitHub the barrier for working on and contributing to open source projects is significantly reduced. In traditional centralized models only certain developers are given commit rights to any given project repository. This creates a problem however for those developers who checkout the project and make changes. Those developers have no way of keeping track of their local changes and history unless they import the project into their own centralized repository. Doing this however means that the developer will lose the ability to easily synchronize their local project with the original project.

This problem goes away when using Git because all commits are local. You can checkout a project from GitHub, makes changes as you wish, and commit them to your local repository. This allows you to keep track of all of your changes and their history. If you want to synchronize with the original project's repository you simply need to issue a "pull" command to get the latest updates. As a bonus, if you come up with some feature that you think is particularly useful to the original project, you can ask the owner of that project to issue a "pull" request from your repository so your changes can be merged back into the master project.

Setting up your own remote Git repository is not that difficult, but GitHub adds a great web interface on top of Git that feels very much like a social networking site and serves as a platform for developers to share and contribute to each other's projects. A lot of big name open source projects (especially in the Ruby community) are hosted on GitHub including Rails, Merb, and Script.aculo.us so developers can feel confident that the guys at GitHub have their act together. GitHub also offers a variety of other features including private repositories (for a fee), a developer API, and integration with other services including Basecamp, Lighthouse, and Twitter.

Git in Practice

It's highly unlikely that a developer will be able to get their current project team to switch version control systems overnight, but with Git you don't have to. Git has powerful migration tools that can be used in conjunction with other version control systems, especially Subversion. Current Subversion users can use Git to manage the source code on their local machine while still staying in sync with their team's Subversion repository using a built in Git command called "git-svn". The git-svn command allows developers to "push" and "pull" commits to and from a centralized Subversion repository. This gives developers the power of local branching/merging capabilities using Git without forcing the technology on their team members. In fact, no one even needs to know that Git is being used.

Git, it's not for everyone

Git is a pretty incredible tool, but it is likely not going to be championed by all developers. To begin with, Git's learning curve can be a bit steep especially when compared with other version control tools. Developers who desire to give Git a shot should be very comfortable working with the command line because Git's visualization support leaves something to be desired.

Git was created by Linux kernel developer Linus Torvalds so it may not come as a huge surprise that Git support on Windows is a bit lackluster. I would guess that a majority of developers use Windows in some capacity so having a tool with weak Windows support is likely to hinder the adoption of that tool. Fortunately, there is a project called MSysgit which allows developers to use Git on their Windows machines. I've personally used this tool on my Windows XP laptop and find that it works pretty well.

Conclusion

Minor drawbacks aside, Git is a pretty amazing tool for source control management and will be the one that I use for my future projects. Its distributed nature, speed, and branching/merging capabilities make it a great tool for managing source code changes and revision history.

Getting it on with Git – Part 1

Justin Spradlin — Wed, 01 Oct 2008 02:40:56 +0000

When I first heard about Git a few months ago the first thing that popped in my mind was: "Why the hell would I ever want to learn a new version control system?" Subversion works perfectly fine, doesn't it? I resisted at first, but after a few weeks I came around and decided to take Git for a test drive. This is what I discovered:

Distributed Source Code Management

Git is a distributed source code management tool which means that there is no centralized source code repository. This is a fundamental shift in the way source control is handled by systems like Subversion, CVS, and Perforce and is probably the most challenging concept to grasp (it certainly was for me) when first learning about Git. Version control using a centralized model is useful for things such as sharing code among developers, providing a history of changes to the code, and backing up data in a single location. However, since Git does not use a centralized repository, a few obvious questions are raised about the Git's workflow model. Questions like:

Where are code commits stored?

All code commits are stored on each developer's local machine. Everyone has a copy of the entire repository and all of its history.

How do developers share code?

Developers share code with Git by executing commands called "push" and "pull". Issuing a push command will push changes made in your repository to another developer's repository. Conversely, issuing a pull command will pull changes from another developer's repository into your repository. This may seem strange at first, but it actually ends up providing developers with a lot of collaborative power. For example, instead of checking code changes into a central repository for everyone to see, two or more developers can share code with one another directly in a controlled fashion by pushing and pulling changes to and from each of their repositories respectively.

How are backups handled in Git?

If all of my code commits are stored on my local machine and my hard drive fails, wouldn't I be in a lot of trouble? Not with Git. If you are working by yourself, regularly backing up your repository using rsync, Mozy, Time Capsule or some other backup system will allow you to completely restore all of your work. This is because Git stores all code revisions and history within each repository.

If you are working with other developers you simply need to clone their copy of the repository. In this sense, Git is actually incredibly fault-tolerant in that it provides geographically redundant archives of a repository because each repository is essentially a mirror of the entire project.

Finally, you could use one of the many publicly available Git repositories including GitHub and repo.or.cz.

Git is Blazing Fast

Since all commits, branching, and merging operations are local, Git is fast. Real fast. Unless you are pushing or pulling code there is no network to worry about. Increased speed means developers can be more productive because they don't have to spend time waiting for costly repository operations to take place across a network.

Git Makes Branching and Merging Easy

In a centralized version control system branching and merging can often times be a source of frustration for developers. All branches must live in the centralized repository and are likely viewable by other developers on the team. If enough developers decide to create branches on the centralized server the repository can quickly become bloated.

Git takes a different approach. Branching and merging in Git occur locally. This dramatically increases the speed in which branching and merging can take place. In addition, a single developer can create and destroy as many branches as they desire without having to worry about polluting a centralized repository. Experimental features can quickly and easily be tested by creating a branch from the current working copy's "master" branch. If the code produced in the branch proves worthy, it can be merged back into the "master" branch. If the code from the branch ends up being useless, the branch can simply be deleted and no one ever needs to know that it existed.

Part 1 Wrap Up

Sunday I'll post Part 2 of my "Getting it on with Git" series where I'll discuss GitHub, Git in practice, and why Git may not be for everyone. Be sure to check back then!

Useful Links

Official Git Site

Git Cheat sheet

Google Tech Talk on Git

Linus Torvalds on Git

GitMagic Book

Groovy Domain Specific Language Tutorial

Justin Spradlin — Mon, 18 Aug 2008 00:42:05 +0000

Although it may seem like an advanced topic, writing a simple Domain Specific Language (DSL) in Groovy is actually pretty easy. Groovy's dynamic nature and metaprogramming capabilities give developers all the tools they need to quickly and easily write their own DSL.

Domain Specific Languages are typically small, simple languages with a highly expressive syntax and grammar. In computing, DSLs are used to provide domain experts (who may or may not have prior programming experience) with programming capabilities limited to their specific area of expertise.

There are two types of DSLs: external and internal. External DSLs allow you to define a new language using any grammar or syntax you wish. External DSLs must then be parsed and executed by the programming language of your choice. Internal DSLs also define a new language, but are limited to the constructs made available by the implementing language. Because of their dynamic features, languages like Groovy and Ruby make writing internal DSLs relatively easy.

A Stock Trading DSL

A few weeks ago I wrote about trading stocks using Google's Finance API. Sticking with that theme, I thought it would be cool to write a Domain Specific Language that could also be used to mimic stock trading activities.

I'm guessing that most Wall Street traders don't know much about programming, but I'm sure most traders could decipher the actions that would take place after executing the following code:

buy 500, "AAPL", 179.30
buy(250, "AAPL", 179.30) //parenthesis are optional
buy 500, "SUNW", 10.14
sell 350, "AAPL", 179.30

show_transactions "AAPL"

print_portfolio_value

The code above is actual Groovy code. It tells the application to buy 750 shares of Apple stock at $179.30 in two separate transactions, buy 500 shares of Sun Microsystems stock at $10.14, and then sell 350 shares of Apple stock at $179.30. After all the transactions take place the code shows the transaction history for the Apple trades and then prints the total value of the portfolio.

The Groovy code to drive the functionality behind this simple DSL looks like this:

portfolio = [:] //Holds all of the stock transactions

/**
 *A simple object to hold the data for each individual transaction
 */
class StockTransaction {
    def tickerSymbol
    def numberOfShares
    def sharePrice
    def type
    def transactionDate
}

def buy(numberOfShares, symbol, sharePrice){
    transaction('Buy', numberOfShares, symbol, sharePrice)
}

def sell(numberOfShares, symbol, sharePrice){
    transaction('Sell', numberOfShares, symbol, sharePrice)
}

def transaction(transactionType, numberOfShares, symbol, sharePrice){
    def transaction = new StockTransaction(tickerSymbol:symbol,
        numberOfShares:numberOfShares, sharePrice:sharePrice,
        type:transactionType, transactionDate:new Date())

    println "${transaction.type}ing ${transaction.numberOfShares} shares" +
        " of ${transaction.tickerSymbol} at ${transaction.sharePrice}"

    //If no transactions exist for a particular stock, check to see that the
    //transaction type is a 'Buy' and if so create a new list to hold
    //transactions for a particular stock.
    if(portfolio[transaction.tickerSymbol] == null){
        if(transactionType == 'Buy')
            portfolio[transaction.tickerSymbol] = [transaction]
        else
            println "You can't sell a stock you don't own."
    } else{
        portfolio[transaction.tickerSymbol] << transaction
    }
}

def show_transactions(tickerSymbol){
    portfolio[tickerSymbol].each { transaction ->
        println "${transaction.transactionDate}: ${transaction.type} " +
          "${transaction.numberOfShares} shares of ${tickerSymbol} at " +
          "${transaction.sharePrice}"
    }
}

def getPrint_portfolio_value(){
    def totalValue = 0

    //Add all of the 'Buy' transactions and subtract all of the 'Sell'
    //transactions from the total portfolio value.
    portfolio.each { tickerSymbol, transactionList ->
        transactionList.each { transaction ->
            if(transaction.type == 'Buy')
                totalValue +=
                    (transaction.numberOfShares * transaction.sharePrice)
            else
                totalValue -=
                    (transaction.numberOfShares * transaction.sharePrice)
        }
    }

    println "Your Total Portfolio Value is: ${totalValue}"
}

The only thing the code really does is define methods for the different types of stock trading activities. I use a HashMap called portfolio to hold each of the individual stock transaction's attributes stored in a simple StockTransaction object. Each time a buy or sell action takes place the portfolio HashMap is updated with the corresponding StockTransaction object.

The readability of the DSL code is greatly improved because Groovy allows for optional parenthesis on methods that take one or more parameters. Notice that the second call to purchase Apple stock uses the parenthesis to illustrate this point. It is important to keep things as simple as possible when creating a DSL. Programmers should try to avoid bogging the domain expert down by forcing him or her to use unnecessary constructs of a language such as parenthesis.

If you look closely at the code that defines the DSL you will notice that in order to make the parenthesis optional for calls to the print_portfolio_value method I had to get a little hacky and define the method as: getPrint_portfolio_value(). This is because Groovy thinks that print_portfolio_value is a call to a property. By adding the "get" in front of the method call, we can "trick" Groovy into calling the method as desired.

After executing the DSL code the output is as follows:

Buying 500 shares of AAPL at 179.30
Buying 250 shares of AAPL at 179.30
Buying 500 shares of SUNW at 10.14
Selling 350 shares of AAPL at 179.30
Sun Aug 17 19:15:05 EDT 2008: Buy 500 shares of AAPL at 179.30
Sun Aug 17 19:15:05 EDT 2008: Buy 250 shares of AAPL at 179.30
Sun Aug 17 19:15:05 EDT 2008: Sell 350 shares of AAPL at 179.30
Your Total Portfolio Value is: 76790.00

Fluency

In the context of trading stocks, the DSL we created above is pretty straightforward, but it does suffer a bit from a problem with fluency. It might be nice if we could make the DSL a bit more readable. For example, what if the DSL was changed to read like this:

buy 500.shares.of("AAPL").at(179.30)
buy 250.shares.of("AAPL").at(179.30)
buy 500.shares.of("SUNW").at(10.14)
sell 350.shares.of("AAPL").at(179.30)

show_transactions "AAPL"

print_portfolio_value

Using categories it is possible to allow for a fluent DSL like the one above. Take a look at the code behind this DSL:

/**
 * The class used for the category
 */
class StockHelper {
    static portfolio = [:]

    static buy(self, stockTransaction) {
        stockTransaction.type = 'Buy'
        transaction(stockTransaction)
    }

    static sell(self, stockTransaction) {
        stockTransaction.type = 'Sell'
        transaction(stockTransaction)
    }

    static transaction(stockTransaction){
        println "${stockTransaction.type}ing " +
        "${stockTransaction.numberOfShares}" +
        " shares of ${stockTransaction.tickerSymbol} at " +
        "${stockTransaction.sharePrice}"

        if(portfolio[stockTransaction.tickerSymbol] == null){
            if(stockTransaction.type == 'Buy')
                portfolio[stockTransaction.tickerSymbol] = [stockTransaction]
            else
                println "You can't sell a stock you don't own."
        } else{
            portfolio[stockTransaction.tickerSymbol] << stockTransaction
        }
    }

    static getShares(self) {
        def startTransaction = new StockTransaction(numberOfShares:self,
            transactionDate:new Date())
        return startTransaction
    }

    static of(self, tickerSymbol) {
        self.tickerSymbol = tickerSymbol
        return self
    }
    static at(self, sharePrice) {
        self.sharePrice = sharePrice
        return self
    }

    static show_transactions(self, tickerSymbol){
        portfolio[tickerSymbol].each { transaction ->
            println "${transaction.transactionDate}: ${transaction.type} " +
              "${transaction.numberOfShares} shares of ${tickerSymbol} at " +
              "${transaction.sharePrice}"
        }
    }

    static getPrint_portfolio_value(self){
        def totalValue = 0

        portfolio.each { tickerSymbol, transactionList ->

            transactionList.each { transaction ->
                if(transaction.type == 'Buy')
                    totalValue +=
                        (transaction.numberOfShares * transaction.sharePrice)
                else
                    totalValue -=
                        (transaction.numberOfShares * transaction.sharePrice)
            }
        }

        println "Your Total Portfolio Value is: ${totalValue}"
    }
}

By chaining methods calls together we can make our code very fluent. We build a StockTransaction object by calling 3 methods: getShares(), of(), and at(). Each of these calls is responsible for building a specific piece of the StockTransaction object and passing it along to the next call. The self keyword is used to reference the target instance (i.e. the StockTransaction object being created). After the StockTransaction is created, the buy or sell methods can be called to update the portfolio HashMap just as in the first example.

Even though parenthesis and the dot notation (for method access) must be used in order to execute this DSL code, I personally think this code is easier to read and understand as compared with the previous example.

Conclusion

The code in this blog entry is pretty brittle, but it illustrates how easily a DSL can be created using the Groovy programming language. In a real life scenario programmers must be diligent about the design and usage of the DSLs they create.

By creating an internal DSL domain experts with some programming knowledge will also be able to exploit other constructs found in the implementing language such as loops and conditional statements. For example, code could be written to do things such as:

if(currentStockPriceVariable < SOME_SHARE_PRICE_CONSTANT){
    buy 500.shares.of("AAPL").at(currentStockPriceVariable)
}

or perhaps something like this:

3.times {
    sell 500.shares.of("AAPL").at(179.30)
}

Over the past few years DSLs have gained popularity along with the rise of dynamic languages. For example, GANT is a popular DSL for the Groovy programming language that can be used to script Ant style builds. It will be interesting to see what other types of DSLs become available in the future. Be sure to look out for Martin Fowler's upcoming book on the subject.

Code

Click here to download the code examples from this post.

Disclaimer

I am in no way endorsing any of the companies mentioned in this blog entry and all company references are for example purposes only.

Ruby Nation: Day 2 Wrap Up

Justin Spradlin — Sun, 03 Aug 2008 22:49:25 +0000

Opening Keynote - Rich Kilmer

Rich Kilmer's opening keynote was by far the best presentation of the day. Rich is an incredibly experienced and skilled Ruby developer who was one of the earlier adopters of the language. During his keynote, Rich walked through his experiences with Ruby dating all the way back to 2001. He has certainly worked on some interesting projects for impressive government agencies including DARPA and the United State Air Force. Rich also talked about projects he has created for the Ruby community including FreeRIDE - a Ruby IDE and RubyForge - a code repository for Ruby projects.

One particularly interesting thing Rich mentioned was that Ruby is becoming a mainstream language. I personally find this both exciting and scary. It is exciting that the community is growing at that there will likely be more opportunities to work on Ruby projects professionally, but it is a bit worrisome that the community could start to be overpopulated with run-of-the-mill developers transitioning from other dying mainstream languages.

Ruby Puzzlers - Mike Furr

Mike Furr, a doctoral student at UMCP, gave a presentation on edge case scenarios that occur in the Ruby language. Since I'm pretty new to Ruby I struggled to follow along, but Mike did do a good job of explaining how Ruby code reacted under various situations. Even though it can be rare to deal with edge cases in a language, it is nice to know how to avoid them or work your way out of them if they do arise.

Practical JRuby - David Keener

A Java(ish) presentation at a Ruby conference? It may seem a bit strange at first, but it actually makes a lot of sense. Java as a platform has some pretty rock solid features including performance, garbage collection, and concurrency. A lot time and thought has also been invested into Java libraries and corporate infrastructure so in many ways it makes sense to take advantage of what is already available. David Keener hit on all of these points in his presentation and gave an example of how to integrate JRuby with the JFreeChart Java library.

Tools for Your Ruby Toolbox - Dave Bock

Dave Bock is another FGM employee that I had the pleasure of working with a few years back. Dave has since moved on and started his own shop called Codesherpas. In his presentation, Dave talked about 3 lightweight tools that can be used to quickly accomplish tasks in Ruby. Dave first talked about StaticMatic, a framework for developing simple static websites using templates, but without using all of the built in functionality of Rails. Another web framework Dave talked about was Sinatra. Again, this framework allows developers to create simple web applications without the full Rails stack, but it also includes some RESTful functionality. Finally, Dave talked about GServer, a generic server library that allows developers to easily create server applications of their own. During his presentation Dave demoed a quick "Knock Knock" client he created using the GServer library.

Lightning Talks

There were five lightening talks given yesterday, but again Bryan Liles stole the show with his presentation on Project Management. The basic takeaway: meet with your reports regularly, project confidence, and set up processes that are easy to follow. Sounds so simple doesn't it?

Closing Keynote: Bad Ruby - Stuart Halloway

Stuart Holloway closed RubyNation with a talk on Bad Ruby. Playing off of a point made earlier in Rich Kilmer's talk about Ruby becoming mainstream, Stuart noted that now was the time for Ruby developers to begin to establish best practices for the language so that common pitfalls can be avoided when mainstream developers start to migrate to Ruby. (Assuming of course "mainstream" developers actually migrate to Ruby).

Conference Wrap Up

Overall the conference was pretty great. Day two was a little less energetic than day one, but that may not be too surprising seeing as how it was on a Saturday. I was a little disappointed that Bruce Tate wasn't able to make the conference because I was really looking forward to hearing him speak, but I was pleasantly surprised at the quality of the other speakers. I know that a lot of people put a lot of hard work into the conference so I'd personally like to thank everyone responsible for putting on this years conference and I can't wait to attend again next year.

Ruby Nation: Day 1 Wrap Up

Justin Spradlin — Fri, 01 Aug 2008 15:59:26 +0000

Ruby experts and enthusiasts from around the country met in Washington, D.C. today for the first annual RubyNation conference. I always think it's awesome to get together with industry leaders to absorb and share new information and insights. I've been to quite a few tech conferences in the past, but I've never been as engaged and attentive as I was today. There is an energy around Ruby and Rails that I have yet to witness with other technologies. I continue to be amazed by how passionate and open the community is as well. I met and talked with more people today than all of the other conferences I've attended combined.

In case you weren't able to make it, here's a quick rundown of today's events:

Opening Keynote: Ceremony vs. Essence - Neal Ford

Neal Ford is a seasoned consultant and speaker that I've seen present on more than one occasion. I've always enjoyed his presentations, but his talk today really hit home with me. Neal's talk basically boiled down to the fact that we developers need to Keep It Simple (KISS) and avoid the "accidental complexity" that is often unnecessarily added to the problems we are trying to solve. I've listen rhetoric like this a 1,000 times, but I've never heard it articulated so elegantly. It was literally like he was looking inside of my head and expressing the way I (and likely a lot of developers) feel about the current state of the industry (especially if you are a Java programmer like me).

DSLs and Ruby: Blurring the Lines Between Programs And Data - Russ Olsen

I used to work with Russ at a FGM so I was really looking forward to hearing him speak. Russ has become quite a leader in the Ruby community especially after publishing his first book: Design Patterns in Ruby. Russ's talk centered around Domain Specific Languages (DSLs) and how relatively easy it is to implement one using the Ruby language. During his presentation Russ put together a simple Internal DSL for example purposes and also discussed use cases for when it is appropriate to create a DSL to solve a problem.

The Culture of Innovation in Ruby - Glenn Vanderburg

Innovation in testing frameworks in Ruby was the topic of Glenn Vanderburg's talk. Glenn discussed some of the interesting testings frameworks available in the Ruby community (Rspec, Mocha, Flex Mock, Shoulda) and compared them to some of the stagnant frameworks in the Java community (JUnit, TestNG). Glenn also pointed out how Ruby's testing frameworks tend to make it much easier to mock objects and create more thorough tests with much less effort than the equivalent tests in the Java world.

Living on the Edge - Yehuda Katz

Yehuda Katz of EngineYard talked about some of the new(er) technologies that are emerging in the Ruby community including Merb, DataMapper, Sake, Thor, YARDoc, and GitHub. DataMapper looks like an especially interesting alternative to the incumbent ORM ActiveRecord. It promises thread safety and increased performance which are obviously two great things. GitHub looks pretty cool too. I'm not quite ready to abandon Subversion just yet, but I will definitely be checking out Git in the near future.

Lightning Talks

Chris Bucchere, David Jones, Simon Kaczor, and Bryan Liles all gave lightening talks today on a variety of topics including social networking, state machines, command line libraries, and testing respectively. All of the talks presented useful information, but Bryan Liles was without a doubt the most entertaining speaker of the day. He made it through about 65 slides in 15 minutes and was still able to convey a very important message about how often developers should be testing their code. Check out his blog at: http://smartic.us/

Archaeopteryx: A Ruby MIDI Generator - Giles Bowkett

I actually wasn't able to attend Giles Bowkett's session because I had a previous engagement to attend, but I did get a chance to view some screencasts that show off his Ruby MIDI Generator. Pretty cool stuff.

Tomorrow

Tomorrow looks to be a pretty exciting day as well. I'm especially looking forward to talks from my former co-worker Dave Bock and author Stuart Holloway.

Putting Google Finance to REST with Ruby

Justin Spradlin — Wed, 23 Jul 2008 06:43:10 +0000

RubyNation is just a week and a half away and in order to prepare myself I thought I'd brush up on some Ruby and Rails in the days preceding the conference. I don't use Ruby on a regular basis, but I'm a huge fan of dynamic languages so I'm really looking forward to attending the conference and finding out what's new in the Ruby community.

By day I'm a Java developer, but I've also spent a fair amount of time studying Groovy. Learning Groovy has been my "training wheels" approach to understanding dynamic languages. Studying Groovy has helped me grasp new (new to me at least) concepts like Metaprogramming and Domain Specific Languages all from the comfort of my Java environment.

As a simple exercise I thought it would be interesting to rewrite the code from my last blog entry in Ruby to see how it compares to the Groovy code I originally wrote. So without further ado let's check out the code:

Authenticating with Google

In order to use the Google Finance API we need to first authenticate with Google by sending our Google Account username and password to Google's client login URL: https://www.google.com/accounts/ClientLogin.

def create_auth_token(username, password)
  https = Net::HTTP.new('www.google.com', 443)
  https.use_ssl = true
  path = '/accounts/ClientLogin'

  query_string = "Email=#{username}&Passwd=#{password}
                 &service=finance&source=company-groovyfinance-1.0"

  https.verify_mode = OpenSSL::SSL::VERIFY_NONE # don't display warnings

  response = https.post(path, query_string)

  if response.code == '200'
    return response.body[/Auth=(.*)/, 1] # The authorization token
  end

  return "Error"
end

In the code above we create an http object and give it the correct URI and path to Google's login service. We then send in a query string with our username, password, service, and source (application name). If everything succeeds Google returns an Authentication token which we will set in the headers in subsequent calls to Google's service.

This code isn't much different from the Groovy code in my previous entry, but I do like the way Ruby allows POST calls directly on the http object. This obsoletes the processRequest() method found in my Groovy code.

Creating a Portfolio

We can create a new portfolio in our Google Finance account by executing calls to the following method:

def create_portfolio(authorization_token, portfolio_name)
  headers={}
  headers["Authorization"] = "GoogleLogin auth=#{authorization_token}"
  headers["Content-Type"] = "application/atom+xml"

  http = Net::HTTP.new('finance.google.com')
  path = "/finance/feeds/default/portfolios"

  atom_string = "
  
         <strong>#{portfolio_name}</strong>
         
  "

  response = http.post(path, atom_string, headers)

  if response.code == '201'
    doc = REXML::Document.new(response.body)
    return doc.root.elements["id"].text[/portfolios\/(.*)/, 1] #portfolio id
  end

  return "Error"
end

This code is almost identical to the Groovy code I wrote last time but I do really like the way Ruby allows you to place header attributes into a Hash and send them directly with a call to http.post().

Placing an Order

Just like the example above, placing an order (stock trade) means that we need to create an Atom feed with our transaction attributes (transaction type, number of shares, etc.) and send them to a Google resource end point. This end point in our case is the URL of the specific portfolio in which we wish to place our new asset.

def create_stock_transaction(authorization_token, ticker_symbol,
                             number_of_shares, transaction_type, portfolio_id)

  headers={}
  headers["Authorization"] = "GoogleLogin auth=#{authorization_token}"
  headers["Content-Type"] = "application/atom+xml"

  http = Net::HTTP.new('finance.google.com')
  path = "/finance/feeds/default/portfolios/#{portfolio_id}/positions/" +
         "#{CGI::escape(ticker_symbol)}/transactions"

  transaction_time = Time.now.strftime("%Y-%m-%dT%H:%M:%S.000")

  atom_string = "
  
         
  "

  response = http.post(path, atom_string, headers)

  if response.code == '201'
    return response.body
  end

  return "Error"
end

Once again the Ruby code looks nearly identical to the Groovy code. We set the correct headers, generate a well formed Atom feed, and the post our data to Google. If all goes well, the transaction will be made and an Atom representation of the transaction will be returned to the client.

Groovy vs. Ruby Thoughts

There are probably "Rubyer" ways to write the code samples listed above, but I did find it interesting that the code I wrote in Ruby was so similar to the code that I wrote in Groovy. I'm not sure whether or not the similarities are due to the subjective nature of having the same programmer write both samples or if it's due to the fact that the languages are so similar to begin with.

Either way, exploring a new language (albeit a similar language) did give me some additional perspective. I found myself bouncing back and forth between the Groovy and Ruby code comparing how similar tasks were performed in each language. This type of information is always useful when trying to determine which technology is the right tool to use for a specific job.

In addition, studying Ruby helped me write better Groovy code. In the create_auth_token() method in the Ruby sample a regular expression is used to parse the authentication token from the returned string. I was able to take this regular expression code and apply it to the same method (createAuthToken()) in my Groovy code to improve its readability.

References

Download the Code

Google Data API Developers Guide
Google Finance Developers Guide
Google Finance Reference

Using Ruby with the Google Data APIs
REXML Guide

Disclaimer

Again... I am in no way endorsing any of the companies mentioned in this blog entry and all company references are for example purposes only.

Putting Google Finance to REST with Groovy

Justin Spradlin — Thu, 17 Jul 2008 03:07:13 +0000

For a while now I've been interested in learning more about building and consuming REST based web services. Fortunately, many tech giants including Google and Yahoo expose much of their data and functionality through REST based APIs. These powerful APIs, combined with Groovy's concise, readable syntax make it very easy to learn about the REST software architecture approach.

This particular blog entry makes use of Google's Finance Data API. I'll explain how it is possible to programmatically authenticate with Google, create a new stock portfolio, and create positions (buy and sell stocks) within this portfolio by making REST based service calls. I won't dive too deep into the details of the REST architecture, but if you have a general understanding of XML and the HTTP protocol it won't be too difficult to follow along. For more information on REST, please see the links in the Reference section below.

Note: If you want to try the code samples on your own you will need to have a Google Account. To sign up for a free account click here.

Authenticating with Google

In order to use Google's service we must first authenticate by POSTing a query string containing our username, password, service identifier, and source (application name) to the following URL: https://www.google.com/accounts/ClientLogin. This code snippet illustrates how a POST request can be made in Groovy:

static createAuthToken(String username, String password){
    def url = new URL("https://www.google.com/accounts/ClientLogin")
    def connection = url.openConnection()

    def queryString = "Email=${username}&Passwd=${password}" +
                      "&service=finance&source=company-groovyfinance-1.0"

    def returnMessage = processRequest(connection, queryString)

    if(returnMessage != "Error"){
        //the authentication token
        return returnMessage.split(/Auth=/)[1].trim()
    }
}

static String processRequest(connection, dataString){
    connection.setRequestMethod("POST")
    connection.doOutput = true
    Writer writer = new OutputStreamWriter(connection.outputStream)
    writer.write(dataString)
    writer.flush()
    writer.close()
    connection.connect()

    if (connection.responseCode == 200 || connection.responseCode == 201)
        return connection.content.text

    return "Error"
}

In the code above we create a connection to Google's authentication URL and POST the queryString by calling the reusable processRequest() method. Within processRequest() we set the connection's request method type to "POST", create a Writer, and write the data to the supplied connection.

Note: The code example for this blog post can be downloaded in its entirety by clicking the "Download the Code" link in the Reference section below.

If our credentials are valid, Google will respond to our request with a string of name value pairs similar to this:

SID=DQAAAIEAAAIgccs7qKgmwXGagt...lu6fg
LSID=DQAAAIIAAACsegj9oEm0Ob8pz...abyi5
Auth=DQAAAIMAAACgadj9oEm0Ob8pz...AGasG

The only piece of information we are interested in is the authorization token value that follows the "Auth" label. We will need to use this token in the rest of our requests in order to authenticate with Google.

Creating a Portfolio

When users log into Google Finance for the first time they will be presented by default with an empty portfolio called "My Portfolio". Note: a portfolio is simply a collection of assets (stocks, bonds, mutual funds, cash, etc.).

Let's pretend however that we do not wish to use the default portfolio, but instead want to create our own. To create a new portfolio we simply need to send a well formed Atom XML feed that contains the name of the new portfolio (and optionally a currency code) to this URL: http://finance.google.com/finance/feeds/default/portfolios

static createPortfolio(String authorizationToken, String portfolioName){
    def url =
    	 new URL("http://finance.google.com/finance/feeds/default/portfolios")

    def connection = url.openConnection()
    connection.setRequestProperty("Content-Type", "application/atom+xml")
    connection.setRequestProperty("Authorization",
                                  "GoogleLogin auth=${authorizationToken}")

    def atomString = """
    
        <strong>${portfolioName}</strong>
        
    """

    def returnMessage = processRequest(connection, atomString)

    //Get the id of the newly created portfolio
    if(returnMessage != "Error"){
    	def entry = new XmlSlurper().parseText(returnMessage)
    	def entryId = "${entry.id}"

    	return entryId[entryId.lastIndexOf('/')+1..-1]
    }
}

Once again we create a connection, but this time we also need to set two request header parameters: Content-Type and Authorization. We set these parameters to "application/atom+xml" and "GoogleLogin auth=${authorizationToken}" respectively. Note that the authorizationToken variable stores the authentication token that was returned by the createAuthToken() method.

After the connection is created we again call the processRequest() method that was discussed in the Authenticating with Google section above. If all goes well, Google will create the new portfolio and return an Atom representation of that new portfolio to the client. If we are logged into our web based Google Finance account we can refresh the screen and will notice our newly created portfolio.

Two code features really shine through in the example above: Groovy's Triple Quotes and Groovy's XmlSlurper. The triple quotes surrounding the Atom XML allow us to easily write multi-line XML strings without having to worry about escaping special characters. The XmlSluper meanwhile allows us to easily parse and locate a specific attribute (the portfolio id in this case) within the returned XML string. Imagine how many lines of Java code it would take to recreate this functionality.

Placing an Order

So far our portfolio is pretty boring. Let's create some stock transactions to add some assets into our portfolio.

static createStockTransaction(authorizationToken, tickerSymbol, numberOfShares,
                              transactionType, portfolioId){

    StringBuffer transactionURL = new StringBuffer()
    transactionURL.append("http://finance.google.com/finance/feeds/")
        .append("default/portfolios/${portfolioId}/positions/")
        .append("${URLEncoder.encode(tickerSymbol)}/transactions")

    def url = new URL(transactionURL.toString())
    def connection = url.openConnection()

    connection.setRequestProperty("Content-Type", "application/atom+xml")
    connection.setRequestProperty("Authorization",
                                  "GoogleLogin auth=${authorizationToken}")

    String timeFormat = "yyyy-MM-dd'T'HH:mm:ss.S"
    SimpleDateFormat dateFormatter = new SimpleDateFormat(timeFormat)
    String transactionDate = dateFormatter.format(new Date())

    def atomString = """
    
    	  '${numberOfShares}' type='${transactionType}' />
    """

    processRequest(connection, atomString)
}

The concept with this code is basically the same as the portfolio creation example above. We create a connection with the correct Content-Type and Authorization headers, define some Atom XML with attributes about the transaction we would like to make, and then POST the XML to a specific URL. The interesting thing to note about this code is that unlike the other two examples, the URL for creating transactions is not static. Instead, this URL needs to be parameterized with the portfolio id and ticker symbol of the stock you wish to trade: http://finance.google.com/finance/feeds/default/portfolios// positions//transactions.

Method calls to createStockTransaction() might look something like this:

createStockTransaction(authorizationToken, "NASDAQ:JAVA", 10000.00, "Buy", pId)
sleep(2000)
createStockTransaction(authorizationToken, "NYSE:F", 10000.00, "Buy", pId)

This would place an order for 10,000 shares of Sun Microsystems and Ford Motor Company stock. Notice that the ticker symbol must be preceded by the name of the market on which it trades. Therefore the ticker symbol we sent to Google to purchase Sun stock is: NASDAQ:JAVA because Sun trades on the NASDAQ stock exchange. Stocks that trade on the New York Stock Exchange would be preceded by the acronym: NYSE (i.e. Ford = NYSE:F). Also notice that it is a good idea to "sleep" between service calls because sending too many requests to Google at once will result in errors.

When we refresh our Google Finance page we will notice these shares placed into our portfolio.

Conclusion

This blog post only skims the surface of what is possible using Google's Finance Data API. The API also allows for RESTful calls to GET, UPDATE and DELETE specific portfolio and transaction feeds. Overall, I've been incredibly impressed by all of the data and functionality made available through Google's Data APIs.

The code provided in this example should give you a solid foundation to start playing around with Google's Finance API, but it is far from bulletproof. For example, I don't really do much in the way of error handling or input validation. So you shouldn't be surprised if things bomb out if you try to sell shares of a stock that you don't own or create a portfolio with the same name as one of your existing portfolios.

The examples I've shown here are very low level because I wanted to learn the basics of REST. A much easier way to implement the same functionality would be to use Google's Data APIs Client Libraries (which by the way work perfectly well with Groovy).

References

Download the Code

Google Data API Developers Guide
Google Finance Developers Guide
Google Finance Reference

Building Web Services the REST Way
REST Wikipedia Entry

Disclaimer

I am in no way endorsing any of the companies mentioned in this blog entry and all company references are for example purposes only.

Groovy Metaprogramming – Creating Behavior on the Fly

Justin Spradlin — Tue, 08 Jul 2008 04:07:37 +0000

In my previous post I talked about dynamically adding behavior to Groovy classes using either the ExpandoMetaClass or Categories. These techniques are especially useful if you know which methods you would like to add to your classes prior to actually writing any code. But what if you don't know which methods you will need before code writing time? What if you want to allow yourself the flexibility to call methods arbitrarily without defining their implementation beforehand?

To the average Java programmer (myself included), the idea of generating methods on the fly might seem bizarre at first, but Groovy's built in method interception capabilities allow programmers to easily realize this type of functionality in their Groovy code. In Groovy Metaprogramming - Adding Behavior Dynamically, I discussed how to add a method to the java.math.BigDecimal class to allow for the conversion of U.S. Dollars (USD) into Euros (EUR). What if, however, I wanted to convert from USD to British Pounds (GBP) or to Japanese Yen (JPY)? I could certainly write a method for each of these conversions, but what if there was a better, cleaner, and more flexible way to build in this functionality?

Groovy's `methodMissing()`

The magic behind Groovy's ability to create methods on the fly comes from the language's built in facilities for intercepting method calls. In particular, one practice for intercepting calls in Groovy is to implement the methodMissing() method on your Groovy classes. Before Groovy throws a MissingMethodException for calls that are made to methods not defined within a class, Groovy first routes the calls through an object's methodMissing() method. This gives programmers a chance to intercept calls to these non-existing methods and define an implementation for them:

import java.text.NumberFormat

def exchangeRates = ['GBP':0.501882, 'EUR':0.630159,
                     'CAD':1.0127, 'JPY':105.87] // (7/2/2008)

BigDecimal.metaClass.methodMissing = { String methodName, args ->
    conversionType = methodName[2..-1]
    conversionRate = exchangeRates[conversionType]

    if(conversionRate){
        NumberFormat nf = NumberFormat.getCurrencyInstance(Locale.US)
        nf.setCurrency(Currency.getInstance(conversionType))

        return nf.format(delegate * conversionRate)
    }

    "No conversion for USD to ${conversionType}"
}

println 2500.00.inGBP()
println 2500.00.inJPY()
println 2500.00.inXYZ()

Notice that in the example above we make calls to the methods: inGBP(), inJPY(), and inXYZ() on the BigDecimal object 2500.00. Also notice however, that we did not actually define any of these methods on the BigDecimal class. Instead we override BigDecimal's methodMissing() method which allows us to intercept calls to these methods (inGBP(), inJPY(), etc.) and create an implementation for them.

Within the methodMissing() method we parse the name of the called method and look up the corresponding conversion rate from the exchangeRate HashMap which stores conversion rate values for various countries. In the future it might be nice if this code called a remote service to actually lookup the exchange rates, but for these examples I'll just keep it simple. If no corresponding conversion rate is found then a message is returned stating that there is "No conversion for USD to " (XYZ in our case).

GBP1,254.70
JPY264,675.00
No conversion for USD to XYZ

Method Caching

One way to improve the performance of the previous example is to cache the implementation of the method calls that we have intercepted. This will allow future calls to these methods to be invoked directly and prevent them from having to be routed through the BigDecimal class's methodMissing() method:

import java.text.NumberFormat

def exchangeRates = ['GBP':0.501882, 'EUR':0.630159,
                     'CAD':1.0127, 'JPY':105.87] // (7/2/2008)

BigDecimal.metaClass.methodMissing = { String methodName, args ->
    println "method missing called"

    def cachedMethod = { Object[] cmArgs ->
        conversionType = methodName[2..-1]
        conversionRate = exchangeRates[conversionType]

        if(conversionRate){
            NumberFormat nf = NumberFormat.getCurrencyInstance(Locale.US)
            nf.setCurrency(Currency.getInstance(conversionType))

            return nf.format(delegate * conversionRate)
        }

        "No conversion for USD to ${conversionType}"
    }

    BigDecimal.metaClass."${methodName}" = cachedMethod

    return cachedMethod(args)
}

println 2500.00.inJPY()
println 2500.00.inGBP()
println 2500.00.inGBP()

In the example above notice that the functionality for the called method is implemented and stored within a closure called cachedMethod. By storing the functionality within a closure we can then assign it to BigDecimal's metaClass so that subsequent method calls are invoked directly.

In the result we can see that the second call to inGBP() does not get routed through BigDecimal's methodMissing():

method missing called
JPY264,675.00
method missing called
GBP1,254.70
GBP1,254.70

Conclusion

It's not too hard to think of ways that the given code samples could be improved. Imagine that instead of assuming all BigDecimal objects were defined in USD, that they could represent any currency type and that our future method calls could actually look something like this:

2500.00.fromEURtoCAD()

The examples given in this blog entry are trivial, but they showcase the power and flexibility that can be gained by using Groovy's metaprogramming capabilities. These metaprogramming techniques give programmers a powerful tool that can be used within their code, but with this power programmers need to show a great deal of caution. Imagine if the following method was called on the BigDecimal object in our examples above:

2500.00.someRandomMethod()

I'm guessing the result would not be desirable. Programmers should always be sure to code defensively and write unit tests when using Groovy's powerful metaprogramming capabilities.

Groovy Metaprogramming – Adding Behavior Dynamically

Justin Spradlin — Mon, 30 Jun 2008 02:00:09 +0000

I've been fascinated with languages like Ruby and Groovy ever since being exposed to their dynamic capabilities. I remember attending a No Fluff Just Stuff conference a few years ago and being awestruck while watching Dave Thomas build a full-blown Rails application within a matter of minutes. Dynamic languages give programmers the ability to write powerful and flexible code that is both readable and concise. But what is it that gives these languages such great power? The secret is their metaprogramming capabilities.

Metaprogramming is the ability of a computer language to manipulate other programs (including itself) to add or create functionality in a dynamic fashion. In Groovy, metaprogramming is especially useful because among other things it gives programmers the ability to easily implement domain specific languages, create builders, or generate mock objects for unit testing. Adding functionality to pre-existing Java or Groovy code can be accomplished by using either the ExpandoMetaClass or Categories.

Note: the best way to test the code examples from this blog entry yourself is to simply copy and paste them into a text file saved with a .groovy extension and then run them from the command line. Running these examples in the GroovyConsole may yield unpredictable results.

The `ExpandoMetaClass`

The ExpandoMetaClass can be used to dynamically add methods, properties, and constructors to Groovy or Java objects. These tasks can be accomplished by simply assigning a closure to an object's MetaClass. For instance, let's pretend that we want to add a method to java.math.BigDecimal that will convert a BigDecimal (in U.S. Dollars) to its equivalent value in Euros.

import java.text.NumberFormat

BigDecimal.metaClass.inEuros = {
    def EXCHANGE_RATE = 0.634961 // (6/27/2008)
    NumberFormat nf = NumberFormat.getCurrencyInstance(Locale.US)
    nf.setCurrency(Currency.getInstance("EUR"))
    nf.format(delegate * EXCHANGE_RATE)
}

println 2500.00.inEuros()

To access BigDecimal's MetaClass we simply use the following syntax BigDecimal.metaClass.inEuros and assign it a closure. Within the closure we set an exchange rate, create a NumberFormat instance, set the currency type to "EUR" (Euros), and multiply the delegate (the BigDecimal we specified) by the exchange rate. Groovy automatically returns the last line of a closure so our call to println 2500.00.inEuros() will yield the following result:

EUR1,587.40

It is also possible to make the code a bit more readable by dropping the parenthesis after the call to inEuros(). This can be accomplished by setting up inEuros as a property on the BigDecimal class instead of as a method:

import java.text.NumberFormat

BigDecimal.metaClass.getInEuros = {->
    def EXCHANGE_RATE = 0.634961 //June 27, 2008
    NumberFormat nf = NumberFormat.getCurrencyInstance(Locale.US)
    nf.setCurrency(Currency.getInstance("EUR"))
    nf.format(delegate * EXCHANGE_RATE)
}

println 2500.00.inEuros

Notice that instead of writing inEuros we create a property on the MetaClass by writing getInEuros. This allows us to drop the parenthesis from println 2500.00.inEuros and yields the same result as the method injection technique described above:

EUR1,587.40

For detailed information on the ExpandoMetaClass and all of its uses, be sure to check out the Groovy User Guide.

Conclusion

As you can see, using metaprogramming gives static language programmers a whole new set of tools that can be used to creatively and elegantly arrive at solutions that can otherwise be overly complex in static languages. In my next blog entry, I'll expand upon this current discussion and explain how we can use metaprogramming to not only add functionality to a class, but to actually create it on the fly with no previous knowledge of the methods that will be called on our class.

Easy AOP with GroovyInterceptable

Justin Spradlin — Wed, 18 Jun 2008 18:56:54 +0000

Aspect Oriented Programming (AOP) is a technique that can be used to eliminate the repetition of cross-cutting concerns (logging, security, transaction management, etc.) in code. AOP is useful because it provides programmers with a single point at which code can be modified and take effect across an entire system, but integrating AOP frameworks into traditional Java applications tends to be somewhat cumbersome and complex. Conversely, adding AOP-like features into Groovy code is as simple as implementing the GroovyInterceptable interface.

Any Groovy object that implements this interface will automatically have all of its method calls routed through its invokeMethod() (assuming of course that this method has been implemented on the Groovy object itself or on its MetaClass).

A Simple Example

Let's take a look at a quick example. Assume that for whatever reason we want to know the time before and after each call to a method in a class called SimplePOGO. To accomplish this task in Groovy, we simply implement the GroovyInterceptable interface and the invokeMethod() like this:

class SimplePOGO implements GroovyInterceptable {
    void simpleMethod1(){
        System.out.println("simpleMethod1() called")
    }

    void simpleMethod2(String param1, Integer param2){
        System.out.println("simpleMethod2(${param1},${param2}) called")
        System.out.println("sleeping...")
        Timer.sleep(2000)
    }

    def invokeMethod(String name, args){
        System.out.println("time before ${name} called: ${new Date()}")

        //Get the method that was originally called.
        def calledMethod = SimplePOGO.metaClass.getMetaMethod(name, args)

        //The "?" operator first checks to see that the "calledMethod" is not
	//null (i.e. it exists).
        calledMethod?.invoke(this, args)

        System.out.println("time after ${name} called: ${new Date()}\n")
    }
}

simplePogo = new SimplePOGO()
simplePogo.simpleMethod1()
simplePogo.simpleMethod2("stringParam", 24)

This code will produce the following output:

time before simpleMethod1 called: Wed Jun 18 12:07:06 EDT 2008
simpleMethod1() called
time after simpleMethod1 called: Wed Jun 18 12:07:06 EDT 2008

time before simpleMethod2 called: Wed Jun 18 12:07:06 EDT 2008
simpleMethod2(stringParam,24) called
sleeping...

time after simpleMethod2 called: Wed Jun 18 12:07:08 EDT 2008

Because SimplePOGO implements GroovyInterceptable all calls to its methods will first be routed through its invokeMethod(). Within the invokeMethod() we print the current time, look up the actual method that was called, call that method, and then print out the current time after execution.

An Example with Nested Methods

What happens if we nest methods? Take a look at this example:

class SimplePOGO implements GroovyInterceptable {

    void simpleMethod1(){
        System.out.println("simpleMethod1() called")
        simpleMethod2Nested("stringParam", 24)
    }

    void simpleMethod2Nested(String param1, Integer param2){
        System.out.println("simpleMethod2Nested(${param1},${param2}) called")
        System.out.println("\nsleeping...\n")
        Timer.sleep(2000)
    }

    def invokeMethod(String name, args){
        System.out.println("time before ${name} called: ${new Date()}")

        //Get the method that was originally called.
        def calledMethod = SimplePOGO.metaClass.getMetaMethod(name, args)

        //The "?" operator first checks to see that the "calledMethod" is not
	//null (i.e. it exists).
        calledMethod?.invoke(this, args)

        System.out.println("time after ${name} called: ${new Date()}")
    }
}

simplePogo = new SimplePOGO()
simplePogo.simpleMethod1()

This code will produce the following output:

time before simpleMethod1 called: Wed Jun 18 12:07:35 EDT 2008
simpleMethod1() called
time before simpleMethod2Nested called: Wed Jun 18 12:07:35 EDT 2008
simpleMethod2Nested(stringParam,24) called

sleeping...

time after simpleMethod2Nested called: Wed Jun 18 12:07:37 EDT 2008
time after simpleMethod1 called: Wed Jun 18 12:07:37 EDT 2008

Not too surprising. When simpleMethod1() is called, the invokeMethod() in SimplePOGO is called which prints the current time stamp and then executes the simpleMethod1(). Notice however that simpleMethod1() calls simpleMethod2Nested(). When this call is made invokeMethod() will again be called so that the "before" time stamp will be printed out again. When simpleMethod2Nested() finishes executing an "after" time stamp is printed to the screen and control returns to simpleMethod1() which also finishes executing and prints the current time to the screen.

Gotcha?

You may have noticed that I used System.out.println() instead of the Groovier println() to print the output to the screen. This is because println() is a method that is built into the GroovyObject which is extended by all other Groovy objects (including SimplePOGO). If a call to println() is made within one of SimplePOGO's methods then the invokeMethod() on SimplePOGO will be called again which may lead to output that is not quite expected:

class SimplePOGO implements GroovyInterceptable {

    void simpleMethod1(){
        println "simpleMethod1() called"
    }

    void simpleMethod2(String param1, Integer param2){
        println "simpleMethod2(${param1},${param2}) called"
        println "sleeping..."
        Timer.sleep(2000)
    }

    def invokeMethod(String name, args){
        System.out.println("time before ${name} called: ${new Date()}")

        //Get the method that was originally called.
        def calledMethod = SimplePOGO.metaClass.getMetaMethod(name, args)

        //The "?" operator first checks to see that the "calledMethod" is not
	//null (i.e. it exists).
        calledMethod?.invoke(this, args)

        System.out.println("time after ${name} called: ${new Date()}")
    }
}

simplePogo = new SimplePOGO()
simplePogo.simpleMethod1()
simplePogo.simpleMethod2("stringParam", 24)

This code will produce the following output:

time before simpleMethod1 called: Wed Jun 18 12:13:32 EDT 2008
time before println called: Wed Jun 18 12:13:32 EDT 2008
simpleMethod1() called
time after println called: Wed Jun 18 12:13:32 EDT 2008
time after simpleMethod1 called: Wed Jun 18 12:13:32 EDT 2008
time before simpleMethod2 called: Wed Jun 18 12:13:32 EDT 2008
time before println called: Wed Jun 18 12:13:32 EDT 2008
simpleMethod2(stringParam,24) called
time after println called: Wed Jun 18 12:13:32 EDT 2008
time before println called: Wed Jun 18 12:13:32 EDT 2008
sleeping...
time after println called: Wed Jun 18 12:13:32 EDT 2008
time after simpleMethod2 called: Wed Jun 18 12:13:34 EDT 2008

Worse still is if you try to place the println() method inside of the invokeMethod() itself. This will lead to recursive calls and an eventual StackOverflowError:

class SimplePOGO implements GroovyInterceptable {

    void simpleMethod1(){
        System.out.println("simpleMethod1() called")
    }

    def invokeMethod(String name, args){
    	try {
	    println("time before ${name} called: ${new Date()}")

	    //Get the method that was originally called.
	    def calledMethod = SimplePOGO.metaClass.getMetaMethod(name, args)

	    //The "?" operator first checks to see that the "calledMethod" is
            //not null (i.e. it exists).
	    calledMethod?.invoke(this, args)

	    println("time after ${name} called: ${new Date()}\n")
	} catch (StackOverflowError soe) {
	    System.out.println("A StackOverflowError was caught")
	    System.exit(0)
	}
    }
}

simplePogo = new SimplePOGO()
simplePogo.simpleMethod1()

This code will produce the following output:

A StackOverflowError was caught

Conclusion

The examples listed above are very simple, but they illustrate how easy it is to implement the powerful features of AOP in Groovy. It's not too difficult to envision how such a capability could be built into an application to handle a lot of the boiler plate code that tends to crowd non AOP applications.

Book Review: Bulletproof Web Design

Justin Spradlin — Fri, 13 Jun 2008 00:06:02 +0000

Author: Dan Cederholm
Rating: 4/5

When I first started toying around with computers in the mid-90s one of my favorite things to do was create websites using free services like Geocities and Tripod. Back then I was guilty of all the deadly web design sins: table based layouts, pointless animated GIFs, heinous font and background color choices, and of course using the dreaded “Under Construction” page. Today I’m a little smarter about my design choices and feel that while web design as a whole isn’t quite where it needs to be, it is getting much better and there’s no doubt it’s because of books like Bulletproof Web Design by Dan Cederholm.

The book centers around designing sites to be accessible and flexible. A lot of the focus is on font sizing which on the surface may seem trivial, but because of differences among browsers, font size actually ends up having some important implications on accessibility. One of the most commonly used browsers (Internet Explorer 6) limits a user’s ability to increase or decrease a site’s font size if the designer of that site specifies the font size using strict pixel values. This creates a lot of problems for users with poor vision. Additionally, table based layouts are also targeted by the author for their excessive markup and rigid design constraints. In Bulletproof Web Design, Cederholm explains in detail some of the techniques designers can use to avoid these and other common design pitfalls.

Other topics included in the book that I found particularly useful were designing using floats, degrading pages nicely when CSS is not available, and a discussion on fluid vs elastic layouts.

While the book does not offer any revolutionary information, the concepts it does present on quality web design are organized in an efficient and easy to read manner. The book is filled with code examples and color pictures making it a quick and easy read. I’d definitely recommend the book to anyone starting out on a new web project or anyone looking for a refresher on best practices in web design.

A bit of a disclaimer: I just launched this blog recently and I know that I’m being a bit of a hypocrite by not following some of the basic web design principles found in the book being reviewed. I’m currently using a simple Wordpress theme built by Bob that gets the job done for now, but I’m looking forward to applying a lot of the principles from the book as I create my own custom theme for this site.

Hello World!

Justin Spradlin — Thu, 12 Jun 2008 01:02:36 +0000

Welcome to Justin Spradlin’s blog. This is my personal technology blog focusing on the latest trends in software development. Be sure to check in often for tutorials, news, and my thoughts on the tools and technologies that I am currently using for development.

Feel free to subscribe to the RSS feed to have the latest posts from this site appear directly in your favorite RSS reader.

If you’d like to contact me, just leave a comment on the site or send me a message via my LinkedIn Profile.

Thanks for visiting!