Alex Rothenberg

The Magic Behind Angularjs Dependency Injection

2013-02-11T00:00:00-08:00

Alex Rothenberg - The "Magic" behind AngularJS Dependency Injection

The "Magic" behind AngularJS Dependency Injection

February 11, 2013

If you’ve built anything with AngularJS you know there’s a lot of “magic” that you can usually ignore because it just works. One of the most magical parts for me is dependency injection. Just by adding a parameter to your controller function you suddenly get access to a powerful Angular service. It’s really pretty amazing but you sorta just have to trust it … until something goes wrong.

It turns out one easy way to break an AngularJS app is to minify your javascript. This happened to me when I deployed my app to production. The Angular app was being served from a Rails application and Rails automatically minifies javascript in prodution. It turns out there’s a simple and a well documented fix in their tutorial (search for “A Note on Minification”) that boils down to “use an array of parameter names” but it wasn’t clear to me why it worked.

In the rest of this article we’re going to

Build a simple AngularJS application
See how magical dependency injection is
Investigate how dependency injection is implemented in AngularJS
Break our app by minifying the javascript
Understand how the fix works

An AngularJS application using the GitHub API

We’re going to build a simple AngularJS app that uses the GitHub API to find the most recent commits on the angular.js project.

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/index.html

Here’s the source for that page:

<html ng-app>
<head>
  <script src="http://ajax.googleapis.com/ajax/libs/angularjs/1.0.4/angular.js"></script>

  <script>
  var MyController = function($scope, $http) {
    $http.get('https://api.github.com/repos/angular/angular.js/commits')
      .success(function(commits) {
        $scope.commits = commits
      })
  }
  </script>
</head>

<body ng-controller="MyController">
  <h1>Recent commits to AngularJS</h1>
  <ul>
    <li ng-repeat="commit in commits">
      {{ commit.commit.committer.date | date }}
      <a ng-href="https://github.com/angular/angular.js/commit/{{commit.sha}}">{{ commit.sha }}</a>
      {{ commit.commit.message }}
    </li>
  </ul>
</body>
</html>

Let’s walk through what’s going on here:

Line 1 - The ng-app attribute tells angular to treat this page as an angular app.
Line 6-11 - Some JavaScript that defines the controller and tell angular to “inject” the $scope and $http services.
Line 15 - The ng-controller="MyController" attribute tells angular the body tag will be scoped by the MyController controller.
Line 18-22 - This section will be expanded to many li elements in the DOM, each containing information about one commit.

This is pretty cool. Of course not everyone is a fan, in Dependency injection is not a virtue DHH argues that it is a legacy of the Java language that is unnecessary in Ruby (and I suspect he would argue JavaScript). AngularJS is figuring out what the hash part of the url is and automatically inserting it in the page.

Dependency Injection is magical

So where does dependency injection come in? Here’s where it gets weird… let’s try reordering the arguments in the controller function.

var MyController = function($http, $scope) {
  $http.get('https://api.github.com/repos/angular/angular.js/commits')
    .success(function(commits) {
      $scope.commits = commits
    })
}

We changed function($scope, $http) to function($http, $scope) and surprisingly it still works!

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/args_swapped.html

What seems to be going on as it runs is AngularJS

Knows what services our controller function needs in each parameter slot (originally $scope first & $http second now $http first & $scope second).
Decides what object should “provide” each of the named services (eg. $httpProvider provides $http).
Calls our controller with the appropriate providers in each slot (either MyController(scope, $httpProvider) or MyController(scope, $httpProvider)).

How does Angular do step #1?

In JavaScript the order of the parameters is important and the names do not matter to the caller (see this egghead.io video: $scope vs. scope)

Let’s take a look at some straight JavaScript and convince ourselves this is true. If we define a function divide that takes two arguments

var divide = function(numerator, denominator) {
  return numerator / denominator;
}

As expected divide(1, 2) == 0.5.

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/divide.html

When we change the order of the parameters, we also change the definition of the function.

var divide = function(denominator, numerator) {
  return numerator / denominator;
}

Now the definition has changed and divide(1, 2) == 2

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/divide_args_swapped.html

It seems Angular is going above and beyond what JavaScript the language supports.

How Dependency Injection implements Named Parameters in JavaScript

We saw that Angular’s dependency injection relies on the name of the parameters not their order which is a language feature called named parameters that does not exist in JavaScript. How do they do it?

AngularJS makes clever use of the fact that every object in JavaScript has a toString function to parse and extract the names of the parameters before deciding what arguments to call your controller function with.

Let’s play around with toString and get a feel of how it works. On function objects it returns the source code definition of the object, including the function signature with the names of the parameters.

When we call it on our divide function we can see this in action, divide.toString() == "function (numerator, denominator) { return numerator / denominator; }"

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/divide_toString.html

Angular takes this to the next level in a function called annotate in injector.js that takes a function and returns the names of its parameters.

$inject = [];
fnText = fn.toString().replace(STRIP_COMMENTS, '');
argDecl = fnText.match(FN_ARGS);
forEach(argDecl[1].split(FN_ARG_SPLIT), function(arg){
  arg.replace(FN_ARG, function(all, underscore, name){
    $inject.push(name);
  });
});
fn.$inject = $inject;
// ...
return $inject;

line 2 - use the toString() trick to get the function definition.
line 3 - do regular expression pattern matching FN_ARGS = /^function\s*[^$]*\(\s*([^$]*)\)/m to find the function signature.
line 4-8 - loop through all the parameters and save their names in an array.
line 11 - return the array or parameter names.

We can see this in action with our trusty divide function, angular.injector().annotate(divide) == ["numerator", "denominator"]

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/divide_annotate.html

Minification breaks our App

The technique of using toString has a big problem. We often minify our javascript before sending it to the browser, in my case the Rails asset pipeline was configured to automatically do this in the production environment.

// whitespace re-added for readability
var MyController = function(e,t) {
  t.get("https://api.github.com/repos/angular/angular.js/commits")
    .success(function(t) {
      e.commits=t
    })
}

When minified the parameter names are mangled to e or t so cannot be mapped to service names by angular. How can it know that t should be implemented by the $httpProvider or that e is the $scope?

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/minified_broken.html

In fact if you open the minified_broken.html example and look at the console you will see the error Error: Unknown provider: eProvider <- e.

Angular’s annotate function is sophisticated enough to handle minification, in fact, I only showed part of it before. When we look at the whole thing, we see it can annotate a function or an array of parameter name strings followed by a function. In the array form, the first string names the first argument, the second string names the second, etc. and the actual names of the parameters are unimportant.

function annotate(fn) {
  var $inject,
      fnText,
      argDecl,
      last;

  if (typeof fn == 'function') {
    if (!($inject = fn.$inject)) {
      // omitting the code we saw before
    }
  } else if (isArray(fn)) {
    last = fn.length - 1;
    assertArgFn(fn[last], 'fn')
    $inject = fn.slice(0, last);
  } else {
    assertArgFn(fn, 'fn', true);
  }
  return $inject;
}

When we can now redefine our MyController

var MyController = ['$scope', '$http', function($scope, $http) {
  $http.get('https://api.github.com/repos/angular/angular.js/commits')
    .success(function(commits) {
      $scope.commits = commits
    })
}]

When it is minified the strings are not touched so annotate will still work.

var MyController= ["$scope", "$http", function(e,t) {
  t.get("https://api.github.com/repos/angular/angular.js/commits")
    .success(function(t) {
      e.commits=t
    })
}]

We can see the annotate function parsing this array angular.injector().annotate(MyController) == ["$scope", "$http"]

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/minified_annotate.html

Of course what really matters is that it works on the page - which it does.

http://alexrothenberg.github.com/examples/angularjs_dependency_injection/minified_working.html

If you’ve made it to the end of this long post, I hope you’ve enjoyed the journey into how AngularJS knows what services to inject in your controllers. Again all you really need to know is to use an array naming the parameters followed by your function but I hope you enjoyed learning how it actually works. I know I did!

Combining D3 and Ember to Build Interactive Maps

2012-05-30T00:00:00-07:00

The Javascript world is exploding with new libraries that let us build really interactive applications. The image on the left is a static image showing how each US State voted in the 2008 Presidential Election - I suspect we’ll be seeing a lot more of this map in the coming months!

Let’s look at how we can draw that map ourselves and make it interactive using:

SVG - a vector graphics format. That will let us draw an good looking map using publicly available data about the geography of each state.
D3.js - a JavaScript library for manipulating documents based on data. Its super fast and will help us interact with the SVG map.
Ember.js - a JavaScript framework for creating ambitious web applications that eliminates boilerplate and provides a standard application architecture. Its going to keep us disciplined and our code organized.

An outline map of the States in the USA

We’ll get stated building an Ember app to draw a map of the USA with each state outlined.

First, we’ll need to know what each State looks like. This information is available to download from http://data.jquerygeo.com/usastates.json.

var MapApp.usaStatesGeoData = {
  "type":"FeatureCollection",
  "features":[
   {"type":"Feature",
    "properties":{"fips":8,"name":"Colorado","abbr":"CO"},
    "geometry":/*...one polygon defining a rectangular shape...*/},
   {"type":"Feature",
    "properties":{"fips":22,"name":"Louisiana","abbr":"LA"},
    "geometry":/*...many polygons defining a very complicated coastline */},

   // all the other States are here too ...
  ]
};

This is a GeoJSON format which Thomas Newton points out in his post Positioning and Scaling maps in D3 is a format that works really well with D3 since it is data that can be rendered into SVG by our code. Each State’s boundaries are defined in complete detail with polygons that a cartographer has drawn out for us. This works for rectangular States like Colorodo or Wyoming and amazingly also works for those with lots of coastline like Louisiana or Alaska (although the polygons are much more complicated).

Now, to create an Ember app we need to create our application and we’ll assign the GeoJSON data into a bindable property.

window.MapApp = Ember.Application.create({
  ready: function() {
    this.set('usaStatesGeoData', usaStatesGeoData);
  }
});

Next we’ll add some handlebars expressions to our html page telling it to display the map as an Ember View (we’ll create that in a minute).

<script type='text/x-handlebars'>
  { {view MapApp.Map geoDataBinding="MapApp.usaStatesGeoData"} }
</script>

Finally we write the view

MapApp.Map = Ember.View.extend({
  didInsertElement: function() {
    var elementId = this.get('elementId');
    var regions = d3.select("#" + elementId).append("svg").append("g").attr("id", "regions");

    var features = this.get('geoData').features;
    var path = this.get('path');
    regions.selectAll("#regions path").data(features).enter().insert("path")
      .attr("d",      path)
      .attr('stroke', '#ccc'  )
      .attr('fill',   'white' );
  }

  path: function() {
    var mapW  = this.$().height();
    var mapH  = this.$().width();
    var mapXY = d3.geo.albersUsa().scale(1000).translate([mapH/2,mapW/2]);

    return d3.geo.path().projection(mapXY);
  }.property(),
});

Whew that is a bit of code so let’s dig in. This is where we use d3 to create the svg that looks like a map of the USA. d3 (Data-Driven Documents) works by binding data to your DOM then applying tranformations to that data. In this case our data is the GeoJSON file and our tranformations turn it into an SVG map. didInsertElement is called by Ember when the view is inserted into the page’s DOM.

The first two lines use d3.select to add the <svg><g id="regions"> elements to the page on this view’s portion of the page.
The remaining use d3.selectAll.data.enter to loop through the features (each State) and add a <path> element.
Looping through each feature (ie. State), we make use of the rest of the code and set the d attribute to the value of the path property. The path logic scales the map and uses the albersUsa projection which moves Alaska and Hawaii to the bottom left where we usually see them on maps.

Looking at code is fun but what does it look like? Below is the page we just built and the you can see that it does, in fact, look like a map of the USA with each State outlined.

http://www.alexrothenberg.com/examples/ember-d3maps/map_outline.html

Coloring the States red or blue

A red/blue map isn’t much use when all the States are white so let’s color them in.

First we’ll need some data with the election results so we know what color each State should be. There’s a source from Google at http://code.google.com/p/general-election-2008-data/source/browse/trunk/json/votes/2008. I found the format to be a little hard to work with so I processed it into a simpler json file with just the information we’ll need. It looks like:

[
  {"name": "Alabama",    "electoral": 9,  "winner": "McCain"},
  {"name": "Alaska",     "electoral": 3,  "winner": "McCain"},
  {"name": "Arizona",    "electoral": 10, "winner": "McCain"},
  {"name": "Arkansas",   "electoral": 6,  "winner": "McCain"},
  {"name": "California", "electoral": 55, "winner": "Obama"},

  // results for the rest of the States too...
]

To use it we’ll save it in our Ember Application.

window.MapApp = Ember.Application.create({
  ready: function() {
    this.set('usaStatesGeoData', usaStatesGeoData);

    var stateResults = usaPres2008Data.map(function(stateResult) {
      return MapApp.StateResult.create(stateResult);
    });
    this.set('usaPres2008Results',  MapApp.StateResults.create({content: stateResults}));
  }
});

We use the javascript map function to turn our json data into ember objects that we can put into an Ember ArrayProxy object. For this example we don’t really need these ember objects but in a real application we’d probably get some requirements that would make them useful. For instance if we wanted the colors to update on election night as new results come in.

Since we are using the StateResult and StateResults objects, we need to define them next.

MapApp.StateResult  = Ember.Object.extend({});

MapApp.StateResults = Ember.ArrayProxy.extend({
  names: (function() {
    return this.mapProperty('name');
  }).property('content'),

  findByName: function(name) {
    var content = this.get('content');
    var index = this.get('names').indexOf(name);
    return content[index];
  }
});

Both objects are pretty simple. In fact they would both be empty except for the findByName accessor we define on StateResults. We’ll see the need for that in a minute when we look at how the view decides whether to color a State red or blue.

This takes us to the view which enhances what we had before:

MapApp.Map = Ember.View.extend({
  // the path property is unchanged

  didInsertElement: function() {
    // same code as before ...

    regions.selectAll("#regions path").data(features).enter().insert("path")
      .attr("d",      path)
      .attr('stroke', '#ccc')
      .attr('fill',   this.updateFillFor ); // This line changed
  },

  // A new function
  updateFillFor: function(d) {
    var stateResult = MapApp.usaPres2008Results.findByName(d.properties.name);
    switch (stateResult.get('winner')) {
      case 'Obama':
        return 'blue';
      case 'McCain':
        return 'red';
    }
  }
});

Most of the view is the same, all that’s different is the way we set the fill on each State (“feature” in d3 terms).

.attr('fill', this.updateFillFor); - Instead of setting the “fill” attribute to ‘white’ for all States we now provide a function to make the decision differently for each State. D3 will call the function with the current State as an argument (it actually passes the D3 “datum” object).
updateFillFor - This function uses the findByName function we defined in StateResults to match the “geographical” State with the “election result” State. They are separate because each came from a different external json source. Once we have the stateResult object we look at the winner and know to return either “red” or “blue”.

We can see it all working together right here and yes it does look like all the other red/blue election maps you’ve seen.

http://www.alexrothenberg.com/examples/ember-d3maps/map_red_blue.html

This is just the beginning of what you can do by combining D3.js with Ember.js. Next I plan to write about how you could add behavior like clicking to select a State and let Ember show details about that State.

Closures can provide encapsulation when Creating Objects in Javascript

2012-03-14T00:00:00-07:00

Objects are funny things in Javascript. If you’re coming from a language like Ruby with classical inheritance you’ll probably be surprised that in Javascript’s prototype system there are no such things as classes. The patterns are different but we can still achieve what’s important about object oriented software namely objects that encapsulate data and behavior. Today I’m going to show you two different ways we can create a simple database object that supports the basic CRUD operations in Javascript.

Object Literal

The simplest way to create our database is to just declare it as a singleton object with some instance data and functions. We’ll use the “starts with underscore” naming convention to denote “private” data but that is just a convention and not enforced.

var Database = {
  _data: {},

  create: function(object) {
    return this._data[object.id] = object
  },
  read: function(id) {
    return this._data[id]
  },
  update: function(object) {
    return this._data[object.id] = object
  },
  delete: function(id) {
    return this._data[id] = null
  }
}

This is pretty simple and if you try it out you’ll see it actually works.

Try it at http://jsfiddle.net/alexrothenberg/Tn8uq/

// We can CREATE
Database.create({id: 7, name: 'Alex'});
Database.create({id: 9, name: 'Pat'});

// We can READ
console.log('7: ' + Database.read(7).name);
  // 7: Alex
console.log('9: ' + Database.read(9).name);
  // 9: Pat

// We can UPDATE
Database.update({id: 7, name: 'Alexander'})
console.log('7: ' + Database.read(7).name);
  // 7: Alexander

// We can DELETE
Database.delete(7)
console.log('7: ' + Database.read(7));
  // 7: null

The biggest downside of this approach is that there is no encapsulation. We can get at the _data instance variable directly to read or change it.

Database._data[7] = {name: 'I changed your name'}
console.log('7: ' + Database.read(7).name);

Encapsulation via a Closure

By wrapping our Database in a closure we can encapsulate our private functions and data. The way to read this is that

Database is a function that defines a closure.
The variables within that closure have access to each other (i.e. _create has access to _data).
When the Database is called it returns an object that explicitly exposes 4 functions and nothing else.

var Database = (function() {
  _data = {}

  _create = function(object) {
    return _data[object.id] = object
  }
  _read =  function(id) {
    return _data[id]
  }
  _update = function(object) {
    return _data[object.id] = object
  }
  _delete = function(id) {
    return _data[id] = null
  }

  return {
    create: _create,
    read:   _read,
    update: _update,
    delete: _delete
  }
}).call(this)

When we test it we see that it still works.

Try it at http://jsfiddle.net/alexrothenberg/kcGLK/

// We can CREATE
Database.create({id: 7, name: 'Alex'});
Database.create({id: 9, name: 'Pat'});

// We can READ
console.log('7: ' + Database.read(7).name);
  // 7: Alex
console.log('9: ' + Database.read(9).name);
  // 9: Pat

// We can UPDATE
Database.update({id: 7, name: 'Alexander'})
console.log('7: ' + Database.read(7).name);
  // 7: Alexander

// We can DELETE
Database.delete(7)
console.log('7: ' + Database.read(7));
  // 7: null

The _data is encapsulated and not accessible from the outside

console.log(Database._data)
  // undefined

There are other patterns for working with objects in Javascript that take advantage of Javascript’s prototype system but I don’t have time to go into that today … perhaps in a future post.

Building a Browser IDE

2012-02-29T00:00:00-08:00

Its become so easy to share code examples as gists but once you start sharing html, css or javascript you can do so much more than share static code. Browsers can run html, css and javascript so we can actually run the code we’re sharing. Let’s look at how we could build a simple IDE in your browser like jsfiddle where where you can experiment with your HTML, CSS and Javascript.

HTML IDE

First we’re going to build an HTML editor. Try it out…I’ll wait.

How does it work? We need three things.

1 - A textarea where you can type in the html

  <textarea id="html" class="content">
    Hi there this is some <b>bold</b> content
    and this is <i>italic</i>.
    <p>Pretty cool huh!</p>
    <p>What else can you type?</p>
  </textarea>

2 - An iframe to preview the page

<iframe id="preview" class="content"></iframe>

3 - Some javascript to copy the html from the text area into the div

$('#html').keypress(function() {
  $('#preview').contents().find('html').html($(this).val())
})

After we add some formatting and put it all together we get a page like this.

http://www.alexrothenberg.com/examples/browser_ide/html_ide.html

<html>
  <head>
    <meta charset=utf-8>
    <title>Browser IDE</title>
    <link rel="stylesheet" href="./styles.css" type="text/css" charset="utf-8">
    <script src="jquery.min.js"></script>
    <script type='text/javascript'>
      $(function() {
        $('#html').keyup(function() {
          $('#preview').contents().find('body').html($(this).val())
        })
        $('#html').keyup() // Initialize
      })
    </script>
  </head>
  <body>
    <h1>Edit the HTML and see it in the Preview</h1>
    <div class="box">
      <div class="label">HTML</div>
      <textarea id="html" class="content">
Hi there this is some <b>bold</b> content
and this is <i>italic</i>.
<p>Pretty cool huh!</p>
<p>What else can you type?</p>
      </textarea>
    </div>
    <div class="box">
      <div class="label">Preview</div>
      <iframe id="preview" class="content"></iframe>
    </div>
  </body>
</html>

HTML and CSS IDE

HTML is good but any self respecting webpage will also have a CSS file. We can add that too.

To get this working we added

1 - A new textarea where you can type in the css

  <textarea id="css" class="content">
    .warning { color: red; }
  </textarea>

2 - Some javascript to copy the css from the text area into the div

  $('#css').keypress(function() {
    preview_contents.find('head style').remove()
    preview_contents.find('head').append("<style>" + $(this).val() + "</style>")
  })

All put together it looks like this.

http://www.alexrothenberg.com/examples/browser_ide/html_css_ide.html

<html>
  <head>
    <meta charset=utf-8>
    <title>Browser IDE</title>
    <link rel="stylesheet" href="./styles.css" type="text/css" charset="utf-8">
    <script src="jquery.min.js"></script>
    <script type='text/javascript'>
      $(function() {
        var preview_contents = $('#preview').contents()
        $('#html').keyup(function() {
          preview_contents.find('body').html($(this).val())
        })
        $('#css').keyup(function() {
          preview_contents.find('head style').remove()
          preview_contents.find('head').append("<style>" + $(this).val() + "</style>")
        })
        $('#html').keyup() // Initialize
        $('#css').keyup() // Initialize
      })
    </script>
  </head>
  <body>
    <h1>Edit HTML and CSS</h1>
    <div class="box">
      <div class="label">HTML</div>
      <textarea id="html" class="content">
<div class="warning">This is a warning!</div>
<div class="message">This is a message.</div>
      </textarea>
    </div>
    <div class="box">
      <div class="label">CSS</div>
      <textarea id="css" class="content">
.warning { color: red; }
/* Try adding a .message style.  Make it bold. */
      </textarea>
    </div>
    <div class="box">
      <div class="label">Preview</div>
      <iframe id="preview" class="content"></iframe>
    </div>
  </body>
</html>

HTML, CSS and Javascript IDE

So the last part of the page we want to edit is javascript.

We implement this following the same pattern we’ve been using.

1 - A new textarea where you can type in the javascript

  <textarea id="javascript" class="content">
    // Change the next line and watch the preview change
    var new_text = 'This text was added by js'

    var js_element = document.getElementById('js_content')
    js_element.innerHTML = new_text
  </textarea>

2 - Some javascript to copy the css from the text area into the div

  $('#javascript').keypress(function() {
    preview_contents.find('head script').remove()
    var created_script = preview_contents[0].createElement('script');
    created_script.text = $(this).val()
    preview_contents.find('head')[0].appendChild(created_script)
  })

Here’s the complete page.

http://www.alexrothenberg.com/examples/browser_ide/html_css_js_ide.html

<html>
  <head>
    <meta charset=utf-8>
    <title>Browser IDE</title>
    <link rel="stylesheet" href="./styles.css" type="text/css" charset="utf-8">
    <script src="jquery.min.js"></script>
    <script type='text/javascript'>
      $(function() {
        var preview_contents = $('#preview').contents()
        $('#html').keyup(function() {
          preview_contents.find('body').html($(this).val())
          $('#javascript').keyup()   // let the javascript change the page 
        })
        $('#css').keyup(function() {
          preview_contents.find('head style').remove()
          preview_contents.find('head').append("<style>" + $(this).val() + "</style>")
        })
        $('#javascript').keyup(function() {
          preview_contents.find('head script').remove()
          var created_script = preview_contents[0].createElement('script');
          created_script.text = $(this).val()
          preview_contents.find('head')[0].appendChild(created_script)
        })
        $('#html').keyup() // Initialize
        $('#css').keyup() // Initialize
        $('#javascript').keyup() // Initialize
      })
    </script>
  </head>
  <body>
    <h1>Edit HTML, CSS and Javascript</h1>
    <div class="box">
      <div class="label">HTML</div>
      <textarea id="html" class="content">
<div class="warning">This is a warning!</div>
<div class="message">This is a message</div>
<div id="js_content"></div>
      </textarea>
    </div>
    <div class="box">
      <div class="label">CSS</div>
      <textarea id="css" class="content">
.warning { color: red; }
      </textarea>
    </div>
    <div class="box">
      <div class="label">Javascript</div>
      <textarea id="javascript" class="content">
// Change the next line and watch the preview change
var new_text = 'This text was added by js'

var js_element = document.getElementById('js_content')
js_element.innerHTML = new_text
      </textarea>
    </div>
    <div class="box">
      <div class="label">Preview</div>
      <iframe id="preview" class="content"></iframe>
    </div>
  </body>
</html>

Syntax highlighting

We now have a very simple IDE but its missing something all code editors have. No I don’t mean “save”, although that is important I’m going to ignore it. When I’m typing text it all starts to run together and become really hard to read. It would be so much easier if we had some syntax highlighting. We’re going to use a javascript library called CodeMirror which knows how to syntax highlight html, css, javascript and a ton of other languages.

When we’re done it will be much easier to read our code

We’ll add CodeMirror in these 3 steps.

1 - First we download CodeMirror and save it along with our files.

2 - Now we need to add it to our page

  <link rel="stylesheet" href="./codemirror/codemirror.css">
  <script src="./codemirror/mode/javascript/javascript.js"></script>
  <script src="./codemirror/mode/css/css.js"></script>
  <script src="./codemirror/mode/xml/xml.js"></script>
  <script src="./codemirror/mode/htmlmixed/htmlmixed.js"></script>

3 - Finally some javascript that enables CodeMirror on our three textareas. Here’s the code for the html pane.

  CodeMirror.fromTextArea($('#html')[0], {
    mode:  "htmlmixed", 
    matchBrackets: true,
    onChange: function(editor) { copyHTML(editor.getValue()) }
  })
  var copyHTML = function(html_text) {
    preview_contents.find('html').html(html_text)
    copyJavascript($('#javascript').val())  // let the javascript change the page
  }
  copyHTML($('#html').val())             // Initialize

CodeMirror actually offers a ton more functionality than just syntax highlighting like emacs or vim keybinding, matching parentheses, undo, and more that you can investigate if you’re interested. When we’re done with all that we have this single page app.

http://www.alexrothenberg.com/examples/browser_ide/index.html

<html>
  <head>
    <meta charset=utf-8>
    <title>Browser IDE</title>
    <link rel="stylesheet" href="./styles.css" type="text/css" charset="utf-8">
    <script src="jquery.min.js"></script>
    <script src="./codemirror/codemirror.js"></script>
    <link rel="stylesheet" href="./codemirror/codemirror.css">
    <script src="./codemirror/mode/javascript/javascript.js"></script>
    <script src="./codemirror/mode/css/css.js"></script>
    <script src="./codemirror/mode/xml/xml.js"></script>
    <script src="./codemirror/mode/htmlmixed/htmlmixed.js"></script>
    <script type='text/javascript'>
      $(function() {
        var preview_contents = $('#preview').contents()
        CodeMirror.fromTextArea($('#html')[0], {
          mode:  "htmlmixed", 
          matchBrackets: true,
          onChange: function(editor) { copyHTML(editor.getValue()) }
        })
        CodeMirror.fromTextArea($('#css')[0], {
          mode:  "css", 
          matchBrackets: true,
          onChange: function(editor) { copyCSS(editor.getValue()) }
        })
        CodeMirror.fromTextArea($('#javascript')[0], {
          mode:  "javascript", 
          matchBrackets: true,
          onChange: function(editor) { copyJavascript(editor.getValue()) }
        })
        var copyHTML = function(html_text) {
          preview_contents.find('body').html(html_text)
          copyJavascript($('#javascript').val())  // let the javascript change the page
        }
        var copyCSS =function(css_text) {
          preview_contents.find('head style').remove()
          preview_contents.find('head').append("<style>" + css_text + "</style>")
        }
        var copyJavascript =function(javascript_text) {
          preview_contents.find('head script').remove()
          var created_script = preview_contents[0].createElement('script');
          created_script.text = javascript_text
          preview_contents.find('head')[0].appendChild(created_script)
        }
        copyHTML($('#html').val())             // Initialize
        copyCSS($('#css').val())               // Initialize
        copyJavascript($('#javascript').val()) // Initialize
      })
    </script>
  </head>
  <body>
    <h1>Edit HTML, CSS and Javascript with Syntax Highlighting</h1>
    <div class="box">
      <div class="label">HTML</div>
      <textarea id="html" class="content">
<div class="warning">This is a warning!</div>
<div class="message">This is a message</div>
<div id="js_content"></div>
      </textarea>
    </div>
    <div class="box">
      <div class="label">CSS</div>
      <textarea id="css" class="content">
.warning { color: red; }
      </textarea>
    </div>
    <div class="box">
      <div class="label">Javascript</div>
      <textarea id="javascript" class="content">
// Change the next line and watch the preview change
var new_text = 'This text was added by js'

var js_element = document.getElementById('js_content')
js_element.innerHTML = new_text
      </textarea>
    </div>
    <div class="box">
      <div class="label">Preview</div>
      <iframe id="preview" class="content"></iframe>
    </div>
  </body>
</html>

This entire sample we’ve built is on github at https://github.com/alexrothenberg/alexrothenberg.github.com/tree/master/examples/browser_ide.

Testing IP Whitelisting in your Specs and Features

2011-11-21T00:00:00-08:00

Rails has so much support for testing built into itself that its rare I come up with something that’s hard to test but HTTP headers is not easy. Normally you don’t have to worry about HTTP headers as they’re set by the browser and you don’t do much with them. Recently I was working on an application where each user has an IP whitelist and they are only allowed to come from their whitelisted IP addresses. This isn’t as crazy as it sounds since the app is in a corporate environment and the users will all be coming from their corporate networks.

Basically this means our authentication method needs 3 pieces of information

username
password
remote ip address

What makes this interesting is that the first two are input by the user but the ip address comes from the browser and network. Writing an RSpec unit test or Cucumber scenario to test user parameters (username and password) is something we’ve all done before but today I’m going to talk about how you can also test the IP address in a header.

Implementation

Before we look at how to test this let’s take a look at the implementation of our SessionController.

app/controllers/sessions_controller.rb

class SessionsController < ApplicationController
  def new
    @session = Session.new
  end

  def create
    remote_ip_address = request.headers['X-Forwarded-For'] || request.headers['REMOTE_ADDR']
    @session = Session.create(params[:username], params[:password], remote_ip_address)

    if @session.valid?
      session[:current_user] = @session.user
      redirect_to root_url
    else
      flash.now[:error] = 'Unable to authenticate. Please try again'
      render :new
    end
  end

  def destroy
    session[:current_user] = nil
    redirect_to session
  end
end

These three actions provide login and logout.

new displays the login form with username & password fields
create uses the username and password from the form as well as the ip address to create a session (i.e. authenticate). In case the request hops through some proxy servers we use the X-Forwarded-For header to get the source IP and not the proxy’s IP.
destroy users need to log out (but we wont talk about that anymore here)

This works, but you shouldn’t trust me. We need tests around the create action!

Unit Testing the IP Whitelist with RSpec

Our Controller Spec needs to pass all 3 pieces of information (username, password & ip address) to the controller. Passing the username and password is pretty standard and something I’m sure you’ve done before. They come from a form so we pass them as a hash in the second argument to post.

spec/controllers/sessions_controller_spec.rb

post :create, {:username => 'alex', :password => 'secret'}

Unfortunately we can’t pass the IP the same way because the post method in ActionController::TestCase doesn’t support passing headers in (but it does take the session or flash - that’s interesting to remember for some other time).

actionpack/lib/action_controller/test_case.rb

def post(action, parameters = nil, session = nil, flash = nil)
  process(action, parameters, session, flash, "POST")
end

If we keep looking around it turns out the ActionDispatch::TestRequest object has a nice convenience method that lets us specify the remote_addr directly.

actionpack/lib/action_dispatch/testing/test_request.rb

def remote_addr=(addr)
  @env['REMOTE_ADDR'] = addr
end

If we add a line to our spec we can handle the case where the IP comes in the REMOTE_ADDR HTTP header.

spec/controllers/sessions_controller_spec.rb

request.remote_addr = '192.168.1.100'
post :create, {:username => 'alex', :password => 'secret'}

We still need to deal with the X-Forwarded-For case. While Rails doesn’t give us a convenience method, by looking at the implementation of the remote_addr= method we can see how to set this header ourselves.

spec/controllers/sessions_controller_spec.rb

request.env['X-Forwarded-For'] = '192.168.1.100'
post :create, {:username => 'alex', :password => 'secret'}

Putting it all together we end up with a controller spec that looks like this.

spec/controllers/sessions_controller_spec.rb

require 'spec_helper'

describe SessionsController do
  describe '#create' do
    describe 'successfully' do
      let(:alex) { mock }
      let(:valid_session) { mock(:valid? => true, :user => alex )}
      before do
        Session.should_receive(:create).with('alex', 'secret', '192.168.1.100').and_return(valid_session)
      end
      describe 'using REMOTE_ADDR' do
        before do
          request.remote_addr = '192.168.1.100'
          post :create, {:username => 'alex', :password => 'secret'}
        end
        it { should redirect_to root_path }
        it { should set_session(:current_user).to(alex)}
      end
      describe 'using X-Forwarded-For' do
        before do
          request.remote_addr = '172.16.254.1'
          request.env['X-Forwarded-For'] = '192.168.1.100'
          post :create, {:username => 'alex', :password => 'secret'}
        end
        it { should redirect_to root_path }
        it { should set_session(:current_user).to(alex)}
      end
    end

    describe 'unsuccessfully' do
      let(:invalid_session) { mock(:valid? => false) }
      before do
        Session.should_receive(:create).with('alex', 'secret', '192.168.1.100').and_return(invalid_session)
      end
      describe 'using REMOTE_ADDR' do
        before do
          request.remote_addr = '192.168.1.100'
          post :create, {:username => 'alex', :password => 'secret'}
        end
        it { should render_template :new }
      end
      describe 'using X-Forwarded-For' do
        before do
          request.remote_addr = '172.16.254.1'
          request.env['X-Forwarded-For'] = '192.168.1.100'
          post :create, {:username => 'alex', :password => 'secret'}
        end
        it { should render_template :new }
      end
    end
  end
end

To sum up we can

pass parameters as a hash in the post method

post :create, {:username => 'alex', :password => 'secret'}
set the remote_addr on the request with a convenience method

request.remote_addr = '192.168.1.100'
et the X-Forwarded-For directly on the requests’s environment hash

request.env['X-Forwarded-For'] = '192.168.1.100'

Integration Testing the IP Whitelist in a Cucumber Feature

We face a similar issue when writing our cucumber scenarios - its easy to pass the username and password but harder to pass the IP address. The solution turns out to be similar but not quite exactly the because our Cucumber steps will use Capybara instead of ActionController::TestCase directly. Before we look into how to implement the steps, let’s write the feature we want which will help us define the steps we need.

features/authentication.feature

Feature: Authentication of a user
  In order to ensure a really secure application
  As a user
  I want my IP address to be validated during login

  Background:
    Given the following user exists:
      | username | password | company                   |
      | alex     | secret   | ip_address: 192.168.1.100 |

  Scenario: Successful log in
    Given I am connecting from ip "192.168.1.100"
     When I log in as "alex" with password "secret"
     Then I should be on the home page

  Scenario: Successful log in with X-Forwarded-For header
    Given I am connecting from ip "192.168.1.100" behind a proxy
     When I log in as "alex" with password "secret"
     Then I should be on the home page

  Scenario: Failed log in from wrong IP
    Given I am connecting from ip "172.16.254.1"
     When I log in as "alex" with password "secret"
     Then authentication should have failed

  Scenario: Failed log in from wrong IP behind a proxy
    Given I am connecting from ip "172.16.254.1" behind a proxy
     When I log in as "alex" with password "secret"
     Then authentication should have failed

We immediately realize we don’t know how to write the first step

features/step_definitions/authentication_steps.rb

Given /^I am connecting from ip "([^"]*)"$/ do |ip_address|
  pending # How do we set the IP Address???
end

To figure this out we need to dig into how capybara works.

We don’t call post in ActionController::TestCase directly instead letting capybara do it for us. To see what capybara is doing we can skip that step and implement the login step

features/step_definitions/authentication_steps.rb

Given /^I am connecting from ip "([^"]*)"$/ do |ip_address|
  # do nothing for now
end

When /^I log in as "([^"]*)" with password "([^"]*)"$/ do |name, password|
  visit(new_session_path)
  fill_in('User name', :with => name)
  fill_in('Password', :with => password)
  click_button('Log In')
end

and edit the SessionsController to show us the stack trace.

app/controllers/sessions_controller.rb

class SessionsController < ApplicationController
  def create
    raise caller.inspect
  end
end

The stack trace is very big but if we look closely, somewhere in the middle of it we see lines below that show how capybara uses the rack-test gem to submit our form.

~/.rvm/gems/ruby-1.8.7-p334/gems/rack-test-0.6.1/lib/rack/test.rb:66:in `post'
~/.rvm/gems/ruby-1.8.7-p334/gems/capybara-1.1.2/lib/capybara/rack_test/browser.rb:62:in `send'
~/.rvm/gems/ruby-1.8.7-p334/gems/capybara-1.1.2/lib/capybara/rack_test/browser.rb:62:in `process'
~/.rvm/gems/ruby-1.8.7-p334/gems/capybara-1.1.2/lib/capybara/rack_test/browser.rb:27:in `submit'
~/.rvm/gems/ruby-1.8.7-p334/gems/capybara-1.1.2/lib/capybara/rack_test/form.rb:64:in `submit'
... more lines omitted...
~/.rvm/gems/ruby-1.8.7-p334/gems/capybara-1.1.2/lib/capybara/node/actions.rb:38:in `click_button'

Looking at the Rack::Test#post method we see something similar to what we saw before in ActionController::TestCase but its not quite identical. It takes the env as a parameter so we need to figure out how to inject our header in there.

rack-test - lib/rack/test.rb

def post(uri, params = {}, env = {}, &block)
  env = env_for(uri, env.merge(:method => "POST", :params => params))
  process_request(uri, env, &block)
end

Following the stack trace up we see the env passed into Rack::Test::Session.post comes from Capybara::RackTest::Browser and it turns out that env is computed in the Capybara::RackTest::Browser#env method.

capybara - lib/capybara/rack_test/browser.rb

def options
  driver.options
end

def env
  env = {}
  begin
    env["HTTP_REFERER"] = last_request.url
  rescue Rack::Test::Error
    # no request yet
  end
  env.merge!(options[:headers]) if options[:headers]
  env
end

The key is in the line env.merge!(options[:headers]) if options[:headers] and those options are delegated to the driver. Now we know how to inject our IP address onto the driver’s options.

features/step_definitions/authentication_steps.rb

Given /^I am connecting from ip "([^"]*)"$/ do |ip_address|
  page.driver.options[:headers] = {'REMOTE_ADDR' => ip_address}
end

Putting it all together we can write all our steps

features/step_definitions/authentication_steps.rb

Given /^I am connecting from ip "([^"]*)"$/ do |ip_address|
  page.driver.options[:headers] = {'REMOTE_ADDR' => ip_address}
end

Given /^I am connecting from ip "([^"]*)" behind a proxy$/ do |ip_address|
  page.driver.options[:headers] = {'X-Forwarded-For' => ip_address}
end

When /^I log in as "([^"]*)" with password "([^"]*)"$/ do |name, password|
  visit(new_session_path)
  fill_in('User name', :with => name)
  fill_in('Password', :with => password)
  click_button('Log In')
end

Then /^I should be on the home page$/ do
  URI.parse(current_url).path.should == root_path
end

Then /^authentication should have failed$/ do
  page.text.should include 'Unable to authenticate. Please try again'
end

Now the scenarios we wrote before all pass.

To sum up

capybara handles form submission superbly with

fill_in('User name', :with => name)

click_button('Log In')
we can set any HTTP header in capybara with

page.driver.options[:headers] = {'REMOTE_ADDR' => ip_address}

Testing is good

Since we’re testing the IP logic at both the unit level with RSpec and integration level with Cucumber and Capybara we can be pretty sure it’s all going to work correctly.

Programming With Kids

2011-11-14T00:00:00-08:00

I’ve started to teach my kids to program. I figured I build websites professionally and it’d be a fun way for me to share what I do and help supplement their learning. And it was something they expressed interest in not something I was pushing. I suggested we build our own small version of facebook or twitter. Very quickly I learned two truths

websites are boring
games are fun

Okay. I’ve never built a game before after a little digging there are plenty of tools in the open source world and many built on Ruby. We’re currently experimenting with three different tools/technologies.

Shoes

I first came across Shoes several years ago and was blown away. It was originally written by why and is now maintained by Team Shoes on github. “Shoes is the best little DSL for cross-platform GUI programming there is. It feels like real Ruby, rather than just another C++ library wrapper”

Writing a Shoes app feels just like writing a Ruby app (it is Ruby!). The best analogy I can use is that what Rails does for websites, Shoes does for GUI apps.

If you want to create a blue rectangle on a page, here’s your app

Shoes.app do
  fill blue
  rect :top => 25, :left => 50, :height => 75, :width => 150
end

We can make it a bit more interactive and allow the user to move our rectangle around with the arrow keys and display the current coordinates

Shoes.app do
  fill blue
  @player = rect :top => 25, :left => 50, :height => 75, :width => 150
  @current_coordinates = para "(#{@player.left}, #{@player.top})"

  keypress do |key|
    @player.left += 10 if key == :right
    @player.left -= 10 if key == :left
    @player.top  += 10 if key == :down
    @player.top  -= 10 if key == :up
    @current_coordinates.replace "(#{@player.left}, #{@player.top})"
  end
end

We’re not limited to blue rectangles. We can replace it with an image

Shoes.app do
  @player = image 'images/Starfighter.png', :top => 25, :left => 50
  @current_coordinates = para "(#{@player.left}, #{@player.top})"

  keypress do |key|
    @player.left += 10 if key == :right
    @player.left -= 10 if key == :left
    @player.top  += 10 if key == :down
    @player.top  -= 10 if key == :up
    @current_coordinates.replace "(#{@player.left}, #{@player.top})"
  end
end

We’re writing Ruby in a fairly natural way. Shoes just gives us GUI methods like rect to create a rectangle or para to create a paragraph. Because this is Ruby, as your Shoes app gets more complex you can create classes and methods to organize and keep it manageable just as you would in any other app.

There are all sorts of great resources out there including

The Shoes Manual which includes a reference for all the elements you may use (like rect or para)
why’s tutorial - the original introduction to Shoes
sample apps - there are some really good ones here!
Teaching Ruby to High School Girls (using Shoes) article by Sarah Mei

Gosu

Gosu is a gaming library so while Shoes lets us build any sort of GUI apps this is seemed like it might be a better fit since we’re interested in gaming. Luckily there’s a gem that wraps up the Ruby interface (Gosu can be used from C++ or Ruby).

gem install gosu

If we want to build a similar game to what we did in Shoes with a play we move around via the arrow keys we need to subclass the Gosu::Window class.

require 'rubygems'
require 'gosu'

class Player
  def initialize(window)
    @image = Gosu::Image.new(window, "media/Starfighter.png", false)
    @x, @y = 125, 50
    @angle = 0.0
  end

  def draw
    @image.draw_rot(@x, @y, 0, @angle)
  end
end

class GameWindow < Gosu::Window
  def initialize
    super(640, 480, false)
    self.caption = "Gosu Tutorial Game"
    @player = Player.new(self)
  end

  def draw
    @player.draw
  end
end

window = GameWindow.new
window.show

And we see a window with a player that looks like a starship

Its not too hard to make it move by overriding the update method in our window class

require 'rubygems'
require 'gosu'

class Player
  attr_accessor :x, :y

  def initialize(window)
    @image = Gosu::Image.new(window, "media/Starfighter.bmp", false)
    @x, @y = 75, 50
    @angle = 0.0
  end

  def draw
    @image.draw_rot(@x, @y, 0, @angle)
  end
end

class GameWindow < Gosu::Window
  def initialize
    super(400, 300, false)
    self.caption = "Our Game"
    @player = Player.new(self)
    @current_coordinates = Gosu::Font.new(self, Gosu::default_font_name, 20)
  end

  def update
    @player.x -= 10 if button_down? Gosu::KbLeft
    @player.x += 10 if button_down? Gosu::KbRight
    @player.y += 10 if button_down? Gosu::KbDown
    @player.y -= 10 if button_down? Gosu::KbUp
  end

  def draw
    @player.draw
    @current_coordinates.draw("(#{@player.x}, #{@player.y})", 10, 10, 0, 1.0, 1.0, 0xffffff00)
  end
end

window = GameWindow.new
window.show

This example can be extended into a full Asteroids like like game where your ship has inertia. You should look at the source or an explanation on the gosu site.

There are all sorts of great resources out there including

Samples from the Gosu Showcase or

Falling Blocks or CptnRuby
Chingu an extension that seems to let us avoid re-writing common tasks
Chipmunk a physics library that makes it easy to do things like collision detection, gravity, etc

Even though we’re still writing Ruby, Gosu feels more like C++ Windows development I used to do long long time ago. I’m not sure if that’s inevitable and need to keep using Gosu to find out.

gamesalad

The last framework we’ve been working with is pretty different. GameSalad advertises it lets you “Create games for iPhone, iPad, Android, Mac, and HTML5. No coding required.”

It follows a model similar to what Adobe Flash uses where you have Scenes containing Actors. You write your programs in a visual editor by dragging and dropping Actors onto Scenes, Rules onto Actors and Behavior onto Rules. For instance if we have a starship actor and we drop these rules onto it

We will get our familiar spaceship that can move left and right

GameSalad is the least familiar to me but seems to be easiest for my kids to start working on. Not having to write any “code” or “do programming” makes it much easier to get started. It also can create iPhone or iPad games and I would never dream of exposing my kids to Objective-C.

What’s next?

We’ve started experimenting with all three of these tools and so far are having fun with all three. Hopefully we’ll figure out what works for us and perhaps try to write about it again in a few months.

After writing this I came across a recent NY Times article Programming for Children, Minus Cryptic Syntax and Scratch also sounds interesting so I may have to look into that sometime too.

Twitter is the new RSS Reader

2011-11-07T00:00:00-08:00

In 2008 I thought RSS was an awesome way to stay abreast of what’s going on, but now its 2011 and I find myself using Twitter more often than Google Reader to find new and interesting articles people have written. Readers tweet and retweet articles they find interesting which seems a lower barrier than leaving an “I like this” comment. As an author Twitter also gives you some idea of who is reading your posts and a way to connect with them.

Back in 2008 I created a blog aggregator site http://waywework.it to group the all the people I work with and promote others to share their thoughts. I was so excited I even wrote an article about it.

Now that its 2011, I’ve been asking myself how could I update http://waywework.it for the twitter world of today?

I decide that if we’re going to follow people on twitter that’s what my site should facilitate. When new posts come in it should tweet them letting you see them if you follow @WayWeWorkIT.

Enabling API access to twitter

Once I had this I added the twitter gem to my application. I have to give a shout out John Nunemaker for writing this fantastic gem which made my task so simple.

In the Gemfile

gem 'twitter'

I created a new twitter account @WayWeWorkIT and registered an application at https://dev.twitter.com/apps so I had my OAuth and access tokens.

The only trick was I had to go Application Settings tab and configure it for Read and Write access then regenerate the tokens.

Now that I had the keys and tokens from twitter I had to tell my application to use them without hardcoding them in my code. This took two steps. First, configuring the app to read the tokens from the environment in config/initializers/twitter.rb. Yes I am somewhat paranoid about accidentally tweeting from development but that if Rails.env.production? should save me.

if Rails.env.production?
  Twitter.configure do |config|
    config.consumer_key       = ENV['TWITTER_CONSUMER_KEY']
    config.consumer_secret    = ENV['TWITTER_CONSUMER_SECRET']
    config.oauth_token        = ENV['TWITTER_OAUTH_TOKEN']
    config.oauth_token_secret = ENV['TWITTER_OAUTH_TOKEN_SECRET']
  end
end

Secondly, setting the tokens on the heroku environment (I typed the real tokens instead of the XXXXXXXX’s).

$ heroku config:add TWITTER_CONSUMER_KEY=XXXXXXXX
$ heroku config:add TWITTER_CONSUMER_SECRET=XXXXXXXX
$ heroku config:add TWITTER_OAUTH_TOKEN=XXXXXXXX
$ heroku config:add TWITTER_OAUTH_TOKEN_SECRET=XXXXXXXX

We can test it out (after deploying with git push heroku)

$ heroku console
>> Twitter.tweet('http://waywework.it aggregates blog articles')
=> # some big object returned
>> Twitter.user_timeline('wayweworkit').first.text
=> "http://t.co/FCmUQdc3 aggregates blog articles"

Great we just tweeted our first tweet for the world to see.

Updating WayWeWork.IT to tweet new posts

The app periodically scans the rss feeds it tracks and when it sees a new post it creates it in the app’s database.

First we add a twitter_username to each feed we’re tracking

class AddTwitterUsernameToFeeds < ActiveRecord::Migration
  def change
    add_column :feeds, :twitter_username, :string
  end
end

Then, add an after_create callback to tweet each time we create a new post.

class Post < ActiveRecord::Base
  after_create :tweet

  delegate :twitter_username, :to => :feed

  def twitter_username_with_at_sign
    "@#{feed_twitter_username || 'WayWeWorkIT'}
  end

  # See https://dev.twitter.com/docs/tco-link-wrapper/faq#Will_t.co-wrapped_links_always_be_the_same_length
  # We should query instead of hardcoding 20
  def short_url_length
    20
  end

  def tweet
    if Rails.env.production?
      non_title_part_of_tweet = " #{'x'*short_url_length} via #{twitter_username_with_at_sign}"
      max_title_length = 140 - non_title_part_of_tweet.length

      tweet = "#{title.truncate(max_title_length)} #{url} via #{twitter_username_with_at_sign}"
      Twitter.update(tweet)
    end
  end
end

Again with the “if Rails.env.production?” paranoia? You do know that you can never be too paranoid :)

With the twitter gem its one line to tweet Twitter.update(tweet). The rest of it is to shorten the title so twitter’s 140 character limit wont cut off the url or the author’s name.

Once this is in we’ll start seeing tweets like

Go ahead and follow @WayWeWorkIT on twitter and you’ll start seeing these blog posts.

Using BDD and the email_spec gem to implement Email

2011-10-31T00:00:00-07:00

When implementing email functionality, the email_spec gem is something I’ve decided I can’t live without. It makes it so easy to write RSpec specs and Cucumber features around your email that you have no excuse not to. Today I’m going to go through an example how I recently used BDD to send an email in an app I was working on.

When I think about email and my Rails environments this is how I typically want them to behave.

test should not send emails and allow us to write specs or features against them
development should not send emails but provide a UI to view what would be sent on localhost
staging should not send emails but provide a UI to view what would be sent on a server
production should send real emails to real people

Last week I wrote about using letter opener to View Sent Email on a Server (without actually sending anything) describing how the letter_opener gem helps us in the development and staging environments. This article focuses on using email_spec gem in the test environment.

Our Sample Project

Let’s imagine we are working on a new startup in stealth mode. We want to generate buzz and prepare for a beta launch. We’re hiding the fact its all vaporware with a splashy homepage where people can request an invitation to the beta and records their email in our database. This is the same example as my article last week and there’s a live demo at http://awesome-site-staging.heroku.com/.

What we’ll do today is not just record the email address but also send a “thanks for your interest” email to the user. We’re going to do this in BDD fashion bouncing back and forth between Cucumber Scenarios and RSpec Unit Tests

Writing a cucumber scenario
While the scenario fails
1. Write a failing rspec spec
2. Write code to make it pass

The Cucumber scenario tells us what should be accomplished and when it fails it we use that to tell us what unit test we should write.

Adding the email_spec gem

We add it to our Gemfile

group :test do
  gem 'email_spec'
end

We bundle and use the email_spec generator to let it initialize itself.

$ bundle
$ rails g email_spec:steps

There’s also a manual step to get cucumber to load the email_spec gem. We need to create a file features/support/email_spec.rb

require 'email_spec/cucumber'

Our first Cucumber Scenario

We know that when a user requests an invitation they should get an email so we write that requirement as a Cucumber Scenario. features/request_an_invitation.feature

Feature: Build excitement for this vaporware
  In order to drum up interest
  As a user
  I will receive an exciting email when I request an invitation

  Scenario: Someone requests an invitation and receives an email
    Given I am on the home page
     When I request an invitation for "gullible@lemmings.com"
     Then "gullible@lemmings.com" should receive 1 email
      And they open the email
      And they should see the email delivered from "alex@awesome-startup.com"
      And they should see "Invitation request for Awesome New Startup received" in the email subject
      And they should see "Dear gullible@lemmings.com," in the email text part body
      And they should see "We have received your request " in the email text part body
      And they should see "Please check back at http://awesome-site-staging.heroku.com" in the email text part body

We run it and it tells us we have a couple of missing steps.

1 scenario (1 undefined)
9 steps (7 skipped, 2 undefined)
0m0.006s

You can implement step definitions for undefined steps with these snippets:

Given /^I am on the home page$/ do
  pending # express the regexp above with the code you wish you had
end

When /^I request an invitation for "([^"]*)"$/ do |arg1|
  pending # express the regexp above with the code you wish you had
end

Oh right, the training wheels came off in cucumber-rails v1.1.1 and we don’t have web_steps.rb anymore. Let’s write these steps using capybara in features/step_definitions/invite_steps.rb

Given /^I am on the home page$/ do
  visit root_path
end

When /^I request an invitation for "([^"]*)"$/ do |email|
  visit root_path
  fill_in 'email', :with => email
  click_button 'Request Invitation'
end

We run one more time and get a failure we expect. Its telling us we haven’t written any code to implement the scenario yet!

  Then "gullible@lemmings.com" should receive 1 email
  expected: 1
       got: 0 (using ==) (RSpec::Expectations::ExpectationNotMetError)
  ./features/step_definitions/email_steps.rb:52:in `/^(?:I|they|"([^"]*?)") should receive (an|no|\d+) emails?$/'
  features/request_an_invitation.feature:9:in `Then "gullible@lemmings.com" should receive 1 email'

Dropping into RSpec unit tests

Our failing feature tells us what we need to implement so we drop down to the unit test level and start implementing it with TDD. The feature tells us an email should be be generated so let’s go. We’ll add to our invites_controller_spec specifying that it should create and deliver an InviteMailer. spec/controllers/invites_controller_spec.rb

require 'spec_helper'

describe InvitesController do
  describe 'PUT #update' do
    let(:invite) { mock(:email => 'someone@someco.com', :save => true) }
    let(:invite_mailer) { mock }
    before do
      InviteMailer.should_receive(:invite_requested).with(invite).and_return(invite_mailer)
      invite_mailer.should_receive(:deliver)
      Invite.should_receive(:new).with('email' => 'someone@someco.com').and_return(invite)
      post :create, :invite => { :email => 'someone@someco.com' }
    end
    it { should redirect_to root_url }
    it { should set_the_flash.to("Thanks for your interest someone@someco.com.  You will hear from us soon.") }
  end
end

Of course we get an error uninitialized constant InviteMailer. We fix that by creating the mailer (it doesn’t do anything yet) app/mailers/invite_mailer.rb

class InviteMailer < ActionMailer::Base
end

The error changes

Failure/Error: InviteMailer.should_receive(:invite_requested).with(invite).and_return(invite_mailer)
  (<InviteMailer (class)>).invite_requested(#<RSpec::Mocks::Mock:0x82c96210 @name=nil>)
      expected: 1 time
      received: 0 times

We add the code to our controller to create and deliver the email in app/controllers/invites_controller.rb

class InvitesController < ApplicationController
  def create
    @invite = Invite.new(params[:invite])
    if @invite.save
      InviteMailer.invite_requested(@invite).deliver
      redirect_to root_path, :notice => "Thanks for your interest #{@invite.email}.  You will hear from us soon."
    else
      render :action => "new"
    end
  end
end

Are we done?

Checking the Cucumber Scenario…

The RSpec unit tests pass now but the Cucumber features are still failing.

When I request an invitation for "gullible@lemmings.com"            # features/step_definitions/invite_steps.rb:5
  undefined method `invite_requested' for InviteMailer:Class (NoMethodError)
  ./app/controllers/invites_controller.rb:10:in `create'
  (eval):2:in `send'
  (eval):2:in `click_button'
  ./features/step_definitions/invite_steps.rb:8:in `/^I request an invitation for "([^"]*)"$/'
  features/request_an_invitation.feature:8:in `When I request an invitation for "gullible@lemmings.com"'

Duh our InviteMailer doesn’t do anything.

Back down to the unit tests

We write our spec/mailers/invite_mailer_spec.rb. We need to include some EmailSpec modules so we have access to its matchers.

require 'spec_helper'

describe InviteMailer do
  include EmailSpec::Helpers
  include EmailSpec::Matchers

  describe '.invite_requested' do
    let(:invite) { Factory.build :invite, :email => 'someone@someco.com' }

    describe 'one email to one user' do
      subject { InviteMailer.invite_requested(invite) }
      it { should deliver_to     invite.email                                                  }
      it { should deliver_from   'alex@awesome-startup.com'                                    }
      it { should have_subject   "Invitation request for Awesome New Startup received"         }
      it { should have_body_text "Dear someone@someco.com,"                                    }
      it { should have_body_text "We have received your request"                               }
      it { should have_body_text "Please check back at http://awesome-site-staging.heroku.com" }
    end
  end
end

Of course it fails because we still haven’t implemented anything. Let’s add some code to app/mailers/invite_mailer.rb

class InviteMailer < ActionMailer::Base
  def invite_requested(invite)
    @invite = invite
    mail :to      => invite.email,
         :from    => 'alex@awesome-startup.com',
         :subject => 'Invitation request for Awesome New Startup received'
  end
end

and we can use haml to format the body in app/views/invite_mailer/invite_requested.text.haml

== Dear #{@invite.email},
We have received your request to be invited into our awesome site. We'll let you know as soon as its available.
Please check back at http://awesome-site-staging.heroku.com
You must be very excited!
Thanks
An Awesome New Startup

We run the rake one more time and …

We’re Done

Everything passes - the specs and the features!

$ rake
ruby -S rspec <a long list of _spec.rb files>
............

Finished in 2.64 seconds
12 examples, 0 failures

ruby -S bundle exec cucumber  --profile default
Using the default profile...
Feature: Build excitement for this vaporware
  In order to drum up interest
  As a user
  I will receive an exciting email

  Scenario: Someone requests an invitation and receives an email
    Given I am on the home page
    When I request an invitation for "gullible@lemmings.com"
    Then "gullible@lemmings.com" should receive 1 email
    And they open the email
    And they should see the email delivered from "alex@awesome-startup.com"
    And they should see "Invitation request for Awesome New Startup received" in the email subject
    And they should see "Dear gullible@lemmings.com," in the email body
    And they should see "We have received your request" in the email body
    And they should see "Please check back at http://awesome-site-staging.heroku.com" in the email body

1 scenario (1 passed)
9 steps (9 passed)
0m0.293s

I hope you’ll consider using the email_spec gem and BDD the next time you have to add email to your app.

Using Letter Opener to View Sent Email on a Server (without actually sending anything)

2011-10-24T00:00:00-07:00

When developing email functionality you don’t want to send real emails to real people before in production. At the same time you need to send them to ensure they are formatted correctly and contain the proper information. You can (and should) write integration tests to verify this but that helps developers gain confidence, what can we do to show non-technical stakeholders that it all works?

Today I’m going to show you how to use Ryan Bates’ letter_opener gem to let you preview your emails without actually sending them. Ryan of course does the always awesome RailsCasts.

Let’s think about the different Rails environments and how we want them to behave with email.

test should not send emails and allow us to write specs or features against them
development should not send emails but provide a UI to view what would be sent
staging should not send emails but provide a UI to view what would be sent
production should send real emails to real people

Looking at this, Rails works really well for test with ActionMailer’s :test delivery method that stores the emails in the ActionMailer::Base.deliveries array so you can then use email_spec to write your specs. Production is also covered as long as you give it your mail server configuration. That leaves development and staging which look identical but we’ll see that they are slightly different. I’ll spend the rest of this article talking about how letter_opener lets us do what we want in these environments.

An example

Let’s imagine we are working on a new startup in stealth mode. We want to generate buzz and prepare for a beta launch. We’re hiding the fact its all vaporware with a splashy homepage where people can request an invitation to the beta and it sends a “thanks for your interest” email. Lastly, we want to test it in development and staging.

Here’s a live demo at http://awesome-site-staging.heroku.com/ if you want to dive in and start clicking.

After you request an invitation it shows a page like this

Logfile testing (we can do better)

When you fill in your email and click the Request Invitation button, our controller uses a mailer to create and deliver the email.

class InvitesController < ApplicationController
  def create
    @invite = Invite.new(params[:invite])
    @invite.save
    InviteMailer.invite_requested(@invite).deliver
    redirect_to root_path, :notice => "Thanks for your interest #{@invite.email}.  You will hear from us soon."
  end
end

The only way to tell whether the email worked is to scroll through the development.log until you see something like this

Sent mail to alex@alexrothenberg.com (20ms)
Date: Fri, 21 Oct 2011 15:09:16 -0400
From: admin@newstartup.com
To: alex@alexrothenberg.com
Message-ID: <4ea1c35cc3d5b_6693830916bc43754@Alex-Rothenbergs-MacBook-Pro.local.mail>
Subject: Invite requested
...
We have received your request to be invited into our awesome site. We'll let you know as soon as its available.
Please check back at http://awesome-site.heroku.com
You must be very excited!
Thanks
An Awesome New Startup

Ok if you’re a developer and you enjoy reading log files but letter_opener lets us do better.

Using letter_opener in development

Letter_opener provides us with a UI so we can view the emails right in our browser. It’s super easy to add this gem and I’ll just copy the instructions from its README

Preview email in the browser instead of sending it. This means you do not need to set up email delivery in your development environment, and you no longer need to worry about accidentally sending a test email to someone else’s address

Rails Setup

First add the gem to your development environment and run the bundle command to install it.

gem "letter_opener", :group => :development

Then set the delivery method in config/environments/development.rb

config.action_mailer.delivery_method = :letter_opener

Now any email will pop up in your browser instead of being sent. The messages are stored in tmp/letter_opener.

Once we’ve done that what happens when we use the site to request an invitation? A new tab opens up with the email right there. Now as a user we can tell it’s correct and any non-technical people on the team can feel their confidence rise.

How does letter_opener actually work?

Rails goes through the standard flow to create the mail object and when it’s ready to deliver the message it calls letter_opener’s deliver! method because we registered letter_opener as the action_mailer.delivery_method. Letter_opener saves the email to your file system as an html file then uses launchy to open it in a browser using the file:// protocol.

There will be a couple of problems once we move onto a server which brings us to staging.

Using letter_opener on staging

If you’re like me you probably have a staging environment where you or your stakeholders can validate your app before releasing it to production and your end users. There are two aspects of this environment that wont work with letter_opener the way it did in development

We need to use http:// not file:// to preview the emails because the browser is not on the same file system where the emails are written
We may not be able to write to the file system. For example if we have deployed to heroku.

I had to make some changes to letter_opener to support this kind of server environment. The fork is available at https://github.com/alexrothenberg/letter_opener/tree/on_a_server and I’ll update the article if my pull requests are merged back in.

We need to make a few changes to our application.

1 - Update our Gemfile to use the fork from github

  gem 'letter_opener',  :git => "git://github.com/alexrothenberg/letter_opener.git", :branch => "on_a_server"

2 - Add a debugging UI link so users can get to the “preview emails” page in something like layouts/application.html.haml

  = link_to 'Preview Emails', letter_opener_letters_path if Rails.env.staging?

3 - If you cannot write to the filesystem let letter_opener know in your config/environments/staging.rb

  config.action_mailer.delivery_method = :letter_opener
  LetterOpener.cannot_write_to_file_system!

Now we can see it all in action.

First, we request an invite.

Then, we click the link “view the Emails that users would have received” link at the bottom and see an “inbox” of everything the app sent.

Fincally clicking one message lets us preview it just as we did in devellopment

We are able to run on heroku without writing to the file system by using the FakeFS gem which simulates the filesystem in memory. FakeFS is designed to be used for testing and one caveat to be aware of is that your old emails will disappear if heroku recycles your dyno due to inactivity.

I hope you think letter_opener is useful and give it a try the next time you need to send email. Remember here’s a live demo at http://awesome-site-staging.heroku.com/ of the site we’ve been talking about here.

Chaining ActiveRecord Scopes from Different Models

2011-10-17T00:00:00-07:00

You probably know you can chain ActiveRecord scopes together and it combines it all together into one sql statement, but did you know that you can also use scopes from associated models in your chain?

Let me show you with an example. Let’s say we have a discussion site where users can post comments with two simple models:

class User < ActiveRecord::Base
  has_many :comments
end

class Comment < ActiveRecord::Base
  belongs_to :user
end

We could display a list of all users with User.all or even display them alphabetically with User.order(:name) (yes I’d create a scope if I were doing this for real).

What if I wanted to display the users sorted so that the ones with the most recent comment was first?

User.includes(:comments).merge(Comment.order('comments.created_at desc'))

There’s a lot going on there, let’s break it down.

User.includes(:comments) - tells active record to query both the users and comments tables
Comment.order('comments.created_at desc') - sorts the results by the date of the comment (we need to specify the table and column name since created_at is also a column on the users table)
merge(Comment.XXX) - lets us use a scope from the Comment model even though we’re dealing with Users

When ActiveRecord and ActiveRelation take all this and convert it into a sql statement it will join the users and comments tables and order by the comments.created_at column. Here’s the sql I actually get in irb (boy am I glad I didn’t have to type that sql myself!).

> User.includes(:comments).merge(Comment.order('comments.created_at desc'))
  SQL (0.4ms)  SELECT "users"."id" AS t0_r0, "users"."name" AS t0_r1, "users"."created_at" AS t0_r2,
                      "users"."updated_at" AS t0_r3, "comments"."id" AS t1_r0, "comments"."user_id" AS t1_r1,
                      "comments"."body" AS t1_r2, "comments"."created_at" AS t1_r3, "comments"."updated_at" AS t1_r4
               FROM "users"
               LEFT OUTER JOIN "comments" ON "comments"."user_id" = "users"."id"
               ORDER BY comments.created_at desc

Adding Scopes to make it usable

It works to type all that but in a real application you’d add scopes to make it easier to work with. Let’s do that!

class User < ActiveRecord::Base
  has_many :comments
  scope :by_most_recent_comment, includes(:comments).merge(Comment.most_recent_first)
  scope :with_recent_comments, includes(:comments).merge(Comment.recently(1.month.ago))
end

class Comment < ActiveRecord::Base
  belongs_to :user
  scope :most_recent_first, order('comments.created_at desc')
  scope :recently, lambda { |date| where('comments.created_at >= ?', date) }
end

Now we can write some nice simple scopes like

User.by_most_recent_comment to get all users sorted so the ones with recent comments are at the top
User.with_recent_comments to get all users who have commented in the past month
User.with_recent_comments.by_most_recent_comment to get users who have commented in the past month sorted by the date of their most recent comment.

Happy scoping!

Alex Rothenberg

The Magic Behind Angularjs Dependency Injection

AlexRothenberg

The "Magic" behind AngularJS Dependency Injection

An AngularJS application using the GitHub API

Dependency Injection is magical

How Dependency Injection implements Named Parameters in JavaScript

Minification breaks our App

Combining D3 and Ember to Build Interactive Maps

An outline map of the States in the USA

Coloring the States red or blue

Closures can provide encapsulation when Creating Objects in Javascript

Object Literal

Encapsulation via a Closure

Building a Browser IDE

HTML IDE

HTML and CSS IDE

HTML, CSS and Javascript IDE

Syntax highlighting

Testing IP Whitelisting in your Specs and Features

Implementation

Unit Testing the IP Whitelist with RSpec

Integration Testing the IP Whitelist in a Cucumber Feature

Testing is good

Programming With Kids

Shoes

Gosu

gamesalad

What’s next?

Twitter is the new RSS Reader

Enabling API access to twitter

Updating WayWeWork.IT to tweet new posts

Using BDD and the email_spec gem to implement Email

Our Sample Project

Adding the email_spec gem

Our first Cucumber Scenario

Dropping into RSpec unit tests

Checking the Cucumber Scenario…

Back down to the unit tests

We’re Done

Using Letter Opener to View Sent Email on a Server (without actually sending anything)

An example

Logfile testing (we can do better)

Using letter_opener in development

How does letter_opener actually work?

Using letter_opener on staging

Chaining ActiveRecord Scopes from Different Models

Adding Scopes to make it usable