Jacob Swanner

PostgreSQL MERGE and Sequences

2022-10-30T00:00:00+00:00

Recently released PostgreSQL version 15 contains a new command: MERGE. We can use MERGE to do “upsert” operations, much like when providing an ON CONFLICT clause to INSERT, but MERGE can be used for other tasks as well. There’s an interesting difference when doing upserts between the two commands, and that’s what we’re going to discuss here.

Using `INSERT` … `ON CONFLICT`

Suppose we have a table defined as (note the unique email constraint):

CREATE TABLE people (
  id bigserial PRIMARY KEY,
  email citext,
  name text,
  UNIQUE(email)
)

After the table is created, we can run the follow INSERT with ON CONFLICT 4 times. The first time will insert a new record, and the other times will update the name since the email conflicts with the existing record:

INSERT INTO people (email, name)
VALUES ('alex@email.com', 'Alex')
ON CONFLICT (email) DO UPDATE SET name = EXCLUDED.name

Then, we run the following query to insert another record:

INSERT INTO people (email, name)
VALUES ('billie@email.com', 'Billie')
ON CONFLICT (email) DO UPDATE SET name = EXCLUDED.name

Afterwards, we query the table for all of its records:

SELECT * FROM people

 id |      email       |  name
----+------------------+--------
  1 | alex@email.com   | Alex
  5 | billie@email.com | Billie

As we can see, there’s a gap in the value of the IDs, and that’s because the sequence used for the id column is advanced with every INSERT query even it ends up doing an update via ON CONFLICT. Because of that behavior, it’s highly suggested to use bigint/bigserial for auto-incrementing columns when you are going to upsert data into the table.

Using `MERGE`

Now, let’s see what happens if we use MERGE instead. We’ll keep the table as it was, and we’ll upsert that last record several times using a MERGE query:

MERGE INTO people
USING (VALUES ('billie@email.com', 'Billie')) AS input(email, name)
ON people.email = input.email
WHEN NOT MATCHED THEN
  INSERT (email, name) VALUES (input.email, input.name)
WHEN MATCHED THEN
  UPDATE SET name = input.name

If we then use MERGE to insert a new record:

MERGE INTO people
USING (VALUES ('charlie@email.com', 'Charlie')) AS input(email, name)
ON people.email = input.email
WHEN NOT MATCHED THEN
  INSERT (email, name) VALUES (input.email, input.name)
WHEN MATCHED THEN
  UPDATE SET name = input.name

And check all the records:

SELECT * FROM people

 id |       email       |  name
----+-------------------+---------
  1 | alex@email.com    | Alex
  5 | billie@email.com  | Billie
  6 | charlie@email.com | Charlie

We now see that there’s no gap with the ID of the last two records, and that’s because our sequence was not advanced when the MERGE was used as an update task. Granted, the MERGE queries are more verbose than the INSERT…ON CONFLICT equivalents, but I believe that verboseness also provides clarity into the intentions of the query. I should also note that neither Ecto nor ActiveRecord currently provide first class support for MERGE.

NimbleCSV: Parsing into Elixir Maps

2022-07-15T00:00:00+00:00

NimbleCSV is a great option when parsing CSV files in Elixir. By design, it will only give you the data for each row as a List – there’s no option for it to generate a Map (using the headers as keys) for each row. When processing data from CSV files, I much prefer to use maps instead of lists, that way the order of the columns in the file doesn’t matter. Luckily, we can using Stream.transform/3 to generate a map for each row of data:

alias NimbleCSV.RFC4180, as: CSV

"path/to/file.csv"
|> File.stream!()
|> CSV.parse_stream(skip_headers: false)
|> Stream.transform(nil, fn
  headers, nil -> {[], headers}
  row, headers -> {[Enum.zip(headers, row) |> Map.new()], headers}
end)
|> Enum.to_list()

And, that’s it! Quite a bit is happening in a few lines of code. For the initial accumulator, we need to use something that will not match the first row of the CSV, so nil or some other atom is a great choice here. We match on our initial accumulator in the first clause of the function to Stream.transform/3 to know we’re on the headers row and we’ll use the headers as the accumulator from thereon. For the rest of the rows, since we know the order of the headers will correspond to the order of the values in each row, we zip the headers and row values together into a list of {key, value} tuples and then turn that list into a map. After that we have an Enumerable of maps that we can use for processing the data.

To see it in action, I’ll use a String instead of a File:

alias NimbleCSV.RFC4180, as: CSV

"""
name,age
Alex,21
Billie,8
Charlie,32
"""
|> CSV.parse_string(skip_headers: false)
|> Stream.transform(nil, fn
  headers, nil -> {[], headers}
  row, headers -> {[Enum.zip(headers, row) |> Map.new()], headers}
end)
|> Enum.to_list()
# [
#   %{"age" => "21", "name" => "Alex"},
#   %{"age" => "8", "name" => "Billie"},
#   %{"age" => "32", "name" => "Charlie"}
# ]

A similar approach can be taken with Enum.flat_map_reduce/3, if you really wanted:

alias NimbleCSV.RFC4180, as: CSV

"""
name,age
Alex,21
Billie,8
Charlie,32
"""
|> CSV.parse_string(skip_headers: false)
|> Enum.flat_map_reduce(nil, fn
  headers, nil -> {[], headers}
  row, headers -> {[Enum.zip(headers, row) |> Map.new()], headers}
end)
|> elem(0)
# [
#   %{"age" => "21", "name" => "Alex"},
#   %{"age" => "8", "name" => "Billie"},
#   %{"age" => "32", "name" => "Charlie"}
# ]

The call to Kernel.elem/2 at the end is there to grab the resulting enumerable and ignore the accumulator.

Production treatment of scripts

2018-04-20T00:00:00+00:00

Applying production code treatment to utility scripts

Those single-file, single-purpose scripts that you might occasionally create are still worthy of the treatment you give production applications. Specifically, the use of explicit dependency declarations and behavior-driven development, and we can do this without adding any additional files.

Background

A few months back, I was tasked with taking some Microsoft Word DOCX files and incorporating their contents into an internationalized, front-end application. Each DOCX file contained the verbiage to use for a particular language, and the end goal was to populate JavaScript locale files with the contents. I found a RubyGem - appropriately named “docx” - to parse DOCX files, and after fumbling around in IRB for a while, I found a sequence of commands that produced the output I desired. I then repeated this sequence of commands for all the files and incorporated the results into the locale files, which completed the task at hand. More recently, I was tasked with updating those same locale files from a new set of files, and I immediately regretted that I had not created a better system for handling this task the previous time. I was resolved to write a script which handled the bulk of this largely repetitive workload.

Utility Scripts

While the processing of these DOCX files is certainly not a responsibility of the application I was working on - instead it’s more of a consequence of how we receive these data from our client - building a utility script to automate as much as possible is still a worthwhile endeavor. When writing these kinds of utility scripts, I often turn to Ruby, as it’s a language I know well and it’s well suited for this type of task. Historically, I’ve had a couple gripes with these scripts: first, I tend to develop them by running them over and over, with small changes between each invocation. Second, it can be tricky to ensure I have the correct version of a dependency installed when I want to use the script months, or years, after it was created. While our applications could potentially suffer from similar problems, they are mitigated through the practice of behavior-driven development for the former problem and dependency management tools for the latter. Why not apply the same techniques and tools when building single-file utility scripts?

Dependency Management

For dependency management in Ruby, we generally use Bundler, where our dependency declarations are all put into one file - typically named Gemfile. For a single-file script, having a separate file for its dependencies seems awkward. Luckily, Bundler has a trick up its sleeve that fits our needs perfectly: bundler/inline. It provides a gemfile method, where we can declare our dependencies using the same interface as Gemfiles.

require 'bundler/inline'

gemfile do
  gem 'rspec', '3.7', require: false
end

Now that we’re able to declare the dependencies for our scripts, we no longer need to worry about their durability when we come back to them months, or years, later.

Behavior-Driven Development (BDD)

The same benefits we receive from using BDD while building our applications can be applied to building scripts. A suite of RSpec examples is usually contained in files that are separate from those of the implementation, but again, that approach is awkward for a single-file script. While I haven’t come across a part of RSpec purpose-built for our use case, we can take advantage of RSpec’s runner to differentiate between how our script file is being used. That is, we’ll inspect the $PROGRAM_NAME global (also aliased as $0) and the __FILE__ constant; if they are the same then the script itself is being executed, otherwise it’s being loaded from another program - RSpec in our case. That will allow us to have our script code and our RSpec examples all in the same file, and the examples will only be evaluated when the script is loaded by RSpec’s runner.

# script logic

if $PROGRAM_NAME == __FILE__
  # the script itself is being executed
  exit
end

require 'rspec'

RSpec.describe 'the script' do
  it 'is expected to do something'
end

Final Script

Now we just need to put it all together into an actual script. For the sake of an example, we’ll use a simplified recreation of Unix’s wc utility, which counts lines, words and characters of the text passed in through standard input (STDIN):

#! /usr/bin/env ruby

require 'bundler/inline'

gemfile do
  gem 'rspec', '3.7', require: false
end

class Program
  def initialize(input)
    @input = input
  end

  def call
    "%8d%8d%8d" % [lines, words, characters]
  end

  def characters
    @input.size
  end

  def lines
    @input.split(/\n/).size
  end

  def words
    @input.split(/\s+/).size
  end
end

if $PROGRAM_NAME == __FILE__
  puts Program.new(STDIN.read).call
  exit
end

require 'rspec'

RSpec.describe Program do
  subject(:program) { described_class.new(input) }
  let(:input) { "line 1\nline 2\nline 3\n" }

  it { expect(program.call).to eq('       3       6      21') }
  it { expect(program.characters).to eq(21) }
  it { expect(program.lines).to eq(3) }
  it { expect(program.words).to eq(6) }
end

With that, we can run our specs:

$ rspec wc.rb
....

Finished in 0.00348 seconds (files took 0.19061 seconds to load)
4 examples, 0 failures

And, we can run the script itself:

$ chmod +x wc.rb
$ echo "hello, world" | ./wc.rb
       1       2      13
$ printf "line 1\nline 2\n" | ./wc.rb
       2       4      14

`exit 0`

Hopefully, you’ll be able to use these tricks when writing your next utility script. With them, you’ll be able to take advantage of BDD while developing the script, and you can be confident about its use of dependencies long after it was written.

Rake: Task Arguments

2017-06-06T00:00:00+00:00

There are times when we want to pass variable data to our tasks; whether it’s to act as a switch between different behaviors, or if it’s to provide needed input for our task to run properly. Whatever the case, it is a useful thing to know about when creating Rake tasks.

This is the fourth post in a series on Rake and I believe this post is the missing piece to complete the series. The topics of the previous posts include: an introduction to the Rakefile format, file tasks, and rule tasks.

The ENV-workaround

It’s quite common to use environment variables in order to pass variable data to Rake tasks – and this is how Rails’ Rake tasks do it.

Let’s start by declaring a task called first which will read the VAR environment variable:

task :first do
  puts "VAR in `first` is #{ENV['VAR'].inspect}"
end

We’re using #inspect here so that the value of our argument is explicitly clear.

Now let’s look at running this task in a shell:

$ rake first
VAR in `first` is nil
$ VAR=a rake first
VAR in `first` is "a"

As you can see in the first invocation, if the environment variable is not set, then the ENV object will return nil for our VAR name. In the second invocation, we’re using our shell’s facility to set an environment variable while running a program, and that value is available to us while our task is running.

None of this is Rake-specific; it’s just Ruby running on a Unix-like system. That said, Rake does provide another way of setting environment variables:

$ rake first VAR=b
VAR in `first` is "b"

As you can see, Rake will add options in the VAR=VALUE format to the ENV object.

Proper Task Arguments

The example above shows a common approach you will see for how to supply variable data to Rake tasks, but Rake actually has real support for task arguments.

You declare task arguments with an Array as the second parameter to the task method, and the values are accessible via the second parameter given to our action block:

task :second, [:var] do |task, args|
  puts "var in `second` is #{args.var.inspect}"
end

When invoking a task, parameters are placed inside of square brackets after the name of the task:

$ rake second[c]
var in `second` is "c"

Arguments are not required, your task will need to handle potential nil values:

$ rake second
var in `second` is nil

But again, Rake provides a handy way of setting default values for our task arguments, by called #with_defaults on our args object:

task :third, [:var] do |task, args|
  args.with_defaults(var: 'default')
  puts "var in `third` is #{args.var.inspect}"
end

Now when our task is invoked, we no longer need to worry about nil argument values:

$ rake third
var in `third` is "default"
$ rake third[d]
var in `third` is "d"

Arguments & Prerequisites

You might be wondering to yourself: how do you declare a task that has arguments and prerequisites? Well, you pass a Hash as the second parameter to the task method; the key of the Hash contains your task’s arguments and the value contains the prerequisites.

Let’s declare a task called fourth that has an argument called var and depends on our task called third:

task :fourth, [:var] => [:third] do |task, args|
  args.with_defaults(var: 'fallback')
  puts "var in `fourth` is #{args.var.inspect}"
end

Parameters given to your task will be passed to its prerequisites as well:

$ rake fourth
var in `third` is "default"
var in `fourth` is "fallback"
$ rake fourth[e]
var in `third` is "e"
var in `fourth` is "e"

In the above invocations, the first line of output comes from the third task, and the second line comes from the fourth task.

Useful Example

Each post in this series has tried to include a useful example of a task using the aspect of Rake described in the post. But, for this post, I’m going to reference back to the example in the first post, as it used task arguments as well.

London

2016-11-19T00:00:00+00:00

I spent the last week of this European trip in London. It was my second time in London, but my previous stay there had been very brief, only a couple days. So, I was certainly excited to see more of the city, and it did not disappoint.

On Saturday – the day after arriving – I joined 2 free walking tours: one of street art in East London and another of the City of London. Both were great, very different from each other, and would recommend both. On Sunday, I booked a bike tour, from the same company that I had used in Berlin, and it turns out they run a much smaller operation in London. Of the tours I did in London, the bike tour definitely covered more of the touristy spots, which is probably why it was the tour I enjoyed the least, even though it was a fine tour.

A friend, and former coworker, Eric lives in London. I hadn’t seen him in a number of years, so it was cool to be able to catch up with him. His company rented desk space from a WeWork location in East London, quite close to where I was staying, so I was able to meet up with him and his coworkers a couple times during my stay.

Edinburgh

2016-11-06T00:00:00+00:00

After leaving France, I took a train – through the chunnel – to London, from there I took another train to Edinburgh, Scotland. I almost missed the Eurostar train in Lille – the one that took me to London – because it took me way longer to find the car rental return location than I thought it would. The tracks that Virgin Trains East Coast uses – which I took between London & Edinburgh – are not what I call smooth, the carriage jostles around a lot. I was constantly worried about stuff falling over or spilling, but the service in first class was probably the best I’ve experienced – first class on a train is not the same kind of difference as on a plane.

It was my first time in Scotland, and I loved the beauty of the country. I don’t think Scotland is usually associated with good food, but I had some lovely meals and drinks while I was there. Edinburgh itself seemed really cool, even though it was probably the dirtiest of all the places I visited during this trip – I can’t believe how much literal shit I saw on the streets & sidewalks. I hiked up Arthur’s Seat, which provided great views of Edinburgh and the surrounding area. But, I think my favorite parts of my time in Scotland would be the two tours I took that got me out in the Scottish countryside.

Wissant

2016-11-02T00:00:00+00:00

After leaving Amsterdam, I spent a week in northern France. This week was the only week of pure vacation during my time in Europe, actually the only week of vacation during all of my nomadic travels thus far.

I was in the area for the wedding of some friends (Guylain & Amandine). My mom and brother traveled to France for the wedding as well. Guylain’s sister, Valérie, rented a large house in the coastal village of Wissant for her extended family to stay in, and my family members and I were lucky enough to be able to stay there as well.

While there, in addition to attending the wedding and associated celebrations, we also hiked up one of the nearby coastal capes, which provided a spectacular view. We also visited some of the nearby cities (Boulogne & Calais) and an aquarium.

I spent the last day of that week in Pontruet, staying other friends in the same extended family. I always enjoy visiting with all these friends in France – it’s like visiting with relatives – which probably explains why I do it so often.

Berlin

2016-11-02T00:00:00+00:00

After Copenhagen, I went to Berlin, Germany. I kind of can’t believe that had only been my third time there, I feel like I’ve spent so much time there. Such a great city, with so much to do, and it’s cheap (way cheaper than Stockholm & Copenhagen).

On my previous visits to the city, I never did any kind of formal tour, on this visit I did two: a free walking tour of the street art in Kreuzberg and a bike tour outside of the typical tourist places. Both were great, would recommend them to any one visiting the city.

Kreuzberg has got to be the most hipster place I’ve ever been, I love it none the less.

Amsterdam

2016-11-02T00:00:00+00:00

I spent the first week of July in Amsterdam, Netherlands. While there, I continued the bike tour trend that I started in Berlin. First, I joined a city bike tour then a couple days later a countryside bike tour. The city bike tour was fine, but I really enjoyed – and recommend – the countryside bike tour.

I was in Amsterdam for the 4th of July. I spent the day on the aforementioned countryside bike tour,visiting the Dutch Resistance Museum, and then having a dinner at a restaurant known for serving traditional Dutch cuisine. I know, I had a very patriotic-American day in Amsterdam.

Copenhagen

2016-11-01T00:00:00+00:00

After Stockholm, I spent a week in Copenhagen, Denmark. This was the third time I had been to Copenhagen, and much like my previous visits to the city: I loved it, it’s certainly one of my favorite cities in Europe.

This time around, I visited places I hadn’t in the past: Kastellet (or The Citadel), Freetown Christiania, and the National Museum of Denmark. I really enjoyed Copenhagen Street Food on PapirØen (Paper Island), so much so that I went back a second time.

On my last night there, I got caught out in the rain while out getting dinner. This was just continuing a trend that started in Chicago, and also happened to be in Stockholm.

My departing train from Copenhagen was 7 minutes late, and I only had a 15 minute connection scheduled in Hamburg, Germany, which made me a bit nervous, but ended up being fine. I must have been in an engine carriage, because it was much louder in the carriage than I’m accustomed to for European trains. A positive thing to say about that train ride: it was really cool when the train pulled into a ferry, and it took us from Denmark to Germany.

Stockholm

2016-10-21T00:00:00+00:00

Over the summer, I spent 7 weeks traveling around Europe, visiting 7 different places, 1 week in each place. My first stop in Europe was Stockholm, Sweden. On 2 different occasions, I have briefly visited Malmo, Sweden, but this was my first time in Stockholm.

While in Stockholm, I had the chance to catch up with a friend from middle school, whom I hadn’t seen since middle school. While it was quite crazy to see him again, I really enjoyed hanging out with him.

During my time there, I learned of a couple Swedish cultural things that I didn’t know before going there. The first one I discovered while trying to figure out how to wash my clothes. I knew ahead of time that my Airbnb rental did not include laundry facilities, so I assumed that there would be a laundromat nearby, but when it came time for me to clean my clothes I could not one. This led me to search the internet how residents of Stockholm wash their clothes, turns out apartment buildings there have shared laundry facilities in the basement, but my Airbnb host did not provide me with the key so that I could use those machines. From a travel forum frequented by Brits and Americans I learned about the only laundromat in all of Stockholm, and it was on the other side of town from where I was staying.

Another thing I did not know about was that Sweden has a civil defense siren system, nor did I know that one day a year at 3pm they test that siren system nation-wide, and I just happened to be there that particular day of the year. That was quite a shock to me when I heard a sound I had always associated with an air raid siren, and I think I’ve really only heard it in movies, prior to that day.

The last thing that I can think of that I learned while in Stockholm, was that when you order pizza there, it is served with pizzasallad, which is similar to a mild sauerkraut.

I left Stockholm by train, which gave me the opportunity to see lots of the Swedish countryside. It was quite beautiful, with lots of birch trees and red barns.

Video Player Debugging Story

2016-10-14T00:00:00+00:00

This is the story of when Dray, Nate, and I spent the better part of two days trying to figure out a bug. Hopefully, you won’t run into this same bug with your applications, but if you do perhaps this can be of some help.

Shortly after the release of iOS 10, we heard users of one of our web applications were no longer able to play videos. We were quickly able to reproduce the issue, and observed that videos were not working with iOS 10, but were working as expected with iOS 9. This is when the journey to find the root of the problem began, what we discovered was more bizarre than expected.

To back up briefly, I should explain how these videos are served, as they are not just requested using a publicly accessible URL. The videos are stored on Amazon S3 and delivered by CloudFront. To restrict video access, the source URL given to the video player is a URL to our application, where authorized users are redirected to a short-lived, signed CloudFront URL.

The first step we took to find the problem was to check if any errors were reported to the browser’s JavaScript console. In order to do so using the tools Apple provides, it means connecting an iOS device to a computer and then use the developer tools of Safari on the computer. Upon doing so, we noticed messages like the following in the console (which didn’t help us very much):

VIDEOJS:
"ERROR:"
"(CODE:4 MEDIA_ERR_SRC_NOT_SUPPORTED)"
"The media could not be loaded, either because the server or network failed or because the format is not supported."

Next step we took was to look at the network requests being sent from the phone’s browser. Unfortunately, that didn’t help us much either, because Safari doesn’t show request headers, response headers, nor response status codes for video requests.

At this point, not knowing where to look for the problem, a couple of my coworkers started making ever increasingly complex test pages, trying to find the specific situation that caused the problem. We spent a lot of time thinking the problem was in the redirection, perhaps when combined with HTTPS requests, but were never able to actually pin down the problem.

After not being able to create a standalone test page that reproduced the issue in iOS 10 devices, we decided to look elsewhere for the problem. We still really wanted to see the details of the requests for the video files, but with Safari’s developer tools not providing those details, something else was needed. We wanted to use an HTTP proxy, but there were a couple problems with doing that: the phone’s network traffic would need to flow through the computer running the HTTP proxy, and we were dealing with HTTPS requests (which means the requests were encrypted).

To solve the first of those problems, my phone needed to use my computer as its wireless access point; which brings up another problem: a Mac can use its WiFi radio to connect to a wireless network or to create a wireless network, not both at the same time, but my computer would still need access to the Internet. Luckily, I own a Thunderbolt Ethernet Adapter and after finally finding an Ethernet port in our office, the first problem was solved: my computer would connect to the Internet via Ethernet and then create a wireless network for my phone to use.

As for our second HTTP proxy problem (reading encrypted HTTPS traffic), we were nicely surprised to learn that the HTTP Proxy we were wanting to use already had a solution to that. We were using Charles Web Debugging Proxy, and it includes instructions on how to capture and display HTTPS traffic, which mostly revolves around installing a root SSL certificate on the computer and on the phone. This is certainly not something you’d want to do under normal circumstances, as we were essentially performing a Man-in-the-middle attack on ourselves.

After getting all of that set up, it didn’t take us long to narrow in on the problem. The first thing we noticed was that the request to our application for the video, which we expected to respond with a redirect to CloudFront, was responding with 401 Unauthorized status. The next thing we noticed was that the Cookie HTTP header in the request didn’t look right, it contained much less data than all the other requests to our application. After that, one of my coworkers discovered a bug had been reported with WebKit, the underlying engine in Safari and many other browsers.

So, the problem ended up being that the browser was not sending the cookies our application was relying on when making requests for media elements.

If your application has a similar problem, the solution will be application specific: wait for Safari to be fixed, remove authentication/authorization from those requests, or use a different authorization scheme. As of this writing, Safari on iOS 10 has still not been fixed.

Hindsight is 20/20

Looking back, there were some things that could have helped us narrow in on the problem sooner. In production, we direct all of our log messages to Logentries, and had we looked more closely we would have seen the 401 Unauthorized responses for the video requests. That would not have told us why our application was responding that way, but at least it would have narrowed our focus.

What would have been of most help is if Safari showed request and response headers for video requests. It shows these details for other request types, and Google Chrome shows these details for video requests. If that data was presented in the developer tools, we would not have needed to use an HTTP proxy.

Atlanta

2016-09-27T00:00:00+00:00

On 4 June, things came full circle, geographically speaking; by returning to Atlanta my drive around North America formed a complete loop. I stayed in Atlanta until 11 June, at which time I flew to Europe, where I stayed for 7 weeks. During this time in Atlanta, I also met up for a friend from college, Rachel, a couple times. Atlanta has a lot of really great restaurants, bars, and breweries – I think a lot of people underestimate it in this respect.

While there, I stayed with old friends (Kevin & Robert), but if that wasn’t generous enough of them, they also took care of my car while I was in Europe. They had told me before that they were thinking of purchasing a Tesla, and I was hoping by leaving it with them that they’d get to experience what it’s like having one for an extended period of time, but I don’t think they drove it all that much.

Washington D.C.

2016-09-11T00:00:00+00:00

After leaving Boston, I spent the following week (28 May - 4 June) in the Washington D.C. area, more specifically, I stayed in Arlington, VA. I hadn’t intentionally planned to be in D.C. for Memorial Day weekend, just how it turned out. But, it was nice having Monday off, which meant I had an extra day to go to museums and such. Speaking of museums, while I was there I went to a couple of old favorites: National Air & Space Museum and the National Museum of Natural History; along with a couple I don’t remember going to before: Smithsonian Arts & Industries Building and the Smithsonian Castle.

While there, I had the pleasure of spending a couple evenings with Todd, Marianna & their children. I also got to hang out with Ashley a few times, one of those times she likes to say that we saved a man’s life. I would say she tended to an unconscious man while I called 911, and the police arrived in under a minute.

Montreal

2016-09-11T00:00:00+00:00

Another in what should be a long line of entries written months after the fact. I was in Montreal from 15 to 21 May. It was not my first time there, but I wish I would have had more time to explore the city, instead I spent a great deal of time working during my stay there. I was able to visit a handful of the many fine restaurants that city has to offer. I really like Montreal, and hope to spend more time there in the future.

Boston

2016-09-11T00:00:00+00:00

After leaving Montreal, I went to Boston, where I stayed in Cambridge and Charlestown. I got to spend a day with Karine and some of her friends, which was great – always nice to catch up with old friends. I was in Boston from 21 to 28 May. I did a great deal of walking around the city at night, not necessarily to go anywhere, just walk and see the city. It had been a while since I had it to that extent, and Boston was a great city to walk around.

Chicago

2016-09-03T00:00:00+00:00

Okay, I’m writing this well after the fact, I’ve got a lot of catching up to do, as far as entries about where I’ve been. I was in Chicago from the 7th to the 14th of May. I was lucky enough to be able to meet up with friends Mike & Raquel and Pete & Laura a few times, it’s always great seeing old friends, them especially. I was out with all of them on my last night in town, and at the end of the night, on the way to the Metra station, Mike, Raquel and myself all got completely soaked in the rain. It turns out that wasn’t the last time that I was rained on during my last night of one of my stops.

Kansas City

2016-06-18T00:00:00+00:00

Kansas City was not a city I was particularly excited about visiting (1 - 7 May). I chose to spend a week there because I wanted to break up the drive between Denver and Chicago, not wanting to drive that much in one weekend; it was a city I was unfamiliar with, and it was conveniently located along the path I was taking.

I stayed on the Kansas [state] side of the city, but only a block away from the state line, and so spent time on both sides throughout my stay. It’s certainly a BBQ town, with that subject coming up in many of the conversations I had while I was there.

Denver

2016-06-18T00:00:00+00:00

I spent the last week of April in Denver, CO (24 April - 1 May). The drive to Denver was quite amazing, driving through the Rockies of Utah and Colorado. The mountain tops were all covered in snow, and even drove past a number of ski resorts still open for the season (although I imagine with a reduced selection of runs open). The drive out was also very pretty, with seemingly endless snow-covered prairies.

Unfortunately, during the week I was in Denver I was working a lot, and so didn’t get to see as much of the area as I would have liked. I did manage to have dinner in Boulder on my last night, as well as visiting a number of restaurants and breweries in the area I was staying. I really enjoyed the Airbnb that I was staying in, which was on the north side of Sloan Lake, with a lovely view of the lake and the park that surrounds it. There was quite the change in weather while I was there: starting out warm and sunny when I arrived, then being blanketed in snow later in the week, and ending with it being cold and wet.

Boise

2016-06-18T00:00:00+00:00

From 17 to 23 April, I stayed in Boise, ID; this was my second visit to Boise, the first being a quick stop while on a previous drive across the country. From that first visit, I learned that it’s actually a cute, little city, just happens to be kind of in the middle of nowhere.

After talking to a handful of people in Boise about their city, I get the feeling that, while they enjoy their city, they are in no hurry to spread the word about where they live, and would prefer to keep it for themselves. While I’m not trying to say that Boise is the greatest place on Earth that no one knows about, I would say that I did come to have an appreciation for what it has to offer.

I have no idea what those offerings might be in terms of cultural activities; it does have an array of outdoor options, an appreciation for local foods (more than just potatoes), a health-conscious population, in a small city package.

Fort Lewis

2016-06-17T00:00:00+00:00

I managed to spend a few extra days with my friends Brad & David, by sacrificing days in beforehand in Portland and later in Boise, and I stayed with them at their home on Fort Lewis, WA (8 - 17 April). While with them, we went to a Mariners baseball game and a Sounders soccer match, both of which were a great deal of fun, the last professional sporting event I had been to prior to that was probably the basketball games I went to for company events.

We took a ferry over to Vashon Island, where we had brunch and visited a [plant] nursery – where Brad purchased a crow statue, I believe he named it Henry. The trip to Vashon Island was a lovely experience, the ferry rides while very short, had great views.

I’ve really enjoyed the time I’ve spent in the Pugent Sound area; in addition to this trip, I’ve spent 2 months in Olympia and another month just north of Seattle. Much like Vancouver, the area has a mix of city, mountains, and water, which provides the area with a wide variety of activitites. Combine that with an environmentally-conscious population, and it certainly makes for an area where I can see myself wanting to spend a lot more time.

Vancouver

2016-05-07T00:00:00+00:00

I went to Vancouver, BC, Canada for my first time (1 - 8 April), and it was certainly one of the places that I have enjoyed most during this journey. I stayed in Yaletown, which was a very nice area to stay, with plenty of bars and restaurants nearby. It’s also along False Creek, with Grandville Island a short ferry ride away. It’s a beautiful city, with a mix of: skyline, water, and mountains.

I really enjoyed Grandville Island, which I visited 3 times; twice to get provisions for a picnic, that I would have shortly there after. Unfortunately, I only spent a few hours in Stanley Park, next time I’m there I will be sure to spend more time there.

I overheard a couple guys from Toronto describing Vancouver to some friends as: San Francisco Bay area mixed with Canada; that seemed true enough to me.

Portland

2016-05-01T00:00:00+00:00

I spent the end of March in Portland, OR (27 March - 1 April), where I met up with some coworkers for EmberConf. I’ve been to Portland a number of times before, but every trip there has involved a conference. I think it’s a great city to hold a conference: public transportation will take you just about anywhere you’d want to go (even to and from the airport), tremendous number of wonderful bars and restaurants, and a conveniently located convention center. Really the only downside is that getting there, especially from the East Coast, is potentially not very direct; that’s a trade-off I’m usually willing to make.

Portland was a bit of a milestone for this journey. The route and the timing leading up to it were planned to get me there in time for the conference. So, it definitely felt good to have successfully accomplished that goal.

My last full day in Portland (31 March) was also the day that Tesla Motors revealed their next vehicle, the Model 3. That morning they began taking reservations for orders for that new vehicle, where people – like myself – gave money for a reservation for a car they had never seen. I believe in an attempt to cause lines of people for news agencies to cover, not unlike lines that form outside Apple stores for people wanting to buy iPhones, reservations were only taken in person at Tesla stores, until the online reservation system became available that night. I went to their store southwest of Portland, in order to join the crowd waiting there to purchase a reservation. It was certainly a fun experience, talking with the people waiting in line around me, and it was great to see how many people were excited about an electric vehicle.

Bay Area

2016-05-01T00:00:00+00:00

I spent two weeks in Albany, CA (13-27 March), staying with Jes and Sadie; I only got to see Doug (Jes’s dad) for a day before he left for another of his epic cruising adventures. Having lived there for 4 months, along with many other visits, I’m quite familiar with the area, but did manage to see many new things during this stay.

Spending the afternoon in Tomales Bay was quite the treat, I look forward to returning to that area in the future. We went to some new wineries in Napa valley, that area is always a fun place to visit. Also went to some dining establishments for the first time.

Experiencing new things is great, but there are also times you want what you are familiar with, and did manage to taken in some of those as well: Acme Bread Co, Lava Vine Winery, Suppenküche, and Zachary’s Pizza.

Silver Lake

2016-04-23T00:00:00+00:00

I spent a week in early March (7-13) staying with friends, Jessie & Tim, in their house in the Silver Lake neighborhood of Los Angeles, CA. I’ve been to LA a handful of times before, but it’s always involves Jessie & Tim, so I don’t really know LA disconnected from them. I’m certainly not complaining about this, I always have fun when I’m with them, and being in a city with someone who lives there provides a different experience than just visiting on your own.

We had some great meals, went to the movie theater to watch The Witch, and went to the Getty museum, which was my first time and it was quite nice. Another LA first for me (and Jessie) was going to Grand Central Market, which I really enjoyed, and would certainly revisit on subsequent trips.

Topanga

2016-04-21T00:00:00+00:00

I was originally planning on spending the weekend of 5-7 March with my brother in Rancho Cucamonga, CA, but due to conflicting schedules, I made a last minute change and booked a wonderful little AirBnb rental in Topanga canyon. The rental was a tiny house that was part of a little compound with a couple other houses and a mini farm. Unfortunately, I was there at the wrong time, so didn’t get to try the homemade goat cheese, but I did get fresh eggs.

I spent a day hiking around the Topanga canyon area, including going up to Eagle’s Rock (highly recommended) and Topanga Lookout (enjoyed the views from the drive up there more than the views from the lookout itself).

There’s all of about 4 restaurants in Topanga canyon, and apparently they are known for having bad service; luckily I only experienced that kind of service in one of the three restaurants I visited while I was there. I would certainly recommend Topanga canyon as a fun weekend getaway.

Santa Barbara

2016-04-21T00:00:00+00:00

I spent the first several days of March (1st - 5th) in Santa Barbara, CA. This was my first time in Santa Barbara, and I really enjoyed it.

I paid a crazy premium for an AirBnb rental in the hearth of the city, and while I felt like it was overpriced, I am glad that I stayed downtown. I enjoyed a number of restaurants there, and found a bar (The Pickle Room) where I ended most nights.

On my last day in the area, I visited a handful of wineries in the area, and really enjoyed the little town of Los Olivos.

Indio

2016-04-18T00:00:00+00:00

After leaving Santa Fe, I stopped for a few days (28 February - 1 March) in Indio, CA. There were a few reasons for stopping here: the place I wanted to stay in Santa Barbara wasn’t available until 1 March, I wanted to see Joshua Tree National Park, and I wanted shorter distances to drive each day.

Joshua Tree was incredibly beautiful, I’m so glad I spent a day there, and would love to go back and spend more time there. Indio seemed nice, for a suburban area in Southern California.

Turning Point

2016-04-17T00:00:00+00:00

Today marks a turning point in this journey: I’ve started driving east. Up until this point, I’ve generally been heading west and north, but that’s no longer the case. For this next leg of the journey it’ll be heading east and south.

It’s a little awkward to have an entry about this turning point right now, as I’m still quite behind on the entries that I’ve been meaning to write about. So having an entry about leaving the Western Washington and driving to Boise, Idaho seems out of place, since I haven’t yet written about visiting various places in California, Portland, Vancouver and the Puget Sound area. But, it seemed important enough that I should write about it today, despite the weird sequence of entries it would cause.

Santa Fe

2016-04-08T00:00:00+00:00

I’m continuing to try to catch up on entries I’ve been meaning to write, so to that end…

I was in Santa Fe, NM from the 21st to the 27th of February. It was my first time in Santa Fe, and I really enjoyed it. I hiked on 3 different occasions, at 2 different locations, and the scenery was really quite beautiful. They like to say they have their own cuisine in New Mexico, although it mostly seems like Mexican with some tweaks. There are a number of chilies (as in peppers) that fall under the name of New Mexico chile, but all it really comes down to is if you’d like red chile sauce or green chile sauce added to your food. And both come from the same kind of chile, it’s just a matter of if the sauce was made with fresh or dried chiles.

My AirBnb hosts, Eric and Ishwari, were incredibly nice, and their “casita” that I stayed in was quite delightful. I would certainly return to Santa Fe again, and stay in the same place as well.

Photo Collection

2016-04-01T00:00:00+00:00

I have been doing a very poor job of keeping this journal current. I haven’t written a new entry in about a month and a half. So today, while I sit in my car in Centralia, WA as my car charges, I will try to catch up with entries for the places I’ve been since that last entry.

I have been much better at keeping the journey map, that I wrote about previously, up to date with where I’ve been. So, if you are curious about of my current location, that’s probably the best thing to check.

I’ve also been creating albums on Flickr, which contain photos that I’ve been taking along the way. Those albums are grouped into a collection for this journey, if you are curious to see those. I’ve been doing a better job of uploading new photos as I go than I’ve been about writing about my experiences here, but currently there are still many photos that I have not yet uploaded.

Austin

2016-04-01T00:00:00+00:00

I was in Austin, TX from the 14th to the 20th of February. That was not my first time in, and I always really enjoy the city when I’m there. During that stay, unfortunately, I did not get out and do too much in the city. For instance, I didn’t attend any live music shows, which I probably should have done.

I did go to a number of nice restaurants and bars, I also went to Franklin’s Barbecue to take in that experience. This is probably heresy to say: I enjoyed the experience of waiting in line more than I enjoyed the food. The food was good, don’t get me wrong, but perhaps my expectations were too high. Given I was with the right group of people, I would certainly do it again.

New Orleans

2016-02-14T00:00:00+00:00

This morning I left New Orleans; I was only there a few nights, I spent the first part of the week visiting my father in Knoxville, TN. Knoxville was cold, too cold, there were snow flurries the first couple days I was there, so the milder temperatures of New Orleans was a welcome change.

New Orleans ticks a number of the boxes for things that I think help make for a great city: decent public transportation, well located green spaces, and good places for food & drink. It also has the added bonus of having a great music scene and culture all its own. I’m not sure if the wealth gap is wider in New Orleans or not, but it is something that’s on my mind whenever I’m there, and that makes me sad.

I do love that city though, while I was standing in d.b.a. on Frenchmen St listening to a band perform, I was so incredibly happy to be in New Orleans.

Journey Map

2016-02-13T00:00:00+00:00

I’ve created a map of my journey, which you can also see below, but for optimal viewing probably best to view full screen. Attached to the map are some photos I’ve taken along the way.

Atlanta

2016-02-13T00:00:00+00:00

I spent the first week of my travels in Atlanta, Georgia, in fact the previous post was written on my first night in Atlanta. I had a great time for the week that I was there: I got to visit good friends, had some lovely meals, and really enjoyed the rental apartment where I stayed.

I’m not doing such a good job of keeping this journal current, as I left Atlanta back on 8 February, hopefully that’s not a trend that will continue.

I ended up staying in Atlanta a day longer than I originally planned, and the reason for that basically comes down to when I was planning I wasn’t thinking about the Super Bowl. So, I stayed an extra night in order to be with old friends during the game, which was definitely worth it.

Nomadic Journey

2016-02-01T00:00:00+00:00

This morning I got in my car and drove away from Orlando; this was the first time since August 2000 that I’ve left Orlando with no idea of when I’ll be back and if I’ll ever live there again. I’ve moved out of my apartment, but instead of finding another permanent residence, I’m going to be nomadic for a while, living out of Airbnb rentals and the like. I don’t have a set end date for when this nomadic journey will end, instead I’m just planning on traveling around until I get tired of what I’m doing and come up with a better plan.

My strategy as I move around is to only drive a day or two between locations, then stay in each location for a week or two. As for where I’ll be going during these travels: since it’s currently winter, I’m starting by going west along the southern part of the country, avoiding the colder northern states. After reaching California, then I’ll head north along the Pacific coast, and most likely continuing in that direction until I reach Vancouver. By which time it’ll be spring, and I’ll probably start heading east along the northern states.

None of these plans are really set in stone, and those ideas I have for what I’ll be doing further in the future are much less defined. If you have any suggestions for where I should go, I would love for you to reach out and let me know.

RailsDiff Rewrite

2015-02-27T00:00:00+00:00

TL;DR

RailsDiff is now an Ember app, built using Ember CLI, and it gets its data from a new API built using Sinatra.

Yesterday morning I released a complete rewrite of RailsDiff, which just so happened to be exactly 31 months since I first launched the site. This new version uses a completely different architecture from the previous; this post is to describe some of the motivations behind the changes, and how those changes were made.

Wait, what’s the point of RailsDiff?

As of yesterday, one out of my six colleagues that predominantly work with Ruby and Rails still didn’t know the purpose of RailsDiff. I’ve obviously done a good job getting the word out.

When you are going to upgrade a Rails application, RailsDiff is there to help you. It shows you what has changed in the default configurations between the version you are currently using and that to which you want to upgrade. The post where RailsDiff was announced goes into a bit more detail.

RailsDiff is about what you should change about your application’s configuration when upgrading Rails versions, not about what Rails has changed internally.

Motivation for the change

When I started this rewrite, my goal was not to use the latest, shiny tools available; instead it was to solve some real problems that I was having. And, in order to explain those problems, I believe a short review of how the app/site was built previously is in order.

The old implementation was a static site hosted on GitHub Pages. I had good reasons to build the site that way at the time. GitHub Pages is free, and I didn’t need an application to generate diffs on-demand because once a diff is generated it won’t change.

In order to have a static site, something needs to generate HTML files, and for that I turned to Rake. I actually made extensive use of Rake for RailsDiff, I used it to:

Generate a new Rails app with each new release of Rails
Generate a diff between each new app and all those that came before it
Generate HTML files for each new diff

And, I was quite happy with that implementation. I really liked the process of updating the site when a new version of Rails was released, which usually took about a minute to do:

rake to generate all the new files
git add -A
git commit
git push origin gh-pages to deploy the site

But, after 2.5 years and many releases of Rails this approach did have its downsides – most notably in disk space. It was approaching 5GB of generated, static files and growing exponentially faster with each new releases of Rails. Which caused me to be reluctant to add new features to the site, because that would mean more HTML files for each release, and even faster growth of disk space usage. I was also worried about the impact on collaborators, if step 1 was giving up 5GB+ of their hard drive space.

Better, stronger, faster

Something needed to change, and the place I decided to start was with the HTML files, as they were taking up the majority of the space. That meant dynamically rendering the diffs as HTML, but I also wanted to keep it a static site for its ease of hosting and speedy responses; using a client-side JavaScript application to do the rendering seemed to meet all the criteria. I chose Ember mostly because of all of the JavaScript frameworks currently available it’s the one I’m most familiar with, and the one we at Envy have the most experience with.

The first pass at the rewrite still had the diffs generated with Rake, which I put into the /public directory of the Ember CLI app; this had the unfortunate side effect of all those diff files being copied to the /dist directory on build, which slowed down the build process, and meant I wasn’t saving as much disk space. I eventually worked around this downside by moving the diffs out of the /public directory and instead modifying the build process to copy the files over, which meant for development builds I could just copy a subset of the diff files; this too was less than ideal, but at least kept development build faster.

At that point I had a working application, but I needed to figure out the deployment strategy. Having enjoyed my experience with GitHub Pages up to that point, I wanted to continue to use it. Unfortunately it’s not a great host for Ember applications unless you use HashLocation (location: 'hash' in the router), which I preferred to avoid. The next option was to deploy to Heroku using the buildpack suggested by Ember CLI. But, this didn’t actually work for me, because all of the diff files made the resulting application slug larger than Heroku allowed limit, which led to me modifying that buildpack to host pre-built Ember apps.

After updating the UI, adding support for legacy URLs, and some other general polishing of the application, I was ready to release it. I was still a bit worried about the disk usage issue, as generating the diff files still had an exponential growth curve. That’s when a couple of my colleagues (Dray & Nate) convinced me that I needed to make an API app to generate the diff files on-demand; the reason being that only generating a new Rails app with each new release, and not the differences between all the releases, leads to a linear growth curve.

Result

And, that’s where we are today. There are now two apps to be maintained: one for UI & behavior, and the other for data. Each with their own deployment schemes, their own narrow focus, and their own optimizations. It’s been a lot of effort and in the end, I’m very happy with the result. Hopefully, you will enjoy these changes as well.

If you see a problem with these changes or anything else that can be improved, please let us know.

Rake: Rule Tasks

2014-03-17T00:00:00+00:00

This is the third post in a series on Rake, see previous posts for an introduction of the Rakefile format along with global tasks , and task dependencies along with file tasks.

At one point, I thought about writing about something else besides Rake (Rails 4.1 perhaps?). But, with the recent passing of Jim Weirich, I felt it better to continue showcasing one of his many contributions to the Ruby community.

So, to that end, in this post we’re going to look at another capability of Rake: rule tasks. We’ll cover how create them, how they work, and then we’ll expand on the example from the previous post to make it usable for more of our configuration files.

We’ll start off with this:

file 'config/database.yml' => 'config/database.yml.example' do |task|
  cp task.prerequisites.first, task.name
end

And, end up with this:

rule '.yml' => '.yml.example' do |task|
  cp task.source, task.name
end

Which is not only less code, but as you’ll see, more useful.

Rule Tasks

Rule tasks, also known as synthesized tasks, have the same characteristics as all other kinds of tasks: they have a name, they can have zero or more actions, they can have prerequisites, and if Rake determines the task needs to be run it will only be run once.

What makes rule tasks different is that you don’t actually give them a name – I know, I just said that rule tasks have names, just bear with me – instead when you declare the task you give it a pattern in place of a name.

Task Declaration

If that made no sense, hopefully looking at some code will clear things up. First, we declare the rule task, and for that we use the rule method:

rule /foo/ do |task|
  puts 'called task named: %s' % task.name
end

In the above example, we used a Regexp pattern to create a rule task that will match any task name with foo in it, and it’s action will report the task’s name. Let’s see it in action:

$ rake foobar
called task named: foobar

As you can see above, once executed, the task’s name was foobar, as that was the name given to the rake command, but the rule’s pattern is /foo/.

A common use for rule tasks is when dealing with files, especially when we don’t necessarily know what the file’s name will be, but we know what to do based on part of the file’s name, like its extension:

rule /\.txt$/ do |task|
  puts 'creating file: %s' % task.name
  touch task.name
end

In the above example, we used a Regexp pattern to create a rule task that will match any task name ending in .txt, and it’s action will create a file with that name. Let’s see how this task works in action:

$ rake hello.txt
creating file: hello.txt
touch hello.txt
$ ls
hello.txt

This approach of matching based on the end of the task’s name is so common in Rake, we actually don’t need to use a Regexp for it, as just a String of what the end of the task’s name will be will suffice:

rule '.txt' do |task|
  puts 'creating file: %s' % task.name
  touch task.name
end

$ rake world.txt
creating file: world.txt
touch world.txt
$ ls
hello.txt world.txt

Dependencies

As with any other kind of Rake task, rule tasks can have dependencies. These dependency tasks can be either regular, file, or other rule tasks. And, they are declared using the same Hash syntax we’ve seen before:

rule '.dependency' do |task|
  puts 'called task: %s' % task.name
end

rule '.task' => '.dependency' do |task|
  puts 'called task: %s' % task.name
end

$ rake rule.task
called task: rule.dependency
called task: rule.task

Rules for Files

Rule tasks don’t need to be about files, as some of the examples above have shown, but if there is a file with the same name as the task’s name, then that task will have the characteristics of a file task. There’s an entire post in this series about file tasks, if you need a reminder of their characteristics and how they are used. But, in a gist, it means: the task will only be executed if the file does not exist, or unless it has a file task dependency with a newer timestamp than itself. Also like file tasks, if the dependency is a rule task matching existing files, you do not need an explicit declaration for the dependency task itself.

rule '.txt' => '.template' do |task|
  cp task.prerequisites.first, task.name
end

For rule tasks, the Task object has an additional source method which, I believe, can improve the readability of our task declaration:

rule '.txt' => '.template' do |task|
  cp task.source, task.name
end

Let’s see this task in action:

$ ls
file.template               # file.txt does not exist
$ rake file.txt
cp file.template file.txt   # output from running task
$ ls
file.template file.txt      # new file has been created
$ rake file.txt
$                           # attempting task again, produces no output

Useful Example

In the prevous post of this series, we used a file task to copy example configuration files to their desired location:

file 'config/database.yml' => 'config/database.yml.example' do |task|
  cp task.prerequisites.first, task.name
end

$ rake config/database.yml
cp database.yml.example database.yml

And, while this certainly works, things start to get a bit unruly once we want to apply this technique to a number of files:

file 'config/database.yml' => 'config/database.yml.example' do |task|
  cp task.prerequisites.first, task.name
end

file 'config/newrelic.yml' => 'config/newrelic.yml.example' do |task|
  cp task.prerequisites.first, task.name
end

file 'config/sidekiq.yml' => 'config/sidekiq.yml.example' do |task|
  cp task.prerequisites.first, task.name
end

This is a great example of where a rule task really shines. All we need to do is create a single task with the correct patterns:

rule '.yml' => '.yml.example' do |task|
  cp task.source, task.name
end

And, we can apply it to any number of files:

$ rake config/database.yml
cp database.yml.example database.yml
$ rake config/newrelic.yml
cp newrelic.yml.example newrelic.yml
$ rake config/sidekiq.yml
cp sidekiq.yml.example sidekiq.yml

Rake: File Tasks

2013-11-25T00:00:00+00:00

This is the second in a series on Rake, see previous post for introduction on the Rakefile format and about global tasks with Rake.

In this post we’re going to look at another capability of Rake: file tasks. We’ll cover how create them, how they work, and then create a useful example. But, before we get into file tasks, we need to have a better understanding of another aspect of the Rakefile format: prerequisites.

Task Prerequisites

Any Rake task can optionally have one, or more, prerequisite tasks – also referred to as dependencies. As with any other Rake task, a prerequisite task is only executed if it is needed, and if it is executed it is only ever done so once.

Let’s start by declaring a couple tasks called one and two in our Rakefile:

task 'one' do
  puts 'one'
end

task 'two' do
  puts 'two'
end

We can run the tasks in a shell, as we’ve seen before:

$ rake one
one
$ rake two
two

Now let’s declare one as a prerequisite for two:

task 'one' do
  puts 'one'
end

task 'two' => ['one'] do
  puts 'two'
end

As you can see, we haven’t changed how one was defined at all. But, for two, we added => ['one'] after the task name: this is how you declare a task’s prerequisites. At first this format may seem foreign, but keep in mind, this is just Ruby code; let’s see how our Rakefile would look like if we added in all of Ruby’s optional syntax:

task('one') do
  puts 'one'
end

task({'two' => ['one']}) do
  puts 'two'
end

As you can see, in both cases, we’re just passing 1 argument and a block to the task method. For one, we pass in just the task name, 'one'; for two, we’re passing in a hash, {'two' => ['one']}, the hash key is the task name 'two' and the hash value is an array of prerequisites ['one'].

Aside: If your task only has one prerequisite, the hash value doesn’t need to be an array:

task 'two' => 'one' do
  puts 'two'
end

Let’s see what happens when we run each of those tasks now:

$ rake one
one
$ rake two
one
two

As you can see above, when we ran two, we get the output from both the one and two tasks. Now that we have a foundation for prerequisite tasks to build on, let’s look at file tasks.

File Tasks

Thus far, all of the tasks we’ve created have used Rake’s task method to declare the task, but for file tasks Rake has a special method: file. File tasks in Rake are very similar to normal tasks: they have a name, they can have zero or more actions, they can have prerequisites, and if Rake determines the task needs to be run it will only be run once. Now, the twist is that those things get modified to be file related: the name of the task is the same as the file’s name, Rake determines that a file task needs to be run if the file doesn’t exist or if any of the prerequisite file tasks are newer.

That’s a bit to wrap your head around, so let’s look at some examples:

file 'foo.txt' do
  touch 'foo.txt'
end

Even though file tasks are meant for dealing with files, you are still responsible for creating the file in the task’s action if the file doesn’t exist.

Aside: Rake includes a modified version of the FileUtils module so that you have access to its methods in your task actions, which is where that touch method above is from.

Aside: FileUtils.touch works like the Unix touch program: updates a file’s timestamps and creates nonexistent files.

And, now when we run that task:

$ rake foo.txt
touch foo.txt   # output from our file task when it's run
$ ls            # showing file was created
foo.txt

Earlier, I mentioned that Rake will not run a file task if the file exists; so let’s see what happens when I delete the file and then run the task twice:

$ rm foo.txt    # deleting file
$ rake foo.txt  # running file task
touch foo.txt   # output from our task
$ rake foo.txt  # running file task again, but no output
$ ls            # showing file was created
foo.txt

As we can see, the first time we ran the foo.txt task we see the touch foo.txt output from the file being created, but the second time we ran the task we get no such output. But, things behave a bit different if we add a prerequisite to our file task:

file 'foo.txt' => 'bar.txt' do
  touch 'foo.txt'
end

Aside: If the prerequisite for a file task is another file, you do not need to create an explicit file task for the prerequisite, just using the name of the file is enough.

$ ls              # showing foo.txt does not exist
bar.txt
$ rake foo.txt    # running file task
touch foo.txt     # output from file task
$ rake foo.txt    # running file task again, but no output
$ ls              # showing file was created
bar.txt foo.txt

So far, things don’t seem much different: our file task creates the file if it doesn’t exist, and if we run the task again nothing happens.

$ touch bar.txt   # update timestamp of prerequisite file
$ rake foo.txt    # running file task again
touch foo.txt     # output! the file was updated!

Because the timestamp for the bar.txt file was newer than that of the foo.txt file, Rake executes the actions for the foo.txt task.

Useful Example

With this series, I’m trying to show you a feature of Rake, then show a useful example of using that feature, hoping that it’ll spark an idea for how you can use Rake in your normal development process; this post is no exception.

In our Rails applications, we typically have a number of configuration files that are critical for the application to run correctly. But, because these files contain either sensitive information or settings specific to where it’s being run, we do not put these files in source control; instead we usually add an “example” file with dummy data, so those who begin working on our application later know what needs to be set. Well, we can use Rake to simplify the creation of our configuration files from these “example” files.

So, let’s say we want to create the config/database.yml file from the config/database.yml.example file:

file 'config/database.yml' => 'config/database.yml.example' do
  cp 'config/database.yml.example', 'config/database.yml'
end

Aside: Here we are using FileUtils’s cp (short for: copy) method.

$ ls config/                          # showing database.yml does not exist
database.yml.example
$ rake config/database.yml            # running file task
cp database.yml.example database.yml  # output from file task
$ ls config/                          # showing file was created
database.yml database.yml.example

Okay, so our task is working properly, and copying the example file as expected. I’m not a huge fan of our task definition as it stands: there’s too much repetition, and we can clean that up. When creating a task, the block that you pass to the task method can also take an argument, which will be the task itself. We can use this task argument to DRY up our task definition:

file 'config/database.yml' => 'config/database.yml.example' do |task|
  cp task.prerequisites.first, task.name
end

That looks much better, and it still behaves the same. With this task in place, anyone who joins the project can run the task and will then have a config/database.yml to use. If they happen to run it again, nothing will happen; until someone updates the config/database.yml.example file, at which point you can then run this task again and get the latest changes.

That means you can think of these “example” files as templates for the actual files we need. Granted, it would be nice if the task didn’t just overwrite our config/database.yml with the contents of the example file and instead allowed it to merge the two files together; in future posts in this series we’ll be looking at expanding this task to do just that!

Rake: Global Tasks

2013-09-23T00:00:00+00:00

I’m hoping this to be the first post in a series about Rake, hopefully demonstrating some of the lesser known capabilities of the tool. But, seeing as this is the first in the series, I should probably give a little background about Rake itself.

Introduction

If you checkout the GitHub repository for Rake, you will see it describes Rake as:

A make-like build utility for Ruby.

And, unless you know what Make is, that description is probably not all that helpful. The Rake tutorial describes Rake as:

A build tool, written in Ruby, using Ruby as a build language.

Okay, so now we’ve seen the “build” term a lot, but again, if you are not familiar with Make, you are probably not too familiar with what a build tool/utility is. I believe the Wikipedia article for Rake does a better job summarizing it:

A software task management tool. It allows you to specify tasks and describe dependencies as well as to group tasks in a namespace.

But, now we’ve replaced one vague term, build tool, with another: task management tool. Because of that, I think we need a more helpful description; while not as succinct as those given above, here’s how I like to describe Rake:

Rake lets you define tasks
Tasks can depend on other tasks
Tasks are only run if needed
Tasks are only run once

But, we still need a definition for tasks. Since you define your tasks in Ruby, your tasks can do anything that you can do in Ruby. Which means you can do pretty much anything you want in your tasks – programmatically speaking.

Rakefile Format

We’ll go more in-depth with Rake’s domain-specific language (DSL) in future posts, but just to cover our bases, here’s a simple example:

task "hello" do
  puts "Hello, World"
end

You’d put this in a file called Rakefile, and to explain what that means: the task method is how you declare a task, you pass to the task method the name of the task ("hello" in our case), and you can give a task an action to run by giving a block to the task method. Now whenever we run the hello task, we’ll see the response of Hello, World:

$ rake hello
Hello, World

Global Tasks

Typically, Rake tasks are defined per project, by writing the task definitions in the Rakefile, placed in the root directory of your project. But, that’s not the only place you can define Rake tasks; Rake does let you define global tasks, these global tasks can be run from any directory – even those that do not have a Rakefile in their hierarchy.

The first step is to make the ~/.rake directory where the task definition files will reside:
```
 $ mkdir ~/.rake
```
Next create the file for the task definitions, the file’s name does not matter, but it must the have .rake extension:
```
 $ touch ~/.rake/hello.rake
```
Add tasks to newly created file just like you would in any Rakefile; I’ll use our above hello task as an example:
```
 $ echo -e 'task "hello" do\n  puts "Hello, World"\nend' > ~/.rake/hello.rake
```
Invoke global task, the -g flag is the important part:
```
 $ rake -g hello
 Hello, World
```

(Hopefully) Useful Example

So, now that we all know about global Rake tasks, perhaps you’re wondering how you can use them in your development workflow. Well, I’ll give you an example I use all the time, it’s a big long, so I’ve packaged it up in a gist.

Here’s a scenario where it can come in handy: suppose you are working in a topic branch of a Rails application, the topic branch has database migrations in it, you need to switch branches, but your app will not run properly in the other branch with the migrations present in the topic branch. What can you do in this scenario?

Well, you need to determine the migrations specific to this topic branch, and roll those migrations back:

$ VERSION=<version> rake db:migrate:down

Or, you can keep running rake db:rollback until all the migrations have been undone, but this assumes all the migrations needing to be rolled back are the latest one; which might not be the case if you’ve merged in the base branch, or rebased the topic branch, at any point.

Another option is to install the Rake task from the gist, and run:

$ rake -g rollback_branch_migrations[master]

Here I’m assuming that master is the branch you want to change to – wrapping it in square brackets like that ([master]) is how you pass arguments to a Rake task. This Rake task will handle determining the migrations specific to your topic branch, and roll those migrations back; that way you can switch branches and your database will be in the state the other branch expects.

TL;DR

To make global Rake tasks:

Define tasks in ~/.rake/*.rake files
Run tasks with -g flag: rake -g task_name

Announcing RailsDiff (finally)

2013-07-29T00:00:00+00:00

When upgrading the version of Rails used in an application, it can be difficult to know precisely every detail that should be changed.

Modifying the Rails gem version in your Gemfile and then running bundle update rails seems to work: your tests still pass, application runs like normal, etc. So, you never think twice and go about continuing to develop your application. But, were there other changes that could/should have been made as well? For instance, maybe Rails has changed the default value for some configuration settings, or perhaps new configuration settings have been added since your application was generated.

What the Rails application generator creates is specific to the version of Rails it was generated for. As new versions of Rails are released, what the application generator creates can change; comparing the output of different versions of the generator can give you some understanding of the changes happening in Rails.

Back when Rails 3.1.1 was released, I created a bunch of diffs from the generated code between 3.1.1 and all versions of 3.1.x that came before it. I then put the diffs in gists and wrote a blog post about it. Those gists helped me while I was upgrading a number of applications, because I didn’t need to guess what I should change about my applications configurations, the diffs told me exactly what to change. And, it didn’t just help me, I heard a lot of great feedback from others that those diffs also helped along the way.

Unfortunately, it was tedious and time consuming to create each of those diffs, and that was just creating a dozen of them. So, the thought of creating more of those with each subsequent release of Rails was quite daunting. Automation was definitely going to be needed to keep this going in the long run.

RailsDiff is about what you should change about your application’s configuration when upgrading Rails versions, not about what Rails has changed internally.

To that end, I’ve created RailsDiff. With it, you can see what changes you would need to make to your application’s configuration when upgrading from your current version of Rails to the new version of your choosing.

I say in the title that I’m finally announcing RailsDiff, and that’s because the site has been up and running for just over a year. Admittedly, I’m a little late with this announcement, but I hope it can help you the next time you need to upgrade a Rails application.

Optional Parameter Gotchas

2012-11-09T00:00:00+00:00

Assuming we have a class that looks like this:

class Foo
  attr_reader :options

  def initialize options = {}
    @options = options
  end
end

What does the following code output?

foo = Foo.new nil
puts foo.options.inspect

The output will be {}, right? Well, let’s check it out:

puts foo.options.inspect
# => nil

What the…? options was an optional parameter, and we set the default value to be an empty hash, what gives? What went wrong? Well, let’s try something a little different:

foo = Foo.new
puts foo.options.inspect
# => {}

This time we get the default value we were expecting, but why didn’t it work before? Well, with optional parameters, the default value is used if the argument is omitted; but, if an explicit nil is passed, that will be the parameter’s value. This behavior is not just for Hash default values, it applies to any default value.

Okay, so knowing that, how do we get around this issue? Well, the answer’s not too bad:

class Foo
  attr_reader :options

  def initialize options = nil
    @options = options || {}
  end
end

foo = Foo.new nil
puts foo.options.inspect
# => {}

So, this time we still have an optional parameter; but, we’re declaring the default value as nil, and when we set the @options instance variable we use || to make the value {}, if options is nil. This may seem like a small change, but can mean a big different in code correctness. If in some other part of our class we call methods on options thinking we’re guaranteed to have a Hash – or Hash-like object – but we in fact have nil, that could lead to some confusing errors.

Okay, so maybe now you are thinking: ‘but, I would never explicitly pass a nil like that.’ Well, you may not knowingly do so, but take a look at the following:

# somewhere else params is defined as: params = {}
Foo.new params[:foo]

In this scenario, maybe you thought you were passing a nested Hash into Foo.new, but if params doesn’t have a value for the key :foo, you’ve just passed a nil into Foo.new. This is just one example of how a nil could end up being explicitly passed as an argument, but it’s far from the only one.

I think the rule of thumb for default parameters should be:

Always declare the default value of an optional parameter as nil
Inside the body of the method, set the default value you actually want

And, that’s it really. I think that’s a pretty simple rule.

Ruby executable invocation

2012-07-22T00:00:00+00:00

Since the proliferation of Bundler, we’ve had multiple options for how to invoke Ruby executables; each option has its own set of advantages & disadvantages. The easiest form of invocation is to use the executable that is installed when the gem is installed; things like: rails, rake, or rspec. But, this form of invocation suffers from the problem of potentially loading a version of a gem that differs from the version your project bundle requires; you know you’ve run into this problem when you see a message like the following:

You have already activated <gem-name> <version-x> but your Gemfile requires <gem-name> <version-y>. Consider using bundle exec.

Binstubs

To get around the above problem, we often prefix our executables with bundle exec; and, while this works, I’ve often felt that the execution time was noticeably slower. But, Bundler provides another solution to this problem: binstubs. I don’t think this option is as well known as bundle exec, but this is how you create them:

$ bundle install --binstubs

Or:

$ bundle config bin bin
$ bundle install

Both of the above options will install all the executables that are provided by your bundle in the bin directory, you can then invoke one of the executables like this: ./bin/rspec.

RSpec executable benchmarks

I was interested in benchmarking the various options, to see how they actually stack up to each other; I also wanted to see how funny-falcon’s performance patches affected the execution time – which you’ll see in the 1.9.3-p194-perf column. I have a separate post with instructions on how to install these patches. So, first off, I wanted to benchmark executing rspec¹ – all the times are listed in seconds ²:

Command	1.9.3-p194	1.9.3-p194-perf
`rspec`	0.27	0.22
`./bin/rspec`	0.49	0.40
`bundle exec rspec`	0.98	0.75

So, my plan going forward for executables like rspec: use the regular gem executable. If I run into a version conflict, hopefully I can fix the issue by upgrading the gem in the appropriate projects; if that is not an option, then at that point I’ll switch to binstubs for the effected projects.

Rails executable benchmarks

Just running rspec does not necessarily load your whole Rails application. So, the following table shows invoking rails runner '' ³, to see what impact that has on run times.

Command	1.9.3-p194	1.9.3-p194-perf
`./script/rails`	1.10	0.83
`rails`	1.27	0.95
`./bin/rails`	1.53	1.11
`bundle exec rails`	2.01	1.47

I showed all the options for comparison’s sake, but I actually don’t think there’s any reason not to use ./script/rails; and, luckily my Rails command shortcuts take care of that for me.

No specs were actually run, the benchmarked command was rspec -h, just enough to load Ruby and RSpec. Version 2.11.1 of RSpec was used. ↩
Times were measured with time, only displaying the “real” value, and are the average of 10 executions. ↩
Version 3.2.6 of Rails was used. ↩

Ruby 1.9.3 with performance patches

2012-07-21T00:00:00+00:00

In my last post, I showed how to install Ruby 1.9.3-p194 on OS X without Xcode or OSX GCC Installer. But, I didn’t cover how to install funny-falcon’s performance patches. I have another post which can show the speed benefits of using these patches.

The following steps assume you’ve followed steps 1-6 of my last post; and, can be applied in addition to, or in replacement of, steps 7-8. In these steps, we will install a new version of Ruby which will be named 1.9.3-p194-perf, which you can keep as a separate installation of Ruby from 1.9.3-p194, or it can be the only installation of Ruby on your development environment.

Installation steps

Install autoconf:
```
 $ brew install autoconf
```

Install Ruby 1.9.3-p194 with performance patches:

 $ curl "https://raw.github.com/gist/3126141/rbenv.sh" | sh

Optionally, set Ruby 1.9.3-p194-perf as global version to use:
```
 $ rbenv global 1.9.3-p194-perf
```

Adjust Ruby environment variables

With those patches in place, we can now improve Ruby’s performance a bit more. Whenever Ruby’s garbage collector (GC) is running, nothing else can run; and, by default, Ruby’s settings are a bit conservative, potentially causing Ruby’s GC to run multiple times within a single Rails request. So, if we can allow Ruby to use more memory, while having the GC run less frequently, allowing more time for our code to run and less GC time, we can improve our applications’ performance.

Install rbenv-vars:

 $ cd ~/.rbenv/plugins
 $ git clone https://github.com/sstephenson/rbenv-vars.git

Add the following to ~/.rbenv/vars, or .rbenv-vars in a project directory:

 RUBY_GC_MALLOC_LIMIT=1000000000
 RUBY_HEAP_FREE_MIN=500000
 RUBY_HEAP_MIN_SLOTS=1000000
 RUBY_HEAP_SLOTS_GROWTH_FACTOR=1
 RUBY_HEAP_SLOTS_INCREMENT=1000000

Ruby 1.9.3 without Xcode or GCC

2012-07-13T00:00:00+00:00

I recently purchased a new Mac, and needed to set up my development environment again; I thought I’d record the steps taken, in case it helps anyone else out in a similar position.

Starting with version 4.3, Xcode no longer ships with command line developer tools, like gcc; instead, they are available as an optional component to Xcode and as a download separate from Xcode.

And, since Ruby 1.9.3 supports being compiled with LLVM/clang, I thought I’d try setting up my environment without Xcode and without the OSX GCC Installer.

Installation steps

Run OS X’s “Software Update”, to make sure the operating system and built-in software are up-to-date.
Download and install Command Line Tools for Xcode.

Install Homebrew:

 $ ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"

Install Git:
```
 $ brew install git
```

Install rbenv:

 $ cd
 $ git clone git://github.com/sstephenson/rbenv.git .rbenv
 $ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_login
 $ echo 'eval "$(rbenv init -)"' >> ~/.bash_login

Install ruby-build:

 $ mkdir -p ~/.rbenv/plugins
 $ cd ~/.rbenv/plugins
 $ git clone git://github.com/sstephenson/ruby-build.git

Install Ruby 1.9.3:

 $ CC=/usr/bin/clang rbenv install 1.9.3-p194

Set Ruby 1.9.3 as global version to use:
```
 $ rbenv global 1.9.3-p194
```

Note:

In steps 3, 5 and 6, I’m showing you the commands I used, which are based on the respective tool’s current installation guide; but, those commands are subject to change. Also, in step 3, I’m using the version of Ruby that ships with OS X.

Result

Homebrew makes sure you are well aware that its support of using “Command Line Tools” without Xcode is experimental:

Error: Experimental support for using the “Command Line Tools” without Xcode. Some formulae need Xcode to be installed (for the Frameworks not in the CLT.)

Despite that, I have not had any trouble installing anything I’ve needed so far – including: PostgreSQL, ImageMagick and Qt. Likewise, I haven’t had any trouble running any Ruby/Rails applications.

Upgrading to Rails 3.1.1

2011-10-07T00:00:00+00:00

Often, at Envy Labs, when we start a new application, we’ll use the latest version of Rails available; even if that version happens to be a prerelease. Unfortunately, as new versions are released, it can be difficult to know what steps need to be taken to upgrade your applications.

There are many times when just modifying the Rails gem version in your Gemfile and then running bundle update rails seems to work: your tests still pass, application runs like normal, etc. So, you never think twice and go about continuing to develop your application. But, were there other changes that could/should have been made as well? Perhaps Rails has changed the default value for some configuration setting; perhaps release candidates did not deal with assets in the same way the final release did. If you are using prerelease versions, more things tend to change as newer versions come out.

What the Rails application generator creates is specific to the version of Rails it was generated for. So, as new versions are released, what the application generator creates changes; and, comparing the output of different versions can give you some understanding of the changes going on in Rails.

So, to that end, I generated applications for the following versions: 3.1.1, 3.1.1.rc3, 3.1.1.rc2, 3.1.1.rc1, 3.1.0, 3.1.0.rc8, 3.1.0.rc6, 3.1.0.rc5, 3.1.0.rc4, 3.1.0.rc3, 3.1.0.rc2, 3.1.0.rc1, 3.1.0.beta1 (3.1.0.rc7 has been yanked from RubyGems); then compared all of them to version 3.1.1. I’ve made a gist for each of the comparisons:

Hopefully, this will come in handy for anyone else – like me – that needs to update some applications that were generated with a Rails version prior to 3.1.1. And, as you can see, depending on what version you used to generate your application, the changes needed will vary.

Common Rails command shortcuts

2010-07-15T00:00:00+00:00

I know many people – including myself – use Bash aliases for common command line tasks. Some popular examples would be: ss short for script/server, sc for script/console, and sg for script/generate.

These are setup by putting the following code in a Bash configuration file – such as ~/.bash_login:

alias ss='script/server'
alias sc='script/console'
alias sg='script/generate'

That works great, until you switch to Rails 3; all of these script files have been removed, and you use the rails command in their place: rails server instead of script/server, rails console instead of script/console, etc. Rails 3 provides shortcuts for the common commands: rails s is short for rails server, rails c is short for rails console, and so on. Some have suggested aliasing the rails command to just r, allowing you to use r s to start a server.

These shortcuts are nice, and is much less typing. But, I don’t want to have to remember when I’m in a Rails 2 app to use ss and then when I’m in a Rails 3 app to use r s; I want ss to just work in every project. Unfortunately, you cannot use a Bash alias to solve this problem; you can however use a Bash function! The following is a drop in replacement for the old alias ss=... stuff in your Bash configuration file:

function ss {
  if [ -e script/rails ]; then
    script/rails server $@
  else
    script/server $@
  fi
}
function sc {
  if [ -e script/rails ]; then
    script/rails console $@
  else
    script/console $@
  fi
}
function sg {
  if [ -e script/rails ]; then
    script/rails generate $@
  else
    script/generate $@
  fi
}

They work by checking for the existence of a script/rails file, which is new to Rails 3; if the file exists use it, otherwise fall back to using the Rails 2 version of the command; any arguments given to the function are passed along to the script, which is the $@ bit above.

I’ve contributed these changes to the Terminal project by Pigment, which includes similar shortcuts for many other common tasks.

Update:

A nice refactoring, courtesy of Xavier Noria:

function rails_command {
  local cmd=$1
  shift
  if [ -e script/rails ]; then
    script/rails $cmd "$@"
  else
    script/$cmd "$@"
  fi
}
function ss {
  rails_command "server" "$@"
}
function sc {
  rails_command "console" "$@"
}
function sg {
  rails_command "generate" "$@"
}
function sr {
  rails_command "runner" "$@"
}

What’s new in Rails 3.0 Beta 2

2010-04-06T00:00:00+00:00

On April 1st, Rails 3.0 Beta 2 was released, I thought I’d take this opportunity to highlight some of the changes between the Beta 1 and Beta 2 releases. According to GitHub, there are 2,691 new, unique commits – contributed by 244 authors – that were included in the Beta 2 release. I’ve picked some of the changes that I felt were important from the perspective of someone using Rails to build a web application and those that build plugins and other extensions for Rails.

The changes are separated by each of Rails’ individual components:

ActionMailer

You can now register interceptors and observers that will be called before and after an email is sent, respectively. The interceptor object must implement the delivering_email method, while the observer object must implement delivered_email. Here’s a simple example:

class MyInterceptor
  def self.delivering_email(mail)
    # do something before sending the email
  end
end

class MyObserver
  def self.delivered_email(mail)
    # do something after sending the email
  end
end

ActionMailer::Base.register_interceptor(MyInterceptor)
ActionMailer::Base.register_observer(MyObserver)

ActionPack

Block helpers now return Strings, so you can use <%= form_for @foo do |f| %>. <% form_for @foo do |f| %> still works but will give you a deprecation notice.

The way that send_file works has changed, it now defers to a Rack middleware called Rack::Sendfile. With that, the following options have been removed from send_file: :stream, :buffer_size, and :x_sendfile. The header used to send the file is configurable via config.action_dispatch.x_sendfile_header – which defaults to “X-Sendfile” – , and servers can configure this setting as well by setting the “X-Sendfile-Type” header. Hopefully, hosting companies specializing in Rails deployments will set this header, so that it will work transparently for those deploying there. If you do need to set it manually, for say Lighttpd, here’s an example:

class MyApp < Rails::Application
  config.action_dispatch.x_sendfile_header = 'X-Lighttpd-Send-File'
end

ActiveModel

For those creating libraries that they want to conform with ActiveModel, there has been some changes to ActiveModel::Lint – a module of tests that you can include in your tests to ensure your library conforms to ActiveModel. #new_record? and #destroyed? were removed, use #persisted? instead. For those of you that are curious, ActiveRecord’s implementation of persisted? is as follows: A model is persisted if it’s not a new_record? and it was not destroyed?.

In addition, #to_key was added to ActiveModel::Lint, this is used to generate DOM IDs for ActiveModel objects.

ActiveRecord

You can now prefix the table names all of the models in a module, you just need to define self.table_name_prefix on the module:

module Prefixed
  def self.table_name_prefix
    'prefixed_'
  end

  class ModelName < ActiveRecord::Base
  end
end

ActiveSupport

There are new assertions you can use in your tests: assert_blank and assert_present. Here they are used in a simple example:

assert_blank [] # => true
assert_present {:data => 'x' } # => true

yajl-ruby was added as a JSON parsing backend; and is the preferred backend, if available. There was an interesting discussion about using ActiveSupport::JSON.encode over #to_json on the patch’s Lighthouse ticket.

Session store configuration has changed. The cookie_verifier_secret has been deprecated and it is now assigned through Rails.application.config.cookie_secret instead and moved into it’s own file: initializers/cookie_verification_secret.rb. Also, session_store configuration has also been changed from ActionController::Base.session to Rails.application.config.session_store and has been moved to initializers/session_store.rb.

If you were building a plugin that inherited from Rails::Railtie, you were probably setting railtie_name and/or engine_name, well both of them have been deprecated. You can now add any object to the configuration object: config.your_plugin = {}. To see how the Rails internals switched to this new configuration approach, view the commit.

OpenID delegates

2008-04-09T00:00:00+00:00

This is more or less to add to what James Duncan Davidson has already said about OpenID and delegates.

Even though I had been using OpenID for a while prior to his post, I hadn’t set things up to use my own domain. After his post, I realized how easy it was to do. In his example he’s using myOpenID, which I don’t use; so, I had to figure out how to do it using my provider, Verisign Labs PIP. I had to do a bit of searching to find the server URL (https://pip.verisignlabs.com/server), but after that it was simple.

So, to amend James Duncan Davidson’s three easy steps, replace the HTML snippet in step 2 with the following:

<link rel="openid.server" href="https://pip.verisignlabs.com/server" />
<link rel="openid.delegate" href="http://yourid.pip.verisignlabs.com" />

Today I also tried out another OpenID provider built on Google App Engine, which authenticates with your Google ID; that’s a plus if you already have a Google account (such as Gmail) because that’s one less account you have to remember (isn’t less IDs part of the point of OpenID?). Anyway, the HTML snippet you need to use that provider is the following:

<link rel="openid.server" href="http://openid-provider.appspot.com/" />
<link rel="openid.delegate" href="http://openid-provider.appspot.com/yourid" />

Update: The above provider built on Google App Engine is currently a bit buggy. I would shy away from changing any of your accounts to use that as your OpenID URL directly. That’s another plus of using your own domain as your OpenID URL, you can change the server you use on your site without having to change the accounts that use your OpenID URL.

Jacob Swanner

PostgreSQL MERGE and Sequences

Using INSERT … ON CONFLICT

Using MERGE

NimbleCSV: Parsing into Elixir Maps

Production treatment of scripts

Applying production code treatment to utility scripts

Background

Utility Scripts

Dependency Management

Behavior-Driven Development (BDD)

Final Script

exit 0

Rake: Task Arguments

The ENV-workaround

Proper Task Arguments

Arguments & Prerequisites

Useful Example

London

Edinburgh

Wissant

Berlin

Amsterdam

Copenhagen

Stockholm

Video Player Debugging Story

Hindsight is 20/20

Atlanta

Washington D.C.

Montreal

Boston

Chicago

Kansas City

Denver

Boise

Fort Lewis

Vancouver

Portland

Bay Area

Silver Lake

Topanga

Santa Barbara

Indio

Turning Point

Santa Fe

Photo Collection

Austin

New Orleans

Journey Map

Atlanta

Nomadic Journey

RailsDiff Rewrite

TL;DR

Wait, what’s the point of RailsDiff?

Motivation for the change

Better, stronger, faster

Result

Rake: Rule Tasks

Rule Tasks

Task Declaration

Dependencies

Rules for Files

Useful Example

Rake: File Tasks

Task Prerequisites

File Tasks

Useful Example

Rake: Global Tasks

Introduction

Rakefile Format

Global Tasks

(Hopefully) Useful Example

TL;DR

Announcing RailsDiff (finally)

Optional Parameter Gotchas

Ruby executable invocation

Binstubs

RSpec executable benchmarks

Rails executable benchmarks

Ruby 1.9.3 with performance patches

Using `INSERT` … `ON CONFLICT`

Using `MERGE`

`exit 0`