*/5 * * * * osascript -e 'tell application "System Events" to if (processes whose name is "iChat") exists then tell application "iChat" to log in'

This uses AppleScript to tell iChat to log in every 5 minutes. Now if the network drops, you’ll only be disconnected a few minutes.

• Michael’s post on using Unicorn with Heroku is a fantastic idea and great way to improve the efficiency of your Heroku application’s web dynos.
• Heroku imposes a limit of 15 threads per Ruby process (there is an open question about this, it might just be a bug in the Heroku Cedar stack, which is still beta). If you are using thin, you get 1 process and 15 threads per dyno. If you are using Unicorn, you get 3 processes^* and 45 threads.
• girl_friday defaults to 5 threads per queue. With the thread limit on Heroku, you’ll want to carefully ration the size of each queue. girl_friday’s Rack status app can tell you at runtime which queues are busy and which are quiet for tuning purposes.
• girl_friday can, in many cases, replace or dramatically reduce your need for separate worker dynos, saving you money.

* Where 3 is based on the memory consumption of your application. YMMV.

Unicorn appears to work great with girl_friday and Heroku’s free service level actually becomes useful for non-trivial applications with these optimizations. Know any other tips? Let me know!

Ruby’s threading has historically been terrible. So we turned to event-based systems like EventMachine but the reactor pattern brings its own set of drawbacks. So we turned to Fibers to solve some of those problems but they bring along another set of drawbacks. In the end though, all of this is a workaround for Ruby’s historically terrible threading implementation. But what if Ruby’s threading didn’t suck?

JRuby has great threads. Rubinius will soon have great threads. Both have really matured into excellent, stable runtimes over the last year. Make no mistake threads have issues too; locks and race conditions are notoriously hard to write well and Ruby code can’t be autoloaded safely in a multithreaded system. The former is what Actors attempt to solve: making concurrency simpler and safer by passing messages rather than sharing mutable state and requiring locks.

I think the Ruby community needs to give threads another chance: ensure their gems are thread-safe and use Actors when needing to write safe multithreaded code. Celluloid and the Rubinius Actor API are two APIs which provide Actors to Ruby. My new gem girl_friday leverages actors for safe concurrency. The autoloading problem is the only hard problem remaining; I hope some time and effort can determine ways around this issue.

Take a simple use case: “I want to send a welcome email when a user registers”. Commonly you want to send this email in the background so it does not impact the user’s experience. Do you need to install ActiveMQ, RabbitMQ or Resque to do this? Certainly not.

Message queueing is a fundamental architectural pattern when building complex systems. Your various system components might be written by different teams but they communicate through messages sent via queues. One component can send a message to another component, saying “please send this email”. But message queueing systems have their cost: they are complex because they are designed to be the foundation of your distributed system. They must be deployed and monitored like the rest of your infrastructure; they must be reliable and highly available.

I think that a lot of people install a message queue to perform simple background processing; it doesn’t need to be that complicated. The fundamental question to me is, “Am I communicating between different subsystems or just trying to spin off some work?” The registration email use case comes up almost immediately when building nearly every website. Consider also the case where you want to perform some action that might take 30-60 seconds and have the user’s browser poll for the result. Spinning off a separate thread to perform this work is entirely sufficient and much simpler. This is the reasoning behind my girl_friday project. I want a simple and reliable way to perform background processing without needing the complexity of an MQ system. Let’s examine a few characteristics of girl_friday:

• In-process – your background processor is part of your Ruby application and has access to the exact same codebase as your webapp. No need to share ActiveRecord models across projects via git or filesystem trickery. No need to deploy or monitor a separate set of processes.
• Threaded – huge memory savings because you don’t have to spin up other processes which load the exact same code. Threads are notoriously tricky to get correct so girl_friday uses Actors for the equivalent behavior in a simpler and safer API.

I have issues with the other contenders in the space:

• delayed_job – stores jobs in your RDBMS and polls for jobs which is a terribly unscalable idea. Spins off processes instead of threads.
• resque – forks a new process for every message. Safe but memory hungry.

The biggest caveat with girl_friday is threading, of course. Typical Ruby deployments aren’t thread-friendly but I’d like to help change that. Rainbows! is thread-friendly, as are all the JRuby app servers. The girl_friday wiki gives more specifics about features and usage. Are there any other dimensions to the problem that I’m missing? Any other projects that solve a similar problem? Post a comment and let me know!

On the other hand, I do think Actors are a good step forward for concurrency by making Threads safer to utilize. By the end of this year, we’ll have several Ruby VMs (JRuby, MacRuby, Rubinius) that can execute threads in parallel, allowing a single Ruby process to peg a multi-core machine; Actors will be helpful in building applications that can take advantage of this.

I’ve written two posts on the Carbon Five blog, the first covering Actors in general and the second covering my new, simple but full-featured asynchronous processing library, girl_friday, which leverages Actors. Take a look and let me know what you think!

• Node.js, Part I: Overview
• Node.js, Part II: Spelunking in the Code
• Node.js, Part III: Full Stack Application

• In MongoDB’s case, they don’t provide true durability so writes can be batched into a large set of writes when actually persisting to disk. This gets them great performance but means they don’t provide true ACID transactions. Side note: the latest release has a new --dur flag which gives true durability with the resultant loss in write performance.
• For Node.js, the trade-off is in programming style: everything is done asynchronously so you have to learn an entirely new style of programming. Great performance but great developer learning curve.
• With Telehash, UDP is a more efficient network protocol than TCP by design. TCP is essentially UDP with reliable delivery baked on top, so it suffers from round trip latency and the state required to track the current network packets in flight in order to ensure delivery. You can use UDP but if a router drops a UDP packet, your application will never know.

When you are looking at a new system that promises better performance or scalability than existing systems, ask yourself “what shortcuts did they take to get that performance or scalability?” Sometimes those shortcuts are worth it but it is completely dependent on your own situation. If you are writing a small, high-traffic network service, Node.js makes sense. Writing a high volume of low-priority logging data with MongoDB makes sense. I would argue there are very few instances where UDP is a good idea, realtime data streaming is the best case I can think of, off hand. Part of being an engineer is learning when these shortcuts are unreasonable and what you are paying for that shortcut.

require 'java'
require 'iText-5.0.6.jar'
 
module Pdf
  include_package "com.itextpdf.text.pdf"
  include_package "java.io"
 
  def self.write
    reader = PdfReader.new('application.pdf');
    stamper = PdfStamper.new(reader, FileOutputStream.new('completed.pdf'))
    form = stamper.acro_fields
    puts "Form has these fields: #{form.fields.key_set.to_a}"
    form.set_field("some_zipcode_field", "94115")
    stamper.close
  end
end
 
Pdf.write

Obviously sample code, not production quality, etc. Really the only hard/tedious part is mapping field names to Ruby object attribute values. If you have a complex form, you may have tens or even hundreds of set_field calls.

Creating RDoc

I’ve never found an easily accessible RDoc markup reference. The only one I know of is unlinkable, buried at the bottom of the RDoc readme instead of front and center as it should be. So I extracted the RDoc markup reference for your reading pleasure. Jan Varwig created a great RDoc cheatsheet PDF for local or offline access.

Next time you’re working on a gem, take 10 minutes to read the reference and document the main class for the gem. It’ll make you a better Rubyist, I guarantee it.

Generating RDoc

Ok, so you’ve learned RDoc markup and your code is documented like a champ. What do you do now? You’ve got several choices:

• Generate rdoc by hand – the rdoc command ships with Ruby and by default generates all .rb files in or below the current directory. You probably want something like rdoc lib to just include your project’s main Ruby code.
• Integrate RDoc into Rake – Rake has an RDocTask which can be configured for your project, so you can just run rake rdoc.

Once generated, you can view the generated output at doc/index.html.

Viewing RDoc

Rubygems has built-in support for generating and viewing the RDoc for installed gems. rdoc is generated when the gem is installed (using gem install --no-rdoc [name] skips the local rdoc generation). You can then use gem server to view your local gem rdoc at http://localhost:8808.

Check out YARD

YARD is an interesting project by Loren Segal to create a next generation Ruby documentation system. Getting started with YARD is well documented; try it out if you want an alternative to RDoc. YARD’s syntax is a superset of RDoc’s so backwards compatibility should not be an issue. The generated documentation looks pretty awesome (note the support for Markdown-formatted README files, which RDoc does not have).

The YARD team also created RubyDoc.info, which hosts documentation for popular gems and is a great resource for finding linkable documentation.

I hope this helps de-mystify rdoc for people. Any other rdoc tips, please leave a comment. Happy documenting!