Virtuous Code

Editing Directory Trees in Emacs

avdi — Mon, 06 Feb 2012 14:30:14 +0000

Sorry about the recording being cut off at the bottom. Tools demonstrated:

On Module Integrity

avdi — Fri, 03 Feb 2012 05:00:00 +0000

Goodness, such strong but mixed feelings provoked by this article: Modules called, they want their integrity back. On the one hand: yes! But on the other: no!

Having used Ruby for upwards of ten years, I have no trouble thinking of #include (and its callback partner-in-crime #included) as just another method. To some degree I think it’s bringing biases in from less dynamic languages to expect Ruby built-in methods like #include to behave like immutable keywords.

To that end, I think I have a looser idea of the “correct” semantics of #include than Josh Cheek does. #include / #included includes new functionality into a class (or module). How they do that is typically to add the referenced module to the inheritance chain and copy in constants. But I have no fundamental problem with more exotic methods of bringing in new functionality.

On the other hand, I do have a slight problem with adding zillions of public class methods in order to establish a some kind of declarative mini-language at class level; methods which then overstay their welcome and clutter up the class API. A chief offender is my usual punching-bag, ActiveRecord:

require 'active_record'
class Foo; end
class Bar < ActiveRecord::Base; end
Foo.public_methods.size # => 153
Bar.public_methods.size # => 481

Holy mackeral! A whopping 328 methods added to the public class API! And that’s calculated after taking into account all the methods which ActiveSupport adds to every class in the system; before requiring active_record the public class method count is 97.

Most of those methods are declarative “macros” which will only ever be used in the class definition: methods like has_many, belongs_to, and validates. But they linger around long after the class is fully defined, like that guy who’s still snoring on your couch at noon the day after a party.

I’ve started to experiment with alternatives to littering classes with “construction macros” which stick around after they have outlived their usefulness. Here’s one approach I’ve been playing with:

require 'make_believe_active_record'

class Post
  ActiveRecord.build(self) do
    has_many :tags
    belongs_to :category
    validates :title, :unique => true
  end

  # has_many, belongs_to etc. are not accessible here
end

At the risk of being a little bit more verbose, this pattern controls the context in which the additional “macro” methods are active. As an added bonus, it gives the implementor a convenient point, at the end of the block execution, to do any post-processing of the data structures that were set up by the builder macros.

Some of Josh’s alternatives reminded me of these experiments. I definitely think we share some concerns, although I think we’re coming at it from slightly different angles. For me, it’s more about avoiding namespace pollution: staying out of the global namespace, avoiding adding methods to base classes (cough acts_as_* cough), and not cluttering up classes with methods which serve no purpose once the class setup is finished.

Ruby Thread Locals are also Fiber-Local

avdi — Thu, 02 Feb 2012 05:00:00 +0000

I was briefly concerned that thread-local variables would not also be Fiber-local, since fibers have their own stack. This would be a problem for any code which uses thread-local variables to delimit a stack context, e.g. to implement dynamically-scoped variables or to prevent recursion. My fears, however, were easily allayed.

f = Fiber.new do
  puts "Fiber started"
  Thread.current[:in_fiber_context] = true
  Fiber.yield
  puts "Fiber-local value: #{Thread.current[:in_fiber_context].inspect}"
  puts "Fiber finished"
end

puts "Starting value: #{Thread.current[:in_fiber_context].inspect}"
f.resume
puts "Value outside of fiber: #{Thread.current[:in_fiber_context].inspect}"
f.resume
puts "Ending value: #{Thread.current[:in_fiber_context].inspect}"
# >> Starting value: nil
# >> Fiber started
# >> Value outside of fiber: nil
# >> Fiber-local value: true
# >> Fiber finished
# >> Ending value: nil

Once again, Ruby gets the little things right.

Preventing Recursion in Ruby

avdi — Wed, 01 Feb 2012 14:00:20 +0000

Sometimes you need to prevent code from executing recursively. I had to do this recently in order to prevent an infinite recursion in ActiveRecord callbacks. A before-save hook needed to save a parent model attribute, which because of an :autosave option would then try to save the child model, which would then try to save the parent… and so on until the stack overflowed.

Read on at the CodeBenders blog…

Decoration is best, except when it isn’t

avdi — Tue, 31 Jan 2012 14:00:00 +0000

I think by now we all know to prefer composition over inheritance. But in a language with a lot of options, what’s the best kind of composition to use?

Composing an adventure

Consider an adventure game, with objects representing player characters.

class Character
  # ...
end

A Character can be described:

class Character
  # ...
  def describe
    puts "You are a dashing, rugged adventurer."
  end
  # ...
end

A Character can look, listen, and smell his environment:

class Character
  # ...
  def look
    list("You can see", ["a lightning bug", "a guttering candle"])
  end

  def listen
    list("You hear", ["a distant waterfall"])
  end

  def smell
    list("You smell", ["egg salad"])
  end

  def list(prefix, objects)
    objects.each do |o|
      puts "#{prefix} #{o}."
    end
  end
  # ...
end

require './decoration-vs-extension'
cohen = Character.new
cohen.describe
cohen.look
cohen.listen

You are a dashing, rugged adventurer.
You can see a lightning bug.
You can see a guttering candle.
You hear a distant waterfall.

The character can also consult all of his senses at once:

class Character
  # ...
  def observe
    look
    listen
    smell
  end
  # ...
end

require './decoration-vs-extension'
cohen = Character.new
cohen.observe

You can see a lightning bug.
You can see a guttering candle.
You hear a distant waterfall.
You smell egg salad.

Characters can have various effects conferred upon them by items, potions, etc. A simple example is a hat:

require 'delegate'
class BowlerHatDecorator < SimpleDelegator
  def describe
    super
    puts "A jaunty bowler cap sits atop your head."
  end
end

At each turn of the game, the Character object will be decorated with whatever effects are currently active, and then a user command will be performed:

require './decoration-vs-extension'
cohen = BowlerHatDecorator.new(Character.new)
cohen.describe

You are a dashing, rugged adventurer.
A jaunty bowler cap sits atop your head.

Seeing in the dark

A more interesting effect is conferred by an infravision potion. It enables your character to see in the dark.

class InfravisionPotionDecorator < SimpleDelegator
  def describe
    super
    puts "Your eyes glow dull red."
  end

  def look
    super
    look_infrared
  end

  def look_infrared
    list("You can see", ["the ravenous bugblatter beast of traal"])
  end
end

While the character is experiencing the effects of an infravision potion, his powers of observation increase:

require './decoration-vs-extension'
cohen = InfravisionPotionDecorator.new(Character.new)
cohen.describe
cohen.look

You are a dashing, rugged adventurer.
Your eyes glow dull red.
You can see a lightning bug.
You can see a guttering candle.
You can see the ravenous bugblatter beast of traal.

There’s just one little problem that crops up when the #observe method is called.

require './decoration-vs-extension'
cohen = InfravisionPotionDecorator.new(Character.new)
cohen.observe

You can see a lightning bug.
You can see a guttering candle.
You hear a distant waterfall.
You smell egg salad.

Hey, where’d that bugblatter beast go?

The Character#observe method calls #look—but since the wrapped object has no knowledge whatsoever of the InfravisionPotionDecorator, it calls the original definition of #look, not the one which also calls #look_infrared.

Now, granted, this flaw actually works out in our intrepid adventurer’s favor, since the ravenous bugblatter beast of Traal is so stupid it thinks that if you can’t see it, it can’t see you. But never mind that: it’s still a bug, and bugs must be blattered.

A solution that’s all wet

We could patch this flaw by overriding #observe as well in the decorator:

class InfravisionPotionDecorator < SimpleDelegator
  def observe
    look
    listen
    smell
  end
end

Yuck! This is the exact same implementation as in Character, just copied and pasted so that the correct implementaiton of #look will be called. Clearly this is non-DRY. But even worse, we’ve introduced a nasty variety of connascence. Every time we introduces a new Character method which calls #look, we’ll have to cull through every single effect decorator which overrides #look, adding copy-and-pasted versions of the new method so that it doesn’t accidentally ignore the effect-wrapped version. Double yuck!

Modules to the rescue

In Ruby, there is an easy solution: extend the character with a module instead of a decorator.

module InfravisionPotionModule
  def describe
    super
    puts "Your eyes glow dull red."
  end

  def look
    super
    look_infrared
  end

  def look_infrared
    list("You can see", ["the ravenous bugblatter beast of traal"])
  end
end

require './decoration-vs-extension'
cohen = Character.new.extend(InfravisionPotionModule)
cohen.observe

You can see a lightning bug.
You can see a guttering candle.
You can see the ravenous bugblatter beast of traal.
You hear a distant waterfall.
You smell egg salad.

This time the overridden method is added directly to the object via its singleton class. So even the object’s own unmodified methods get the new infravision version of #look.

Sadly, by enabling him to see the monster we have sealed our protagonists’s fate. But at least we fixed the bug!

Are decorators overrated?

So what can we learn from this? When composing objects, Is it always better to use module extension than decoration?

In a word, no. For one thing, decoration is a simpler structure to understand. Given object A wrapped in object B wrapped in object C, it’s easy to reason about how method calls will be handled. They’ll always go one-way: a method in object A will never reference a method in B or C.By contrast, method calls in a module-extended object can bounce around the inheritance heirarchy in unexpected ways.

A second consideration is that once you’ve extended an object with a module, its behavior is changed for all clients, including itself. You can’t interact with the “unadorned” object anymore. You might extend an object for your own purposes, then pass it to a third-party method which doesn’t understand the modified behavior of the object and barfs as a result.

Finally, there’s a performance penalty. While it varies from implementation to implementation, dynamically extending objects can slow down your code as a result of the method cache beign invalidated. Of course, as with all performance-related guidelines, be sure to profile before making any code changes based on this point.

Conclusion

Decoration and module extension are both viable ways to compose objects in Ruby. Which to use is not a simple black-or-white choice; it depends on the purpose of the composition.

For applications where you want to adorn an object with some extra functionality, or modify how it presents itself, a decorator is probably the best bet. Decorators are great for creating Presenters, where we just want to change an object’s “face” in a specific context.

On the other hand, when building up a composite object at runtime object out of individual “aspects” or “facets”, module extension may make more sense. Judicious use of module extension can lead to a kind of “emergent behavior” which is hard to replicate with decoration or delegation.

At least, this has been my experience. Got some experiences or opinions on decoration vs. module extension? Feel free to leave a note in the comments!

A Game Development Curriculum

avdi — Tue, 24 Jan 2012 14:30:00 +0000

Along with every other hacker on the planet, I got into programming because I wanted to write games. It wasn’t long at all, though, before I learned that the greatest game of all is teaching the machine to do your will and get things done. I lost interest in game development pretty quickly and moved on to systems and application programming.

My son J. is almost 16, and he wants to write games too. However, in his case I have reason to believe he’s a lot more serious about the “games” aspect than I ever was. So I’m trying to put together a curriculum with an emphasis on game programming. This is a little tricky, since it’s a sub-field I have little connection to.

Here’s what I have so far, with many thanks to the folks on Twitter who helped me narrow down the list: (Book links include my Amazon affiliate code.)

Learn to Program, by Chris Pine. While I already have him working on Hackety Hack, I honestly hesitated to make Ruby his first language. Ruby is a programmer’s language; it has an enormous amount of syntax and many ways to do something. I worry that, as with C++, it will make learning the basics more difficult because it will be hard to separate the fundamentals from the incidentals.

After thinking about it for a while I came around, though. My reasons:

Ruby is the language I’m most fluent in at the moment. I’ll have no trouble helping him with any roadblocks he might run into.
He’s going to run into OO programming eventually, and I want his first contact to be with a strongly Alan Kay-inspired language. Rather than having to come around to that understanding the long way round, as I did. So no C++, Java, or Python as a first OO language. That pretty much means Smalltalk or Ruby, and of the two Ruby is probably more generally useful these days.
Finally, there are some Ruby game development bindings like Gosu that will let him have some fun even before diving into the more hardcore C/C++ stuff.

I suppose Javascript would fit the bill as well, and has the advantage of being ever-so-accessible inside the nearest browser. That said, for all that there is a beautiful language hidden in it, Javscript has more than it’s share of warts. I think in some ways Javascript might have even more of a fundamentals-obscured-by-incidentals problem than Ruby. Tough call though; I could definitely see starting with Javascript, especially if the student were most interested in programming for the web.

Land of Lisp, by Conrad Barsky. A timely book: it’s our next Ruby Rogues book club book. It’s about Lisp, which I want to give J. early exposure to, for ever so many reasons. And it’s about writing games. Win!

C Primer Plus, by Stephen Prata. For someone interested in web applications I might wait C until much later. But if it’s games you want to build, you simply can’t avoid C. Eventually you’re going to find yourself writing OpenGL or DirectX code. Even if you’re writing it in a higher level language, chances are the API documentation will be for the C/C++ API, so a basic understanding of C is essential.

And anyway, a little C goes a long way towards understanding just what the hell the computer is up to.

Blender Foundations, by Roland Hess. J. is particularly interested in modeling. I don’t know jack squat about 3D modeling, but I know Blender is free, Open Source, and widely used. I suspect I won’t need to give him much guidance in this area beyond giving him the tools he needs to get started.

Some C++ book, TBD. I suspect this will be a necessity for understanding later OpenGL texts and working with various game SDKs. If he does well with the earlier languages I’ll probably give him Accelerated C++; or maybe I’ll just go over the basics with him myself without resorting to a text.

OpenGL SuperBible: this one seems to come reasonably well recommended. And honestly I would love to learn some OpenGL along with him.

That’s what I’ve got so far. Any other suggestions?

ActiveRecord Default Association Extensions

avdi — Thu, 19 Jan 2012 19:07:00 +0000

Let’s say we want a helper method #build_fuzzy everywhere we have a collection of socks.

Sock.build_fuzzy # => #
sock_drawer.socks.build_fuzzy # => #

At first we might think to put it in an a collection extension module:

class Sock < ActiveRecord::Base
  module SockCollectionHelpers
    def build_fuzzy
      build(type: "fuzzy")
    end
  end
  # ...
end

Now we have to remember to add it to every Sock association:

class Drawer < ActiveRecord::Base
  has_many :socks, extend: Sock::SockCollectionHelpers
  # ...
end

And we also need to include it in Sock:

class Sock < ActiveRecord::Base
  # ...
  extend SockCollectionHelpers
end

…but this doesn’t work:

sock_drawer.socks.build_fuzzy # => #
Sock.build_fuzzy # => Raises no method exception for Sock.build()

And anyway, we want this to be available globally without explicitly extending associations.

Next, we try adding a class method to the Sock class. Since ActiveRecord association proxies delegate missing methods to the assoication class, it seems like this should work.

class Sock < ActiveRecord::Base
  def self.build_fuzzy
    build(type: "fuzzy")
  end
  # ...
end

This is a complete failure. While the association does forward the .build_fuzzy call to Sock, once in the call it is operating in the context of the class object, which as we saw before has no #build method.

sock_drawer.socks.build_fuzzy # Raises no method exception for Sock.build()
Sock.build_fuzzy # => Raises no method exception for Sock.build()

However, a slight change makes everything work as hoped:

class Sock < ActiveRecord::Base
  def self.build_fuzzy
    scoped.build(type: "fuzzy")
  end
end

Now when we call #build_fuzzy on the class it builds a fuzzy sock unassociated with any drawer, and when we call it on an association it builds a fuzzy sock with the appropriate drawer ID set:

Sock.build_fuzzy # => #
sock_drawer.socks.build_fuzzy # => #

All this thanks to the #scoped method, which is aware of the current scope.

Thank you to Dan Kubb for figuring this out for me.

UPDATE: Fixed def build_fuzzy to be def self.build_fuzzy in the last two examples.

My authoring tools

avdi — Thu, 12 Jan 2012 14:00:00 +0000

Several people have asked about my self-publishing process. I’d really like to write something comprehensive about this, along with code/scripts/config for my whole build stack, but I haven’t had time yet. Until that time, here are some notes on tools I use.

I do all of my writing in Emacs Org-Mode. (Including this blog post.) Org-Mode has many advantages for technical writing, but chief among them is probably the integration with various Emacs programming language modes. As a result, I am able to execute code samples “live” from within the manuscript and have the output automatically inserted. This saves me from having to maintain separate sample files and manually sync up the manuscript with the examples.
HTML export is handled directly by Org-Mode.
For PDF I use Org-Mode’s LaTeX-to-PDF support. I’ve made a number of customizations to the way the LaTeX is generated, something that Org-Mode makes very easy.
PDF code typesetting is handled by Pygments via the Minted LaTeX package.
For Mobi and Epub formats I pipe the generated HTML into the Calibre command-line tools.
I automate the various file conversions and packaging the outputs into a ZIP file with good-ole’ GNU Make.
I use DPD for digital distribution, payment processing, discount codes, everything having to do with selling the books.
I use 99Designs for cover designs.

I can recommend all of these tools to anyone looking to self-publish a technical book. I also recommend that anyone thinking of doing this read Jeremy McAnally’s book on the subject; it has a lot of good advice on how to plan, structure, write, and sell an eBook.

Should I rename my gem?

avdi — Mon, 09 Jan 2012 14:00:49 +0000

The Ruby community has some diversity challenges. I don’t think the Ruby community is any more biased than other tech communities; if anything, I think Rubyists are more apt to make a big deal over offensive content which would be considered business-as-usual in much of the traditionally white-male-dominated tech industry. Which, ironically, may make us seem like a hotbed of scandal.

As a community we also like our colorful gem names. A little while back someone wrote a blog post about some of the more questionable naming choices. Which got me thinking about one of my own gems.

The gem in question is HookR. It’s a library I wrote a few years ago for adding event hooks to arbitrary classes and objects.

The name amused me at the time. Actually, let’s be honest, the name still amuses me. But I’m also concerned it could be offensive. Not in an overt, groping-bottoms-at-a-conference kind of way, but in that insidious way that monocultures tend to be subtly threatening to outsiders without even realizing it.

As I personally read it, it is not gender-offensive since “hooker” is a gender-neutral term. However, it is potentially offensive to sex workers. And anyway, as I’ve noted before, ultimately I’m not the one who gets to decide which of my statements are offensive.

So my question for you: is HookR an offensive name? I realize it’s an unconventional, non-enterprisey name; that much I’m OK with. But is it offensive? Is it amusing, or is it off-putting?

And if you do find it problematic: can you suggest a better name?

Stop reading this blog and learn something

avdi — Thu, 29 Dec 2011 14:00:00 +0000

Software development is a wonderful field to be a noob in. Perhaps more than any other discipline, there is a wealth of information available for free online–everything from fundamental computer science courses, to the night-by-night learning notes of a master programmer. And, of course, there are millions upon millions of lines of Open Source code free for the reading.

Unfortunately, this embarassment of riches can be as much stumbling block as well as a stepping-stone. You can learn a lot from this information overload, but you can also waste a great deal of time and find yourself with very little to show for it. Personally, I’ve probably wasted thousands of hours in this way over the years. It’s only recently that I’ve come to identify the antipattern behind this waste. I hope that in writing this I can help you to avoid my mistakes.

The pattern usually looks something like this:

Oh man, this web programming stuff is HOT! I need to get in on this!

Let’s see… if I want to program for the web, I’ll need to learn a programming language better suited for the web. And I don’t want to get stuck with on that sucks, so I need to pick the very best one. Research ALL the languages!

[insert 3 months of reading about Perl, Python, Ruby, and PHP and maybe writing one or two "hello world" examples]

OK, my “hello world” was totally better in Blub than in any of the other languages. But now I need to pick the right web stack for my awesome app ideas. Subscribe to ALL of the blogs!

[Insert three months of voraciously reading web programming blog articles.]

I’ve been reading some of these blogs, and while it seems like everyone is using “Ruts”, it has almost become passé now. Some of the more obscure competitor frameworks look a lot cooler. I’ve read all of their design rationales, and despite the fact that I’ve never written an app in Ruts, I totally agree with their critiques. Yeah, I don’t want to to be stuck in a 6-month- old outmoded framework like Ruts.

Also I need to remember to use the “Box Turtle Pattern” that everyone is talking about.

No wait, now they are realizing Box Turtle doesn’t scale, and the Elephant Plop pattern is the way to go. Man, why did we ever think Box Turtle was a good idea? Whew, glad I hadn’t started my app yet. Dodged a bullet!

Oops, no, looks like Classic Box Turtle coding is making a comeback with a vengeance. Man they are totally right, when you do it correctly it’s way more robust than Elephant Plop.

Gosh, what a vibrant community! These advances are hard to keep up with! When am I ever going to find time to write code?

Oh man, this smartphone programming stuff is really heating up…

Maybe you mastered web programming long ago, but if you’re still learning at all there’s probably some technology that you’re at this point in. Maybe you’re like me and right now it’s client-side JavaScript frameworks. You know it’s there, you know you’re interested, you know where to find out more information about it. And you know that there are a lot of alternatives, and that despite the overwhelming amount of information available it isn’t at all clear where to start.

Without further ado, here is Avdi’s autodidact guide to learning a new programming technology:

Do not subscribe to all of the blogs. As yet you have no way of contextualizing the information you will receive from this firehose, and it will just be a paralyzing stream of often-contradictory information.
Make a list of the alternative languages/libraries/frameworks/whatever.
Pick the most popular one. Here are the criteria for “most popular”:
- You can easily discover high-traffic fora–mailing lists, IRC rooms, forums–dedicated to it. When you ask newbie questions on these fora, they are quickly answered.
- There is at least one reputable book about the topic. Books aren’t a perfect indicator of technology viability, but the fact that someone has taken the time to organize knowledge in a systematic fashion is a decent heuristic pointing to a certain level of stability.
- There is a healthy users group dedicated to the technology, meeting regularly in your nearest big city.
- At least a few people in the community are too hip for it, and have vocally moved on.
Yes, I said the most popular. Not the “best”. You don’t have enough information to figure out the “best” one yet. We’ll get to that. Right now, you need to pick the one where any problem you run into as you learn can be easily googled.
Join that local users group. Attend it diligently. You will need those people.
Buy, and read, a book about it. Yes, an actual book that you can read on your actual iPad. Or even, (gasp) on paper. Books have some nice properties which most online learning resources lack:
- They are (relatively) complete. Free online tutorials have a nasty tendency to have gaping “to be written” sections, or to simply stop right in the middle.
- They usually focus on fundamentals, rather than the latest nifty techniques. Right now you need fundamentals.
- They have structure. Right now you need structure.
- Authors of capital-B Books typically spend more time making sure that you’ll be able to reproduce the examples shown in the book using a given version of the tools. Not always (sadly), but usually. For that matter, they usually have fully-worked examples rather than snippets.
- Fewer hyperlinks, and thus fewer opportunities to be distracted by rabbit holes which lead everywhere except to you writing actual working software.
Now use what you’ve learned in a project. For the love of pete do not use it on a project you get paid for, unless you have an explicit understanding with your employer that you are using something you’ve never used before, and will probably get it totally wrong and have to rip it up and start again in six months. Otherwise, use it on a personal project. A small personal project.
When you run into trouble, consult the people in the aforementioned fora and local users groups. You are attending those groups regularly, right?
Congratulations! You now have the beginnings of context and perspective on this new realm of knowledge. The annoyances which caused those aforementioned hipsters to abandon technology X are no longer academic to you; they are real things which you have also dealt with.
With your newfound perspective in hand, review that list you made in step 2. Toss out the projects which have become defunct since you made the list. Pick out one which specifically addresses the problems and irritations you encountered in step 6. Use online tutorials, blogs, whatever; you know the territory now and don’t need to be so pedantic.

This is the framework I’m going to try to stick to, going forward. If you decide to try it, please let me know how it works out for you.