paydrotalks

Hacking Redis To Save Money

2016-01-03T00:00:00-08:00

Check out my latest post on redis!

Site note – I’m kind of diggin’ medium for writing.

Scaling the 8tracks Music Player

2015-12-24T00:00:00-08:00

I started using medium to see if I would write more. First post is Scaling the 8tracks Music Player. check it out!

MacVim Tip: Tabs Per Project

2013-11-20T00:00:00-08:00

I dislike having too many windows open on my desktop. If I can consolidate them then I will. Here’s a tip to do this with MacVim.

Use :lcd path/to/proj instead of :cd path/to/proj in the new tab.

:cd will change the working directory for the whole window, while :lcd does it for the current window.

My vim setup uses ack.vim, NERDTree, and command-t plugins. These depend on the current working directory. If you’ve ever decided to open another project in another tab instead of another window, you know that running these plugins will not search within the new project directory. Well, now when you open a new tab use :lcd to move to that directory first.

For more info on :cd and :lcd you can run :help current-directory to find out more.

To ORM Or Not To ORM

2013-07-23T00:00:00-07:00

I spend most of my time building/fixing/maintaining the infrastructure at 8tracks, but every now and then I have the opportunity to do some good old fashion ruby web application programming. This week happens to be one of those times. Everything was glorious … until I needed to make a decision.

Do I use a gem or roll my own library to handle feature X?

I hate making this decision. It’s amazing when it’s easy. For instance, choosing to use the aws-sdk gem over rolling my own AWS library (it sucks just generating the stupid signature much less making an normal API call). I face so many facets dealing with choosing a gem vs. rolling my own. I always tell myself to suck it up and choose one so I can move on, but I never fail to change my mind shortly afterwards. And then change it again. And again. And again.

I was in this horrible state yesterday. Working on an internal sinatra/backbone.js/mongo application I had to choose whether or not to add a mongo object relational mapping (ORM) gem or roll my own. For the past several months I didn’t need to have any ORM as simple procedural code was more than sufficient. The application logic mostly existed in the browser with Backbone.js. Sinatra was a simple pipe to load mongo data to the browser. It was extremely efficient and simple.

But I needed more logic in ruby land. I thought I could add more helper methods to manage hashes, but that soon became unmanageable (for me anyways - I have a hard enough time remembering two variables much less 10 different helper methods). The features I wanted to add begged for more behavior that suits a model class. I attempted to add one of the mongo ORMs, but some of the conventions I used in the application conflicted with the way the ORMs behaved and would require that I write code to massage the gem into place. I decided to attempt rolling my own ORM that behaved like ActiveRecord. This was fun until my Superman ego deflated and I realized the scope of what I was trying to do. As usual, the scope was huge. I was sad. I spent many hours yesterday trying out each approach but to no avail. It’s hard replicating these huge libraries!

Then something clicked last night. What I really needed was a class around the hash that would allow me to CRUD it. All other access to the hash would behave like a normal hash. This concept basically made normal ruby hashes in to super hashes. Hashes with logic. Hashes with awesome. And if you’ve ever worked with mongo, that’s basically what is stored in that database. Hashes!

I had been asking myself the wrong question the whole time. It’s not to ORM or not to ORM, but rather CRUD this hash and add a few domain specific logic to the hash. That’s not a question, I know. It punted me out of the horrible decision making though.

So, I spent my time today crafting a small library that would make it easier to manage mongo documents. It was glorious. It even has tests! And I hate writing tests.

Unix `cat` For File Creation

2012-04-26T00:00:00-07:00

I spend copious amounts of time on the servers at 8tracks.com. On occasion, I upload a ruby/base/etc script to the server to test. In the past I normally fire up vi on the server, paste the code in, save it, and execute it. When the script fails because of a bug, I fix the file on my local editor, copy the code, reopen the file on the server with vi, save it, and execute it. This is slow. Here’s a faster way using cat and the CTRL+d keyboard shortcut in a terminal.

$ cat < my_new_file
Hello, world!
<hit CTRL+d here>
$ cat my_new_file
Hello, world!

This removes vi from the equation. This works for my previous tip about executing arbitrary ruby code on the shell.

Punctuation In Quotes

2012-04-19T00:00:00-07:00

I’m always confused by punctuations marks with quotations. For instance, where do you place the period when the sentence ends with quotes? Does the period go inside the quotes or outside? I know the rules (inside), but it still bothers me and I couldn’t figure out why … until I found this (emphasis mine)!

When it comes to commas and periods, though, logic doesn’t enter into the equation, at least not in the United States. Universal American usage places commas and periods inside the quotation marks, regardless of logic.

I was doing it all wrong with logic! It’s like measuring distance in America. Obviously, one mile is 5,280 feet. And one foot is 12 inches! DUH.

Ruby Scripting with Shell Here Documents

2012-04-11T00:00:00-07:00

Ever need to run a command you know how to do very well in Ruby but don’t know off hand with bash? It’s pretty easy if you have a single line handy for ruby like:

$ ruby -e 'puts "hello!"'

But, what happens if the ruby code is more complex and has multiple lines? You can obviously use semi-colons, but that’s less elegant. Let me introduce you to bash heredocs. Here’s an example of uploading a file to S3 with the right_aws gem.

$ cat < # Write my rubyies here!
> require 'right_aws'
> s3 = RightAws::S3.new('key', 'secret_key')
> bucket = s3.bucket("my_awesome_bucket")
> bucket.put("path/in/s3/my_large_file.csv", open('/path/to/local/file.csv', 'r'))
> RUBIES

Load mysqldump CSVs into Ruby FasterCSV

2012-04-11T00:00:00-07:00

Working with mysqldump CSV files using the FasterCSV is a pain. mysqldump converts NULL fields into \N and it uses backslashes (‘\’) to escape characters. The only character in a well formed CSV that you need to escape is the double quote (‘"’). You can change the mysqldump escape characters and force double quotes around the fields, but FasterCSV::MalformedCsvError errors still occur.

After multiple tries I looked to shell scripting to get this working. Modifying large files this way wont be fast, but I do this with scripts that run in the background so I don’t mind the time it takes.

First thing is to dump your table(s) into CSV formats. The following mysqldump command outputs tab separated fields with no surrounding quotes.

  # Creates a "/path/to/store/csvs/your_table_here.txt" file
  $ mysqldump -u<user> \
  > -p<password> \
  > -t --tab=/path/to/store/csvs/ your_db_here your_table_here

Since the only double quote in the file belongs to the content, I can escape them with sed. At the same time, I like to change \N to NULL. This sed command does the trick.

  # Edit in place
  $ sed -i \
  > -e 's/"/""/g' \
  > -e 's/\t[[:space:]]*\\N/\tNULL/g' \
  > /path/to/store/csvs/your_table_here.txt

Now that everything is properly escaped I double quote the fields and replace the tabs with commas.

  # Edit in place
  $ sed -i \
  > -e 's/\t/","/g' \
  > -e 's/^\(.*\)$/"\1"/g' \
  > /path/to/store/csvs/your_table_here.txt

You can combine both of those steps into one sed command, but I like having them separate in my scripts. It’s easier to digest.

Now you’re ready to use that file with FasterCSV (or CSV if you’re on ruby 1.9) with the default settings!

  FasterCSV.open('/path/to/store/csvs/your_table_here.txt') do |row|
    puts row[0], row[1] # ...
  end

I like to use a converter to convert those NULL fields to nils.

  FasterCSV.open(
    '/path/to/store/csvs/your_table_here.txt', 
    :converters => lambda{|f| f == "NULL" ? nil : f}
  ) do |row|
    puts row[0], row[1] # ...
  end

Let's rock out

2012-04-05T00:00:00-07:00

I Love Rock ‘n’ Roll from paydro on 8tracks.

Chrome Browser Just Made My Day

2011-08-17T00:00:00-07:00

Chrome added the ability to view :hover or :focus styles in the web inspector.

This is amazing.

If only they added :before and :after as well.