Core Dump

Goodbye Utah

noreply@blogger.com (Amjith Ramanujam) — Thu, 24 May 2012 07:29:00 +0000

The time has come to part ways. I'm leaving Utah to move to Portland on May 25th 2012. This July would have marked the 10 year anniversary of living in Utah. I can't believe I spent one third of my life so far in Utah. Some of the best memories in my life were formed here.
True Love:
I met my beautiful wife here in Utah. She kicked my ass in TaeKwonDo, I asked her out, and the rest is history. Happily married for four years with a baby on the way.

See the full gallery on Posterous

First Job:
My first real programming job was with Delcam USA. I still have my first paystub from Delcam. Tom my boss at Delcam is still the best boss I've had so far.

Higher Education:
University of Utah. So many memories, so many sleepless nights at the computer lab. I still get a nostalgia when I walk through the campus.

Parksvan:

Eight clueless kids from India got together to share accomodation while going to school and ended up sharing the best parts of our life. Although we have all parted ways since our college days, I can't help but feel they are part of my family.

See the full gallery on Posterous

Aikido:

When I went to my first class, I thought I was going there to give my wife some company. Four years and 6 belts later, it has become a dominating force in my life. I never knew getting thrown around was the way to make new friends. :)

See the full gallery on Posterous

Along the way I discovered the wonderful art of Iaido. I call it the art of playing with Japanese swords.

See the full gallery on Posterous

Python:

I couldn't believe there was a group of people who met every month to geek out about their favorite language. My heartfelt thanks to herlo, mharrison, smcquay, travis and seth. You guys welcomed me into the group and helped me shape my future in the world of Python.

Outdoor:

I took up Mountain Biking, Hiking and Snowboarding and I've loved every minute of it. I will miss the rocky moutains, Uintahs and the snowy hills.

See the full gallery on Posterous

Leaving all of this behind makes me sad. Utah made me into what I am today.

But now I'm headed to Portland to work for New Relic. I'm told Oregon is a wonderful place, but I'll always have fond memories of Utah wherever I go.

Python Profiling - Part 1

noreply@blogger.com (Amjith Ramanujam) — Wed, 16 May 2012 02:28:00 +0000

I gave a talk on profiling python code at the 2012 Utah Open Source Conference. Here are the slides and the accompanying code.
There are three parts to this profiling talk:

Standard Lib Tools - cProfile, Pstats
Third Party Tools - line_profiler, mem_profiler
Commercial Tools - New Relic

This is Part 1 of that talk. It covers:

cProfile module - usage
Pstats module - usage
RunSnakeRun - GUI viewer

Why Profiling:

Identify the bottle-necks.
Optimize intelligently.

In God we trust, everyone else bring data

cProfile:

cProfile is a profiling module that is included in the Python's standard library. It instruments the code and reports the time to run each function and the number of times each function is called.

Basic Usage:

The sample code I'm profiling is finding the lowest common multiplier of two numbers. lcm.py

# lcm.py - ver1 
    def lcm(arg1, arg2):
        i = max(arg1, arg2)
        while i < (arg1 * arg2):
            if i % min(arg1,arg2) == 0:
                return i
            i += max(arg1,arg2)
        return(arg1 * arg2)

    lcm(21498497, 3890120)

Let's run the profiler.

$ python -m cProfile lcm.py 
     7780242 function calls in 4.474 seconds
    
    Ordered by: standard name
   
    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
         1    0.000    0.000    4.474    4.474 lcm.py:3()
         1    2.713    2.713    4.474    4.474 lcm.py:3(lcm)
   3890120    0.881    0.000    0.881    0.000 {max}
         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
   3890119    0.880    0.000    0.880    0.000 {min}

Output Columns:

ncalls - number of calls to a function.
tottime - total time spent in the function without counting calls to sub-functions.
percall - tottime/ncalls
cumtime - cumulative time spent in a function and it's sub-functions.
percall - cumtime/ncalls

It's clear from the output that the built-in functions max() and min() are called a few thousand times which could be optimized by saving the results in a variable instead of calling it every time.

Pstats:
Pstats is also included in the standard library that is used to analyze profiles that are saved using the cProfile module.

Usage:
For scripts that are bigger it's not feasible to analyze the output of the cProfile module on the command-line. The solution is to save the profile to a file and use Pstats to analyze it like a database. Example: Let's analyze shorten.py.

$ python -m cProfile -o shorten.prof shorten.py   # saves the output to shorten.prof  
$ ls shorten.py shorten.prof

Let's analyze the profiler output to list the top 5 frequently called functions.

$ python 
>>> import pstats
>>> p  = pstats.Stats('script.prof')   # Load the profiler output
>>> p.sort_stats('calls')              # Sort the results by the ncalls column
>>> p.print_stats(5)                   # Print top 5 items

    95665 function calls (93215 primitive calls) in 2.371 seconds
    
   Ordered by: call count
   List reduced from 1919 to 5 due to restriction <5>
    
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10819/10539    0.002    0.000    0.002    0.000 {len}
           9432    0.002    0.000    0.002    0.000 {method 'append' of 'list' objects}
           6061    0.003    0.000    0.003    0.000 {isinstance}
           3092    0.004    0.000    0.005    0.000 /lib/python2.7/sre_parse.py:182(__next)
           2617    0.001    0.000    0.001    0.000 {method 'endswith' of 'str' objects}

This is quite tedious or not a lot of fun. Let's introduce a GUI so we can easily drill down.

RunSnakeRun:
This cleverly named GUI written in wxPython makes life a lot easy. Install it from PyPI using (requires wxPython)

$ pip install SquareMap RunSnakeRun

$ runsnake shorten.prof     #load the profile using GUI

The output is displayed using squaremaps that clearly highlights the bigger pieces of the pie that are worth optimizing.

It also lets you sort by clicking the columns or drill down by double clicking on a piece of the SquareMap.

Conclusion:
That concludes Part 1 of the profiling series. All the tools except RunSnakeRun are available as part of the standard library. It is essential to introspect the code before we start shooting in the dark in the hopes of optimizing the code.

We'll look at line_profilers and mem_profilers in Part 2. Stay tuned.

You are welcome to follow me on twitter(@amjithr).

PyCon 2012 Review

noreply@blogger.com (Amjith Ramanujam) — Wed, 21 Mar 2012 04:56:00 +0000

PyCon 2012 was held at Santa Clara, California.

Tutorial:

I was there on the Thrusday to attend a tutorial called Python Epiphanies. The tutorial was educational in understanding some of the inner workings of Python. But I have a hard time trying to figure out how to use the knowledge I gained there.

Opening Ceremony:

We had ROBOTS.

See the full gallery on Posterous

And they were dancing.... how cool was that? It was a splendid opening ceremony.

Socializing:

Thursday evening was bag stuffing. Where we helped out by stuffing the swag bag. I got to work side-by-side some well known figures in the community (Jesse Noller, Pydanny). After that I hung out with some Heroku folks and learned about their awesome work culture.

Later that night, Yannick and Bryan gave a Pycon newbie orientation. I took their advice and gave a Lightning Talk about bpython (about 10mins into the video).

I socialized plenty and got a lot of useful contacts from different companies. I got to meet the founders of Octopart, my favorite Electrical Engineering based startups.

I also met with Kenneth Reitz who is famous for his requests library and this awesome talk Python for Humans.

I was quite thrilled when I first saw Guido at the Lunch hall sitting right next to my table. I was too shy to talk to him, but I managed to get a picture with him (in the frame).

Real gutsy! Maybe next year I'll actually shake his hand and get a picture with him.

Keynotes:

I enjoyed Paul Graham's keynote quite a bit. He talked about daring startup ideas. His keynote is summarized in these two essays.
David Beazly's keynote was a walk-through (demo) of tinkering with PyPy. It looked hard as balls and I kept hoping a happy ending where he declared victory. But it ended up being one of those art movie endings that leaves the listeners in a confused and inconclusive state.
Guido's Keynote on the other hand was interesting. His talk was sprinkled with unintended hilarity that ensued due to Google's presentation software. He was sporting an awesome T-shirt that read "Python is for girls" and talked about dealing with Python Trolls.

Talks:

I knew that all the talks were video taped and posted online, so I didn't worry too much about missing some when I had conflicts.

The following talks piqued my interest and will make me go exploring a little bit.

Permission or Forgiveness - Quaint. Applying Grace Murray Hopper's logic to Python programming.
pandas: Powerful data-analysis tools for Python - Useful.
Webserver Performance Tuning - Sounded like a sales pitch for New Relic, but not in a bad way.
Angry Birds playing Robot - Hilarious and Informative.
Pyed Piper: Modern Alternative to Awk, Sed etc. - Interesting.

Capoeira:

I went to the open space organized by Pydanny where he brought his Capoeira instructor who taught us some awesome moves. By the end of the class, we are all breathing heavily and energized. Once I tried to do a hand stand and lost my balance, but Aikido training kicked in and I gracefully rolled out of my fall with just a carpet burn on my elbow.

Open Spaces:

I didn't get a chance to go to many of them, but I did attend the open space for SaltStack and sat with Seth House to try and get Salt running on my Macbook. After a few failed attempts, I decided to give Salt a chance on my linux desktop once I got home.

Babbage Difference Engine: (Not Pycon related)

Sunday afternoon Stephen McQuay (a fellow Utah Pythonista) and I decided to take up Vijay's offer to go visit the Computer History Museum where they were doing a live demo of the Babbage Difference Engine. OMG! It was awesome to watch the history come alive.

Memoization Decorator

noreply@blogger.com (Amjith Ramanujam) — Fri, 10 Feb 2012 16:42:00 +0000

Recently I had the opportunity to give a short 10 min presentation on Memoization Decorator at our local UtahPython Users Group meeting.

Memoization:

Everytime a function is called, save the results in a cache (map).

Next time the function is called with the exact same args, return the value from the cache instead of running the function.

The code for memoization decorator for python is here: http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize

Example:
The typical recursive implementation of fibonacci calculation is pretty inefficient O(2^n).

def fibonacci(num):
        print 'fibonacci(%d)'%num
        if num in (0,1):
            return num
        return fibonacci(num-1) + fibonacci(num-2)

>>> math_funcs.fibonacci(4)   # 9 function calls
    fibonacci(4)
    fibonacci(3)
    fibonacci(2)
    fibonacci(1)
    fibonacci(0)
    fibonacci(1)
    fibonacci(2)
    fibonacci(1)
    fibonacci(0)
    3

But the memoized version makes it ridiculously efficient O(n) with very little effort.

import memoized
@memoized
def fibonacci(num):
    print 'fibonacci(%d)'%num
    if num in (0,1):
        return num
    return fibonacci(num-1) + fibonacci(num-2)
    
>>> math_funcs.mfibonacci(4)  # 5 function calls
    fibonacci(4)
    fibonacci(3)
    fibonacci(2)
    fibonacci(1)
    fibonacci(0)
    3

We just converted an algorithm from Exponential Complexity to Linear Complexity by simply adding the memoization decorator.

Slides:

http://speakerdeck.com/u/amjith/p/memoization-decorator

Presentation:
I generated the slides using LaTeX Beamer. But instead of writing raw LaTeX code I used reStructured Text (rst) and used rst2beamer script to generate the .tex file.

Source:
The rst file and tex files are available in Github.
https://github.com/amjith/User-Group-Presentations/tree/master/memoization_de...

Update: Comment by dasht via HackerNews pointed out that the terminology Cache is used when you have a fixed size container, whereas the memoization decorator above does not have a restriction on the size. Hence it's technically not a cache.

Follow the comments in HackerNews: http://news.ycombinator.com/item?id=3576620

Productive Meter

noreply@blogger.com (Amjith Ramanujam) — Fri, 10 Feb 2012 06:09:00 +0000

A few weeks ago I decided that I should suck it up and start learning how to develop for the web. After asking around, my faithful community brethren, I decided to learn Django from its docs.

::Django documentation is awesome::

Around this time I came across this post about Waking up at 5am to code. I tried it a few times and it worked wonders. I've been working on a small project that can keep track of my productivity on the computer. The concept is really simple, just log the window that is on top and find a way to display that data in a meaningful way.

Today's 5am session got me to a milestone on my project. I am finally able to visaulize the time I spend using a decent looking graph. Which is a huge milestone for someone who learned how to display html tables 3 weeks ago.

Tools:

Django for backend
Sqlite
Haystack/Solr - search backend for Django
FancyBox - jquery plugin
flot - jquery plotting lib
Bootstrap - html/css

A huge thanks to my irc friends and random geeks who wrote awesome blog posts and SO answers on every problem I encountered.

I will be open-sourcing the app pretty soon. Stay tuned.

See the full gallery on Posterous

Too Many Classes Too Little Time

noreply@blogger.com (Amjith Ramanujam) — Tue, 22 Nov 2011 06:47:00 +0000

I'm taking a couple of the free online classes offered by Standford. One on Artifical Intelligence and one on Machine Learning.

I haven't had so much fun since kindergarten. Actually that's not fair, I didn't enjoy kindergarten this much. I'm listening to the classes during my lunch, after work, during weekends. I'm working on my assignment with so much enthusiasm, I dread the day when this class ends.

Stanford just announced a slew of new online classes offered starting in Jan 2012. I was way too excited when I first read the description on them. Now I'm a little sad, becasue I want to take 8 out of the 11 courses that are being offered and I don't have enough time. :(

Woe is me.

ps: If you are not taking any of these classes you are missing out big time. Please do yourself a favor and sign up.

Picking 'k' items from a list of 'n' - Recursion

noreply@blogger.com (Amjith Ramanujam) — Tue, 18 Oct 2011 05:01:00 +0000

Let me preface this post by saying I suck at recursion. But it never stopped me from trying to master it. Here is my latest (successful) attempt at an algorithm that required recursion.

Background:
You can safely skip this section if you're not interested in the back story behind why I decided to code this up.

I was listening to KhanAcademy videos on probability. I was particularly intrigued by the combinatorics video. The formula to calculate the number of combinations of nCr was simple, but I wanted to print all the possible combinations of nCr.

Problem Statement:
Given 'ABCD' what are the possible outcomes if you pick 3 letters from it to form a combination without repetition (i.e. 'ABC' is the same as 'BAC').

At first I tried to solve this using an iterative method and gave up pretty quickly. It was clearly designed to be a recursive problem. After 4 hours of breaking my head I finally got a working algorithm using recursion. I was pretty adamant about not looking it up online but I seeked some help from IRC (Thanks jtolds).

Code:

def combo(w, l):
    lst = []
    for i in range(len(w)):
        if l == 1:
            lst.append(w[i])
        for c in combo(w[i+1:], l-1):
            lst.append(w[i] + c)
    return lst

Output:

>>> combinations.combo('abcde',3)
    ['abc', 'abd', 'abe', 'acd', 'ace', 'ade', 'bcd', 'bce', 'bde', 'cde']

Thoughts:

It helps to think about recursion with the assumption that an answer for step n-1 already exists.
If you are getting partial answers check the condition surrounding the return statement.
Recursion is still not clear (or easy).

I have confirmed that this works for bigger data sets and am quite happy with this small victory.

Python Profiling

noreply@blogger.com (Amjith Ramanujam) — Fri, 14 Oct 2011 04:50:00 +0000

I did a presentation at our local Python User Group meeting tonight. It was well received, but shorter than I had expected. I should've added a lot more code examples.
We talked about usage of cProfile, pstats, runsnakerun and timeit.
Here are the slides from the presentations:

http://speakerdeck.com/u/amjith/p/python-profiling-1

The slides were done using latex-beamer, but I wrote the slides in reStructuredText and used rst2beamer to create slides.
The source code for the slides are available on my github page.

Programming - A Gateway Drug to Math

noreply@blogger.com (Amjith Ramanujam) — Wed, 05 Oct 2011 05:44:00 +0000

I decided to try my hand at the Stanford's AI Class. The pre-requisites mentioned Probability and Linear Algebra. So I started watching Probability videos on KhanAcademy.
Sal Khan was teaching how to find the probability of 2 heads when you toss a coin 5 times.
A classic nCk problem:

The probability of getting 2 heads while tossing a coin 5 times is:

But I wanted to find out the probability of getting at least 2 heads when I toss 5 coins.

Its really simple. All I had to do is P(2) + P(3) + P(4) + P(5).

But then computingby hand (or a calculator) was painfully slow, let alone do it 4 times.

So I wrote two little functions in Python that will calculate factorial (yes I reinvented the wheel) and.

Nothing teaches you math faster than trying to write a program to do the math for you.

Writing a program is the same as teaching the computer how to do a certain task. The only way you can teach someone to do a task is to become a master at doing that task yourself.

Bonus: It also teaches you corner cases like 0! = 1 and that you wouldn't think of otherwise.

Rant about C++ dependency hell

noreply@blogger.com (Amjith Ramanujam) — Wed, 05 Oct 2011 03:39:00 +0000

When was the last time I vented about C++? The answer for that is always:

"TOO LONG AGO".

The initial friction to setup a substantial project using C++ is unfucking bearable.

When we started code revamp at work recently, I decided to be a good citizen and decided to incorporate cpptest, a unit testing framework.

It made me realize how unreasonably complicated Makefiles can become. After 3 hours of peeling away at the complexity I managed to add cpptest to the build dependency of the project.

Now time to write a few tests and check it out. I'm thinking "We are almost there".

FALSE!

Compilation gives me a gazillion error messages that make absolutely no sense. After about 30mins of StackOverflowing and Googling, I find out that string and map that I declared in the headers files need to be namespaced. Of course there is no indication (not even a hint) of that in the error messages. So I add 'using namespace std' and get past it.

Awesome my first test is compiling successfully. Time to run this baby and declare victory.

Close! But no cigar.

The executable was unable to load the CppTest library during runtime. Argh!

I set my LD_LIBRARY_PATH env variable and now it's running. But I can't ask everyone in my team to do that, so I have to figure out how to statically link that library.

It's already 6pm and I'm hungry. That'll have to wait for another day.

TL;DR - C++ and Makefile can burn in a fire of thousand suns.

Rapid Prototyping in Python

noreply@blogger.com (Amjith Ramanujam) — Mon, 26 Sep 2011 03:02:00 +0000

I was recently assigned to a new project at work. Like any good software engineer I started writing the pseudocode for the modules. We use C++ at work to write our programs.
I quickly realized it's not easy to translate programming ideas to English statements without a syntactic structure. When I was whining about it to Vijay, he told me to try prototyping it in Python instead of writing pseudocode. Intrigued by this, I decided to write a prototype in Python to test how various modules will come together.
Surprisingly it took me a mere 2 hours to code up the prototype. I can't emphasize enough, how effortless it was in Python.

What makes Python an ideal choice for prototyping:

Dynamically typed language:
Python doesn't require you to declare the datatype of a variable. This lets you write a function that is generic enough to handle any kind of data. For eg:

def max_val(a,b): 
    return a if a >b else b

This function can take integers, floats, strings, a combination of any of those, or lists, dictionaries, tuples, whatever.
A list in Python need not be homogenous. This is a perfectly good list:

[1, 'abc', [1,2,3]]

This lets you pack data in unique ways on the fly which can later be translated to a class or a struct in a statically typed language like C++.

class newDataType 
{ 
    int i; 
    String str; 
    Vector vInts; 
};

Rich Set to Data-Structures:
Built-in support for lists, dictionaries, sets, etc reduces the time involved in hunting for a library that provides you those basic data-structures.
Expressive and Succinct:
The algorithms that operate on the data-structures are intuitive and simple to use. The final code is more readable than a pseudocode.
For example: Lets check if a list has an element

>>> lst = [1,2,3]    # Create a list 
>>> res = 2 in lst   # Check if 2 is in 'lst' True

If we have to do it in C++.

list lst; 
lst.push_back(3); 
lst.push_back(1); 
lst.push_back(7); 
list::iterator result = find(lst.begin(), lst.end(), 7);  
bool res = (result != lst.end())

Python Interpreter and Help System:
This is a huge plus. The presence of interpreter not only aids you in testing snippets of code, but it acts as an help system. Lets say we want to look up the functions that operate on a List.

>>> dir([]) 
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__'
,  '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', 
'__len__', '__lt__', '__mul__', '__ne__','__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', 
'__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 
'reverse', 'sort']  
>>> help([].sort) 
Help on built-in function sort:  
    sort(...) 
      L.sort(cmp=None, key=None, reverse=False) 
      -- stable sort *IN PLACE*; cmp(x, y) -> -1, 0, 1

Advantages of prototyping instead of pseudocode:

The type definition of the datastructures emerge as we code.
The edge cases start to emerge when you prototype.
A set of required supporting routines.
A better estimation of the time required to complete a task.

Scripting Tmux Layouts

noreply@blogger.com (Amjith Ramanujam) — Thu, 04 Aug 2011 05:20:00 +0000

Tmux is an awesome replacement for Screen. I have a couple of standard terminal layouts for programming. One of them is shown below.

Vim editor on the left.
Top right pane has the bpython interpreter.
Bottom right pane has the bash prompt.

I have a small tmux script in my ~/.tmux/pdev file that has the following lines

selectp -t 0              # Select pane 0
splitw -h -p 50 'bpython' # Split pane 0 vertically by 50%
selectp -t 1              # Select pane 1
splitw -v -p 25           # Split pane 1 horizontally by 25%
selectp -t 0              # Select pane 0

In my tmux.conf file I have bound <prefix>+P to sourcing this file. So now anytime I want to launch my python dev layout, I hit <prefix>+<shift>+p.

bind P source-file ~/.tmux/pdev

How to Find Local Tech Jobs

noreply@blogger.com (Amjith Ramanujam) — Tue, 07 Jun 2011 00:28:00 +0000

Looking for tech jobs can be daunting. Networking is touted as the magic bullet for job seekers. But where do you start?

Here are some robust ways to build your network.

Users Group:

User groups are typically monthly meetings for geeks who get together to talk about their favorite programming language or operating system. Usually they are accompanied with a mailing list which is used to announce the meetings, ask questions and post job openings. So sign up to the mailing list and start attending the meetups. They are full of really nice people who are willing to help.

Utah Python - Utah Python Users Group
URUG - Utah Ruby Users Group
SLLUG - Salt Lake Linux Users Group
- SLLUG-JOBS - Mailing list to announce job postings
PLUG - Provo Linux Users Group

Local Conferences:

Most cities have some tech conferences that are a great source for networking. I found out about a lot of the user group by going to one of the following conference.

UTOSC - Utah Open Source Conference.

HackUTOS - Utah Open Source Project Day - Geeks, snacks and open source.

LaunchUp - A local entreneurship clinic. A great way to learn about the local start-up scene. You can meet new CEOs and fresh companies looking to hire tech talent. A must for job-seekers.

I hope this helps someone.

Contributing to Open Source

noreply@blogger.com (Amjith Ramanujam) — Thu, 05 May 2011 05:50:00 +0000

Last week I successfully submitted my first patch to an open source project and it was accepted.

I like the bpython interpreter for all my python needs. It is quite handy for a python newbie like me. A few weeks ago I was in the middle of building an elaborate datastructure to learn list comprehension in python, when bpython crashed and took all the history with it. I whined about it on twitter and one of the developers of the project prompted me to submit a bug report. I was quite impressed by the fact that a core developer of bpython replied to my bitching on twitter.

After I filed the bug report, I decided to get the source code and poke around. I finally implemented a feature that will save the history after each command instead of waiting till the end of a session.

The following factors were the main impetus that led me to contribute to the project.

Project Hosting:

The project was hosted on bit bucket which is a Github equivalent for mercurial. This makes it so easy to fork a project and issue pull requests, compared to the traditional source forge model of submitting patches in a mailing list. The social coding sites like Github and BitBucket have reduced much of the initial friction in starting an open source project.

Project Size:

This one has a huge impact when I decide to dive into the code. Traditional C projects tend to have a ton of files that are too big and daunting for a beginner. The bpython project was written in python and had a total of 13 .py files. This makes it dead simple to make a quick change and run the project without compiling it. Again the choice of language has a lot to do with this.

IRC:

The welcoming nature of the community around a project does a lot to encourage a new comer. The IRC channels are a great way to interact with the developers compared to a passive form of communication such as emails. I jumped on #bpython irc channel and started asking questions when I ran into an issue with bpython source code. People on that channel are really helpful and prompt in answering questions.

Persistence:

My first pull request was scrutinized by the core developers and some suggestions for improvements were given. During that process I learned a lot about code review and how to check for corner cases. Finally after I made all those improvements the pull request was accepted and merged with the main repo. So having a beginners mind (no ego) is an absolute must when getting started on any project. Don't be discouraged if your first attempt is unsuccessful.

Now I'm proud to say my name is listed in the AUTHORS file of bpython project.

Synchronize Panes in Tmux

noreply@blogger.com (Amjith Ramanujam) — Tue, 26 Apr 2011 06:38:00 +0000

Tmux is an alternative for screen. For anyone who doesn't know screen, it is a terminal multiplexer which means, it allow multiple windows in terminal. It can split your window into multiple panes (vertical/horizontal), detach a session which can be attached at a later time. Detach/Attach is very useful for running a job in a remote server without having to keep the ssh open the whole time.

Tmux can be configured by ~/.tmux.conf file.

My prefix key is Ctrl-q.

Synchronizing panes:

If you want to send your keystrokes to all the panes in your tmux window:

<prefix> :setw synchronize-panes

In my case I do:

Ctrl-q:setw synchronize-panes

tmux.avi Watch on Posterous

This is immensely useful if you want to execute the same set of commands on multiple servers.

Why do I hate Gnome?

noreply@blogger.com (Amjith Ramanujam) — Tue, 01 Mar 2011 05:49:00 +0000

I've been using Ubuntu Linux on my netbook for the past couple of days and I'm quite pleased with the whole experience, except for the initial issues (I just won't trust the auto-update).

Ubuntu uses the Gnome desktop environment by default with a little bit of tweaking. Gnome UI designers have a sense of aesthetic cognizance to their designs. I've always appreciated the crisp icons and the polished dialogs. I've been known to throw around the word stunning, quite generously, while describing Gnome.

All these initial infatuations almost made me forget the reasons why I abandoned Gnome a few years ago. I hate the absence of a central control center to tweak the default behavior of Gnome. There is however a severely handicapped version called gconf-editor which is like a terrible cousin of Windows Registry. So now if you want sloppy focus on gnome that doesn't raise your window when you click on it, you just have to do the following simple steps:

Open gconf-editor
apps
metacity
general
raise on click (uncheck)

Quite intuitive wouldn't you agree?

Oh you want to enable compositing, so your gnome-do can have some slick skins, here's how you achieve that:

Open gconf-editor
apps
metacity
general
compositing_manager (check)

Why? Why would you think this is more intuitive than having a simple GUI driven control center? I'm told this was a conscious choice by Gnome developers because giving choices tend to confuse their users.

If you think your users are idiots, only idiots will use it.

No wonder Linus was pissed at Gnome and started recommending KDE.

Ubuntu and I have some trust issues

noreply@blogger.com (Amjith Ramanujam) — Mon, 21 Feb 2011 07:31:00 +0000

I've been exclusively using Meego on my netbook and I was moderately happy with what it provided. So I did what any self-respecting hacker would do - I tried to make it better. Well, we all know how that usually ends. I managed to uninstall every single kernel in the system and rendered the system unbootable. Taking this as an opportunity I decided to try a grown-up OS. Enter Ubuntu into the picture.

I've heard nothing but great things about Ubuntu and I fuckin' hated it. Why? Because it was stealing all the limelight from the ever superior Mandriva (my favorite distro). So I got the installation process going which was smooth but surprisingly it didn't give me the option to keep my old partition, it was an all or nothing approach (grown up OS my ass). I had all my "stuff" backed up, so I decided to repartition my disk. It even detected my Broadcom wireless card and offered to install a proprietary driver for it. Wireless was working and the visual candy was stunning. I've never seen such a beautiful font set on my computer before. All the visual components looked hand crafted and the notifications were done with style. It was just a gorgeous piece of artwork. Visually stunning (if you didn't get that part already).

I was quite pleased with the decision to go with Ubuntu and was having some fleeting thoughts about replacing Mandriva on my desktop. Around 15 minutes in my playful prodding and poking, I was prompted by Ubuntu to notify that Software Updates were available. After it successfully installed the updates and rebooted the computer (some Kernel updates were involved), I couldn't connect to the internet anymore. The network icon wouldn't show up on my system tray anymore. After some googling oI managed to find the "Additional Drivers" program which installed the Broadcom Drivers once again. But this time it would show me the available networks but wouldn't connect to any of them. The system update had successfully screwed over the authentication routine for connecting to WPA2 networks. I tried to plug in my network cable to the ethernet port only to find out the system hadn't recognized the existence of an ethernet port. Are you kidding me? Have you heard of a little thing called hot-plugging? I've been spoiled by Mandriva that can automatically choose between the wireless card and the network cable on the fly, depending on which was available. Now I had to muck around the network interface files to even get my ethernet card detected. How hard is it to develop a Central Control Center that can manage your hardware? After getting that to work, I tried at least 10 different methods described on various forums to get the wireless chipset to work again without any success. I gave up after 10 hours of tweaking and started trying other OSes. Here is a list:

Fedora looked nice but wouldn't connect to internet. Same problem as Ubuntu, but this time, it wouldn't offer to download proprietary drivers.
Mandriva image hangs up in the middle of booting (I'm personally embarrassed by this). Btw, Mandriva has to be the ugliest of all the OSes. Seriously man, you gotta up the ante a little bit if you want to stay in this game.
Mint OS same as Ubuntu, only it looked even more gorgeous. I might try this out in the future.

I should've tried Arch Linux but I was too tired at this point. So I did what any self-respecting hacker would do..... I re-installed Ubuntu and rejected the offer to auto-update my system. Now I have wireless internet and a usable desktop that looks pretty.

Ubuntu had so much potential. I will always remember today as the day I almost replaced my desktop Mandriva with Ubuntu. Maybe next time.

LaTeX on Mac OS X

noreply@blogger.com (Amjith Ramanujam) — Sun, 19 Dec 2010 17:17:00 +0000

I used LaTeX when I was in school to create reports, presentation (using beamer) and even sometimes class notes and assignments. Recently when I was looking for a presentation program in Linux I was crestfallen by the lack of polish in OpenOffice Impress. So I created my presentation for the Salt Lake Linux User Group in LaTeX and it looked professional (nothing surprising there).

But that was created on my tiny Netbook running Meego. I wanted to make some edits to it with my wife's MacBook, so I started looking around for LaTeX on Mac. I found MacTex which completely took me by surprise, because the download size of the MacTex package was 1.6GB and the installed size on the computer was well over 3GB. It took me a good one hour to download, install and configure that thing her laptop. Seeing the 1.6GB zip file getting downloaded made me realize how much work has gone into LaTeX which I've been taking for granted all these years.

I bow before thee LaTeX.

Hands on Intro - Git

noreply@blogger.com (Amjith Ramanujam) — Thu, 09 Dec 2010 06:33:00 +0000

Today I presented in the Salt Lake Linux User Group meeting. The topic was "Hands on Intro - Git". It went well and I actually enjoyed it quite a bit. I choked twice once while trying to explain how to git apply patches that you receive via email, but then recovered from it with some help from the audience. But the second time I choked while trying to explain how to pull from multiple remote repositories, I couldn't recover from that. I do that so rarely it never occurred to me. Oh well, the first time is the hardest.

Off to prepare for the talk tomorrow at the Utah Python Group. I'm presenting "Hands on Into - PyQt4".

Here are the slides from that talk. LaTeX Source: https://github.com/amjith/git_present

Download now or preview on posterous

git_present.pdf (135 KB)

pTimer listed in Softpedia

noreply@blogger.com (Amjith Ramanujam) — Sat, 13 Nov 2010 05:10:00 +0000

One of my hobby projects is now included in the Softpedia Mac OS database. I don't know if that means anything but it is fun to see that 9 people have downloaded that utility so far.

A while ago I wrote a little PyQt4 application called pTimer to try out the famous Pomodoro Technique for productivity.

The original reason why I wrote that timer was to get myself acclimatized to Qt programming using Python. But since I liked it so much I decided to publish the code and host it on Google Code hosting.

Yesterday I got an email from Softpedia about including that utility in their website. Pleasantly surprised.

Moving MySQL databases to a new location

noreply@blogger.com (Amjith Ramanujam) — Sat, 06 Nov 2010 00:46:00 +0000

Default MySQL uses /var/lib/mysql folder to store the database files. If you'd like to change that default location to something else, you can modify the mysql config file called my.cnf located in /etc/my.cnf.

So edit /etc/my.cnf (you'll need root privileges). Lets say you want to change the default location to /home/mysql.

WAS:

[mysqld]
user = mysql
datadir = /var/lib/mysql

IS:

[mysqld]
user = mysql
datadir = /home/mysql

Create the folder /home/mysql and copy the folders that are inside /var/lib/mysql to /home/mysql.

cp -r /var/lib/mysql /home/mysql

The last and important step is to change the ownership of /home/mysql and its contents to the username mysql.

chown -R mysql:mysql /home/mysql

Now restart the mysqld service.

service mysqld restart

It should be ready to go and you can remove the folders from the old location /var/lib/mysql/* after you've checked to make sure everything is working properly.

Making Standalone Application for Mac OS X from PyQt app using py2app

noreply@blogger.com (Amjith Ramanujam) — Wed, 29 Sep 2010 03:46:00 +0000

In a previous blog post I talked about a software I wrote in Python called ptimer. I decided to bundle that software into a Mac OS X application that can be used without installing any dependencies.

Download Link - ptimer.dmg

py2app is a setuptool that can convert python scripts into a standalone Mac OS X App bundle. First install py2app using easy_install. Type the following in a Terminal.

$ sudo easy_install -U py2app

Next step is to create a setup.py that defines our package.

from setuptools import setup

APP = ['ptimer.py']
DATA_FILES = []
OPTIONS = {'argv_emulation': True,
 'iconfile': 'bell.icns',
 'includes': ['sip', 'PyQt4']}

setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)

You'll notice that we included both sip and PyQt4 as dependencies, they will be bundled with the application.

The icon file for this application comes from 'bell.icns'. If your icon file is in the form of an image you can convert it to icns using the img2icns.

Now let us create our app:

$ python setup.py py2app

This will create two folders in the current directory called 'build' and 'dist'. Inside the dist folder you'll see the Mac OS X app bundle called ptimer.app. That is the app bundle for distribution.

If you double click on the ptimer.app in finder, you'll get the following error message:

Lets fix that error.

Using Terminal navigate inside your app bundle and open the __boot__.py file inside the dist/ptimer.app/Contents/Resources folder.

$ cd dist/ptimer.app/Contents/Resources
$ open __boot__.py

Towards the end of the file there is a function def _run(*scripts):
Add this line after the import statement:

    sys.path = [os.path.join(os.environ['RESOURCEPATH'], 'lib', 'python2.6', 'lib-dynload')] + sys.path

You might have to change python2.6 to the appropriate folder name inside the dist/ptimer.app/Contents/Resources/lib folder. Double-click to make sure it is working.

Not it is time to create the .dmg file which is an apple disk image that is usually used to distribute software for Mac OS X.

$ hdiutil create ptimer.dmg -srcfolder dist/ptimer.app

This will create the ptimer.dmg file that is ready for distribution. Release it in the wild for everyone to enjoy.

Developing PyQt4 Applications in Mac OS X

noreply@blogger.com (Amjith Ramanujam) — Mon, 27 Sep 2010 01:48:00 +0000

PyQt4 is Python binding for the Qt library. Qt is developed by Nokia and PyQt bindings are provided by Riverbank.

PyQt4 is available only for 32-bit Intel architecture for Mac OS X. So we have to keep that in mind while we install the various dependencies for PyQt4.

The dependencies for PyQt4 are Qt4 and SIP.

Use the following command for configuring SIP and PyQt4 in 32-bit mode:

python configure.py -d /Library/Python/2.6/site-packages -b /usr/local/bin --use-arch=i386

This link is a walk through of the installation process: http://blog.oak-tree.us/index.php/2010/05/27/pyqt-snow-leopard.

Now that we have installed PyQt4 we are ready to start writing scripts in Python using PyQt4.

Lets check if PyQt4 is working. Type the following the Python interpreter:

>>> import PyQt4

If there are no errors, then we are good.

Since PyQt4 is installed as a 32-bit library, we have to run the Python scripts in the 32-bit mode as well.

$arch -i386 python2.6 pyqt_test.py

Lets try it out:

Copy the code to a file and name it pyqt_test.py


#!/usr/bin/env python

import sys
from PyQt4 import QtCore, QtGui

class Ui_Dialog(object):
   def setupUi(self, Dialog):
       Dialog.setObjectName("Dialog")
       Dialog.resize(400, 300)
       self.pushButton = QtGui.QPushButton(Dialog)
       self.pushButton.setGeometry(QtCore.QRect(80, 80, 271, 151))
       self.pushButton.setObjectName("pushButton")
       self.retranslateUi(Dialog)
       QtCore.QObject.connect(self.pushButton, QtCore.SIGNAL("clicked()"), Dialog.close)
       QtCore.QMetaObject.connectSlotsByName(Dialog)
   def retranslateUi(self, Dialog):
       Dialog.setWindowTitle(QtGui.QApplication.translate("Dialog", "Dialog", None, QtGui.QApplication.UnicodeUTF8))
       self.pushButton.setText(QtGui.QApplication.translate("Dialog", "Close", None, QtGui.QApplication.UnicodeUTF8))

class DialogTest(QtGui.QMainWindow):
   def __init__(self, parent=None):
       QtGui.QWidget.__init__(self, parent)
       self.ui =  Ui_Dialog()
       self.ui.setupUi(self)

if __name__ == "__main__":
   app = QtGui.QApplication(sys.argv)
   myapp = DialogTest()
   myapp.show()
   sys.exit(app.exec_())

Now try running the script:

$arch -i386 python2.6 pyqt_test.py

It should pop-up a dialog box with a big close button. Congratulations you are now ready to develop PyQt4 applications on your Mac OS X.

If you use the following without the arch -i386

$python2.6 pyqt_test.py

You will get the following error message:

Traceback (most recent call last):
File "pyqt_test.py", line 4, in 
from PyQt4 import QtCore, QtGui
ImportError: dlopen(/Library/Python/2.6/site-packages/PyQt4/QtCore.so, 2): no suitable image found. Did find:
/Library/Python/2.6/site-packages/PyQt4/QtCore.so: mach-o, but wrong architecture

Happy Hacking!

Pomodoro Timer written in Python and Qt4

noreply@blogger.com (Amjith Ramanujam) — Wed, 22 Sep 2010 03:57:00 +0000

I recently came across the Pomodoro Technique which is a method for improving concentration and getting things done without distractions. Since my attention span rivals the lifetime of a Muon, I jumped on the opportunity to try it out.

Pomodoro in 5 simple steps (from the website):

Choose a task to be accomplished
Set the Pomodoro to 25 minutes (the Pomodoro is a timer)
Work on the task until the Pomodoro rings, then put a check on your sheet of paper
Take a short break (5 minutes is OK)
Every 4 Pomodoros take a longer break

Since I wasn't planning on bringing my kitchen timer to work, I started to look for a software timer that can fit my needs.

To my surprise I couldn't find a good timer that can notify me every 25 mins, followed by 5 mins and repeat the cycle. I took it as my calling to implement the perfect Pomodoro Timer and use this opportunity to learn Python and Qt.

Rest of this post is about the timer. I decided to call it 'ptimer', partly because the name 'pytimer' was already taken.

Download Link: http://ptimer.googlecode.com/files/ptimer-src-latest.zip

In order to run this timer you need to have python 2.6 or above and PyQt4 installed on your system. I plan to bundle this into a single executable in the future.

How to Use:

Download and unzip the ptimer
Launch ptimer.py

Windows: Double click on ptimer.py
Linux/Mac OS X: Type './ptimer.py' from Terminal

Set the appropriate alarm times in the settings box and press Ok.
The ptimer will reside in the system tray.
When the timer expires it will flash a small display.
Left click to start the second alarm timer.
Repeat as necessary.
Right click on the timer display or the systray icon to control the behavior of the timer display.

I plan to go over the details of the code in some of the future posts.

Until then, enjoy the ptimer!

Intro to Caving in Timpanogos

noreply@blogger.com (Amjith Ramanujam) — Tue, 04 Aug 2009 15:55:00 +0000

Caving (or Spelunking) is the sport of exploring caves. Yosh and I recently had the opportunity to go on our first spelunking adventure. It was offered at the Timponogos caves in Utah for a group of five. It wasn't quite as dangerous and difficult as it sounds, but it was definitely more than just walking inside a cave. We donned our hard-hats with head lamps before entering the dark hole in the mountain. We were accompanied by a Ranger who knew what he was doing.

We had already taken the regular tour of the cave which in retrospect was dull in comparison to the spelunking experience. The path we took had a pretty low ceiling at many spots, sometimes so low we had to crawl in all fours. At one point we had to descend down the cave through a rock-climbing rope which was exciting. The cave walls were adorned with gorgeous calcite formations. At the end of our path there was a hidden lake (more like a pond really) which had a crystal clear surface. We all turned off our head lamps at this point and experienced the total darkness for about a couple of minutes. This is the kind of darkness that our eyes never get used to. No matter how long you stay in that darkness, it is impossible to detect anything. Experiencing the absence of even a single photon was quite unnerving. It was a lot of fun and definitely worth trying. We are both planning to pursue more spelunking in the future, lets see how deep it takes us.

Picture Time:

All ready to go in

Posing with the ranger

The whole group