I can do this (and remove a DNS lookup) by updating my Feedburner count using an automated script on my server instead of having each client request it.

Using the FeedBurner Awareness API

Most of the time you only care about getting your total subscribers at the moment. The FeedBurner Awareness API is far more capable than just doing that, but we’re going to keep it simple today.

For the simple case you just need your feed ID or URI. Try:

curl -s 'https://feedburner.google.com/api/awareness/1.0/GetFeedData?uri=YOUR_FEED_NAME'

You receive an XML response like:

<?xml version="1.0" encoding="UTF-8"?>
<rsp stat="ok">
  <!--This information is part of the FeedBurner Awareness API. If you want to hide this information, you may do so via your FeedBurner Account.-->
  <feed id="foo0nbta7tscktjrgddc95gg3s" uri="EricWendelin">
    <entry date="2009-11-03" circulation="1181" hits="3901" reach="21" />
  </feed>
</rsp>

Now we just need to parse out the “circulation” which is your subscriber count.

Quick and dirty bash script

This is what I used to use until FeedBurner started returning 0s or blanks in the XML returned:

#!/bin/bash
FEED_COUNT=`curl -s https://feedburner.google.com/api/awareness/1.0/GetFeedData?uri=EricWendelin | egrep -o circulation=\"[0-9]+\" | egrep -o [0-9]+`

sed -r -i "s/(\"numsubscribers\">)[^<]+</\1$FEED_COUNT</g" /my/path/to/sidebar.php

echo $FEED_COUNT

You can use cron to replace some HTML, log the count, etc. every so often.

The Python

The Python version is much lengthier but has error checking and does not have to do file-replacement. The following is also available on GitHub for your extending pleasure ;).

#!/usr/bin/env python

# Usage: ./check_feedburner.py MyFeedName

import re, sys, urllib, fileinput
from xml.dom import minidom

API_URL = 'https://feedburner.google.com/api/awareness/1.0/GetFeedData?uri=%s'
# To replace the feed count in a file, put it here
FEED_COUNT_FILE = '/path/to/your/file.php'
# HTML to replace with feed count - REPLACE ME
HTML_REGEX = r'("numsubscribers">)[^<]+<'
REPLACE_STRING = r'"numsubscribers">%s<'

def get_circulation(feed_uri):
    dom = minidom.parse(urllib.urlopen(API_URL % feed_uri))
    entry = dom.getElementsByTagName('entry')[0]
    count = entry.getAttribute('circulation')
    if count == '' or count == '0':
     print 'Error getting feed count'
     sys.exit(1)
    return count

def replace_feedburner_count(count):
    '''Replaces feedburner count in FEED_COUNT_FILE'''
    try:
        for line in fileinput.input(FEED_COUNT_FILE, inplace=1):
     # Comma at the end prevents explicitly writing a newline
            print re.sub(HTML_REGEX, REPLACE_STRING % count, line, 1),
    except OSError, ose:
     print 'File "%s" not found' % FEED_COUNT_FILE

if __name__ == '__main__':
    # First arg is URI
    feed_count = get_circulation(sys.argv[1])
    print feed_count
    if FEED_COUNT_FILE:
        replace_feedburner_count(feed_count)

Something like this would be especially cool if you could set thresholds or use optparse for advanced options. Useful? Hope so!

Related posts:

1. Python first impressions
2. Site monitoring with Python and cron
3. How to highlight search results with JavaScript and CSS

Getting started

Before I could really start working on a major project, I had to choose a VCS. I went with mercurial based on a friend’s insistence that I introduce myself to a DVCS (specifically Hg) immediately. I knew what it was but never took time to try it myself. I am very glad I tried it.

Looking back, though, I think git may have been a (very slightly) better choice. Regardless, mercurial has served me very well and I strongly encourage anyone who hasn’t given a DVCS a spin to do so!

After getting some code and tests written, it was time to pick a project host. I chose kenai.com because it integrated well with NetBeans (which is THE IDE for JavaFX), it supported mercurial, and I like the direction Kenai is taking.

Communicating with Remember The Milk: GroovyRTM

RTM has a REST API that you can use to interact with the service. I decided to start by creating my own Groovy consumer of that API that would do the heavy lifting of interacting with Remember The Milk so I could keep my options open when looking for a UI technology.

I chose Groovy because:

• I didn’t want to write and maintain a lot of code
• I wanted to be able compile to Java classes I could use with other languages that run on the JVM
• It has great utilities for parsing XML and making HTTP requests

In the end, Groovy totally came through for me. It did everything I needed and more, easily.

UPDATE: I’ve been polishing GroovyRTM a lot since May and I’m confident in saying it is, by far, the best Java API kit available. If you have suggestions or improvements, let me know!

The UI

JavaFX was a natural choice for me since I’d been playing with it since before it was 1.0. I was eager to develop something more than a demo app with it. The two key features here were:

• Easily using Java classes – just an import
• Deployment to the desktop AND mobile phones (hoping to deploy to Android eventually). I’m holding off on mobile right now because RTM introduced their own Android app which I can’t compete with (yet)
• Java Web Start deployment gave me a lot of goodies for the desktop (title, icon, auto-update etc.)

Here is where I thank Jim Weaver and Rakesh Menon in particular for their useful JavaFX examples, helping me find a couple bugs, and their suggestions. If you use JavaFX and you’re not subscribing to their blogs, it’s time to come out from under that rock.

Frankly, I don’t think JavaFX was ready for building real applications until version 1.2 (released in June). Even then, including the JFXtras project is a must. There is no way I could have done this without them. I still had to create or adapt a few custom components. In all fairness, JavaFX is a young technology and I know that the team is working very, very hard on producing a lot more for JavaFX 1.3. Oh, and I can’t wait for the Sun to release the JavaFX visual designer.

Launch

Last but not least, I needed to easily make a screencast to introduce Cheqlist. After listening to Faceoff Show’s excellent screencasting review, I chose Screenr. The only thing that caught me by surprise is that the video didn’t embed in my aggregation feeds. Could have been my idiot mistake, though.

You can catch my intro video here if you haven’t for some reason.

Updates!

• Where’d my edit button go? Oh yeah, you don’t need it anymore since all task edits occur immediately now.
• Create and delete lists (including smart lists)
• Toaster looks and acts much better ;)
• Cheqlist is now on twitter! Follow @cheqlist for news and updates.
• Yet another update: Cheqlist has been approved for the Java Store!

In addition, you should know that I’m currently working on notes management, minimizing to taskbar icon, and undo functionality. Anything else you want? Hit me up on twitter, email me or discuss in the forum!

What would you have done differently? Comment!

Related posts:

1. Cheqlist: A free, open-source desktop app for Remember The Milk
2. Introducing GroovyRTM: A Groovier way to Remember The Milk
3. Lifehacker: 88 tech tricks to turbocharge your day

With that in mind, I am introducing an application that I think will help you with that: Cheqlist. A desktop application powered by Remember The Milk that does 2 things very well: make managing tasks very efficient and look sweet on your desktop.

What makes Cheqlist awesome

I wanted to build an application that I, myself, would use everyday. There are a few key features that would make that happen:

• Adding tasks quickly using RTM’s new Smart Add feature
• Visual appeal and lots of room for customization. Special thanks to Rakesh Menon for allowing me to use his JavaFX color picker
• Easily search tasks and use RTM’s custom search keywords
• Must work on all the OSes I use: Windows, Mac OS X, and Linux

On top of that, I think there are some things you’ll come to appreciate:

• Easy-update: Cheqlist checks for updates on startup, and if you want to upgrade, it takes just seconds!
• Free and open-source
• Lots of nice “easter egg” features for those of you who take time to find them ;)

A quick look

I’ve created a short (3 minutes) introductory video to show you the main features. UPDATE: Apparently screenr videos don’t show up in RSS properly. View it here.

Here are a few screenshots:

You can find more on the Cheqlist Wiki.

Try it out!

What you need:

• A marginally recent (6u11 or newer) version of Java
• Remember The Milk account
• Launch Cheqlist

Other resources

Something missing? Request a feature at the issue tracker.

Having trouble? Check out the support forums or contact me directly.

For more information, you can hit the project page and Wiki on Kenai.

Enjoy!

Related posts:

1. Introducing GroovyRTM: A Groovier way to Remember The Milk
2. What and why: the tech behind Cheqlist
3. How I use multiple desktops to get things done

While browsing my GitHub account I came across Mark Sanborn’s site monitoring script and thought: “Hey this is a good idea, let’s see what I can make of it”. I have been meaning to post more Python here so I updated his code a bit and thought I’d share it with you. I hope you have ideas for improvements.

UPDATE: It looks like Mark has made this a full project on GitHub and added timing the requests and command-line options! This is a perfect example of how OSS projects are started. Check out his introductory post.

Checking site availability with Python

I didn’t feel that this script was big enough to go full OO with it, but if you want to add to it, fork the gist on GitHub and provide a link in the comments. You know what’d really be cool is if someone used timeit to get the response time and set thresholds for when the site is too slow.

#!/usr/bin/env python

# sample usage: checksites.py yoursite.com othersite.org

import pickle, os, sys, logging
from httplib import HTTPConnection, socket
from smtplib import SMTP

def email_alert(message, status):
    fromaddr = 'you@gmail.com'
    toaddrs = 'yourphone@txt.att.net'

    server = SMTP('smtp.gmail.com:587')
    server.starttls()
    server.login('you', 'password')
    server.sendmail(fromaddr, toaddrs, 'Subject: %s\r\n%s' % (status, message))
    server.quit()

def get_site_status(url):
    response = get_response(url)
    try:
        if getattr(response, 'status') == 200:
            return 'up'
    except AttributeError:
    	pass
    return 'down'

def get_response(url):
    '''Return response object from URL'''
    try:
        conn = HTTPConnection(url)
        conn.request('HEAD', '/')
        return conn.getresponse()
    except socket.error:
    	return None
    except:
        logging.error('Bad URL:', url)
        exit(1)

def get_headers(url):
    '''Gets all headers from URL request and returns'''
    response = get_response(url)
    try:
        return getattr(response, 'getheaders')()
    except AttributeError:
    	return 'Headers unavailable'

def compare_site_status(prev_results):
    '''Report changed status based on previous results'''

    def is_status_changed(url):
    	status = get_site_status(url)
    	friendly_status = '%s is %s' % (url, status)
    	print friendly_status
    	if url in prev_results and prev_results[url] != status:
            logging.warning(status)
            # Email status messages
            email_alert(str(get_headers(url)), friendly_status)
        prev_results[url] = status

    return is_status_changed

def is_internet_reachable():
    '''Checks Google then Yahoo just in case one is down'''
    if get_site_status('www.google.com') == 'down' and get_site_status('www.yahoo.com') == 'down':
        return False
    return True

def load_old_results(file_path):
    '''Attempts to load most recent results'''
    pickledata = {}
    if os.path.isfile(file_path):
        picklefile = open(file_path, 'rb')
        pickledata = pickle.load(picklefile)
        picklefile.close()
    return pickledata

def store_results(file_path, data):
    '''Pickles results to compare on next run'''
    output = open(file_path, 'wb')
    pickle.dump(data, output)
    output.close()

def main(urls):
    # Setup logging to store time
    logging.basicConfig(level=logging.WARNING, filename='checksites.log',
            format='%(asctime)s %(levelname)s: %(message)s',
            datefmt='%Y-%m-%d %H:%M:%S')

    # Load previous data
    pickle_file = 'data.pkl'
    pickledata = load_old_results(pickle_file)

    # Check sites only if Internet is_available
    if is_internet_reachable():
    	status_checker = compare_site_status(pickledata)
    	map(status_checker, urls)
    else:
        logging.error('Either the world ended or we are not connected to the net.')

    # Store results in pickle file
    store_results(pickle_file, pickledata)

if __name__ == '__main__':
    # First arg is script name, skip it
    main(sys.argv[1:])

Basically, this script just checks if the internet is available, then checks each site. If the previous result is available and is different, it sends an email with the headers received so you might get a good idea what’s going on. Even cooler, you can use the email specific to your cell phone carrier to get text messages when your sites’ availability changes.

NOTE: You must have some sort of mailer daemon installed. See How to setup Gmail with sSMTP. You can try it out by editing the appropriate parts of the script and then doing:

chmod +x checksites.py
./checksites.py eriwen.com yoursite.com

Scheduling it up with cron

I’ve already showed you the ins and outs of basic cron scheduling. We can have this run every 5 minutes by typing crontab -e and then adding:

*/5 * * * * ./path/to/checksites.py yourwebsite.com othersite.org

What do you think? Tell me how you’d make it more “pythonic” or otherwise improve it in the comments.

Related posts:

1. Using Python to update your FeedBurner stats
2. Python first impressions
3. Get groovy for better shell scripts

Not just small talk

A lot of people seem to think that Twitter is all about “sitting on patios” or “my MBP did these things”, and that’s understandable. There is a lot of generally useless stuff and a lot of people just use it for that. You don’t have to. You don’t have to follow anyone that only posts that and you certainly don’t have to tweet that way. It is what you make of it.

Only one piece of the puzzle

Twitter is great, but it’s not a good replacement for anything except for the questions you might have asked on IRC. You still should be subscribing and reading DZone and individual programming blogs. You still need to try to make it to your local programming group meetings. You will still need to check out Google Groups or random forums for some answers. If you do it right, though, you can enhance your experience by engaging others who choose to be in community with you.

http://www.flickr.com/photos/walkn/ / CC BY 2.0

It requires extra effort on your part to sift through posts to find stuff you’re really interested in. I usually spend about 20 minutes a day reading and tweeting. This time doesn’t come from my work time, but my personal time at home or my lunch break. You might not have that kind of time. You may not have an iWhatever to check it while you’re waiting for your coffee to brew. Or maybe you just don’t like people and have no intention of interacting with anything without a screen.

Don’t just consume, engage

I find that a lot of programmers tweet about what they’re working on and give sneak peeks (because they want opinions) to twitter before any of their stuff hits the mainstream aggregators (DZone, Proggit, whatever). The big advantage here is that this is the point of “involvement”. You tend to play a more active role in the programming community when you have access to things in early stages. Furthermore, you open yourself up to more joint projects because you share a greater connection to mutual followers.

I’m not saying that you can’t engage on DZone or whatever, but I am saying that it’s harder to do so. For me, all those “stupid fluff” tweets get me more in tune with a larger variety of programmers habits and abilities.

Conclusion

Obviously, Twitter is not for everyone. The bottom line is that you can’t knock Twitter until you’ve given it an honest try. Like a whole week, not just a skim through someone’s tweets. It’s not going to be what you expect but I think most of you can find a way to make it valuable.

Now, a poll

I’m introducing polls (requiring Javascript for you subscribers out there ;) to this blog, starting with “How useful is twitter to you as a programmer?”. I want you all to come vote and then comment on why you voted the way you did. Be specific, too. This is not going to be a flame war, and I reserve the right to re-word all mean-spirited comments to sound like toddler temper tantrums.

Related posts:

1. Why programmers should twitter
2. Get groovy for better shell scripts
3. A programmer’s 2009 resolutions

You might be laughing to yourself if you already do tweet, but you’d be surprised at how many don’t.

Network with other programmers

I’ve said before that networking is one of the best things you can do for your career. Not only can networking help you find a job, but I’ve found that programmers are generally very helpful and accessible on twitter. Programmers love it when you ask about a project they are involved with. Call it an ego thing.

One other really cool networking use for twitter: having something to talk about when you meet at JUGs or conferences. A follow request is an instant icebreaker and gets the networking process started for you.

Keep up on trends

You already know that software engineering is a constantly changing field. Twitter is a great way to learn how and when to branch out from your areas of focus. Even better, if you have a question about a new technology you can ask your posse without fear.

Be careful, though, there are a TON of cool things out there. Pursuing all of them is idiotic, you’ll just end up burning yourself out. Trust me ;)

Promote yourself

Networking in today’s world seems to be just as much about building your personal brand as meeting others. Hiring managers love to find passionate and honest programmers. There is little better proof than tweets that show you’re thinking in the programming world (the best proof, though, is open-source code IMO).

Why do you use twitter? Share with a comment or a tweet!

WARNING! Shameless plug: Don’t forget to follow me on twitter. If you show passion for programming, it’s likely I’ll follow you back :)

Related posts:

1. Follow-up: Why programmers should twitter
2. A programmer’s 2009 resolutions
3. 5 great ways to take advantage of JavaOne

Over the last couple months I’ve been taking a bit of spare time to write something that I hope all of us can benefit from: GroovyRTM

What is GroovyRTM?

GroovyRTM allows you to easily take advantage of the Remember The Milk REST API using any language on the JVM. In short, you can now write apps for Remember The Milk without having to worry about all the HTTP transaction stuff, error handling, etc. As its name implies, it’s written in Groovy, which made it much easier to write and test.

How can I use it?

To get started, you’ll need 3 things:

1. Download groovyrtm-all.jar from the project on kenai.com
2. Request an API key (you’ll probably want an RTM account to test it with, too)
3. Create a RtmService.properties file as explained in the project Wiki

UPDATE: I’ve created a quick-start project that has everything you need to get started quickly! I included an application authorization example because that’s the first hurdle. Download it here

Just how easy is it? Suppose you want to write a little Java app that adds a new task:

import org.eriwen.rtm.*;

class MyGroovyRtmTest {
    public static void main(String[] args) {
        RtmService groovyrtm = new RtmService();
        groovyrtm.tasksAdd("Try GroovyRTM!");
    }
}

It’s that easy! Now, it’s a little deceiving because the first time your app runs you have to allow it access via the RTM website. The best part is… it’s open source! You can check out the source with Mercurial:

hg clone https://kenai.com/hg/groovyrtm~mercurial groovyrtm

Awesome! Can I help?

Of course! Get the code, give me suggestions, use it and tell me what you think. I am willing to add people to the project that show me their interest and some good Groovy code. Just hit up the forum or use my contact form for questions or suggestions.

If you think it’s cool but you can’t use it, help me get it out there by digging it or voting on your favorite aggregator.

Related posts:

1. Cheqlist: A free, open-source desktop app for Remember The Milk
2. What and why: the tech behind Cheqlist
3. Crush images on the command-line with Groovy

Today I’m going to explain the use of features like history, brace and file expansion, and other tricks by example and give you references for later.

Master your history

Those who forget history are doomed to repeat it. This is arguably one of the best productivity enhancing features of any shell.

You can check your history with the history command, which prints your last 500 commands (or so) by default. Alternatively, you can filter the list:

# Print last 10 entries
history 10

# searches history for cmd
history | grep cmd

Each entry has a number, which you can then execute with !<number>

Now suppose I want to copy a file to a directory and then change to that directory. The quick way to do that with history is:

cp myfile.txt my/directory/path
cd !$  # cd my/directory/path

or if I forget to run a command as super-user:

vi /etc/fstab  # oops!
sudo !!  # sudo vi /etc/fstab

to execute the last command starting with "mount", since I don’t want to type it all out:

# Previously...
mount 192.168.0.100:/my/path/to/music /media/music

# Later...
!mount
# Repeats last mount command

Note that I often (but not always) prefer Ctrl-R, which will search history as you type. As an added bonus you can view the command before executing it.

Other examples:

eric@sawyer:~$ echo foo -a bar baz
foo -a bar baz
eric@sawyer:~$ echo !:3-4
bar baz
eric@sawyer:~$ !-2 #2nd-to-last command
foo -a bar baz
eric@sawyer:~$ ^ba^ya #replace 1st "ba" with "ya"
foo -a yar baz
eric@sawyer:~$ !^:p #MUCH cooler than "echo ..." ;)
foo
eric@sawyer:~$ !?bar #Last command containing "bar"
foo -a bar baz
eric@sawyer:~$ !:gs/ba/ya #replace all "ba" with "ya"
foo -a yar yaz

Quick reference

Download as PDF

Helpful .bashrc entries for history

Copy and paste these into ~/.bashrc

# Don't put duplicate lines in the history
export HISTCONTROL=ignoredups
# Ignore successive duplicate entries.
export HISTCONTROL=ignoreboth

# Store a lot history entries in a file for grep-age
shopt -s histappend
export HISTFILE=~/long_history
export HISTFILESIZE=50000

# No reason not to save a bunch in history
# Takes up several more MBs of RAM now, oOOOooh
export HISTSIZE=9999

# Ignore dupe commands and other ones you don't care about
export HISTIGNORE="&:[ ]*:exit"

Another neat trick with .inputrc

If you are still particularly fond of the up and down arrows, copy and paste the following into a ~/.inputrc file. This will allow you to start typing a command and then hit the up-arrow to search backwards through your history for commands starting with what you typed. I prefer other methods usually but this is pretty cool, huh?

"\eOA": history-search-backward
"\e[A": history-search-backward
"\eOB": history-search-forward
"\e[B": history-search-forward
"\eOC": forward-char
"\e[C": forward-char
"\eOD": backward-char
"\e[D": backward-char

Further reading

Peteris Krumins has an excellent write-up called The Definitive Guide to Bash Command Line History which goes in-depth on many of the above topics, should you crave more bash history goodness.

Brace expansions

No shorthand list would be complete without the (in)famous brace expansions. Basically, they allow you to specify part(s) of an command to repeat substituting different values of a set within braces. Let me show you what I mean:

# Quickly make a backup
cp file.txt{,.bak}
# Equivalent to 'cp file.txt file.txt.back'

This is obviously very useful to prevent having to repeat parts of files or directory paths.

Suppose I wanted to make a template folder structure, I could make most of the directories I need with:

mkdir -p {src,test}/com/eriwen/{data,view}

This will expand every combination so I end up with src/com/eriwen/data, src/com/eriwen/view, test/com/eriwen/data, and so forth. Being able to create a template directory structure with one line is a big time saver!

Better filename expansion

I'm sure you often use the * operator to match files beginning or ending with something, but bash goes far beyond that. It should be noted, though, at some point a good find | grep is more powerful and useful. See Find is a beautiful tool for more information.

In addition to wildcards with *, you can use ? to match any single character. You can also limit matching to certain characters with []. For example:

ls
# prints "myfile netbeans.conf netbeans-6.5rc2 netbeans-6.5 netbeans-6.7 src"
ls netbeans-6.?
# matches "netbeans-6.5 netbeans-6.7"
ls netbeans-6.[1-5]*
# matches "netbeans-6.5rc2 netbeans-6.5"

.bashrc entries for better filename expansion

# Include dot (.) files in the results of expansion
shopt -s dotglob
# Case-insensitive matching for filename expansion
shopt -s nocaseglob
# Enable extended pattern matching
shopt -s extglob

cd shorthand

There are a couple quick tricks to change to oft-used directories. For example:

# Lame way to go home
cd ~

# The cool way
cd

You can also switch to the previous directory with cd - like so:

pwd  # prints /home/eriwen/src
cd /my/webserver/directory

# Do something...

cd -
# Now I'm back in /home/eriwen/src

To get more advanced with this, try mastering your directory stack with pushd and popd.

Conclusion

Effective use of history, brace expansions, and other shortcuts will to save you a lot of time. However, nothing is more productive than no typing. Automating things is best where possible. If you can't automate, use smart aliases.

I know I did not cover tilde expansions, shell parameter expansions or bash key commands. You will want to check those out as well, but I find them less useful than what I've covered here.

Have any quick shortcuts you love? Share them in the comments!

Related posts:

1. grep is a beautiful tool
2. Using aliases and command-line functions for speed
3. Start using crontab for automation