jaXzin.com

Best Practices with Maven: OSS forks

2010-03-21T12:05:00.000-04:00

Recently I came across a company that is forking several open source Java projects. I saw they were making a mistake that I also made a few years ago and have since learned from.

In Maven's distributed repository architecture project artifacts, like JAR files, are uniquely identified by a coordinate system composed of a group identifier, an artifact identifier, a version number, optionally a classifier, and a packaging type. For instance, the most recent version of the Apache Commons Lang project has a Maven coordinate (i.e.groupId:artifactId:version:classifier:type) of commons-lang:commons-lang:2.5::jar.

A few years ago, if I wanted to make custom changes to this project I would get the source, make my changes and then deploy the result to our private Nexus repository under a new groupId such as com.jaxzin.oss:commons-lang:2.5::jar. That might seem reasonable. Then a year later or so I tried something different and changed the artifactId like this commons-lang:commons-lang-jaxzin:2.5::jar.

Unfortunately there is a serious problem with both of these approaches. Maven supports transitive dependencies which means, if you include a dependency you get its dependencies 'for free'. But what happens when you depend on com.jaxzin.oss:commons-lang and indirectly include commons-lang:commons-lang? With either approach, Maven has lost all knowledge that these two artifacts are actually related. And when I say 'related' I mean they include different versions of the same classes. When Maven loses this relationship, it can't perform version conflict resolution and will include both versions in the output. It will compile against both in the classpath. If you are building a WAR file, it will include both in the WEB-INF/lib directory. If you are assembling or shading an "uber"jar, it will include the classes from both in your giant jar with all its dependencies. And unfortunately, the one that 'wins' is nearly indeterministic.

So what's the solution? How do you properly fork an open-source project privately?

The trick is to change the version, and leave the groupId and artifactId alone. That way, Maven still can detect the relationship and can perform version conflict resolution. So to complete the example I would fork Commons Lang 2.5 to a new coordinate commons-lang:commons-lang:2.5-jaxzin-1::jar.

Now I do have one further suggestion, but it's of questionable practice and I'm not sure how well it works. You might consider forking version 2.5 to version 2.6-jaxzin. This way, if Maven attempt to resolve version conflicts, it will know that your fork is 'newer/better' than 2.5. Maven sees version with qualifiers as being older than the unqualified version. I think the assumption is that if you are qualifying a version its a pre-release version like 1.0-alpha-1, 1.0-beta-1, or 1.0-rc-1. You can read more about how Maven version conflict resolution works and I know they have a major overhaul of this logic available in Maven 3.0 with the Mercury project.

But, in practice, when I've run into version conflicts like this I will add an exclusion clause where I depend on an artifact that is including the conflict transitively.

Not for Adoption

2010-03-18T15:20:00.002-04:00

Last night was my first session as a volunteer at the Danbury Animal Welfare Society (DAWS). I had attended an orientation a few weeks back and that's when I saw the facility for the first time, learned about the standard operating procedures and policies, and got to meet some of the cats I'll be working with. Now I'm not a person that enjoys change or meeting new people, but other than my immediate family I don't think many people are aware because I try very hard to hide my discomfort. Who knows, I could be wrong so feel free to call me out in the comments!

So far, this entire experience is quite out of my normal comfort zone, but I'm forcing myself to do this for several reasons. I learned about DAWS after attending their Puppy Love Ball, a fund-raiser they held in February, in support of a friend of mine who was honored as their Person of the Year. They premiered this mission video at the event and it had me hooked. To learn that DAWS is a shelter that doesn't euthanize animals for non-medical reasons was a real inspiration to me. I didn't even know this idea of a 'no-kill' shelter even existed. Another reason, I have wanted to volunteer somewhere simply to give back to a community. I'm not sure where my misunderstanding came from, but I've always imagined volunteering to be an unpleasant activity, like cleaning or manual labor. So when I found out that I could volunteer to be a 'cat socializer', it sounded like a perfect fit. Since my wife is very allergic to animal hair and dander, this would also be my chance to interact with cats on a regular basis without owning one.

This next statement may come off as brutally honest, another reason I was outside my comfort zone is the facility. Inside, its a clean, spacious and wonderful facility for the animals, but when I first arrived its exterior isn't exactly what I had expected especially since my introduction to DAWS was the grandeur of the Ball held at the Ethan Allen. The building looks like a large residential home and could easily be overlooked as a place of business. After visiting the DAWS location in Bethel I understood the importance of the sign and landscaping that my friend Paul created. Hearing about their goal to fundraise for a brand-new state-of-the-art facility by 2019, it made me even more impressed and connected with DAWS. I'd love to be a small part of them reaching that goal.

I arrived last night a few minutes early, uncertain what to do. Seeing others waiting outside for the door to open, I introduced myself and found out they were waiting to become adopters. I was starting to get a little nervous since none were other volunteers. Was I late? Was there a different entrance? Did I not read an email fully? After a few minutes I discovered I hadn't made a mistake when other volunteers arrived.

After a few minutes we were let in and I introduced myself and was introduce to the other cat program volunteers that were there. Unfortunately I'm joining the volunteer program at a time when everyone needs to be more hands-off with the cats because of a skin infection that is making the rounds in the shelter. They have done an amazing job handling the situation. I hope in a few weeks the issue will behind DAWS.

From what I gathered from the my first session, a two-hour evening consists of feeding the cats, playing with them, cleaning litter boxes and then finishing the night by getting everyone back in their cages. This is the kind of volunteering I can handle!

The thing I really took away from last night though was my experience with a cat named Daphne. She's a beautiful black & white cat and very reminiscent of the family cat I grew up with for 14 years named Purrina. She has some trouble with her hind legs, and she seems a bit of a special-needs case because of it. On her cage is a sign that reads "Not for Adoption" and those three words seemed to say so much about DAWS. Here is a cat that in many cases would be euthanized because no one goes to a shelter looking for a special-needs pet. But instead DAWS, with their no-kill policy, chose to give this cat a chance to be adopted and she's so close to going home.

Daphne cemented why I want to give two hours of my week to DAWS, in a time when I feel like I don't have 5 extra minutes to spare. If you are located in the greater Bethel/Danbury area, I encourage you to donate your time or donate your money.

First Impressions from NoSQL Live

2010-03-12T00:40:00.003-05:00

Today I drove up to Boston for the day to attend NoSQL Live. My experience so far within the NoSQL community has been limited to what we've built in-house at Disney and ESPN over the past decade to solve our scaling issues, more recently has been ESPN's use of Websphere eXtreme Scale, and the very latest has been my own experimentation with HBase which hasn't gotten much further than setting up a four node cluster. I've read a little about Cassandra, memcached, Tokyo Cabinet and that's about it. So before the sandman wipes away most of my first impressions of the technologies discussed today, I wanted to record my thoughts for posterity or, at the very least, tomorrow.

Cassandra
Cassandra seems to be the hottest NoSQL solution this month with press about both Twitter and Digg running implementations. My impression, I'm wary of "eventual consistency". I don't feel I understand the risk and ramifications well enough to design a system properly. When Jonathan Ellis of Rackspace Cloud mentioned that Digg needed to implement Zookeeper-based locking on top of Cassandra so that diggs get recorded correctly, I realized how poorly I understand eventual consistency and how risky it could be. But my impression of Cassandra isn't all negative, it definitely seems to have less baggage than HBase by not being built on top of HDFS. I'll get into what that means a little later.

Memcached
Unfortunately the speaker that 'represented' memcached gave off a vibe that really turned me off to the product. I know that's incredibly shallow, but this is first impressions after all and not perfectly-evaluated impressions. Mark Atwood sat on the first panel of the day "Scaling with NoSQL" and his whole attitude seemed to say "memcached is all you'll ever need and these guys next to me are just overdesigning hacks". His answers were short and his tone was quite condescending even when addressing audience questions. Not a very good first impression of him. But luckily today wasn't my first impression of memcached as I was pointed in its direction just last week by a Disney colleague. My research before today has me intrigued about using it as a replacement for ehcache as a second-level cache provider to Hibernate which we use as an ORM in one system at ESPN right now.

Document Oriented Databases (Riak, CouchDB, MongoDB)
Wow, this is a subgroup of NoSQL technology that I had heard of in passing but was really unaware of what problem they were trying to solve. Riak had the best answers for scaling and operational-ease. With homogeneous nodes and consistent hashing, Riak promises that adding and removing nodes are seemless. CouchDB and MongoDB sounded like a 'me too' answer so I'm interested to find out what that really means for each, or better yet what it doesn't mean. But the concepts of document-oriented databases really meshes well with ESPN's current fantasy user database. Our fantasy user profiles are stored in a traditional RDBMS as serialized maps of maps, one row for each user. Since its serialized to a BLOB column its completely opaque to reporting and analytics. To keep that model but have vendor support for divining information and having transparency into it sounds exciting. I really need to look into these. Riak definitely won this round of first impressions.

Tokyo Cabinet
This was a technology I was referred to by a colleague and read through their site last week. I was far from impressed then since it seems much too low-level for my taste, similar to my impressions of Carbonado which we use at ESPN. The lightning talk by Flinn Mueller got me a little more interested. He seems to be doing interesting things but from an analytics and reporting perspective. He was vague on how loads the data from his primary store and what the scale of the data is, so my first impression: its a toy. I'm sure that's an unfair characterization but I'm not trying to be fair tonight. But honestly, Tokyo Cabinet makes no bones that it punts on horizontal scaling which is the deal breaker for me.

Hypertable
I looked at Hypertable (as in read their website) about 18 months ago on the suggestion of a colleague when discussing HBase around the same time. This conference didn't change my opinion, which is "It's HBase but written in C". It doesn't seem to bring anything else to the table which to me is a blocker. JVM implementations are available for all the operating systems I use and so I don't like the idea of needing to find the right binary to download for a given box. When it comes to Java vs. C, I choose Java but I'm also extremely biased as I've been a Java developer nearly my entire career.

Full-stack JavaScript
This was my favorite of the lightning talks, and possibly my favorite of the conference. It felt a little tangential to the NoSQL topic, mainly because Jim Wilson covered more than just data storage. The idea, what if you could use JavaScript on your server, in your client and use JSON for talking between the layers and as the storage format? Crazy, right? I say brilliant. His few slides were mildly embarrassing that dissected each of the popular stacks of today by how many languages you need to learn (Java, XML, etc) as well and the various impedance mismatches between layers (ORM, Object marshaling to JSON or XML, etc). "ORM is an antipattern" was an enlightening take on something I've accepted as necessary. Full-stack JavaScript is something I'll be lusting after for a long time, especially since he made it sounds so attainable with node.js, rhino and MongoDB. As soon as his slides are online I'll be linking to them as well as passing them around the office.

HBase
Well I saved HBase for last. It's the one I've had the most experience with, though that experience can still be measured in hours. As I hinted at earlier, this conference gave me the first impression that HDFS is a weight around the neck of HBase. I was surprised to get that feeling from the room, since my impression has been purely positive so far. It is also getting a lot of flack from the 'single point of failure' problem associated with the current HDFS architecture's Name Node. Apparently performance is a dog since it was "only" designed to be highly distributed with no promise of when you'll get your data. This burden seems to carry over to HBase. But after talking to Ryan Rawson one-on-one at the end of the HBase lab, it's clear he is of the strong opinion that its getting a bad wrap. He also makes very convincing arguments about the scale of what HBase is currently doing in real production environments vs. competitors like Cassandra. It's very pursuasive and you can read more of the details in a very active thread I kicked off on the HBase user group earlier this week.

Conclusion
HBase is still the front-runner of my personal candidates for a NoSQL option for ESPN as it has been for a long time. Cassandra's design choice of eventual consistency is a little scary to me because I don't know yet how to design for it, not because it is inherently bad choice. Documented-oriented databases just made a big blip on my radar. Memcached is interesting if I want to stick with a traditional ORM-based architecture. Tokyo Cabinet and Hypertable are all but off my radar. And the lusty vixen of them all is a full-stack JavaScript architecture.

Disclaimer: Though I mention my employer ESPN in this post, these are my own personal opinions and don't represent the opinions of the company. The final decision on this stuff is "above my pay-grade" as they say.

Java Puzzler

2010-02-24T13:10:00.013-05:00

I bought the book Java Puzzlers by Josh Bloch and Neal Gafter a few years ago and enjoyed it. If you aren't familiar with it, it covers rare, odd and usually counter-intuitive edge cases of Java in a brain-teaser style of presentation. For both a Java user and an analytical mind its a fascinating book.

Well today I stumbled onto a puzzler of my own and thought I'd share. Can you tell me what the main method of this class will output?

1 : public class Parser {
2 :
3 :  private static <T> T parse(String s, Class<T> type) {
4 :    // Simple implementation only supports long primitive
5 :    if(type == long.class) {
6 :       return (T) Long.parseLong(s);
7 :    }
8 :    throw new UnsupportedOperationException(
9 :                   "parse() only supports long right now!");
10:  }
11:
12:  public static void main(String[] args) {
13:    System.out.println(parse("1234", long.class).equals(1234));
14:  }
15: }

So what gets written to standard output? If you said 'true', then you'd be wrong. If you said 'false', then you'd also be wrong! The code above won't even compile! There are a few features of Java that are colliding here to give odd results, including generics, type erasure, primitive types and autoboxing.

So what happens exactly? You'll get a compile error like this:

Error:line (6) inconvertible types
found   : long
required: T

The error is on the return statement line of the parse() method. Long.parseLong() returns a primitive long, but since a parameter type can only represent an Object type, you can't cast a primitive to an parameter type. Also, it appear auto-boxing doesn't work in this case, so it fails with a compiler error.

But wait, doesn't the code explicitly use the Class object that represents the long primitive, long.class? But see 'long.class' is syntactic sugar for the constant field Long.TYPE which is of the parameterized type Class<Long> since type variables only support using Objects.

So realizing this, we can change the return statement to use a Long instead of a long:

    return (T) Long.valueOf(s);

Now the code will compile, but something is still wrong because it prints out 'false'.

Knowing that the return type of parse("1234", long.class) is Long, take another look at the puzzle. Can you see the problem? This one is subtle. The numeric literal '1234' that is inside the call to equals() needs to be auto-boxed and since it is a literal int, it is boxed into a Integer. But an Integer object isn't considered equal to a Long object because they are different types, so equals() returns false.

So it a few interesting features combine for some strange behavior. Hope you enjoyed this puzzle!

UFOs, Ghosts and Bigfoot

2010-01-25T23:29:00.006-05:00

So I hope you don't think I'm a complete loon, but I've always been interested in the paranormal, cryptozoology and lights in the sky. I think it is driven by my need to try to answer all those unanswered questions in life.

Lately I've been watching various 'reality' shows on these topics like MonsterQuest, Paranormal State and Ghosthunters. Don't get me wrong, I'm not gullible enough to believe these shows are finding evidence of anything, its just some good ol' fashioned mind-numbing popcorn TV for me to fall asleep to.

Watching them has reminded me of a few unexplained incidents I've had in my life and I wonder if you've experienced anything similar? I've got three that all happened to me over a decade ago but were so unnerving that I remember them vividly. So here are my campfire ghost stories, feel free to share yours!

Incident #1: The Mysterious Flash
I don't remember the exact date or even the season of this first incident, but my best guess is it happened during the summer of 1993. It was the summer before my sophomore year of high school. I had a goal to read 50 books that summer. I know what you're thinking "wow, was he really that cool?" My motivation was a challenge from my freshman English teacher that I couldn't do it, and I ended up proving him wrong. The reason I'm telling you all this is to set the scene that I was in my bedroom reading a book around 10 o'clock at night.

My Dad is awesome and years earlier had installed overhead track lighting in my room attached to a touch-sensitive dimmer switch. The track lights hung from the ceiling and I had most of them pointed outward to face the walls so I wouldn't be blinded. However the one light that hung near the head of my bed I faced inward to act as a reading light. For whatever reason, I was laying in bed with the lights dimmed as low as I could and still be able to read. I can't explain the inner workings of my 15 year old self but I liked the lighting that way.

As I'm laying there in bed with my back propped against the wall my entire room is flooded with brilliantly blinding light! I stare at my book and realize the light isn't coming from above me it's from behind me as I can see my shadow and the shadow of the book I'm holding cast against the far wall of my room like someone is holding a spotlight a few feet behind my shoulder. And before I can react it's gone! The whole flash takes about half a second. Then as I turn to see where the light came from I come to the realization that I'm laying on the wall, it couldn't have possibly come from behind me and cast the shadows I saw. Now being rational, my first thought is that I imagined the shadows and it was a simple power surge so I go downstairs and ask my parents if they saw anything, but they hadn't.

Incident #2: The 727 at 500'
For this next incident I wasn't alone. I was still in high school and my Mom and I were in the car together coming home from an after-school activity, probably band practice or musical rehearsal. It was twilight and as we drove home I see a plane on the horizon. As I watch it I realize it is not near the horizon because its far away, it's because it is close but flying low to the ground. I pointed out the plane to my Mom and wondered out loud if something is wrong with it. She was surprised and decided to pull over to watch it. As the plane drew closer we both realized that it was a commercial airliner, not a small prop plane out of Danbury as I was expecting. Now mind you, I grew up in the hills of New Milford where the closest airport that can land an airliner is 40 miles away. It continued to approach us and at this point I realized it was traveling extremely slow for a jet of its size. Finally as it passed directly overhead I realized it was the size of a 727, no more than 500 feet from the ground, and going maybe a 100 miles an hour which is very close to, if not less than, the stall speed for an jet like that. But if that all wasn't strange enough, here's the part that haunts my mother and I to this day...it was dead silent. I mean so silent that the crickets stopped chirping. The jet passed and disappeared past the treeline and we never heard a thing. In that pre-internet age, my only recourse was to check the newspaper every day for at least a week. Nothing.

Incident #3: Close Encounters of the Orange Kind
My last weird event is probably the clearest of the three in my mind. It was September of 1995 near the beginning of my senior year of high school. My Dad and I were flying to Florida for the weekend so I could tour the Ringling School of Art and Design and see if it was where I wanted to spend the next four years of my life studying computer graphics. In the end, I went to Trinity College in Hartford but that's another story.

On this trip we flew down on a Friday night. It was one of the first night-time flights I'd ever been on so I was pretty excited. We were seated on the left side of the plane that faced the Atlantic Ocean for most of the trip. Even without much to see but the blackness of the ocean I still sat watching out the window almost the entire trip. Midway through the trip I got to see thunderhead clouds out on the horizon. It was a lot of fun watching a lightning storm from 200 miles away. But as I watching I realize that one of the clouds has begun to light up with a constant orange light. The cloud wasn't lit in a diffuse way, it was very clear it was emanating from a point within the cloud. If you've seen the movie Close Encounters of the Third Kind in the scene where the UFOs arrive you can only see points of lights within the clouds, it looked exactly like that but for only one light source. I'm not saying I think this was a UFO, only that the special effects in this movie are a good approximation of what I saw. So I immediately got my Dad to lean over and take a look to see if he had any idea what it was. As we sat there and watched, it became extremely bright. It was so bright that you could see the light cast through the windows onto the right wall similar to what happens when the setting sun is off the left wing. We watched it brighten for about 30 seconds, stay at full brightness for a minute or two and then fade to nothing in about the same amount of time it took to arrive. To this day I have no idea what I saw.

My thoughts on Google's Chrome OS

2009-11-24T15:22:00.001-05:00

Unless you've been living under a rock, you have heard that last week Google demonstrated their Chrome OS for the first time in public. They also open-sourced what they have so far as the Chromium OS project. You can watch an overview video here:

I followed the event, watched the YouTube videos and even played with a copy of Chromium OS via gdgt.

So here are my thoughts on my limited experience so far. It seems like a perfect fit for the netbook market. "Living in the cloud" seems like a great idea for the ability to access your data anywhere. A verified boot system will be a godsend to anyone who has ever had reinstall an OS because it just got 'crufty'.

But I don't understand why they have, so far, omitted the concept of a desktop. One of the coolest features of Chrome is its ability to turn a web app into a 'desktop' app, which prompted me to to find Fluid for the Mac. Why force new users of Chrome OS to live in a browser when they could have still kept the concepts of a desktop with 'desktop' applications? There is also a nearly direct analogy in Chrome OS between its tabs and the Windows task bar or Mac Dock. So why make it feel so different instead of trying to fit into the conventions every other GUI-based OS already established?

I know it is fun to "rock the boat" and "shake the tree", but I'm just not finding the benefits to tying so closely to existing browser paradigms instead of reusing some of the desktop conventions for this new 'web' OS.

Maven and Legacy Projects

2009-11-16T16:40:00.005-05:00

Recently I've been helping a colleague introduce a dependency on a legacy project that hasn't been "Maven-ized". Often the "easy way" to add a legacy dependency is to throw the JAR into our Nexus repository and be done with it. But this legacy project was a little trickier and is my new poster child for why taking the easy way can lead to headaches and frustration.

Your legacy project might have compile-time dependencies you need.
Maybe your legacy project implements an external API, or maybe it uses a library and exposes that usage in its API. "So what", you say? "I'll find it at compile-time for my project and I'll add the dependency." Well, now every project that wants to use your legacy project is going to go through the same discovery and embed the dependency in their POM. The right way is to define this dependency in the pom.xml for your legacy project so all dependent projects retrieve it transitively.

Your legacy project might have run-time dependencies you need.
Similar to the last example, your legacy project might use a library but not expose it's use directly. You won't discover the missing dependency until you try and run your project. Here's where the power of the 'runtime' scope of a transitive dependency becomes obvious.

Your legacy project might include library classes within its JAR.
Now here's a real yuck factor when it comes to dealing with other people's code. Our example legacy project bundled a small subset of its dependencies right in its JAR. It wasn't until after we threw it into Nexus that we discovered this rather severe violation of Maven standards. The best solution, get access to the project's source, mavenize the project properly and release a 'pure' artifact. If you don't have access to the legacy project's source you'll need to strip the extra classes from the artifact and construct a pom that includes the right dependency.

Summary
So here's a really quick summary of why not to half-ass adding legacy projects into your Maven infrastructure:

Any transitive dependencies will be missing.
With legacy jars that embed the classes they depend on you risk class file collisions

Medicine is so Young

2009-11-10T21:50:00.000-05:00

I think its undeniable that one of the hottest topic of this year is health care. While I've been casually following the political discourse, its far from my inspiration for today's post. My one year old son, Wally, had a reaction to penicillin this week and I question why current medicine is still so reactive rather than proactive. Since allergies are something that can change and fluctuate, that led me to also wonder why we don't have more frequent testing too.

Standardized testing
I'm looking forward to the day when there is a standard battery of tests for allergies and conditions. It's beginning with the prenatal screenings available during both of my wife's pregnancies, but why, for instance, is there no standard allergy testing post-birth? From the perspective of a patient, current medicine waits for a reaction to an allergen which seems so dangerous to me.

Continuous diagnostics
When can I get my little implant that continuously monitors my blood-work and vitals? I've been dealing with the medical 'miscellaneous bucket' that is IBS for 6 years now and don't feel like I'll ever get an answer about what is really afflicting me. I see my doctor once a year, and she doesn't really seem to have an answer. I know, I know, demand answers, fight for myself, blah blah, that's not my point. It's currently not 'bad enough' to force me to demand anything yet. I bring up my current state during my annual checkup, and that's about all I feel I'm empowered to do right now. I strongly believe if there were a way to easily and continuously monitor medical data points (blood, hormone, etc) within my body and let me correlate that with the ebb and flow of symptoms there would be a much higher chance of being able to find the problem.

Online records
I've been 'using' Google Health since it was released last year. But my doctor's office has no tie to my account so I'm forced to update it by hand. Not to mention all my historical data is no where to be found. For the providers that Google Health does currently support linking, I have a 50% success rate getting them to work. I was able to link my CVS prescriptions but failed to link my Quest test results. When I followed Quest's instructions, the receptionist at the testing facility looked at me like I had three heads when I asked to get my results online. She had no idea what I was talking about.

What REST is NOT, Updated

2009-11-04T13:26:00.003-05:00

In a surprising update to my post earlier this week about what REST is NOT, I just spoke with Rooly Eliezerov, the President and co-founder of Gigya. We had a nice conversation about the goals of Gigya, primarily that they want to aggregate these other social web services and their design philosophy is to mimic Facebook and Twitter's API. I expressed that using GET to change a resource isn't a REST principle either company's API breaks. He was surprised to hear it and said he'd follow-up with his team to validate that and get back to me.

Whatever the outcome, I really appreciate that a company with their goals and reach is interested in what little ol' me has to say. It speaks volumes to their passion and customer support.

Problems of the MPAA's Digital Copy system

2009-11-03T18:59:00.009-05:00

When I got 'Wanted' on DVD as a gift, it was the first movie on disc (DVD or Blu-ray) that included a Digital Copy. Since my home media setup is all iTunes/Apple-based I easily popped in the extra disc, entered the included key and added the "Wanted" to my media collection in less than 5 minutes. It was beautiful and enlightening. I'm on the side of the fence that copying a movie onto a private home media network is an ethical, though not currently legal, fair use of copyrighted works.

Usually ripping a DVD for me is a 'set it and forget' operation using Handbrake that takes several elapsed hours but little interaction. But with the latest copy-protection features of DVDs it can be a hit-or-miss prospect which has caused me to add a few corrupted movies to my collection. And if I want to take the time to attempt to circumvent the copy-protection using Mac the Ripper or RipIt, it's again hit-or-miss and makes the process that much more labor-intensive, at a time in my life where I have little desire or time desire to dedicate to building my media collection.

Seeing how quick, easy and reliable using the Digital Copy process was and how it resulted in a copy that can play on my mobile device (iPhone), HTPC (Mac mini) and set-top box (AppleTV) was exactly what I've wanted! And now having a Blu-ray player but no way to import those movies into my media collection, I thought Blu-ray + Digital Copy was the killer combination. But with my third purchase of a feature that included a Digital Copy, the honeymoon is over.

Here's what I think is missing from the current Digital Copy system:

DRM-free
HD
Quality consistency
Ownership transfer/lending

DRM-free
At first, the DRM didn't bother me. Then I started to worry about what happens when I have a catastrophic data failure of some kind? Sure I back up my files including my ripped videos, and I've even started making a second off-site backup but will I have to jump through hoops to play them again? How simple is the process of re-adding them to my collection if I get a new computer?

Also, I found expiration dates for all three keys I currently have, what if I have to reauthorize them after the expiration date? In fact my third purchase was made after the listed expiration date, but luckily I didn't have a problem activating the Digital Copy, which was a bit of a relief. In general I'm worried about the ramifications of an expiration date on the activation key.

High Definition
Now if I buy a DVD with a Digital Copy, I'm not greedy and I don't expect an HD Digital Copy. But for the one Blu-ray I've purchased with a Digital Copy I'd expect at least a 720p HD copy as if I had purchased it in HD the iTunes Store.

Quality consistency
Another surprising thing for one of the movies I purchased, "The Nightmare Before Christmas", I noticed a drastic difference in the aspect ratio of the Digital Copy compared to the DVD video. The DVD was clearly full-frame 16:9 but the Digital Copy was somewhere between 4:3 and 16:9. My guess is its the original theatrical aspect ratio and the DVD version was cropped to 16:9.

Ownership transfers and lending
What happens when I want to lend, give or sell the DVD to a friend? I think what Barnes&Noble is doing with the Nook is the most realistic option, allow 14-day lending of digital rights to a friend. If they can make it work, why can Apple and Microsoft?

What REST is NOT

2009-11-02T21:09:00.017-05:00

Update: I spoke with the president of Gigya.

Having only familiarized myself with REST a few months ago, I still struggle to explain to colleagues what REST is. My best one sentence summary so far is that it's "using HTTP as it was designed". Not a perfect or even complete summary, but I think it covers the highlights.

This week I've been reviewing Gigya's REST APIs for an upcoming project at work. And I've found it to be a shining example of designing an API incorrectly and calling it RESTful. It solidified a couple of REST anti-patterns for me and I wanted to capture them:

REST is not RPC, though RPC can be done RESTfully.
GET is not for state changes. If you're operation has side effects, don't use GET.
The response body is not the only way to send the client information. Don't duplicate information in the HTTP headers in the response body.
Authentication is not part of your resource. Don't create a query parameter to do authentication, use the HTTP header that already exists for the purpose.

REST is not RPC
Many developers coming from a SOAP background think RPC calls can be ported to 'REST' and then call it a day. They are missing a big mental shift in the way to think of web services when designing using REST principles. Instead of thinking of services as functions that can be called, change your thinking to performing operations on objects, formally known as 'resources' in REST parlance. The operations you can perform on your resource are the existing HTTP methods, GET, POST, PUT, DELETE, OPTIONS and HEAD. So instead of exposing a 'setStatus' function via a web service, instead expose a 'status' resource that can be POSTed or PUT to change it, call GET to retrieve the current status, or DELETE it to clear the status. REST is not functional RPC, its a paradigm shift to treating web services in an object-oriented way.

GET is not for state changes
The most important REST principle that Gigya's APIs violate is using GET calls to affect the state of a resource. Short of recording access, like last viewed by/on, GET calls should never affect the state of the resource it represents. If you need to change a resource, there are three other HTTP methods for that purpose, namely POST, PUT and DELETE. The use cases for DELETE are pretty self-apparent, but I've got to admit to being confused of when I should choose POST vs. PUT. John Calcote clarified the answer nicely of when to choose POST over PUT.

The response body is not the only way to send the client information
HTTP headers, for both requests and responses, can be an invaluable way to simplify your API. They can contain rich information about the success (200 OK, 201 Created, 204 No Content) and failure (404 Not Found, 401 Not Authorized, 503 Service Unavailable) of a request. Gigya for example includes the HTTP status code in their XML response body. It seems like a waste of bandwidth to me, when the status code is already included in the response headers. To their credit their documentation seems to imply the codes are more detailed than available in the HTTP spec (500-001, etc) but their examples conflict with that and only show the XML return simple HTTP codes with HTTP reason messages. Even if they provide more detailed information, it should not duplicate what already exists in the request. Trust that your API users can understand and use the headers. Its much easier to only check a status code instead of parsing an XML response.

Authentication is not part of your resource
This antipattern is directly related to my last point of using HTTP headers properly but its important enough to warrant its own bullet. This violation is a version of "don't reinvent the wheel". Gigya has for some strange reason taken OAuth and permuted its use rather than sticking to the spec. They even have API users sign their requests using OAuth libraries but then fail to carry that through into the API usage. The signature a part of the GET query string rather than use the already established HTTP header that the OAuth spec has defined for carrying that data point.

Summary
REST is a very amorphous set of principles but that doesn't mean there are no concrete guidelines. Patterns are emerging as well as anti-patterns and Gigya has examples of both. I don't mean to pick on Gigya but they just happened to be freshest examples in my mind this week. The big points I'd like you to take away from this are, don't reinvent the wheel, use the existing HTTP methods properly, don't forget that HTTP headers exist and design your APIs by thinking of them as objects rather than functions.

Parenthood | Sweetness or Manipulation?

2009-09-22T17:24:00.002-04:00

So Sunday night (Sept 20), I realized my three year old daughter Penelope is either extremely manipulative or extremely sweet. We were doing bedtime, reading a book and bedtime was over. She was supposed to lay down and go to sleep. I went to get a drink before heading up for the night and as I walk past her room again I see her standing at the foot of her bed in the dark. She calls out "Daddy, daddy! Poopies are coming!". Now I've learned this is a common stall tactic with her.

So I said, "No! Poopies are not coming. Go to sleep". But she insisted and so I threatened, that if she didn't really have to go to the bathroom I was going to take away her favorite bunny, which has become her security blanket.

So she agrees and gets out of bed and goes to the bathroom. So she's sitting there and nothing's coming out. And I'm like "Penelope, I told you what would happen right?" So I give her some more time and then come back.

I open the door and say "Time's up, any poopies?". She says "No.", and hangs her head in shame. She then follows it with one of the most heartwrenching statements yet, "It's okay Daddy, I not cry. I big girl. You can take bunny."

So I think oh no, I'm a terrible father, I can't take her bunny... So I said, "Just because of that, you get to keep your bunny." She gives me a huge grin and says "Oh Daddy! Thank you Daddy! I love you Daddy!" And she gives me a big hug and I pick her up, and she's patting my back and says "Good Daddy! Good Daddy!" like I bought her a new car.

Kids are awesome.

1Password is an essential app

2009-07-07T22:28:00.003-04:00

Today I posted a Facebook status that I've come around and consider 1Password a must-have app for every Mac user. A friend commented that he didn't get it, much for the same reasons I held out for a long time. So here's how I explained my new love:

Yeah I didn't 'get' the benefit for a long time. But recently I've been frustrated with the countless number of passwords I've got and how every site has a different policy that forces me to rethink my password scheme. I started to feel like my scheme was becoming too simple just so I can meet the lowest common denominator. Also, I recently find myself forgetting the less frequently used password and needing to reset them. I needed a solution.

The killer features for me:
Killer browser integration, replaces Firefox and Safari password managers
Password generator, now I don't need to fear that my online passwords are crackable. Again #1 makes #2 awesomely simple.
Slick iPhone integration. I can take and update my passwords anywhere. Better Mobile Safari integration would be nice, but that's due to Apple's limitations.
Master password: I know, I know, single point of failure right? I'm now of the opinion its much more likely a hacker will get access to one of my passwords online and figure out my simple scheme rather than them gaining access to my computer and cracking my rather complex master password. For me its more peace of mind and the simplicity of not having to memorize all my different passwords.
Easy syncing via Dropbox. I want autofill not just cross-browser but cross-computer too. http://webworkerdaily.com/2008/09/29/1password-dropbox-sync/
Oh and I remembered to search for coupon codes before buying it. I used "iSlayer" which gave me 20% off.

HandleCheck: My Google App Engine and Open Source Project

2009-04-19T23:48:00.006-04:00

So I signed up for a Google App Engine account a while ago, almost right after it's Python version was released last year. Of course, life prevented me from taking the time to learn Python, and so the learning curve prevented me from building or deploying anything.

Fast-forward to two weeks ago when Google released the Java API for the Google App Engine (GAE from here on out). Being a full-time Java programmer for the past 10 years, now we're running on all cylinders! I applied for a Java account on the day it was released and my Java GAE account was approved within 24 hours, but it took a few days for me to make some time to start writing some code. After about 8 hours of development I had my first working prototype of a web application running on the GAE.

My first GAE webapp is a service that connects to a set of social web sites that provide user names and profile pages for their users to see if a username you are interested is still available across those sites. My app is a called Handlecheck (handle in the sense of "username") and is live now at http://handlecheck.jaxzin.com.

This is also the first project I've open-sourced. I did a little reading on how to pick from the various open source licenses and settled on the GPLv3. I'm maintaining the source on Google Code.

Back Online

2008-09-03T11:07:00.001-04:00

Finally http://www.jaxzin.com is back online after almost a week outage. I'm still trying to get http://jaxzin.com back online, unfortunately the Google Apps hosting doesn't play nicely with Blogger.com.

Jon & Kate + Hate

2008-08-26T01:43:00.002-04:00

Tonight I watched the "Beach Trip" episode of Jon&Kate+8 and was inspired to find out exactly where they went since it looked so much like where my parents own a condo. It turned out to be close but no cigar. This googling session happened to also lead me to a video from Aunt Jodi. After dismissing Aunt Jodi's sister's blog a couple weeks ago as jealous ramblings, I finally took the time to read it through which took me about an hour. The latest entry which includes this video of Jodi verifying the claims her sister has made 'in print', was what changed my mind. I read the posts in chronological order this time which helped me to better understand her tone, rather than just reading the last entry as I did previously. So for those that haven't taken the time to read it here's the summary. Keep in mind this is just my recap of her posts (in case the Gosselins see this little post and get pissed):

The production company offered to pay Jodi for opening her home to the camera crew via Kate starting with Season 2
Kate, without discussing it with Jodi, said no and said "no one but us gets paid" and "no one will make money off our family"
The production company broached the issue again at the start of filming season 3
Kate refused again
The production company approached Jodi directly, and off-the-record shared the backstory with her
She didn't believe it and her husband Kevin, Kate's brother, asked Kate it this was true.
Kate verified it and laughed.
Kate then decided to replace Jodi's services and got TLC to pay for child-care (Jenny)
Jodi approached Kate and said it wasn't about the money and still wants to be a part of the kids' life.
Kate refused saying how could she expect her to do it for free (even though she was doing it for free for years).
Jodi is now involuntarily estranged from the eight children she helped raise.

Between the video from Aunt Jodi and this 3 year old article in a Pittsburgh newspaper I'm beginning to believe Jodi's sister's account of what happened. Reading it through was really heartbreaking especially when she recounts her son's reaction to watching GumGate.

Boom-dee-ah-duh

2008-05-21T22:48:00.003-04:00

Something about this new commercial for the Discovery Channel got me. I don't if its the catchy song, or seeing all their main personalities singing it, or having Dr. Stephen Hawking in it. Whatever it is, I like it.

My family tree from Geni.com

2008-02-20T12:16:00.004-05:00

I saw Geni.com now allows you to embed your family tree in other sites so I decided to try it here. Check out my family tree which we traced back 18 generations for one line!

View more of Brian's tree | Get your own family tree at Geni.com

Importing old emails into Gmail via IMAP

2007-11-13T22:44:00.000-05:00

So Gmail added IMAP support this month and the biggest benefit I got from the new addition was a simple way to import my pre-Gmail email. When I first got my account several years ago I looked into importing all of my old email messages into my Gmail account but found it would be tedious and fraught with problems. The biggest one being there seemed to be no way to import the emails and preserve their old header information such as who sent it, what address they sent it to, when they sent it and when I originally received it. All the import tools I found were nothing more than glorified resenders that simply emailed my old emails to my new account. But now with an IMAP link between Gmail and Apple Mail, my current email client, importing my old email is as easy as dragging and dropping them into my Gmail inbox. Unfortunately Apple Mail 3.0 is still flaky and trying to move too many emails at once causes it to crash. If I import them in about one-month batches it seems to handle it. With seven years of email to import, I'm about three-quarters of the way through the import after three nights of importing a month, waiting, moving the next month, etc. But it's going smoothly and all the original header information is preserved accurately as well as the attachments to the originals.

Identity Theft and You

2007-09-07T23:49:00.000-04:00

I recently was forwarded an email about Google's "scary" (for some) feature of reverse phone searching. You can read more about the email at Snopes.com.

Anyway, I was bothered enough by the false sense of security a FUD email like this provides for those who aren't as wise to today's Internet issues that I was forced to write the following response for my family to read as a counterpoint.

I receive many forwarded emails, but I award few with more than a passing glance. This one is so misguided though, I felt it deserved a response at least to those I know and care about.

Phone numbers and addresses aren't private information and shouldn't be or they betray the very nature of what they are, location devices. Reverse searching for a person or address based on a phone number has been available for a very long time on many websites, the most well known, for me at least, being http://anywho.com.

The advice the original email provided will lead many who read it into a false sense of security while making things harder only for legitimate people looking to contact them. Removing yourself from Google's listing makes it not only harder for the "bad" people to find you but also your friends, family, and business associates. Someone looking to steal your identity or find out information about you has many resources beyond Google in this era of the Information Age.

If you are truly looking to protect yourself from identity theft there are more effective ways:

Create a Google Search Alert for your social security number and/or credit card numbers. This will notify you via email if Google finds one of these numbers on the Internet. That way, if one of these important numbers have been stolen and appears online, you'll be one of the first to know.

Check your credit report annually for free at http://www.annualcreditreport.com. Review it to make sure no one has stolen your identity

NOTE: Beware of freecreditreport.com. It is a well-advertised scam.

Place a fraud alert or credit freeze on your credit report. Its an extreme measure, but it then allows only you can get credit in your name using a private PIN.

NEVER use a debit card or personal checks. They are linked directly to your rightfully earned money with no safeguard and provide you no recourse if someone fraudulently takes money from your bank account. A credit card provides many protections a debit card does not.

Ask your credit card company if they provide single-use credit card numbers. Many, such as Discover, provide an online utility to generate a new credit card number to use for each of your online purchases. This will protect you because if that single-use number is stolen it is worthless.

Find out even more ways to protect yourself: http://www.google.com/search?q=protect+identity+theft.

Now I, Brian Jackson, personally wrote every word of this email and I hope THIS is what you pass on to friends and family!

Maven Quirks: Parents of Dependencies

2007-08-24T14:29:00.001-04:00

So try this in Maven. Create a project A and don't install it. Create two subprojects, B and C, that have A as their parent. Make B depend on C. Install C. Try to build B. You won't be able to as Maven claims it can't download A. Huh? Why did C build then?

The cause is that Maven can resolve a project's parent on the local drive using a relative path. But if a project has a dependency, it will only look up that dependency's parent through the repositories, even if that parent is available via a relative path.

The solution, install A into your local repository. If another developer gets B out of your SCM and tries to build they will fail with the same problem. So I've now learned. Deploy more snapshot builds of parent POMs to keep my fellow developers happy.

Maven for Ant Users

2007-06-08T16:42:00.001-04:00

My coworker asked me to describe how Maven compares and differs to Ant. I realized how hard it was for me to describe what a developer gets for moving their build process from Ant scripts to Maven. I can't make the argument that you can do X with Maven but not with Ant because I don't really have a valid example. However, I am inclined to make the argument that you can do X with both but with Maven it's easier.

With Maven, you collect metadata about your project instead of writing scripts defining the steps your build will take. For example, if you look at an Ant script you'll immediately see that it is organized as targets which many developers use to define and group the steps of their build process. In a Maven POM file, you won't see similar build steps defined. This can be confusing for a new Maven user coming from the world of Ant. I know it was for me. Instead, you'll see lots of metadata about the project such as dependencies, Maven plugins to use, locations for the source control repository and project website, the working directory structure, contributors and more, but nothing that defines what should happen when you want to build the project's jar, or generate a documentation site for the project. These steps are defined by various Maven plugins.

Maven plugins are the most tangible thing you get from using Maven over Ant. What the plugins provide are nothing you can't do with Ant, but they encompass many of the tasks Ant users have written and rewritten time and time again. Goals like compiling your classes, running your unit tests, jaring your classes, generating javadoc, building a JAR of your source files, and on and on. So if you want to build your project with Generics support, you'll define in your Maven POM file that the compiler plugin should target JDK 5.0, but you won't have to define what Maven needs to do to complete the goal.

So when it comes time to build your project, you haven't spent anytime rewriting goals, instead you've only spent time describing how your project differs from other projects. So when you call "mvn jar:jar" instead of "ant jar", you won't have written any of the boilerplate script to tell Ant how to build a jar and which steps it depends on (compile, test), you'll just have reused the jar goal that a Maven developer wrote.

With Maven you get to focus on the high-level task of defining your project instead of the low-level task of defining your build.

Jen on The Weather Channel

2007-02-17T16:37:00.000-05:00

Jen, had a report on The Weather Channel! It's the first time I got to see her on TV. I took the time to grab it off my TiVo, decrypt the video, reencode it, trim out the leader and trailer and upload it to YouTube. Take a look!

Flocking Applet

2007-01-12T13:49:00.000-05:00

So I was brushing up on my CS fundamentals recently and was reminded of the topic of Markov chains. In college, I created an applet that simulated flocking behavior in animals using a Markov chain and it turns out its still up on my college site.

CSS: max-width in IE 6

2006-10-06T11:58:00.000-04:00

I found this great example of how to implement the CSS2 properties of max-height, min height, max-width and min-width in Internet Explorer.