About Nothing

HTML Parsing With Groovy and TagSoup

2009-12-29T08:43:00.003-06:00

I'm working on an app where I need to parse some HTML. This is the first time I've had to do screen-scraping with Groovy. After a bit of trial and error I think I'm getting the hang of it. The HTML I'm working with isn't well-formed, so the default Groovy XmlSlurper and XmlParser puke. After some digging I found TagSoup. It "parses HTML as it is found in the wild: poor, nasty and brutish, though quite often far from short".

It made my parsing much easier. Thanks John Cowan!

Groovy XmlSlurper and HTTP 503 Response Code

2009-12-29T05:38:00.008-06:00

I struggled a bit when trying to parse some XHTML with Groovy's XmlSlurper (and XmlParser). I was receiving the following:

Caught: java.io.IOException: Server returned HTTP response code: 503 for URL: http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd

It turns out that the guys from W3C got sick of dealing with the excessive traffic for their DTDs. So now they return a Service Unavailable (HTTP 503) if they detect parser requests.

To solve the problem I had to set the loading of external DTDs to false. Here's the code.

def slurper = new XmlSlurper()
slurper.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false)
def results = slurper.parseText(htmlResponse)

Googling for the answer wasn't extremely helpful. This blog post helped (I think it's in Japanese). This post also helped. Thanks guys!

I decided to re-post the solution since it took me awhile googling for the answer.

The Science of Avatar

2009-12-27T06:30:00.004-06:00

An interesting read on the science of Avatar. I still haven't seen the movie; just too much going on with the holidays.

(Near) Real-Time Analytics

2009-12-08T06:32:00.004-06:00

At my new gig, I've been asking whether the team has considered the possibility of using map/reduce or a similar grid-based solution to conduct our analytics in (near) real-time. Interestingly enough, I ran across Nati Shalom's post on real-time analytics yesterday. This should help give me some ammunition to convince everyone that we need to move in this direction for the solution we're building. Thanks Nati!

A Feast for Crows

2009-12-08T06:16:00.003-06:00

I just finished re-reading George R. R. Martin's A Feast for Crows. I enjoyed it more than the first time I read it. My favorite still continues to be A Storm of Swords. Now if he'd just publish A Dance with Dragons!

The Definitive Guide to Grails

2009-05-13T06:45:00.003-05:00

I finished up The Definitive Guide to Grails close to a month ago, but I forgot to blog about it (I'm using my blog to help keep track of which books I've read). It was an excellent read. I'm sold on Groovy and Grails, particularly for Java shops. Given the recent SpringSource purchase of G2One, I expect Groovy and Grails to gain much wider adoption in the enterprise.

The Productive Programmer

2009-03-22T05:58:00.003-05:00

I just finished up The Productive Programmer by Neal Ford. It was so good I decided to buy my own copy. It definitely made me realize how much more efficient I can make myself. There are a ton of tips for both Mac and Windows. One of the major themes was automate everything you can. Thanks Neal!

No Fluff Just Stuff

2009-03-13T15:15:00.007-05:00

Today is the first day of the Twin Cities Software Symposium. The first talk I attended was REST: Information Driven Architectures for the 21st Century by Brian Sletten. Very informative. Definitely not an introductory REST talk. I'm curious to hear Brian's Semantic SOA talk later today. A couple things that stuck included Jon Postel's "Be liberal in what you accept, and conservative in what you send."

A couple of things Brian talked about that sound like they're worth investigating include Sinatra, a DSL for building web applications in Ruby and retrievr, which lets you find Flickr photos by creating a sketch of what you're after.

Brian's new gig sounds pretty cool: League of Legends.

Einstein

2009-03-01T06:05:00.002-06:00

I recently finished reading Einstein: His Life and Universe, a biography by Walter Isaacson. I enjoyed it very much. It gave excellent insight into the man. The most interesting thing to me was that even though he was brilliant, he struggled with things in his everyday life just like everyone else. His family life wasn't perfect and neither were many other aspects of his life. I love the fact he was a non-conformist not only in science, but in political affairs as well.

Crystal Clear

2009-02-14T17:04:00.008-06:00

I just finished up Crystal Clear by Alistair Cockburn. A very good book, but it was a stretch to get it to 300 pages. The first chapter threw me for a loop the way it was structured and the last chapter, a case study, was a dud. While the team size for Crystal should be 8 or less, a case study with 1.5 developers doesn't sound like a very good case study. However, the chapters in between were excellent. Cockburn admittedly structured each chapter differently attempting to cater to different readers. It gave me some insights into how a successful team should interact and was very complementary to the other Agile documentation that I've seen. I definitely liked his guidance on Walking Skeleton and Incremental Re-architecture. It helped me reinforce the concept of an Architectural Slice that I've been conveying to the folks on the large Java project that I'm currently working on.

Kudos to Andy Miller for recommending this book!

Shipped It!

2009-02-14T09:13:00.003-06:00

Jared Richardson of Ship It! fame spoke at the TCJUG on Monday. His topic was your career. He started out a bit slow, but the pace picked up as his presentation progressed. I think most folks got quite a bit out of it, but it reminded me of several presentations I had seen before. Particularly there was some overlap with a presentation I went to several years ago by Dave Thomas at NFJS Denver. Dave's theme was about investing in your career. That was the first time I heard Dave's infamous "Herding Racehorses and Racing Sheep".

I did learn some new stuff at Jared's presentation. In particular I learned about qik, which looks pretty cool. It allows you to share a live video feed from your phone; Jared had someone in the audience do the live feed to qik using Jared's iPhone. I liked Jared's acronym for public speaking (L)ock eyes, (I)ntonation, (P)ause as well. This will definitely come in handy for me in the future.

The part that surprised me the most was how few people in the audience new about Blogs and Feed Readers. Another shocker was how few people had heard of The Pragmatic Programmer. Maybe people were just too lazy to raise their hands. If not, c'mon TCJUG attendees!

Thanks for flying all the way to Minnesota to enlighten us Jared!

Groovy Encapsulation - Say What?

2009-01-31T06:19:00.007-06:00

I'm reading Groovy Recipes by Scott Davis and find this troubling:

class Book3{
private String title
private String getTitle(){}
private void setTitle(title){}
}
def b3 = new Book3()
b3.@title = "Groovy Recipes"
println b3.@title
===> Groovy Recipes

In Groovy, private attributes can be modified, even if you use private setters. That's not cool. I'm hoping there's some way to enforce encapsulation, but it's not looking good right now.

Groovy!

2009-01-25T06:52:00.001-06:00

I just finished up Getting Started with Grails the free book from InfoQ by Jason Rudolph. An excellent book. Thanks Jason! I thought it was so good I decided to pay for it (even though it's free) to help support the author and InfoQ. There's a few discrepancies because the book is almost two years old and quite a bit has changed in Grails since then, but I had very few problems working through the examples with the latest version of Grails.

I'm definitely on the Groovy and Grails learning train. Both still seem very promising to me (being a Java guy) and I'm going to continue investigating both. While I set out to learn more about Ruby and Rails this year, the winds have shifted and I'm now focused on Groovy and Grails. I purchased the PDF version of Scott Davis' book Groovy Recipes and The Definitive Guide to Grails.

Hockey Day Minnesota

2009-01-17T06:38:00.001-06:00

Today is Hockey Day Minnesota. I'm going to try to honor that by getting my daughters out on the neighbor's pond followed up by taking them to the University of Minnesota Women's game against Bemidji State. It'll then be on to termite practice where the kids will get to play pond hockey instead of practicing. We'll wind it down with watching some of the Gopher and Wild games. It should be a ton of fun!

Code Freeze

2009-01-17T06:14:00.001-06:00

I attended the Code Freeze conference at the University of Minnesota on Thursday. It was an excellent local event, especially considering it was an all-day event for only $90. This was the first year I've attended Code Freeze; this year's theme was Maximizing Developer Value. Neal Ford kicked things off and as usual he knocked it out of the park. His topic was On the Lamb from the Furniture Police. It covered the fact that as programmers we're hired to concentrate for long periods of time, yet corporate environments provide the exact opposite affect.

Other speakers included Luke Francl, Nate Schutta, Susan Standiford, Andy Miller and Tomo Lennox. I was very impressed with Nate's presentation, it seemed to directly pick-up where Neil left off. I was particularly interested in Nate's comments about the working of the human brain, as it is an area of interest for me.

I was also intrigued by Andy's presentation entitled "Why I don’t estimate with "points" (and how you too can be delivered from the tedium of repetitive estimation)". Andy and I are currently at the same client working together on a large re-engineering project. I haven't work with Andy for long, but I was very impressed with his presentation and was impressed with his pragmatic approach to estimating. It definitely opened my eyes to new ideas.

Hackers and Painters

2009-01-10T07:38:00.001-06:00

One of my 2009 resolutions is to read more. I just finished up Hackers and Painters. I had made it half-way through a couple of years ago and decided to start over and I made it all the way through. A quick and insightful read. Paul Graham is fairly opinionated, which makes for a good read. I've never seen Lisp, but I am very curious if it as good as he claims. Based on my experiences, I definitely agree with his thoughts on development productivity. It makes me that much more interested in learning Ruby and Rails. The last time I worked seriously with a dynamic language was with Perl in college when we were building the Heil X6 SMT's kernel, simulator and assembler. The Heil X6 was the computer we (a team of 6 graduate and undergraduates) built in our ECE 554 Digital Engineering Lab using FPGAs that we had to hand-wire. What a trip!

Groovy and Grails

2009-01-10T07:13:00.002-06:00

At the DevJam Jam Session on Wednesday, Mike Hugo gave a presentation on Groovy and Grails. I had heard Scott Davis speak about Groovy and Grails over a year ago and thought it was pretty cool, but back then it didn't feel real. Since then Grails has produced a 1.0 release; put simply I was extremely impressed. Mike is a very good presenter, his slides had very few words and some great pictures. He kept it lively with some humorous slides plus some succinct demos. I've been looking at Ruby on Rails off and on over the last couple of years. One of my New Year's resolutions for 2009 was to really dig into Rails and write a decent sized application. However, after Mike's presentation I began to think that a better investment of my time may be in Grails given that it runs on the JVM and plays very nicely with the Java language. I downloaded Grails yesterday and played around with it a bit. Very nice. Just like with Rails, it's amazingly easy to get a site up and running quickly. Grails is still a 1.0 release, so I have some concerns about it's maturity. However, it runs on the JVM and uses Spring, Hibernate and SiteMesh under the covers. Obviously all of them are very mature.

After the presentation by Mike, we entered the Fish Bowl round table discussion, which I believe is the best part of the Jam Sessions. I'm amazed by the brain power in the room at these sessions and I definitely like the Fight Club mentality. The conversation touched on a number of topics including the advantages / disadvantages of statically vs dynamically typed languages. The consensus seemed to be we're definitely seeing a movement toward dynamically typed languages. Someone even dropped the Lisp quote along the lines of Hackers and Painters ("...we could do that in Lisp back in 1968..."), which was interesting given that I just finished the book. One person, who seemed to know a ton about language theory (he even dropped F#), mentioned that he thinks that the gulf between the two is getting bigger and bigger: static languages are becoming more strongly typed and dynamic languages are/will become more dynamic. It makes sense to me given where I see the Java language headed and makes my desire to learn Ruby and Groovy that much stronger.

Building Scalable Web Sites

2008-01-05T09:31:00.000-06:00

I just finished reading the book Building Scalable Web Sites by Cal Henderson of Flickr fame. It was a very interesting read and covered quite a few things I wasn't expecting. Most of the code was PHP, which I've never worked with, so that in itself was cool to see for the first time. He covered a ton of topics, including walking through how they scaled Flickr. I got the most out of chapter 9: Scaling Web Applications. He also gives a decent overview of Web Services in the last chapter, which I wasn't expecting. The book was easy to read and some of the difficult topics he explained clearly and concisely.

Overall it was an excellent book and I would recommend it to anyone building web sites regardless of whether or not they're using PHP.

Spring 2.5 Upgrade

2007-12-21T08:44:00.000-06:00

I upgraded out platform infrastructure to Spring 2.5 this morning. It was pretty straight-forward, although I had one painful issue. For any of our tests that use Hibernate, I was getting the following error:

Caused by: org.hibernate.HibernateException: saveOrUpdate is not valid without active transaction
at org.hibernate.context.ThreadLocalSessionContext$TransactionProtectionWrapper.invoke(ThreadLocalSessionContext.java:297)
at $Proxy39.saveOrUpdate(Unknown Source)

After a bit of digging, I realized we had the following Hibernate property set:

<prop key="hibernate.current_session_context_class">org.hibernate.context.ThreadLocalSessionContext</prop>

Basically, this was preventing Spring from managing the transaction context correctly. Once I removed it, all of our tests passed. I wonder if this is the reason why we've had some problems with NOT_SUPPORTED and SUPPORTS previously. This will require some additional testing...

Spore

2007-12-20T10:57:00.000-06:00

I stumbled across a new PC game today entitled Spore. It's not GA yet, but it looks extremely interesting. It looks like Civilization meets The Sims. Spore was created by Will Wright, the creator of The Sims. Here's the video I watched. You start as a single-celled organism and you evolve your species all the way up to an advanced spaced-based civilization.

Desktop Matters (TSE Day 4)

2007-12-15T10:08:00.000-06:00

In the second session of the morning, I decided to attend the talk entitled Desktop Matters by Jim Moore. I didn't learn much in this presentation. He walked through the history of computers from a UI perpsective: mainframe to client server to web. He then went into UI technologies that are available today to build rich applications: Flex, Apollo, Silverlight, GWT, WebStart, etc. He had a couple of bullets points for each, but he didn't get into much detail in terms of what each does. I would've liked to have seen more in the way of the positives vs. negatives of each and some recommendations.

Architecture Enforcement with AspectJ (TSE Day 4)

2007-12-15T09:43:00.000-06:00

I just attended an extremely interesting session entitled Architecture Enforcement with AspectJ and Other Tools given by Alef Arendsen and Ramnivas Laddad. They first showed Structure101, which gives you a visual representation of your architecture, including displaying package structure, cyclic dependencies, etc. I hadn't heard of Structure101 previously. It looks like there are several versions including both free and for-fee.

They next showed several examples of using AspectJ to enforce architecture rules both at compile-time and runtime. This included rules like:

- don't use straight JDBC, instead use Spring JDBC
- don't return null Collections
- ensure that non-thread safe objects are never accessed by multiple threads simultaneously

It was all AspectJ plus Java. Very cool! The compile-time errors show up as standard compile errors in Eclipse and the runtime errors can be exceptions or some type of log message using Log4j or similar. This presents some very interesting options if you're working on a medium to large-sized team and want to enforce a clean architecture. I can already think of a number of places that this would help us immensely. It sounds like it's well worth the investment. Obviously you'll still want to have a Software Architecture document, but this gives you a way to enforce what's in the document.

I am a little concerned about the runtime overhead of AspectJ for the enforcement of the runtime rules, but it's worth experimenting with to see how much overhead is actually involved. I asked the speakers about this and I'm not sure they understood the question, but they mentioned the overhead was minimal. I'm worried about the case where you wind up with hundreds (or even thousands) of runtime rules and the overhead that is introduced.

GigaSpaces - The Spring Experience Day 3 Afternoon

2007-12-14T13:30:00.000-06:00

I just finished attending the session presented by Nati Shalom from GigaSpaces. It reinforced a lot of the ideas I've been reading about in his various white papers and blog entries. I'm extremely impressed with the GigaSpaces technology. If you're not familiar with GigaSpaces, I'd highly recommend taking a look at it if you are writing a mid-size or large-scale application. Basically, it's a grid platform for messaging, processing and data. One interesting thing I learned is that they're holding an OpenSpaces Developer Challenge that officially started on December 10th. Final submissions are required by April 2nd, 2008. First prize is a $10,000 gift card from Amazon, Best Buy, Circuit City and Apple. That's a lot of toys for any geek to have in their hand!

Practical Enterprise Concurrency

2007-12-14T09:49:00.000-06:00

In the second session of the morning, I attended a talk by Rob Harrop on Practical Enterprise Concurrency. It's a highly technical topic and Rob did a great job of presenting it. It's obvious he knows what he's talking about. He covered a variety of topics including:

- The Java Memory Model
- Common Bug Patterns
- JDK Concurrency Utilities
- Testing Concurrent Applications
- Concurrency in Java 7

The bug patterns were pretty common ones that I had seen previously. He touched on the Condition interface as a replacement for wait/notify, which I wasn't previously aware of. He also touched on CountDownLatch as being useful for scheduling parallel tasks. I really need to read the Java Concurrency in Practice book I purchased a couple of months ago.

Overall it was an excellent presentation. In the future, I'll ensure I attend any of Rob's presentations, regardless of the topic. Tonight he's hosting a BOF entitled Extreme Scalability, which should be very interesting given the scalability research I've been doing as of late.

The Spring Experience - A Bad Start to Day 3

2007-12-14T07:41:00.000-06:00

I'm off to a horrible start this morning. I couldn't get coffee (or orange juice) from the servers at breakfast. I'm a coffee junkie, so needless to say it wasn't a good start.

The talk I'm sitting in is entitled Batch Processing Performance and is being done by a couple of consultants from Accenture. The first speaker is having all kinds of difficulties: microphones, projector and believe it or not, even PowerPoint. I'll try to take the optimistic approach and know that the day can only get better.

After struggling to get going for about 15 minutes, the speaker started off by reciting word-for-word a 1,000 word article published in Computer Weekly. What a complete waste. I should have walked out 5 minutes ago when he mentioned his title, which included the word "manager". After listening for another 10 minutes without any improvement, I'm walking out now...