The first one to yell loses

There’s a reason that yelling is often called “losing it”. Yelling shows you’ve lost control, it shows that you’ve lost sight of what it is you’re talking about. Yelling means that you’re trying to beat the other person, not win the argument or express your point.

If you start to feel yourself lose it, just:

1. Smile
2. Breathe
3. Go slowly

There’s something much more compelling about a whisper than a yell. This wallpaper might help you remember.

My first screencast

Okay, it’s not my first screencast, but it’s the first that’s been created simply as a how-to. Most of my screencasts have been in direct response to client questions on “how do I do x in my CMS again?”. Much faster than typing an email, typically.

I use ScreenFlow for capture, and this was all done in one take while waiting for some updates to compile on a Gentoo server.

Get to the point!

I ramble a lot. I need to learn to edit this stuff, but hey, you don’t learn if you don’t post.

Feedback?

Oh, and it looks much better in HD.

Languages, frameworks, design patterns, blogs, Photoshop, Fireworks, off-the-shelf products, roll-your-own solutions, operating systems, servers, desktop PCs, laptops, phones, shoes, socks, paradigms: they’re all just tools.

Craftsmen create. They use their tools and they get the job done.

The end result is the job. The tools are irrelevant to the craftsmen. Don’t debate others’ choices of tools - learn from them. That is the path of the artisan.

The more tools you have, the better. The more you use them all, the better.

Anything else is just spending your apprenticeship looking for the perfect hammer, then spending your career painting it.

My second guess is that if you called me arrogant, your development language of choice for your killer CMS was PHP.

I know. I feel your pain. I’ve been there. There is a better way.

PHP was good enough for me father, so it’s good enough for me

Okay, so now I’m going to hate on PHP for a bit? What next? Sugar in your gas tank?

Let’s get one thing straight. I have no problem with PHP as a language. Well, okay, I have a few problems, but in principle I think it’s a fine platform for developing, well, anything you want. The syntax is a bit flakey, and language consistency is practically non-existent, but it’s really as good as any other language geared to web development.

I also think there are some great apps written in PHP. In my last post, I mentioned three content management systems that I use regularly. They’re all PHP-based, and I recommend them all.

So what gives? PHP is a ghetto? How come?

PHP is too easy

While it will take me a little while to justify it, my conclusion is simple. PHP is far too easy to hack on and get fast results.

You know what else is too easy? HTML. Let’s take a little analogy trip back to the late 90s.

Back in the day, HTML was a horrendous tag soup. “Progress” was measured by browsers adding proprietary tags to a badly-enforced spec that nobody cared about. We all laid out in tables, we all used massive sliced images to get our message across. Our sites were inaccessible and unsearchable.

But who cared? It was easy to build a website. Fire up Front Page, hack about in Photoshop for an afternoon, and boom - one corporate intranet coming right up!

Of course, you could follow the standards if you wanted. You could create great sites without resorting to the nasty tables, but if you did so you were an one of those guys - what was the point?

The point, of course, was maintainability, with an added bonus of learning a whole load about how the web was supposed to work: semantics, presentation separation, scalability, inter-app communcability. These days the cool kids call it “Web 2.0″.

Thankfully, the sort of table-driven web design that we saw back then is dying a slow, lingering, but ultimately satisfying death. Designers who still think in terms of CSS being “that new thing that nobody really uses” aren’t occupying many high level positions any more, and it’s all a bit more disciplined a discipline.

PHP is still amateur night

Let’s haul our analogy back on track. What does this have to do with PHP? The problem is that while the folks running the show at php.net are desperately trying to enforce some standards, and generally discipline up their users, the users are having none of it. PHP4 just won’t die, and there are still a lot of professional PHP developers out there who haven’t moved beyond a few bits of PHP injected into flat files on their server.

PHP is great for quickly adding some functionality to a flat site. That’s how most of us got into building “PHP apps” in the first place. A file here, a login page there, some admin forms and suddenly we’ve got an app! It’s untestable, unmaintainable, but it’s small enough that we don’t care.

Except we then get adventurous and start hacking at some sort of “framework” idea that we have, without realising that we’ve crossed over to the point where we really need some computer science understanding here. Design patterns? What are they?

Suddenly we have hundreds of .php files in our public_html directory, each one slavishly copying the authentication header include to make sure the right users get to administer the site. Adding a new section? Fire up the FTP client, create a new directory, and make four copies of the “page” script.

We might even get creative and use .htaccess to start hacking up our URLs to look nice.

But we’re still just using glorified server-side includes to activate our pages.

Go pro

Most of the good, object oriented, application-thinking PHP developers I know only became that way because of one of two things in their careers:

• They came to PHP from a comp. sci. background
• They left PHP to program in a different language, and came back enlightened

This is one of the main reasons that I try to wean PHP developers off PHP as fast as possible when I meet them. I don’t care if you use Rails, Django, Zope, Erlang or frigging Perl (okay, maybe not Perl) - just get out of the PHP ghetto, and see how the big boys do it.

It’ll seem weird at first (”There’s only one file in the public_html directory!!”) and you won’t like it (”I could do this SO MUCH FASTER if I just wrote some inline PHP to process the form submission”), but gradually you’ll start to see all the wins you’re getting (”You mean I don’t have to instantiate the database connection?”) and the time you end up saving (”Wow, all my HTML is bundled in a neat package, and I can drop in a new template any time I like!”).

You’re then free to come back to PHP and start coding like a real programmer. You’ll have a much greater appreciation for how it all works, and that a little discipline saves a lot of time.

Why the venom?

The last couple of posts have been pretty vitriolic. They say we hate most in others the faults we see in ourselves, and I’ve recently been reminded of my “bad old days” a few years ago where PHP was my only language, and I loved Rails but didn’t want to learn Ruby.

I’ve been working recently on a legacy project from those days, and realise how far I’ve come. I hope this post and the last one might save someone else the pain.

PHP is bad shit, kids - just say no.

Picking the engine itself, though, has taken about 2 years, and was based on one thing: immediacy.

Choices, choices

Choice is good, right? Competition is even better. I’ve recently become jaded with my own erstwhile approach to site management systems. That approach could be boiled down to two simple statements:

• Content Management Systems are simple
• Given enough projects, I can incrementally build my own while getting paid for it

The implication being that “my own” will be inherently better than any of the off-the-shelf systems that are already out there, because I’ll understand it from the ground up.

What really happens can be summed up by these parallel statements:

• CMS is as “simple” as word processing: easy to describe, easy to prototype, hard to perfect
• Clients are not software designers, and I’m a scorched-earth perfectionist

All of which results in me, having idealised the first two statements, but followed the second two, writing this post two years later.

Build it in increments -> (step 2) -> profit!

If you are planning on building an in-house CMS, I’m assuming that you’ve already come up with all the rationalising arguments about why it will be “better” than off-the-shelf. It doesn’t matter whether you’ve convinced yourself that the client’s requirements are beyond the scope of any existing system, or whether you’re just full of “Not Invented Here” Kool-aid, the point is that you and your team are committed to delivering a CMS to fulfil your client’s spec.

Of course, what I’ve just described is a bespoke CMS for a single client. Everyone knows that’s not scalable to larger projects, so how do we get around that?

All together now:

We write the core of the CMS for this client, and build it modularly so that we can just give them what they asked for, but reuse the core on the next project!

Bingo. Sounds great, doesn’t it? You make a bit of a time loss on this project, but it’s an investment, because next time around you’ll be mainly writing modules for the next client’s functionality. If the next client has the same spec as this client, it’s instant profit! What can go wrong?

This is where reality creeps in. You build your CMS nice and modularly for the first two weeks. You then realise that you’re not quite finished the core yet, and you’ve only got two weeks left to implement the various bits of functionality as modules. You panic. You know what modules you need, so you start building the remainder of the core with those modules in mind. You maybe even start building some modules into the core, promising to refactor them out on the next project, because you finally remembered your Pragmatic Programmer, and suddenly it’s all about “just get it working”.

You complete the project on time, and what do you have? A half-bespoke, half-modular monster that is going to require a serious refactoring job to make usable for the next project.

However, you’ve learned a lot. This is a key realisation, and often missed. You’ve learned a hell of a lot, and this is where the rot starts. You’ve learned so much that you convince yourself that, on the next project, it would be insanity to try to reuse any of what you’ve just coded. What you decide to do instead is to reuse the knowledge. Oh yeah, baby - we salvaged something after all!

So you start coding again from scratch. This time it will be better, you say, since you know how it all went wrong last time, and you’ve come up with five great new ways of modularising from the get-go. Then client number 2 rolls along.

One of these kids is kinda different

So with your newfound “I get it now!” knowledge, you start implementing The Plan again. Client 1 was a false start - client 2 is the real deal. Now you know how to make it modular, you’ll do it right this time.

Except the same thing happens. This time, because you’re having so much fun coding, and because you’ve convinced yourself that last time you caved and started coupling the core to the modules too early, you hold out to week three to produce the core CMS. But again, week three rolls around and you’re still not done. With only one week left to implement the functionality that took you two weeks last time, you start panicking and even more coupling happens. Worse, your project runs over time and is late.

So far, with two projects in the bag, you’ve overserviced both, pissed off one of the clients by being late, given each a rushed bespoke system (with all the attendant maintenance overheads), and have precisely zero reusable code. Not exactly the outcome you’d hoped for.

So what went wrong?

Addiction is curable

What went wrong is that you failed to notice that your stated goals were not really your goals. Worse, you actually got what you wanted all along.

Huh?

Your implicit goal in building an in-house CMS was to reduce the amount of work you do. You’ve noticed that servicing your clients is boring, because you’re constantly writing the same stuff. To fix that, you decide to consciously write that stuff in a way that you can reuse next time.

The logical next step of that is to write an entire system that you can reuse next time. This is where the logic breaks down, because there already are systems that you can reuse. Right now. You can download MODx right now and be at precisely the point you say you want to be at in four weeks.

So why didn’t you? You didn’t because your goal isn’t to reduce the amount of work you do, it’s to increase it. What you secretly want to do is build a bespoke CMS for every one of your clients. You hope that maybe, one day, you might hit upon a pattern that you can reuse, but you don’t really care about that. You love coding, you love tweaking and refining and burning it down and starting again. You love it, and you got it.

Step zero is realising you’re addicted

So what are you addicted to? Coding? No - you’re addicted to the knowledge you’re learning, to the experience, to “figuring it out”. That’s why it’s so difficult to locate the source of the problem. When I said “you wanted to increase your workload”, you balked, because to you the work is becoming more and more painful.

The knowledge isn’t, though. You’re putting up with the pain for the rush of having refined your perfect plan for next time.

And the truth is that your CMS prototypes are getting better all the time. After 5 or 6 more clients, you might have a really solid platform.

Too bad for your first half-dozen alpha and beta testing clients, though.

Where’s the exit?

So how do you break this cycle? You decide which of these is more important to you right now. Do you want to:

• Build a CMS; or
• Streamline your client servicing?

You don’t get both, not right now. If you want to build an in-house CMS, you’re going to have to commit 6 to 9 months of development time to it, and the client has to be your company, not some real-world project you’re using to guinea-pig your sketches on. You need to take yourself (and perhaps another developer) off paid work and become software developers, with the attendant investment and payoff cycles that don’t exist in direct-to-client sales.

That’s the price. Once you have your CMS, you can start flogging it to people. At that point you can start to “Streamline your client servicing”.

Of course, by then, I’ll be in your sector, stealin’ your clients, because I chose to “Streamline my client servicing” straight up using WordPress and Expression Engine, with MODx in there for the really tricky stuff, and have been having a whale of a time developing modules, plugins and extras for those mature platforms you’re only starting to catch up with.

Your CMS had better do something unique.

Certainly not me, and certainly not my visitors.

Offering sensible choices

Let’s look at all the places I live:

• here
• flickr
• ma.gnolia
• twitter
• last.fm

Each of these services provides a simple feed of content (though for some reason, last.fm has stopped scrobbling for about a week now. Grumble. Any ideas? Solved: it seems that the Mac OS X last.fm client doesn’t report password problems. It just fails silently and looks like you’ve logged in. Nasty bug.). If a visitor just wants to follow me on flickr, they can click on the flickr link on relativesanity.com and then add that feed to whatever reader they use. Job done, right?

Well, not really. I’m aggregating content on relativesanity.com, so wouldn’t it make sense to aggregate feeds? By default, relativesanity.com’s feed link tag used to be a direct pull of my twitter feed, so someone subscribing to the autodetected feed on relativesanity.com would get my twitter updates.

That was fine, but I got more and more requests to provide direct links to the rss feeds from my other accounts. Family, for example, had no idea how to subscribe to flickr (or even that it was an option.

So what to do?

Relationships for relatives

Let’s do some anatomy. link is made up of a few elements:

rel

this specifies the relationship of the link to the page, e.g. “stylesheet” or “alternate”

type

this is the MIME type of the file. For feeds, it’s often “application/rss+xml” or “application/atom+xml”

title

this is the optional title of the link. For multiple feeds, these titles may be displayed by your browser when you try to subscribe to the feed.

href

this is the actual address of the feed itself.

I mentioned “multiple feeds” there. You can, it may surprise you to hear, have multiple feeds for any given page. Most people seem to see this as a means to add different formats of feed to their page. Flickr is a great example:

I say “great” here, but what I mean is “terrible”. What are my options here? “Atom” or “RSS”? I know they’re different formats, but why do I care which one to pick? I don’t know of any feed readers these days that only handle one or the other, so as a visitor, what differentiator is there for me to choose between these? Do they provide the same content? If so, is one more “efficient” at delivering that content than the other? Will one become obsolete soon? Do I lock myself into something by choosing one over the other?

It’s a minefield, and that’s all from the point of view of someone who understands what Atom and RSS are. How’s my Mum going to feel about this, other than stupid?

Let’s look at some code:

  <link rel="alternate" type="application/atom+xml"
    title="Flickr: Your Photostream Atom feed" href="...">
  <link rel="alternate" type="application/rss+xml"
    title="Flickr: Your Photostream RSS feed" href="...">

This is how flickr codes up its photostream feeds. To my mind, it’s completely pointless. If the feeds offer different content, telling me the format doesn’t tell me what the difference is. If the feeds don’t offer different content, why offer two?

Here’s how relativesanity.com’s feeds now look:

Isn’t that neat? Some code:

  <link rel="alternate" type="application/rss+xml"
    title="articles" href="http://journal.relativesanity.com/feed/" />
  <link rel="alternate" type="application/rss+xml"
    title="thoughts" href="http://twitter.com/statuses/user_timeline/6040962.rss" />
  <link rel="alternate" type="application/rss+xml"
    title="photos" href="http://api.flickr.com/services/feeds/..." />
  <link rel="alternate" type="application/rss+xml"
    title="links" href="http://ma.gnolia.com/rss/full/people/relativesanity" />
  <link rel="alternate" type="application/rss+xml"
    title="music" href="http://ws.audioscrobbler.com/1.0/user/relativesanity/..." />
  <link rel="alternate" type="application/atom+xml"
    title="everything" href="http://friendfeed.com/relativesanity?format=atom" />

Now, when visitors arrive, they can choose what to subscribe to, and the differences between each choice are clearly marked.

One feed to rule them all (sorry)

Even better, there’s an “everything” feed that aggregates all the others. This is built using friendfeed.com, which has been on my radar for ages, but for all the wrong reasons. My impression was that friendfeed primarily aggregated feeds of your friends. What I wanted was to aggregate all my feeds (and here’s the key bit) into a single RSS feed. Friendfeed not only lets you do this, it actually provides it as a first-class feature (they should have called it “myfeed”, but myfeed.com is gone).

So there we have it. Nice semantic feeds of content, all in one place, and the “everything” feed can be updated whenever I sign up to some other life-sucking service that lets me tell the world what I’m doing.

No, I’m not on facebook.

To do this, I use OmniPlan from The Omni Group. I love Omni’s software (OmniFocus runs my life, and OmniOutliner stores most of my most important notes), and OmniPlan is built with the same attention to detail as the rest of their apps.

Outlining the effort

The key thing for me is the fact that it uses an outline view for building up your project. Just like OmniOutliner, you can happily drill down deeper and deeper into each “node” in the outline. Each node has a number of attributes, the key one being “Effort”. The amount of effort assigned to each node then translates into a bar on a Gantt chart, and also an addition to the project cost, once you’ve told OmniPlan who’s working on that node, and how much they cost per hour.

Over the couple of years I’ve been doing this, I’ve developed a standard “web development” template that stubs out most of the tasks I carry out during a project. Just like any good outliner, the nesting facility means that I can group the project into discrete blocks, which is great for being able to produce a “top line” summary of what each area of the project will cost.

Where do all these top line numbers come from? Let’s expand out all the items in Core Development outline:

As you can see, the project timeline can now start to build itself from the bottom up. Every task starts out with one hour assigned to it. My first task at the start of a costing is to go through the default outline and prune what isn’t needed for this project, before adding in what *is* needed. I then go through and start to apply time estimates to each task based on previous experience and “best guess” work.

Bubbling up the costs

The great thing about OmniPlan is that it tends to remove the best guess factor from most tasks. Let’s say, for example, that in my first pass I reckon that I’ll need 4 wireframes:

I reckon each layout is going to take me half an hour to do, so in they go, and suddenly my Site Wireframes budget sits at 2 hours. If I realise I need more wireframes as I plan the project, in they go and bump up the cost in a fully justifiable way.

Let’s say, though, that I reckon one of those wireframes is particularly complex because there are lots of variations. I might not have a good handle on how long, say, a progressively enhanced “personal content aggregator” might take to wireframe. No worries - I can nest the various sub-tasks to come up with a proposal:

I now have a better feel for how easy or hard that’s going to be. This is all the kind of stuff you do when planning a project anyway, but the fact that you can make simple changes to subdivisions of tasks and have them percolate up your estimates automatically is a real boon for tracking complex projects.

Show me the money

Once I’ve gone through and assigned time estimates to each part of the project (practically writing the spec as I go, incidentally), I can then use OmniPlan to build timescale estimates and cost them up.

First I switch to the Resource view:

By selecting me (or creating a new resource) and viewing the inspector panel:

I can set hourly rates, and how “available” I am for the project. No, I don’t charge £100 per hour for development. I wish I did.

Back to the Task view, and I expand all tasks, select all, and choose Structure -> Assign Resource -> Jonathan. This hammers through all the hourly estimates and gives me a cost based on my hourly rate, as well as displaying a project total in the status bar at the bottom of the window. By selecting View -> View Options then enabling “Total Cost” in the Outline Columns pane, I can now see how my top line works out:

Awesomeness!

Time keeps on slippin’

Next up is to get a timescale plan. To the right of the outline we get a Gantt chart. The Gantt chart is smart enough to build up a timing plan automatically based on how many people you have working on the project, how available they are, and what dependencies you build into the project.

Since we’ve only got one person working on this full time, we can’t have any concurrency, so it’s all about the dependencies. As you can see, there are some black arrowed lines running from the end of each task group to the beginning of another:

Each one of these arrows represents a dependency, and we set those up quite simply by selecting the task which depends on another one, opening the dependency inspector, and dragging in that task’s pre-requisite and dependent tasks:

This ensures that our Gantt chart never schedules anything before its pre-requisites are completed.

Once all THAT’s done, we select all the tasks and choose Project -> Level Resources. This rebuilds the Gantt based on what we’ve said about the project, and gives us a good estimate of how long the project will take to complete.

All you do is write code, right?

The final super-smart thing about OmniPlan is the ability to restrict a Resource’s availability. Go into Calendar view, and select your resource. By default, that resource is made available to the project full time. Let’s say we only want to spend half of each day on this project, the rest of the time left for maintenance, admin, phone calls, email, and other freelance stuff. Simple - just make sure that the “Normal Work Week” for your resource is set from 8am to 12noon (just drag on the calendar like in iCal).

Now if we go back to the Task view and run Project -> Level Resources again, we’ll see that we now have a much longer looking Gantt chart:

Even better, we can see that the Duration of the project (see the status bar, and click on that to cycle through the various bits of info this can display) is over 2 weeks, but the actual effort involved only spans 4 days.

Now all we need to do is write up our proposal document based on these timings and specs, send to the client, and wait for approval so we can bill them their deposit.

You do charge your clients deposits, don’t you?

Downloads

• Web Development OmniPlan Template (zip)