Coding matters

MPS is simple

noreply@blogger.com (Dmitry Kandalov) — Mon, 10 May 2010 03:08:01 PDT

If you have ever seen or tried MPS and it didn’t make a lot of sense, do not believe your first impression, because meta-programming in MPS is simple.

There are only two things you have to know to create something useful in MPS:

editors and
generators.

Editors

Editors is the way you interact with your language in MPS. The thing is that MPS doesn’t edit text files. It works with abstract syntax tree. When you edit a program in MPS, you edit AST. Here is AST for hello world program written in Java:

To define a language you describe types of nodes that can be in your language AST. In MPS this is done with concepts (they are green).

In a way concepts are classes for AST nodes and AST nodes are instances of concepts.

For each concept you define an editor.

Editors allow you to edit and view AST nodes in a text-like form. In a way editors that you define for concepts are “editor classes” and the area in which you actually edit programs consists of “editor instances”.

If you like MVC pattern and its flavours, you can think about AST nodes as a model and editors as view and controller. You can read more about concepts here and here and about editors here.

Generators

While having AST and being able to edit it is great, usually you would like to produce something useful like java code or text files. This is what generators are for. They take AST and transform it into real files on your hard drive.

Generators can transform AST directly into real files:

…or they can transform AST into another AST. This is useful when you want to build your language on top of another language. For example, in MPS there a bundled language called base language which is basically Java extended with features like closures. You can create generator to translate your language into base language and generator from base language will automatically transform it into java files:

This way you have a chain of AST transformations and don’t have to deal with file generation.

You can read more about generators here. There are also quite good generator demos.

What’s next?

There are a lot of other interesting things in MPS like type system, IDE support and much more, but to do something useful you don’t have to know anything except editors and generators. In my view, the best way to get started is to go through MPS tutorial and generator demos. Do not just read them and look at screenshots, install MPS and try to repeat all the steps. Be aware, that tutorials are not too easy and might cover too many topics to understand at once. Don’t get frustrated if something is not clear. After tutorials you might read parts of user guide or try doing something simple yourself. It doesn’t take a lot of time, but it’s rewarding and it’s the best way to understand what MPS is good for.

Pomodoro in practice

noreply@blogger.com (Dmitry Kandalov) — Thu, 15 Apr 2010 10:51:21 PDT

Do you ever feel at work like you’re doing thousand things at the same time but never finish anything? You start writing some code, then a bug report comes in and you have to respond because it stops QA from testing. You go back to code, but then colleague comes with a question. Then yo go to meeting. You go back to code again, but it doesn’t go very well so you start reading blogs. Then you receive an email you have respond to. And before you know the day is over and you don’t feel like you have really accomplished anything.

If you experience anything like this, stop doing what you’re doing and try pomodoro.

What is pomodoro?

It’s a focus (and time) management technique or put more simply “a way to have less distractions”. In its basic form pomodoro is like this:

you do a task for 25 minutes avoiding any interruptions;
after 25 minutes you take 5 minute break;
if despite all your efforts, you’re distracted during 25 minutes, you reset timer and this pomodoro doesn’t count.

(Notice, that “pomodoro” term is overloaded, because “pomodoro” is not only the name of the technique, it also means 25 minute interval.)

The main reason to do it is that any non-trivial task requires a lot of mental effort to get into the state of flow (a state of mind when you’re really productive). Getting into the flow is quite expensive so you should avoid all the internal (checking email, reading RSS) and external (a colleague taking to you, instant messenger pop-up) interruptions.

On the other hand, working for 10 hours in a row is also not productive. It’s also important to have small breaks. I remember quite a few examples when after hours of hectic coding I realized that I had created a mess, while there is a much simpler solution to my problem, or even worse that the problem was wrong.

There is more to it, but in a nutshell pomodoro is like a vise:

forces you to concentrate on the task by having less interruptions;
forces you to have breaks.

(Another metaphor I like “pomodoro is your personal Scrum sprint”.)

Pomodoro in practice

I’ve been using pomodoro for about 4 months now. In three words “it is great”. In ten words “It’s not a silver bullet, but totally worth doing”.

What works

I get more satisfaction. Having small tasks and accomplishing them is somehow more satisfying than doing the same amount of work without tasks (I think, it’s also one of the ideas in Getting things done). It works especially well on the most insane days because doing small focused tasks makes them less insane and at the very least at the end of the day you can look at the paper where you track pomodoros and see that you have actually accomplished something.
makes the cost of context switching explicit. It’s explicit for me because I know that I’m doing a certain task and if I’m distracted I’ll have to reset timer. It’s explicit for others because they know I’m doing pomodoro and they can see my timer. (I use Focus Booster on Windows and Pomodoro on Mac OS.)
less procrastination. Before each pomodoro I choose a task (or several tasks if they’re small) that I’m going to accomplish. If something has to be done to accomplish the task, I do it immediately. With 25 minute “sprints” there no time to procrastinate. It works especially well with tasks which are not exciting but just have to be done.
time tracking. I used to track how much time I spend on tasks before using pomodoro. With pomodoro time tracking became easier and better. Easier because I do it anyway whenever I use pomodoro. Better because my time is tracked with 100% focus factor.
focus factor. “Focus factor” a term from Agile (see XP and Scrum from trenches). What I mean here is how much time a day you can spend without being interrupted. With pomodoro you get your personal focus factor for free. On average I can do 8 pomodoros or 4 hours of uninterrupted work. This is 50% focus factor. (It may seem low, but it’s better to know the truth than to avoid it.)
having breaks is relaxing. It helps a lot to keep focus while doing boring tasks because I know I’ll have a break after all :)
it’s fun. It’s like a game, but really useful one.

What doesn’t work

often 25 minutes is not enough. I haven’t figured out the reason yet. Probably, it depends on the time of the day and kind of task. When I do something really exciting, I tend to skip every second break.
obeying the timer. When I started using pomodoro, I would actually start or stop doing task when timer rang. I don’t know whether I adopted pomodoro to my needs or I’m just lazy, but now I don’t care about timer that much. I can stop few minutes later or have a longer break if I feel like doing it.
exploratory tasks. Pomodoro is great for goal oriented tasks but being goal-oriented is not always a good thing (see awesome talk by Daniel Pink). Sometimes you may want to enjoy yourself without thinking of time or how much you can accomplish. I don’t watch movies or read books using pomodoro.
breaks can be distracting if you involve in a discussion.
some people don’t care about pomodoro. Even though they know what it is, even though they see timer, they keep distracting me anyway. It’s not a big deal, just the way it is.
since pomodoro is like a game, you may start cheating yourself to get better results and more satisfaction. For example, I noticed that I reset pomodoro less often than I used to three months ago.

Consequences

I feel happier. Even if have to do something really boring.
I have less distractions:
- I turned off email and instant messenger’s notifications;
- I check instant messenger only when decide to do it, not when someone sends me a message;
- I broke email addiction. I don’t check email or read RSS feeds when I feel like I’m bored.
I became more realistic in task estimates (at least I want to think so).
I became more productive.

Try pomodoro now!

I started using pomodoro by simply typing “pomodoro timer” in google and choosing a timer. I installed Focus Booster and started my first pomodoro without knowing too much about it. You’re not going to lose anything, but there is a chance to get rid of distractions and become happier :)

Meta Programming System: generator for Miss Grant's controller

noreply@blogger.com (Dmitry Kandalov) — Sun, 04 Apr 2010 01:32:11 PDT

This is the second part of Martin Fowler’s state machine example in MPS. The first one was about creating editor for Miss Grant’s Controller and at the end of it there was a nice editor like this:

While this is cool, this editor doesn’t help to create anything useful like Java code which you could actually execute. So this part will be about code generation for Miss Grant’s Controller.

What it’s going to be like in the end?

There will be java code based on existing set of java classes. Something like this:

Importing java classes

In introductory example Martin Fowler shows some code and a diagram to explain what the state machine is like. To reproduce this state machine framework I copied all the code from introductory example into IDE. (There are some problems with code, so if you try doing it yourself, don’t expect it to compile out-of-the-box.) I also didn’t feel ok copying diagram or linking to it (after all it’s a part of the book), so here is a diagram I got in IntelliJ after copy-pasting code and making some minor changes:

In a way this diagram is very similar to AST from the previous part. StateMachine receives Events and changes its State. States can have several rules called Transitions describing which State StateMachine will be in after receiving some Event. When state machine transitions to a new state, it executes Commands associated with new State. There is also StateMachineFactory which creates StateMachines.

First off, let’s create generator and empty template.

In order to use existing java classes in template, we need to add these java classes to MPS project:

After adding classes to language properties we have to import them into template. There are two ways to do it. The simple one is to press Ctrl+M and start typing package name:

The more complicated way is to add package in generator properties:

After importing the package we’ll be able to use all the state machine classes in template code:

Now that we have simple template let’s map StateMachine concept to this template…

… and see if it’s working

Adding initialState

Before going further it worth noticing that there is probably a missing part in Martin Fowler’s example. There is a notion of resetEvents which return state machine to its initial state and there is even code which does it…

… but state machine DSL doesn’t have any constructs to specify initial state. Let’s fix it by adding reference to StateMachine concept:

Let’s also configure initial state to idle, as it is in the introductory example.

Generating Events, Commands and States

First off, let’s make generated class name more descriptive by prefixing it with state machine name. In editor state machine name is the topmost line:

We add it as property macro and replace spaces in state machine name, otherwise it will be invalid java class name.

Let’s add generating code to create Event, Command and State objects. It will cover the following areas of the state machine DSL (note that State objects won’t be fully configured after this step):

Let’s start writing template by adding some code as if it was written in plain java. Code for creating Event object will be like this:

Now we can add macros to loop over all the events:

Inside the $LOOP$ macro we add property macro to change variable name for each loop cycle:

Similarly we wrap with property macros event “name” and “code” parameters. After writing similar macros for Command and State, template will look like this:

Let’s see what this template generates…

Generating initial state

So far we’ve been generating code which instantiates StateMachine with null initial state. Let’s add macro to initialize it with real state. We replace null with reference to state variable and wrap this variable with reference macro:

Reference macro can return node or string so we just copy initial state name (which is string) from StateMachine node:

After generation StateMachine will be constructed with idle initial state.

Adding transitions and actions

Event and Command objects are now fully constructed but States has yet to be configured with transitions and actions. Let’s start by writing template code for adding transition as if it was plain java:

What we want now is to copy that line for each transition in each state. In pseudo code it will look like this:

At the end equivalent template code will be:

We start by creating outer loop…

… and then inner loop

Parameterizing event and target state is easy. We can simply take names of triggerEvent and targetState references:

Getting name of the current state is more complicated. The problem is that inner $LOOP$ macro doesn’t know anything about outer $LOOP$ macro. There is a way to do it using mapping label (see this thread) but in this case we can do it by taking Transition node’s parent which is a State node:

When generated, the code will look like this:

We do similar thing to generate addAction() statements.

Reset events

The only missing part now is reset events. This is pretty straightforward. We loop over resetEvents and use reference macro to generate parameter names forv addResetEvents() method.

Setting up output path

It’s great to have generated code, but we need to be able to actually use it. Let’s save generated files in the folder with source code from introductory example. It can be done in Solution properties by specifying generator output path:

Now java project for introductory example can be configured to use src_gen folder as a source folder, so whenever state machine changes in MPS, generated files will be seen by IntelliJ.

What’s next?

While having all this code is great we still need:

unit tests for state machine
unit tests for state machine DSL

baseLang.collections.list and set cheatsheets

noreply@blogger.com (Dmitry Kandalov) — Mon, 08 Mar 2010 02:31:34 PST

Here are cheatsheets for list and set types from jetbrains.mps.baseLanguage.

[...] is literal for sequence.
should be is assertEquals() which can compare sequences.
given list is equivalent to

{
  list<int> aList = new linkedlist<int>;
  aList.addAll(...);
  aList.someListOperation();
  ...
}

(Project source code is here.)

baseLang.collections.sequence cheatsheet

noreply@blogger.com (Dmitry Kandalov) — Mon, 08 Mar 2010 02:33:20 PST

Here is a cheatsheet for sequence type from jetbrains.mps.baseLanguage.

[...] is literal for sequence I came up with.
should be is assertEquals() which can compare sequences.

(Project source code is here.)

Meta Programming System: editor for Miss Grant's controller

noreply@blogger.com (Dmitry Kandalov) — Mon, 15 Feb 2010 13:42:52 PST

I’ve been playing with MPS over the last couple months so I felt like I have to write something about it. Here is Martin Fowler’s introductory example from his upcoming book about DSLs.

If you haven’t read introductory example, it might be a good idea to take a look now because without it the following example won’t make much sense. I will also assume that you know basics of MPS and at least have read MPS tutorial.

What it’s going to be like in the end?

There will be nice editor for state machine language with text completion and syntax highlighting. The syntax will be close to Miss Grant’s Controller. Something like this:

(To be fair, there is already secretCompartmentLanguage bundled with MPS which is kind of the same thing. There are some differences between Martin Fowler’s example and secretCompartmentLanguage, though. In secretCompartmentLanguage there are no commands and states don’t have actions.)

Editors for StateMachine, Event and Command

First off, let’s create concepts for AST which will be used in editors. We have state machine which receives events, changes its state and sends commands. Here is the first version of StateMachine, Event, Command and State:

Note that all concepts implement INamedConcept interface and both Event and Command implement IHaveCode:

Now when we have concept let’s create editor for StateMachine:

After all the changes editor for StateMachine will look like this (note that cell collections for %event%, %command% and %state% are vertical):

Let’s create simple editors for Event and Command concepts…

… and this is it. On the whole editors will work like this:

After generating language …

… we’ll be able to create instance of StateMachine concept and actually use state machine editor:

Reset events for state machine

In his state machine Martin Fowler has notion of reset events. Events which return machine to its initial state. In terms of our language AST, reset events are just links (or references) to already defined events. Let’s add resetEvent reference to StateMachine concept:

Turns out that unlike child nodes, there can’t be references to multiple nodes (this is not conceptual problem, ”it’s just not implemented”). As a workaround let’s create EventRef concept which will hold one reference to Event. Then, we’ll be able to add several EventRefs as StateMachine children.

Because EventRef concept is different from Event, we’ll need new editor for EventRef which will show name of Event it references to:

After language generation …

… editor for state machine will look like this:

Editor for State concept

At the moment State concept is pretty much empty and doesn’t have editor:

In Martin Fowler’s example State have actions (commands which are send on state entry) and transitions (rules which say when state machine will change its state). Let’s add actions to the State concept. There are multiple actions and they reference to already defined commands so there is the same problem as with resetEvents. We have to create CommandRef concept:

Creating concept and editor for transition is pretty much straightforward:

Now we can create editor for the State and see how it works:

“Add Action” intention

When all the states from Miss Grant’s controller example are added, states declaration will look like this:

Unfortunately, some of the states don’t have any actions and empty actions statements add noise. Let’s add condition to hide actions statement if there are no actions to show.

After language generation all empty actions are gone:

This creates another problem. When new state is added, it doesn’t have any actions. Because of that actions statement is not shown and there is no way to add new actions. This is one of the downsides of editing AST directly. There is a good solution, though. We can use intention which will add empty actions.

Syntax highlighting

The last thing that’s left is syntax highlighting. It can be easily done with stylesheets. (Note that stylesheets are created in editor module.)

Now we can use this stylesheet in editors to attach different styles to different parts of state machine language.

After applying styles to all editors, the final result will look like this:

Books I read (2009, 2008)

noreply@blogger.com (Dmitry Kandalov) — Mon, 08 Feb 2010 13:13:21 PST

Recently I was thinking about which book to read next and came up with idea to make a list of books I've read over the last two years to understand where I'm now, where I'm heading to and may be read some of them again.

I hope you can find something interesting in this list. (Books I considered not related to software development are excluded.)

2009

[!!] Made to stick - why nobody remember company mission statement right after they hear it, but everyone know urban legends? It's a very practical book telling why it happens and how to make you ideas spread. The book of the year.
[!] Refactor your wetware - great collection of ideas about how to improve and rewire your mind. Contains a lot of interesting references to other books. Can be a life-changing book!
Don't make me think - web usability 101. Awesome reading. Nice pictures. I wish I had read this book years ago.
Presentation Zen - make presentations like Steve Jobs :) It's real.
Domain Driven Design - software design as it should be.
Refactoring to patterns - the name says everything. Refactoring + patterns.
Real World Haskell - hands-on book about Haskell. I haven't finished it though :(
User stories applied - a guide how to use user stories in XP and Scrum.
Manage it! - practical agile project management.
The Humane Interface - interesting and sometimes surprising view on human interaction with computer interfaces.
Developing Multi-agent Systems with JADE - makes sense to read only if you like multi-agents or use JADE.

2008

[!!] My job went to India (was renamed to The Passionate Programmer) - very honest and realistic view on programming. Nothing about coding, only real-life pieces of advice. This is like The Pragmatic Programmer 2nd no-technology-edition. The book of the year.
[!] Getting real - concise and opinionated book about web development and software development in general. It's awesome. It's free and available online in HTML format.
Ruby Way - my favorite ruby cook-book. I still read it every now and then.
Effective Java 2nd edition - a must-to-read book for every java developer. (Even if you've read 1st edition.)
Implementation patterns - for me it's a bit unusual, but very interesting view on programming. Kent Beck can always take ordinary things and turn them around. This is Small Talk Best Practices rewritten for java.
Analysis patterns - code is not the only thing you can reuse. Conceptual models might be more appropriate. Martin Fowler's first book. I'm a fan.
Clean code - you should really care about your code. Agile Principles, Patterns, Practices was better, though.
Release it! - real-life description how to create web applications which don't fail after going into production. (This book is focused on java.)
Pragmatic SVN - concise and pragmatic guide to SVN.
How to become a highly paid corporate programmer - controversial book. I didn't agree with a lot of things. That's why I kept reading.
Scrum and XP from the trenches - real-life view on Scrum. Concise and easy to read. It's free to download from infoq.com.
A Guide to the PMBOK - "old-school" ideas about management. Boring to read but very useful.
Black swan - accept that there are unknowns you don't know about. Models we use to predict future will never really work. This is pragmatic nihilism.

Win + key shortcuts in IntelliJ

noreply@blogger.com (Dmitry Kandalov) — Mon, 15 Feb 2010 13:35:40 PST

I always thought it’s not possible to assign Win + key shortcuts in IntelliJ. The reason was that if you press Win + key in shortcut editor, you see something like this:

It looks very much like IntelliJ cannot use this kind of shortcuts. Turns out it can!

The trick is to use Win button as a first key-stroke and key as a second one. Surprisingly enough it works when Win + key are pressed simultaneously.

(In the above picture I assign Win + S shortcut to split editor. I love left-handed shortcuts which easy to use :))

I’ve only tried it in Windows XP. Would be nice if it also works in Vista and Windows 7.

Design Limits

noreply@blogger.com (Dmitry Kandalov) — Mon, 23 Nov 2009 08:04:12 PST

"A lot" doesn't mean "good" even there is a lot of something good. You should know when to stop.

One of the worst thing I've ever done as a developer is over-designing. I've overdone many other things too, but over-designing is probably the number one. Like many other words the word "design" is overloaded. In software development world it usually means a structure of classes or the process of creating this structure. For example, Design Payoff (from Martin Fowler's Design Stamina Hypothesis) has always meant for me that "if you have a good structure, you will be able implement a lot of stuff; if you have no structure, you're out of luck". I can't say for sure whether this diagram implies "design" not only as structure but also as activity. Probably, the main point of it is just "payoff line". If you redraw this diagram in terms of "features per iteration", you will get something like this. From this diagram it's easy to see that the amount of features you can implement with "good design" is limited. This is kind of obvious thing which, probably, every developer knows very well from he's own experience. "Design" as the process of creating "design structure" comes to play every time you want to implement something which turns out to be cumbersome or very hard to do with current design. This is what I blame myself most of all. First off, it's the amount of time spent redesigning. I do not claim it to be true, but thinking about last several months for me the following dependency seems to close to reality. After a certain point the harder you try to "improve" design, the worst result you get. In other words, there is also a limit of much effort it's worth spending on design. The worst part of it is that determining how close you're to "masturbation point" is always a judgment call. Basically you don't know where you are on the curve. Even when you feel you've crossed the point, there is always temptation instead of Killing Your Darlings to continue in hope of getting some insight and turning things right. Spending more time redesigning something doesn't necessarily mean that design becomes more complicated. It can become simpler but less appropriate. It's also true that the time you spend designing is the time you could've spent adding new feature. The only good way I found to avoid this is to add feature with minimal changes and then consciously experiment with design ideas. Or just wait.

Going meta, this can be applied to other areas like code conventions, static code analysis or static dependencies rules.

Slides from "DSL explained" presentation

noreply@blogger.com (Dmitry Kandalov) — Sun, 08 Nov 2009 07:21:22 PST

View more presentations from dkandalov.

IntelliJ good intentions screencast

noreply@blogger.com (Dmitry Kandalov) — Wed, 04 Nov 2009 04:41:01 PST

Here is the screencast where I show IntelliJ some tricks (mostly intentions) while doing kata 4. This is a rough version. I would never think that creating 20 minutes screencast can take so much effort. Actually, I stopped at some point and focused on delivering it. In other words I’m not satisfied with how this screencast looks. But putting even more efforts in it without feedback is even worse. Therefore, I would very much appreciate any kind of feedback. (“It sucks, don’t ever do it again” kind of feedback is acceptable :) but I would prefer it said in a less direct way.)

Technical details:

video captured with CamStudio;
video editing done with VirtualDub;
audio recorded and processed with Audacity;
on-screen keyboard is Comfort On-Screen Keyboard (trial version; it’s not free).

Code formatting vs TDD

noreply@blogger.com (Dmitry Kandalov) — Mon, 26 Oct 2009 01:08:48 PDT

If you are a professional programmer, I'm pretty sure that you format your code. But could you convince someone who doesn't to do so? Not force, but make the person believe it's the right thing to do.

A while ago I was lucky to participate in a team-building game called "Council of the Marble Star". In this game you are divided in several teams. Each team has a certain amount of resources (like triangle and square pieces skin) and several cards with rules (like "after the first round teams will get 10 points for each square they own"). At the end of the game all the teams which have more than say 1000 points win, other teams lose. There are no other rules. You can trade your resources and information with other teams in any way you want. I played similar games before and knew that there are enough points for all teams to win. The only real problem for teams is to trust each other and share information. If no one shares, few teams win. If everyone shares, everybody win. There was another guy in the same team who also knew it. Given that, our team strategy was to convince others to share information so that everybody could win. We don't trade. We publish all our information for free. We tell people everything we know. Guess what happened. We lost. As well as several other teams. Why?! From the very beginning we knew how this game works. People we were trying to convince were co-workers, not strangers. It was just a game, not a matter of life and death. And still we failed to convince other teams to do the right thing. (In case I get into marketing one day and HR manager reads this... I learned the lesson, I can do much better now! I'll tell "how" on interview :))

While this is a funny story, it's not about programming.

Here is another story about programming. I was sitting next to my younger brother writing his first real-world program. ARP ping in C for Linux is a pretty cool start, isn't it? The program wasn't big. Just command-line arguments handling, scanning network in the same thread and printing results. The program worked fine. The only problem was that he did not use indentation. Ok, it was a "problem" from my point of view. Everything works, he understands code. Where is the problem? Why should he care that every piece of C code he can find on Internet has indentation? I think, it is not necessary to say that I failed to convince him to use indentation. (As a side note, he used a lot of procedures and no nested ifs, so the code didn't look bad.)

Can it be that there is something similar happening with other arguments? You know you're right, but you just cannot effectively communicate that. If it's the case, for me the first one on the list is TDD, followed by other XP and agile practices. It's true that the fact that you can't persuade someone to do X, doesn't make X a good thing. I know there are always new problems with the solutions which are supposed to solve your old problems. On the other hand, how do you know where is the line between obvious and not so obvious, line indentation and TDD? I'm serious. If I don't have lots of nested structures, if I can understand what I'm doing, all this code formatting stuff is just a waste of time.

Note to myself: read "Fearless change"

Zen Programming

noreply@blogger.com (Dmitry Kandalov) — Wed, 21 Oct 2009 15:07:32 PDT

Are you tired of moving methods around to “improve” design? Are you tired of trying to understand other developers’ code? Are you struggling to learn IDE shortcuts? Don’t know what to do with automated refactorings in “Refactor” menu? There is good news, you don’t need it all. Acknowledge the true nature of software. The only thing you need is to know the problem you’re solving and write code.

I knew I was out of place from the very beginning. I started the application and after splash screen it displayed an error. “What’s is wrong with it?” I asked another developer. “It’s ok. Just try starting it several more times, and it will work,” he said. I was, and still am, proponent of technical practices which are often used under “agile” term. Almost everything in this place was contradictory to my believes. (I will omit non-technical problems.)

There was no up-to-date documentation and no one could cohesively explain what application does, what are the inputs and outputs. (I don’t mean no one knew, only that no one could explain.)
There was only one development environment shared between everyone. (To make the matters worse databases were located far away and application start-up time was more than 10 minutes.)
At times application won’t start at all for unapparent external reasons.
There was no dedicated build server.
There were no unit tests (existing ones were broken and abandoned long ago). The only meaning of the world “test” was to test things manually.
There were no full-time testers on the project.
There were a lot of classes with 1000 lines and more. Sometimes with mixed business logic, UI and inputs.
The worst thing was that no one cared about it. And there was a reason…

While my productivity sunk, most of other developers were making good constant progress. If application wouldn’t start, they would know which dependent-on services to restart and who could do it. They could test code manually in debugger and it would work just fine. They could make sense of huge piles of multi-threaded code by just stepping through it in debugger. They were not writing clean or perfect code. They were making mistakes sometimes, but non-critical ones). They were adding features in time and bringing business value in spite of everything.

It’s said “change your organization or change your organization”. I believe there is another option. You can learn what others are doing and do exactly that. This is what I’ve been trying to do over the last year. I called this “zen-programming” (name inspired by this book) and from my humble observation here is the most contradictory things Zen Programmers do.

You discard complicated tools, because you don’t need them. They are distraction.

You don’t automate tasks, because you can do them yourself fast enough.

You don’t write documentation, because source code is the best and the only credible documentation.

You don’t pay much attention to your code appearance, because the essence is not in how it looks, but how it works.

You don’t let code drive you, because you’re the master of code. Beautiful cloud castles from code are waste of your time.

You never rely on UML-like pictures to explain a program. They are inaccurate and misleading.

You are not afraid of duplication, because you know well enough where it is.

You don’t spend time beatifying code and making it easier to understand, you live in code, you know it too well.

When working with someone else’s code, you don’t think “why” you think “how”.

You don’t need to double-check your ideas, because you know you’re right. Even though your solution may not be perfect, it will work just fine. You totally trust yourself.

You never attempt to try create anything perfect, because it costs too much and no one needs it. “Most useful” doesn’t mean “perfect” (far from this).

As a Zen Programmer you acknowledge messy and chaotic nature of software. You don’t struggle with it, but embrace it by becoming a part of it. While there is a value in well designed, thought through systems, code is not the purpose of software nor they reflect what software really is.

Update: what I meant by Zen Programmers seems similar to Duct Tape Programmers except that zen-guys are not in the least afraid of "C++, templates, multiple inheritance, multithreading".

Misc IntelliJ tips

noreply@blogger.com (Dmitry Kandalov) — Wed, 18 Nov 2009 13:14:43 PST

Here are miscellaneous IntelliJ tips. Some of them are very simple and yet took me a long time to discover. I hope you’ll find something new and useful among them. But please be aware that while using cool IDE features can make your work easier and more fun there is a limit to which they can help. After all, software development is not about using IDEs. “Use a Single Editor Well” is only one tip out of 70 in pragmatic programmer tips. Be aware also of baby duck syndrome and try another editor or IDE. Emacs, for example :)

DEBUG

When debugging you can see variable value under mouse cursor with alt + < left button click >

“Alt + click” can be more convenient compared to alternatives like pointing mouse on a variable, waiting for popup and then clicking on ”+”. Not to mention selecting text and using “Evaluate expression” action. “Alt + click” fits well with “browsing mode” (left hand on keyboard, right with mouse).

You can disable breakpoint with alt + < right button click on the gutter >

There isn’t much to say about it, just a fast way to disable one or several breakpoints. To disable all breakpoints it’s still easier to use “Mute breakpoints” in debug window.

You can create “tracing breakpoint” with shift + < left button click on the gutter >

This is just a shortcut for adding a breakpoint and then configuring it. By default “tracing breakpoint” will have configuration like in the picture below.

It’s easier to browse external stacktrace with stacktrace analyzer, rather than navigate to lines manually

While writing I found that stacktrace analyzer has already been mentioned in several posts. See JetBrains posts or post on java lobby. There is also a nice feature in stacktrace analyzer that it takes content from system clipboard, so that you don’t have to paste stacktrace in it manually. You select stacktrace in email (or where it is), copy it into clipboard, go to IntelliJ and press Alt+Z, S, Enter (I use menu :)).

If you want to find exceptions of which you are not aware, consider adding exception breakpoint

How well it will work depends very much on how exception are handled in your project. I found exception breakpoints useful while working on a project where the main pattern for handling exceptions is catching java.lang.Exception and writing stacktrace to log. You may silently get NPE and the only place to see it is the log file which has tons of other stuff. Moreover, you have to search log for exceptions manually. With breakpoint exceptions there is no way you wouldn’t notice the problem.

More generally it may be a good idea to have NPE breakpoint and breakpoint for uncaught exceptions to spot problems right away. (IIRC this is what VisualStudio does.)

If you debug objects which form a tree, consider using custom data type renderers

Basically, “data type renderer” is just a way to tell debugger what it should show and what should not. Here is post about type renderers in IntelliJ blog with more details. You can use type renderers for any type. I only found it useful to explore tree-like object structures. In particular to observe UI components hierarchy, antlr AST and IntelliJ PSI trees.

If you debug program with recursive (or complicated) call stack, consider marking instances to better understand it

Put simply, you can mark class instances so that to distinct them from other instances.

I had a hard time coming up with realistic example so here is not elaborated one. There is a Dog class with methods eat(Dog) and barkAt(Dog). Suppose that dog instances are dynamically created in some complicated way. You run the program and see the following call stack (in real world it will be diluted with calls on instances of other classes):

From the picture we see that the Dog#eat() method is being called recursively. It doesn’t necessarily mean that there is a bug in code. It may be that function is called recursively unless some condition is met. Conditions are usually based on object state so the above picture is not very helpful since it doesn’t show instance on which method is called. This is where marking instances can help. You go through call stack marking “this” objects:

And now, when stack shows instances of Dog, you can reason about what is going on.

If you have external logs files which you want to watch, consider watching them in IntelliJ

While there’s nothing wrong with using external editor to watch logs (for example, less waiting for new input) it can be easier to watch logs in IntelliJ. IntelliJ can filter out standard and error output and do usual text-file stuff like highlighted searching (with regexps).

There is “Logs” tab in the “Run/Debug Configuration” window where you can setup which log files to show (for example, latest of rolling log files). The downside is that with big logs and lots of input console window in IntelliJ doesn’t work very fast :(

BOOKMARKS

If there are several points in code to which you frequently navigate, consider using numbered bookmarks

There isn’t much to say about it. If you navigate to some class or method often enough, try numbered bookmark to see if it helps you to navigate faster. ctrl+shift+number to add numbered bookmark and ctrl+number to go to numbered bookmark.

Alternatively you can add classes and methods to “Favorites” lists (“Add to favourites” in context menu). See also bookmarks post in IntelliJ blog.

You can add bookmark with ctrl + < left button click on gutter >

Does the same thing as pressing F11. Can be better than F11 when you are in “browsing mode”.

TABS

You can close tabs and windows by clicking with middle button or shift + < left button click >

This feature was especially useful before IntelliJ 8, because there were no crosses on tabs with code. But it still works very well for closing windows. See also recent post in IntelliJ blog. This feature is not IntelliJ-specific. There are many other programs (it might be better to say “user interfaces”) which allow user to close tabs with middle button click (eclipse, firefox, opera for example).

You can see full path to file with ctrl + < left button click > on its tab

I don’t actually use it. I imagine it could be useful if clicking on folder in the popup would open this folder in external file manager. May be you have ideas why is it there and how to use it :)

TEXT

If you want to copy several text chunks from one file to another, it may be easier to copy them using clipboard stack

Suppose you want to copy three keys from property file and insert them into java code. Doing it “normal way” you would go to property file, copy one value, go to java file and paste it, then go back to property file, copy next value, go to java file and so on. There is another way to do it. You can go to property file copy all values you want (ctrl + c), then go to java file and paste them using clipboard stack (ctrl + shift + v). It’s simple and can save you from switching between contexts (which you do going back and forth between files).

You can also use numbers to paste text from keyboard. For example, if you want to copy three values, you can copy them into clipboard, go to the target file and press “ctrl + shift + v, 3, enter” three times.

To configure clipboard stack size see “Settings - Editor - Limits - Maximum number of content to keep in buffer”.

If you want to compare file with another one from outside of the project, you can use “Compare with clipboard”

Every now and there is a need to compare two files. It’s straightforward to do if both files are within project. You select these two files and choose “Compare two files” in the context menu. However, it’s not going to work if one of the files is not within project folder, because there is no way to select it. In this case you can copy content of external file to clipboard and use “Compare with clipboard” context action.

Sure, external diff programs can compare anything.

If you want to copy part of log without several left (or right) columns, you can use “column mode”

“Column mode” can be turned on it “Edit - Column mode” or with alt + shift + insert.

Alternatively you can delete text you don’t need with “Replace text” action using regexp mode.

You can copy opened file’s path to clipboard with ctrl + shift + c

You can also do it with folders in the project view. Useful to paste path into external file manager or console. See also IntelliJ features page.

If you want to use some constant or static method from another class, consider copying its reference

This is just a way to avoid typing class name and then constant/method name. Note that you can also use it to paste {@link ..} value into javadocs. See also IntelliJ features page.

Other

You can go to class or method by clicking middle button on it

Does the same thing as ctrl + < click > or pressing F4. Useful in “browsing mode”.

You can go to method or class implementation(s) with ctrl + alt + < left button click >

Useful in “browsing mode”. Alternatively you can open class or method hierarchy (it involves more fiddling, though).

It’s easy to see where checked exceptions are thrown by pressing ctrl + shift + F7 on catch/try keywords

It can also be handy if try-catch block catches java.lang.Exception. In this case it shows all checked exception the block catches. By the way, I still like alt + a much more than the default shortcut.

It’s easier to remove anonymous classes (try/catch blocks and etc.) surrounding some code with ctrl+shift+del, rather than do it manually

In case you haven’t seen it, take a look at IntelliJ features (unwrap). This works in IntelliJ 8 only.

Try “Appearance - Automatically position mouse cursor on default button”

If you’ve never used it, give it a try and may be you’ll like it. Positioning mouse on default button is not specific to IntelliJ, you can as well turn on similar option in your OS.

Finally, try using the latest version of IntelliJ by participating in Early Access Program

Most developers don’t like using “nightly builds” for real-world programming and this is very reasonable. We want to get our work done, not to struggle with buggy software. But there are also reasons to use development builds. The most compelling one is that you can use latest features (it assumes that they help you with what your work on). Another reason is that not all development builds are buggy. Sure, they have more problems than a release which has been out for a year (at least it’s supposed to be so), but not all problems might affect your work. It’s likely that you use only a subset of functionality and there may be problems in the parts which you don’t use or use rarely.

Regarding IntelliJ, I used EAP builds of 7 and 8 versions and from what I’ve seen EAP builds are good enough to be used for day-to-day work. So wait until April or so for IntelliJ 9 and give it a try ;)

Yak scratching

noreply@blogger.com (Dmitry Kandalov) — Fri, 27 Feb 2009 15:20:37 PST

Recently when reading Thoughtworks blogs I coined the “yak-shaving” term. It means “any seemingly pointless activity which is actually necessary to solve a problem which solves a problem which, several levels of recursion later, solves the real problem you’re working on”. There is also an interpretation such as “doing it well now is much better than doing it perfectly later”, but I like the first one better.

As one can expect from working in a big company on “legacy system” it allows you to experience a lot of yak-shaving activities. For example, you want to catch breakpoint to understand a piece of code or make sure you change actually works (ask me why you cannot.. don’t write unit test). To catch breakpoint you would need database server on other continent, several feeds and services to be up (they can occasionally be down) and probably a remote desktop connection to market emulator which is shared by a whole bunch of people.

After reading about yak-shaving I retold the story to several co-workers. Some of them didn’t get it, some (those who know the pain of yak-shaving) enjoyed it, but best of all I liked one response. I told a guy the yak-story, he spent about half-an-hour trying to get a breakpoint according to the above “methodology” and then exclaimed ”I’m scratching the yak”. This, I believe, is more precise description of what we sometimes do. You’re not just shaving it, but scratching. You may know with some degree of certainty how to get to the actual problem, but obstacles you face are so uncertain, unrepeatable and numerous, that despite all efforts you get nowhere and the only thing you left with is misery and despair. Indeed “shaving” sounds to optimistic in this case. “Shaving” assumes you’re making progress. But when you stuck with some totally unrelated problem for hours and then you get another one and then another, “scratching the yak” seems much more appropriate term.

Does it have to be this way? Why do we do yak-scratching? (I won’t go into five why’s.) What can we do about it? May be it’s our incompetence and lack of knowledge? May be it’s the way big systems are developed? Or may be it’s yaks’ conspiracy? From technical point of view we could separate system into modules with clear contracts. Develop modules separately. Create fakes for external systems and write automated tests. Sounds good but it requires political will. You have to convince management it will pay off for business. You have to convince people who work on project it won’t change one yak to another. Not to mention you have to be sure you’re right. (I realize it can be self-deception and blaming others is not a solution anyway.)

Not all technical problems are really technical. Beware of yaks.

Debugging managers

noreply@blogger.com (Dmitry Kandalov) — Thu, 19 Feb 2009 02:38:45 PST

The other day I was debugging the application I work on. At some point looking at the debug window in a “context free” way I realized that there is a whole lot managers in it and what I’m doing is basically debugging managers of managers.

It’s no that I think it’s a bad design or these classes are badly named (they could’ve been better, though). And it’s not that I think this structure is too complicated, after all there are only three levels of managers in the picture (actually more).

What drew my attention was the idea that choice of class names is influenced by organization, its structure and its language. The place I work at use model like: coding monkey –> manager –> superhero manager. Where the last two stages are obviously bound to be attractive. Now imagine how many times you can read the word “manager” in code (fields, parameter, local variables) given the huge number of manager classes. Admittedly it’s not very useful idea, since I can hardly imagine how one could benefit from it.

As the Conway’s Law (which sound like “any piece of software reflects the organizational structure that produced it”) the idea of code being influenced by corporate language boils down to the fact that it’s people who produce software and they inevitably leave traces.

IntelliJ IDEA 2.6

noreply@blogger.com (Dmitry Kandalov) — Sun, 08 Feb 2009 01:45:46 PST

It’s always interesting to see what things were like in the past. It’s even more interesting to see things that are not easily accessible. I can always start vi and pretend I’m in 70s but since it’s so easy to do and because vi (or vim) installed almost on any *nix box it’s not fun. Therefore, I tried old versions of IntelliJ.

I got into java somewhere about 2004. As you can guess in 2004 IntelliJ IDEA 4 was the latest version (you can guess because IntelliJ versions correspond to the current year, so in 2011 there will be IntelliJ 11) and I’ve never used any version prior to 4. Recently I got hold of IntelliJ version 2 and 3 (you can download version 3 here). Summarizing my impression in three words “they are fast”. Here are some screenshots of IntelliJ 2 to get a feel of what it was like.

It's not cracked ;) Just don't want to disclose the owner.

Estimation

noreply@blogger.com (Dmitry Kandalov) — Mon, 09 Feb 2009 13:37:15 PST

Quick! How long does it take to execute about 200 lines of java code (no I/O) on your machine? Why does it take 70 milliseconds?

This is the kind of problem I faced yesterday (although, I didn’t actually measure how many line of java code there was). So I add several breakpoints logging current time to console. In IntelliJ you can click on gutter with Shift pressed (doesn’t work in KDE4 at the time of writing) and get window like this one.

After running code with breakpoints output shows that there is from 15 to 30 milliseconds delay between each of breakpoints, which add up to some 70ms. Is it all? I thought so, but it’s not. After suggestion from a colleague to try “profiling” without writing to console straightaway, I added code like this one (not real code):

    public static final long TIMES[] = new long[20];

    public static void takeTimestamp(int i) {
        TIMES[i] = System.currentTimeMillis();
    }

    public static void main(String[] args) {
        takeTimestamp(0);
        // ... some code
        takeTimestamp(1);

I ran it without debugger attached and see that output is a bit strange because all timestamps are the same. I do make stupid mistakes sometimes, but with code like the above it seems very unlikely. I change timestamps to nanoseconds and see quite surprising result. There are no mistakes, it's just that code I’m “profiling” executes in less than 1 millisecond.

Lessons learned:

don’t “profile” with debugger attached.
“for interval timing, always use System.nanoTime in preference to System.currentTimeMillis” (see EJ 2nd edition). I could’ve saved some time if I followed this advice. I knew I should do it, but didn’t remember why and, seeing that several other developers use currentTimeMillis() in preference of nanoTime(), decided to use it too. Really bad idea. The reason is that it’s platform specific and on XP machine it may have “a granularity of 15 ms” (see this post).
most importantly you should be able to “Estimate to Avoid Surprises” (see The Pragmatic Programmer Tip 18).

Update: I wasn't benchmarking but rather measuring how long it takes for specific code path to be executed in live environment under certain conditions. Should you want to do benchmarking you probably should look at something like framework from this article.

Software history tidbits from Joel

noreply@blogger.com (Dmitry Kandalov) — Wed, 04 Feb 2009 13:26:38 PST

I’ve been reading old articles by Joel Spolsky lately. Starting with post mocking java installation dialog, I went to “People don’t like to read” link and was absorbed by adjacent articles. While enjoying old stories about UI design I found that there are quite a lot of things I don’t know about the past of software industry. Here are some things I read on Joel’s site and googled afterwards (they might be not surprising in the least to you, though):

in the 80’s and early 90’s there was “almost complete monopoly” by Lotus 123 on the spreadsheet market and Microsoft overtook it by making Excel compatible with Lotus.
“when Microsoft Excel 3.0 came out in 1990, it was the first application to sport a new feature called a toolbar”
there was a complete rewrite of Netscape Navigator which lasted for 3 years and gave IE an opportunity to take over the browser market
IE was default browser on Mac for 5 years (I believe I’ve heard about it but since I never owned a Mac I didn’t really care)
Opera is much older than I though with the first release in 1994 (this is from browsers timeline)

I wonder if there is some book about software history over the last 15 years or so.

Update:

There was a good illustration of why I don’t read Joel often in his podcast #38 and following blog post by Robert Martin (with lots of interesting comments). In my view Joel and Jeff make a good point that software development techniques are just tools to provide “business value”. On the other hand, they obviously didn’t try to understand SOLID principles before criticising them. Not to mention that regarding usefulness of TDD I’m totally on the side of Robert Martin :) Looking forward to listening stack overflow with Robert Martin.

I plugged in to IntelliJ :)

noreply@blogger.com (Dmitry Kandalov) — Mon, 29 Dec 2008 14:26:01 PST

This year I found some time to take part in IntelliJ plugin contest. I started writing code only two weeks before the end of the contest, though. So the most reasonable prize I could hope for from the very beginning was “I plugged in to IntelliJ” t-shirt which all the participants get. The reason I chose to write html export is that I didn’t like in the default IDEA export. It lacks annotations and static fields highlighting and have problems with multiline java-docs. Javascript-based highlighters that I tried weren’t very good at highlighting static fields and methods either. Therefore, I rolled up plugin for html export. (There is also Copy as HTML plugin which does mostly the same thing. It might be a good idea to update it to IntelliJ 8 API but after looking at the code I though better off it. Sorry. I’m a bit tired of making sense of and changing code I didn’t write at work.)

All in all I really enjoyed the process of plugin-writing. I don’t think of it as being “finished” and plan to add more features. Here are some things I learned while doing it.

TDD doesn’t replace thinking beforehand

I admit I’ve been and probably still is over-enthusiastic about unit testing and TDD in particular. In my understanding the idea behind doing TDD and not thinking through the whole problem is that you cannot possibly know all the obstacles you’re going to encounter and, therefore, it might be more practical to write some code and learn from this experience. Sometimes this approach works very well, sometimes doesn’t. The only rule that holds true most of the time is that there are no silver bullets and and it’s a judgement call to decide whether some technique can be applied or not (well.. unless you do the same thing all the time). As I found it might be more effective to write some code (I don’t say write “spike code”) and run system in live environment to get broader view of problem, rather than start with unit tests right away.

There is no “right” scope for unit tests

It was after reading xUnit Patterns and Working Effectively with Legacy Code books that I became keen on writing small-scoped tests which wouldn’t interact with other classes or even worse with filesystem of database. These are actually the qualities that make them “unit” tests. It seems to me now unit tests are not the best choice in every case (I mean even when it’s possible to write meaningful unit tests). Sometimes it’s better to write larger-scoped system-level tests. The reason is that unit tests are coupled to interfaces of classes they are written for and, should you want to make big changes in implementation (throwing away or changing a lot of classes along the way, for example), unit tests will be of little help. Here is a pretty good article by Kent Beck on this topic.

“find” and “grep” are faster than IDE search

I’ve been aware for quite a while that I could use something like

find . -name * | xargs grep “some text”

I used it occasionally on remote machines but I’ve never seriously thought of using it on desktop. After all why would I want to with modern IDEs and file managers? The answer is because “find” with “grep” is way faster. I don’t have exact numbers but on projects with large code base textual search in IntelliJ, Eclipse or Krusader may take ten seconds or more. On the same code bases grep does the same thing in just seconds

Checklists are great

They really help to make sure you don’t forget anything. Although the idea of using checklists is not new, I’ve never seen a developer sitting next to me who would use a checklist. I wouldn’t use it either if not for Igor Stepin who mentioned checklists in some discussion.

To make it more concrete here is a snippet from htmlexport “release checklist”.

- fix all todos
- review code (optional)
- run DSM
- run Intellij code review
- make sure all tests pass
- test plugin manually with java, xml
- update plugin xml
- run plugin build from IntelliJ
…

As an afterthought, refactoring descriptions are in fact checklists, but I admit I use built-in IDE refactorings or refactor without any guide.

Gem 'ruby-debug-ide' not found in IntelliJ under Kubuntu

noreply@blogger.com (Dmitry Kandalov) — Sun, 09 Nov 2008 07:29:19 PST

A while ago when IntelliJ got the ability to debug ruby scripts I tried to set it up under Kubuntu what turned out to be not very straightforward. Firstly, there must be ruby-debug-ide gem installed. It seems to be not a big deal? Right, it’s not a big deal to type

gem install ruby-debug-ide

But with default Kubuntu installation (8.4 at that time) it would fail when building some system dependent sources. Thanks to Igor Stepin, who suggested installing build-essentials package, I got around this problem.

The second problem was that even though ruby-debug-ide was installed IntelliJ kept saying it cannot find it. The solution is also not very tricky: you should change “Gem bin directory” in JDK setting from “/usr/bin” (which is default) to “/var/lib/gems/1.8/bin”.

The same applies to debugging in RubyMine. I didn’t manage to find SDK configuration dialog in 435 build and, therefore, fixed configuration through xml configs by copying part of IntelliJ’s jdk.table.xml to ~/.RubyMine10/config/options/jdk.table.xml (text configurations rule!).

Bluetooth in kubuntu 8.10 by hands

noreply@blogger.com (Dmitry Kandalov) — Sat, 08 Nov 2008 05:44:40 PST

I recently updated my desktop Kubuntu to 8.10 which among other things doesn’t have KDE 3.5 anymore but only KDE 4.1. I really liked KDE 3 which all things considered is the best window manager I’ve ever used (three years ago I wouldn’t believe I’d say it but here I am) and, therefore, transition to KDE 4 is pretty much painful for me. One of the problems I encountered was the lack of bluetooth support in KDE 4 which IMHO is something inconceivable for a major distribution like Ubuntu.

Anyways, the only thing I use bluetooth for is to download and remove photographs from cell phone (Sony Ericsson 500i). My first attempt was to install bluez-gnome and use bluetooth-applet as it is suggested in this bug report. Unfortunately, for some reason bluetooth-applet didn’t work. I could see cell phone but “Browse files on device” menu item was disabled no matter what I did. Being desperate after several attempts, I even tried to setup bluetooth in Vista (which was shipped with my laptop). To my surprise it was not easy either because Vista wanted some drivers which it couldn’t find on Internet. And all this after KDE 3 where it took me literally 1 minute to setup bluetooth so that to edit cell phone filesystem.

Of course, no bluetooth in KDE 4 doesn’t mean you cannot use it at all. So after some googling I found out that there is obextftp command line tool which can be used as shown here or here. After a little while I ended up with small script which finds directory with photos and downloads photos to the current directory:

#!/usr/bin/ruby

PHONE_MAC = "00:1E:DC:1E:82:AA" # can be found with `hcitool scan`
CHDIR_TO_CAMERA_ALBUM = "obexftp --bluetooth #{PHONE_MAC} --chdir 'Phone memory' --chdir 'Camera album'"

output = `#{CHDIR_TO_CAMERA_ALBUM} --list`
photos_folder = output.match(/folder name="(.+?)"/)[1]

output = `#{CHDIR_TO_CAMERA_ALBUM} --chdir '#{photos_folder}' --list`
photos = output.scan(/file name="(.+?)"/).flatten

photos.each do |photo|
  `#{CHDIR_TO_CAMERA_ALBUM} --chdir '#{photos_folder}' --get #{photo} --delete #{photo}`
end

It was the best thing KDE 4 forced me to do so far. No more clicking around with mouse, console is your best friend :)

Test::Unit design

noreply@blogger.com (Dmitry Kandalov) — Mon, 03 Nov 2008 04:46:05 PST

A while ago I posted a couple of diagrams which I used to make sense of JUnit design. Almost at the same of drawing those ones I took a look at implementation of Test::Unit and RSpec frameworks. As a consequence here is a simplified overview of Test::Unit design.

Design in general

Given a TestSuit, TestRunner creates TestRunnerMediator, registers itself as TestRunnerMediator listener and then asks TestRunnerMediator to execute tests.
TestRunnerMediator creates TestResult, registers itself as TestResult listener and then relays request to execute tests to TestSuite.
TestSuite contains TestCase instances for each test method and upon request runs each of them.
TestCases actually execute test code (including setup and teardown methods) and notify TestResult on assertions, errors and failures. These events are propagated to TestRunner which actually provides some feedback to user, for example, write results to console.

All notifications are implemented using Observer module, so MediatorListener and TestResultListener interfaces in the diagram show roles played by classes in collaboration rather than some kind of interfaces in terms of the language.

How it actually works

Since the above description is somewhat abstract here is how Test::Unit is actually used from scripts if you ran TestCase as a standalone script:

When you require test/unit module in a test script, Unit adds a block to be executed when script is about to exit through the Kernel::at_exit method. This block asks the AutoRunner class to execute tests.
AutoRunner creates OptionsParser which reads parameters from command line and based on them creates TestRunner. Also AutoRunner chooses and creates Collector instance which finds tests and packs them into a TestSuite.
AutoRunner passes TestSuite to TestRunner and asks it to run tests.
Given a TestSuit, TestRunner creates TestRunnerMediator… etc

Should you want to see available TestRunners, Collectors or other options, the AutoRunner is probably the best place to get started. Another interesting place to go (it’s obvious and can be easily found from Test::Unit documentation but anyway) is Test::Unit::Assertions module, which contains available assertions.

If you don’t want to use AutoRunner, there is another way to execute tests which is to manually create TestRunner and pass to it a TestSuite (in this case Unit module won’t register block to execute at exit and AutoRunner won’t be used). It looks like this (code snippet is taken from Test::Unit rdoc):

require ‘test/unit/ui/console/testrunner’
Test::Unit::UI::Console::TestRunner.run(TCMyTest)

P.S. there is also a bit confusing (at least for me) RUnit package bundled with Ruby. I couldn’t find any authoritative information about how it’s related to Test::Unit. Judging from the code, it seems to be deprecated API left for backward compatibility which shouldn’t be used in new code.

JUG with Joshua Bloch

noreply@blogger.com (Dmitry Kandalov) — Mon, 20 Oct 2008 13:03:28 PDT

The weekend before the last one I was lucky enough to attend JUG meeting with Josh Bloch invited as a speaker. The event took place in St. Petersburg. It was a bit far away to go for me, but even though I’ve never been a huge fan of Josh Bloch (I read and enjoyed Effective Java :)) I had decided it was worth going and was right.

Josh gave two talks: “More Effective Java” which essentially highlighted some new parts of the 2nd edition of Effective Java and “How to Design a Good API and Why it Matters” talk which had also a lot in common with the Effective Java book. The bad thing for me was that I had read Josh’s book before (including 2nd edition) and I have even watched him giving a talk about good API on the Internet, so I’ve read or heard probably about 98% of content. I don’t mean I know all these things really well, but still it wasn’t exciting. There was a chance to see java-puzzlers with generics but unfortunately everyone voted against it in favor of API speech. Josh Bloch is an excellent and quite articulate speaker so it still was a real pleasure to listen to him. The good side of it is should you be interested in Josh Bloch’s views on how to develop software and pieces of advice, you can read his book (or may be also watch a video) to get the most of it… 98% or so :)

The most surprising for me was how everything except the speech itself went. First off, it was Josh Bloch setting up his (well.. Google’s) laptop and projector totally on his own sometimes crawling among the chairs. It’s by no means a complaint about organization of the event, it’s just was quite different from what I had expected or seen on MS events. Another thing was that there was almost no excitement about Josh Bloch, although, he was very friendly readily posing for photographs and singing books. After the speeches there were not more than 20 people out of more than 100 who decided to join the dinner and at most 7 of them who were talking or listening to Josh. For me it was the best part where everyone could ask any questions and as many of them as they would like. I admit I wasn’t prepared and didn’t have interesting technical questions :( Still it was a great experience to see in details what the man who influenced java-world a great deal thinks, what values has. It can be found “offline” but real-life experience is much better and it surely changed my understanding of Josh Bloch’s writings and made me think how geeky I got over the last few years.

These are The Only Really Important Answers ;) I got (reproduced as I remembered them):

Q: What IDE do you use?
A: In our team at Google we use IntelliJ, so it’s IntelliJ. I haven’t used other IDEs a lot.
Q: What OS do you use?
A: We use Ubuntu at Google. I have several XP boxes at home and I used to have Mac. I hate all OS equally :)

JUnit design

noreply@blogger.com (Dmitry Kandalov) — Tue, 21 Oct 2008 12:36:04 PDT

JUnit design

It started a while ago when I read Jay Fields’ inspiring post about being your own start-up where he mentioned two books one should read. One of the books was My Job Went to India. Despite its weird name, it is a great writing with honest pieces of advice how to be a better developer. After reading and truly enjoying it, I realized the worst thing can happen now is that I mark it as “read” on the list and move on. So I decided to follow some of the ideas from “act on it”-parts. One of them is to find new ideas and inspiration from projects that are generally considered good.

Here is just a high-level overview of how junit works in my over-simplified interpretation in case I loose a piece of paper I drew it on :) For more substantial overview of junit 3.8, take a look at cooks’ tour or excellent paper by Dirk Riehle.

JUnit 3.8

Given a class TestRunner tries to use it as a TestSuite. If it’s not a TestSuite, TestRunner creates TestSuite with TestCases for each test method in this class. TestRunner creates TestResult object which collects results of running tests and passes them to TestListeners. Tests notify TestResult which passes events to TestListeners. TestRunners bundled with junit are also TestListeners since they subclass BaseTestRunner.

JUnit 4.x

Given a class JUnitCore creates a Request. Request decides which Runner to use for executing tests (this is where @RunWith annotation is used; see Mark Burnett’s post for example of how to create custom Runner). Runner finds test methods and executes them. For each test Runner creates Description which is sent to RunNotifier and then passed to RunListener. After tests are executed JUnitCore sends Result to RunNotifier and RunListener.