Einar Ingebrigtsen

Windows 8 Developer Kickstart

einar@dolittle.com (Einar Ingebrigtsen) — Thu, 12 Jul 2012 11:36:45 -0300

Windows 8 is around the corner and I was asked to do a talk for a kickstart event Microsoft is doing.

My Talk will be on doing metro apps with HTML, CSS and JavaScript, how you can leverage your existing Web experience and knowledge and take advantage of some of the functionality specific to Windows 8.

The event will be in Oslo @ Microsoft Norway. If you're interested, jump over to here to register for it.

CQRS applied : a summary

einar@dolittle.com (Einar Ingebrigtsen) — Thu, 12 Jul 2012 02:19:15 -0300

Every now and then in a software career you get a chance to write something from scratch and try out new things; a proper greenfield project. I've had that luck a couple of times and latest a project that proved to be the complete game-changer for me personally. Game changer in the sense that I gained a knowledge that I am pretty sure I will treasure for, if not the rest of my career, at least for quite a few years moving forward. The knowledge I am talking about can be linked back to applying CQRS, but it is not CQRS in itself that is the knowledge, its the concepts that tag along with it and the gained knowledge of how one can write code that is maintainable in the long run. Its also about the things we discovered during the project, smart ways to work, smart code we wrote - techniques we applied in order to meet requirements, add the needed business value, and at the same time deliver on time with more than was asked for.

This is a more in-depth post than the talk I did @ NDC 2011

… from the top ...

For the last couple of years, till March this year, I had the pleasure of being hired by Komplett Group, the largest e-commerce in Norway. At first I was assigned tasks to maintain the existing solution and was part of the on-premise team to do just that. As a consultant, that is very often what you find yourself doing - unless you're hired in to be a particular role, like I've been in the past; system architect. I helped establish some basic architectural principles at that time, applying a few principles, like IOC and other parts of our favorite acronym; S.O.L.I.D. I remember feeling a bit at awe of just being there, they had a solution that could pretty much take on any number of clients and still be snappy and they never went down. I've learned to respect systems like that, even though it requires a lot of work - not necessarily development work, but a lot of the time IT or DevOps help keep systems alive. Anyhow, after a few months, back in 2009 I was asked by the department manager if I wanted to lead a small team on a particular project, an administration-tool for editing order details. With my background earlier as a team lead and also as a department manager myself, I kinda missed that role a bit and jumped at it. It was to be a stand-alone tool, accessible from the other tools they had, but we were given pretty much carte-blanche when it came to how we did it, whatever technology within the .net space we wanted. We settled on applying ASP.net MVC, Silverlight for some parts, WCF for exposing service for the Silverlight parts and nHibernate at the heart as the ORM for our domain.

Part of the project was also to try out Scrum, having had quite a bit of experience with everything ranging from eXtreme Programming to MSF Agile and later Scrum, that excited me as well. So we applied it as well.

Half-way through the project we started having problems, our domain was the one thing we shared with the others and we started running into nightmare after nightmare because we worked under the one-model-to-rule-them all idea. Which is really hard to actually get to work properly, and looking back I realize that most projects I've been have suffered from this. We ran into issues were for our purpose we needed some things in *-to-many to be eager fetched, which had consequences we could not anticipate in other systems that was using the same model. But we managed to come up with compromises that both systems could live with - still, we weren't seeing eureka, just brushing up against the problems that a lot of projects meet without seeing that the approach was wrong. A bit after this we started brushing up against something that really got us excited; Commands. Without really knowing about CQRS at this point, but more coming from working with Silverlight and WPF, the concept of modeling behavior through commands. The reason we needed these commands was that we needed to perform actions on objects over a long period of time; potentially days, and at the end commit the changes. We came up with something we called a CommandChain - a chain of commands that we appended to and persisted. Commands represented behavior and modified state for the most part on entities when executed. We came up with a tool were we could debug these chains, and we could inspect which Command was causing problems and not.

All in all, we were quite pleased with the project; we had done a lot of new things, applied TDD in a behavioral style, started exploring new corners of the universe we had yet to realize the extent of. Delivered not too badly on time, not perfectly on time - but close enough.

The turning point

After yet another 6 months or so, there were initial talks about the need to expose functionality from the web-shop to other systems used internally, a few design meetings and meetings with management lead to a new project. The scope of the project turned out to be not only exposing some services, but also a new web-shop frontend targeted and optimized for smartphone devices. The project was initiated from a technical perspective and not one with a specific business need in mind. From a technical perspective, the existing codebase had reached a point were it was hard to maintain and something new needed to replace it to gain back velocity and control over the software. It was to be a complete greenfield project, totally throw things overboard and just basically work with existing database but add flexibility enough that even that could be thrown out the door, if one ever wanted to do that. Early on I was vocal about them needing an architect to be able to deliver this project, I pointed in a couple of directions to internal resources they had - but people pointed back to me and I soon found myself as the system architect for the project.

Requirements

When dealing with e-commerce at this level, there are quite a few challenges. Lets look at a few numbers; in the product catalog there was at the time I got off the project about 13.000 products, there was an order shipped every 21 seconds, in 2011 that amounted up to 1.454.776 orders, ~30.000 living sessions at any given time. Sure, its not Amazon, but for our neck of the woods its substantial. These numbers are of course on an average, but come busy times like Christmas, these numbers are more focused and the pressure is really on for that period in particular.

Decisions, decisions, decisions...

Before we started production, back in November 2010, we needed to get a few things straight; architecture, core values for the project, process and then getting everyone on-board with the decisions. We early on decided that we were going to learn all about CQRS, as it seemed to fit nicely with the requirements - especially for performance, and we were also requiring ourself that we wanted a rich domain model that really expressed all aspects of the system. We also decided that we wanted to drive it all out applying BDD, and we wanted to be driving the project forward using Scrum and really be true to the process, not make our own version of Scrum. A dedicated product owner was assigned to the project that would have the responsibility for the backlog, make sure that we refined as needed, planned as needed and executed on it.

Adding the business value

As I mentioned, this project came out of a technical need, not a concrete business need. We had the product owner role in place and he needed to fill the backlog with concrete business value. This was not an easy task to do, basically because the organization as a whole probably didn't see the need for the project. In their defense, they had a perfectly fine solution today, not entirely optimal for smaller screens like a smartphone, but manageable. To the different store owners that normally provided the needs to the backlog, they were in desperate need of new features on existing solution, rather than this new thing targeting a platform they didn't see much business value in adding. In combination with the fact that the organization had been in migration mode and all developer resources partly or close to full-time in periods being tied down to work related to migration of systems that was a result of merges and acquisitions, the organization had gotten used to not getting things done anyways. All this didn't exactly create the most optimal environment for getting the real business value into the project. Something we really wanted. Early on we realized that the project could not be realized if we had user stories that were technical in nature. The first couple of months we did have quite a few technical user stories, and statistically these failed on estimation. We didn't have any business value to relate them directly back to, and ended up in many cases as over-engineering and way out of their proportions as we as developers got creative and failed at doing our job; add business value. So we came to the conclusion; no technical user stories were allowed - ever. Something that I still today think was one of the wisest decisions we had on the project. It helped us get back and focus on why we were writing code every day; add business value. Even though this project was a spawn of the developers, there was clearly business value to guide us through. The approach became; lets pretend we're writing an e-commerce solution for the first time. This turned out to be a good decision, it helped us be naïve in our implementations - keeping in line with core principles of agile processes; the simplest thing that could possibly work. Our product owner was then left with the challenge of dragging the business value out of the business, he did a great job in doing that and at the same time getting them to realize the need for the change of platform that was in reality taking place. Something that became evident further down the line; we were in fact not building an e-commerce front-end for smartphones, but an entire new platform. More on that later.

YES, we did create a framework

One of the realizations we had early on was that we needed to standardize quite a few things. If you're going to do that many new things and have a half-way chance of getting everyone with you and feel productive in the new environment, you need to get a basis working that people can work with. Back in 2008 I started a project called Bifrost, you can read more here. We looked at it and decided it was a good starting point for what we wanted to achieve. We also wanted the framework to be open-sourced. The philosophy was to create a generic framework to be the infrastructure sitting at the core of the application we were building. It would abstract away all the nitty gritty details of any underlying infrastructure, but also serve as the framework that promoted CQRS and the practices that we wanted. It was to be a framework that guided and assisted you, and very clearly not in your way. I'm not going to go in-depth in the framework, as there are more posts related to it specifically in the making and already out there.

CRUDing our way through CQRS

Well on our way, we had quite a few things we really couldn't wrap our heads around. Coming from a very CRUD centric world, the thought of decoupling things in the way that CQRS was saying was really hard. And at the same time, there were potential for duplication in the code. I remember being completely freaked out at the beginning of the project. All my neural cells were screaming "NO! STOP!" - but we had to move on and get smarter, get passed the hurdles, learn. At first we really started making a mess out of things, just because we were building it on assumptions - the assumptions that CQRS is similar to doing regular old CRUD with what we used to know as a domain model. It was far from it, and we had a true eureka at one point were we realized something important; we were working hard an entire day on how to represent some queries in a good way so that they would be optimal in the code but also execute optimally - and it hit us as a ton of brick after leaving work that day. We were doing everything wrong, and we even came up with a mantra; "if a problem seems complicated, chances are we're doing it wrong". That was the turning point that helped us write code that was simpler, more testable, more focused, faster and we picked up pace in the project like I've never experienced before.

From that point we had our mantra that really proved as a guiding star. Whenever we ran into things we didn't have an answer to straight away and we started finding advanced solutions to the challenges, we applied the mantra and went back to rethink things.

Tooling

Early in the project we realized we needed a tool for both visualizing the events being generated, but also be able to republish events. We came up with a tool built in Silverlight, using the pivot control from Microsoft to visualize.

The real benefits

Looking back at what we did and trying to find the concrete benefits, I must say we now have gained serious amount of knowledge in a few areas. The thing that CQRS specifically gave us was the ability to model our domain properly. We achieved the separation we wanted between the behavior of the application and the things the behaviors caused changes to, the data on the other side. It helped us achieve greater flexibility, easier maintenance. Since we decided to not only just apply CQRS, but also build a reusable framework sitting at the bottom, we achieved a certain pattern of working that made things really easy to get started with development, and also a recognizable structure that made it easy to know were to put things if the core principles was explained to you.

I think by far the biggest benefit we achieved was the insight into how we should be developing software. Keeping things simple really have huge benefits. Decouple things, staying true to single-responsibility in every sense of the word single.

Another huge realization I had, something I have been saying all the time throughout my career but really got re-enforced with this project; concrete technology doesn't really matter. Sure things will end up as a certain concrete technology - but stop thinking concretely when designing the system. Try to get down to the actual business needs, model it and let the concrete technology fall into place later. With this approach, you gain another useful possibility; doing top-down development. Start with the user interface, move your way down. Keep the feedback loop as tight as possible with the business. Don't do more than is needed. This approach is something I know I will be missing the most in future projects. A tight feedback-loop is were the gold is hidden.

Were did we screw up?

This project must come across as a fairly peachy story. And sure, it was by far in my experience the project with the best code-base, the most structured one, the one that I personally learned the most from and also the one project in my career that we really managed to be on schedule and in fact for a couple of the releases we delivered more business value than was asked for. But it came at a price. One of the things we struggled with early on was to spread the knowledge across to the entire team and get everyone excited about the architecture, the new way of working with things and so forth. Personally I didn't realize how invested people were in their existing solution, and also in the existing way of doing things. Me as the architect, should have seen this before we got started. The problem with not realizing this ended up being a growing problem in the group. You had a divide in the group of people buying into the entire story and those who didn't or didn't quite get the entire story. My theory is that we should have given the most invested members of the group a time for mourning. Get time to bury their friend through many years; the old project. We should have realized that we were in fact building for the future and would replace the existing solution at the beginning of the project and this should have been the official line. Instead it kind of organically became the official line. We did at the beginning do training in all the new techniques, and gave people time to learn. Basically didn't give them any tasks for a few weeks and just pointed them in the general direction of things they should look at. What I think we failed on was that we didn't point out that these things were not optional, these new ideas were in fact mandatory knowledge. We should have been much clearer about this and been vocal about the expectations. Another thing I think I would have done a bit different; involve more people in the framework part of things. With the risk of stepping on toes, I think it is not wrong of me to say that I was the framework guy. For the most part, I ended up working on the framework. Don't get me wrong, I love doing that kind of work - but I think the experience, the design decisions got lost in translation and not everyone in the group understood why things were done as they were.

Conclusion

The project and opportunity that was given to the team was awesome, I really appreciate the trust that was given to me for leading the way in this project. The pace we had, the stuff we did has so far amounted up to be the coolest project I've ever worked on - and I am happy to admit it; I miss the project. Hadn't it been for a great opportunity that was given to me, I would have loved to stay on further. We had ups and downs, as with any software project, but overall I am wildly impressed with our accomplishments as a team and also by the end result.

Ohh… By the way. The end result can be found here.

Azure WebSites - deployment tip

einar@dolittle.com (Einar Ingebrigtsen) — Sat, 09 Jun 2012 09:36:12 -0300

The latest preview of Azure looks really good, very impressed with what they've accomplished and Microsoft seem to be doing a great job with catching up with their competitors. Personally I'm using a few cloud offerings already, including Amazon for Virtual Machine handling - since their VMs have been persistent and Azures has not been up till this release. Also I'm using AppHarbor for the ease of deploying web sites, but am now in the process of considering Azure for this as well.

I signed up for all the preview features and got them one by one within hours of signing up, and I must admit I got really impressed with the ease of getting things up and running. It is totally comparable to both AppHarbor and Heroku. I did however run into one tiny little problem, not sure if this is something that is common or not. Inside the CSProj file for a Web project, it holds an import statement for importing the default WebApplication targets that will help build the Web site for ASP.net. In mine the line was :

It needed to point to the 32 bit build extension path

This might not be a problem one gets if using Visual Studio, but might be just related to the fact that I do most of my .net development in MonoDevelop. Anyways, doing that certainly made it all build and was running smoothly in a matter of seconds after that.

If you haven't had the time to get started with Azures new WebSite feature, Maarten Balliauw has a great post for setting it up - although, the UI on the new Azure portal is just really intuitive.

Time to explore Azure a bit more - looking good so far!

Philosophy of Bifrost

einar@dolittle.com (Einar Ingebrigtsen) — Tue, 05 Jun 2012 15:58:49 -0300

Back in 2008 I started as a consultant after having worked at different ISVs since I started my career back in 1994. In the beginning my employer back then sent me to short contracts to get the consultant life under my skin. Moving around from client to client like that I realized something; I am rinsing and repeating a lot of mundane tasks, things I quickly realized that I really didn't want to be repeating. This is probably a realization most consultants do, but nevertheless I felt I had to do something about it; out came the idea of Bifrost, an open-source library that I would be able to reuse at clients if the client permits.

The philosophy
At the core of Bifrost and its philosophy lies an theory that although different domains have in general different needs, the abstract concepts that sits as pillars supporting a domain are the same. Bifrost would therefor be the provider of these abstractions. The abstractions should be very lightweight and focused and just support the concepts Bifrost is promoting.

The things that Bifrost aims at doing is to make things simpler, both on the backend as well as the frontend. There are so many things out there that we're repeating, Bifrost aims to either take away tasks or expose APIs that will make it a lot easier to accomplish things. I will not go into details about all the aspects of Bifrost in this post, as we're on a constant move and have incorporated quite a lot just the last 6 months.

With that being said, you're probably already thinking; geez that must a bloated framework. No! It is not. The reason for it not being bloated in my opinion, is just because of the fact that the different APIs are really focused and are not generalized to support all scenarios out there. Bifrost is opinionated, and will remain so. It is not a one size fits all necessarily. If you want to apply it, you will have to adjust to the philosophy behind it. This does not mean we're not open for suggestions, improvements and so forth. But it means we're keeping an eye on the road and we want it not blow up.

One aspect that was really important is to follow good development practices. Creating highly flexible, maintainable code and also highly testable to achieve the best possible quality.

The Evolution
Late 2010, when working for Komplett ASA we revitalized the project as a joint venture between Komplett and my own company; DoLittle Studios. Re-focusing some of the effort and changing around some of the core principles applied to it. For one we wanted it to be more focused around separation and more specifically implement and support CQRS as the preferred backend solution. We had already done an internal project using Commands to express behaviors in the system, but didn't do the entire CQRS stack at that point, but rather had the commands chained up and replayed whenever we wanted to achieve a state, leaving events out of the equation at that point. What was great about that is that we got a chance to dive into the concept, get our hands dirty without applying the entire stack; get some experience, basically.

CQRS
Command Query Responsibility Segregation Coined by Greg Young a few years back was something we saw quite a lot of benefits in applying. Although, we now see a bunch of different benefits from doing CQRS; the road leading up to CQRS and what followed, the product we ended up with forced us to learn so much. Everything became much clearer when it comes to identifying the different concerns and responsibilities in the code. The basics of CQRS states that you keep your read side optimized for that purpose only, and the execution is behavioral in nature and expresses in a verbose and explicit way what is to happen to the system, the read side will then be flattened or specialized as a consequence of whatever behavior has been applied.

Once we had it applied we started realizing the power of the concepts that we applied. We started seeing that applications are not about data, but rather have a very rich domain that expresses the behavior and that data might not even exist at all, but it might be as crazy as statically generated HTML files for our Web views, or at least statically generated JSON files we could pull directly from a CDN into our JavaScript code. It basically provided us with the scalability, flexibility and fueled us as developer with a set of mindsets that are really powerful.

MVVM
Back when I originally started the project, I always kept a very open eye on bridging the gap between the backend and the frontend. A pattern I've been loving for a few years now is Model View ViewModel. Modern Web applications are doing more and more on the client using JavaScript. Combine that with the growing popularity of single page web applications, MVVM seems to be a perfect fit. Bifrost has been built on top of Knockout JS, extending and formalizing a few things.

Much much more..
There is quite a few more things related to Bifrost. But I'm not going to take on the task of going through it all in this post. But on the official site. The site is really a work in progress some of the elements of Bifrost are documented, but for the most part not at this moment in time. Stay tuned and we'll get more of documentation up and running. We're also focusing on an API documentation that goes into detail.

Our conclusion

Although we jump-started the framework again and wanted to focus on the CQRS parts, we quickly realized that Bifrost was not just a CQRS library, it was something else. Its place in life is to facilitate any line-of-business application development. We see great opportunities to simplify a lot of the everyday developers life and is also something we would love to hear from you about. Don't hesitate to engage in a conversation with us at our GitHub site or our Google Forum.

Webinar for Typemock on BDD

einar@dolittle.com (Einar Ingebrigtsen) — Mon, 23 Apr 2012 12:58:03 -0300

I'm doing a live webinar on BDD and how it fits in with your general application development and can help you focus on getting your business needs covered and increase your general software quality.

The webinar is brought to you by

For more details and to register for the webinar, please go here.

Update: Video and slides has now been released here

Forseti - JavaScript Test/Spec runner

einar@dolittle.com (Einar Ingebrigtsen) — Sun, 18 Mar 2012 15:12:37 -0300

A couple of months ago, Pavneet Singh Saund and me decided to go down the road of creating our own test/spec runner for JavaScript. The rationale behind creating our own was basically that we had been having trouble getting existing solutions working, and the feedback loop also being higher than one should expect. Also, we felt that running all tests / specs in a through real browsers was more of integration rather than doing it every time we need to run.

Another aspect was that we had this idea of creating a better developer experience for these kind of runners. Integrate better with IDEs and also have a tool that does not sit in the terminal window. Even though all the cool kids are having their terminal windows real close, we still feel there is room for abstracting that away and create a very slick and simple UI sitting on top.

The runner we came up with is called Forseti, it is open sourced and can be found on GitHub here. The first release we have out at this point is a simple console app, but we will be moving forward on it as much as we can and hopefully bring all the goodness to it that we have in mind as soon as possible.

In the meantime, we'd love your input on it, anything from getting it up and running as is now, to building it, feature requests and bugs you might find. Please let us know what you think and how you experience it!

Embracing change

einar@dolittle.com (Einar Ingebrigtsen) — Sun, 18 Mar 2012 10:20:34 -0300

As I wrote in this post, back in September 2008, I used to be a Windows guy. In fact I was borderline fanboy of Windows. But decided that I wanted to try out this Mac thingy that everyone was talking about and all the alpha-geeks were using.

Now, some 3 and a half years later, I'm still on the Mac - in fact, It turns out that what I mentioned in that post; of minimizing the need for Windows has been a true goal of mine and something that I've accomplished quite good. Some would probably call me a Apple fanboy now, with the amount of Apple devices I've acquired since my small start in 2008. I can't blame them. But I'm not preaching it!

Ever since I started that branch of my life, I must say that the operating system has become less and less important, although I now have my preference going the general direction of away from Windows. But I think doing a switch like this has had a very positive effect on a number of other things, especially my perspective on software development. Around the areas of how I work, what I think is important when writing software and such. One could of course argue that I would broaden my mind naturally, without going to the extreme of doing my work on a different platform. But I beg to differ. The reason for that is that I think, no matter what platform you're on, that your mindset is colored by the culture on the platform, and if that platform is a commercial one driven by a company, you're also likely to be colored by the values of that particular company.

I think by constantly seeking out new things, changing platforms every now and then, or just trying out things in general, will make you a better and more reflected person. There are so many things one is missing out on if you believe in one truth, and from my experience, there is also a lot of truths that people will tell you about the competitor - from people belonging to the community surrounding a platform, this applies to most platforms.

To me, it has been an awesome journey switching - I've learnt so much from just doing that. It has opened my mind to things like Ruby, and crazy ass rocket science things like HTML, CSS and JavaScript (Sarcasm applied). I've grown in productivity at the expense of simplifying my tools - no longer need Visual Studio, nor Resharper to accomplish being productive in .net. I demand more of myself, more knowledge, more precision, higher velocity - as a result.

So, if you're sitting there - stuck on a platform and haven't really tried out anything else. Go crazy, try out something else, go to a user group - if available, representing that parallel universe. Chances are you might learn something, expand your horizon and become a better you. Go and embrace change - trust me, you'll hate me the first weeks but thank me in the long run.. :)

Slides + URLs from NNUG talk 29th of February 2012

einar@dolittle.com (Einar Ingebrigtsen) — Wed, 29 Feb 2012 23:43:17 -0300

Yesterday, Pavneet Singh Saund and I held a talk for NNUG Vestfold about modern Web development using the MVVM pattern and compositional UIs.

The slides can be found here.

In addition we talked about a few things, these are the resources of the talk :

KnockoutJS
Bifrost
Forseti

I did mention and tried to demo another site I'm working on, which was down at the time - but up now :

Geekrider - it shows navigation of features, using Bifrost.

and lastly, the actual demo app we did can be found here. We did also show the inkFinder that was developed at Komplett that uses Knockout and MVVM, which you can find here.

Removed ratings

einar@dolittle.com (Einar Ingebrigtsen) — Sun, 19 Feb 2012 04:13:01 -0300

Recently I've had a couple of strange ratings on posts here. People, in plural, rating simple posts that basically just announce a talk to a 1 out of 5 and not leaving a comment. That just messes with my head. There is no way I can improve posts when there is just a bunch of negative rating but no comments. As a consequence, I decided to remove ratings all together - hopefully forcing people to engage in a conversation instead.

If there is something you don't like in a post, or the entirety of a post - please don't hesitate to post a comment. I'm very open to listening to feedback, but I can't just read into the lack of a star and figure out what I'm doing wrong.

NNUG Vestfold talk, 29th of February 2012

einar@dolittle.com (Einar Ingebrigtsen) — Sat, 18 Feb 2012 02:15:05 -0300

On the 29th of February, Pavneet Singh Saund and myself will be holding a talk about modern web development for NNUG Vestfold.

For years, for a lot of developers, the Web has been a mess and JavaScript has not made that mess any better. There has been a lot of improvement over the years and a lot of efforts in bringing structure as well as promoting JavaScript into a first class citizen of your every day application development. The topic of the talk will be around this, how can you as a developer work with JavaScript and stop having the feeling that it is just something you have to do, but don't like to do.

Here's the agenda.

Don't hesitate to register, this will be great fun!