Colin Jack's Blog

SOA Manifesto

2009-10-26T22:45:00.002+00:00

I've just gotten through reading the SOA Manifesto and reactions from (among others) Jim Webber and Stefan Tilkov and I can't help feeling the whole thing is a missed opportunity.

In particular you'll notice is that the manifesto doesn't deal with the "what is a service" issue and so fails to deal with the Service Oriented Ambiguity issue. Although Martin Fowler covers multiple ways of viewing services in his post I think two are probably most relevant when considering the manifesto:

Business Level SOA (B-SOA) - Services for business capabilities/functions (think Strategic Design in DDD, Enterprise SOA). This approach focuses on bringing IT and the business together and in my view should lead to large and loosely coupled autonomous services.
Technical Level SOA (T-SOA) - Services here are the individual endpoints. These end-points could be Web Services (RPC/WS-*), they could be resources (REST), they could be messaging endpoints, or they could be something else.Each B-SOA service exposes one or more T-SOA services.

Stefan Tilkov has a good blog entry about the two which is worth a read, and with these two types of SOA in mind its worth looking at the values in the manifesto to see how each apply in each context (context is king).

Values In Context

Whether considering T-SOA or B-SOA I agree with the first and last values, favouring business value and evolutionary refinement are difficult to argue with. Intrinsic interoperability also makes sense in either case, especially as it is so vague, which leaves us with three other values:

Strategic goals over project-specific benefits - At the B-SOA level this is certainly true but at the T-SOA level its a tradeoff as Stefan points out.
Shared services over specific-purpose implementations - From a T-SOA angle this could be taken to refer to the argument that you should design your endpoints to be reusable in different contexts, such as when you go for an entity service based approach. However there are arguments against too much premature focus on reuse and use is far more important than reuse. I also think that reuse becomes less key if you view each T-SOA service as being within the context of a B-SOA service. In addition I'm not really sure what this value means when considering the B-SOA services themselves, probably not much.
Flexibility over optimization - This one is very vague. You could read it about designing your T-SOA services to handle real variations in behaviour/data (patterns like Workflodize come to mind) and if so it is obviously entirely valid. However I think this is coming back to reuse again, specifically designing each T-SOA service to support reuse by making them more general purpose/agnostic. In addition at the T-SOA level optimisation does have to be considered because it is at this level that QOS becomes a consideration, but in many cases considering optimizations of individual T-SOA services won't involved sacrificing flexibility. So I'm not sure how to take this one, in addition I'm unclear as to what this statement means in terms of B-SOA, again probably not much.

So although I can nod my head to most of the points in the manifesto someone with a completely different views on SOA could do exactly the same, and even if we did have the same views we'd need to be clear which type of SOA we were considering when thinking about the manifesto.

That's is a pity and I can't help feeling that maybe Martin Fowler had it right in 2005:

So what do we do? For a start we have to remember all the time about how many different (and mostly incompatible) ideas fall under the SOA camp. These do need to be properly described (and named) independently of SOA. I think SOA has turned into a semantics-free concept that can join 'components' and 'architecture'. It's beyond saving - so the concrete ideas that do have some substance need to get an independent life.

With this in mind I actually think that Steve Jones could be right, the term SOA is overloaded and too often used just for T-SOA so maybe its time to separate out the notion of B-SOA completely by using a term like Business Service Architecture.

RESTful Web Services Cookbook - Review of Rough Cut

2009-09-22T22:05:00.001+01:00

I've just finished reading the rough cut of RESTful Web Services Cookbook by Subbu Allamaraju and Mike Amundsen and thought I'd write a very quick review.

Ultimately the book fills a gap in the market by providing practical advice on designing high quality HTTP/REST solutions and I found the recipe approach really accessible.

Its also an easy read so if you've been toying with REST but haven't found it very easy to get your head around then I'd definitely recommend having a look. I know that when I started learning REST I wasted large amounts of time reading unhelpful and downright misleading content so its good to have a book that should help you avoid falling into any obvious traps.

In summary, highly recommended.

WCF and REST

2009-09-12T12:53:00.002+01:00

I'm on a project that was using WCF and we've managed to make the transition to using REST (well, to be honest so far its just POX).

REST is a joy and I really find it such a pleasant experience compared to RPC let alone WS-*.

However WCF and REST just are not good bed fellows. Some of the problems are that it doesn't support linking out of the box, content negotiation isn't there, no support for common HTTP/REST patterns, it limits the design/granularity of your resource handlers (think MVC controllers), and the REST starter kit (at least when I looked at it) was appalling.

There are also lots of annoyances that you only discover when you come to use it and unfortunately, as Seb discovered when he developed our REST framework on top of WCF, WCF lacks some key extension points meaning you end up having an extremely painful time getting the functionally you need.

The point of this post isn't to beat up on WCF, its just to make you aware of the issues. If all you want to be able to do is a bit of simple CRUD then you might find WCF to be a good solution, however just be aware that it isn't designed to scale out to handle more interesting problems or more RESTful solutions.

On the bright side the WCF team did contact me and Seb and we did give feedback which apparently will feed into a release post-WCF 4. My feeling is that if they don't make massive improvements then WCF will lose out on the REST battle, especially as there are some great REST frameworks appearing in .NET-land.

E-VAN Blog and Videos

2009-07-11T22:13:00.001+01:00

Jan has already posted about it but we have a new E-VAN blog which includes links to the recording of the excellent presentation Alan Dean gave on REST and a link to the next one that Ian Robinson and Jim Webber are giving (please submit questions/topics for this one, it's going to be superb).

E-VAN - Alan Dean talking on REST (6th July 2009)

2009-06-24T21:10:00.003+01:00

First off if you didn't watch Udi Dahan talk on the SOA then you missed a treat, a real pro speaking on a topic that he and a very few others in the .NET space are way ahead of the rest of us on. Brilliant stuff and the video is now online at Vimeo.

Next E-VAN

We've also now planned the next couple of E-VANs which will focus on REST, the first will be with Alan Dean where he will cover facts and fallacies related to REST.

Alan is a real authority on REST and he's got a great talk ready so this is a great chance to find out more about REST. In addition if you have any particular questions for Alan, or topics you'd like to discuss in detail, then you can add them to the existing topic on the google group.

Start Time on Monday July 06 (7:00PM GMT)

In France/Germany/Belgium: 9:00PM
In UK is: 8:00PM
EST in the US is: 3:00PM (EST)
PST in the US is: 12:00PM (Midday)

The session will last around 90 minutes and on timings see this world clock site or more information.

Attendee URL

http://snipr.com/virtualaltnet (Live Meeting)

Future E-VANs

Stefan Tilkov has kindly agreed to do another E-VAN one on REST at the start of August. In addition we are going to see if we can organize another one for mid-July, creating a bit of a REST theme for the next month or so which we hope you'll enjoy.

We've also got a couple of people in mind that we want to get on looking further into August and beyond, more to come on that once myself and Jan contact the people :P

Feedback - Tell us how to do better

So far I think the E-VAN is going well, great people talking about interesting topics. Live Meeting has been a bit flaky and time zones are a pain but other than that I'm pretty happy with it.

However we want to know where you guys think things can improve so if you get a chance we'd be interested in hearing your feedback using the new google group. I'm interested to see if we can get more open discussions going and to explore whether an online book club is a good idea...

Book Review - SOA Patterns (First 5 Chapters)

2009-05-24T13:29:00.001+01:00

I'll cut straight to the chase, I definitely recommend SOA Patterns to anyone interested in SOA.

Even though its currently only available in MEAP, and is a bit rough and ready, and isn't complete, it's still great stuff and well worth your time.

Now I've said that I'll give a quick run through. Essentially the author, Arnon Rotem-Gal-Oz, has decided to focus on the practical aspects of SOA and in particular with a focus on the software architecture aspects of SOA. It thus doesn't cover the business aspects of SOA, which is fine with me as those topics are covered elsewhere.

With that tight focus in mind Arnon has already managed to pack in a range of very interesting and valuable patterns. Each pattern includes the problem, solution, technology mapping and quality attributes. There is already the feeling of a pattern language about it which I think will evolve as later chapters are added, this should allow you to describe a complex system with just a few high level patterns or to dig into more detail by discussing the lower level patterns they compose.

You can actually read a few of the patterns online, see the linked PDFs and also the Blogjecting Watchdog pattern, and the anti-pattern called The Knot is also worth a read.

In addition Jan has posted a review of the book.

Next Europe VAN - 1st June 2009 - Questions for Udi Dahan

2009-05-17T20:06:00.001+01:00

Udi Dahan has been kind enough to agree to do the next Europe VAN and suggested that we all put forward our questions for him to answer.

Its a great chance to ask him about DDD/SOA/CQS/messaging/NServiceBus, or any other topic, so if you have any questions post them here or on the associated thread on the DDD/ALT.NET forums.

Jan and I will aggregate the questions on the 25th May (after adding our own!) and then submit them to Udi so make sure you've submitted yours before then. Just to confirm, you need to get your questions to us no later than Monday the 25th May or we'll ignore them.

Details

Here's the details of the live meeting:

Start Time: Monday, June 01, 2009 07:00 PM GMT

End Time: Monday, June 01, 2009 08:30 PM GMT

Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Calendar

This meeting should also be in the VAN Calendar, this allows Google calendar users to get a reminder before the big event.

Book Review - RESTful .NET

2009-05-10T21:53:00.001+01:00

Since we are using REST on top of WCF on my current project I was glad to see that this book had been written as I was hoping it would answer some questions I had.

However my first issue with this book was the title. As I say we're using WCF but WCF isn't the only way to implement REST in .NET, however the book does not take the time to evaluate alternatives (including building RESTful applications on top of ASP.NET MVC) so RESTful WCF would have been a more accurate title.

In fact if we wanted to truly represent the books focus I think we would call it WCF REST because the primary focus is on WCF. There was a lot of WCF detail in here, starting from chapter 2, and personally I found some if quite boring especially where the author has chosen to put in property listings of some key WCF classes. I just don't find that sort of content all that useful and I'd have expected a lot of it to be in an appendix at best.

However the WCF focus has some other implications. Take HATEOS which is an important part of REST, my company has been lucky enough to have Sebastien Lambla and Alan Dean in our office and both emphasized its importance. However there is almost no discussion of it in the book, and the reason for this becomes clear on page 246 where the author states that although hypermedia is important in REST it isn't covered in the book because WCF has poor support for it. As I say we're using WCF so I knew that WCF's support for links in normal representations was non-existent, but I do consider it a flaw of the book and in my view the author would have been better stating this big problem up-front or tried to add framework to make linking a lot easier (something Seb did for us).

Strangely the author actually states that he actually avoids discussing custom infrastructure code because it takes away from learning about the technology (page 97). Personally I would prefer the book if had a focus on REST and how to get the advantages of REST using WCF, low level WCF plumbing/architecture is a necessary evil not something I want to read much about and more importantly if we need extra framework then I'd like to see it discussed.

I also thought it was doubly odd that chapter 9 discussed "Using Workflow to Deliver REST Services", all centred around Windows Workflow, when to me when I see the words workflow in a REST book I'm again thinking of HATEOS (the RESTbucks example at InfoQ shows this approach brilliantly and will form part of a rival book).

Before wrapping up I also wanted to identify one bit of the book where the advice is very questionable. Chapter 10 is called "Consuming RESTful XML Services Using WCF" and one approach described is to take the interface attributed with ServiceContract and use it client side. Just to be clear we're talking about making class like channel.CreateAuthority(authority) and then letting WCF work out what HTTP request to make. You won't want to be doing that.

Anyway I personally wouldn't recommend this to anyone wanting to learn about REST, instead I'd recommend (for now) RESTful Web Services. If however you are using WCF and REST, and feel you already understand REST, then this book will give you some insight into support for REST in WCF.

E-VAN Videos Reminder

2009-05-06T19:49:00.001+01:00

Just to remind people that Jan has put the videos for the first two E-VAN presentations online:

http://www.vimeo.com/user1286822

Mark Nijhof's excellent presentation on FubuMVC will be up as soon as its available and we'll announce the next presentation as soon as we organize it :P

Next Europe VAN - 25th February 2009

2009-02-15T17:44:00.001+00:00

After a bit of discussion Jan and I have decided that for now it seems to make sense to run the Europe VAN every two weeks, we have discussed having more open discussions on alternate weeks but we'll have to see if there is an appetite for that.

Seb has agreed to do the next one on REST and his OpenRasta framework.

Start Time: Wednesday Feb 25, 2009 7:00 PM GMT

End Time: Wednesday, Feb 25, 2009 09:00 PM GMT

Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Europe VAN - DDD with Greg Young (9th Feb)

2009-02-09T19:03:00.003+00:00

The delayed first E-VAN is on now with Greg Young, discussing DDD and CQS. The URL is: http://snipr.com/virtualaltnet (Live Meeting).

NOTE: This one was a success and Greg did a superb job explaining DDD and messaging, really enjoyable stuff.

P&P Guidance - Positive Progress

2009-02-08T20:39:00.003+00:00

I put up a post a while back about the P&P REST guidance which I considered to be quite flawed. The reaction from the P&P team was superb and Greg Young, Sebastien Lambla and myself immediately got involved in an e-mail exchange with them regarding this and other guidance.

They've shown they are incredibly open to feedback and we're still involved in this exchange but it looks like the result may be changes to the existing article and/or new articles.

One thing I hadn't realized before this started is that P&P actually have three different sites:

Guide – http://www.codeplex.com/AppArchGuide
Knowledge Base – http://www.codeplex.com/AppArch
Community Site – http://www.codeplex.com/AppArchContrib

The community site is quite open so even if our examples use open source frameworks they can still be published on it. I'm therefore planning on writing one example using WCF, NHibernate and Castle where the two primary areas covered will be:

REST - Although the current example says its using REST I think this is arguable.
Domain Model - I'll be using DDD patterns but I've tried to explain that DDD isn't always appropriate (e.g. if you don't have access to domain experts). Regardless I'm obviously a big fan of object-oriented domain models and the barrier to entry is so low with modern ORM's that I think its a no-brainer to use one in the example.

I'm not saying I've chosen the best technologies, in particular using WCF is highly questionable, but since I'm not trying to say this is the "best" way to do things I'm comfortable with the choices especially as I'd expect other guidance to be given over other approaches (perhaps using an MVC style approach).

First Europe VAN

2009-02-02T19:58:00.001+00:00

The first Europe VAN went off without a hitch and was an amazing success. Ha, yeah sorry about that.

Good news is next one is BOUND to be better and we are trying again very soon. Next time we'll come armed with a backup plan too, possibly involving one us loading up the Dark Knight DVD so we can all watch it for an hour or two.

First Europe VAN

2009-02-01T10:10:00.001+00:00

Just a reminder that the first Europe VAN is tomorrow:

Start Time: Monday, Feb 02, 2009 7:00 PM GMT

End Time: Monday, Feb 02, 2009 09:00 PM GMT

Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

Greg Young has been kind enough to agree to speak and I'm hopeful it'll go well as we've had a couple of dry runs so it's looking good, even if I was perturbed to found out Jan speaks better English than me.

P&P - Web Service with REST Application Pattern

2009-01-22T20:09:00.001+00:00

It was interesting that all the fuss over Oxite had a positive impact. Unfortunately Microsoft are also now giving us their views on REST (WCF REST starter kit, argh) and more recently REST with a domain model in Two-Tier Service Application Scenario (REST) which J.D. Meier calls the Web Service with REST Application Pattern.

If you understand REST/DDD, or like Fowlers original meanings for his patterns, or believe that "Business Process Objects" with methods like "Promote (object[] data)" could be improved on then you might want to look away now. Come on though I've piqued your interest, have a look, it makes you feel dirty but yet at the same time makes you glad you now realize that perhaps there are other ways to build applications.

Might be interesting to see how this relates to the old design documents MS used to pass out, pretty sure its not that different except resource is now used in some places instead of service. Service + procedural code + database, rock on.

Anyway I'm going to go back to writing my own comments on it to put on the page, its going to take me a while...

SOA Principles of Service Design

2008-12-28T15:31:00.001+00:00

I've been learning a lot more about SOA and REST recently and so have been doing a lot of reading including SOA Principles of Service Design and I thought I should write a short review.

Short version, this is one of the worst technical books I've read and I wouldn't recommend it to anyone. My main gripes are:

Style - Although there are lots of colour diagrams they don't tell you much and don't take away from the fact that the text is dry, repetitive and at times very boring. Since I've read one of Erl's previous books, well read about half of it because I just couldn't face the rest, I should have known what to expect but it was still a big big struggle to finish it.
Ideas - Reusable entity services (Customer/Order) shared throughout the enterprise might sound sensible but I'm not at all convinced by that approach and the focus on it has always blunted my interest in SOA. Unfortunately that style of layered SOA is key to this book so I found a lot of it pretty un-inspiring.
Single Viewpoint - There is more to SOA than RPC services but this book doesn't really talk about anything else. REST gets no coverage (as an alternative to aspects of SOA), EDM/messaging doesn't really come into it, business focused SOA (analysis/design with business experts) is hardly discussed...it just all feels too narrowly focused on the one approach the author uses and so there is little discussion of alternatives and tradeoffs.
Book Assembly Line Approach - Having one book on SOA principles and another on the patterns, with references from the principles book to the patterns one, seems overkill. I know SOA is a big topic but do we really need an eight book SOA series? In my view you could easily have cut 30% of the content in this book (including most of the diagrams) and merged it with the patterns book.

Some might take this as me saying I don't rate SOA but that is not true at all, I just don't rate this book and don't think the ideas behind it are what I'm interested in. However I still think SOA has a lot of value and in fact the next two books I plan to read/finish are both about SOA:

SOA Patterns - Not the SOA patterns book by Erl (which I'll admit from it's site looks interesting), this one is by Arnon and from the Manning EAP version I've read it's going to be superb.
Enterprise SOA Adoption Strategies - I learned about this book because I enjoyed reading Steve Jones' posts on the SOA forum and it seems to focus on the business oriented side of SOA, something which doesn't seem to get discussed nearly enough.

Since I'm far more interested in business focused services each wrapping one or more domain models and exposing multiple end-points, instead of entity services first approach, I'm looking forward to reading these two as from the looks of it they'll give me lots of new ideas.

BDD - Available Frameworks

2008-11-19T21:16:00.004+00:00

I've been using the Astels style of BDD for a while now but so far I've just done it using MSTest/NUnit and a few custom base classes. I think that's a good way to start out, as with so many good things in life it doesn't require a new whizzy tool/framework.

However I've just joined a new project and we've been looking at different frameworks that are available for unit testing and for BDD so I thought I'd post about what I've seen so far. Any opinions would be gladly received.

MSpec
This project is very cool, not only does it allow you to create superb specifications but you also get some nice reporting. Here's a simple example of a sample spec:


public class When_adding_a_contact_to_a_user_with_no_existing_contacts
{
  private static User _user;
  private static Contact _contact;

  Establish context_once =()=>
  {
      _user = new TestUserBuilder().Build();
      _contact = new ContactBuilder().Build();
  };

  Because a_context_is_added =()=>
      _user.Contacts.Add(_contact);

  private It should_associate_the_contact_with_the_user = () =>
      _user.Contacts.Contains(_contact).ShouldBeTrue();
                                                           
}

One thing to note is if your planning to look at MSpec then you'll probably want to download the Machine codebase since there aren't many examples of using MSpec on the Web the examples with Machine are a good starting point.

So far there's no R# integration but that doesn't worry me at all as if needed it will come and this is still a very early version.

The reporting seems to work well, but we primarily use this style for unit/integration tests and so we are unlikely to present the reports outside the development team. Having said that Aaron pointed out that they can be useful within the development team, which makes a lot of sense.

Overall my main worry is the syntax could be a bit much for some people, in particular if you go for the compact style:

NUnit
I think there's a good argument for just using a base class, especially when you are getting going with the approach:

public class When_adding_a_contact_to_a_user_with_no_existing_contacts : SpecificationBaseNUnit
{
  private User _user;
  private Contact _contact;

  protected override void EstablishContext()
  {
      _user = new TestUserBuilder().Build();
      _contact = new ContactBuilder().Build();
  }

  protected override void Act()
  {
      _user.Contacts.Add(_contact);
  }

  [Test]
  public void should_associate_the_contact_with_the_user()
  {
      _user.Contacts.Contains(_contact).ShouldBeTrue();
  }
}

This is a hopelessly naive example but you get the idea. You lose some of the syntax niceness, suddenly the specs themselves take up multiple lines because of all the curlies. You've also lost reporting, unless you put in some work yourself. However it is a little easier to understand and when introducing TDD/BDD that could be important.

XUnit.net
I'm no XUnit.net expert but Ben Hall convinced us to give it a shot by recommending it and it is very nice. You can read about an approach that works here. If you use the specification base class described in that post you might end up with this:

// Using base class influenced by http://www.bjoernrochel.de/2008/10/04/introducing-xunitbddextensions/
public class When_adding_a_contact_to_a_user_with_no_existing_contacts : SpecificationBase
{
  private User _user;
  private Contact _contact;

  protected override void EstablishContext()
  {
      _user = new TestUserBuilder().Build();
      _contact = new ContactBuilder().Build();
  }

  protected override void Because()
  {
      _user.Contacts.Add(_contact);
  }

  [Observation]
  public void should_associate_the_contact_with_the_user()
  {
      _user.Contacts.Contains(_contact).ShouldBeTrue();
  }
}

One aspect of XUnit that might throw you is how opinionated it is, which could be an advantage or a disadvantage. An example is that it's aiming for each test to run in isolation, so the fixture class is re-created each time and if you really want to reuse the fixture you implement IUseFixture. I guess this is a very safe approach because it means tests/specs are extremely unlikely to affect each other, but it actually seems over-kill if you're using a style where the specification methods only assert (no side-effects).

The lack of messages on assertions seems sensible, and it is for small focused BDD specifications, but if you use it for integration testing you would want the option of adding a message in.

On the syntax front we could always go for more flexibility:

public class When_a_user_has_no_contacts : FlexibileGrammarSpecificationBase
{
  private User _user;
  private Contact _contact;

  protected override void EstablishContext()
  {
      _user = new TestUserBuilder().Build();
      _contact = new ContactBuilder().Build();
  }

  [Because]
  protected void and_we_give_them_a_new_contact()
  {
      _user.Contacts.Add(_contact);
  }

  [Observation]
  public void the_contact_should_be_associated_with_the_user()
  {
      _user.Contacts.Contains(_contact).ShouldBeTrue();
  }
}

However this seems a little pointless to me so I've dumped the idea.

Summary
We decided to go with XUnit.net but we also plan to look at Ruby based solutions including RSpec and Cucumber. Cucumber seems exciting as it lets you specify table based specifications, theoretically allowing us to get the advantages of a FIT style approach without having to use FIT (or SLIM) itself.

Ultimately there is a lot going on in the BDD space in Ruby-land (and a book on the way) and the language does suit it quite well so we intend to do some playing.

BDD and Parameterized testing

2008-11-19T20:58:00.004+00:00

Although I really like Astels style BDD (to me a constrained/enhanced style of TDD) I still use a lot of parameterized testing and though I should give you an example why, using XUnit.net.

Lets say we're testing simple SPECIFICATION style rules, we might write:

[Concerning(typeof(ValidEmailRule<TestEntity>))]
public class When_using_rule_on_a_null_string : SpecificationBase
{
  protected TestEntity _testEntity;
  private bool _isSatisfied;

  protected override void EstablishContext()
  {
      _testEntity = ContextSetup.CreateTestEntityWithValue(null);
  }

  protected override void Act()
  {
      _isSatisfied = new ValidEmailRule<TestEntity>(_testEntity, x => x.Value).IsSatisfied();
  }

  [Observation]
  public void is_satisfied()
  {
      _isSatisfied.ShouldBeTrue();
  }
}

This just tests how the rule handles a null value, but we'd then want to test with all sorts of other values (valid and invalid). To compare lets thus look at how easy it is to test a variety of invalid e-mail address using one of XUnit.net's parameterized testing approaches (see Ben Hall for more options):

[Concerning(typeof(ValidEmailRule<TestEntity>))]
public class When_evaluating_invalid_email_addresses
{
  [Theory]
  [InlineData("sddas.com")]
  [InlineData("sddas@")]
  [InlineData("@")]
  [InlineData("@blah.com")]
  [InlineData("sddas@@blah.com")]
  [InlineData("1213231")]
  public void is_not_satisfied(string invalidEmailAddress)
  {
      var testEntity = ContextSetup.CreateTestEntityWithValue(invalidEmailAddress);

      var isSatisfied = new ValidEmailRule<TestEntity>(testEntity, x => x.Value).IsSatisfied();

      isSatisfied.ShouldBeFalse();
  }
}

Now you may disagree with my approach here, this isn't as readable as it could be, but I think you can see why you'd use this approach if you have a lot of values to validate.

DDD - Making meaningful relationships

2008-10-28T22:43:00.001+00:00

A recent discussion on the DDD forum made me want to post about what I consider to be an under-appreciated aspect of domain modelling, namely relationships. In the thread Randy Stafford said the following:

Note that RoleRegistration is an example of a Relationship Object – arguably a fourth archetype of domain object alongside Entity, Value Object, and Aggregate.

I couldn't agree more with this, in a domain model I worked on recently we had a stack of very useful and very meaningful relationship objects that served a range of purposes.

For example if a Client has Accounts then it is possible that you can get away with just having that as a direct relationship but its equally possible that the relationship itself is meaningful and carries its own data/behavior. In this case you might have a type association with the relationship that would explain if the Client owns the Account, or whethery just manage it, or whether they are actually just one of several owners.

Aggregates

You need to consider aggregate boundaries especially carefully when using Relationship Objects.

In the case of an association between a Client and an Account the relationship probably belongs to the Client aggregate.

Then again if you choose to model the association between a Client and SalesAdvisor using a full blown party/role/relationship approach then things become a big more complex. Are all parties and roles and relationships different aggregates or does a relationship own the two roles it composes?

If its the latter then you may be breaking an aggregate design rule because the party now refers to a piece of the relationship aggregate other than root.

Temporal Associations

Another common case is that the relationship is temporal which brings with it a lot of complexity and should only be done with extreme care. If your sure you need temporal associations then you will find Martin Fowlers patterns invaluable.

Encapsulating Relationships

Most of the relationships have real meaning in their own right but sometimes they are just an artifact of the design, in those cases you can demote the associations to being just an encapsulated detail.

Take the association between Client and Account, maybe when you ask for the Accounts for a Client you want to get the Account objects rather than the ClientAccountRelationship objects.

If this were the case you could keep the ClientAccountRelationship class, which has its own data/behaviour and makes mapping easier, but entirely hide it from users of the domain. One way to do this is to create custom collection called ClientAccounts and have it wrap an IList<ClientAccountRelationship> whilst acting like it is just a simple IList<Account>, it can also provide helpful methods like FindPrimaryOwner.

Summary

I mention all of this because when I got started with DDD relationship objects bothered me especially as we were working with a legacy database and I saw the relationship objects as being a problem caused by the number of join tables. At the time my plan was to get rid of a lot of them by taking advantage of NHibernate magic.

However as I got a bit more experience I realized that they were key and although we encapsulated some (when they were just an artifact of the design) we made others totally key parts of the design. In both cases the relationships themselves were very useful.

Domain Driven Design - Knowledge Level

2008-10-28T22:18:00.002+00:00

A recent thread on the ALT.NET forum got me thinking about KNOWLEDGE LEVEL, a seldom discussed but useful pattern from domain driven design. As with many patterns near the end of the book its a pattern that isn't going to suit every situation but it is a pattern that I think has a lot of value. In summary we aim to explicitly split our model into:

Operations Level - The place that we do our day-to-day business. For example Shipment, Account.
Knowledge Level - Objects that describe/constrain the objects in the basic (operations) level. For example EmployeeType, RebatePaymentMethod, ShipmentMethod.

I'm not a big fan of just repeating what's already in books and to my mind if you want to understand DDD then I'd suggest you read the DDD bible. I'm thus not going to go over the pattern, but I did think it was worth a post about what I've found when coming to the detailed design and implementation of classes in a knowledge layer. I thought I'd start by discussing how you can persist/load/model objects in the KNOWLEDGE LEVEL, moving on to other topics in later posts. Two caveats:

Simple Example - Rather than using a real example I've chosen to use the shipping method example from the thread, I've never worked in that domain so this may be a bad idea but it felt right...
Might seem obvious - If you are an object/behaviour first type of developer then you'll be looking at using OO techniques and so some of this will just be waffle. However I've seen situations where people used data-driven techniques when approaching their KNOWLEDGE LEVEL and so I wanted to explain why I think it's a bad idea.

Option 1 - Data First Approach

Since we store our operational objects in the database why not do the same for our KNOWLEDGE LEVEL? If we go down this path then we'd have a ShippingMethod table with one row for each shipping method we support. Of course each shipping method has different characteristics, perhaps certain shipping methods are only available to long standing customers or customers in certain countries. To support this we'll end up with extra columns in the ShippingMethod table and maybe even associations to other tables. To get the ShippingMethods from the database we'd probably use a REPOSITORY, probably making sure that the REPOSITORY is read-only (maybe not always, see pattern for information on modifying the KNOWLEDGE LEVEL).

Disadvantages

This approach obviously has disadvantages:

Loss of clarity - Our ubiquitous language might refer to a "local carrier shipping method" but it won't be visible in the domain model and in fact to see details about this shipping method you need to look at the appropriate row in the ShippingMethod table. Even if you don't value the ubiquitous language this is a problem because as a developer you'll end up switching between the code and database in order to understand even simple aspects of the behaviour.
Goodbye OO, hello procedural - Since there is no LocalCarrierShippingMethod class I have nowhere to put its behaviour, instead I have to fall back on a procedural style of program where we load in the data about the ShippingMethod with an ID of 3 (which happens to be the local carrier shipping method) and then look at its SupportsInternationalDelivery boolean flag before deciding how to proceed (hopelessly naive example alert). We can encapsulate this logic in a service, perhaps a ShippingMethodSelectionService but its still more than a little smelly.
No explicit associations - Imagine if we know that the local carrier shipping method is only available for certain types of Orders and that the decision making also takes into account the Customer placing the Order. If we're using the database-driven approach then managing this becomes very difficult, we can't just look at our LocalCarrierShippingMethod class or at some code that sets up the associations. Instead we fall back and run a database query involving all sorts of joins.

Advantages

Those are, as I see it, the primary disadvantages of hiding this important information in the database. Ofcourse its not all bad:

Consistency - Our KNOWLEDGE LEVEL is handled in the same was the rest of the domain model, loaded from the database as required. Not sure its a massive advantage though because we've explicitly chosen break out the KNOWLEDGE LEVEL its fair enough to say that the consistency has limited value.
Flexibility - If we want to add a new kind of ShippingMethod we just add a row to the ShippingMethod table, no need to redeploy. That's the idea anyway, but its not always going to work especially when the related procedural code has to change.

Its also fair to say that the KNOWLEDGE LEVEL changes at a different pace to the rest of the model, we probably don't add ShippingMethods too often. However we might want to be able to change characteristics of existing ShipmentMethod's, such as changing their price (remember this probably only affects future Shipments) or changing what Countries they can be used in. So on the flexibility angle you've actually got a couple of types of changes:

Adding/removing concepts - Perhaps adding a donkey based shipment method, to me its safe to require a redeploy for this sort of change and in any case without some serious thought we're not necessarily going to be able to do just by modifying a table anyway.
Changing configuration/associations - Its maybe fair enough to expect to be able to ban using donkey based shipping for all future orders in Spain without requiring the code to be redeployed. As I'll show below this can work even if you go for an object-oriented approach and I also think this is a case where a DSL would really add value (not tried that though).

Those are the main advantages I've heard about, and as I say I'm not sold on them. That brings me on to how I think you should go about it...

Option 2 - Object Oriented Approach

Take that "local carrier shipping method" concept and turn it into a LocalCarrierShippingMethod class, probably inheriting from a ShippingMethod base class. There's only one instance of this class and since its read-only (at least as far as normal usage goes) it's totally safe to share it. The ShippingMethod has any data it needs to support the behaviour it contains and to support the interface that it exposes, so for example you might have an IsApplicableForSending(Shipment) with each subclass providing their own implementation.

Implementation

How does this look in practice, well one approach I've used is to a variant of the typesafe enum pattern. The following is just pseudo code to show the idea:

public abstract class ShipmentMethod
{
  private static Dictionary<ShipmentMethodKind, ShipmentMethod> _shippingMethods = new Dictionary<ShipmentMethodKind, ShipmentMethod>();

  static ShipmentMethod()
  {
      // NOTE: In practice you wouldn't do it like this but it is just an example....
      _shippingMethods.Add(ShipmentMethodKind.LocalCarrier, new LocalCarrierShippingMethod());
      _shippingMethods.Add(ShipmentMethodKind.DonkeyBased, new DonkeyBasedShippingMethod());
  }

  public IEnumerable<ShipmentMethod> GetAll()
  {
      return _shippingMethods.Values;
  }

  public ShipmentMethod GetByKey(ShipmentMethodKind key)
  {
      return _shippingMethods[key];
  }

  public abstract bool IsApplicableFor(Shipment toEvaluate);
}

public enum ShipmentMethodKind
{
  LocalCarrier = 0,
  DonkeyBased = 1
}

public class LocalCarrierShippingMethod : ShipmentMethod
{
  public override bool IsApplicableFor(Shipment toEvaluate)
  {

Don't get too hung up on the implementation, some of it is optional (e.g. ShipmentMethodKind) and it could be refractored quite a bit, I'm really just trying to show the idea not to show how to actually implement a solution.

You can see that in this case ShipmentMethod is really just a SPECIFICATION but in real situations a ShipmentMethod might well have more behaviour and data. The base class gives us an easy way to access all the ShipmentMethods that the system handles, this can be useful because it could easy provide useful methods like GetAllShipmentMethodsThatCanHandle(Shipment).

In reality there are multiple ways of implementing this, in particular you might want to load in the configuration for each ShipmentMethod from a database or other data source. This is more flexible than including it in the code but slightly more complicated to implement.

Referential Integrity

If you choose not to load the data from the database then we might seem to have a problem, how do we associate this ShipmentMethod with other objects in the KNOWLEDGE LEVEL and operational level?

It would seem that we've lost referential integrity but we have choices:

Generate tables - The code in the static constructor in ShipmentMethod can be used to generate a table in the database.
Use enum value as key - We can instead treat the value of the ShipmentMethodKind as a "key" in the database, automated tests will make spotting any issues really quick.
Have separate config table - As I said earlier we may want to load the configuration information for each ShipmentMethod from the database, if so we have a ShipmentMethod table which resolves the problem.

In any case I haven't found this to be a major issue, by and large I don't care that my ShipmentTable has a ShipmentMethodId table which doesn't lead me anywhere as if I want to understand what's going on I go back to the domain model.

In Closing

Not sure this topic needed as much details I've put in here, really its the KNOWLEDGE LEVEL pattern that's key but I hoped that by enumerating some of the implementation choices that I've seen I'd help you make an informed decision.

Biztalk and WCF - When two big beasts collide

2008-10-28T21:34:00.003+00:00

I spent the entirety of last week trying to create a ridiculously simple Biztalk orchestration and trying to get it to talk to a simple WCF service and I thought I should describe what I "learned".

Biztalk

If you follow me on Twitter you'll know how unbelievably annoyed the results made me and although I didn't learn much from the experience I thought I should put down some tips:

If Biztalk gives you an error DO NOT read it, the message itself is bound to be utter jibberish and the correct response is to put it straight into Google.
If Biztalk behaves like a problem is with step N don't assume that step N-1 passed especially if step N-1 is a transformation. You can test the transformation in isolation within the IDE using a sample document so do it,
If you are having real problems working out why Biztalk and WCF aren't playing ball then it might well be XML namespaces that are the issue.
If you're thinking of renaming the orchestration or anything in it be careful and take a backup first.

WCF

Whilst Biztalk left me cold the WCF side of it was a joy, mainly because Johnny Hall pointed me at the Castle WCF Facility and his own usages of it. Using the WCF Facility configuring your services is an utter joy, definitely when compared to the XML based approach that you get with bog-standard WCF. The documentation isn't great but the tests that you get with the Castle source code are the real way to see how to use its fluent interface.

Johnny also suggested we use a console application to host the service when testing and a Windows Service when deploying for a real. The console application makes testing locally a lot easier, just CTRL+F5 and your host is loaded and ready for you to fire requests at it.

If only Biztalk was as enjoyable to use...

BDD - Files/Folders/Namespaces (BDD)

2008-10-28T17:11:00.001+00:00

Files/Folders
One thing that can be troublesome when moving from TDD to BDD is how to organize your files and folders, so far I've tried two approaches:

One class in each file - So if you have When_associating_an_order_with_a_customer and When_associating_an_order_with_a_preferred_customer then they'd be in seperate files even though they are very closely related. If they share a base class, or a class they both compose, then that would be in yet another class (presumably).
Multiple classes per file - As an example you might group the Order addition contexts into a file called OrderPlacementSpecifications, the file could also contain the shared base class (if you went down that road).

To me the second approach has a couple of advantages:

Gives the reader extra information - By grouping the two order placement classes we tell the reader that they are quite closely related.
Simplifies folder structure - If we go for the other approach, one class in each file, then we're probably going to have to have more folders. The addition of the extra files and folders definitely makes the solution file harder to structure.

To give you an idea here's a screen shot of a part of the folder structure for a sample app we're doing:

Namespaces

In addition to files/folders I've tried a few approaches to structuring namespaces but the approach I'm trying now groups related artifacts. For example:

Specifications.Users.Domain
Specifications.Users.Domain.Contacts
Specifications.Users.Services
Specifications.Users.Domain.Repositories
Specifications.Users.UI.Controllers

The "Specifications" bit adds very little but I think grouping all specifications related to users is useful, not least as R# makes it easy to run all the specifications in a namespace. This can be useful if you have a big solution and only want to run the specifications for the area your working on. Its also worth saying that its "Users" to avoid clashing with the "User" class.

Folder wise however we're using a standard approach where your Repositories are in a completely seperate folder from your controllers, even though they might both relate to a particular entity. To me the lack of relationship between our folders and namespaces isn't a problem though, with R# its easy to find a file/type and in addition the folder/namespace tell you two different things about your codebase (one by "layer", one by "feature").

So I'm interested in peoples views? I'm guessing you'll all dislike it though because from what I've seen no matter what you do people will be unhappy with your file/folder/namespace scheme. Pluse we'll probably turn against this approach next week....

What I want from an ORM

2008-10-23T19:58:00.001+01:00

Thought I'd blog about some of the things I'd like to see in an ORM in the future, particularly to support DDD cleanly:

No enforced associations - I never want to create an association in the model just to support persistence, regardless of where keys are stored. So if I want to use uni-directional associations then I should be able to do that without having to go for workarounds.
Aggregate locking - Currently, with NHibernate at least, its difficult to lock an entire aggregate. For example NHibernate's optimistic concurrency approach involves applying a version to rows, however aggregates can span tables so we really want to be able to give each aggregate a shared version (coarse-grained locking approach). See coarse-grained lock pattern.
Validating before saving - I'd like hooks to automatically and cleanly validate an entire aggregate before persistence.
Disabling unit of work - I'd like to be able to disable my unit of work, in many cases when working with DDD the UOW becomes more of a hindrance than anything else. I really want to be 100% sure that the only way to save a Customer is through a CustomerRepository.
Revalidate value objects when reloading - Value objects only validate their data in their constructors, if your ORM does not ensure that a constructor that performs the validation is also used when reloading the object then its possible to end up with an invalid Value object. This is something you definitely want to avoid. Greg Young has raised this issue a few times, including in the NHibernate forum, and made some very good points.
Value objects - Choosing to view something as a value object is a design decision that you make irrespective of the underlying DB design, so whilst the NHibernate component mapping is useful it should be as powerful as the mappings used elsewhere. Unfortunately with NHibernate components don't support inheritance nicely and if your value object is stored in a separate table things get confusing.

There may be other things I'd want but those are the ones that come to mind.

New Home

2008-10-23T12:44:00.002+01:00

Just to let you know that I'll now also be posting over at lostechies. Obviously I'll continue to post everything here though so you only need to do anything if your already subscribed to the lostechies feed and getting my blog entries twice is annoying you.

DDD and Complexity

2008-10-22T21:06:00.003+01:00

There have been some interesting and entirely valid discussions on Twitter about the validity of DDD and about deciding when its useful and when its too complex (including good comments from Casey Charlton and Jimmy Bogard). I full agree with Casey and Jimmy and the comments, when combined with an interesting discussion on the topic at the latest ALT.NET UK discussion, let me wanting to blog about how I think DDD can be used in different types of projects.

First off I must admit that even for a simple business system I'd be thinking about using some of the patterns from DDD, in particular I think these usually have value:

Aggregate
Repository
Service

For example I find that thinking about aggregates makes me think much more carefully about how to map those classes to the database, for example how far to cascade, which in my view is a good idea as not thinking about boundaries can lead you to some fairly annoying debugging sessions. You can take these three patterns and with a good ORM and create and map a simple domain model to a green fields database very quickly.

Tradeoffs

Here's some of the tradeoffs we're making when using DDD on a system that it doesn't necessarily suit:

Analysis/Design - We map completely skip the analysis/design and in particular the discussions with the domain expert, instead starting from our requirements and letting TDD and our own design skills guides us to the correct design.
Encapsulation - We might expose everything using getters and setters and the domain may be anaemic, for example validation may be in attributes and/or use a custom framework.
Design Clarity - If our primary focus is on getting going fast then we're going to have to cut some corners. We'd bind the domain to the GUI, design it to be as easy to map to the database, make it easy to create domain objects (default constructors) and generally make tradeoffs in the quality of the domain model to make our own lives easier.
Flexibility - A model that is quick to create/map/bind is not likely going to be flexible, this may or may not be a problem.
Patterns - We're ignoring half the patterns in DDD, patterns that have a lot of value in a complex domain model but may not be justified when the domain is simpler/more closely bounded.

We make these tradeoffs to make our lives easier and in particular I wanted to cover two of the tradeoffs you may choose to make.

Design Clarity

If we want to be able to bind your GUI to the domain and map it to the database quickly then we can being to fray our domain model:

Value objects make binding and displaying validation errors trickier.
If our user interface uses wizards then the GUI will want to create domain objects early on in the wizard, possibly without giving us any meaningful data. We thus end up with default constructors or constructors with very few arguments.
If we use an ORM with a unit of work it will probably fight against our need to validate aggregates before saving them.
If we use an ORM we'll find it hard to version the entire aggregate.

With discipline you can make these tradeoffs whilst still maintaining a comprehensible model but there is no doubt that we are making tradeoffs.

Flexibility

We're also sacrificing flexibility, for example a lot of talk about repositories right now focuses on generic repositories. For example we'd have a Repository<T> class where T is an aggregate root and you'd have query methods on this repository that would take Linq specifications. That's going to be fine for a lot of cases but in more complex models (or where we don't control our environment fully) our repository can encapsulate complex logic, for example we should be able to ensure the following:

Instances of aggregate are never deleted or they are simply archived.
Instances of aggregate Y are built up from multiple data sources.
Instead of mapping aggregate Z to a (legacy) DB we map some simple DTO's and then in the repository convert them to our nicely designed aggregate.
Query Z is very expensive and needs to be done using SQL.

Those are all things I’ve had to be involved in when working with a moderately complex domain model and repositories helped encapsulate those details. So whilst I think generic repositories might have their place ins some systems I think you have to be aware of the choices your making.

What was the point of all this?

My point with all this is that you can get a lot of value from DDD without following it too closely, but you need to be aware of what tradeoffs your making and why. Choosing to go for a simple domain model when you have a complex problem to solve is a really bad choice and going from active record (true active record) to a real domain model is not going to be a smooth transition.

However I don't think the choice is easy, for example there's been a lot of discussion recently on whether DDD and complex models are suitable for CRUD related problems. In general I can see why using DDD on an average CRUD system is a mistake but sometimes it is worth using. For example we used DDD on a CRM system that among other things handled parties and their associations in a temporal manner. This was a complex modelling problem solved with a complex pattern but primarily we were handling CRUD (including constraints/validation) and setting the stage for other systems to use the information. Trying to do this using active record would, in my view, have been a big mistake.

So as Casey pointed out DDD is expensive and whilst it can pay off you need to do in with eyes open fully aware of the tradeoffs involved.