Seeing as it’s been just about a year since the last NSBCon we ran in New York City, I’m happy to say we’re bringing NSBCon back to the US – this time in Dallas Texas, Dec 1-4.

Click here for the full details.

First – the war stories

For those of you who have been wondering about what it would be like to apply the Service-Oriented principles I blog about, we’ll hear the war story of one brave architect’s journey down that path, which kinds of messaging patterns helped, and which it didn’t.

Another point of concern for people considering event-driven approaches to building systems is how to keep control of all of these loosely-coupled bits and pieces. Luckily we’ll have a session talking about various monitoring and devops techniques to keep on top of everything that’s happening – and not just a theory of how that could be done, but a “from the trenches” account of how well they actually worked.

Then – down to the metal

In addition to some really “down and dirty” sessions about all the cool nifty NServiceBus features that have come out this past year, we’ll also give you a sneak peek into what’s coming in the future (after keeping very tight lipped about our future roadmap).

And finally, we’ll talk a bit about the actor model and Akka.net – where they fit in message-driven architectures and how to get the two worlds to play together nicely.

And finally – an unconference

When all is said and done, what you really want is some time to talk through your specific challenges with people who’ve “been there” and “done that”. That’s why our last day is a full interactive “unconference” meaning that you get to set the agenda, get direct access to all the speakers, and even grab some NServiceBus experts to debug some code with you.

If you’ve never been to an unconference before, you don’t know what you’re missing.

So, hurry up!

Discounted “early bird” tickets are almost sold out.

I really hope to see you all there.

Check it out.

Wegmans have been long-time customers of NServiceBus and I recently got the opportunity to go on a guided tour in their flagship store in Rochester and hear all the ways that they’ve been leveraging our platform – even the scales that people use to weigh their produce are linked up.

Yup – you may even be eating NServiceBus and not know it

There are also some pretty powerful back-office processes at play – things that your regular consumer wouldn’t even notice but that matter quite a bit.

In any case, I’ll hand this over to Adam Fyles – the person who has done the actual work of getting NServiceBus into Wegmans, from the early proof-of-concept days to leveraging it across the enterprise to tell the story himself.

Here’s Adam presenting at NSBCon USA 2014:

And if you want the slides or to hear more from Adam, check out his blog at: http://adamfyles.blogspot.com/.

For more information on NServiceBus and the rest of the Particular Service Platform, check out http://particular.net/

The venue was packed, the speakers were great, and all in all the conference was a smashing success. If you don’t believe me <wink/> you can check out Roy Cornelissen’s coverage here.

All videos now online

For those of you who couldn’t make it, we have all the videos recorded for you and you can access them here.

Some of the cooler demos included:

• Szymon Pobiega showing how he plugged the Event Store under NServiceBus for complete transport, subscription, timeout, and saga persistence – video here
• Dylan Beattie videoing the audience with his iPhone and having that processed through NServiceBus and uploaded to their site in “real time” – check it out

Of course, I really think you should check out the full list.

Onward – to New York

And for those of you in the US and Canada, we’re happy to announce that registration is open for NSBCon New York!

The early bird is available until the end of this week (Aug 8), so you should really get going.

Register here.

We’ve got some really great speakers lined up for you – Jimmy Bogard will be there, and we’ll have both Oren Eini and Ted Neward speaking as well (who you might remember from the infamous ORM Smackdown).

And Carl and Richard from Dot Net Rocks will be there as well – rockin’ it the way that only DNR can.

For those of you who have been wondering, yes, you can come to just one of the days if you aren’t able to take both days off of work. Also, if you want to come but can’t convince your boss to pay for it, drop us a line and we’ll hook you up with a discount. Just send an email to nsbcon@particular.net and we’ll take care of everything.

One last thing – unfortunately, Oren won’t be able to come speak in London (but still will be speaking in NYC). While no one can quite fill in those shoes (as Oren is quite a big guy), we have someone I think you’ll really enjoy hearing: James Lewis, one of the main voices in the Microservices arena, will be coming to talk about … Microservices!

Hope to see you there: NSBcon.com.

The speaker lineup for NSBcon London 2014 is now complete and we’ve got a really great mix of talks, if I do say so myself.

I’ve already mentioned that Oren and Greg will be there, but I wanted to talk a bit about the rest of the roster:

First up – Wonga

If you’re living in the UK, you almost can’t avoid seeing these ads.

What you probably didn’t know is that Wonga has been running on NServiceBus for years now.
(No – we had nothing to do with the ads, and there’s nothing we can do about them.)

Charlie Barker was there from the beginning and has lived to tell the tale:

We faced the problem of scaling our platform to meet rapid growth in customers whilst at the same time increasing the team from 15 to 200 people spread across five countries. We did this by transitioning from N-Tier to SOA even as we were delivering new features and keeping the platform stable. No small feat.

This is an interesting story to hear, not only from the point of view of NServiceBus, and I’d definitely recommend grabbing Charlie over lunch or over a beer. This is probably one of the higher profile startup successes in the UK you’ll find and, as always, the behind the scenes story is just fascinating.

And to the cloud!

No self-respecting technology conference these days can go without spending at least some time talking about the cloud – and we won’t be the ones to buck the trend, even though we have a healthy disrespect for all sorts of things, including the ourselves

You’ll hear from one of the foremost Azure MVPs and all around cloudy Belgian Yves Goeleven (whose last name nobody is really sure how to pronounce). Yves has been the driving force behind getting all the various bits of Azure infrastructure integrated into NServiceBus and, if you get him to just the right level of inebriation, will spill all the dirty little secrets of the Azure platform that Microsoft doesn’t want anyone to know.

Dylan Beattie will then relate his tales of creating loosely-coupled encoding workflows for audio and video on the cloud at Spotlight – one of the world’s leading resources for professional actors, casting directors, and production professionals. It ain’t easy making ordinary people into stars.

And so much more

To see the complete lineup, go to NSBcon.com.

And me – what will I be talking about? That’s a good question.

Not to steal my own thunder (after stealing everybody else’s), but you’ll hear about the deeper integration we’ve got planned for SignalR, making your event-driven architecture extend from the back of your systems all the way to the browser, so much simpler and smoother than you ever thought possible.

I’ll also tell you about how we’re going to enable you to run on queues that don’t support distributed transactions (like Service Bus for Windows Server) without having to worry about making your logic idempotent. For some background, see my blog post on Life without distributed transactions.

In short – it’s going to be a kick-ass conference.

Check it out.

* and for those of you paying out of pocket, contact nsbcon@particular.net for a discount.

For everybody who has been building message-driven systems on .NET – I really do believe that this is the dawn of a new era (and if you know anything at all about me, you know that I don’t tend to exaggerate).

What is it?

Imagine designing your solution by graphically creating endpoints and messages, seeing the routing laid out in front of you visually, being able to specify system wide error and audit queues, having your cross-cutting authentication applied automatically to all endpoints, correlating events together into sagas, pressing F5, and as everything springs to life, on your second monitor you can see the messages flowing through your specific debug session synchronized in real time as you step through your code.

Well, you don’t have to imagine – that’s now a reality.

Top that off with some nice production monitoring that includes heartbeats for all of your endpoints, notification of message processing failures (with the ability to send them back for reprocessing), and some cool extensibility to plug in your own custom checks.

It’s a whole new world of service-oriented development.

This short video will give you a sense of what it’s like:

Here’s some other things you might find interesting.

Testing you can trust

I’ll bet you didn’t know the level of testing that we’re now putting into each release.

Every time we put out a new release, we test it on cloud servers configured across all permutations of transports, containers, persistence technologies, server operating systems, as well as for compatibility with previously released versions 2 years back (!). This way, when you want to have a new system subscribe to events published by a system already in production, you’ll know that it will all “just work” – no matter what.

Running all of these tests in the cloud is something that we could never afford back when we were a regular free/open-source project, but is definitely worth it. We know you count on us for your mission-critical systems and will continue doing our very best to be worthy of that trust.

The roadmap

Now that the platform is out, the next big thing you can expect is the removal of all distributed transactions.

Over the past months (and even years) we’ve been making preparations for this, making all sorts of small backwards compatible changes and enhancements so that we can fully control the inbound and outbound behavior of messages. The investments we’ve been making in automated tests is also in support of this. We want to be absolutely positive that you will get the exact same logical behavior as before without losing any of the transactional guarantees you’ve come to expect from NServiceBus.

We’ll give you updates on this and the other things we’re working on soon, as part of our larger roadmap so that you’ll know roughly what kind of big features you can expect and when. Of course, we always leave some “slack” in our iterations for working on critical patches, enhancements, and refactoring.

In any case, the main idea here is for you to be able to give us feedback and input on what’s important to you and for us to communicate back what we will be able to commit to and when.

Expressing our thanks

It’s been quite a journey this past year – the company has doubled in size and we’ve really got a great group of people here. While we’re constantly looking ahead and seeing all the wonderful things that we want to do, I wanted to take a moment and look back.

It’s been almost two years of work on ServiceMatrix (previously named NSB Studio) – the absolute biggest investment we’ve made. This past year was really the biggest push, and when taken together with all of the other tools – the production monitoring in ServicePulse that so many people have been clamoring for; the under-the-hood visibility of ServiceInsight; and the unseen glue of ServiceControl tying them all together; well, it was a very ambitious year.

I wanted to thank everyone who took part in our betas – your feedback was absolutely invaluable;

To our Community Champs who keep asking us the difficult questions and pushing us to do better;

To all of you in our community who have believed in us;

To the growing ranks of paying customers who voted with their wallets as well as their hearts;

You have my personal thanks as well as those of the entire team.
It continues to be an honor and a privilege.

Go and get it

That’s it – go to www.particular.net and click the big “download now” button and take the new platform for a spin!

I can’t wait to hear what you all think.

I’m happy to say that the recording of that session is now online, courtesy of the good folks at Skills Matter, and you can now view that here.

In this presentation, you’ll also see a lot of the new capabilities that will be coming out soon with the release of our new Particular Service Platform – the suite of tools that makes NServiceBus that much more awesome. From monitoring of production issues, to debugging cross-endpoint message flows, to modeling of message-driven & service-oriented systems, it really takes distributed systems development to the next level.

Check it out!

Oh, I almost forgot, there are still some seats left for my course in New York City. Details here.

I’ve given my Programming in 4D talk already several times and I haven’t yet uploaded the code for it.

And seeing as I’m going to be giving it again today (at the DevWeek conference in London), I figured that I should finally get my act together and put it online.

Interestingly enough, the other conferences I’ve spoken at either didn’t record it or didn’t put the recording online. Hopefully DevWeek will do better <FingersCrossed/>.

The overall solution

The scenario I talk about in this presentation is (again) a standard one in the world of retail: customers who want to return products that they’ve purchased and get a refund.

I’ve used our new ServiceMatrix tooling to model the solution like this (click for a larger image):

If you’d like to download the complete solution, click here.

Now, we’re going to focus on the RefundPolicy object that you can see towards the bottom left.

The Refund Policy

So, in this scenario, what we’re going to implement is a process whereby if you return your products within 30 days of the purchase, you’ll receive a 100% refund; if you return your products within 60 days you’ll receive a 50% refund, and anything longer than that and no refund for you.

Here’s the code:

public class RefundPolicy : Saga<RefundPolicySagaData>, 

                            IAmStartedByMessages<OrderAccepted>, 

                            IHandleMessages<ProductsReturned>, 

                            IHandleTimeouts<Percent>, 

                            IHandleSagaNotFound

{

    public void Handle(OrderAccepted message)

    {

        Data.OrderId = message.OrderId;

        Console.WriteLine("OrderAccepted");

        Data.Percent = 100;

        RequestTimeout(TimeSpan.FromDays(30), 50.Percent());

        RequestTimeout(TimeSpan.FromDays(60), 0.Percent());

    }

    public void Handle(ProductsReturned message)

    {

        Console.WriteLine("ProductsReturned");

        Bus.Send<IssueRefund>(m => m.Percent = Data.Percent);

        MarkAsComplete();

    }

    public override void ConfigureHowToFindSaga()

    {

        ConfigureMapping<ProductsReturned>(m => m.OrderId).ToSaga(s => s.OrderId);

    }

    public void Timeout(Percent state)

    {

        Console.WriteLine("Timeout");

        Data.Percent = state;

        if (state == 0)

            MarkAsComplete();

    }

    public void Handle(object message)

    {

        if (message is ProductsReturned)

            Console.WriteLine("No refund for you");

    }

}

public class RefundPolicySagaData : ContainSagaData

{

    [Unique]

    public int OrderId { get; set; }

    public Percent Percent { get; set; }

}

And what’s so good about that?

Well, not to steal my own thunder and give you a reason not to watch the video when it comes out, the trick is in the two RequestTimeout calls that are invoked when an OrderAccepted message arrives. You see, what that does is that it takes the data that defines the behavior of the refund policy and persists that via a queue, which will play back the data to our object according to the defined schedule.

This way, if/when the business decides to change the rules (say, reducing the timeframes to 20 days and 40 days), that will only affect users who are placing new purchases. The customers who made a purchase 50 days earlier (when the rules were still 30/60) will get the correct behavior applied to them.

Next steps

Of course, nobody would want a developer to have to open this code and change it in order to make a simple change like 30/60 -> 20/40, so we could externalize the data by creating a dictionary property on the saga where the key is the TimeSpan and the value is the Percent. Then, we could pull those values from either a config file or database and inject them into the property on the saga.

Here’s how the code of the saga would change:

public class RefundPolicy : Saga<RefundPolicySagaData>, 

                            IAmStartedByMessages<OrderAccepted>, 

                            IHandleMessages<ProductsReturned>, 

                            IHandleTimeouts<Percent>, 

                            IHandleSagaNotFound

{

    public Dictionary<TimeSpan, Percent> DataDefinitions { get; set; }

    partial void HandleImplementation(OrderAccepted message)

    {

        Data.OrderId = message.OrderId;

        Console.WriteLine("OrderAccepted");

        Data.Percent = 100;

        foreach(var kv in DataDefinitions)

            RequestTimeout(kv.Key, kv.Value);

    }

And the code to do the property injection would look like this:

public class RefundPolicyDataConfigurator : INeedInitialization

{

    public void Init()

    {

        Dictionary<TimeSpan, Percent> data = null /* get from config instead */;

        Configure.Instance.Configurer

            .ConfigureProperty<RefundPolicy>(p => p.DataDefinitions, data);

    }

}

Classes the implement INeedInitialization are invoked at process startup, and that call to ConfigureProperty instructs the container to set the DataDefinitions property of our RefundPolicy every time it is resolved (which is on every message).

In closing

We now have a solution which allows non-developers to make changes to the refund policy definitions without requiring us to deploy any new code to production. Also, any changes that are made will preserve the promises we made to our users in the past.

Of course, if you want changes to rules to impact all users immediately, this approach wouldn’t be so good.

Again, if you’d like to download the complete solution, the code is here, and if it wasn’t already obvious, this code makes use of NServiceBus quite heavily.

When the recording comes online, I’ll update this post with a link as well as do another blog post.

Hope you like it.

But since I’ve been getting repeated requests to weigh in on the topic, I guess I’ll have to risk fanning the flames up again.

Also, since quite a few reactions have already been written on the topic (and I don’t want to repeat them here), I’ll just point to this post by Arnon which sums them all up pretty well.

Now, I don’t entirely agree with all the commentary Arnon pointed to, or all of his thoughts on the topic, but I’ll try to take those up some other time.

And before jumping into it, let me say that there is a lot of good stuff in the article and that, regardless of naming, spreading the word more broadly on these approaches has value.

So, where do I stand on the topic

First of all, for those of you who have been following my blog for a while I’d say this:

Microservices almost equals Autonomous Components.

Why “almost”?

Because an Autonomous Component (AC) isn’t necessarily a physical unit of deployment – very often we’ll see multiple ACs deployed in the same physical process. One of the most common occurrences is in a web front end built as a composite UI. In the same web server process we’ll see components from multiple Services.

This is something that was hardly mentioned in the original article.

On Services and Systems

In my world, Services are a larger organizing principle that are meant to align solution domain boundaries with problem domain boundaries.

Now, that might sound very similar to this passage from the original article:

“The microservice approach to division is different, splitting up into services organized around business capability. Such services take a broad-stack implementation of software for that business area, including user-interface, persistant storage, and any external collaborations. Consequently the teams are cross-functional, including the full range of skills required for the development: user-experience, database, and project management.”

Now, this isn’t entirely surprising because I did have several conversations with both James and Martin on the topic over the past couple of years.

Still, there is something important missing here that I believe is very important to achieve loose-coupling, and that is that Services necessarily have to span system boundaries.

Let me repeat that: a Service will need to have components that are deployed to more than one system.

Here’s why:

Let’s say you have a piece of data like the price of a product. Not only will that data be visible in one system, often it will need to be shown (as well as updated) in other systems too. In order to have appropriate encapsulation of that concept, the owning service will need to be the one that owns the components that operate on that concept in the other systems.

This means that if we need to show the product price on an invoice in a back-end system, then that invoice would have to be a composite UI as well, and the service which owns the price will have a component deployed there which would be responsible for showing the price on the invoice.

In this manner, no code outside the service boundary would know about the concept of the product price and thus could not end up coupled to it.

Although the original article does get into this to some degree (when talking about Decentralized Data Management), I don’t really see how a microservice/AC could end up having this level of ownership of data.

Still, the point made about different persistence technologies is valid at the level of services (though not ACs).

How big is a service

Now, if the price is not shared outside the boundary of the service, then how would order totals be calculated?

The answer is that the totals must (MUST) be calculated in the same service.

This shouldn’t be surprising as it’s just good old OO – encapsulating data with the logic that operates on it. Or, if you’d like, call it the Single Responsibility Principle: there should only be a single service impacted by a change to the definition of this data.

As a result, you’ll tend to see services that aren’t all that small, and probably not so many of them. In my experience, I’ve seen between 7 and 15 services the majority of the time.

Cross-service collaboration

Although I am glad to see the recommendation for event-based interactions between microservices, the focus on cross-process communication ignores some extremely important collaboration scenarios – the most important of which is in the client tier.

In a web application, it is quite common to have components written in javascript from multiple services interacting among themselves in the browser – one publishing events, others subscribing to those JS events. It is also quite common to see those JS components request some data from back-end components in the same service in response to those JS events.

This type of synchronous RPC communication within a service boundary is perfectly acceptable, although it stands in contrast to the recommendations of the microservices approach.

Caveat on sharing data

I’ve been going on and on about the importance of not sharing data, but there is one exception to that rule.

There is a special service that I call IT/Ops which (among other things) is responsible for integration with 3rd party systems. As a part of this integration, it encapsulates data transformation logic and, as a result, needs to be able to receive data from the other more business-centric services.

As you can imagine, this puts IT/Ops in the risky position of coupling itself to a lot of things and thus needs to be done carefully. As a result, I recommend that many of the most skilled technical people work within the IT/Ops team, also serving in a consultative capacity to the other service teams.

In closing

I am extremely thankful to Martin and James for writing the Microservices article.

I think that the conversations it has sparked are timely, and hopefully more people will ponder these questions of how to structure their code-bases in order to avoid them becoming monolithic.

And while I think that it’s great to consider aligning team boundaries with service boundaries, people need to understand that it will need to be an evolutionary process – it will take time to transition an existing code base and an existing team structure to this new model, especially since these teams will have to continue to deliver features and bug fixes through the transition period. Jumping to the new model directly may cause more harm than good.

This is actually one of the most salient topics of my course (next one in NYC in May) – how do you get there from here. In my experience there are 4 phases that companies go through, often taking at least a couple of years, with larger environments taking potentially longer.

In any case, let’s keep the conversation going.

What are your thoughts? Have you been applying the Microservices approach, or possibly the one I talk about (I really should give it a name). What’s been working well for you? What hasn’t?

Leave me a comment or write your own blog post.

<Applause/>

It’s been a long time since my first blog post on NServiceBus (about 7 years ago) and the community has come a long way since then.

In any case, this won’t be a “looking back” post but rather one looking forwards.

In addition to myself and the rest of the crew, we’ve got some great speakers coming out including the inimitable Greg Young and the formidable Ayende Rahien (aka Oren Eini).

Register here.

For those of you who can’t make it to London for June 26-27, we’ll be running another NSBcon in The Big Apple (New York City) on September 29-30. Register here

If you’d really like to come but have to pay out of pocket (either because you’re a contractor or your employer just won’t pay), send an email to nsbcon@particular.net and we’ll get you a nice discount, but please hurry as we don’t have many of these tickets left.

Hope to see you there!

Register here

I’ll be teaching the 5-day Advanced Distributed Systems Design course on the week of March 24-28 in London. Registration is currently open here.

I’ve also decided to come back to New York City and teach the course there on the week of May 12-16. Registration isn’t open just yet but if you’d like to reserve a spot, send me an email to training@udidahan.com.

You might have also heard some whispering of an NServiceBus conference – that too will be coming this year.

Stay tuned.

While we’ve been making some pretty amazing progress, it still a bit premature for big announcements.

In any case, here’s a partial list of goings-on:

Monitoring – sneak peek

One of the most requested additions to NServiceBus was strong production monitoring capabilities.

Some of the things we have coming include:

1. Endpoints periodically send heartbeats to a central location, and if they stop coming, the administrator gets notified.
2. If messages can’t be processed for some reason and are sent to the error queue, the administrator gets a nice visual notification with the ability to send the message to be reprocessed.
3. Developers can also plug in their own custom code to periodically report on various things, for example the existence of a folder, accessibility of a URL, etc.

Check out this short video to see how it all works:

And yes – it will work across all our supported transports (MSMQ, RabbitMQ, ActiveMQ, SQL Server) as well as in the cloud (on Azure).

Partners

This was also a big year for the larger ecosystem.

In addition to our long-time partner Headspring, home of the legendary Jimmy Bogard, many more companies are now Official Partners.

I don’t want to play favorites here so this list is not in any particular order (except alphabetical). There are also a number of independent consultants that do amazing work but I’m not including them here for brevity so you might see some prominent countries not listed.

In the US, we have:

• Architecting Innovation which covers all of the US.
• CTS – focusing on the South East.
• Headspring – Nation-wide but with a Texas flavor.
• ILM – focusing on the Mid-West.

ILM also happens to be the home of David Boike – one of our community champions and author of the first NServiceBus book.

In Europe, we have:

• BBV – focusing on Switzerland, Germany, and Austria
• InfoSupport – focusing on the Netherlands
• WebStep – focusing on Scandinavia

BBV were the ones who did the heavy lifting for getting our support for ActiveMQ off the ground.

And not to forget the folks down under, there’s Driven Logic in the Sydney area.

Keep an eye out as there’ll be some interesting things coming from our partners this coming year.

In Closing

I’ll close with some thank-you’s:

Thank you to the hundreds of new paying customers, as well as to the customers who’ve stuck by us over the years who make it possible for us to make enterprise development on .net that much more enjoyable.

Thank you to all the employees at Particular Software – now twice as many as there were last year, and probably going to double again in 2014. This team, I’ve got to tell you, is absolutely world-class.

Thank you, my dear reader, for coming back and reading my rambling thoughts time after time. I can’t believe I’ve been doing this 10 years now.

And, to my wife and kids, well, they don’t read my blog, but on the off chance they might one day, I’d better thank them as well.

I wish you all a wonderful 2014.

For the complete list, see http://particular.net/courses-and-events.

Jimmy Bogard will be coming back to London next week to teach the Enterprise Development with NServiceBus course – Nov 11-14. Register here.

David Boike, author of the first NServiceBus book, will be teaching that same course in Minneapolis USA next month – Dec 9-12. Register here.

*

If you’re not up for the full, intensive, 4-day deep-dive – there’s still something for you:

Mauro Servienti will be speaking at the Italian Developer Conference in Milan on Nov 27 about the use of NServiceBus in Azure. Check it out.

And, even though it’s not technically in 2013:

Jimmy Bogard will be delivering a 1-day NServiceBus crash-course at the CodeMash in Sandusky Ohio on Jan 7. Although it’s only been a week since registration has opened, CodeMash is sold out but if you’re already registered, make sure to go and see Jimmy. He’s a great presenter.

*

And, just so that you can get in early before this sells out too, I’ll be offering my 5-day SOA course again next year – first in London and then in New York City. The date for NYC isn’t set yet, but London is good to go: March 24-28, 2014.

Register here.

There’s also a 10% discount if you book before Jan 23.

Hope to see you there!

Boris Tveritnev, the architect of the Kaspersky Protection Center, had this to say:

“NServiceBus is the backbone of the messaging infrastructure of Kaspersky Protection Center – our innovative platform providing security-related services for millions of connected devices around the globe. Millions of events produced by devices and services are handled by the NServiceBus-powered codebase every day. We chose NServiceBus for its clear architecture and easy yet powerful programming model, which provides reliable solutions for our distributed systems.”

Dmitry Aleshin, the Director of Product Management added:

“Kaspersky Protection Center is our vision of how online security in a modern, multi-device era should be offered. This is a single interface for managing licenses and security features on all the devices which a person uses on a daily basis, along with extended convenience and protection offered by the Kaspersky Password Manager service. In future, we are going to increase the number of applications and services to make the virtual lives of people who use Kaspersky Lab products even more comfortable and secure.”

This is the latest in a series of milestones for Particular Software and NServiceBus in Particular as we see more of the world’s largest and most trusted brands introduce our Service Platform into their business critical systems.

For more information on the Particular Service Platform, click here.

It’s pretty thorough, going from the absolute basics of one-way messaging and publish/subscribe, through how to handle encryption, fault tolerance, long-running processes, and into aspects of administering a running system like logging, virtualization, monitoring, and scaling.

You can buy the book from the publisher’s website here and of course on Amazon as well.

If you’d like to win a free copy of the book, go to David Boike’s blog post here.

This is something of a milestone for NServiceBus and has, indeed, been a long-time coming.

Please pop over to David’s blog and join me in thanking him for his great work.

Mogens Heller Grabe, creator of Rebus, will be joining us.

The advantages to the NServiceBus community are pretty clear – having more people in the core team with experience building service bus technology will help us get more done, faster.

What I want to talk about is the impact to the larger open-source ecosystem.

On Open Source

Although NServiceBus moved away from the permissive Apache license some time ago, the source is still and will continue to be available under the OSI approved Reciprocal Public License.

Rebus has always been licensed under the permissive Apache license and, as far as I know, will continue that way.

In order to make sure that the Rebus code base is in no way tainted and ensure that its users remain protected, Mogens will not be able to continue contributing code to Rebus. To hear Mogens comments on this change, see his blog post on the topic.

So you’re killing Rebus?!

That question always come up whenever some company hires someone whose working on an open-source project, doesn’t it? I can only speak for NServiceBus here, but our focus is on getting additional talent – growing the team, so that we can achieve all the great things we’ve got planned, not on killing anything.

I don’t think we could “kill” Rebus even if we wanted to. There is a thriving community of users and committers around Rebus who have the choice and ability to keep it going (or not).

And there isn’t really anything that we could gain commercially from the death of Rebus – its users probably wouldn’t switch to NServiceBus anyway. If anything, they’d most likely go with one of the other free, open source service buses out there – most notably, Mass Transit.

So, it’s just business as usual then

Pretty much.

We’re going to continue hiring more people to help realize our vision of the Particular Service Platform, as well as partnering with companies that help complement this vision (like we’ve done in the past with RavenDB).

And if you’re either an independent consultant or working at a consulting company and have experience delivering NServiceBus projects, we have an interesting proposal for you, so please contact us.

Exciting things are coming.

NServiceBus version 4.0 is now out.

Hard to believe it’s been more than 7 years since I started on this journey.

I know I always say this, but this is our biggest release ever. It’s been over a year since we released version 3.0 and in that time, the adoption of NServiceBus has totally exploded. I know I always talk about the big brand names but there have been hundreds of small to medium companies that have been seeing real success on this platform.

OK – one big name. I feel like I have to. If you’ve been to McDonald’s and purchased a happy meal for your kids – that’s going through NServiceBus. Yeah, at any McDonald’s around the world – and not just happy meals.

What’s in the box

While NServiceBus has historically used MSMQ as its primary queuing infrastructure, we now support RabbitMQ and ActiveMQ out of the box so you have full AMQP compliance at the wire level, and we’re soon going to be releasing a fully supported IBM MQ transport.

You can see the complete release notes for version 4 of NServiceBus here.

For those organizations who’ve been ignoring NServiceBus until now because they prefer having everything in a central database, we now support using database tables as queues.

While I’ve always been big on queuing infrastructure, I understand that it takes time for an organization to change and I hope that this will provide a smooth transition path.

Try it online now

If you’re in a locked-down corporate environment that won’t let you install anything on your box, but you’d still like to try NServiceBus out, we’ve got the solution for you.

Try our new hands-on labs – you’ll get remote access to a machine in the cloud with NServiceBus pre-installed on it ready to go.

We’ve even got a scale-out lab where you’ll see just how easy it is to build a 4 machine NServiceBus solution.

Try the labs.

Modeling

I’ve talked before about some modeling tools we’ve been working on for NServiceBus and I’m happy to say that they’re now ready for public beta. Instead of the name NServiceBus Studio (which overlapped far too much with Visual Studio), we’re now calling this ServiceMatrix.

ServiceMatrix makes building message-driven service-oriented solutions an absolute joy allowing you to operate at much higher levels of abstraction. Find out more here.

Debugging

While the looser-coupling of message-driven systems makes them more scalable and maintainable, it does make debugging harder since you can’t just step-through one function to another when everything is invoked asynchronously.

For that reason, we’ve been working on visualization and management tooling to make things easier. By leveraging the built-in auditing of NServiceBus, we can show you the message causation chain – which command caused which event, which in turn caused what other messages to flow. We also hook into the error queue to show you when messages fail (in orange below) and the full stack trace of the exception.

Of course, you can always see the full body and headers of each message.

This tool, called ServiceInsight is now in its second beta and the feedback has been overwhelming – “oh my God, how could you have waited this long to release this?! Even our testers and admins are using it.”.

In Particular

You might have noticed that all of these links are going to the Particular.net domain, and that the existing NServiceBus.com now redirects there as well.

After talking things over internally, the team agreed that we wanted to be much more than “just” a service bus. The new products I mentioned above are the first step in that direction, but there is more to come.

We wanted a name that wouldn’t pigeon-hole us but, I’ve got to tell you, it’s extremely hard to find a name for a software company (and a corresponding domain) that isn’t already taken.

When we hit on the name Particular Software and saw that ParticularSoftware.com was open, we were quite thrilled. You see, we’re very particular about the software we use, and we want to write software for people who are just as discerning as us. Getting Particular.net was nice too.

In closing

I want to take this opportunity to thank our community for believing in us all this time, and an especially big thank you to our community champions who have been gently shepherding thousands of new users in their first NServiceBus steps.

To our partners that have been building solutions for their customers on NServiceBus – you have our deepest appreciation. We’ve also got some new things coming just for you that you’ll really like.

By the way, if you’re an independent consultant or working at a consulting company and would like partner with us, please reach out to us.

And, although we’re constantly growing the team – it seems like there’s always so much more to be done. When I started down this path over 7 years ago, I never imagined that this could go on this long. Today, I’m pretty sure that there’s at least another 7 years more to go.

We’re dedicated to making this the best service platform out there.

Check it out.

I think the reason for this interest is the rise in popularity of cloud platforms and queuing systems like RabbitMQ (which don’t support distributed transactions) and the difficulty of setting up distributed transactions even in on-premise.

Of course, there’s also the regular scalability hand-wringing going on even though most people wouldn’t bump up against those limits anyway.

In this post, I’ll talk about the nature of the problem, explain the pitfalls in some of the common solutions, but I’ll put off the description of how to provide consistency without distributed transactions to a future post as this one is already going to be quite long.

I’ll start with the basic fault-tolerance issues and then explain how things spiral out from there.

Starting with the basics

OK, so we have a queuing system in place that dispatches messages to our business logic which does some transactional work against a database.

Let’s say that we completed the transaction against our database but before we could acknowledge to the queue that the message was processed successfully, our machine crashed. What our machine comes up again, the queue will once again dispatch us the same message. Unless we have some logic to detect that we’ve already processed it (called “idempotence” in the REST community), we will end up processing it again.

In short, the problem is duplicates.

Attempted solutions to the duplicate problem

Most queuing systems don’t do anything about duplicates, actually giving it a proper architectural name: At-least-once message delivery, as opposed to the Once-and-only-once model that a queue that supports distributed transaction provides.

The solution often suggested is to have your logic check to see if it has already processed a message with that ID before – in essence storing the ID of each message processed for some period of time. Of course, there is some performance overhead with that, but it might be a small price to pay compared to dealing with it in the logic of every use case.

On the other hand, you’ll often have some messages (like Update commands) for which it looks like you can safely process them multiple times, in which case you might want not to pay the performance overhead there. The thing is, if your logic publishes an event in addition to the regular database work (something that is quite common) and you process the same message twice, you will probably end up publishing the event twice as well.

These duplicates are different in that here we have two distinct messages with different IDs that contain the same business data. This means that recipients of these messages will not be able to filter them out at an infrastructure level anymore.

NOTE: Deduplication abilities in queues

Although the Azure Service Bus doesn’t support distributed transactions meaning you still have the issue mentioned above, Microsoft added the ability to detect and filter out duplicates based on message contents rather than just the ID. This helps quite a bit but it’s important to understand that that doesn’t cover everything for you. Let me explain:

More complex logic

In some of your most important use cases, you may have both entity updates as well as entity creation happening together in your domain model. You might be using some kind of event model (like I wrote about here) to percolate out the information that an entity was created in order to keep your service layer decoupled from the internals of the domain model.

In the callback code from these domain events, you will likely publish out an event on the queuing system containing information like the ID of the entities created as well as other business data. And there’s the rub.

You see, without distributed transactions, you can run into some problematic scenarios:

For example, if you don’t make sure that your event publishing calls to the queuing system include the same transaction object as the one you used when retrieving the original message from the queue, then those calls could “escape” before you know if the database transaction is going to succeed. Deadlocks always happen at the lousiest times. Anyway, if you’re using database generated IDs for your entities, then those IDs will get published out in events despite the database rolling back and your subscribers will now be making decisions on wrong data – not just eventually consistent data.

In this case, processing the message again doesn’t really solve the problem – it just means that you’ll be publishing events with different IDs, so an infrastructure like Azure Service Bus couldn’t really de-duplicate them.

On the other hand, if you do use the same transaction and combine in the infrastructural message ID based de-duplication described above (as identifying duplicate calls for complex business logic is damn hard), you’ll run into another problem.

Consider what would happen if your server crashes right after finishing its database work but before it completes the transaction against the queuing system. When going to retry the message, the infrastructure filtering thing would know not to call your business logic again and that message would be quietly swallowed. Unfortunately, the event publishing calls to the queuing system from the first time the message was processed were rolled back and since your business logic isn’t called again, the event publishing won’t happen again.

Oops.

In closing

I hope I’ve been able to clarify what kind of scenarios distributed transactions solve for you and some of the difficulties in solving them yourself.

Now, to be clear, you could solve these problems by going in-depth on each of your use cases, analyzing the consistency needs and structuring the code differently to address those needs. But give this another thought, if our consistency is dependent on calling otherwise independent APIs in exactly the right order, and that a change in this order would not cause any visible functional effects, what would happen when developers with less expertise maintain this code?

The folks in the event sourcing community have their solution to this which is based on writing their business logic differently. As the adoption of this pattern is still pretty limited (probably still in the Innovator section of the Technology Adoption Curve), it’ll be interesting to see how it holds up with larger teams in the mainstream.

Oh, and in case it wasn’t clear from before, the guys in the REST community haven’t even begun addressing this problem when it comes to server-to-server integration.

We’re working on a solution for this with NServiceBus that won’t require you to change how you write business logic. We’ve got one big release to do before we can roll this in, and that’s coming soon (with all sorts of cool things like support for ActiveMQ and queues in the database). The solution we’ve found is architecturally sound but you’ll have to wait for my next post to hear about it.

Stay tuned.

A short intro to SOA, done right

In the “vertical services” approach I espouse, you often see components from multiple services deployed to any given endpoint. While these services usually don’t need to communicate with each other at all, occasionally you’ll see them leaving “breadcrumbs” behind for each other – things like UserId or OrderId in the session. What’s especially important is that these IDs are accessible even before the entity is finalized – this enables each service to collect its own data without needing any other service to know about that data.

In an ecommerce environment, we would see one service owning money, another the product catalog (excluding prices, those would be owned by the previous service), another service owning the customers’ payment info (credit cards, etc), and yet another owning shipping addresses – all of these separate from the one that owns the shopping cart. Let’s call these Finance, Catalog, Payment, Shipping, and finally Shopping – just so that we have something to reference later.

The API

While we can do all sorts of cool browser composition with the UI in our own system, enabling each service to collect and display its own information, if we want to expose an API for clients to call, we wouldn’t want to force those clients to have to make a separate call to each service in order to make a purchase. Instead, we’d want something that looks like:

MakePurchase(Guid orderId, Dictionary cart, CreditCardInfo cc, Address shippingAddress)

In case you were wondering, the service which owns the definition of this API is different from all of the above services – it is a service that is primarily technical in nature and is responsible for things like integration and data transformations. I call this service IT/Ops.

Getting the data from the API to the services

So, we don’t want any of our business-centric services to know about anybody else’s data structures, so that leaves it to IT/Ops to pass the data to them. The thing is that we still want to do that in the most loosely-coupled way possible – with messaging being a good candidate for that.

So, what we’ll do is have IT/Ops send a message containing the data to the other services, but with a slight twist.

Here’s what the code would look like with NServiceBus:

Bus.SendLocal(new Order { Id = orderId, Cart = cart, 
                          CreditCard = cc, ShippingAddress = shippingAddress });

Why the SendLocal?

So that the components from the other services can all run together with IT/Ops in the same process giving a nice tight deployment model.

UPDATE:

If you’re using a transport like RabbitMQ that is set up as a remote broker, the overhead of going back through the broker might not be worth the improved reliability you’d get by going through messaging. In that case, you might want to consider the Domain Events approach. This would give you a similar level of decoupling but then IT/Ops would need to set up an surrounding transaction, well, that is if you want all the services processing to succeed/fail as one unit. If you don’t, you’d probably be better off just sending messages from IT/Ops to endpoints that host each of the components of the other services.

End Update

Before we get to the services, let me show you the Order class – specifically, the interfaces it implements:

public class Order : ShoppingOrder, PaymentsOrder, ShippingOrder
{
    public Guid OrderId { get; set; }
    public Dictionary<Guid, int> Cart { get; set; }
    public CreditCardInfo CreditCard { get; set; }
    public Address ShippingAddress { get; set; }
}

public interface ShoppingOrder
{
    Guid OrderId { get; set; }
    Dictionary<Guid, int> Cart { get; set; }
}

public interface PaymentsOrder
{
    Guid OrderId { get; set; }
    CreditCardInfo CreditCard { get; set; }
}

public interface ShippingOrder
{
    Guid OrderId { get; set; }
    Address ShippingAddress { get; set; }
}

Each of the above interfaces represents the data that each service cares about. Therefore, each service will provide an assembly that handles that message/data and persists it to its database, like this:

public class ShoppingAPIHandler : IHandleMessages<ShoppingOrder>
{
    public void Handle(ShoppingOrder message)
    {
        //persist to shopping service db
    }
}

public class PaymentsAPIHandler : IHandleMessages<PaymentsOrder>
{
    public void Handle(PaymentsOrder message)
    {
        //persist to payment service db
    }
}

public class ShippingAPIHandler : IHandleMessages<ShippingOrder>
{
    public void Handle(ShippingOrder message)
    {
        //persist to shipping service db
    }
}

Since the Order object being sent is a polymorphic match for all of these interfaces, NServiceBus knows to invoke all of these handlers. By the way, if you care about the order of invocation, then you can control that as well (but I won’t get into that here).

Also, since all of these handler assemblies are deployed to the same endpoint, and the Order object is sent just once, this means that all the handlers will be invoked in a single transaction on a single thread – either all of them succeeding, or all failing. Since they’re all connected on the same Order Id, referential integrity can be preserved as well.

Wrapping up

When you are building a system on SOA principles, you’ll often find that you need a service like IT/Ops to handle data transformation and other broker-centric tasks. While much of SOA is based on the Bus Architectural Style – meaning primarily publish/subscribe interaction between services – that doesn’t mean that your business-centric services cannot have their components deployed in the same process.

I’d go so far as to say that if you aren’t deploying components from multiple services in process with each other at least some of the time then it’s quite likely that your service boundaries are probably incorrect.

Anyway, I hope you found this post interesting. Shout out to Slawek who gave me the idea for this post.

By the way, if you’d like to learn more about these kinds of patterns, the next batch of courses is open for registration – but the early bird prices are almost over, so you’d better hurry.

Register for   Denver CO, USA,     Bad Ems, Germany,     Perth, Australia.

These full-day events will include topics like CQRS, REST, as well as async and functional programming.

Full information and registration is available here.

Full information here.

Here’s the full information.

Here’s the plan:

• Sep 25 in Berlin
• Sep 26 in Dresden
• Sep 27 in Leipzig

If you’re in and around one of those areas, drop by and support the community. If you have some friends or colleagues that are working on distributed systems, drag them along as well. They’ll thank you for it later.

Werner posts:

As this is a support forum, the nature of the threads are almost always about problems; which is fine. But I just feel like posting something not related to problems

After spending 2 weeks with [version] 3.x I must say that I like it a lot, and have recommended that we make a global change to the new version (we’re using 1.9 and 2.0).

I like the support that I see in this forum. Both from other users and especially from the people now making a living developing and supporting NServiceBus. No one mentioned, no one forgotten; thanks to all and keep up the good work!

————–

The most incredible thing about NServiceBus is our community – over 1600 of the best and brightest of the .NET developer ecosystem – over 500 of which have gone to the 5-day Advanced Distributed Systems Design course. In addition to going about building the most reliable software their industries have ever seen, many are there to lend a helping hand to those just getting started.

Not surprisingly, we’ve seen more people downloading NServiceBus than actually in the community, at least a 2 to 1 ratio. In my work with clients, I often see somewhere in the area 5-10 people actively developing with NServiceBus in a company, but only one of them downloading new releases and bringing them into the company.

I’ve heard mention of this “power law” before, but I’ve got to tell you, when I started on this journey with NServiceBus 6 years ago, in my wildest dreams I wouldn’t have ever thought that tens of thousands of developers around the world would be using my baby.

Thank you all.