The Angry .Net Developer

NHibernate WCF Error: HTTP request context being aborted by the server??

noreply@blogger.com (Rudy Lacovara) — Wed, 19 Dec 2012 15:34:00 +0000

I’ve been ambivalent toward ORMs for quite some time now, but I’m really starting to dislike NHibernate in a special way. My experience has been that ORMs lead to a lot of wasted time troubleshooting code problems that are really ORM problems. Here’s one example.

I recently got this mysterious and unhelpful error message on an ASP.Net MVC3 web app that was pulling data via a WCF web service, which in turn was using NHibernate for persistence.

An error occurred while receiving the HTTP response to http://localhost:8080/JobsService/ws. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.

Ok, my service endpoint was definitely using HTTP and none of that other stuff was happening either. So what was the problem? NHibernate lazy load proxies.

Let’s say I have a Company entity that contains a child collection of type List<Job> as shown in the diagram below. So if we look at the Company entity, all of the fields (Name, State, City) exist in my data table, except for the Jobs. Jobs represents a relation between my Company and Job tables and to get Job data NHibernate has to query the Job table.

At this point it’s important to note that I’m using NHibernate, all of my entity properties are virtual, and I have lazy loading enabled. So what happens when I run a query to get a Company? What data does NHibernate populate? It’s going to populate all of the data that’s in the Company table (Name, State, City) , but it’s not going to populate the Jobs collection. It’s going replace that virtual property with a proxy that will execute a query to get Jobs only when that property is accessed.

I actually think this lazy loading proxy technique is pretty neat and it’s technically impressive. Unfortunately it’s also the root cause of a lot of application errors.

So the lazy loading proxy works great when we’re accessing an entity directly in our web app while it’s still has the context of the NHibernate session, but what happens when we try to return our Company over WCF and WCF tries to serialize that Jobs property that hasn’t been lazy loaded yet? That’s right, you get the ambiguous error message above.

The fix is simple. You just need to eager load the Jobs. You can do this any number of ways. Here’s how to do it in a criteria query.

return session.CreateCriteria<Company>()
.Add(Restrictions.Eq("Id", _Id))
.SetFetchMode("Jobs", FetchMode.Eager);

Now that we’re eager loading the Jobs there’s real data there instead of the lazy load proxy, WCF has no problem serializing the Jobs, and our mysterious ambiguous error goes away.

So, I hope this saves someone a little time, and please remember, friends don’t let friends use ORMs.

The problem with ORMs

noreply@blogger.com (Rudy Lacovara) — Wed, 18 Jul 2012 17:35:00 +0000

Over the last 8 years I’ve done a lot of thinking and experimenting with different persistence models. If you’d asked me 6 years ago what a persistence layer should look like I would have whole heartedly suggested an ORM like nHibernate. Linq to SQL came along and initially looked promising but I quickly soured on it in favor of Entity Framework. Then I actually tried to use Entity Framework in a production app and quickly found that in spite of the initial query writing efficiencies, it caused problems in other areas and the net result was that it didn’t save me any time at all but did add limitations and complexity to my app.

So what was the problem? As I spoke with more developers I started to hear the same complaints, even though some were using EF, some were using nHibernate, others were using LinqToSql. I heard the problem stated best by a guy I met at the speaker’s dinner for the Rocky Mountain Tech Trifecta. I can’t remember the guy’s name but his words stuck with me. I was telling him that I would never use EF again if the choice was up to me. He said it’s not that he has a problem with Entity Framework, it’s that he’s done with ORMs. I think that’s really the heart of it. There’s a fundamental problem with ORM as a pattern. It just doesn’t adequately solve the problem and it ads significant complexity and inefficiency to an application.

I recently ran across a couple of old blog posts that make for interesting reading, Is OR/M an anti pattern? by Ayende, and ORM is an anti-pattern by Laurie. I’m hearing this line of thought more often as more developers use EF and other ORMs in production apps and then have a couple of years of living with the consequences.

So it’s easy to knock something, but what’s the alternative? For me it’s been moving to a simpler data mapper instead of a full on ORM. I first wrote my own simple data mapper, but since then I’ve discovered Dapper.net by Sam Saffron and Marc Gravel. On the surface Dapper works almost identically to the mapper I wrote, but under the covers it’s way better and much more efficient. So I’ve been porting all of my old code over to use Dapper.

The results so far have been fantastic. Sometimes I have to write more code that I would have to write with a full-on ORM, but I find that I can make major application changes (relatively) quickly and easily and I have never once spent hours struggling with my persistence architecture trying to understand why I’m erroring out on an update for an object graph, or troubleshooting lazy load issues, or select N+1 issues, or thinking to myself “If only this was SQL I would know exactly how to do this”. Instead it just works, exactly the way I expect it to, and in the end that saves a lot of time and effort.

Great Backbone.js Tutorial

noreply@blogger.com (Rudy Lacovara) — Mon, 04 Jun 2012 18:10:00 +0000

I definitely think that JavaScript is the future (at least the near term future) of app development, and I recently converted HireFlo to a single page JavaScript app. You know the type, where you only have 1 real HTML page and it defines the main content areas of your site. Then the rest of the app is all JavaScript that is used to load content into those areas mostly through AJAX and client side templates (check out jsrender).

After doing the HireFlo rework, and seeing the rats nest of JavaScript that I created, I can definitely see the value of using a JavaScript MVC or MVVM framework to manage my JavaScript UI code. One framework that I’ve been looking into is Backbone.js. It’s a full-on MVC framework for JavaScript. How cool is that? The more I learn the more I like, and I think I might be refactoring HireFlo in the near future to use Backbone.

If you want to take a look at Backbone, check out the Backbone.js Wine Cellar Tutorial by Christophe Coenraets. It’s the best intro I’ve seen so far.

Awesome Startup Architecture List

noreply@blogger.com (Rudy Lacovara) — Sun, 13 May 2012 07:07:00 +0000

I wanted to put together my short list of awesome stuff you must have if you’re building a new web application on .Net. Each of these libraries is free, open source, and takes the simple, no-nonsense approach to getting things done that is essential in a startup.

Before I get into the list just let me say that jQuery isn’t on the list because I just assume that you’re already using it, and because it’s already included by default when you create a new .Net web app.

Dapper
You really can’t make the claim that there is any one right way to do data access... but this is the one right way to do data access. It was written by two of the guys at StackOverflow and it gives you reflection like functionality without reflection. They actually emit IL. It’s cool.
http://code.google.com/p/dapper-dot-net/

Automapper
When you find yourself writing that mapping code between your DTOs (or entities) and your ViewModels, save yourself some time and just use this incredibly useful convention based mapping library by Jimmy Bogard
https://github.com/AutoMapper/AutoMapper

NLog
It’s a good idea to bake logging into your app from the beginning. I have no idea why people still use Log4Net (confusing) when NLog (simple and awesome) is out there.
http://nlog-project.org/

Quartz.Net
Surprisingly powerful scheduler. Ideal for writing processors, console apps, and windows services that do things on a schedule.
http://quartznet.sourceforge.net/

Twitter Bootstrap
CSS and Javascript that gives you a whole UI framework and a bunch of themed widgets to build a modern webapp on top of. Before you write any UI code, spend a day experimenting with Bootstrap. It’s amazingly useful and it gives your app a super polished look.
http://twitter.github.com/bootstrap/

JsRender
Javascript templating done right. Indispensable if you’re writing a single page javascript app using Bootstrap, BackboneJs, or KnockoutJs.
https://github.com/BorisMoore/jsrender

dotLess
A .Net port of the Less CSS library. In the style of FakeGrimlock: IT MAKE CSS MORE AWESOME!!
http://www.dotlesscss.org/

jqModal
Simple jQuery based dialog boxes and modal popups.
http://dev.iceburg.net/jquery/jqModal/

The 5 Step Recipe for Building a Startup: Developer Edition

noreply@blogger.com (Rudy Lacovara) — Wed, 04 Apr 2012 15:20:00 +0000

Building a startup is hard. If you spend any time around newtech meetups or startup groups you’re constantly running into people who all seem to have the same problem, they’re a business person with this great idea, if only they could find a developer to build it. Seems like if you’re a developer and have a startup idea you’ve got it made, right? Wrong.

Developers have the skills to make their own vision a reality. That’s a huge advantage, but it’s also a problem. Because developers are really good at building things with code, the first thing they usually do is…write code. That’s the absolute wrong place to start, and that’s why most developers wind up producing something that nobody wants, with no users, and no idea where to go from there.

The key to not being “that developer” is to start with the business side. Don’t start writing code right away. Instead start thinking through your app, start talking to potential customers, start doing things like collecting emails that measure the interest in your idea. Now I’ve done this wrong a couple of times, but I like to think that I can be taught, and this post lists the 5 simple steps that I’ve learned for making a successful startup.

1. Mock

Buy a copy of Balsamiq mockups and mock your app. Not just the landing page, all of it. Every screen. This will force you to focus on the user experience side of your app (which is the only part the user cares about) and work through a lot of the logical problems, application flow problems, stuff like that. It will also give you something you can show to other people when describing your idea, which leads us to....

2. Talk

Take your mocks and show them to as many people as you can who are potential users. That part's important. You want to talk to people who are in your target market. Talk through the idea, talk through the app. Listen. Listen most to the people who love your idea and the people who hate your idea. Those are the people who care. Figure out what the key values that your product provides are from your customer's perspective. Also, as you get feedback, you should be updating the mockups incorporating the things that you'

This is the step that most developers will want to skip. Don’t. Talking to potential customers is the most important thing you can do as a Founder. It’s uncomfortable, it’s tough, you’ll find yourself fumbling for words and talking in circles trying to figure out how to describe your product. You’ll feel like an idiot for the first 10 or 20 conversations, but then something will start to happen. Just by sheer force of repetition you’ll find that you’re starting to figure out how to describe your product clearly, and you’ll find that you’re saying a lot fewer words while doing it. You’ll also develop a picture of what the benefits your product brings that seem to interest people the most. This is what you’re after. This whole process is necessary to teach you what your product really is and what your key value propositions are. It’s not easy but it’s absolutely necessary. If you’re not willing to do it then go work for somebody else, you’re not an entrepreneur.

3. Announce

Now that you have mocks, and you know what your key value props are (because you talked about it with many potential customers) it's time to put up the "coming soon" page. I would recommend doing a single page that describes the key value props and includes a screenshot. The screenshot isn't a real screenshot of your app. You don’t have an app yet. You can hire a designer to dummy it up from one of your mocks. I would recommend spending a few hundred dollars on 99designs. You can get every thing done at once, a good looking design for your "coming soon" page, a dummy screenshot, and a nice text treatment (no icon needed at this point) for the logo. Get this page up and start collecting emails of potential users.

If you're really ambitious and have something to say, you may also want to start a blog at this point. But don't blog about building a startup, blog about the problem space that your product is for. Blog about things that show your expertise in that area, and write things that your target customer will want to read because it's relevant to the problem that you solve. If you're building small business accounting software, then blog about small business accounting issues. Remember, the purpose of the blog is to attract and reach potential users, not other startup founders or the hackernews crowd.

4. Bizspark

If you’re building on the .Net stack you absolutely need to join Bizspark. Microsoft will give you free licenses to basically every product they make, including SQL Server, and a free MSDN gold subscription, for 3 years. They figure 3 years is long enough for you to get going so after that they want you to pay for new licenses, but here's the great part, they let you keep the licenses you're already using. So you don't wind up in a situation where you're just scraping by, then 3 years is up and you have a big Microsoft bill to pay. It’s great. Microsoft has basically taken the cost factor completely out of the equation for new startups, and it ‘s a perfect program for a .Net developer who’s building their first startup.

It may seem odd that I didn’t start with Bizspark as step 1. There’s a reason. You’re not ready to join Bizspark until you reach this point. Keep in mind that Microsoft doesn’t just accept everyone who applies to Bizspark. You actually are required to have a company, a website, and a sponsor. Your sponsor can be a company that has some kind of partner relationship with Microsoft, like Rackspace, or it can be a person employed by Microsoft, like Aaron Stannard who is the startup evangelist for Microsoft (aaronst@microsoft.com). So you’ll need be ready to describe your app and you’ll need to have a website up, not a completed product, just some type of website for your company. The “coming soon” site is fine for this requirement.

5. Build

Now that all that other stuff is done, now you start building the first version of your app. Note that you should continue to talk to potential users, you may even want to establish a small group of users who you show your app to as it's being built. The key point is that step 1 is not finding a developer, step 1 is not start building. Building the app doesn't happen until step 4.

While building, do not follow the common advice "don't worry be crappy". Nothing you produce should ever be crap. Instead I would recommend taking the 37Signals approach "Build half a product, not a half-assed product". In other words, your MVP should have half the features you want it to have, but what you do build should be awesome.

To give some technical specifics, I highly recommend building on MVC, not WebForms, and I recommend using a virtual server like an OrcsWeb cloud server or a Rackspace cloud server, not Azure. The reason is that for an early stage startup speed and flexibility are paramount. You’re going to be making major changes to your app on a regular basis for the first few years. MVC is a framework that embraces change and makes those changes faster and easier than Webforms.

The same principle applies in reverse for Azure. Azure solves the problem of scale, but scale isn’t your problem. My experience, and the experience of a couple of other startups I have first hand experience with, is that Azure slows down development. I think it makes your development slower and less flexible. Debugging is a pain, setting up your environment is a pain, deployment is a pain, the DAL becomes something you actually have to spend time and effort on, plus… what if you decide you need to run Redis or MongoDb or some other open source thing? Microsoft will probably make it possible so they can check off that box on the feature list and claim that it works, but odds are it will be a whole lot harder than just installing that software on a regular server. What if you end up needing to have 10 different services running? Is each of those going to be a separate Azure instance, and thus an additional bill? You don’t want to spend time worrying about stuff like that in the early days when your app can change massively every 2 weeks.

At the beginning, I think you’re much better off just buying a single cloud server. It runs just like a real windows server, it provides a familiar environment, it’s going to be a solid single platform that you can load up with whatever open source code and services you need (without paying extra), and you can scale it up if you you need to. Later, when you know what all the pieces of your app are and you have customers pushing the limits of your cloud server, that is the right time to look at Azure.

Should you use .Net for your Startup Company?

noreply@blogger.com (Rudy Lacovara) — Sat, 24 Mar 2012 21:14:00 +0000

I get this question a lot, from both people inside and outside of the .Net community, and it came up again the other day on Quora. Below is my answer. Please keep in mind that this is my opinion, and I can’t promise that I’m right, but I can promise that this is what I really think and if you walked up on the street and asked me “should I use .Net for my startup?” this is what I would tell you.

In spite of the fact that I’m a pretty high end .Net guy, I would caution new startups that are considering using .Net. I think .Net is awesome, and I use it for my own startup HireFlo. However there's one big landmine that could handicap your startup before it even gets going, webforms.

You see, ASP.Net development is divided into 2 camps, and there are major architectural differences, and cultural differences between them.

ASP.Net MVC is awesome for startups

ASP.Net MVC is the awesome new framework that is heavily influenced by Rails and is the platform of choice for startups like StackExchange. It’s a breath of fresh air for skilled .Net developers who want a framework that embraces the way the web works instead of struggling against it. This camp is full of people who care about the craftsmanship of writing code and who like to actually ship software. A programmer in this camp is likely to be familiar with lots of open source projects, and is most likely a great fit for your startup.

WebForms is death for startups

The other camp is WebForms, a festering bog of evil, spaghetti code, and hate. It's the domain of corporate developers who prize process and documentation over shipping code and hope that the glacial pace of WebForms development will hide the fact that they haven't shipped a product in years and can't tell the difference between an HTTP GET and an HTTP POST. I mean it. WebForms and it’s poisonous PageLifeCyle pattern are a sink hole of productivity and they are going to resist you every step of the way as your startup struggles to find the right fit between your application and your market (product market fit). Plus Webforms and the cursed UpdatePanel make it incredibly difficult to do the fancy new style of client side javascript app that tools like Backbone.js, Knockout.js, and jQuery have made possible.

So the answer is…Heck yes you should (as long as it’s .Net MVC)

So .Net is awesome, C# is amazing, MVC3 is hugely productive, if you find a .Net developer who works with those, I'd recommend using them. But you've got to be careful. A WebForms developer with a corporate dev background is most likely going to be death for your startup.

There are three other points I want to add that I think are relevant.

Don’t use Azure for an early stage startup

Microsoft is really pushing Azure as a platform for startups. I strongly recommend that early stage startups do not use Azure. Azure development is significantly slower than straight up .Net development that runs on a windows server or VS. Deployment takes longer, debugging takes longer, the Azure environment is difficult to replicate on your local dev computer, backups are a pain, and you're locked into a single hosting provider. It all ads up to a lot of inflexibility and friction at a time when your startup needs flexibility and speed over all other things. Azure can have a place later, when scale is your problem, but at the beginning of your startup that isn't the issue.

The ASP.Net MVC/C#/SqlServer stack scales like a madman

Listen to the middle days of the first StackOverflow podcast and you'll hear that for a long time they ran StackOverflow on a single server. They were serving a million uniques with the web app and database running on a single box! It wasn’t even a very big box. This is consistent with my experience with .Net. It scales well. So, if your startup does make it, you'll probably have a much easier time scaling the .Net stack than you would with say Ruby or PHP.

Bizspark is proof that Microsoft loves programmers and startups

If you want to build a startup on the Microsoft stack, they will give you free licenses to basically every product they make, including SQL Server, and a free MSDN gold subscription, for 3 years. They figure 3 years is long enough for you to get going so after that they want you to pay for new licenses, but here's the great part, they let you keep the licenses you're already using. So you don't wind up in a situation where you're just scraping by, then 3 years is up and you have a big Microsoft bill to pay. They don't do that, they just let you keep using the software. So Microsoft has basically taken the cost factor completely out of the equation for new startups.

HireFlo, an actual startup built on .Net

noreply@blogger.com (Rudy Lacovara) — Sat, 03 Mar 2012 18:28:00 +0000

What have I been doing for 10 months?

I’ve gotten a number of emails asking for part 2 of the ValidationLabel post that I wrote 10 months ago, and a number of people have said there’s problems with the sample code on aapl.codeplex.com/ and asked for updated code samples. Sorry about that. There is a reason I’ve dropped off the face of the earth for the last year.

That reason is HireFlo. I took all of the principles I’ve been discussing here, the techniques for creating an architecture that embraces change and can quickly adapt to massive changes to the application, and I put them into practice with my own startup. HireFlo is an Applicant Tracking System designed for the small business market. It takes all the tools that recruiters use like jobsites, aggregators, social networks, resume databases, and wraps them up in a simple app designed for small business users who probably don’t even know that most of those tools exist.

I’ve been building HireFlo for about 18 months. It’s an ASP.NET MVC app that uses AAPL for persistence, and I have to say the MVC and AAPL combination has been a huge success. I’m on version 4 of the app and I’ve made some massive changes since version 1. That’s what happens in startups, and why an architecture that embraces change but can still scale, is so incredibly important. Each time I’ve made major changes, MVC and AAPL have made the experience as quick and painless as possible.

The last major change was November 2011 when I decided to completely throw away the old UI, and rewrite a simpler, streamlined version based on a Kanban board, and also make it a single page Javascript app in the style of backbone.js and knockout.js apps. I completed the complete rewrite of every page, controller, HTML template, CSS file, and Javascript file in the app in just 14 days. This would never, never, NEVER have been possible if I’d used WebForms or a more Microsofty approach to persistence that used a bunch of sprocs and hard coded mapping logic in my data and business layers.

So anyway HireFlo is up. It’s been officially launched since August 2011 but I would say the real launch was end of December 2011 when I released version 4 of the app and the redesigned marketing site. So far there about 300 companies using it. It’s a start.

What’s coming next

So the first thing I need to do is write Part 2 of my last post. This part details how to take a helper method and turn it into a strongly typed helper method by using expressions and lambdas because…. well because doing that is awesome. After that I’ve got a lot of tech problems that I’m solving for HireFlo and I’d like to share them. I just need to find some time and get back on a somewhat regular schedule. BTW, I’d also be happy to give updates on HireFlo if you’re interested. Just let me know. -rudy

Creating a Validation Label Helper – Part 1

noreply@blogger.com (Rudy Lacovara) — Sun, 17 Apr 2011 15:05:00 +0000

Presenting validation messages is a pain. Programmers usually tack validation messages on to a User Interface is a way that gets the job done, but also is clunky and detracts from the overall UI design. Let me show you an example. Below is a really simple form from my HireFlo application.

Here’s the markup that creates the form fields and buttons. I just use a simple layout with a label followed by a validation message, and a Textbox created using MVC’s strongly typed Html helper methods.

    <div class="form-row">
        <label class="form-label" for="folder.FolderName" title="The name for this folder" >
           Enter a name for your folder:
        </label>
        <%= Html.ValidationMessageFor(model => model.Folder.FolderName)%><br /> 
       <%= Html.TextBoxFor(model=>model.Folder.FolderName, new {style="width:300px;", maxlength="50"})%>
    </div>
                             
    <div>
        <input id="SaveButton" type="submit" class="button-green" value=" Save Folder " />  
        <input id="CancelButton" type="button" class="button-gray" value=" Cancel " />           
    </div>

The only validation on the form is that you have to enter a name for the folder you’re creating. If you don’t, you get a validation error that looks like this.

While functional, this doesn’t look very good. The validation message for the field is just kind of tacked on after the label, but where else am I going to put it. I could put it underneath the field, beside the field, but these options all result in extra text, and an interface that looks cluttered to me.

I’ve become more and more of a minimalist when it comes to UI design. I want to put as few things on the page as possible. So I started thinking, what is the function of the label on a form? It’s there to tell the user what data to put into a given field. Now what is the purpose of the label on a form with a validation message? The validation message and the label are both trying to tell you what goes into a given field. Maybe it makes sense to combine them. So I tried that out and the results look like this.

All right. This is looking better to me. The interface is clean and clear. I have to make sure that my validation messages always start with the field name, and they need to be fairly short, but those both sound like good design constraints so I’m willing to work with them. The ValidationLabel is now the standard way that I’m going to handle error messages in my application.

So how should I implement the ValidationLabel? I’m already using the ValidationMessage Html helper method and that works pretty well, I’ll just customize that and add the Label functionality to it. I write an extension method for HtmlHelper that will take all of the parameters required to behave as both a label and an error message. If there’s no error, my helper will render a label. If there is an error, my helper will render an error message. Simple. Here’s how my markup will look with the new helper.

    <div class="form-row">
        <%= Html.ValidationLabel("Folder.FolderName", "Enter a name for your folder", null)%>
        <%= Html.TextBoxFor(model=>model.Folder.FolderName, new {style="width:300px;", maxlength="50"})%>
    </div>
                            
    <div>
        <input id="SaveButton" type="submit" class="button-green" value=" Save Folder " />  
        <input id="CancelButton" type="button" class="button-gray" value=" Cancel " />           
    </div>

Hmmmm, cleaner interface and cleaner markup. I like the way this is going. Now let’s write the ValidationLabel helper method. It’s actually surprisingly simple. Here’s the code.

        // ValidationLabel
        public static MvcHtmlString ValidationLabel(this HtmlHelper htmlHelper, 
                                                    string modelName, 
                                                    string labelText, 
                                                    IDictionary<string, object> htmlAttributes)
        {
            if (modelName == null) { throw new ArgumentNullException("modelName"); }
            ModelState modelState = htmlHelper.ViewData.ModelState[modelName];
            ModelErrorCollection modelErrors = (modelState == null) ? null : modelState.Errors;
            ModelError modelError = ((modelErrors == null) || (modelErrors.Count == 0)) ? null : modelErrors[0];
            // If there is no error, we want to show a label.  If there is an error,
            // we want to show the error message.
            string tagText = labelText;
            string tagClass = "form_field_label_normal";
            if ((modelState != null) && (modelError != null))
            {
                tagText = modelError.ErrorMessage;
                tagClass = "form_field_label_error";
            }
            // Build out the tag
            TagBuilder builder = new TagBuilder("span");
            builder.MergeAttributes(htmlAttributes);
            builder.MergeAttribute("class", tagClass);
            builder.MergeAttribute("validationlabelfor", modelName );
            builder.SetInnerText(tagText);
            return MvcHtmlString.Create(builder.ToString(TagRenderMode.Normal));
        }

Here’s how it works. This is an extension method on the HtmlHelper class so the first parameter is the HtmlHelper. Next we pass in the modelName, the labelText, and a dictionary of Html Attributes that can be used for custom formatting. The first thing we do is make sure that we have a modelName and then check to see if there’s an item in the ModelErrorsCollection for that modelName.

Note that modelName is not the name of your entire model, it’s the name of your field. In my folder form example the model name is “Folder.FolderName”, because I have view model class that contains a Folder object and that Folder object has a property called FolderName. If you’re ever uncertain about what a field’s modelName is, just load up your page and look at the markup for your input tag. It will look something like this, and your model name is the value that ASP.Net MVC puts in the name attribute for your tag.

<input id="Folder_FolderName" maxlength="50" name="Folder.FolderName" style="width:300px;" type="text" value="">

Let’s get back to our method. Now we know we have a modelName, we know what the modelName is, and we’ve checked the ModelState’s ModelErrorsCollection to see if we have any errors for our modelName. From this point on the logic is pretty simple. If we do have an error we set the tagText to the error message text and we set the tagClass to “form_field_label_error”. If we don’t have an error we set the tagText to the labelText and we set tagClass to “form_field_label_normal”. Then we just use a TagBuilder to render out a span using the tagText and tagClass that we defined. I also added a custom “validationlabelfor” attribute that holds the model name and is used for client side (javascript) validation.

So that it! We now have a method that renders our field label, and will magically present any error messages for that field if they exist in the ModelState (In case you’re wondering, ASP.Net MVC automatically stores all errors resulting from model validation or model binding in the ModelState). Plus the syntax is really clean and simple. A ValidationLabel, input pair is just 2 lines.

<%= Html.ValidationLabel("Folder.FolderName", "Enter a name for your folder", null)%>
<%= Html.TextBoxFor(model=>model.Folder.FolderName, new {style="width:300px;", maxlength="50"})%>

There is one more thing I want to do though. Notice how my ValidationLabel requires me to manually enter the modelName “Folder.FolderName”, but the TextBoxFor method uses a lambda instead? That’s pretty cool, plus it eliminates the need to use a “magic string” in our UI code. For Part 2 I’m going to cover how we can create our very own strongly-typed ValidationLabelFor method that will take a lambda as a parameter instead of that modelName string.

.Net as an Awesome Startup Platform

noreply@blogger.com (Rudy Lacovara) — Fri, 04 Mar 2011 20:20:00 +0000

This is the short (10 slide) deck from my presentation at the Denver Tech Trifecta. I used slideshare to share the presentation. If you click the menu button there is a View Fullscreen option.

First, this is all my own personal opinion, not the views of Microsoft or anybody associated with the Tech Trifecta. If it seems a little terse that’s because most of the content was spoken. I favor short slides and lots of talking with code demos. The main theme of the presentation was that classic ASP was an awesome startup platform. Then Microsoft pushed out ASP.Net WebForms, and they took over corporate software development, but they lost the startup community. I think the reason is that it takes too long and costs too much money to build anything real with WebForms (it’s also painful to make significant changes to an existing app). Open source tools like Ruby/Rails and PHP on the other hand provide a much faster and lower friction development experience, plus they’re free.

But now that’s all changed. Microsoft is supporting startups in a big way with the BizSpark program which will basically give your startup every product that Microsoft makes for FREE for 3 years. The biggest and most important change for me personally is ASP.Net MVC. Three years ago I was ready to throw away a decade of work on Microsoft platforms and go do Ruby/Rails. WebForms is a never ending source of bad code and frustration for me. Then ASP.Net MVC came along and it was like a love letter written to web developers. Every place I turned the MVC Framework had given me just the right levels of abstraction and access to support me in what I needed to code. Plus there’s no ridiculous artificial page lifecycle that tries to make the web look like Windows, awesome! I love MVC and because of it I am excited to keep building on the .Net stack. Unfortunately, now that I’ve experienced the light, I cry every time I’m forced to go back and work in WebForms. One place that I do think the page event lifecycle model works well is Silverlight. Silverlight gives you the ability to write apps with a user experience that is so good that I think building with it will become a competitive advantage for line of business apps. All these changes add up to Microsoft once again being an awesome platform to build your web startup on.

Net as an awesome startup platform

View more presentations from rlacovara.

Startup Links

For those who are just looking for the resource links at the end of the presentation, here they are. These are all great startup resources and well worth checking out, especially if you live in the Boulder/Denver area. -rudy

newtech Meetup
About 4 tech startups present in Denver and Boulder every month. The Denver meetup meets on Auraria Campus. The Boulder meetup has free beer!
http ://www.bdnewtech.com /

TechStars
One of the first Y Combinator style accelerator programs for early stage startups.
http ://www.techstars.org /

Open Angel Forum
Angel Investors meet in Boulder about twice a year to hear pitches from early stage startups and possibly fund them.
http ://openangelforum.com /

StartupDigest
Weekly email digest by Chris McCann that tracks all of the tech startup related events in the Boulder / Denver area.
http ://startupdigest.com /

Lean Startup
Best practices for launching a startup by Eric Reis. Must see video here:
http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2329

Feld Thoughts
Amazing blog by Brad Feld, Boulder resident and vocal venture capitalist.
http ://www.feld.com/wp/

Easy XML Feed using LINQ to XML and ASP.Net MVC2

noreply@blogger.com (Rudy Lacovara) — Mon, 21 Feb 2011 16:06:00 +0000

I recently had to create an XML feed for an app I’m working on. The use case is simple. I have a database of jobs and I need to provide an XML feed containing any jobs that were posted in the last week. I’m going to need to do this for several different 3rd party companies that each use a different XML format, so I want to do a little object oriented design to make sure that creating a feed using a new XML format is as easy as possible.

I’ve always heard that LINQ to XML makes working with XML much easier, and I know that MVC2 makes it easy to package up and return any type of data so I’m going to use them. It should make for a pretty painless XML feed. Let’s see.

Starting at the controller

I want to start by writing the consuming code and let that drive the shape of my business layer code. That means I start by writing a Feed action method on one of the controller classes in my MVC2 app. I want to return XML data so i pick a return type of ContentResult, which will allow me to set the content type to “text/xml”. The logic for this method is simple. I have a feedKey that distinctly identifies the 3rd party company / XML schema that I want to generate a feed for. All companies will hit the same action method, but they’ll pass in a different feedKey, which will result in them getting the appropriate XML feed. My action method takes the feedKey as a parameter and then passes it to a factory method on my FeedService class that returns an instance of the appropriate FeedBuilder object. The FeedBuilder is the class that I’ll use to encapsulate logic for getting job data and formatting it as an XML document. I’ll have a different FeedBuilder for each feedKey. The design is a variation on the Strategy pattern. So, back to controller action method. Here’s what it looks like.

        // Feed
        [AcceptVerbs(HttpVerbs.Get)]
        public ContentResult Feed(string feedKey)
        {
            var feedBuilder = this.FeedService.GetFeedBuilderFor(feedKey);
            return this.Content(feedBuilder.GetFeedXml(), "text/xml");
        }

The Factory Method

The factory method on my FeedService is pretty straight forward. It just returns an instance of the right FeedBuilder for the feedKey that we pass in to it. Note that the return type is FeedBuilderBase which is an abstract class that defines the shape of a FeedBuilder.

        // GetFeedBuilderFor
        public virtual FeedBuilderBase GetFeedBuilderFor(string feedKey)
        {
            switch(feedKey)
            {
                case FeedKey.SimplyHired:
                    return new SimplyHiredFeedBuilder();
                case FeedKey.Indeed:
                    return new IndeedFeedBuilder();
                default:
                    return new EmptyFeedBuilder();
            }
        }

The FeedBuilderBase

Remember we’re using a Strategy pattern. Usually you would define the shape of your strategy class with an interface, but I’m going to use an abstract class because there’s some boilerplate code that will be the same for all of my concrete FeedBuilder classes and I want to implement that code in a single base class. My abstract class is FeedBuilderBase and it looks like this.

    public abstract class FeedBuilderBase
    {
        //**************************************************************************************
        // PROPERTIES
        //**************************************************************************************
 
        // FeedService
        private FeedService _feedService;
        public virtual FeedService FeedService
        {
            get { return NewIfNull(_feedService); }
            set { _feedService = value; }
        }
 

        //**************************************************************************************
        // ABSTRACT METHODS
        //**************************************************************************************
 
        // GetFeedData
        public abstract List<JobFeedItem> GetFeedData();
  
        // BuildXmlFor
        public abstract XDocument BuildXmlFor(List<JobFeedItem> list);
 

        //**************************************************************************************
        // HELPER METHODS
        //**************************************************************************************
        
        // NewIfNull
        public virtual T NewIfNull<T>(T obj) where T : new()
        {
            if (obj == null) { obj = new T(); }
            return obj;
        }
  
        // GetFeedXml
        public virtual string GetFeedXml()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriterUtf8(sb);
            xdoc.Save(sw);
            return sb.ToString();
        } 
    }

I have two abstract methods, GetFeedData and BuildXmlFor. These methods represent the two things that change for each XML feed. Each company has slightly different business rules around what data they want to pull and each company has it’s own proprietary XML format. These 2 methods will have to be overridden whenever we create a concrete FeedBuilder.

You’ll also notice that we implemented GetFeedXml which is the method we called in our action method to get the text of our XML document. GetFeedXml should have been very easy to write, but it took a little time due to an unexpected encoding problem that I described in How to Create XML in C# with UTF-8 Encoding.

The Concrete FeedBuilder

Now I’m ready to create my first concrete FeedBuilder class. I’m going to start with a jobsite aggregator called SimplyHired. So I create a new SimplyHiredFeedBuilder class that inherits from FeedBuilderBase an I override my GetFeedData and BuildXmlFor methods. GetFeedData is just a wrapper for the appropriate data access method in my FeedService class. GetFeedData returns a list of JobFeedItem objects. JobFeedItem is a simple Data Transfer Object (DTO) that I created to contain the aggregate of data needed to build a feed item.

BuildXmlFor is more interesting. It takes a list of JobFeedItem objects returned by our GetFeedData method and transforms it into an XDocument using a LINQ query. The LINQ query is just a select with a bunch of nested XElement constructors that create the XML elements needed to represent each job. We then take the expression created to represent the list of jobs as XElements and we wrap it in a single top level XElement called, wait for it…. jobs. BTW, it is at this point that the LINQ expression actually executes. Up to now it has just been an expression. As soon as we use it in the constructor for our jobs XElement the expression compiles and executes.

So, we now have an XML tree contained in a single XElement called jobs. To complete the method we just wrap the jobs XElement in a new XDocument, return the XDocument, and we’re done.

    public class SimplyHiredFeedBuilder:FeedBuilderBase
    {
        //**************************************************************************************
        // FEEDBUILDERBASE OVERRIDES
        //**************************************************************************************
 
        // GetFeedData
        public override List<JobFeedItem> GetFeedData()
        {
            return this.FeedService.GetJobFeedItemsForSimplyHired();
        }
 
 
        // BuildXmlFor
        public override XDocument BuildXmlFor(List<JobFeedItem> list)
        {
            var xmlExpression = 
                from JobFeedItem j in list
                select new XElement("job",
                    new XElement("title", j.JobTitle),
                    new XElement("job-code", j.JobGuid),
                    new XElement("job-board-name", j.CompanyName),
                    new XElement("job-board-url", j.CompanyJobPageUrl),
                    new XElement("detail-url", ""),
                    new XElement("apply-url", GetApplyUrl(j.CompanyKey, j.JobGuid)),
                    new XElement("job-category", j.JobCategory),
                    new XElement("description",
                        new XElement("summary", j.JobDescription),                   
                        new XElement("required-skills", ""), 
                        new XElement("required-education", ""),
                        new XElement("required-experience", ""),
                        new XElement("full-time", j.IsFullTime),
                        new XElement("part-time", j.IsPartTime),
                        new XElement("flex-time", ""),
                        new XElement("internship", j.IsInternship),
                        new XElement("volunteer", ""),
                        new XElement("exempt", ""),
                        new XElement("contract", j.IsContract),
                        new XElement("permanent", j.IsPermanent),
                        new XElement("temporary", j.IsTemp),
                        new XElement("telecommute", "")
                        ),
                    new XElement("compensation",
                        new XElement("salary-range", ""),                   
                        new XElement("salary-amount", ""),
                        new XElement("salary-currency", ""),
                        new XElement("benefits", "")
                        ),
                    new XElement("posted-date", GetPostedOnDate(j)),
                    new XElement("close-date", GetClosedOnDate(j)),
                    new XElement("location",
                        new XElement("address", ""),  
                        new XElement("city", j.JobCity),  
                        new XElement("state", j.JobState),  
                        new XElement("zip", ""),  
                        new XElement("country", "") 
                        ),
                    new XElement("contact",
                        new XElement("name", j.ContactName),
                        new XElement("email", j.ContactEmail),
                        new XElement("hiring-manager-name", ""),
                        new XElement("hiring-manager-email", ""),
                        new XElement("phone", j.ContactPhone),
                        new XElement("fax", "") 
                        ),
                    new XElement("company",
                        new XElement("name", j.CompanyName),
                        new XElement("description", j.CompanyDescription),
                        new XElement("industry", ""),
                        new XElement("url", j.CompanyUrl)
                        )
                    );
            var jobs =  new XElement("jobs", xmlExpression);
            var declaration = new XDeclaration("1.0", "utf-8", null);
            return new XDocument(declaration, jobs);
        }     
    }

Conclusion

So I built it, tested it, and it works. Except for the hour that I spend trying to figure out my UTF-8 encoding problem, the LINQ / XElement method was a relatively painless way to create XML. I love the simplicity of the MVC action method and the fact that it gives me a ContentResult type that gives me access the right parts of the stack. The LINQ / XElement query was concise and the style of passing element data in XElement constructors makes it hard to mess up the document structure. My conclusion is that I like it and I’ll definitely be using this technique again.

How To Create XML in C# with UTF-8 Encoding

noreply@blogger.com (Rudy Lacovara) — Mon, 14 Feb 2011 06:26:00 +0000

I can’t believe that I spent over an hour on this problem. It’s one of those things that you can’t understand how it’s possible that it doesn’t just work. But it doesn’t, so here’s the solution I found for anyone who might have the same issue.

The Problem: My XML refuses to use UTF-8 encoding

I have some code where I’m doing a query and creating an XML doc from the data that’s returned. I used LINQ to XML to create my XML and package it up in an XDocument. The XDocument allows me to set the XML version and the encoding. Here’s the code I used to create and return my XDocument.

            var declaration = new XDeclaration("1.0", "utf-8", "yes");
            return new XDocument(declaration, jobs);

So far so good. Now the last (and what should be trivial) step is to write that XML to a string and return it. This is where things go wrong. To get an XDocument object to write out a full XML document you call it’s Save method. No problem, the Save method has an overload that takes a StringWriter parameter so I’ll just create a StringWriter and a StringBuilder and I’ll be in business. Here’s my initial code.

        // GetFeedXML
        public virtual string GetFeedXML()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriter(sb);
            xdoc.Save(sw);
            return sb.ToString();
        }

That all looks good. So I run it and here’s the result.

    <?xml version="1.0" encoding="utf-16" standalone="yes" ?>
    <jobs>
      <job>
        <title>Audit Engagement Manager</title>

What the heck is that??? UTF-16?? It’s like the UTF-8 encoding that I set in my XDocument was completely ignored and .Net decided to use UTF-16 encoding. How can this not work? Where else do I even have an option to set the encoding? The StringWriter won’t let me set the encoding, the StringBuilder certainly won’t let me set the encoding.

So I started Googling and found that other people were having the same problem and I found several solutions that sounded plausible. Most of them centered around creating an XMLWriter with a settings object that would allow me to set the encoding. All of these required a fair bit of extra code and I never got one of them to actually work, but they did get me looking at the StringWriter as the place where I needed to set my encoding.

The Solution: Subclass StringWriter

At this point I’m looking into progressively more and more complex solutions when I stumble across a post by Ian Dykes called Writing XML with UTF-8 Encoding using XmlTextWriter and StringWriter. Ian basically says to create a new StringWriterWithEncoding class that inherits from StringWriter but allows you to set the encoding in the constructor. I used the same idea and created the StringWriterUtf8 class below. Instead of taking the encoding in a constructor, I opted to make the Encoding property always return UTF8.

    public class StringWriterUtf8 : StringWriter
    {
        public StringWriterUtf8(StringBuilder sb) : base(sb)
        {
        }
 
        public override Encoding Encoding
        {
            get { return Encoding.UTF8; }
        }
    }

Now I just need to use StringWriterUtf8 in my code instead of StringWriter and I’ll be using UTF-8 encoding in my writer. I did it, I tested it, and it worked. My XML output now looks like this:

    <?xml version="1.0" encoding="utf-8" standalone="yes" ?>
    <jobs>
      <job>
        <title>Audit Engagement Manager</title>

Thank you Ian for understanding more about this than I do and taking the time to put the solution out on your blog. By the way, I think it’s worth mentioning that several people posted more “settings type” solutions in the comments to Ian’s post. I tried them. They didn’t work for me. The only thing that did work was Ian’s idea of subclassing the StringWriter. So I now have a working method and here’s the final version of GetFeedXml that uses my StringWriterUtf8 class.

        // GetFeedXML
        public virtual string GetFeedXML()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriterUtf8(sb);
            xdoc.Save(sw);
            return sb.ToString();
        }

Refactoring a TrySave Method

noreply@blogger.com (Rudy Lacovara) — Tue, 08 Feb 2011 07:34:00 +0000

When writing code, it’s unusual for me to get it right the first time. Typically I’ll just write a big block of statements that do something that I’ll need done. Then I’ll verify that it works. Then I’ll look at it, notice what an ugly mess it is, and refactor it so that the poor developer who inherits this code (and that developer might very well be me) will have some hope of maintaining it. Here’s a typical example that came up this week.

The scenario

I’m working in a controller class in an ASP.Net MVC2 application. I have some action methods that need to save the data passed in via a signup form. The signup form collects all of the data needed to create a new company and a new user in my application. So my signup form is a composite of Company data, User data, an other data that’s needed for the UI (like a ConfirmPassword field). I’ve chosen to encapsulate all of this stuff that the signup form needs in a view model class called SignupVM. It looks like this.

    public class SignupVM
    {
        public User User { get; set; }
        public Company Company { get; set; }
        public string ConfirmPassword { get; set; }
        public string FormError { get; set; }
        public string FormMessage { get; set; }
 
        public SignupVM()
        {
            this.User = new User();
            this.Company = new Company();
            this.ConfirmPassword = string.Empty;
            this.FormError = string.Empty;
            this.FormMessage = string.Empty;
        }
    }

So that part works really well. Here’s where it starts to get a little messy. I have a couple of Action Methods in my controller that receive a SignupVM that’s been populated with data by the View (or more accurately, the ModelBinder) and they need to save that data. Saving data involves three main steps. First I need to validate the Company, the User, and my ConfirmPassword field. Second I need to actually save the data, but I need to save the Company first and then make sure I set the correct CompanyGuid on the User before I save the User data. Third, I have some standard data that needs to get created for all new signups. Right now the only standard data is the GeneralApplicantsJob but I know that there will be a need for more standard data down the road. To handle this stuff, I created a TrySave() method in my controller class that looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            var userValidator = new UserValidator();
            var companyValidator = new CompanyValidator();
            var isError = false;
            // validate user
            try
            {
                userValidator.Validate(vm.User);
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetUserFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "User");
            }
            // validate company
            try
            {
                companyValidator.Validate(vm.Company);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetCompanyFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "Company");
            }
            // validate vm level fields
            try
            {
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.CopyToModelState(ModelState, "");
            }
            // if error, return false
            if (isError) { return false; }
            // if no error let's save the Company
            this.CompanyService.Save(vm.Company);
            // set the companyguid for the user
            vm.User.CompanyGuid = vm.Company.CompanyGuid;
            //save the user
            this.UserService.Save(vm.User);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(vm.Company.CompanyGuid);
            return true;
        }

Let’s Refactor

So, this block of code isn’t unmanageable, but I could definitely see it becoming unmanageable if I leave it like this and it gets added to a few times. Plus, it’s not immediately clear what this code is doing. What am I returning? Where am I returning it? What logic decides what my return value is? Looking at it I realize that I’m writing in a very Imperative style that puts the focus on the individual steps of “how” to do something, instead of writing in a Declarative (or Functional) style that puts the focus on “what” is being done. The code above is just a big list of steps that are a little difficult to follow without the comments. I’m just validating data and saving data, so why do I have all of these statements in my method? I can do better.

First let’s extract all of the validation code into an IsValidSignup() method. Just that one refactoring helps quite a bit. Now our TrySave method looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            // if no error let's save the Company            
            this.CompanyService.Save(vm.Company);            
            // set the companyguid for the user            
            vm.User.CompanyGuid = vm.Company.CompanyGuid;            
            //save the user            
            this.UserService.Save(vm.User);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(vm.Company.CompanyGuid);
            return true;        
        }

The next step is to refactor the data saving code. There’s two things that bother me about this code. First it’s critical that the the entities (Company and User) are saved it the right order, and that the user.CompanyGuid needs to be set in between the saves. Second, it’s critical that the GeneralApplicantsJobForCompany get created after the Company has been saved, plus I know that there will be a need for more standard data later on, which means at some point a developer is going to have to modify this helper method in a controller class to make sure signups are created with the right standard data. That feels like something that should be in my business layer, not in my UI code. So, I decided to extract all of the persistence logic (including the creation of standard data) to a new method on my SystemService class in my business layer. The mehtod is called SignupNewCompany() and now my TrySave method looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            this.SystemService.SignupNewCompany(vm.Company, vm.User);
            return true;
        }

This looks much cleaner to me. The focus is on what I’m doing instead of how I’m doing it, and the intent of the code is much more clear even though it no longer has comments. Now I do have three methods instead of one, but each of those three methods does one thing that is easy to understand at a glance. Best of all, I realized that I had put some logic in my UI that really needed to live in my business layer where it would be easier to reuse and easier to maintain. For anyone who’s interested, here are all 3 methods in their final form.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            this.SystemService.SignupNewCompany(vm.Company, vm.User);
            return true;
        }

        // IsValidSignup
        public bool IsValidSignup(SignupVM vm)
        {
            var userValidator = new UserValidator();
            var companyValidator = new CompanyValidator();
            var isError = false;
            // validate user
            try
            {
                userValidator.Validate(vm.User);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetUserFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "User");
            }
            // validate company
            try
            {
                companyValidator.Validate(vm.Company);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetCompanyFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "Company");
            }
            // validate vm level fields
            try
            {
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.CopyToModelState(ModelState, "");
            }
            return !isError;
        }

        public void SignupNewCompany( Company newCompany, User newUser)
        {
            // create the new company
            this.CompanyService.Insert(newCompany);
            // create the new user
            newUser.CompanyGuid = newCompany.CompanyGuid;
            this.UserService.Save(newUser);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(newCompany.CompanyGuid);
        }

How To Not Hate Dependency Injection.

noreply@blogger.com (Rudy Lacovara) — Mon, 17 Jan 2011 17:57:00 +0000

The Dependency Injection pattern is all the rage these days and more and more DI containers are becoming part of how we build .Net applications. I still have a love / hate relationship with Dependency Injection. I love the idea of being able to inject any external dependencies into my classes, thereby breaking those dependencies, but I hate the idea of creating an interface for every class that I need to inject, then maintaining that interface as well as any classes that implement it. Let me show you what I mean.

Dependency Injection on an MVC Controller Base Class With Interfaces

Typical scenario, I have an ASP.Net MVC2 application that needs to get data from a database. All of my data access code is contained in service classes like UserService, SystemService, CompanyService, etc. I don’t want to manually declare and instantiate whatever service classes I need in each of my controller classes, so I created an abstract base class for my controllers called AtsControllerBase which has properties that will provide instances of my service classes using a lazy creation pattern. Here’s an abbreviated version of the base class.

    public abstract class AtsControllerBase : Controller
    {
        //******************************************************
        // PROPERTIES
        //****************************************************** 
 
        // StateBox
        private HfStateBox _stateBox;
        public HfStateBox StateBox
        {
            get { return NewIfNull(_stateBox); }
            set { _stateBox = value; }
        }
        // AuthToken
        public AuthToken AuthToken { get { return StateBox.AuthToken; } }
        // ApplicantService
        private ApplicantService _applicantService;
        public ApplicantService ApplicantService
        {
            get { return NewIfNull(_applicantService); }
            set { _applicantService = value; }  
        }
        // CompanyService
        private CompanyService _companyService;
        public CompanyService CompanyService
        {
            get { return NewIfNull(_companyService); }
            set { _companyService = value; }
        }
        // FolderService
        private FolderService _folderService;
        public FolderService FolderService
        {
            get { return NewIfNull(_folderService); }
            set { _folderService = value; }
        }        
        // JobService
        private JobService _jobService;
        public JobService JobService
        {
            get { return NewIfNull(_jobService); }
            set { _jobService = value; }
        }
        // SecurityService
        private SecurityService _securityService;
        public SecurityService SecurityService
        {
            get { return NewIfNull(_securityService); }
            set { _securityService = value; }
        }
        // SystemService
        private SystemService _systemService;
        public SystemService SystemService
        {
            get { return NewIfNull(_systemService); }
            set { _systemService = value; }
        }
        // ServicePlanService
        private ServicePlanService _servicePlanService;
        public ServicePlanService ServicePlanService
        {
            get { return NewIfNull(_servicePlanService); }
            set { _servicePlanService = value; }
        }        
        // UserService
        private UserService _userService;
        public UserService UserService
        {
            get { return NewIfNull(_userService); }
            set { _userService = value; }
        }
        // WorkflowService
        private WorkflowService _workflowService;
        public WorkflowService WorkflowService
        {
            get { return NewIfNull(_workflowService); }
            set { _workflowService = value; }
        }
 
 
        //******************************************************
        // UTILITY METHODS
        //******************************************************
        
        // NewIfNull
        public T NewIfNull<T>(T obj) where T:new()
        {
            if (obj == null) { obj = new T(); }
            return obj;
        }
    }

This pattern works well for me because my concrete controllers never need to instantiate a service, they can just access any of the service properties in the base class. I don’t have the overhead of newing up a bunch of service classes that the concrete controller won’t use because the services aren’t instantiated until the first time the service property is accessed. That’s the lazy creation part. Finally, each property has a public setter, so I can replace any of my services if I need to inject a mock or stub or something.

So this already looks a lot like Dependency Injection. After all we have public setters for each of the services so we could switch them out if needed. There’s one thing missing though. We have no interfaces. Our properties are all using the concrete type of the service class (like FolderService) instead of some public interface (like IFolderService). So we could inject something, but it would have to be another instance of FolderService. That get’s us nowhere. How can we inject something else like a MockFolderService?? Hmmmm, I guess the right way to implement Dependency Injection is to create an IFolderService interface for my FolderService class. At least that’s the pattern that I always see in books and code samples. Let’s try that. Ok, FolderService only has 10 public methods. That’s not too bad so we’ll start with that one. Here’s the service code.

    public partial class FolderService : ServiceBase
    {
 
        //**************************************************************************************
        // FOLDER METHODS
        //**************************************************************************************
 
        // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT * 
                           FROM [Folder]                             
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }
 
        // GetListOfFolderWithCountsForCompany
        public virtual List<FolderWithCount> GetListOfFolderWithCountsForCompany(Guid companyGuid)
        {
            string sql = @"SELECT f.*, (SELECT COUNT([ApplicantGuid]) 
                          FROM [ApplicantInFolder] WHERE [FolderGuid] = f.[FolderGuid]) As ApplicantCount
                          FROM [Folder] f
                          WHERE f.[CompanyGuid] = @CompanyGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@CompanyGuid", companyGuid));
            return (dao.GetList<FolderWithCount>(command));
        }
 
         // GetListOfFoldersForCompany
        public virtual List<Folder> GetListOfFoldersForCompany(Guid companyGuid)
        {
            string sql = @"SELECT f.*
                                  FROM [Folder] f
                                  WHERE f.[CompanyGuid] = @CompanyGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@CompanyGuid", companyGuid));
            return (dao.GetList<Folder>(command));
        }
 

        // GetListOfFoldersForApplicant
        public virtual List<Folder> GetListOfFoldersForApplicant(Guid applicantGuid)
        {
            string sql = @"select f.*
                        from Folder f
                        join ApplicantInFolder aif on aif.FolderGuid = f.FolderGuid
                        where aif.ApplicantGuid = @ApplicantGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@ApplicantGuid", applicantGuid));
            return (dao.GetList<Folder>(command));
        }
  
        // GetCachedListOfFoldersForCompany
        public virtual List<FolderWithCount> GetCachedListOfFoldersForCompany(Guid companyGuid)
        {
            if (companyGuid.Equals(NullValues.NullGuid))
            {
                // return empty list if a null guid was passed
                return new List<FolderWithCount>();
            }
            string key = "__GetCachedListOfFoldersForCompany_" + companyGuid.ToString();
            CacheHelper helper = new CacheHelper();
            List<FolderWithCount> list = helper.TryGet<List<FolderWithCount>>(key);
            if (list == null)
            {
                RefreshCachedListOfFoldersForCompany(companyGuid);
                list = helper.TryGet<List<FolderWithCount>>(key);
            }
            return list;
        }
 

        // RefreshCachedListOfFoldersForCompany
        public virtual void RefreshCachedListOfFoldersForCompany(Guid companyGuid)
        {
            string key = "__GetCachedListOfFoldersForCompany_" + companyGuid.ToString();
            CacheHelper helper = new CacheHelper();
            List<FolderWithCount> list = helper.TryGet<List<FolderWithCount>>(key);
            list = GetListOfFolderWithCountsForCompany(companyGuid);
            helper.Add(key, list);
        }
  
        // Insert
        public virtual void Insert(Folder folder)
        {
            this.FolderPersister.Insert(folder);
        }
 

        // Save
        public virtual void Save(Folder folder)
        {
            this.FolderPersister.Save(folder);         
        } 
 
        // Delete
        public virtual void Delete(Folder folder)
        {
            this.FolderPersister.Delete(folder.FolderGuid);
        }
 

        // AddApplicantToFolder
        public virtual void AddApplicantToFolder(Guid applicantGuid, Guid companyGuid, Folder folder)
        {
            this.FolderPersister.SafeInsertApplicantInFolder(applicantGuid, folder.FolderGuid);
            RefreshCachedListOfFoldersForCompany(companyGuid);
        } 
    }

We’ve basically got 10 public data access methods. Now I need to create a public interface that defines all of the public methods, so I’m going to create a new IFolderService interface. ReSharper even makes it easy for me with a handy Extract Interface tool. So my interface looks like this.

    public interface IFolderService
    {
        Folder GetFolderByFolderGuid(Guid folderGuid);
        List<FolderWithCount> GetListOfFolderWithCountsForCompany(Guid companyGuid);
        List<Folder> GetListOfFoldersForCompany(Guid companyGuid);
        List<Folder> GetListOfFoldersForApplicant(Guid applicantGuid);
        List<FolderWithCount> GetCachedListOfFoldersForCompany(Guid companyGuid);
        void RefreshCachedListOfFoldersForCompany(Guid companyGuid);
        void Insert(Folder folder);
        void Save(Folder folder);
        void Delete(Folder folder);
        void AddApplicantToFolder(Guid applicantGuid, Guid companyGuid, Folder folder);
        SqlDao SharedSqlDao { get; set; }
        ApplicantPersister ApplicantPersister { get; set; }
        CareerSiteSettingsPersister CareerSiteSettingsPersister { get; set; }
        CompanyPersister CompanyPersister { get; set; }
        CompanySettingsPersister CompanySettingsPersister { get; set; }
        FolderPersister FolderPersister { get; set; }
        HistoryItemPersister HistoryItemPersister { get; set; }
        JobPersister JobPersister { get; set; }
        JobStubPersister JobStubPersister { get; set; }
        ResumePersister ResumePersister { get; set; }
        UserPersister UserPersister { get; set; }
        WorkflowPersister WorkflowPersister { get; set; }
        T NewIfNull<T>(T obj) where T : new();
        string GetSqlCsv(List<Guid> list);
        string GetSqlCsv(List<int> list);
    }

There, that wasn’t too bad. Now I just need to change my controller FolderService property to type IFolderService and I have Dependency Injection implemented for the FolderService. I can create new classes that implement IFolderService and inject them into my controller using the public FolderService property like this.

            var controller = new FolderController();
            IFolderService mock = new MockFolderService();
            controller.FolderService = mock;

That’s great! Now I just need to do that for… wait… I’m going to have to do that for every service class in my application. Then I’m going to have to maintain each of those interfaces every time I make a change to a service class. That’s a lot of extra code and a lot of work that’s going to have to be repeated every time I make a change to my service classes. Why am I doing this again? This is starting to feel like the hate part of the love / hate relationship.

Dependency Injection on an MVC Controller Base Class Without Interfaces

So I don’t really want to create an interface for every single dependency in my application and then have to maintain that interface as well as my concrete classes, but what other option do I have? Everybody implements DI using interfaces right? Interfaces are like a badge of honor that demonstrate you know how to architect an application the right way. You haven’t been to architecture land until you have your interface badge. We love interfaces!

Well I don’t love any code that I have to spend extra time maintaining every time I make a change to my application. All that stuff adds up after a while. I call it the death of a thousand cuts. One day you look up and all those little extra tasks like maintaining interfaces add up to a mountain of friction that makes it miserable to make changes to your code. Now there are places for interfaces, but if possible I’m opting for an easier alternative.

So step one in the easier alternative, throw out the interface that I just created and change my FolderService property back to type FolderService (no more IFolderService). Step two…. actually we don’t need a step two we’re done. Lets take a closer look at one of the data access methods in our FolderService class.

       // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT *
                           FROM [Folder]
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }

Hmmm, that’s a virtual method. That means that I can create a class that inherits from FolderService and just override the GetFolderByFolderGuid() method. The FolderService class actually functions as my interface in this scenario. So, if I’m writing a unit test where I want to stub out my FolderService I could create the following class.

    public class FolderServiceStub: FolderService
    {
        public override Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            return new Folder() {FolderGuid = folderGuid, FolderName = "Test Folder"};
        }
    }

FolderServiceStub inherits from FolderService so I can use that stub class anywhere that an object of type FolderService is expected. The end result is that my testing code (or any other DI code) winds up looking almost exactly like my code did with full on interface based Dependency injection

    var controller = new FolderController();
    FolderService mock = new FolderServiceStub();
    controller.FolderService = mock;

Summary

So, make all of your public methods virtual and you can implement a Poor Man’s Dependency Injection that doesn’t require interfaces. It’s not the right solution every situation, but I’ve found that it works most of the time for me. It also has the one quality that I prize most in code. It’s so simple that I forget it’s even there.

AAPL Part 7: Building a Generic Data Mapper

noreply@blogger.com (Rudy Lacovara) — Mon, 10 Jan 2011 06:23:00 +0000

I got bored with this series a while back and I haven’t been writing on it, mostly because I’ve been using the architecture (Agile ADO.Net Persistence layer) to write a bunch of code. The basic idea of AAPL was to create an application architecture that embraced code changes instead of resisting them. I took a look back at all of the application architectures I’ve worked with over the years and thought carefully about the parts of those architectures that caused me pain whenever made changes to the application. I then threw all those things out and tried to put together an application architecture based on ADO.Net (not LINQ, EF, or nHibernate) that was optimized for making code changes easy and reliable. I’ve been using the AAPL architecture for a while now and I’m happy to report that those decisions have really paid off.

Why are we doing this again?

Just a a quick reminder. The point of this architecture is to give us the ability to write persistence code that looks like this.

        // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT * 
                           FROM [Folder]                             
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }

Whenever we need a new data access method we want to just write the new query, package it up in an ADO.Net SqlCommand object, then pass both the SqlCommand and the desired model class type to a Data Access Object that will then handle all of the plumbing required to execute the command, get the data, package it up as whatever model type we passed in and then give it back. This is a very low friction way to handle data access. It’s easy to maintain, new data methods take seconds to create, and we never have to touch the plumbing code. The end result is that maintaining code and making changes to our model should feel like this…

instead of this…

What is a data mapper?

The key piece of plumbing code that makes this architecture so painless is the GenericDataMapper. As described above, when we read data we execute an SQLCommand, get the resulting IDataReader, then map the data in that IDataReader to a model class (an entity class). The DataMapper is the class that encapsulates the code that knows how to map the fields from an IDataReader to the properties in a model class. For example, the UserMapper class contains the logic needed to take an IDataReader and use it’s data to populate a new object of type User.

What does the Generic Data Mapper do?

We could write a separate DataMapper class for each model type. We could have a UserMapper, a CompanyMapper, a JobMapper, a WorkflowMapper, an ActionMapper, a StepMapper… you get the idea. But doing it that way would require that 1) We write all those DataMapper classes, and 2) We modify those DataMapper classes whenever we make a change to one of our models. No thanks.

It would be better to write a single DataMapper class that could handle the mapping for any model class that we passed to it. Remember that early on we made the decision that the columns in our database would have the exact same names as the data properties in our model classes. That constraint makes the mapping logic pretty simple. We can create a GenericDataMapper that uses reflection to find all the properties on our model class and matches them to any fields with the same name in our IDataReader. No hard coded mapping logic needed. Just a little bit of reflection and a standard naming convention and we can create a single GenericDataMapper that will handle mapping for all of our model classes.

DataMapperFactory

We use a DataMapperFactory to get the right DataMapper for a model type. The logic is simple. We have a GetMapper() method that takes a type as it’s parameter. It checks to see if we have a custom mapper for the type, if not it returns the GenericMapper . In the example below we have just one custom data mapper, ListItemDTOMapper. Every other model class is handled by the GenericMapper.

    class DataMapperFactory
    {
        public IDataMapper GetMapper(Type dtoType)
        {
            switch(dtoType.Name)
            {
                case "ListItemDTO":
                    return new ListItemDTOMapper();
                default:
                    return new GenericMapper(dtoType);
            }       
        }
    }

IDataMapper

All of our DataMappers implement a custom IDataMapper interface.

    interface IDataMapper
    {
        // Main method that populates dto with data
        Object GetData(IDataReader reader);
        // Gets the num results returned. Needed for data paging.
        int GetRecordCount(IDataReader reader);
    }

IDataMapper requires that we implement just two methods. GetData() takes an IDataReader and returns a model class that contains the data for the current record in that reader (note that one call to GetData() returns one model object, lists can be created by calling GetData() multiple times ). GetRecordCount() will return the total number of records returned by the IDataReader and is used for data paging. So these are the only two public methods that we’ll need to implement.

GenericMapper

Now we get down to it. Here is the complete code for the GenericMapper.

    class GenericMapper : IDataMapper
    {
        public System.Type DtoType { get; set; } 
        private bool _isInitialized = false;
        private List<PropertyOrdinalMap> PropertyOrdinalMappings;
 
 
        public GenericMapper(System.Type type)
        {
            DtoType = type;
        }
 
 
        private void InitializeMapper(IDataReader reader)
        {
            PopulatePropertyOrdinalMappings(reader);
            _isInitialized = true;
        }
 
 
        public void PopulatePropertyOrdinalMappings(IDataReader reader)
        {
            // Get the PropertyInfo objects for our DTO type and map them to 
            // the ordinals for the fields with the same names in our reader.  
            PropertyOrdinalMappings = new List<PropertyOrdinalMap>();
            PropertyInfo[] properties = DtoType.GetProperties();
            foreach (PropertyInfo property in properties)
            {
                PropertyOrdinalMap map = new PropertyOrdinalMap();
                map.Property = property;
                try
                {
                    map.Ordinal = reader.GetOrdinal(property.Name);
                    PropertyOrdinalMappings.Add(map);
                }
                catch { }
            }
        }
 
 
        public Object GetData(IDataReader reader)
        {
            if (!_isInitialized) { InitializeMapper(reader); }
            object dto = Activator.CreateInstance(DtoType);
            foreach (PropertyOrdinalMap map in PropertyOrdinalMappings)
            {
                 if (!reader.IsDBNull(map.Ordinal)) 
                 { 
                     map.Property.SetValue(dto,reader.GetValue(map.Ordinal),null); 
                 }
            }
            return dto;
        }
 
 
        public int GetRecordCount(IDataReader reader)
        {
            Object count = reader["RecordCount"];
            return count == null ? 0 : Convert.ToInt32(count);
        }
 
 
        private class PropertyOrdinalMap 
        {
            public PropertyInfo Property { get; set; }
            public int Ordinal { get; set; }
        }
 
    }

This isn’t very much code, but the logic can be a little difficult to follow. Let’s break it down and see how it works.

When the user creates an instance of GenericMapper, the constructor requires them to pass in the model type they want to map to. That type is assigned to our local DtoType property (ModelType might have been a better name). Note that we’re creating an instance, not using static methods. This is important because once the GenericMapper is created it can be used multiple times for multiple records without having to recalculate the mappings.

The user calls the GetData method which takes an IDataReader as a parameter. GetData is the heart of our mapper.

        public Object GetData(IDataReader reader)
        {
            if (!_isInitialized) { InitializeMapper(reader); }
            object dto = Activator.CreateInstance(DtoType);
            foreach (PropertyOrdinalMap map in PropertyOrdinalMappings)
            {
                 if (!reader.IsDBNull(map.Ordinal)) 
                 { 
                     map.Property.SetValue(dto,reader.GetValue(map.Ordinal),null); 
                 }
            }
            return dto;
        }

GetData first checks to see if the GenericMapper has been initialized. If not it calls InitializeMapper() which handles all initialization including a call to PopulatePropertyOrdinalMappings(). This is where the magic happens.

        public void PopulatePropertyOrdinalMappings(IDataReader reader)
        {
            // Get the PropertyInfo objects for our DTO type and map them to 
            // the ordinals for the fields with the same names in our reader.  
            PropertyOrdinalMappings = new List<PropertyOrdinalMap>();
            PropertyInfo[] properties = DtoType.GetProperties();
            foreach (PropertyInfo property in properties)
            {
                PropertyOrdinalMap map = new PropertyOrdinalMap();
                map.Property = property;
                try
                {
                    map.Ordinal = reader.GetOrdinal(property.Name);
                    PropertyOrdinalMappings.Add(map);
                }
                catch { }
            }
        }

We have a private class named PropertyOrdinalMap which is just a simple container for a System.Reflection.PropertyInfo object and an int. The int represents the ordinal (field index) in our IDataReader for the field that matches the name of the property stored in PropertyInfo. The PopulatePropertyOrdinalMappings method just loops over all of the properties in our DtoType and tries to find an ordinal in IDataReader for a field with the same name. If a match exists, a new PropertyOrdinalMap is created an added to the PropertyOrdinalMappings List. If a match doesn’t exist, no PropertyOrdinalMap is created. The result is that we just skip any properties in our DtoType that the reader doesn’t have a value for. That’s a key design decision. We don’t throw an error if a mapping doesn’t exist, we just skip that property and get the data that we can map.

Once the mappings are created it’s all pretty straight forward. We return to GetData(). We use System.Activator() to create a new instance of our DtoType called dto (short for Data Transfer Object). Then we iterate over our PropertyOrdinalMappings and use reflection to set the value of each dto property that we have a mapping for. At that point we have a brand new object of whatever type was passed in to the constructor and it’s been populated with the data from the current record in the IDataReader. We’re done! We can just return the dto.

So that’s it. A little object oriented design, some super-simple reflection code, and I have a GenericMapper that allows me to never right another line of data mapping code (as long as I’m willing to use the same name for my model properties and data table columns). I haven’t spent any time optimizing this. In fact, I haven’t really looked at it since I first wrote it. If anyone has suggestions for improvements please let me know.

Type Argument Specification is Redundant

noreply@blogger.com (Rudy Lacovara) — Mon, 03 Jan 2011 17:16:00 +0000

One of the things I like about ReSharper is that it teaches me things I didn’t know. Recently I was writing a property that calls a generic helper method that I’d created and ReSharper gave me the little popup below letting me know that I might want to refactor that line of code.

“Type argument specification is redundant”, what the heck does that mean? So I looked into it and learned something interesting that I didn’t know before. To understand what’s going on let’s first look at my NewIfNull() method.

NewIfNull() is a simple generic helper method that takes an object of any type that has a parameterless constructor, it then checks to see if that object is null. If it is null, the helper returns a new instance of that object’s type. If the object isn’t null, it just returns the object. It’s kind of like lazy loading for type instantiation. It’s useful in situations where I have properties for things like data services that I don’t want to instantiate until the first time I call them (if they’re called at all). Here’s the NewIfNull() code:

You can see that NewIfNull() is a generic method that takes a type parameter T which represents the type of the object that I’m passing in. A typical call to NewIfNull looks like

This isn’t too bad, but it’s a little annoying that I have to pass _workflowService as a parameter and then I also have to specify WorkflowService as my type inside of the angle brackets. Well apparently that part is completely unnecessary. That little helper message from ReSharper was trying to tell me that since T is just the type of my parameter obj, .Net can infer the type of T from parameter obj. Passing type T in the angle brackets is actually redundant. Brilliant! Now my calls to NewIfNull() look like this.

Much cleaner. No angle brackets or generic type parameters needed. I’ve been writing and using generic methods for quite a while and I never knew .Net would infer types like this. Thank you ReSharper. If you want to learn more, the MSDN documentation on Type Inference is pretty good. Check out Section 26.3.3 Type Inference.

Muffin Top architecture is killing us

noreply@blogger.com (Rudy Lacovara) — Mon, 02 Aug 2010 17:00:00 +0000

I want to share a story. I recently encountered a company that had somehow navigated the many hazards that face technology startups and managed to create a solid, successful business. They had gone public and now after years of struggling with a collection of applications that consisted largely of simple markup and script pages that directly accessed database queries, they were in the process of transitioning everything over to a real architecture, a single unified “Framework”.

The “Framework” was a thing of architectural beauty, object oriented design as far as the eye could see. It had been designed by some incredibly bright developers. So bright in fact that they had progressed beyond the point where they were even involved in writing code for a shipping application. Some were so smart, that they skipped the mundane part of a programmer’s career where they spend years writing and maintaining application code, and instead they moved directly into the architecture world.

The core of the “Framework” was the data access module which was based on LINQ to SQL, but hid all of the entities as well as the data context behind interfaces. These interfaces would of course need to be updated whenever changes were made to the LINQ data model, but they made the “Framework” feel a lot more object oriented and provided the additional benefit of hiding any functionality that the architecture group didn’t think regular application developers should have access to.

The LINQ data model also benefited from some extra architectural love. Mapping directly to tables was out of the question, so every LINQ entity mapped to a collection of sprocs. After all, the idea was to create an entity model, not just map directly to table data. No, every entity was carefully planned and modeled, every field the application developers might need was anticipated and aggregated using those sprocs. Now this did create yet another point (like the interfaces) at which data mapping logic could go wrong and throw errors, but that wasn’t a problem because the architecture team was able to use Visual Studio to generate a massive collection of database unit tests that would exhaustively test every possible crud scenario for every sproc and assert that it worked as expected. Of course the massive collection of db unit tests and the sprocs themselves would need to be rewritten every time even the smallest change was made to the data model, but architecturally speaking it was just the right thing to do.

Finally the day came when the “Framework” was pushed out to the unwashed masses of developers who would benefit from it’s glorious design. But an unexpected thing happened. New application development done on top of the “Framework” slowed to a crawl. It was taking 3 to 5 times longer for application developers to build code with the “Framework”. To make matters worse, the code that was produced, ran slower, sometimes much, much slower than the old legacy code that accessed the database directly. Also, the application developers kept coming back to the architecture team requesting that additional fields be added to the entities. They said these fields were required in order to provide features that were already available in the legacy applications. It was as if these guys had never been part of the endless modeling meetings where everyone worked out in advance every field each entity would ever need. The problems just kept piling up, and application ship dates slipped further and further. Eventually the architects had to face up to an unpleasant conclusion… the company’s application developers just weren’t very good. I’m not kidding, that really was their conclusion. I actually heard that straight from the mouth of one of the architects. (BTW, in case you’re wondering the application developers were actually some of the most skilled that I’ve ever run into. They were very good developers.)

The point of this story is that the problems we face as programmers in the .Net world are rapidly changing. It used to be that the most common problem I ran into was horribly constructed spaghetti code that was then copied and pasted into 20 different pages of the application. No code reuse, poor logic structure, and of course direct access to the database in the code behind or even in the markup pages. The problems were largely the result of us as .Net developers just not being skilled enough to properly design our code. But .Net is a developer-friendly technology and most of us were able to hack together code that worked even if it wasn’t pretty. Note that I did say “us”. Some of my early code is horrific.

Now, increasingly I see a much different problem. As a community we’ve learned a lot more about real software development. We understand all of those architectural things that our Java buddies used to talk about. We’re using design patterns, object oriented design, automated testing, ORMs, true layered architectures, dependency injection, component architectures, class inheritance all over the place, interfaces, contract based programming, service oriented architecture and a bunch of stuff that sounds like it should enable us to ship more reliable code, and to ship it faster. The problem is that in my experience, most of the time it doesn’t. Don’t get me wrong, I'm not saying that the practices listed above are bad, or that we should go back to writing simple markup pages with SQL queries written directly in our scripts. What I am saying is that in our eagerness to flex our new-found coding muscles, we’re just throwing these things into our applications because we think they’re a “best practice” and the result is a lot of over-architected, impossible to maintain, fat, sprawling code. I just get a sense that many very intelligent .Net developers are throwing this stuff into their applications because this is what a real architecture looks like, not because they are getting some measurable benefit. At the end of the day it’s all about shipping code, not creating an architecture that makes us feel like real architects.

A smart guy that I met at the unspecified company mentioned earlier put a name to it. He called it Muffin Top Architecture. I didn’t know what “muffin top” meant until he explained it to me. If you’re confused do a search on Google images for muffin top and you’ll figure it out. Hint, it doesn’t refer to the images of real muffins. Anyway, I’m starting to see quite a bit of Muffin Top code. The company I told the story about is by no means atypical. I could go on for pages and pages. I’ve even encountered companies who are unable to make changes to their own code, that they wrote. There are just so many layers of architecture and class inheritance that changes are incredibly painful even for the original developers.

I don’t have an easy answer, but I do think we need to start producing code that is simpler, easier to maintain, and quicker to produce. .Net is a great stack and it isn’t difficult to work with, but we’re developing this reputation of “.Net is a solid platform HOWEVER it takes so long to build on that it’s not appropriate for many projects, and definitely not for a startup project”. I don’t think that’s true. The problem isn’t .Net, the problem is that we’re over-architecting, and in many cases Microsoft is more than happy to help us further down the path to destruction with new persistence frameworks, designers, and component architectures. I think we should be looking more and more at technologies like Ruby on Rails and trying to figure out how they achieve the remarkable efficiencies that are possible on that platform. It’s certainly not through a designer. I also think that the answer lies in less code that is simpler and easier to understand, not piles of generated code and component architectures that make it impossible to figure out what code is executing when.

Another big part of the problem is the emergence of the non-coding architect. This is the person who doesn’t write any shipping code themselves, they just tell everyone else how they should be writing code. I think the role of architect is tremendously important. I see the architect as the lead developer, the mentor, the overall system designer. But let me make this clear, an architect who doesn’t write code isn’t a real programmer, they are an obstacle to real programmers. It doesn’t matter how skilled that person is, if the architect isn’t writing shipping code then their incentives are not aligned with the other developers. The real programmer’s number one priority in life is to ship working code. The non-coding architect’s priority is to create beautiful architecture.

If you have any thoughts please don’t hesitate to post them. Personally, I think that Muffin Top Architecture is killing us. Is there a solution or am I just destined to go be a Rails developer like Mike Moore, Jeff Cohen, and Scott Bellware? Check out Herding Code 84 for some interesting discussion by the .Net expatriates.

AAPL Part 6: Building a DataMapper and DataMapperFactory

noreply@blogger.com (Rudy Lacovara) — Tue, 27 Apr 2010 08:50:00 +0000

I recently decided that I needed to take a fresh look at how to build a persistence architecture that would provide the flexibility of an ORM, that would embrace change instead of resist it (making maintenance code easier), and would use ADO.Net. I started building from the ground up, threw away any best practices that resulted in friction, and Agile ADO.Net Persistence Layer is the result. Part 1 in the series can be found here: Agile ADO.Net Persistence Layer Overview.

In this post, we’ll get into the DataMapper. If you’ve been waiting to see some real ADO.Net code, this is the class you’ve been waiting for.

Where the DataMapper classes fit in

Let’s review our solution structure. Core is our main business layer project (Note: in previous posts and in the sample code Core is actually the BAL project. I never liked the name BAL so I’ve renamed it Core. Same code, just a different name. Sorry for any confusion.). Core contains Service classes which are our one stop shop for all persistence (see AAPL Part 3 and AAPL Part 4 for more on how the service classes work). The Service classes use the SqlDao for all communication with the database. The SqlDao uses the DataMapper classes to handle all mapping between the DataReader returned by executing a query, and the desired data shape. Here’s what our project looks like:

Last post we covered the SqlDao class and saw how it’s data access methods take a query wrapped in an SqlCommand, execute that query, then return either a Single<DTO> (where DTO is our desired data shape), List<DTO>, DataPage<DTO>,or a scalar value. The data access methods in SqlDao handle all the boilerplate logic for getting a connection, executing a query, stuff like that, but the logic that is specific to a data shape, things like get the value from this column in my reader put it in this property on my returned data shape, that stuff get’s delegated to a DataMapper class. This is what that SqlDao code looks like:

SqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
    reader.Read();
    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
    dto = (T)mapper.GetData(reader);
    reader.Close();
}

What does the DataMapperFactory do?

You’ll notice in the code above that we get our IDataMapper object from a DataMapperFactory. If we need to get the DataMapper for type User, we just call the DataMapperFactory.GetMapper method, pass type(User) as a parameter, and GetMapper returns the appropriate concrete DataMapper (Note that the code example above uses generics, so we use generic T instead of a specific type like User, but the logic is the same). DataMapperFactory is a simple factory class with one method, GetMapper. The code looks like this:

class DataMapperFactory
{
    public IDataMapper GetMapper(Type dtoType)
    {
        switch(dtoType.Name)
        {
            case "ListItemDTO":
                return new ListItemDTOMapper();
            case "User":
                return new UserMapper();
            default:
                return new GenericMapper(dtoType);
        }       
    }
}

Whenever we create a new IDataMapper for a specific type, like User, we need to add it to this factory method. The method is just a switch statement that uses the type.Name to look for a type-specific implementation of IDataMapper. If it finds one, it returns it. If it doesn’t find a type specific IDataMapper, it returns an instance of GenericMapper, which is our reflection based DataMapper that can be used to handle mappings for any query / data shape combination. We’ll cover GenericMapper in the next post.

What does the DataMapper do?

The behavior of a DataMapper class is defined by the IDataMapper interface. The interface requires DataMappers to implement two methods.

interface IDataMapper
{
    // Main method that populates dto with data
    Object GetData(IDataReader reader);
    // Gets the num results returned. Needed for data paging.
    int GetRecordCount(IDataReader reader);
}

The GetData method is the important one. That’s the method that our SqlDao calls whenever it needs to map the current record in an SqlDataReader to a specific data shape. Remember, this method doesn’t do any mapping of an SqlDataReader to a List, or DataPage, or anything like that. That’s handled by the SqlDao data access methods. The purpose of the DataMapper GetData function is to take an SqlDataReader that’s already open and on a record, read that record, put the column data into an object (DTO) of the desired data shape, then return that single DTO.

Note that GetData returns an Object, so the SqlDao has to cast the GetData result to the proper data shape. I tried all kinds of designs to get around this cast, I used interfaces, generics, base classes, but in the end all of these options added complexity to the code that I just couldn’t accept. So, I opted to stick with GetData returning an object. I think it was the right tradeoff, but if anyone has a better solution please let me know.

Here is a diagram of our IDataMapper interface and a couple of concrete DataMapper implementations.

Inside a DataMapper class

So let’s take a look inside the UserMapper class and see what an IDataMapper implementation looks like. The user class is a data shape that represents, wait for it… a user. The class looks like this.

So the job of UserDataMapper is to take a SqlDataReader, get the values from it’s columns, and put those values into and instance of the User class above. The DataMapper implementation that I use accomplishes this task in a very structured way that is easy to copy and reproduce for new data shapes.

Before we look at the code, I should mention that I write a fair amount of extra code to get the ordinals for each data field instead of just using the string field names. The reason for this is that using the ordinals allows me to use strongly typed methods on the SqlDataReader to get each column. This allows me to avoid the cast to and from Object that occurs when you get a column by name which results in boxing and unboxing for value types. It also gives me a little type safety. If the query is returning a column called UserGuid that has a type of Int32, when I call reader.GetGuid() on that column, my code will throw an error and alert me that there is a problem with my query. For more on the benefits of using ordinals take a look at High Performance Data Access Layer Architecture Part 3.

Now, with no further ado, here’s the UserMapper implementation.

class UserMapper : IDataMapper
{
    private bool _isInitialized = false;
    private int _ordinal_UserGuid;
    private int _ordinal_UserRole;
    private int _ordinal_CreatedUtc;
    private int _ordinal_ModifiedUtc;
    private int _ordinal_CompanyGuid;
    private int _ordinal_FirstName;
    private int _ordinal_LastName;
    private int _ordinal_Email;
    private int _ordinal_Password;
    private int _ordinal_PhoneWork;
    private int _ordinal_PhoneMobile;
 
    private void InitializeMapper(IDataReader reader)
    {
        PopulateOrdinals(reader);
        _isInitialized = true;
    }
 
    public void PopulateOrdinals(IDataReader reader)
    {
        _ordinal_UserGuid = reader.GetOrdinal("UserGuid");
        _ordinal_UserRole = reader.GetOrdinal("UserRole");
        _ordinal_CreatedUtc = reader.GetOrdinal("CreatedUtc");
        _ordinal_ModifiedUtc = reader.GetOrdinal("ModifiedUtc");
        _ordinal_CompanyGuid = reader.GetOrdinal("CompanyGuid");
        _ordinal_FirstName = reader.GetOrdinal("FirstName");
        _ordinal_LastName = reader.GetOrdinal("LastName");
        _ordinal_Email = reader.GetOrdinal("Email");
        _ordinal_Password = reader.GetOrdinal("Password");
        _ordinal_PhoneWork = reader.GetOrdinal("PhoneWork");
        _ordinal_PhoneMobile = reader.GetOrdinal("PhoneMobile");
    }
 
    public Object GetData(IDataReader reader)
    {
        // This is where we define the mapping between the object properties and the 
        // data columns. The convention that should be used is that the object property 
        // names are exactly the same as the column names. However if there is some 
        // compelling reason for the names to be different the mapping can be defined here.
 
        // We assume the reader has data and is already on the row that contains the data 
        //we need. We don't need to call read. As a general rule, assume that every field must 
        //be null  checked. If a field is null then the nullvalue for that  field has already 
        //been set by the DTO constructor, we don't have to change it. 
        if (!_isInitialized) { InitializeMapper(reader); }
        User dto = new User();
        // Now we can load the data
        if (!reader.IsDBNull(_ordinal_UserGuid)) { dto.UserGuid = reader.GetGuid(_ordinal_UserGuid); }
        if (!reader.IsDBNull(_ordinal_UserRole)) { dto.UserRole = (UserRole)reader.GetInt32(_ordinal_UserRole); }
        if (!reader.IsDBNull(_ordinal_CreatedUtc)) { dto.CreatedUtc = reader.GetDateTime(_ordinal_CreatedUtc); }
        if (!reader.IsDBNull(_ordinal_ModifiedUtc)) { dto.ModifiedUtc = reader.GetDateTime(_ordinal_ModifiedUtc); }
        if (!reader.IsDBNull(_ordinal_CompanyGuid)) { dto.CompanyGuid = reader.GetGuid(_ordinal_CompanyGuid); }
        if (!reader.IsDBNull(_ordinal_FirstName)) { dto.FirstName = reader.GetString(_ordinal_FirstName); }
        if (!reader.IsDBNull(_ordinal_LastName)) { dto.LastName = reader.GetString(_ordinal_LastName); }
        if (!reader.IsDBNull(_ordinal_Email)) { dto.Email = reader.GetString(_ordinal_Email); }
        if (!reader.IsDBNull(_ordinal_Password)) { dto.Password = reader.GetString(_ordinal_Password); }
        if (!reader.IsDBNull(_ordinal_PhoneWork)) { dto.PhoneWork = reader.GetString(_ordinal_PhoneWork); }
        if (!reader.IsDBNull(_ordinal_PhoneMobile)) { dto.PhoneMobile = reader.GetString(_ordinal_PhoneMobile); }
        return dto;
    }
 
    public int GetRecordCount(IDataReader reader)
    {
        Object count = reader["RecordCount"];
        return count == null ? 0 : Convert.ToInt32(count);
    }
}

You’ll notice that the first thing we do is declare an _ordinal_FieldName member for each property in our User class. These local members are used to store the ordinal location for the column in the SqlDataReader that maps to that User field. We also create an _isInitialized flag that let’s us know whether we’ve set our _ordinal_FieldName values yet. This is important when the UserMapper is being used to get more than one User from a DataReader. The SqlDao code will get a single IDataMapper instance and will then call that mapper over and over for each record that exists in the SqlDataReader. We only want to find the ordinals once, so we create our _isInitialized flag that can be set to true after we set all the ordinal values.

Next is the InitializeMapper method. This just calls our PopulateOrdinals method and then sets that _isInitialized flag, that we just discussed, to true.

PopulateOrdinals is the method that takes our SqlDataReader, finds the ordinal location of each field we need, then stores those ordinal values in our local _ordinal_FieldName members.

Finally we arrive at GetData, the heart of our class. We’ve structured our code in such a way that GetData is actually pretty simple. We check to make sure the _isInitialized flag has been set. If it hasn’t we call PopulateOrdinals. We then create a new instance of our User class called dto. After that it’s just a matter of repeating the same line of code for each field.

if (!reader.IsDBNull(_ordinal_UserGuid)) { dto.UserGuid = reader.GetGuid(_ordinal_UserGuid); }

This one line of code performs a null check on the ordinal, if the column exists it uses a strongly typed Get method to get the value of that column and assign it to the dto, if the column is null then we do nothing because the constructor for User already set each property to it’s null value on creation. And that’s it, after setting each field value (or not setting it) we just return our dto and we’re done. Simple maintainable, easy to copy and reproduce code.

The last method is GetRecordCount. We need this method to support the DataPage<DTO> shape. Our SqlDao needs to know the total record count if it’s returning a data page. If it’s not returning a data page this method isn’t used.

Conclusion

That’s it. Now that I look at the DataMapper code its a little anti-climactic because it’s really very simple stuff. But then that’s the idea, break a complex operation into smaller pieces so that each one is easy to understand, and if needed, reproduce. To add a new DataMapper we just copy and paste the existing UserMapper and change the field names. Easy. We don’t need to touch any of the code at the SqlDao level that handles getting Single<DTO>, List<DTO>, or DataPage<DTO> return types. We also don’t touch any code that affects any other data shape. All of our code is encapsulated in this one easy to handle class.

Also, keep in mind that the real power of this architecture is that we don’t need to create a type specific DataMapper at all. For the vast majority of tasks, the GenericMapper is going to work just fine. In practice I never create any type-specific DataMappers during development because it seems like at that early stage data shapes are always in flux, and when a data shape changes, the last thing I want to do is go back and update a bunch of mapping code. I look at the type specific DataMappers as something that I never want to use unless an app has gone into production and it’s experiencing performance issues related to the GenericMapper. If that happens then I have a hook in the framework that I can use to create a simple DataMapper for that data shape that contains what I think is the fastest ADO.Net code possible.

Next time we’ll go over the GenericMapper.

AAPL Part 5: How to Write a Data Access Object (DAO)

noreply@blogger.com (Rudy Lacovara) — Mon, 19 Apr 2010 12:32:00 +0000

In this post, we’ll get into what is probably the most useful class in the framework, the Data Access Object (DAO).

Where does the DAO fit in?

Let’s backtrack a bit and take a quick look at the design of core. Below we see a snapshot of Core’s folder structure. We have a Persistence folder that contains our SqlDAO (a DAO written specifically for SqlServer), as well as our DataMappers and Persisters (we’ll go into these later). The Rules folder contains business rules and validation rules for different entities / data shapes. Security has a little plumbing code for business layer authorization. Finally, the Services folder contains the service classes that we’ve focused on during the previous few posts.

The service classes are the one stop shop that our UI code uses to get and save data. A typical data access method looks like this.

public User GetUserByEmail(string email)
{
    string sql = @"SELECT * 
                  FROM [User] 
                  WHERE [Email] = @Email";
    SqlDao dao = SharedSqlDao;
    SqlCommand command = dao.GetSqlCommand(sql);
    command.Parameters.Add(dao.CreateParameter("@Email", email, 100));
    return dao.GetSingle<User>(command);
}

You’ll notice that this is a pretty simple method. There’s no real Ado.Net code. We’re just defining a query, wrapping it in an SqlCommand, and then passing both the SqlCommand and our desired data shape <User> to our DAO. This mystery DAO object does all the heavy lifting for us and even helps us create our query parameter in very efficient, single line of code. This is a very intentional design. After writing data access code for a few years, I finally noticed that the only things I really change from one data access method to the next are: the query; the parameters; and the return data shape. Everything else is just boilerplate plumbing code that gets repeated over and over again. So, I took all that data access boilerplate code and extracted it out to a helper class called SqlDao. The main functions of the DAO are:

The DAO is the single access point to our database. All communication with the database goes through the DAO. No code goes around it.
The DAO encapsulates common ADO.Net tasks like getting a connection, creating query parameters, executing queries, etc.
The DAO defines methods that take an SqlCommand as a parameter, execute it, and return the data in one of our standard formats, a single DTO, as List<DTO>, a DataPage<DTO>, or a string value.

There’s a lot of good stuff in there that makes writing data access method almost frictionless. I can’t go over everything so I included a complete listing for the SqlDao at the end of this post.

The GetSingle<T> and GetList<T> methods

Now let’s look at something a little more interesting. The SqlDao.GetSingle<T> method takes an SqlCommand as a parameter and returns the restults of that command as a single T where T is the class of our desired datashape. So, the line dao.GetSingle<User>(command); will execute command and then return an object of type User. If no User was found then the method just returns null. Here’s how it works.

// GetSingle
public T GetSingle<T>(SqlCommand command) where T : class
{
    T dto = null;
    try
    {
        if (command.Connection.State != ConnectionState.Open)
        {
            command.Connection.Open();
        }                
        SqlDataReader reader = command.ExecuteReader();
        if (reader.HasRows)
        {
            reader.Read();
            IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
            dto = (T)mapper.GetData(reader);
            reader.Close();
        }
    }
    catch (Exception e)
    {
        throw new Exception("Error populating data", e);
    }
    finally
    {
        command.Connection.Close();
    }
    // return the DTO, it's either populated with data or null.
    return dto;
}

The code is pretty simple. We declare a new object of type T called dto (Data Transfer Object) and initialize it to null. We then check the connection on our command and make sure it’s open. We call ExecuteReader on the command which gets us an SqlDataReader. If the reader has any rows then we go to our DataMapperFactory and get an IDataMapper for the type T. A cool thing that is happening behind the scenes is that the DataMapperFactory checks to see if we have an IDataMapper that has been created specifically for type T, if we don’t it returns a GenericMapper that uses reflection to map SqlDataReader fields to fields on our type T. Once we have a mapper it’s just a matter of calling the mapper.GetData(reader) method and returning the dto.

Now let’s take a quick look at GetList<T>. This method is very similar to GetSingle<T>, but it returns a generic list of objects of type T instead of a single object of type T. If no results are returned when we execute the SqlCommand passed to GetList<T>, we don’t return null, instead we return an empty list. Here’s the code.

// GetList
public List<T> GetList<T>(SqlCommand command) where T : class
{
    List<T> dtoList = new List<T>();
    try
    {
        if (command.Connection.State != ConnectionState.Open)
        {
            command.Connection.Open();
        }   
        SqlDataReader reader = command.ExecuteReader();
        if (reader.HasRows)
        {
            IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
            while (reader.Read())
            {
                T dto = null;
                dto = (T)mapper.GetData(reader);
                dtoList.Add(dto);
            }
            reader.Close();
        }
    }
    catch (Exception e)
    {
        throw new Exception("Error populating data", e);
    }
    finally
    {
        command.Connection.Close();
    }
    // We return either the populated list if there was data,
    // or if there was no data we return an empty list.
    return dtoList;
}

It’s really the same as GetSingle<T>. We just have a different return type List<T>, and a while loop that get’s our individual DTOs and adds them to our return dtoList. The main logic for getting an IDataMapper from the DataMapperFactory and calling mapper.GetData(reader) to parse the reader results into an object of type T remains the same.

Conclusion

At this point you may be thinking, “That’s it???, you didn’t answer anything! All you did is get some mapper object and call the GetData method on it.” That’s true to some extent. Our GetSingle<T> and GetList<T> methods just contain the logic that is always going to be common to those tasks regardless of the data shape passed to them. Anything that is data shape specific, things like how do I get data from this field in my SqlDataReader and put it in this field in my returned data shape, are delegated to the IDataMapper. This separation of logic means that it’s very easy to maintain our code. We can add DataMapper logic for new data shapes and never touch any of the logic for GetSingle, GetList, and GetPage. In fact, our logic for a specific data shape is encapsulated in a single IDataMapper class that touches nothing else in the application. That means it’s easy to change and maintenance friendly. Plus, don’t forget that we have the reflection-based GenericMapper, so we never have to create a new IDataMapper if we don’t want to.

Next time we’ll take a look at a data shape specific IDataMapper and the DataMapperFactory. In the meantime, the full source for SqlDao is included below. Have a look at the implementation for GetDataPage<T> and see if it makes sense now that we’ve gone over GetSingle<T> and GetList<T>. The CreateParameter methods are worth a look too. I went with a design where I just overload the method CreateParameter for each type of parameter that I need to create, so the usage syntax for CreateParameter(Guid value) and CreateParameter(int value) are almost identical. Each CreateParameter method was designed with the target of enabling that single line syntax that I use: command.Parameters.Add(dao.CreateParameter("@Email", email, 100)); so each method contains automatic null value checking. You’ll also see some methods that weren’t in my original spec. While using this framework on different projects I’ve found it convenient to have methods like GetSingleInt32(command) and GetStringList(command).

Finally you can download a small, one page sample app that uses aapl at http://aapl.codeplex.com/.

using System;
using System.Linq;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.Data;
using System.Configuration;
using hf.Common;
using hf.Common.DataShapes;
using hf.Core.Persistence.DataMappers;
 
 
namespace hf.Core.Persistence
{
    public class SqlDao
    {
 
        #region "Database Helper Methods"
 
 
        // Connection
        private SqlConnection _sharedConnection;
        public SqlConnection SharedConnection
        {
            get
            {
                if (_sharedConnection == null)
                {
                    _sharedConnection = new SqlConnection(ConfigurationManager.ConnectionStrings["hf"].ConnectionString);
                }
                return _sharedConnection;
            }
            set
            {
                _sharedConnection = value;
            }
        }
 
 
        // Constructors
        public SqlDao() { }
        public SqlDao(SqlConnection connection)
        {
            this.SharedConnection = connection;
        }
 
 
        // GetDbSqlCommand
        public SqlCommand GetSqlCommand(string sqlQuery)
        {
            SqlCommand command = new SqlCommand();
            command.Connection = SharedConnection;
            command.CommandType = CommandType.Text;
            command.CommandText = sqlQuery;
            return command;
        }
 
 
        // GetDbSprocCommand
        public  SqlCommand GetSprocCommand(string sprocName)
        {
            SqlCommand command = new SqlCommand(sprocName);
            command.Connection = SharedConnection;
            command.CommandType = CommandType.StoredProcedure;
            return command;
        }
 
 
        // CreateNullParameter
        public  SqlParameter CreateNullParameter(string name, SqlDbType paramType)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.ParameterName = name;
            parameter.Value = DBNull.Value;
            parameter.Direction = ParameterDirection.Input;
            return parameter;
        }
 
 
        // CreateNullParameter - with size for nvarchars
        public  SqlParameter CreateNullParameter(string name, SqlDbType paramType, int size)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.ParameterName = name;
            parameter.Size = size;
            parameter.Value = DBNull.Value;
            parameter.Direction = ParameterDirection.Input;
            return parameter;
        }
 
 
        // CreateOutputParameter
        public  SqlParameter CreateOutputParameter(string name, SqlDbType paramType)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.ParameterName = name;
            parameter.Direction = ParameterDirection.Output;
            return parameter;
        }
 
 
        // CreateOuputParameter - with size for nvarchars
        public  SqlParameter CreateOutputParameter(string name, SqlDbType paramType, int size)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.Size = size;
            parameter.ParameterName = name;
            parameter.Direction = ParameterDirection.Output;
            return parameter;
        }
 
 
        // CreateParameter - uniqueidentifier
        public  SqlParameter CreateParameter(string name, Guid value)
        {
            if (value.Equals(NullValues.NullGuid))
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.UniqueIdentifier);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.UniqueIdentifier;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateParameter - int
        public  SqlParameter CreateParameter(string name, int value)
        {
            if (value == NullValues.NullInt)
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.Int);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.Int;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateParameter - bool
        public SqlParameter CreateParameter(string name, bool value)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = SqlDbType.Bit;
            parameter.ParameterName = name;
            parameter.Value = value ? 1 : 0;
            parameter.Direction = ParameterDirection.Input;
            return parameter;
        }
 
 
        // CreateParameter - datetime
        public  SqlParameter CreateParameter(string name, DateTime value)
        {
            if (value == NullValues.NullDateTime)
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.DateTime);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.DateTime;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateParameter - nvarchar
        public  SqlParameter CreateParameter(string name, string value, int size)
        {
            if (String.IsNullOrEmpty(value))
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.NVarChar);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.NVarChar;
                parameter.Size = size;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateTextParameter
        public SqlParameter CreateTextParameter(string name, string value)
        {
            if (String.IsNullOrEmpty(value))
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.Text);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.Text;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
        #endregion
 
 
 
        #region "Data Projection Methods"
 
 
        // ExecuteNonQuery
        public void ExecuteNonQuery(SqlCommand command)
        {
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                command.ExecuteNonQuery();
            }
            catch (Exception e)
            {
                throw new Exception("Error executing query", e);
            }
            finally
            {
                command.Connection.Close();
            }
        }
 
 
        // ExecuteScalar
        public Object ExecuteScalar(SqlCommand command)
        {
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                return command.ExecuteScalar();
            }
            catch (Exception e)
            {
                throw new Exception("Error executing query", e);
            }
            finally
            {
                command.Connection.Close();
            }
        }
 
 
        // GetSingleValue
        public T GetSingleValue<T>(SqlCommand command)
        {
            T returnValue = default(T);
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    if (!reader.IsDBNull(0)) { returnValue = (T)reader[0]; }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnValue;
        }
 
 
        // GetSingleInt32
        public Int32 GetSingleInt32(SqlCommand command)
        {
            Int32 returnValue = default(int);
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    if (!reader.IsDBNull(0)) { returnValue = reader.GetInt32(0); }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnValue;
        }
 
 
        // GetSingleString
        public string GetSingleString(SqlCommand command)
        {
            string returnValue=null; 
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    if (!reader.IsDBNull(0)) { returnValue = reader.GetString(0); }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnValue;
        }
 
 
        // GetGuidList
        public List<Guid> GetGuidList(SqlCommand command)
        {
            List<Guid> returnList = new List<Guid>();
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    returnList = new List<Guid>();
                    while (reader.Read())
                    {
                        if (!reader.IsDBNull(0)) { returnList.Add(reader.GetGuid(0)); }
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnList;
        }
 
 
        // GetStringList
        public List<string> GetStringList(SqlCommand command)
        {
            List<string> returnList = new List<string>();
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    returnList = new List<string>();
                    while (reader.Read())
                    {
                        if (!reader.IsDBNull(0)) { returnList.Add(reader.GetString(0)); }                    
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnList;
        }
 
 
        // GetSingle
        public T GetSingle<T>(SqlCommand command) where T : class
        {
            T dto = null;
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }                
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
                    dto = (T)mapper.GetData(reader);
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            // return the DTO, it's either populated with data or null.
            return dto;
        }
 
 
        // GetList
        public List<T> GetList<T>(SqlCommand command) where T : class
        {
            List<T> dtoList = new List<T>();
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }   
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
                    while (reader.Read())
                    {
                        T dto = null;
                        dto = (T)mapper.GetData(reader);
                        dtoList.Add(dto);
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            // We return either the populated list if there was data,
            // or if there was no data we return an empty list.
            return dtoList;
        }
 
 
 
 
        // GetDataPage
        public DataPage<T> GetDataPage<T>(SqlCommand command, int pageIndex, int pageSize) where T : class
        {   
            DataPage<T> page = new DataPage<T>();
            page.PageIndex = pageIndex;
            page.PageSize = pageSize;
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
                    while (reader.Read())
                    {
                        // get the data for this row
                        T dto = null;
                        dto = (T)mapper.GetData(reader);
                        page.Data.Add(dto);
                        // If we haven't set the RecordCount yet then set it
                        if (page.RecordCount == 0) { page.RecordCount = mapper.GetRecordCount(reader); }
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return page;
        }
  
        #endregion
 
    }
}

Agile ADO.Net Persistence Layer Download is on CodePlex

noreply@blogger.com (Rudy Lacovara) — Fri, 29 Jan 2010 19:29:00 +0000

I’ve have quite a few requests for sample code so I pulled the AAPL code out of my app, put it in a stripped down ASP.Net MVC sample app and published it on CodePlex. You will need Visual Studio 2008, ASP.Net MVC Framework version 1.0, and Sql Server 2008 to run the sample app. A db backup file is included in the zip and in the source code. You can download the code at:

http://aapl.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=39653

Agile ADO.Net Persistence Layer Part 4: Writing data access for new data shapes

noreply@blogger.com (Rudy Lacovara) — Mon, 25 Jan 2010 16:17:00 +0000

What’s a new data shape?

Has this ever happened to you? You have a persistence architecture that works beautifully. It’s effortless to get and save entities, everything works great. Then you get a requirement for a search results page that will display data from your entity, fine. But then they also want the grid to display fields from another entity entirely. It’s not even in the same aggregate! You find yourself thinking things like what a stupid requirement, why would anyone want to do that. But the real problem isn’t with the requirement. The problem is that your beautiful architecture is inflexible. It works great as long as you’re dealing with nicely segmented entities, but as soon as you need to handle data that is a composite of fields from two different entities, things fall apart. The worst part of it is, in the back of your mind you’re thinking “it would be so easy to just write this in TSQL”.

Another scenario you once again have your beautiful architecture. This time you need to add a couple of fields to an entity. It’s a pretty simple change, but you’re dreading it because you know that in order to make this simple change, first you have to modify all of the CRUD sprocs, then you need to modify the method signatures on every data access method for this entity in the DAL, then you need to modify the parser (or data mapper methods), then you need to modify the BAL methods that call the modified DAL methods, then you need to modify the entity class itself. Once again, the architecture is inflexible. Changes are painful. I call this friction.

Changes like this happen all the time. The majority of our time as developers is spent coding changes to existing systems. Perversely, we usually design architectures to be really easy to use for green field projects, but really hard to use when we need to make changes to those same systems. Making it easy to deal with change was my main objective when coming up with this architecture. In this post I’m going to go over solutions to the two scenarios above using Agile ADO.Net Persistence Layer.

Scenario 1: Adding fields to an existing entity

Just to refresh your memory, our solution looks like the screenshot below. Common contains a DataShapes folder that contains the class definition for every DTO (Data Transfer Object) used in our BAL. For this scenario we’re going to be working with the Category DTO. Our BAL contains a Services folder that contains a service class that provide business logic and persistence logic for each aggregate (that’s service in the DDD sense, not a web service). We’ll be working with the CategoryService.

So, our Category data shape is a straight DTO. It has no methods, just properties. It looks like this.

public class Category
{
    public Guid CategoryGuid { get; set; }
    public string CategoryName { get; set; }
    public string CategoryKey { get; set; }
 
    public Category()
    {
        CategoryGuid = NullValues.NullGuid; 
        CategoryName = NullValues.NullString;
        CategoryKey = NullValues.NullString;
    }
}

Our CategoryService class contains business logic and data access methods. Each data access method does three things, it defines a query, it selects a data shape to return, then it passes both the query and data shape to our DAO (Data Access Object) which executes the query and maps the results to the data shape. Our data access methods in the Category Service look like this. You can see both methods returns a List<Category>.

public List<Category> GetAllCategories
{
      string query = @"SELECT * 
                    FROM Category 
                    ORDER BY CategoryName";
      SqlDao dao = SharedSqlDao;
      SqlCommand command = dao.GetSqlCommand(query);
      return dao.GetList<Category>(command);
}
 
 
public List<Category> GetAllCategoriesInUse
{
     string query = @"SELECT DISTINCT c.*
                    FROM Category c
                    JOIN PostCategory p on p.CategoryGuid=c.CategoryGuid
                    ORDER BY CategoryName";
     SqlDao dao = SharedSqlDao;
     SqlCommand command = dao.GetSqlCommand(query);
     return dao.GetList<Category>(command);
}

Now for our simple change. We forgot to add a CreatedUtc field to our Category entity to store the date when a category was created. Crap. We already have a bunch of code written that uses these classes. How hard is it going to be to make this change without breaking any of our data access methods and UI code. Well the first step is obvious, we need to add CreatedUtc to the database table. So we do that. Now for the tough part. Do we need to modify any sprocs? No. Do we need to modify any DAL methods? No. How about the data access methods in our service class, we need to change those right? Nope. The queries defined in the data access methods use “SELECT *” so they’ll pick up any new fields added to the Category table. How about the DAO, surely we need to change some mapping code there? Nope, it uses a little reflection to automatically map query results to a data shape based on field name. As long as we use CateogryUtc for both our database column name and our Category DTO field name, no code changes are needed. So what do we need to do?? Just one thing, we need to add CreatedUtc to our Category data shape like this.

public class Category

{

public Guid CategoryGuid { get; set; }

public DateTime CreatedUtc { get; set; }

public string CategoryName { get; set; }

public string CategoryKey { get; set; }

public Category()

{

CategoryGuid = NullValues.NullGuid;

CreatedUtc = NullValues.NullDateTime;

CategoryName = NullValues.NullString;

CategoryKey = NullValues.NullString;

}

Done. You have to admit, even for a simple change, that was pretty easy. Just add the new field to the Category DTO, add the new column to the table, and the persistence layer handles the rest. Now I know some of you are thinking “But what if I don’t want to use the same names for my columns and DTO fields” or “No way would I ever use reflection in my DAL, it’s too slow”. Those are valid concerns, and I have an easy way to tune the DAO to address both of those issues, but the nice part is that if you’re willing to work within these constraints, your development can go quickly and with very little friction.

Scenario 2: Adding a new data shape

Now let’s look at the case where we need a new composite data shape that consists of data from two different aggregates. Our site has Categories, Blogs, and BlogPosts. The customer tells us that we need to display a list of the the most popular categories along with a count of how many BlogPosts are in each Category. This doesn’t really fit with how we did our initial modeling for this project. A BlogPost contains a collection of Categories not the other way around. Still, the data model will support it, we just have a simple join table between the Category table and the BlogPost table. There’s no reason we can’t write a query that uses the relationship in the other direction.

So, our solution is to add a new data shape called CategoryWithPostCount that will contain the category data plus a count of blog posts that are using that category. Because we’re using an architecture that anticipates and embraces change, adding this new data shape will be a simple 2 step process. First we create the CategoryWithPostCount DTO and add it to our Common/DataShapes folder.

public class CategoryWithPostCount
    {
        public Guid CategoryGuid { get; set; }
        public string CategoryName { get; set; }
        public string CategoryKey { get; set; }
        public int PostCount { get; set; }
 
        public CategoryWithPostCount()
        {
            CategoryGuid = NullValues.NullGuid; 
            CategoryName = NullValues.NullString;
            CategoryKey = NullValues.NullString;
            PostCount = NullValues.NullInt;
        }
    }

Second, we open up our CategoryService class and add a new GetTopCategoryList data access method. Remember that our framework does most of the heavy lifting for us. All we need to do is define the query, pick a data shape, and then pass both to the DAO which will automatically map them by field name. Writing the method is easy. We fire up Sql Server Management Studio, write and test our query, copy one of the other data access methods, and paste our new query into it. Here’s the result.

public List<CategoryWithPostCount> GetTopCategoryList(int pageSize)
{
    string query = string.Format(
                    @"SELECT TOP {0} c.*, q1.PostCount
                    FROM Category c
                    JOIN (
                        SELECT c.CategoryGuid, count(pc.PostGuid) as PostCount
                        FROM PostCategory pc 
                        JOIN Category c ON c.CategoryGuid = pc.CategoryGuid 
                        Group By c.CategoryGuid
                        ) q1 ON q1.CategoryGuid = c.CategoryGuid
                    ORDER BY q1.PostCount DESC
                    ", pageSize);
    SqlDao dao = SharedSqlDao;
    SqlCommand command = dao.GetSqlCommand(query);
    return dao.GetList<CategoryWithPostCount>(command);
}

That’s it, we’re done. A brand new data shape complete with data access method ready to use in minutes. When it’s this easy to add new data shapes, it really frees you up to work in a much more agile way. You don’t need to worry about getting entity classes designed exactly right the first time, or missing pieces of data like a count or date created. If you do, the architecture is designed to make it painless to add that stuff as it’s needed. And none of this is hard, none of it is magic, we’ve just made some decisions and accepted some limitations (like DTO field names and table column names will be the same) that allow us to take the friction out of design changes.

Next time we’ll get into the DOA and the associated DataMapper classes.

Agile ADO.Net Persistence Layer: Part 3 Service Class Single Data Access Method

noreply@blogger.com (Rudy Lacovara) — Tue, 19 Jan 2010 17:11:00 +0000

When I say data access methods, I’m talking about the methods my UI is going to call whenever it needs to get data. When my Posts controller needs a list of BlogPosts to display, it’s going to call a BAL method like GetAllBlogPosts() or GetAllBlogPostsForCategory(). Last time I mentioned (over and over) that I like to keep things simple. When I need to get or save data, I don’t want to have to search through 3 different classes just to find the one with the method I need. Instead, I’m putting all my persistence logic for a given aggregate in just one place, a service class. This is not a web service. I’m using service in the Domain Driven Design sense here. That means that I have a BlogService class that is my one stop shop for all persistence that has to to with Blogs, BlogPosts, SubmittedBlogUrls, and anything else that falls within the Blog aggregate. Here is what my BlogService class looks like. You can see that it’s mostly “Get” data access methods.

What’s an Aggregate?

I keep using the word aggregate. If you’re not familiar with the term, it just means a group of entities that all share the same persistence class (whether that be a repository, a service, or something else). This is a key concept in Domain Driven Design. If you want to know more I would recommend picking up Eric Evans’ book or Jimmy Nilsson’s book on DDD. For now, all you need to know is that a BlogPost can never exist without a Blog, so there’s no point in BlogPost having it’s own persistence class. In fact we find that if we do break BlogPost out into it’s own persistence class, it will lead to problems down the road due to BlogPost’s dependency on Blog. What’s the solution? We put data access methods for both Blog and BlogPost in the same persistence class and call it an aggregate. That is why BlogService has methods for both Blog and BlogPost entities.

What type of data will data access methods return?

We covered this last post, but to recap all data will be returned as a Data Transfer Object (DTO). The DTOs are all defined in our Common assembly in the DataShapes folder. Our BAL will return data in one of the following 4 formats.

a single DTO
a List<DTO>
a DataPage<DTO>
a string value

For more see last week’s post Agile ADO.Net Persistence Layer: Part 2 Use DTOs.

A simple Single<DTO> data access method

Let’s look at the simplest possible data access method. GetBlogPost() takes a postGuid for a parameter, defines the query to find the BlogPost entity for that postGuid, and then returns the result as a single BlogPost DTO. Here’s the complete method.

public BlogPost GetBlogPost(Guid postGuid)
{
    string query = @"SELECT p.*, s.Score 
                    FROM [dbo].[BlogPost] p
                    LEFT JOIN [dbo].[BlogPostReputationScore] s on s.PostGuid = p.PostGuid
                    WHERE PostGuid = @PostGuid";
    SqlDao dao = new SqlDao();
    SqlCommand command = dao.GetSqlCommand(query);
    command.Parameters.Add(dao.CreateParameter("@PostGuid", postGuid));
    return dao.GetSingle<BlogPost>(command);
}

The first thing you’ll notice is that this isn’t a lot of code. All we’re really doing here is defining a parameterized TSQL query, wrapping that query up in a SqlCommand, and then passing the command and our desired return type off to a Data Access Object (DAO) that automagically executes the command and maps the results to our desired type. It may seem counter intuitive to write code like this when we haven’t even written the DAO yet, but that’s exactly how I did it when I wrote this code for the very first time. I decided that my data access methods should be very simple. I would start with the query and the DTO type that I wanted it to return, then I would pass them both to some type of helper class that would handle the details of running the query and figuring out how to map the query results to the properties of my DTO. By using this top down approach, I gave myself a very clear picture of how I needed my DAO to behave.

What’s a DAO (Data Access Object)?

By looking at the query logic above, you can see that I have this thing called a DAO or Data Access Object. This is a class that encapsulates the helper logic for working with my database. The DAO handles things like creating parameters, getting a connection, and most importantly it implements methods to return my four main data formats, GetSingle<DTO>, GetList<DTO>, GetDataPage<DTO>, and GetStringValue(). The DAO and it’s associated DataMappers are where you’ll find the special sauce that makes this architecture work. We’ll get into their implementation later on.

A BAL that embraces change

It’s easy to look at the simple code above and miss something that I think is very important. In fact that thing is the whole reason that I wrote this framework. That simple data access method above is the blueprint for a flexible persistence layer that makes changing your entities and associated persistence code easy and almost painless. It sets up a simple 3 step process for all data access in your application.

Define a DTO in the exact data shape that you’re looking for. That means create a DTO property for each data field that you need out of the database.
Define a query that gets the data. It can be as simple or as complex as you like. You can develop it in Sql Server Management Studio. You can easily optimize it. Use whatever process or tools work for you. When you’re done just paste the query into your data access method.
Pass both the query and your DTO to the DAO and it will automatically handle field mappings and pass the results back in the data shape you requested.

This is a very powerful way to work. I can’t count the number of times that I’ve worked with and architecture where I dreaded any changes because I knew that any data fields added would require me to modify a sproc, a DAL method, a BAL method, parsing logic, an entity class, it all adds up to a lot of friction that resists any change. This BAL design embraces change. It’s written with the attitude that we know change is going to happen so we’re going to give you as few things as possible to modify, and make sure we don’t have any cross cutting dependencies, so that you can make changes easily.

Next time, more on the service classes.

Next Post: Agile ADO.Net Persistence Layer Part 4: Writing data access for new data shapes

Agile Ado.Net Persistence Layer Part 2: Use DTOs (Data Transfer Objects)

noreply@blogger.com (Rudy Lacovara) — Wed, 13 Jan 2010 15:45:00 +0000

What container are we going to use to pass data between the layers of our application? The usual answers I hear are either DataTables/DataSets or full business objects. I don’t like any of these options. DataSets and DataTables come with significant overhead and they don’t contain strongly typed data. Business objects do contain strongly typed data, but they typically contain a lot of extra business logic that I don’t need, and they may even contain persistence logic. I don’t want any of that. I want the lightest weight, simplest possible container that will give me strongly typed data, and that container is a Data Transfer Object (DTO). DTOs are simple classes that contain only properties. They have no real methods, just mutators and accessors for their data. Below is a class diagram for several DTO classes. You’ll notice that each one has only properties and a constructor, no other logic at all.

So I’m going to have a strongly typed DTO class for each “data shape” that needs to go in or out of my BAL. This concept of a data shape is something that will come up again and it’s actually pretty central to how I’ve designed this architecture. For now just know that a data shape is a DTO.

DTO, List<DTO>, DataPage<DTO>, and string

Sometimes getting a single DTO will be good enough. For example, when I want to get a single BlogPost, code like the following works just fine.

BlogService service = new BlogService();
BlogPost latestPost = service.GetMostRecentBlogPost();

But most of the time I’m going to be dealing with collections of objects. The collection of choice is a generic list, List<T>, where T is my DTO type. So if I need to get a collection of all BlogPosts, that data would come back to me as a List<BlogPost> as shown below.

BlogService service = new BlogService();
List<BlogPost> list = service.GetListOfAllBlogPosts(BlogPostSortOption.ByDate);

So between single DTO and List<DTO>, we’ve got most of our data needs covered but there’s still a couple more. What about paging? Take the example above. There’s no way that I’m ever going to get a list of all BlogPosts and display it all at once. I’m going to break it up into pages. What I need is a generic DataPage<T> class that I can use to encapsulate a single page of data along with some metadata like PageSize and PageIndex. The DataPage class is defined in our Common assembly.

public class DataPage<T>
{
    public List<T> Data = new List<T>();
    public int RecordCount = 0;
    public int PageSize = 20;
    public int PageIndex = 0;
    public int PageCount { get{ return (RecordCount ==0 || PageSize==0) ? 0 
                         : (RecordCount + PageSize - 1) / PageSize;}}
}

It’s a pretty simple class. Note the very concise algorithm for calculating PageCount. That isn’t mine, I picked it up on some website but I can’t remember where. Anyway it works really well. So now that we have our DataPage<T> defined in the Common assembly, if we want to divide our BlogPosts up into pages of 20, and we want to get the third page, we’ll use code like this.

BlogService service = new BlogService();
int pageSize = 20;
int pageIndex = 2;
DataPage<BlogPost> page = service.GetPageOfAllBlogPosts(pageSize, pageIndex, BlogPostSortOption.ByDate);

We’ll see later on that having this simple DataPage<T> class really makes paging easy to deal with. Now we’ve almost got all of our bases covered, but there’s one more input/output type that we need to consider, a simple string value. At some point we’re going to want to get a simple value out of our persistence layer. If we don’t think of it ahead of time we’ll be tempted to do something silly like create a DTO class that contains a single string field, not I would ever do something like that … at least not again. Anyway, we will want to get a simple value at some point. Let’s just plan for it now and require that our persistence layer has the capability to return string values.

Next time, a look at Service classes and query logic

So, we haven’t written any actual persistence code yet, but we know a lot about how our persistence layer is going to behave. We know that all data going in or out of our BAL is going to be in the shape of a single DTO, a List<DTO>, a DataPage<DTO>, or a string value. We also have some sample code that demonstrates how we expect our service classes to work. By the way, this isn’t just a case of hindsight being 20/20. When designing code I’ll typically write the consuming code first, because it really guides my development of the lower level framework code and sets up requirements for how that code will need to behave. Next time we’ll continue this top down approach and write a service class.

Next Post: Agile ADO.Net Persistence Layer Part 3: Service Class Single<DTO> Data Access Method

Agile Ado.Net Persistence Layer Part 1: Design Overview

noreply@blogger.com (Rudy Lacovara) — Sat, 09 Jan 2010 22:54:00 +0000

Last year I did a blog post series on how to design a High Performance DAL using Ado.Net. Judging by the response I’ve gotten from that series, there must be a lot of developers out there who believe that even with the availability of LINQ, Entity Framework, and a host of other ORM technologies, Ado.Net is still your best option when designing a persistence layer. BTW, I’m one of them.

After that series I started digging into Entity Framework and LINQ, and I was impressed by how effortless those technologies made certain parts of application development. Once the EF or LINQ mappings were in place, I found myself writing much less code and focusing more on the business logic of my application. I also found myself driven right back to ADO.Net as I found myself struggling with how do something that I already knew how to do in T-SQL, or fighting errors resulting from an attached data context object that I really didn’t want to start with.

So, I found myself back with ADO.Net, but I didn’t want to give up the ease of development and coding efficiencies that I got from the ORMs. I decided to take a fresh look at how to design an ADO.Net persistence layer. I started at the top (the application layer) and thought about how I want my app code to consume business logic, and then I worked down from there. I incorporated many of the best practices that I’ve used over the years, but I also looked critically at each one and whenever I found that something was slowing me down or leading to duplicate code, I threw it out. The resulting architecture is quick to develop on,testable, easily maintainable, and can be easily optimized for performance. This series of posts will detail the entire design, from application code to database.

A peek at the final design

I always find it’s easier to follow along if I have some idea where I’m going, so this is a quick look at where we’re headed. We’re going to cover the entire architecture for a simple blog aggregator called RelevantAssertions.com. RelevantAssertions is an Asp.Net MVC application that uses our new Agile Ado.Net Persistence Layer. We have 4 projects in the RA solution, WebUI, Tests, Common, and BAL Here’s a quick look.

WebUI

WebUI is our Asp.Net MVC application, that’s our application layer. This contains all UI and presentation logic, but it contains absolutely no business logic.

Common

Common contains classes that we need at all layers of our code. The DataShapes folder is where we define all of our DTO classes.

Tests

This project contains all of our automated tests for both the BAL and the UI.

BAL

I know it’s probably more correct to say BLL, but I like the term BAL. It just sounds better. This is the big class where everything interesting happens. The main workhorses of the BAL are the service classes. These are not web services. They are service classes in the DDD sense. The service classes are going to be the one stop shop where our application code goes to do whatever it needs to do. The service classes will also contain query logic, that’s right I said query logic. Behind the scenes the service classes will use DAOs (Data Access Objects), Data Mappers, and Persisters to do their thing in an efficient object oriented way, but the only classes our application code will use directly are the services.

What, no DAL??

You’ll notice that there is no DAL. It seems a little strange to have an architecture that focuses on ADO.Net but doesn’t have a DAL, but there’s a reason. Usually, the DAL is where I’ll put my query logic, mappings to query results, and any other database specific code. The DAL would allow me to keep all of my TSQL and ADO.Net code separated from the rest of my application, and this separation provided me with some important benefits like:

1) Separation is it’s own virtue, that’s just the right way to do it.
2) I wouldn’t have leakage of db or query logic into my business logic.
3) I could easily swap out SQL Server with another database if needed.
4) It encapsulates code that would otherwise be repeated.
5) We need to hide TSQL from programmers, it scares them.
6) It’s fun to make changes to 3 layers of code every time I add a new data member to an entity class.

At least that’s what I was always taught. But after working with more ORM oriented architectures and the Domain Driven Design way of doing things, I started to look at things differently. Let’s look at some of these benefits (at least the ones that aren’t sarcastic).

I’ve never met anyone who’s ever switched out their database

YAGNI means You Ain’t Gonna Need It. The idea is that we spend a lot of time building stuff that we don’t really need. We build it because it seems like the architecturally correct way to do it, or we think we’ll need the feature one day, or maybe we’re just used to doing it that way. Whatever the reason, the result is that we spend a lot of time coding features that are never used, and that’s not good. After doing this for 14 years or so, I’ve never, ever, run into a single project where they’ve decided “hey, let’s trash the years of investment we’ve made in SQL Server and switch over to MySQL, or any other database. Now I am aware that a db switch is likely if you’re writing a product that clients install onsite and it has to work with whatever their environment is, but for 99% of .Net developers this is just never going to happen. I call YAGNI on this one.

Query logic IS business logic

One of the biggest gripes I had when I started investigating LINQ, EF, and Hibernate (yes I was looking at Java code) architectures is that they had query logic in their repository classes. Now the query logic was written in LINQ, or EntitySQL, or some other abstracted query language, but it was still query logic. Blasphemy!! You can’t put query logic in a BAL class! That stuff has to be abstracted away in the DAL or it will contaminate the rest of the application architecture! Our layered architecture is being violated! Worlds are colliding! It’ll be chaos!! Then I started to notice something, it’s really easy to develop business logic when you include queries in the BAL. In the past I would put my queries in a sproc, then I would write a DAL wrapper for the sproc, and a BAL wrapper for the DAL method. Then, if the query changed, or if I needed an identical query but with a slightly different parameter list I would write a new sproc, then write a new DAL wrapper, then write yet another BAL wrapper method for the DAL wrapper method. By the time all was said and done I would have this crazy duplication of methods across all layers of my application including my database! And don’t even get me started on the crazy designs that I implemented to try and pass query criteria structures (basically the stuff that goes in the where clause) between my BAL an my DAL. I came up with these crazy layers of abstraction that basically existed so that I wouldn’t have to create a simple TSQL WHERE clause in my BAL. Then there’s the problem of handling sorting and data paging, that required even more DAL methods, and each of these DAL methods had corresponding wrapper methods in the BAL that did nothing but pass the call through to the DAL! Why?? I was doing the right thing by separating my business logic from my query logic, why was it so painful? The answer I finally arrived at is simply that query logic is business logic. I’d been putting a separation where no separation belonged.

All real programmers know TSQL

I’ve heard the argument that TSQL is too hard for programmers so we’re going to create something much easier for programmers to use like LINQ or EF. The problem is that these tools require almost exactly the same syntax as TSQL but they put an extra layer of stuff in there that can break and a data context (or session for you nHibernate folks) that throws errors whenever you try to save a complex object graph. How did this attitude that TSQL is a problem for programmers gain any traction? Have you ever met a real programmer who can’t write TSQL? And if you did meet such a person, would you let them touch your business layer code? Why would we ever want to abstract TSQL away? It’s the perfect DSL for accessing SQL Server data and every programmer in the world is already familiar with it.

Using good object oriented design and encapsulating data access code is a good thing

I fully believe this one, but once we decide that query logic is business logic and that we don’t need to hide TSQL from programmers, there’s no reason to put our well designed object oriented data access code in a separate project and call it a DAL. I decided to just put it in a Persistence folder in my BAL and now I have one less DLL to worry about.

So, that’s some of what I was thinking when I made the decisions that I did. It made sense to me. I’m sure it won’t make sense to everyone, but I do think that it resulted in a very usable architecture. Before I wrap up for today, I want to look at one more thing.

The target application code experience. What will it be like to use?

When I’m writing code in my application layer, consuming the logic that is provided through my BAL service classes, what does that code look like. Well I know a couple of examples of code I don’t want it to look like.

I’ve been in a few environments where there were huge libraries of BAL classes, any of which could contain the logic I want. I would often have to resort to a solution wide text search looking for sproc names or keywords that might exist in the method that I needed. I don’t want that. I want everything I need to be in one easy to find place.

I’ve also seen a practice that’s common in the DDD (Domain Driven Design) crowd where you’ll need to go to a factory class to create a new entity, you need to go to a repository class to get an entity from the database, if you have complex logic that involves more than one entity you need to go to a service class, and saving entities is a toss up between using either the repository or a separate service class. There may be a good reason to use that kind of class design inside of the BAL, but when I’m writing code in my application layer, I don’t want to have to worry about which of 4 different classes I’m going to use. So again, I’m a simple guy, when I need to get, save, or validate a BlogPost entity, I want a single service class that I can go to for everything. My app code should look something like this.

// instantiate service classes 
BlogService blogService = new BlogService();
CategoryService categoryService = new CategoryService();
// Get data shaped as lists and pages of our DTOs
DataPage<BlogPost> page = blogService.GetPageOfBlogPosts(pageSize.Value, pageIndex.Value, sortBy);
List<Category> categoryList = categoryService.GetTopCategoryList(30);
// create and save a new BlogPost
BlogPost newPost = new BlogPost();
newPost.BlogGuid = blog.BlogGuid;
newPost.PostTitle = item.Title.Text;
newPost.PostUrl = item.Links[0].Uri.AbsoluteUri;
newPost.PostSummary = item.Summary.Text;
newPost.Score = 0;
blogService.Save(newPost);

Next time we’ll focus less on discussion and more on code. We’ll look at DTO classes and the 4 main data shapes that will go into and come out of our BAL: DTO, List<DTO>, DataPage<DTO>, and String.

Next Post: Agile ADO.Net Persistence Layer Part 2: Use DTOs

How do you test CRUD methods? Just hit the database and be happy.

noreply@blogger.com (Rudy Lacovara) — Tue, 29 Dec 2009 00:21:00 +0000

It’s time for a little programmer heresy. For the last few months I’ve been reevaluating all of my “best practices” and throwing out anything that creates friction. Friction is a term I’ve heard Jeff Atwood use fairly often. It’s a fuzzy term but I think I understand what he’s getting at. When you’re writing code and you’ve got some momentum going, any time you find yourself struggling against the persistence framework, any time you find yourself jumping through hoops to find that bit of code that’s throwing an error (and is probably called through reflection), any code that makes you scream inside but you have to write it to comply with some standard, pretty much anything that slows you down, that is friction. Sometimes friction is justified. Most of the time it isn’t.

Background

So I’ve been doing more and more unit testing lately. One thing that I run up against often is how to test persistence (Create Read Update Delete) methods. Below is a diagram of some persistence classes from an application I’m working on. The classes below handle persistence for my SubmittedBlogUrl entity. BlogService is the one stop shop that my application layer uses to do everything related to blogs. In Domain Driven Design parlance, the BlogService handles getting and saving data for all classes that are part of the Blog aggregate. Getting and saving BlogProfiles, BlogPosts, and SubmittedBlogUrls all takes place through the BlogService class. That doesn’t mean that BlogService contains all of the persistence logic though. I use a strategy pattern that delegates the actual Delete, Insert, Save, and Update methods to separate persister classes.

Now the code I want to test are the Delete, Insert, and Update methods in my SubmittedBlogUrlPersister class. The methods look like this.

public class SubmittedBlogUrlPersister
    {
 
        public void Save(SubmittedBlogUrl dto)
        {
            if (dto.Id == NullValues.NullInt)
            {
                Insert(dto);
            }
            else
            {
                Update(dto);
            }
        }
 
        public void Insert(SubmittedBlogUrl dto)
        {
            SqlRepository repository = new SqlRepository();
            string sql = @"INSERT INTO [dbo].[SubmittedBlogUrl]
                                ([SubmittedByUserGuid]
                                ,[SubmittedOnUtc]
                                ,[IpAddress]
                                ,[Status]
                                ,[BlogUrl])
                            VALUES
                                (@SubmittedByUserGuid
                                ,@SubmittedOnUtc
                                ,@IpAddress
                                ,@Status
                                ,@BlogUrl)
                            SELECT SCOPE_IDENTITY()";
 
            SqlCommand command = repository.GetSqlCommand(sql);
            command.Parameters.Add(repository.CreateParameter("@SubmittedByUserGuid",       
              dto.SubmittedByUserGuid));
            command.Parameters.Add(repository.CreateParameter("@SubmittedOnUtc",  
              dto.SubmittedOnUtc));
            command.Parameters.Add(repository.CreateParameter("@IpAddress", dto.IpAddress, 20));
            command.Parameters.Add(repository.CreateParameter("@Status", dto.Status));
            command.Parameters.Add(repository.CreateParameter("@BlogUrl", dto.BlogUrl, 100));
            Object result = repository.ExecuteScalar(command);
            dto.Id = Convert.ToInt32(result);
        }
 
 
        public void Update(SubmittedBlogUrl dto)
        {
            SqlRepository repository = new SqlRepository();
            string sql = @"UPDATE [dbo].[SubmittedBlogUrl]
                           SET [SubmittedByUserGuid] = @SubmittedByUserGuid
                              ,[SubmittedOnUtc] = @SubmittedOnUtc
                              ,[IpAddress] = @IpAddress
                              ,[Status] = @Status
                              ,[BlogUrl] = @BlogUrl
                         WHERE Id = @Id";
 
            SqlCommand command = repository.GetSqlCommand(sql);
            command.Parameters.Add(repository.CreateParameter("@Id", dto.Id));
            command.Parameters.Add(repository.CreateParameter("@SubmittedByUserGuid",  
              dto.SubmittedByUserGuid));
            command.Parameters.Add(repository.CreateParameter("@SubmittedOnUtc",       
              dto.SubmittedOnUtc));
            command.Parameters.Add(repository.CreateParameter("@IpAddress", dto.IpAddress, 20));
            command.Parameters.Add(repository.CreateParameter("@Status", dto.Status));
            command.Parameters.Add(repository.CreateParameter("@BlogUrl", dto.BlogUrl, 100));
            repository.ExecuteNonQuery(command);
        }
 
 
        public void Delete(SubmittedBlogUrl dto)
        {
            SqlRepository repository = new SqlRepository();
            string sql = @"DELETE FROM [dbo].[SubmittedBlogUrl]
                           WHERE Id = @Id ";
 
            SqlCommand command = repository.GetSqlCommand(sql);
            command.Parameters.Add(repository.CreateParameter("@Id", dto.Id));
            repository.ExecuteNonQuery(command);
        }
 
    }

As you can see, these methods just create some parameterized T-SQL, package it in a command, and then pass the command off to my repository to be executed. The key thing that I need to test here is the query logic. I need to make sure I didn’t make any mistakes when writing the SQL and creating parameters, and I need to make sure that the SQL works when run against my database.

Getting to the Tests

So how should I test this? The TDD purists might say that I should fire up a mocking framework, create some mock objects of my repository and make sure that all right things are passed in. I also might need to look at a Dependency Injection framework to make it easier to switch out my real SqlRepository with the mock SqlRepository. And at the end of all this coding I’ll know what??? Well, I’ll just know that I passed a command to a mock object without anything blowing up. I won’t know if my SQL syntax is right, I won’t know if my SQL works with my database, I won’t know anything that I actually need to know. I call this friction. Lots of effort that at the end of the day doesn’t even get me what I need, a valid test of the SQL in my persister class.

My solution, let’s dumb down the tests and get on with coding. I’ll create a single CRUD test that instantiates a real SubmittedBlogUrlPersister (not a mock) then creates a new SubmittedBlogUrl object, saves it to the database, updates it, and finally deletes it. If I make sure that I’m using a dev database (not production) and I make sure that my test cleans up after itself (deletes the data it creates) this should work just fine. The resulting test looks like this.

 [TestClass()]     
publicclassSubmittedBlogUrlPersisterTest{
    [TestMethod()]
    public void CrudTest()
    {
        // create our test object
        var persister = new SubmittedBlogUrlPersister();
        var service = new BlogService();
        var dto = new SubmittedBlogUrl();
        SubmittedBlogUrl dto2;
        var utcNow = DateTime.UtcNow;
        dto.BlogUrl = "testUrl";
        dto.IpAddress = "testIp";
        dto.SubmittedByUserGuid = Guid.NewGuid();
        dto.SubmittedOnUtc = utcNow;
        // insert it
        persister.Insert(dto);
        // get it
        dto2 = service.GetSubmittedBlogUrl(dto.Id);
        Assert.AreEqual(dto.Id, dto2.Id);
        // update it
        dto2.BlogUrl = "NewUrl";
        persister.Save(dto2);
        dto2 = service.GetSubmittedBlogUrl(dto.Id);
        Assert.AreEqual(dto2.BlogUrl, "NewUrl");
        // delete it
        persister.Delete(dto2);
        dto2 = service.GetSubmittedBlogUrl(dto.Id);
        Assert.IsNull(dto2);
    }
}

Technically, this isn’t a unit test, it’s an integration test since it’s hitting a real database, but I think it’s the right approach for this situation. The principle here is that when you reach the point where you’re writing tests for methods that contain query logic (like CRUD methods), that might be the right place to ditch the unit tests and switch over to some simple integration tests that write to a real database and test what you really need tested, your query logic.

Addendum

Great comment that I wanted to draw attention to from Steve Freeman who co-authored the book Growing Object-Oriented Software, Guided by Tests. Steve said..

As a TDD purist who "wrote the book" on Mock Objects and once wrote a foolish paper on mocking out JDBC, I would say that testing against a real database is the right thing to do here (provided you've set up your environment to do this cleanly). The other right thing is to only need such tests at the boundaries of your application, the persistence code should be nicely contained and not leak into the domain code.

That makes a lot of sense to me. I might be off a bit on what I expect to hear from TDD purists. I like Steve’s statement that you need integration tests like this only at the boundaries of your application. In my example, if I’m testing app logic in an MVC controller class that uses my BlogService for persistence, then it makes sense to mock my BlogService and write real unit tests because I really just want to test the logic in my controller. But at the edge of the application, where I’m touching the database, it’s probably more appropriate to switch over to integration tests. I think a good rule of thumb is that if the code you’re testing contains any query logic (SQL, LINQ, etc..), go ahead and hit the db.

Angry-Coders: Requirements Gathering

noreply@blogger.com (Rudy Lacovara) — Mon, 28 Sep 2009 12:53:00 +0000

Links for 1969-12-31 [del.icio.us]

Thu, 01 Jan 1970 00:00:00 PST

something went wrong