Reference Bits

Programmers could learn a thing or two from bridge engineers

noreply@blogger.com (Rudy Lacovara) — Fri, 17 Feb 2017 19:25:00 +0000

I was recently watching a documentary that described the engineering and design of bridges. These bridges, some of them 80 years old, have been in constant use from the moment they were built.

There really was no cookie cutter recipe. For the most part, each bridge was a custom design job that resulted from the engineering best practices at the time it was designed, the available materials, and the opinions of the engineers. Sounds a lot like the software development.

One thing that didn’t sound like software at all was the attitude toward maintenance. The bridge engineers all had respect for the original design and applied their skills to maintain it. They would incorporate new techniques and materials where it supported the intent of the original designers, but there was no push to “refactor” the bridge. As a result the bridges keep working, fulfilling their original purpose and providing value every day.

Imagine how this might be different if each new engineer on the project came in and said “This bridge design is crap. I’m surprised it even works. We know so much more about the right way to build a bridge now. We’re going to have to gut this architecture and update it to implement current best practices.”

How long do you think that bridge would remain in service? Even worse, how reliable do you think that bridge would be after a new set of engineers came in and tried to implement a brand new architecture, producing a mish-mash of the old and the new.

It sounds like an obviously bad idea when talking about bridges, but for some reason we don’t give it a second thought in the software world. We have a chronic lack of respect for those who came before us. We’re constantly rewriting code THAT IS WORKING for no better reason than we don’t like the design.

I can look back at my career and see multiple instances where I’ve been the chief refactor offender, rewriting code that was working, even entire application architectures because I thought they were written poorly. It’s easy to do. There’s no shortage of legitimately bad code or developers who rush to implement the latest tools and ideas without really understanding what they’re doing.

These days I’m a little older and a little wiser. I try to keep 2 important principles in mind

1. Software is not an end unto itself.
Our end customers don’t really care if our services are perfectly RESTful, or if we’ve properly implemented dependency injection throughout our app. They care if it works.

2. There’s always a reason the developer who came before me did things the way they did.
Here’s a cycle I’ve gone through countless times. I see code that works but it’s stupidly implemented, what was this idiot thinking. It’s going to take me an extra hour but let me fix this nonsense. Ok I’m pretty much finished and wait, this key use case doesn’t work. I can fix it, but now the rest of my refactor is broken. Well if I do this and this it will work, but…. oh… it pretty much works the same way it did to start with.

Core Frameworks Don’t Work (most of the time)

noreply@blogger.com (Rudy Lacovara) — Mon, 06 Apr 2015 17:41:00 +0000

I work as a consultant, mostly for large software development organizations at public companies that maintain large systems that handle millions of transactions each day. My typical engagement is at least 1 year, no longer than 2 years. It’s long enough that I’m able to dig deep but it’s short enough that I get exposed to many different organizations. This gives me a unique and informed perspective on how software development works at a fair number of .Net shops. I’m kind of like Marlin Perkins, able to observe the natural behavior of cutting edge software development organizations in the wild.

One interesting ritual that I’ve seen practiced over and over is rite of THE CORE FRAMEWORK. Some organizations actually go as far at to call it THE FRAMEWORK. Others are content to simply call it core. Regardless of what you call it, it’s usually a terrible idea that will cripple the productivity of your organization.

A point of clarification

I want to be clear. This post is about company-wide core frameworks. That’s a common framework that is used as the foundation for multiple applications across the company. That is not the same as an application core library.

An application core library is usually just the business layer for a single application that can be shared across different assemblies at the application layer. For example, the SquareHire core library is used for the SquareHire public website, the SquareHire ATS (webapp), the SquareHire admin app (another web app), the SquareHire processor, and the SquareHire webapi. All of these are different assemblies that share a common data and service backend. Together they make up the SquareHire application so it makes sense that they would share a common SquareHire core library.

A company wide Core Framework is something different. That’s when a company has multiple applications that do not share a common data and service backend. They may have nothing to do with each other. However, the company wants to standardize software architecture across the company so the decision is made to build every application on top of a common, company-wide Core Framework. Often these Core Frameworks are named after the company, like AcmeInc.core. These company-wide core frameworks force a common architecture and code base on apps that do NOT share a common data and service backend. These are the frameworks this post is about.

How does it happen?

A group of programmers build an application. The application works, the company grows, and it hires more developers. As time goes on the more features are added to the application by developers who really don’t understand the original designs. They just kind of bolt code on that works in the way they’re used to doing things. The new code doesn’t follow the patterns established by the original designers, but it does accomplish the short term goal. Besides, the new approach is probably better than the original design anyway.

More time goes on. The company becomes even more successful which means the app is supporting more users. It’s now maintained by a completely new group of programmers, nobody’s really sure who the original developers were. Each new wave of developers implements features using what they feel are the best practices. The new features hang off the original application like tumors. Growths that are powered by the original organism, but refuse to cooperate with it.

Things go on this way. The company is now so successful that it is developing multiple applications, or maybe they’re acquired and merged with another organization and their software projects. However it happens, the point is that we now have a situation where the organization has multiple teams maintaining multiple applications. Each application has a different architecture and is becoming increasingly more difficult to maintain as new generations of programmers keep bolting stuff on that doesn’t work in the way the original app was designed.

This is the moment when somebody has the bright idea “You know what would make our software development efforts much more efficient and would make all of our applications work much better… if we standardized on a single perfect architecture and made everybody in the entire organization adopt it. In fact, we should create a set of Core libraries, and then every new project will be built on top of them.”

The theory of company-wide Core

On the surface, a Core Framework makes a lot of sense. There are some good arguments for standardizing and implementing a common framework across your organization. Here’s a few of them.

1. New development should be faster/easier
New projects will be much easier to get off the ground. They won’t need to write plumbing code like persistence, they’ll get everything they need from the “Core”.

2. Managing development resources (people) should be easier
If you standardize it should be easier for developers to move between teams. They’ll already be familiar with the architecture.

3. The smartest guys can guide us
Since the “Core” will be designed by the architect (or architects) it will embody the best possible architecture for the organization and it will enforce a set of best practices that will be implemented by less capable programmers who might otherwise go astray.

I may have overdone the sarcasm on that last one but the point is that creating a core really does seem like a good idea on paper.

The reality of company-wide Core

The theory sounds good, but the reality doesn’t work out that way. Here’s some real world observations pulled from my time at real organizations that were in various stages of implementing Core libraries. Keep in mind that the architects and developers at these organizations are some of the smartest that I’ve ever worked with. The problem definitely was not a lack of skill or programming experience.

1. New development should be faster/easier
The Core may actually get the initial phase of a new project off the ground a bit quicker, but it doesn’t last. Quickly developers find that the Core brings with it limitations as well as benefits. It’s not long until the developers of the new application are scheduling time with the architect and the Core team to try and figure out how Core can be changed to accommodate the needs of this new application. Did you catch that? Not only does core not meet the needs of the new app, the new application actually winds up driving changes to Core. This happens for every application that consumes it. What’s really been created is a massive cross-application dependency. Nothing brightens my day like pulling the latest build of core and discovering that my entire application is now broken because some other team decided that we needed a better way to manage transactions or the DI container or any of the other things that find their way into these core libraries.

2. Managing development resources (people) should be easier
This benefit actually does happen to some extent. Different apps in the organization will be structured in a similar way so that it’s easier for developers to move between them. Unfortunately this commonality comes with negative side effects.

The worst effect is that software development becomes uniformly slow and difficult. The Core is a cross application dependency that forces the decisions made by one team onto every other team in the organization. This also means that when your team needs a change made to core, it’s not going to happen quickly. You’re going to have to wait for buy in from the architects and other team leads.

I’ve also seen Core negatively effect organizations’ ability to hire. Core usually goes hand in hand with a dramatic increase in complexity. I’ve heard managers express that it’s incredibly difficult to find programmers and once they do it takes 5 or 6 months for them to be productive because the app architecture is just so complex. Does the Core have to be that complex? No, but for some reason it always is… which brings us to number 3.

3. The smartest guys can guide us
The core is indeed designed by the smartest programmers in the organization. This has to be a good thing right? Wrong. There are two big problems here.

First is the problem of misaligned incentives. When a group of programmers is working together to ship a product, their incentives are all pretty much aligned. They want to ship the product. If the product doesn’t ship it’s a big problem for everyone. When you take your smartest programmers and you pull them out of that team and tell them their priority is to build the core architecture that all other applications will be built on, you’ve just given them a different set of incentives. You just told them that it’s not their job to ship product. Their job may be to create beautiful architecture or develop new techniques for modeling business domains, or researching new technologies, or to instruct developers who are shipping products on how they should be implementing the core. The point is, incentives are no longer aligned.

The second problem is a phenomenon that I’ve labeled Smart Guy Disease. This is another thing I’ve seen repeated in multiple organizations. The basic idea is that the worst mistakes that have the most negative impact on an organization are consistently made by the “smart” programmers not the “dumb” programmers. If you want more detail, I wrote a whole post on it here http://rlacovara.blogspot.com/2014/03/smart-guy-disease.html

Conclusion

The company-wide core framework is an idea that seems like an obvious win to management and architects. But be warned. Here there be monsters. I have never, not once, seen a company-wide architecture solve more problems than it solved. I have seen two examples where company-wide frameworks had hugely negative impacts on their respective organizations. So think twice before starting down that path.

One alternative I would suggest is to maintain a base application code-base. But instead of forcing every application to consume it and inherit from it (thus creating cross-application dependencies), just fork it when you create a new app. That way you have a solid starting point, but you’re free to make changes without worrying about how those changes will affect other apps in your organization.

You might be using TFS

noreply@blogger.com (Rudy Lacovara) — Mon, 12 Jan 2015 02:48:00 +0000

If your dev computer has 500GB of disk space taken up by 237 branches of the same code even though 99% of the files in those branches are identical… you might be using TFS.

If you spend 45 seconds watching a little spinning icon each time you rename a file… you might be using TFS.

If your supervisor requires that you log your time in 15 minute increments and track those increments against work items in your current sprint… you might be using TFS.

If the workflow for tracking a bug fix requires a simple 27 click process that includes setting values in 3 different status fields and manually reassigning the work item and still doesn’t map to your actual process… you might be using TFS.

If the most productive team in your company manages their sprints in Excel instead of the company standard ALM software – and you wish you were too… you might be using TFS.

If creating a new branch requires more than 10 seconds… you might be using TFS.

If you ARE NOT CONFUSED when you hear people say things like “It took me 5 minutes before I was able to switch to disconnected mode” or “I spent a full day trying to fix a corrupted workspace”… you might be using TFS.

If you’ve ever heard the phrase “Yeah it’s kind of inefficient and it has some rough spots and the constantly connected model is a pain, but it’s sooooo much better than Source Safe”… you might be using TFS.

If you’ve ever lost 2 days trying to fix history after a bad merge… you might be using TFS

If your company has ever sent all of the devs home because the source control server was down and nobody could work… you might be using TFS.

I find it tragic that so many .Net teams use what I consider to be the “worst of breed” solution just because it’s made by Microsoft. TFS is a cumbersome, poorly designed system that extracts a productivity tax on every developer, every day that they use it. The constantly connected model alone is reason enough for me to never use this tool. So why do people use it if it’s so bad? I think it comes down to 3 unfortunate facts of life:

1. SCM (Source Control Management) and ALM (Application Lifecycle Management) software is typically selected by managers, not by the developers who will need to use those systems.

2. Microsoft makes TFS. It’s a safe choice. Nobody is going to get fired for picking the solution made by Microsoft.

3. Most of the teams using TFS have either never used another SCM, or were previously using Microsoft Source Safe, the only SCM so horrible that it actually makes TFS look good by comparison.

So the next logical question is “Ok Mr. Complainer, if TFS is so bad what do you use instead?”

I’m glad you asked. In my opinion there is only one choice, git. A git repository hosted on either Bitbucket (from Atlassian) or GitHub, and managed using SourceTree (also from Atlassian) is such a gloriously simple and low friction solution that I’ve never found cause to use anything else. If you find yourself in the position of picking an SCM, do yourself a favor and try the git/SourceTree/(GitHub or Bitbucket) solution first and see if it meets your needs.

ALM/sprint planning/bug tracking is another matter. There are lots of SAAS apps that provide this functionality and I find all of them easier to use than TFS. GitHub itself has some basic ALM functionality that will be good enough for most small teams. There are also 3rd party SAAS apps that plugin to GitHub or Bitbucket to provide additional features. Huboard is one such app that provides a Kanban board for your issues on GitHub. The bottom line is try something simple and see if that works for you before moving on to a more complex solution like Jira or (shudder) TFS.

One last thing. I know some people will read this and say “You’re totally out of touch, you can use git as your source control within TFS, get all the git advantages, and still get all the great TFS vertical integration.” That might be true. I’ve never used TFS with git and I’ve never seen it used in any of the TFS shops that I’ve worked with. That’s worth noting. I’ve heard a lot of people talk about TFS with git, but I’ve never actually seen it implemented in the wild. So, it might be the land of milk and honey. It might fix every problem I’ve ever had with TFS. I doubt it, but it is possible.

To conclude, I don’t have an informed answer to the TFS with git question because what I’m doing works great and I’m not willing to invest another iota of my precious time into the sinkhole of productivity that is TFS. I think the great philosopher Samuel Jackson (Pitt) from Pulp Fiction said it best. “Sewer rat might taste like pumpkin pie, but I wouldn’t know, because I wouldn’t eat the filthy mother-scratcher.”

SafeSubstring Extension Method

noreply@blogger.com (Rudy Lacovara) — Fri, 23 May 2014 18:47:00 +0000

Extension methods are a great way to keep you’re view code tight and readable. I always include a UIExtensionMethods class in my projects that contains extensions for the HtmlHelper and for my View Model classes.

The Problem

I often want to restrict the max number of characters that will display in the UI. If a field has 2000 characters, I just want to show the first 100. Easily done right? Just use a substring like so:

<div>
    @Model.JobSummary.Substring(0, 300)
    <a href="#" class="empub_feed_readmore_link">read more...</a>
</div>

The problem is that there are a number of things that will make Substring throw errors. If JobSummary is null, error. If JobSummary has less than 300 characters, error. I can check for these things in my view code, but that’s going to make my view look messy and I’ll have to copy and paste that code every time I do a substring in a view.

A Quick Solution

Anytime I have view code that needs to be used more than once, or even hurts the readability of my view, I extract that code into an extension method. Since I want a version of Substring that I can run in a view without throwing an error, I call the method SafeSubstring.

// SafeSubstring
public static MvcHtmlString SafeSubstring(this HtmlHelper htmlHelper, string text, int start, int stop)
{
    if (string.IsNullOrEmpty(text) || text.Length < (start + stop))
    {
        return new MvcHtmlString(text);
    }
    return new MvcHtmlString(text.Substring(start, stop));
}

Now my view code looks clean, and I have another tool in my toolbox that can be used next time

<div>
    @Html.SafeSubstring(Model.JobSummary, 0, 300)
    <a href="#" class="empub_feed_readmore_link">read more...</a>
</div>

Smart Guy Disease

noreply@blogger.com (Rudy Lacovara) — Mon, 31 Mar 2014 05:59:00 +0000

As a consultant I’ve had the opportunity to see first-hand how software development works at some of the most cutting-edge .Net shops in the Denver area. One of the things I’ve seen over and over is a phenomenon I call “Smart Guy Disease”.

The basic concept is this. The biggest, most destructive mistakes are always made by the “smart” programmers, not the “dumb” programmers. The smarter the programmer, the greater the capacity for catastrophe.

It seems counter-intuitive, but it’s true. Think about it for a minute. “Dumb” programmers…. actually I have to stop a moment because I don’t really like the term dumb programmers. One of the reasons I like software development is that I consistently get to work with smart people. Even programmers who aren’t up on the latest technologies and are just showing up to do their job every day tend to be very intelligent people. However, I think we all have worked with people who just weren’t up to the task. They consistently ship code rife with bugs, you have to code review every line they commit with a fine tooth comb, they just aren’t very good at programming and they consistently produce bad code. Bad programmers might be a better term, but I’ll keep using “dumb” because it contrasts well with smart.

Anyway, “dumb” programmers do produce crappy code, but they almost never make the kind of massive mistakes that cripple an entire organization’s ability to ship software. They may implement features that perform poorly, but they don’t deploy code that destroys performance on a global scale. Nope, when you want to cause that kind of damage, you need a smart guy.

I’ve seen it. It happens. The smartest guys in an organization are the ones that make the truly big mistakes.

Shipping architecture instead of software

The first mistake I’ve seen is shipping architecture instead of software. Companies take the smartest guys in the organization and tell them that they’re architects now and they are no longer responsible for shipping an actual application. Instead they will create a company-wide enterprise architecture, or Core Framework, that other app development teams, who do ship production applications, will be required to build on top of.

I’ll go into more detail on why this is a bad idea in a later post, but for now just know that in this situation the incentives of the architects and the app developers are not aligned. The architects’ priority is to create architecture, something that implements cutting edge technologies, looks great in diagrams, is impressive when presented to C level executives, and if it’s overly complex… well that just demonstrates how smart they are. The app development team’s priority is to produce code that performs well, is easy to maintain, and ships on time. The architects really have no idea if their designs are easy to implement or perform well until they are implemented by the app team. Not a good way to ship code.

Overly complex solutions

Another common problem, that I alluded to above, is overly complex solutions. You might be tempted to dismiss this one as no big deal. Don’t. This is probably the most common and most damaging mistake that I’ve seen in the real world. Smart programmers usually like to show how smart they are. You can’t really do that if you produce code that is easy to understand and maintain. Why implement a simple solution when you can do something clever instead?

So what’s the cost of extra complexity? If you look only at the first 9 months of an application’s lifecycle then the answer is “not much”. However, if your application is successful and has a lifecycle that extends beyond the first year then that extra complexity becomes a tax that you pay over and over, every time someone touches that code base. That’s the real problem. The price for additional complexity isn’t just paid by that initial developer, it’s paid again and again by every developer who touches that code for the entire lifetime of the application. It’s far too common for smart developers to ship a new product with needlessly complex architectures, then declare the project a success and move on, leaving a maintenance nightmare for the “average” developers who come after them.

Complexity also leads to brittle code. We all understand intuitively that the more complex something is, the more likely something will go wrong and it will break. That’s bad enough by itself. Now think a year down the road when that complex system is being modified by a developer who doesn’t necessarily understand the intent of the original “smart” programmer. Now you’ve got a high probability that something will get broken.

The crazy thing is that teams sometimes look at the complexity of their code as a badge of honor. If you ever catch yourself saying something like “We are so cutting edge that we can only hire the best developers, and even then it takes them a good 4 months before they’re productive with our architecture”…. maybe you should ask yourself if that’s really a good thing.

Smart on paper, disaster in reality

One last mistake, solutions that look great on paper but are a complete disaster when implemented in the real world. The best example I’ve seen of this was when a company decided that they wanted to rewrite their ASP.Net WebForms applications so that they could be managed with a 3rd party CMS. The idea was that the WebForms pages were mostly made up of .ascx controls anyway, so it should be possible to make every .ascx responsible for loading and saving it’s own data, then publish that data to and coordinate with the other controls on the page at runtime. These individual controls could then be composited together in any combination by designers or business people using the CMS. There was some kind of limitation with the CMS that required all of the heavy lifting to be done in the browser with JavaScript and Ajax. Did I mention this was WebForms code with update panels and all that nonsense. Remember how fun it was to do real JavaScript and manage DOM Ids browser-side with update panels?

To be fair, it’s been a while and I don’t really remember the details. I just remember that the big brains decided this made sense, and I’m sure it did make sense in a high-level planning meeting. But, every hands-on-keyboard developer who worked with those apps immediately recognized that this was a terrible idea. Every developer who actually shipped code could see that even if a miracle happened and this idea worked, it would make the code much, much harder to maintain and it would slow development to a crawl. It did by the way.

How to avoid smart guy disease

So what’s the point? Is this just a rant because I hate smart people? No of course not. I like smart people and I like working with smart people. The point is that it’s possible, even common, for the smartest developers, who should be our greatest asset, to instead create massive problems. But it doesn’t have to be that way.

I have 2 simple rules that can help prevent smart guy disease from crippling your organization.

Rule 1. Make sure your smartest developers are always in a role where they are shipping production code.

Most of the problems mentioned above happened when the smartest developers became disconnected from the reality of building and maintaining production software. That disconnect messes up the incentives and puts your smartest programmers at odds with the programmers who actually ship code.

Instead, keep your best and brightest developers embedded in teams that are shipping production applications. Let them be a team leader or a team member. Give each application team their own architect. Or, if they can’t be a permanent team member, at least let them be a temporary resource embedded inside the team. They can work hands-on-keyboard with the other developers to actually implement their architectural designs. That kind of hands on experience will help everyone involved and will keep those incentives aligned.

Rule 2. Value simplicity.

Make simplicity a cultural value, a goal that every developer strives to attain. Make it clear that simple solutions are valued and needlessly complex solutions won’t be tolerated. Keep a close eye on any developers who want to use multiple layers of base classes, Unity Interceptors, or other techniques that are used to make things happen automagically.

Never, ever boast that your architecture is so complex that only the very best developers can work on it. Instead your boast should be that your architecture is so well designed that any junior developer can come in off the street, understand the basics in a day or two, and be productive within the week.

NHibernate WCF Error: HTTP request context being aborted by the server??

noreply@blogger.com (Rudy Lacovara) — Wed, 19 Dec 2012 15:34:00 +0000

I’ve been ambivalent toward ORMs for quite some time now, but I’m really starting to dislike NHibernate in a special way. My experience has been that ORMs lead to a lot of wasted time troubleshooting code problems that are really ORM problems. Here’s one example.

I recently got this mysterious and unhelpful error message on an ASP.Net MVC3 web app that was pulling data via a WCF web service, which in turn was using NHibernate for persistence.

An error occurred while receiving the HTTP response to http://localhost:8080/JobsService/ws. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.

Ok, my service endpoint was definitely using HTTP and none of that other stuff was happening either. So what was the problem? NHibernate lazy load proxies.

Let’s say I have a Company entity that contains a child collection of type List<Job> as shown in the diagram below. So if we look at the Company entity, all of the fields (Name, State, City) exist in my data table, except for the Jobs. Jobs represents a relation between my Company and Job tables and to get Job data NHibernate has to query the Job table.

At this point it’s important to note that I’m using NHibernate, all of my entity properties are virtual, and I have lazy loading enabled. So what happens when I run a query to get a Company? What data does NHibernate populate? It’s going to populate all of the data that’s in the Company table (Name, State, City) , but it’s not going to populate the Jobs collection. It’s going replace that virtual property with a proxy that will execute a query to get Jobs only when that property is accessed.

I actually think this lazy loading proxy technique is pretty neat and it’s technically impressive. Unfortunately it’s also the root cause of a lot of application errors.

So the lazy loading proxy works great when we’re accessing an entity directly in our web app while it’s still has the context of the NHibernate session, but what happens when we try to return our Company over WCF and WCF tries to serialize that Jobs property that hasn’t been lazy loaded yet? That’s right, you get the ambiguous error message above.

The fix is simple. You just need to eager load the Jobs. You can do this any number of ways. Here’s how to do it in a criteria query.

return session.CreateCriteria<Company>()
.Add(Restrictions.Eq("Id", _Id))
.SetFetchMode("Jobs", FetchMode.Eager);

Now that we’re eager loading the Jobs there’s real data there instead of the lazy load proxy, WCF has no problem serializing the Jobs, and our mysterious ambiguous error goes away.

So, I hope this saves someone a little time, and please remember, friends don’t let friends use ORMs.

The problem with ORMs

noreply@blogger.com (Rudy Lacovara) — Wed, 18 Jul 2012 17:35:00 +0000

Over the last 8 years I’ve done a lot of thinking and experimenting with different persistence models. If you’d asked me 6 years ago what a persistence layer should look like I would have whole heartedly suggested an ORM like nHibernate. Linq to SQL came along and initially looked promising but I quickly soured on it in favor of Entity Framework. Then I actually tried to use Entity Framework in a production app and quickly found that in spite of the initial query writing efficiencies, it caused problems in other areas and the net result was that it didn’t save me any time at all but did add limitations and complexity to my app.

So what was the problem? As I spoke with more developers I started to hear the same complaints, even though some were using EF, some were using nHibernate, others were using LinqToSql. I heard the problem stated best by a guy I met at the speaker’s dinner for the Rocky Mountain Tech Trifecta. I can’t remember the guy’s name but his words stuck with me. I was telling him that I would never use EF again if the choice was up to me. He said it’s not that he has a problem with Entity Framework, it’s that he’s done with ORMs. I think that’s really the heart of it. There’s a fundamental problem with ORM as a pattern. It just doesn’t adequately solve the problem and it ads significant complexity and inefficiency to an application.

I recently ran across a couple of old blog posts that make for interesting reading, Is OR/M an anti pattern? by Ayende, and ORM is an anti-pattern by Laurie. I’m hearing this line of thought more often as more developers use EF and other ORMs in production apps and then have a couple of years of living with the consequences.

So it’s easy to knock something, but what’s the alternative? For me it’s been moving to a simpler data mapper instead of a full on ORM. I first wrote my own simple data mapper, but since then I’ve discovered Dapper.net by Sam Saffron and Marc Gravel. On the surface Dapper works almost identically to the mapper I wrote, but under the covers it’s way better and much more efficient. So I’ve been porting all of my old code over to use Dapper.

The results so far have been fantastic. Sometimes I have to write more code that I would have to write with a full-on ORM, but I find that I can make major application changes (relatively) quickly and easily and I have never once spent hours struggling with my persistence architecture trying to understand why I’m erroring out on an update for an object graph, or troubleshooting lazy load issues, or select N+1 issues, or thinking to myself “If only this was SQL I would know exactly how to do this”. Instead it just works, exactly the way I expect it to, and in the end that saves a lot of time and effort.

Great Backbone.js Tutorial

noreply@blogger.com (Rudy Lacovara) — Mon, 04 Jun 2012 18:10:00 +0000

I definitely think that JavaScript is the future (at least the near term future) of app development, and I recently converted HireFlo to a single page JavaScript app. You know the type, where you only have 1 real HTML page and it defines the main content areas of your site. Then the rest of the app is all JavaScript that is used to load content into those areas mostly through AJAX and client side templates (check out jsrender).

After doing the HireFlo rework, and seeing the rats nest of JavaScript that I created, I can definitely see the value of using a JavaScript MVC or MVVM framework to manage my JavaScript UI code. One framework that I’ve been looking into is Backbone.js. It’s a full-on MVC framework for JavaScript. How cool is that? The more I learn the more I like, and I think I might be refactoring HireFlo in the near future to use Backbone.

If you want to take a look at Backbone, check out the Backbone.js Wine Cellar Tutorial by Christophe Coenraets. It’s the best intro I’ve seen so far.

Awesome Startup Architecture List

noreply@blogger.com (Rudy Lacovara) — Sun, 13 May 2012 07:07:00 +0000

I wanted to put together my short list of awesome stuff you must have if you’re building a new web application on .Net. Each of these libraries is free, open source, and takes the simple, no-nonsense approach to getting things done that is essential in a startup.

Before I get into the list just let me say that jQuery isn’t on the list because I just assume that you’re already using it, and because it’s already included by default when you create a new .Net web app.

Dapper
You really can’t make the claim that there is any one right way to do data access... but this is the one right way to do data access. It was written by two of the guys at StackOverflow and it gives you reflection like functionality without reflection. They actually emit IL. It’s cool.
http://code.google.com/p/dapper-dot-net/

Automapper
When you find yourself writing that mapping code between your DTOs (or entities) and your ViewModels, save yourself some time and just use this incredibly useful convention based mapping library by Jimmy Bogard
https://github.com/AutoMapper/AutoMapper

NLog
It’s a good idea to bake logging into your app from the beginning. I have no idea why people still use Log4Net (confusing) when NLog (simple and awesome) is out there.
http://nlog-project.org/

Quartz.Net
Surprisingly powerful scheduler. Ideal for writing processors, console apps, and windows services that do things on a schedule.
http://quartznet.sourceforge.net/

Twitter Bootstrap
CSS and Javascript that gives you a whole UI framework and a bunch of themed widgets to build a modern webapp on top of. Before you write any UI code, spend a day experimenting with Bootstrap. It’s amazingly useful and it gives your app a super polished look.
http://twitter.github.com/bootstrap/

JsRender
Javascript templating done right. Indispensable if you’re writing a single page javascript app using Bootstrap, BackboneJs, or KnockoutJs.
https://github.com/BorisMoore/jsrender

dotLess
A .Net port of the Less CSS library. In the style of FakeGrimlock: IT MAKE CSS MORE AWESOME!!
http://www.dotlesscss.org/

jqModal
Simple jQuery based dialog boxes and modal popups.
http://dev.iceburg.net/jquery/jqModal/

The 5 Step Recipe for Building a Startup: Developer Edition

noreply@blogger.com (Rudy Lacovara) — Wed, 04 Apr 2012 15:20:00 +0000

Building a startup is hard. If you spend any time around newtech meetups or startup groups you’re constantly running into people who all seem to have the same problem, they’re a business person with this great idea, if only they could find a developer to build it. Seems like if you’re a developer and have a startup idea you’ve got it made, right? Wrong.

Developers have the skills to make their own vision a reality. That’s a huge advantage, but it’s also a problem. Because developers are really good at building things with code, the first thing they usually do is…write code. That’s the absolute wrong place to start, and that’s why most developers wind up producing something that nobody wants, with no users, and no idea where to go from there.

The key to not being “that developer” is to start with the business side. Don’t start writing code right away. Instead start thinking through your app, start talking to potential customers, start doing things like collecting emails that measure the interest in your idea. Now I’ve done this wrong a couple of times, but I like to think that I can be taught, and this post lists the 5 simple steps that I’ve learned for making a successful startup.

1. Mock

Buy a copy of Balsamiq mockups and mock your app. Not just the landing page, all of it. Every screen. This will force you to focus on the user experience side of your app (which is the only part the user cares about) and work through a lot of the logical problems, application flow problems, stuff like that. It will also give you something you can show to other people when describing your idea, which leads us to....

2. Talk

Take your mocks and show them to as many people as you can who are potential users. That part's important. You want to talk to people who are in your target market. Talk through the idea, talk through the app. Listen. Listen most to the people who love your idea and the people who hate your idea. Those are the people who care. Figure out what the key values that your product provides are from your customer's perspective. Also, as you get feedback, you should be updating the mockups incorporating the things that you'

This is the step that most developers will want to skip. Don’t. Talking to potential customers is the most important thing you can do as a Founder. It’s uncomfortable, it’s tough, you’ll find yourself fumbling for words and talking in circles trying to figure out how to describe your product. You’ll feel like an idiot for the first 10 or 20 conversations, but then something will start to happen. Just by sheer force of repetition you’ll find that you’re starting to figure out how to describe your product clearly, and you’ll find that you’re saying a lot fewer words while doing it. You’ll also develop a picture of what the benefits your product brings that seem to interest people the most. This is what you’re after. This whole process is necessary to teach you what your product really is and what your key value propositions are. It’s not easy but it’s absolutely necessary. If you’re not willing to do it then go work for somebody else, you’re not an entrepreneur.

3. Announce

Now that you have mocks, and you know what your key value props are (because you talked about it with many potential customers) it's time to put up the "coming soon" page. I would recommend doing a single page that describes the key value props and includes a screenshot. The screenshot isn't a real screenshot of your app. You don’t have an app yet. You can hire a designer to dummy it up from one of your mocks. I would recommend spending a few hundred dollars on 99designs. You can get every thing done at once, a good looking design for your "coming soon" page, a dummy screenshot, and a nice text treatment (no icon needed at this point) for the logo. Get this page up and start collecting emails of potential users.

If you're really ambitious and have something to say, you may also want to start a blog at this point. But don't blog about building a startup, blog about the problem space that your product is for. Blog about things that show your expertise in that area, and write things that your target customer will want to read because it's relevant to the problem that you solve. If you're building small business accounting software, then blog about small business accounting issues. Remember, the purpose of the blog is to attract and reach potential users, not other startup founders or the hackernews crowd.

4. Bizspark

If you’re building on the .Net stack you absolutely need to join Bizspark. Microsoft will give you free licenses to basically every product they make, including SQL Server, and a free MSDN gold subscription, for 3 years. They figure 3 years is long enough for you to get going so after that they want you to pay for new licenses, but here's the great part, they let you keep the licenses you're already using. So you don't wind up in a situation where you're just scraping by, then 3 years is up and you have a big Microsoft bill to pay. It’s great. Microsoft has basically taken the cost factor completely out of the equation for new startups, and it ‘s a perfect program for a .Net developer who’s building their first startup.

It may seem odd that I didn’t start with Bizspark as step 1. There’s a reason. You’re not ready to join Bizspark until you reach this point. Keep in mind that Microsoft doesn’t just accept everyone who applies to Bizspark. You actually are required to have a company, a website, and a sponsor. Your sponsor can be a company that has some kind of partner relationship with Microsoft, like Rackspace, or it can be a person employed by Microsoft, like Aaron Stannard who is the startup evangelist for Microsoft (aaronst@microsoft.com). So you’ll need be ready to describe your app and you’ll need to have a website up, not a completed product, just some type of website for your company. The “coming soon” site is fine for this requirement.

5. Build

Now that all that other stuff is done, now you start building the first version of your app. Note that you should continue to talk to potential users, you may even want to establish a small group of users who you show your app to as it's being built. The key point is that step 1 is not finding a developer, step 1 is not start building. Building the app doesn't happen until step 4.

While building, do not follow the common advice "don't worry be crappy". Nothing you produce should ever be crap. Instead I would recommend taking the 37Signals approach "Build half a product, not a half-assed product". In other words, your MVP should have half the features you want it to have, but what you do build should be awesome.

To give some technical specifics, I highly recommend building on MVC, not WebForms, and I recommend using a virtual server like an OrcsWeb cloud server or a Rackspace cloud server, not Azure. The reason is that for an early stage startup speed and flexibility are paramount. You’re going to be making major changes to your app on a regular basis for the first few years. MVC is a framework that embraces change and makes those changes faster and easier than Webforms.

The same principle applies in reverse for Azure. Azure solves the problem of scale, but scale isn’t your problem. My experience, and the experience of a couple of other startups I have first hand experience with, is that Azure slows down development. I think it makes your development slower and less flexible. Debugging is a pain, setting up your environment is a pain, deployment is a pain, the DAL becomes something you actually have to spend time and effort on, plus… what if you decide you need to run Redis or MongoDb or some other open source thing? Microsoft will probably make it possible so they can check off that box on the feature list and claim that it works, but odds are it will be a whole lot harder than just installing that software on a regular server. What if you end up needing to have 10 different services running? Is each of those going to be a separate Azure instance, and thus an additional bill? You don’t want to spend time worrying about stuff like that in the early days when your app can change massively every 2 weeks.

At the beginning, I think you’re much better off just buying a single cloud server. It runs just like a real windows server, it provides a familiar environment, it’s going to be a solid single platform that you can load up with whatever open source code and services you need (without paying extra), and you can scale it up if you you need to. Later, when you know what all the pieces of your app are and you have customers pushing the limits of your cloud server, that is the right time to look at Azure.

Should you use .Net for your Startup Company?

noreply@blogger.com (Rudy Lacovara) — Sat, 24 Mar 2012 21:14:00 +0000

I get this question a lot, from both people inside and outside of the .Net community, and it came up again the other day on Quora. Below is my answer. Please keep in mind that this is my opinion, and I can’t promise that I’m right, but I can promise that this is what I really think and if you walked up on the street and asked me “should I use .Net for my startup?” this is what I would tell you.

In spite of the fact that I’m a pretty high end .Net guy, I would caution new startups that are considering using .Net. I think .Net is awesome, and I use it for my own startup HireFlo. However there's one big landmine that could handicap your startup before it even gets going, webforms.

You see, ASP.Net development is divided into 2 camps, and there are major architectural differences, and cultural differences between them.

ASP.Net MVC is awesome for startups

ASP.Net MVC is the awesome new framework that is heavily influenced by Rails and is the platform of choice for startups like StackExchange. It’s a breath of fresh air for skilled .Net developers who want a framework that embraces the way the web works instead of struggling against it. This camp is full of people who care about the craftsmanship of writing code and who like to actually ship software. A programmer in this camp is likely to be familiar with lots of open source projects, and is most likely a great fit for your startup.

WebForms is death for startups

The other camp is WebForms, a festering bog of evil, spaghetti code, and hate. It's the domain of corporate developers who prize process and documentation over shipping code and hope that the glacial pace of WebForms development will hide the fact that they haven't shipped a product in years and can't tell the difference between an HTTP GET and an HTTP POST. I mean it. WebForms and it’s poisonous PageLifeCyle pattern are a sink hole of productivity and they are going to resist you every step of the way as your startup struggles to find the right fit between your application and your market (product market fit). Plus Webforms and the cursed UpdatePanel make it incredibly difficult to do the fancy new style of client side javascript app that tools like Backbone.js, Knockout.js, and jQuery have made possible.

So the answer is…Heck yes you should (as long as it’s .Net MVC)

So .Net is awesome, C# is amazing, MVC3 is hugely productive, if you find a .Net developer who works with those, I'd recommend using them. But you've got to be careful. A WebForms developer with a corporate dev background is most likely going to be death for your startup.

There are three other points I want to add that I think are relevant.

Don’t use Azure for an early stage startup

Microsoft is really pushing Azure as a platform for startups. I strongly recommend that early stage startups do not use Azure. Azure development is significantly slower than straight up .Net development that runs on a windows server or VS. Deployment takes longer, debugging takes longer, the Azure environment is difficult to replicate on your local dev computer, backups are a pain, and you're locked into a single hosting provider. It all ads up to a lot of inflexibility and friction at a time when your startup needs flexibility and speed over all other things. Azure can have a place later, when scale is your problem, but at the beginning of your startup that isn't the issue.

The ASP.Net MVC/C#/SqlServer stack scales like a madman

Listen to the middle days of the first StackOverflow podcast and you'll hear that for a long time they ran StackOverflow on a single server. They were serving a million uniques with the web app and database running on a single box! It wasn’t even a very big box. This is consistent with my experience with .Net. It scales well. So, if your startup does make it, you'll probably have a much easier time scaling the .Net stack than you would with say Ruby or PHP.

Bizspark is proof that Microsoft loves programmers and startups

If you want to build a startup on the Microsoft stack, they will give you free licenses to basically every product they make, including SQL Server, and a free MSDN gold subscription, for 3 years. They figure 3 years is long enough for you to get going so after that they want you to pay for new licenses, but here's the great part, they let you keep the licenses you're already using. So you don't wind up in a situation where you're just scraping by, then 3 years is up and you have a big Microsoft bill to pay. They don't do that, they just let you keep using the software. So Microsoft has basically taken the cost factor completely out of the equation for new startups.

HireFlo, an actual startup built on .Net

noreply@blogger.com (Rudy Lacovara) — Sat, 03 Mar 2012 18:28:00 +0000

What have I been doing for 10 months?

I’ve gotten a number of emails asking for part 2 of the ValidationLabel post that I wrote 10 months ago, and a number of people have said there’s problems with the sample code on aapl.codeplex.com/ and asked for updated code samples. Sorry about that. There is a reason I’ve dropped off the face of the earth for the last year.

That reason is HireFlo. I took all of the principles I’ve been discussing here, the techniques for creating an architecture that embraces change and can quickly adapt to massive changes to the application, and I put them into practice with my own startup. HireFlo is an Applicant Tracking System designed for the small business market. It takes all the tools that recruiters use like jobsites, aggregators, social networks, resume databases, and wraps them up in a simple app designed for small business users who probably don’t even know that most of those tools exist.

I’ve been building HireFlo for about 18 months. It’s an ASP.NET MVC app that uses AAPL for persistence, and I have to say the MVC and AAPL combination has been a huge success. I’m on version 4 of the app and I’ve made some massive changes since version 1. That’s what happens in startups, and why an architecture that embraces change but can still scale, is so incredibly important. Each time I’ve made major changes, MVC and AAPL have made the experience as quick and painless as possible.

The last major change was November 2011 when I decided to completely throw away the old UI, and rewrite a simpler, streamlined version based on a Kanban board, and also make it a single page Javascript app in the style of backbone.js and knockout.js apps. I completed the complete rewrite of every page, controller, HTML template, CSS file, and Javascript file in the app in just 14 days. This would never, never, NEVER have been possible if I’d used WebForms or a more Microsofty approach to persistence that used a bunch of sprocs and hard coded mapping logic in my data and business layers.

So anyway HireFlo is up. It’s been officially launched since August 2011 but I would say the real launch was end of December 2011 when I released version 4 of the app and the redesigned marketing site. So far there about 300 companies using it. It’s a start.

What’s coming next

So the first thing I need to do is write Part 2 of my last post. This part details how to take a helper method and turn it into a strongly typed helper method by using expressions and lambdas because…. well because doing that is awesome. After that I’ve got a lot of tech problems that I’m solving for HireFlo and I’d like to share them. I just need to find some time and get back on a somewhat regular schedule. BTW, I’d also be happy to give updates on HireFlo if you’re interested. Just let me know. -rudy

Creating a Validation Label Helper – Part 1

noreply@blogger.com (Rudy Lacovara) — Sun, 17 Apr 2011 15:05:00 +0000

Presenting validation messages is a pain. Programmers usually tack validation messages on to a User Interface is a way that gets the job done, but also is clunky and detracts from the overall UI design. Let me show you an example. Below is a really simple form from my HireFlo application.

Here’s the markup that creates the form fields and buttons. I just use a simple layout with a label followed by a validation message, and a Textbox created using MVC’s strongly typed Html helper methods.

    <div class="form-row">
        <label class="form-label" for="folder.FolderName" title="The name for this folder" >
           Enter a name for your folder:
        </label>
        <%= Html.ValidationMessageFor(model => model.Folder.FolderName)%><br /> 
       <%= Html.TextBoxFor(model=>model.Folder.FolderName, new {style="width:300px;", maxlength="50"})%>
    </div>
                             
    <div>
        <input id="SaveButton" type="submit" class="button-green" value=" Save Folder " />  
        <input id="CancelButton" type="button" class="button-gray" value=" Cancel " />           
    </div>

The only validation on the form is that you have to enter a name for the folder you’re creating. If you don’t, you get a validation error that looks like this.

While functional, this doesn’t look very good. The validation message for the field is just kind of tacked on after the label, but where else am I going to put it. I could put it underneath the field, beside the field, but these options all result in extra text, and an interface that looks cluttered to me.

I’ve become more and more of a minimalist when it comes to UI design. I want to put as few things on the page as possible. So I started thinking, what is the function of the label on a form? It’s there to tell the user what data to put into a given field. Now what is the purpose of the label on a form with a validation message? The validation message and the label are both trying to tell you what goes into a given field. Maybe it makes sense to combine them. So I tried that out and the results look like this.

All right. This is looking better to me. The interface is clean and clear. I have to make sure that my validation messages always start with the field name, and they need to be fairly short, but those both sound like good design constraints so I’m willing to work with them. The ValidationLabel is now the standard way that I’m going to handle error messages in my application.

So how should I implement the ValidationLabel? I’m already using the ValidationMessage Html helper method and that works pretty well, I’ll just customize that and add the Label functionality to it. I write an extension method for HtmlHelper that will take all of the parameters required to behave as both a label and an error message. If there’s no error, my helper will render a label. If there is an error, my helper will render an error message. Simple. Here’s how my markup will look with the new helper.

    <div class="form-row">
        <%= Html.ValidationLabel("Folder.FolderName", "Enter a name for your folder", null)%>
        <%= Html.TextBoxFor(model=>model.Folder.FolderName, new {style="width:300px;", maxlength="50"})%>
    </div>
                            
    <div>
        <input id="SaveButton" type="submit" class="button-green" value=" Save Folder " />  
        <input id="CancelButton" type="button" class="button-gray" value=" Cancel " />           
    </div>

Hmmmm, cleaner interface and cleaner markup. I like the way this is going. Now let’s write the ValidationLabel helper method. It’s actually surprisingly simple. Here’s the code.

        // ValidationLabel
        public static MvcHtmlString ValidationLabel(this HtmlHelper htmlHelper, 
                                                    string modelName, 
                                                    string labelText, 
                                                    IDictionary<string, object> htmlAttributes)
        {
            if (modelName == null) { throw new ArgumentNullException("modelName"); }
            ModelState modelState = htmlHelper.ViewData.ModelState[modelName];
            ModelErrorCollection modelErrors = (modelState == null) ? null : modelState.Errors;
            ModelError modelError = ((modelErrors == null) || (modelErrors.Count == 0)) ? null : modelErrors[0];
            // If there is no error, we want to show a label.  If there is an error,
            // we want to show the error message.
            string tagText = labelText;
            string tagClass = "form_field_label_normal";
            if ((modelState != null) && (modelError != null))
            {
                tagText = modelError.ErrorMessage;
                tagClass = "form_field_label_error";
            }
            // Build out the tag
            TagBuilder builder = new TagBuilder("span");
            builder.MergeAttributes(htmlAttributes);
            builder.MergeAttribute("class", tagClass);
            builder.MergeAttribute("validationlabelfor", modelName );
            builder.SetInnerText(tagText);
            return MvcHtmlString.Create(builder.ToString(TagRenderMode.Normal));
        }

Here’s how it works. This is an extension method on the HtmlHelper class so the first parameter is the HtmlHelper. Next we pass in the modelName, the labelText, and a dictionary of Html Attributes that can be used for custom formatting. The first thing we do is make sure that we have a modelName and then check to see if there’s an item in the ModelErrorsCollection for that modelName.

Note that modelName is not the name of your entire model, it’s the name of your field. In my folder form example the model name is “Folder.FolderName”, because I have view model class that contains a Folder object and that Folder object has a property called FolderName. If you’re ever uncertain about what a field’s modelName is, just load up your page and look at the markup for your input tag. It will look something like this, and your model name is the value that ASP.Net MVC puts in the name attribute for your tag.

<input id="Folder_FolderName" maxlength="50" name="Folder.FolderName" style="width:300px;" type="text" value="">

Let’s get back to our method. Now we know we have a modelName, we know what the modelName is, and we’ve checked the ModelState’s ModelErrorsCollection to see if we have any errors for our modelName. From this point on the logic is pretty simple. If we do have an error we set the tagText to the error message text and we set the tagClass to “form_field_label_error”. If we don’t have an error we set the tagText to the labelText and we set tagClass to “form_field_label_normal”. Then we just use a TagBuilder to render out a span using the tagText and tagClass that we defined. I also added a custom “validationlabelfor” attribute that holds the model name and is used for client side (javascript) validation.

So that it! We now have a method that renders our field label, and will magically present any error messages for that field if they exist in the ModelState (In case you’re wondering, ASP.Net MVC automatically stores all errors resulting from model validation or model binding in the ModelState). Plus the syntax is really clean and simple. A ValidationLabel, input pair is just 2 lines.

<%= Html.ValidationLabel("Folder.FolderName", "Enter a name for your folder", null)%>
<%= Html.TextBoxFor(model=>model.Folder.FolderName, new {style="width:300px;", maxlength="50"})%>

There is one more thing I want to do though. Notice how my ValidationLabel requires me to manually enter the modelName “Folder.FolderName”, but the TextBoxFor method uses a lambda instead? That’s pretty cool, plus it eliminates the need to use a “magic string” in our UI code. For Part 2 I’m going to cover how we can create our very own strongly-typed ValidationLabelFor method that will take a lambda as a parameter instead of that modelName string.

.Net as an Awesome Startup Platform

noreply@blogger.com (Rudy Lacovara) — Fri, 04 Mar 2011 20:20:00 +0000

This is the short (10 slide) deck from my presentation at the Denver Tech Trifecta. I used slideshare to share the presentation. If you click the menu button there is a View Fullscreen option.

First, this is all my own personal opinion, not the views of Microsoft or anybody associated with the Tech Trifecta. If it seems a little terse that’s because most of the content was spoken. I favor short slides and lots of talking with code demos. The main theme of the presentation was that classic ASP was an awesome startup platform. Then Microsoft pushed out ASP.Net WebForms, and they took over corporate software development, but they lost the startup community. I think the reason is that it takes too long and costs too much money to build anything real with WebForms (it’s also painful to make significant changes to an existing app). Open source tools like Ruby/Rails and PHP on the other hand provide a much faster and lower friction development experience, plus they’re free.

But now that’s all changed. Microsoft is supporting startups in a big way with the BizSpark program which will basically give your startup every product that Microsoft makes for FREE for 3 years. The biggest and most important change for me personally is ASP.Net MVC. Three years ago I was ready to throw away a decade of work on Microsoft platforms and go do Ruby/Rails. WebForms is a never ending source of bad code and frustration for me. Then ASP.Net MVC came along and it was like a love letter written to web developers. Every place I turned the MVC Framework had given me just the right levels of abstraction and access to support me in what I needed to code. Plus there’s no ridiculous artificial page lifecycle that tries to make the web look like Windows, awesome! I love MVC and because of it I am excited to keep building on the .Net stack. Unfortunately, now that I’ve experienced the light, I cry every time I’m forced to go back and work in WebForms. One place that I do think the page event lifecycle model works well is Silverlight. Silverlight gives you the ability to write apps with a user experience that is so good that I think building with it will become a competitive advantage for line of business apps. All these changes add up to Microsoft once again being an awesome platform to build your web startup on.

Net as an awesome startup platform

View more presentations from rlacovara.

Startup Links

For those who are just looking for the resource links at the end of the presentation, here they are. These are all great startup resources and well worth checking out, especially if you live in the Boulder/Denver area. -rudy

newtech Meetup
About 4 tech startups present in Denver and Boulder every month. The Denver meetup meets on Auraria Campus. The Boulder meetup has free beer!
http ://www.bdnewtech.com /

TechStars
One of the first Y Combinator style accelerator programs for early stage startups.
http ://www.techstars.org /

Open Angel Forum
Angel Investors meet in Boulder about twice a year to hear pitches from early stage startups and possibly fund them.
http ://openangelforum.com /

StartupDigest
Weekly email digest by Chris McCann that tracks all of the tech startup related events in the Boulder / Denver area.
http ://startupdigest.com /

Lean Startup
Best practices for launching a startup by Eric Reis. Must see video here:
http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2329

Feld Thoughts
Amazing blog by Brad Feld, Boulder resident and vocal venture capitalist.
http ://www.feld.com/wp/

Easy XML Feed using LINQ to XML and ASP.Net MVC2

noreply@blogger.com (Rudy Lacovara) — Mon, 21 Feb 2011 16:06:00 +0000

I recently had to create an XML feed for an app I’m working on. The use case is simple. I have a database of jobs and I need to provide an XML feed containing any jobs that were posted in the last week. I’m going to need to do this for several different 3rd party companies that each use a different XML format, so I want to do a little object oriented design to make sure that creating a feed using a new XML format is as easy as possible.

I’ve always heard that LINQ to XML makes working with XML much easier, and I know that MVC2 makes it easy to package up and return any type of data so I’m going to use them. It should make for a pretty painless XML feed. Let’s see.

Starting at the controller

I want to start by writing the consuming code and let that drive the shape of my business layer code. That means I start by writing a Feed action method on one of the controller classes in my MVC2 app. I want to return XML data so i pick a return type of ContentResult, which will allow me to set the content type to “text/xml”. The logic for this method is simple. I have a feedKey that distinctly identifies the 3rd party company / XML schema that I want to generate a feed for. All companies will hit the same action method, but they’ll pass in a different feedKey, which will result in them getting the appropriate XML feed. My action method takes the feedKey as a parameter and then passes it to a factory method on my FeedService class that returns an instance of the appropriate FeedBuilder object. The FeedBuilder is the class that I’ll use to encapsulate logic for getting job data and formatting it as an XML document. I’ll have a different FeedBuilder for each feedKey. The design is a variation on the Strategy pattern. So, back to controller action method. Here’s what it looks like.

        // Feed
        [AcceptVerbs(HttpVerbs.Get)]
        public ContentResult Feed(string feedKey)
        {
            var feedBuilder = this.FeedService.GetFeedBuilderFor(feedKey);
            return this.Content(feedBuilder.GetFeedXml(), "text/xml");
        }

The Factory Method

The factory method on my FeedService is pretty straight forward. It just returns an instance of the right FeedBuilder for the feedKey that we pass in to it. Note that the return type is FeedBuilderBase which is an abstract class that defines the shape of a FeedBuilder.

        // GetFeedBuilderFor
        public virtual FeedBuilderBase GetFeedBuilderFor(string feedKey)
        {
            switch(feedKey)
            {
                case FeedKey.SimplyHired:
                    return new SimplyHiredFeedBuilder();
                case FeedKey.Indeed:
                    return new IndeedFeedBuilder();
                default:
                    return new EmptyFeedBuilder();
            }
        }

The FeedBuilderBase

Remember we’re using a Strategy pattern. Usually you would define the shape of your strategy class with an interface, but I’m going to use an abstract class because there’s some boilerplate code that will be the same for all of my concrete FeedBuilder classes and I want to implement that code in a single base class. My abstract class is FeedBuilderBase and it looks like this.

    public abstract class FeedBuilderBase
    {
        //**************************************************************************************
        // PROPERTIES
        //**************************************************************************************
 
        // FeedService
        private FeedService _feedService;
        public virtual FeedService FeedService
        {
            get { return NewIfNull(_feedService); }
            set { _feedService = value; }
        }
 

        //**************************************************************************************
        // ABSTRACT METHODS
        //**************************************************************************************
 
        // GetFeedData
        public abstract List<JobFeedItem> GetFeedData();
  
        // BuildXmlFor
        public abstract XDocument BuildXmlFor(List<JobFeedItem> list);
 

        //**************************************************************************************
        // HELPER METHODS
        //**************************************************************************************
        
        // NewIfNull
        public virtual T NewIfNull<T>(T obj) where T : new()
        {
            if (obj == null) { obj = new T(); }
            return obj;
        }
  
        // GetFeedXml
        public virtual string GetFeedXml()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriterUtf8(sb);
            xdoc.Save(sw);
            return sb.ToString();
        } 
    }

I have two abstract methods, GetFeedData and BuildXmlFor. These methods represent the two things that change for each XML feed. Each company has slightly different business rules around what data they want to pull and each company has it’s own proprietary XML format. These 2 methods will have to be overridden whenever we create a concrete FeedBuilder.

You’ll also notice that we implemented GetFeedXml which is the method we called in our action method to get the text of our XML document. GetFeedXml should have been very easy to write, but it took a little time due to an unexpected encoding problem that I described in How to Create XML in C# with UTF-8 Encoding.

The Concrete FeedBuilder

Now I’m ready to create my first concrete FeedBuilder class. I’m going to start with a jobsite aggregator called SimplyHired. So I create a new SimplyHiredFeedBuilder class that inherits from FeedBuilderBase an I override my GetFeedData and BuildXmlFor methods. GetFeedData is just a wrapper for the appropriate data access method in my FeedService class. GetFeedData returns a list of JobFeedItem objects. JobFeedItem is a simple Data Transfer Object (DTO) that I created to contain the aggregate of data needed to build a feed item.

BuildXmlFor is more interesting. It takes a list of JobFeedItem objects returned by our GetFeedData method and transforms it into an XDocument using a LINQ query. The LINQ query is just a select with a bunch of nested XElement constructors that create the XML elements needed to represent each job. We then take the expression created to represent the list of jobs as XElements and we wrap it in a single top level XElement called, wait for it…. jobs. BTW, it is at this point that the LINQ expression actually executes. Up to now it has just been an expression. As soon as we use it in the constructor for our jobs XElement the expression compiles and executes.

So, we now have an XML tree contained in a single XElement called jobs. To complete the method we just wrap the jobs XElement in a new XDocument, return the XDocument, and we’re done.

    public class SimplyHiredFeedBuilder:FeedBuilderBase
    {
        //**************************************************************************************
        // FEEDBUILDERBASE OVERRIDES
        //**************************************************************************************
 
        // GetFeedData
        public override List<JobFeedItem> GetFeedData()
        {
            return this.FeedService.GetJobFeedItemsForSimplyHired();
        }
 
 
        // BuildXmlFor
        public override XDocument BuildXmlFor(List<JobFeedItem> list)
        {
            var xmlExpression = 
                from JobFeedItem j in list
                select new XElement("job",
                    new XElement("title", j.JobTitle),
                    new XElement("job-code", j.JobGuid),
                    new XElement("job-board-name", j.CompanyName),
                    new XElement("job-board-url", j.CompanyJobPageUrl),
                    new XElement("detail-url", ""),
                    new XElement("apply-url", GetApplyUrl(j.CompanyKey, j.JobGuid)),
                    new XElement("job-category", j.JobCategory),
                    new XElement("description",
                        new XElement("summary", j.JobDescription),                   
                        new XElement("required-skills", ""), 
                        new XElement("required-education", ""),
                        new XElement("required-experience", ""),
                        new XElement("full-time", j.IsFullTime),
                        new XElement("part-time", j.IsPartTime),
                        new XElement("flex-time", ""),
                        new XElement("internship", j.IsInternship),
                        new XElement("volunteer", ""),
                        new XElement("exempt", ""),
                        new XElement("contract", j.IsContract),
                        new XElement("permanent", j.IsPermanent),
                        new XElement("temporary", j.IsTemp),
                        new XElement("telecommute", "")
                        ),
                    new XElement("compensation",
                        new XElement("salary-range", ""),                   
                        new XElement("salary-amount", ""),
                        new XElement("salary-currency", ""),
                        new XElement("benefits", "")
                        ),
                    new XElement("posted-date", GetPostedOnDate(j)),
                    new XElement("close-date", GetClosedOnDate(j)),
                    new XElement("location",
                        new XElement("address", ""),  
                        new XElement("city", j.JobCity),  
                        new XElement("state", j.JobState),  
                        new XElement("zip", ""),  
                        new XElement("country", "") 
                        ),
                    new XElement("contact",
                        new XElement("name", j.ContactName),
                        new XElement("email", j.ContactEmail),
                        new XElement("hiring-manager-name", ""),
                        new XElement("hiring-manager-email", ""),
                        new XElement("phone", j.ContactPhone),
                        new XElement("fax", "") 
                        ),
                    new XElement("company",
                        new XElement("name", j.CompanyName),
                        new XElement("description", j.CompanyDescription),
                        new XElement("industry", ""),
                        new XElement("url", j.CompanyUrl)
                        )
                    );
            var jobs =  new XElement("jobs", xmlExpression);
            var declaration = new XDeclaration("1.0", "utf-8", null);
            return new XDocument(declaration, jobs);
        }     
    }

Conclusion

So I built it, tested it, and it works. Except for the hour that I spend trying to figure out my UTF-8 encoding problem, the LINQ / XElement method was a relatively painless way to create XML. I love the simplicity of the MVC action method and the fact that it gives me a ContentResult type that gives me access the right parts of the stack. The LINQ / XElement query was concise and the style of passing element data in XElement constructors makes it hard to mess up the document structure. My conclusion is that I like it and I’ll definitely be using this technique again.

How To Create XML in C# with UTF-8 Encoding

noreply@blogger.com (Rudy Lacovara) — Mon, 14 Feb 2011 06:26:00 +0000

I can’t believe that I spent over an hour on this problem. It’s one of those things that you can’t understand how it’s possible that it doesn’t just work. But it doesn’t, so here’s the solution I found for anyone who might have the same issue.

The Problem: My XML refuses to use UTF-8 encoding

I have some code where I’m doing a query and creating an XML doc from the data that’s returned. I used LINQ to XML to create my XML and package it up in an XDocument. The XDocument allows me to set the XML version and the encoding. Here’s the code I used to create and return my XDocument.

            var declaration = new XDeclaration("1.0", "utf-8", "yes");
            return new XDocument(declaration, jobs);

So far so good. Now the last (and what should be trivial) step is to write that XML to a string and return it. This is where things go wrong. To get an XDocument object to write out a full XML document you call it’s Save method. No problem, the Save method has an overload that takes a StringWriter parameter so I’ll just create a StringWriter and a StringBuilder and I’ll be in business. Here’s my initial code.

        // GetFeedXML
        public virtual string GetFeedXML()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriter(sb);
            xdoc.Save(sw);
            return sb.ToString();
        }

That all looks good. So I run it and here’s the result.

    <?xml version="1.0" encoding="utf-16" standalone="yes" ?>
    <jobs>
      <job>
        <title>Audit Engagement Manager</title>

What the heck is that??? UTF-16?? It’s like the UTF-8 encoding that I set in my XDocument was completely ignored and .Net decided to use UTF-16 encoding. How can this not work? Where else do I even have an option to set the encoding? The StringWriter won’t let me set the encoding, the StringBuilder certainly won’t let me set the encoding.

So I started Googling and found that other people were having the same problem and I found several solutions that sounded plausible. Most of them centered around creating an XMLWriter with a settings object that would allow me to set the encoding. All of these required a fair bit of extra code and I never got one of them to actually work, but they did get me looking at the StringWriter as the place where I needed to set my encoding.

The Solution: Subclass StringWriter

At this point I’m looking into progressively more and more complex solutions when I stumble across a post by Ian Dykes called Writing XML with UTF-8 Encoding using XmlTextWriter and StringWriter. Ian basically says to create a new StringWriterWithEncoding class that inherits from StringWriter but allows you to set the encoding in the constructor. I used the same idea and created the StringWriterUtf8 class below. Instead of taking the encoding in a constructor, I opted to make the Encoding property always return UTF8.

    public class StringWriterUtf8 : StringWriter
    {
        public StringWriterUtf8(StringBuilder sb) : base(sb)
        {
        }
 
        public override Encoding Encoding
        {
            get { return Encoding.UTF8; }
        }
    }

Now I just need to use StringWriterUtf8 in my code instead of StringWriter and I’ll be using UTF-8 encoding in my writer. I did it, I tested it, and it worked. My XML output now looks like this:

    <?xml version="1.0" encoding="utf-8" standalone="yes" ?>
    <jobs>
      <job>
        <title>Audit Engagement Manager</title>

Thank you Ian for understanding more about this than I do and taking the time to put the solution out on your blog. By the way, I think it’s worth mentioning that several people posted more “settings type” solutions in the comments to Ian’s post. I tried them. They didn’t work for me. The only thing that did work was Ian’s idea of subclassing the StringWriter. So I now have a working method and here’s the final version of GetFeedXml that uses my StringWriterUtf8 class.

        // GetFeedXML
        public virtual string GetFeedXML()
        {
            List<JobFeedItem> list = GetFeedData();
            XDocument xdoc = BuildXmlFor(list);
            var sb = new StringBuilder();
            var sw = new StringWriterUtf8(sb);
            xdoc.Save(sw);
            return sb.ToString();
        }

Refactoring a TrySave Method

noreply@blogger.com (Rudy Lacovara) — Tue, 08 Feb 2011 07:34:00 +0000

When writing code, it’s unusual for me to get it right the first time. Typically I’ll just write a big block of statements that do something that I’ll need done. Then I’ll verify that it works. Then I’ll look at it, notice what an ugly mess it is, and refactor it so that the poor developer who inherits this code (and that developer might very well be me) will have some hope of maintaining it. Here’s a typical example that came up this week.

The scenario

I’m working in a controller class in an ASP.Net MVC2 application. I have some action methods that need to save the data passed in via a signup form. The signup form collects all of the data needed to create a new company and a new user in my application. So my signup form is a composite of Company data, User data, an other data that’s needed for the UI (like a ConfirmPassword field). I’ve chosen to encapsulate all of this stuff that the signup form needs in a view model class called SignupVM. It looks like this.

    public class SignupVM
    {
        public User User { get; set; }
        public Company Company { get; set; }
        public string ConfirmPassword { get; set; }
        public string FormError { get; set; }
        public string FormMessage { get; set; }
 
        public SignupVM()
        {
            this.User = new User();
            this.Company = new Company();
            this.ConfirmPassword = string.Empty;
            this.FormError = string.Empty;
            this.FormMessage = string.Empty;
        }
    }

So that part works really well. Here’s where it starts to get a little messy. I have a couple of Action Methods in my controller that receive a SignupVM that’s been populated with data by the View (or more accurately, the ModelBinder) and they need to save that data. Saving data involves three main steps. First I need to validate the Company, the User, and my ConfirmPassword field. Second I need to actually save the data, but I need to save the Company first and then make sure I set the correct CompanyGuid on the User before I save the User data. Third, I have some standard data that needs to get created for all new signups. Right now the only standard data is the GeneralApplicantsJob but I know that there will be a need for more standard data down the road. To handle this stuff, I created a TrySave() method in my controller class that looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            var userValidator = new UserValidator();
            var companyValidator = new CompanyValidator();
            var isError = false;
            // validate user
            try
            {
                userValidator.Validate(vm.User);
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetUserFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "User");
            }
            // validate company
            try
            {
                companyValidator.Validate(vm.Company);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetCompanyFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "Company");
            }
            // validate vm level fields
            try
            {
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.CopyToModelState(ModelState, "");
            }
            // if error, return false
            if (isError) { return false; }
            // if no error let's save the Company
            this.CompanyService.Save(vm.Company);
            // set the companyguid for the user
            vm.User.CompanyGuid = vm.Company.CompanyGuid;
            //save the user
            this.UserService.Save(vm.User);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(vm.Company.CompanyGuid);
            return true;
        }

Let’s Refactor

So, this block of code isn’t unmanageable, but I could definitely see it becoming unmanageable if I leave it like this and it gets added to a few times. Plus, it’s not immediately clear what this code is doing. What am I returning? Where am I returning it? What logic decides what my return value is? Looking at it I realize that I’m writing in a very Imperative style that puts the focus on the individual steps of “how” to do something, instead of writing in a Declarative (or Functional) style that puts the focus on “what” is being done. The code above is just a big list of steps that are a little difficult to follow without the comments. I’m just validating data and saving data, so why do I have all of these statements in my method? I can do better.

First let’s extract all of the validation code into an IsValidSignup() method. Just that one refactoring helps quite a bit. Now our TrySave method looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            // if no error let's save the Company            
            this.CompanyService.Save(vm.Company);            
            // set the companyguid for the user            
            vm.User.CompanyGuid = vm.Company.CompanyGuid;            
            //save the user            
            this.UserService.Save(vm.User);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(vm.Company.CompanyGuid);
            return true;        
        }

The next step is to refactor the data saving code. There’s two things that bother me about this code. First it’s critical that the the entities (Company and User) are saved it the right order, and that the user.CompanyGuid needs to be set in between the saves. Second, it’s critical that the GeneralApplicantsJobForCompany get created after the Company has been saved, plus I know that there will be a need for more standard data later on, which means at some point a developer is going to have to modify this helper method in a controller class to make sure signups are created with the right standard data. That feels like something that should be in my business layer, not in my UI code. So, I decided to extract all of the persistence logic (including the creation of standard data) to a new method on my SystemService class in my business layer. The mehtod is called SignupNewCompany() and now my TrySave method looks like this.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            this.SystemService.SignupNewCompany(vm.Company, vm.User);
            return true;
        }

This looks much cleaner to me. The focus is on what I’m doing instead of how I’m doing it, and the intent of the code is much more clear even though it no longer has comments. Now I do have three methods instead of one, but each of those three methods does one thing that is easy to understand at a glance. Best of all, I realized that I had put some logic in my UI that really needed to live in my business layer where it would be easier to reuse and easier to maintain. For anyone who’s interested, here are all 3 methods in their final form.

        // TrySave - SignupVM
        private bool TrySave(SignupVM vm)
        {
            if (!IsValidSignup(vm)) { return false; }
            this.SystemService.SignupNewCompany(vm.Company, vm.User);
            return true;
        }

        // IsValidSignup
        public bool IsValidSignup(SignupVM vm)
        {
            var userValidator = new UserValidator();
            var companyValidator = new CompanyValidator();
            var isError = false;
            // validate user
            try
            {
                userValidator.Validate(vm.User);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetUserFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "User");
            }
            // validate company
            try
            {
                companyValidator.Validate(vm.Company);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.SetUiFieldNames(GetCompanyFieldNameMappings());
                e.ErrorList.CopyToModelState(ModelState, "Company");
            }
            // validate vm level fields
            try
            {
                ValidateConfirmPassword(vm);
            }
            catch (RuleException e)
            {
                isError = true;
                e.ErrorList.CopyToModelState(ModelState, "");
            }
            return !isError;
        }

        public void SignupNewCompany( Company newCompany, User newUser)
        {
            // create the new company
            this.CompanyService.Insert(newCompany);
            // create the new user
            newUser.CompanyGuid = newCompany.CompanyGuid;
            this.UserService.Save(newUser);
            // create the GeneralApplicants job, it's created the first time we get it.            
            this.JobService.GetGeneralApplicantsJobForCompany(newCompany.CompanyGuid);
        }

How To Not Hate Dependency Injection.

noreply@blogger.com (Rudy Lacovara) — Mon, 17 Jan 2011 17:57:00 +0000

The Dependency Injection pattern is all the rage these days and more and more DI containers are becoming part of how we build .Net applications. I still have a love / hate relationship with Dependency Injection. I love the idea of being able to inject any external dependencies into my classes, thereby breaking those dependencies, but I hate the idea of creating an interface for every class that I need to inject, then maintaining that interface as well as any classes that implement it. Let me show you what I mean.

Dependency Injection on an MVC Controller Base Class With Interfaces

Typical scenario, I have an ASP.Net MVC2 application that needs to get data from a database. All of my data access code is contained in service classes like UserService, SystemService, CompanyService, etc. I don’t want to manually declare and instantiate whatever service classes I need in each of my controller classes, so I created an abstract base class for my controllers called AtsControllerBase which has properties that will provide instances of my service classes using a lazy creation pattern. Here’s an abbreviated version of the base class.

    public abstract class AtsControllerBase : Controller
    {
        //******************************************************
        // PROPERTIES
        //****************************************************** 
 
        // StateBox
        private HfStateBox _stateBox;
        public HfStateBox StateBox
        {
            get { return NewIfNull(_stateBox); }
            set { _stateBox = value; }
        }
        // AuthToken
        public AuthToken AuthToken { get { return StateBox.AuthToken; } }
        // ApplicantService
        private ApplicantService _applicantService;
        public ApplicantService ApplicantService
        {
            get { return NewIfNull(_applicantService); }
            set { _applicantService = value; }  
        }
        // CompanyService
        private CompanyService _companyService;
        public CompanyService CompanyService
        {
            get { return NewIfNull(_companyService); }
            set { _companyService = value; }
        }
        // FolderService
        private FolderService _folderService;
        public FolderService FolderService
        {
            get { return NewIfNull(_folderService); }
            set { _folderService = value; }
        }        
        // JobService
        private JobService _jobService;
        public JobService JobService
        {
            get { return NewIfNull(_jobService); }
            set { _jobService = value; }
        }
        // SecurityService
        private SecurityService _securityService;
        public SecurityService SecurityService
        {
            get { return NewIfNull(_securityService); }
            set { _securityService = value; }
        }
        // SystemService
        private SystemService _systemService;
        public SystemService SystemService
        {
            get { return NewIfNull(_systemService); }
            set { _systemService = value; }
        }
        // ServicePlanService
        private ServicePlanService _servicePlanService;
        public ServicePlanService ServicePlanService
        {
            get { return NewIfNull(_servicePlanService); }
            set { _servicePlanService = value; }
        }        
        // UserService
        private UserService _userService;
        public UserService UserService
        {
            get { return NewIfNull(_userService); }
            set { _userService = value; }
        }
        // WorkflowService
        private WorkflowService _workflowService;
        public WorkflowService WorkflowService
        {
            get { return NewIfNull(_workflowService); }
            set { _workflowService = value; }
        }
 
 
        //******************************************************
        // UTILITY METHODS
        //******************************************************
        
        // NewIfNull
        public T NewIfNull<T>(T obj) where T:new()
        {
            if (obj == null) { obj = new T(); }
            return obj;
        }
    }

This pattern works well for me because my concrete controllers never need to instantiate a service, they can just access any of the service properties in the base class. I don’t have the overhead of newing up a bunch of service classes that the concrete controller won’t use because the services aren’t instantiated until the first time the service property is accessed. That’s the lazy creation part. Finally, each property has a public setter, so I can replace any of my services if I need to inject a mock or stub or something.

So this already looks a lot like Dependency Injection. After all we have public setters for each of the services so we could switch them out if needed. There’s one thing missing though. We have no interfaces. Our properties are all using the concrete type of the service class (like FolderService) instead of some public interface (like IFolderService). So we could inject something, but it would have to be another instance of FolderService. That get’s us nowhere. How can we inject something else like a MockFolderService?? Hmmmm, I guess the right way to implement Dependency Injection is to create an IFolderService interface for my FolderService class. At least that’s the pattern that I always see in books and code samples. Let’s try that. Ok, FolderService only has 10 public methods. That’s not too bad so we’ll start with that one. Here’s the service code.

    public partial class FolderService : ServiceBase
    {
 
        //**************************************************************************************
        // FOLDER METHODS
        //**************************************************************************************
 
        // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT * 
                           FROM [Folder]                             
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }
 
        // GetListOfFolderWithCountsForCompany
        public virtual List<FolderWithCount> GetListOfFolderWithCountsForCompany(Guid companyGuid)
        {
            string sql = @"SELECT f.*, (SELECT COUNT([ApplicantGuid]) 
                          FROM [ApplicantInFolder] WHERE [FolderGuid] = f.[FolderGuid]) As ApplicantCount
                          FROM [Folder] f
                          WHERE f.[CompanyGuid] = @CompanyGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@CompanyGuid", companyGuid));
            return (dao.GetList<FolderWithCount>(command));
        }
 
         // GetListOfFoldersForCompany
        public virtual List<Folder> GetListOfFoldersForCompany(Guid companyGuid)
        {
            string sql = @"SELECT f.*
                                  FROM [Folder] f
                                  WHERE f.[CompanyGuid] = @CompanyGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@CompanyGuid", companyGuid));
            return (dao.GetList<Folder>(command));
        }
 

        // GetListOfFoldersForApplicant
        public virtual List<Folder> GetListOfFoldersForApplicant(Guid applicantGuid)
        {
            string sql = @"select f.*
                        from Folder f
                        join ApplicantInFolder aif on aif.FolderGuid = f.FolderGuid
                        where aif.ApplicantGuid = @ApplicantGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@ApplicantGuid", applicantGuid));
            return (dao.GetList<Folder>(command));
        }
  
        // GetCachedListOfFoldersForCompany
        public virtual List<FolderWithCount> GetCachedListOfFoldersForCompany(Guid companyGuid)
        {
            if (companyGuid.Equals(NullValues.NullGuid))
            {
                // return empty list if a null guid was passed
                return new List<FolderWithCount>();
            }
            string key = "__GetCachedListOfFoldersForCompany_" + companyGuid.ToString();
            CacheHelper helper = new CacheHelper();
            List<FolderWithCount> list = helper.TryGet<List<FolderWithCount>>(key);
            if (list == null)
            {
                RefreshCachedListOfFoldersForCompany(companyGuid);
                list = helper.TryGet<List<FolderWithCount>>(key);
            }
            return list;
        }
 

        // RefreshCachedListOfFoldersForCompany
        public virtual void RefreshCachedListOfFoldersForCompany(Guid companyGuid)
        {
            string key = "__GetCachedListOfFoldersForCompany_" + companyGuid.ToString();
            CacheHelper helper = new CacheHelper();
            List<FolderWithCount> list = helper.TryGet<List<FolderWithCount>>(key);
            list = GetListOfFolderWithCountsForCompany(companyGuid);
            helper.Add(key, list);
        }
  
        // Insert
        public virtual void Insert(Folder folder)
        {
            this.FolderPersister.Insert(folder);
        }
 

        // Save
        public virtual void Save(Folder folder)
        {
            this.FolderPersister.Save(folder);         
        } 
 
        // Delete
        public virtual void Delete(Folder folder)
        {
            this.FolderPersister.Delete(folder.FolderGuid);
        }
 

        // AddApplicantToFolder
        public virtual void AddApplicantToFolder(Guid applicantGuid, Guid companyGuid, Folder folder)
        {
            this.FolderPersister.SafeInsertApplicantInFolder(applicantGuid, folder.FolderGuid);
            RefreshCachedListOfFoldersForCompany(companyGuid);
        } 
    }

We’ve basically got 10 public data access methods. Now I need to create a public interface that defines all of the public methods, so I’m going to create a new IFolderService interface. ReSharper even makes it easy for me with a handy Extract Interface tool. So my interface looks like this.

    public interface IFolderService
    {
        Folder GetFolderByFolderGuid(Guid folderGuid);
        List<FolderWithCount> GetListOfFolderWithCountsForCompany(Guid companyGuid);
        List<Folder> GetListOfFoldersForCompany(Guid companyGuid);
        List<Folder> GetListOfFoldersForApplicant(Guid applicantGuid);
        List<FolderWithCount> GetCachedListOfFoldersForCompany(Guid companyGuid);
        void RefreshCachedListOfFoldersForCompany(Guid companyGuid);
        void Insert(Folder folder);
        void Save(Folder folder);
        void Delete(Folder folder);
        void AddApplicantToFolder(Guid applicantGuid, Guid companyGuid, Folder folder);
        SqlDao SharedSqlDao { get; set; }
        ApplicantPersister ApplicantPersister { get; set; }
        CareerSiteSettingsPersister CareerSiteSettingsPersister { get; set; }
        CompanyPersister CompanyPersister { get; set; }
        CompanySettingsPersister CompanySettingsPersister { get; set; }
        FolderPersister FolderPersister { get; set; }
        HistoryItemPersister HistoryItemPersister { get; set; }
        JobPersister JobPersister { get; set; }
        JobStubPersister JobStubPersister { get; set; }
        ResumePersister ResumePersister { get; set; }
        UserPersister UserPersister { get; set; }
        WorkflowPersister WorkflowPersister { get; set; }
        T NewIfNull<T>(T obj) where T : new();
        string GetSqlCsv(List<Guid> list);
        string GetSqlCsv(List<int> list);
    }

There, that wasn’t too bad. Now I just need to change my controller FolderService property to type IFolderService and I have Dependency Injection implemented for the FolderService. I can create new classes that implement IFolderService and inject them into my controller using the public FolderService property like this.

            var controller = new FolderController();
            IFolderService mock = new MockFolderService();
            controller.FolderService = mock;

That’s great! Now I just need to do that for… wait… I’m going to have to do that for every service class in my application. Then I’m going to have to maintain each of those interfaces every time I make a change to a service class. That’s a lot of extra code and a lot of work that’s going to have to be repeated every time I make a change to my service classes. Why am I doing this again? This is starting to feel like the hate part of the love / hate relationship.

Dependency Injection on an MVC Controller Base Class Without Interfaces

So I don’t really want to create an interface for every single dependency in my application and then have to maintain that interface as well as my concrete classes, but what other option do I have? Everybody implements DI using interfaces right? Interfaces are like a badge of honor that demonstrate you know how to architect an application the right way. You haven’t been to architecture land until you have your interface badge. We love interfaces!

Well I don’t love any code that I have to spend extra time maintaining every time I make a change to my application. All that stuff adds up after a while. I call it the death of a thousand cuts. One day you look up and all those little extra tasks like maintaining interfaces add up to a mountain of friction that makes it miserable to make changes to your code. Now there are places for interfaces, but if possible I’m opting for an easier alternative.

So step one in the easier alternative, throw out the interface that I just created and change my FolderService property back to type FolderService (no more IFolderService). Step two…. actually we don’t need a step two we’re done. Lets take a closer look at one of the data access methods in our FolderService class.

       // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT *
                           FROM [Folder]
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }

Hmmm, that’s a virtual method. That means that I can create a class that inherits from FolderService and just override the GetFolderByFolderGuid() method. The FolderService class actually functions as my interface in this scenario. So, if I’m writing a unit test where I want to stub out my FolderService I could create the following class.

    public class FolderServiceStub: FolderService
    {
        public override Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            return new Folder() {FolderGuid = folderGuid, FolderName = "Test Folder"};
        }
    }

FolderServiceStub inherits from FolderService so I can use that stub class anywhere that an object of type FolderService is expected. The end result is that my testing code (or any other DI code) winds up looking almost exactly like my code did with full on interface based Dependency injection

    var controller = new FolderController();
    FolderService mock = new FolderServiceStub();
    controller.FolderService = mock;

Summary

So, make all of your public methods virtual and you can implement a Poor Man’s Dependency Injection that doesn’t require interfaces. It’s not the right solution every situation, but I’ve found that it works most of the time for me. It also has the one quality that I prize most in code. It’s so simple that I forget it’s even there.

AAPL Part 7: Building a Generic Data Mapper

noreply@blogger.com (Rudy Lacovara) — Mon, 10 Jan 2011 06:23:00 +0000

I got bored with this series a while back and I haven’t been writing on it, mostly because I’ve been using the architecture (Agile ADO.Net Persistence layer) to write a bunch of code. The basic idea of AAPL was to create an application architecture that embraced code changes instead of resisting them. I took a look back at all of the application architectures I’ve worked with over the years and thought carefully about the parts of those architectures that caused me pain whenever made changes to the application. I then threw all those things out and tried to put together an application architecture based on ADO.Net (not LINQ, EF, or nHibernate) that was optimized for making code changes easy and reliable. I’ve been using the AAPL architecture for a while now and I’m happy to report that those decisions have really paid off.

Why are we doing this again?

Just a a quick reminder. The point of this architecture is to give us the ability to write persistence code that looks like this.

        // GetFolderByFolderGuid
        public virtual Folder GetFolderByFolderGuid(Guid folderGuid)
        {
            string sql = @"SELECT * 
                           FROM [Folder]                             
                           WHERE [FolderGuid] = @FolderGuid";
            SqlDao dao = SharedSqlDao;
            SqlCommand command = dao.GetSqlCommand(sql);
            command.Parameters.Add(dao.CreateParameter("@FolderGuid", folderGuid));
            return dao.GetSingle<Folder>(command);
        }

Whenever we need a new data access method we want to just write the new query, package it up in an ADO.Net SqlCommand object, then pass both the SqlCommand and the desired model class type to a Data Access Object that will then handle all of the plumbing required to execute the command, get the data, package it up as whatever model type we passed in and then give it back. This is a very low friction way to handle data access. It’s easy to maintain, new data methods take seconds to create, and we never have to touch the plumbing code. The end result is that maintaining code and making changes to our model should feel like this…

instead of this…

What is a data mapper?

The key piece of plumbing code that makes this architecture so painless is the GenericDataMapper. As described above, when we read data we execute an SQLCommand, get the resulting IDataReader, then map the data in that IDataReader to a model class (an entity class). The DataMapper is the class that encapsulates the code that knows how to map the fields from an IDataReader to the properties in a model class. For example, the UserMapper class contains the logic needed to take an IDataReader and use it’s data to populate a new object of type User.

What does the Generic Data Mapper do?

We could write a separate DataMapper class for each model type. We could have a UserMapper, a CompanyMapper, a JobMapper, a WorkflowMapper, an ActionMapper, a StepMapper… you get the idea. But doing it that way would require that 1) We write all those DataMapper classes, and 2) We modify those DataMapper classes whenever we make a change to one of our models. No thanks.

It would be better to write a single DataMapper class that could handle the mapping for any model class that we passed to it. Remember that early on we made the decision that the columns in our database would have the exact same names as the data properties in our model classes. That constraint makes the mapping logic pretty simple. We can create a GenericDataMapper that uses reflection to find all the properties on our model class and matches them to any fields with the same name in our IDataReader. No hard coded mapping logic needed. Just a little bit of reflection and a standard naming convention and we can create a single GenericDataMapper that will handle mapping for all of our model classes.

DataMapperFactory

We use a DataMapperFactory to get the right DataMapper for a model type. The logic is simple. We have a GetMapper() method that takes a type as it’s parameter. It checks to see if we have a custom mapper for the type, if not it returns the GenericMapper . In the example below we have just one custom data mapper, ListItemDTOMapper. Every other model class is handled by the GenericMapper.

    class DataMapperFactory
    {
        public IDataMapper GetMapper(Type dtoType)
        {
            switch(dtoType.Name)
            {
                case "ListItemDTO":
                    return new ListItemDTOMapper();
                default:
                    return new GenericMapper(dtoType);
            }       
        }
    }

IDataMapper

All of our DataMappers implement a custom IDataMapper interface.

    interface IDataMapper
    {
        // Main method that populates dto with data
        Object GetData(IDataReader reader);
        // Gets the num results returned. Needed for data paging.
        int GetRecordCount(IDataReader reader);
    }

IDataMapper requires that we implement just two methods. GetData() takes an IDataReader and returns a model class that contains the data for the current record in that reader (note that one call to GetData() returns one model object, lists can be created by calling GetData() multiple times ). GetRecordCount() will return the total number of records returned by the IDataReader and is used for data paging. So these are the only two public methods that we’ll need to implement.

GenericMapper

Now we get down to it. Here is the complete code for the GenericMapper.

    class GenericMapper : IDataMapper
    {
        public System.Type DtoType { get; set; } 
        private bool _isInitialized = false;
        private List<PropertyOrdinalMap> PropertyOrdinalMappings;
 
 
        public GenericMapper(System.Type type)
        {
            DtoType = type;
        }
 
 
        private void InitializeMapper(IDataReader reader)
        {
            PopulatePropertyOrdinalMappings(reader);
            _isInitialized = true;
        }
 
 
        public void PopulatePropertyOrdinalMappings(IDataReader reader)
        {
            // Get the PropertyInfo objects for our DTO type and map them to 
            // the ordinals for the fields with the same names in our reader.  
            PropertyOrdinalMappings = new List<PropertyOrdinalMap>();
            PropertyInfo[] properties = DtoType.GetProperties();
            foreach (PropertyInfo property in properties)
            {
                PropertyOrdinalMap map = new PropertyOrdinalMap();
                map.Property = property;
                try
                {
                    map.Ordinal = reader.GetOrdinal(property.Name);
                    PropertyOrdinalMappings.Add(map);
                }
                catch { }
            }
        }
 
 
        public Object GetData(IDataReader reader)
        {
            if (!_isInitialized) { InitializeMapper(reader); }
            object dto = Activator.CreateInstance(DtoType);
            foreach (PropertyOrdinalMap map in PropertyOrdinalMappings)
            {
                 if (!reader.IsDBNull(map.Ordinal)) 
                 { 
                     map.Property.SetValue(dto,reader.GetValue(map.Ordinal),null); 
                 }
            }
            return dto;
        }
 
 
        public int GetRecordCount(IDataReader reader)
        {
            Object count = reader["RecordCount"];
            return count == null ? 0 : Convert.ToInt32(count);
        }
 
 
        private class PropertyOrdinalMap 
        {
            public PropertyInfo Property { get; set; }
            public int Ordinal { get; set; }
        }
 
    }

This isn’t very much code, but the logic can be a little difficult to follow. Let’s break it down and see how it works.

When the user creates an instance of GenericMapper, the constructor requires them to pass in the model type they want to map to. That type is assigned to our local DtoType property (ModelType might have been a better name). Note that we’re creating an instance, not using static methods. This is important because once the GenericMapper is created it can be used multiple times for multiple records without having to recalculate the mappings.

The user calls the GetData method which takes an IDataReader as a parameter. GetData is the heart of our mapper.

        public Object GetData(IDataReader reader)
        {
            if (!_isInitialized) { InitializeMapper(reader); }
            object dto = Activator.CreateInstance(DtoType);
            foreach (PropertyOrdinalMap map in PropertyOrdinalMappings)
            {
                 if (!reader.IsDBNull(map.Ordinal)) 
                 { 
                     map.Property.SetValue(dto,reader.GetValue(map.Ordinal),null); 
                 }
            }
            return dto;
        }

GetData first checks to see if the GenericMapper has been initialized. If not it calls InitializeMapper() which handles all initialization including a call to PopulatePropertyOrdinalMappings(). This is where the magic happens.

        public void PopulatePropertyOrdinalMappings(IDataReader reader)
        {
            // Get the PropertyInfo objects for our DTO type and map them to 
            // the ordinals for the fields with the same names in our reader.  
            PropertyOrdinalMappings = new List<PropertyOrdinalMap>();
            PropertyInfo[] properties = DtoType.GetProperties();
            foreach (PropertyInfo property in properties)
            {
                PropertyOrdinalMap map = new PropertyOrdinalMap();
                map.Property = property;
                try
                {
                    map.Ordinal = reader.GetOrdinal(property.Name);
                    PropertyOrdinalMappings.Add(map);
                }
                catch { }
            }
        }

We have a private class named PropertyOrdinalMap which is just a simple container for a System.Reflection.PropertyInfo object and an int. The int represents the ordinal (field index) in our IDataReader for the field that matches the name of the property stored in PropertyInfo. The PopulatePropertyOrdinalMappings method just loops over all of the properties in our DtoType and tries to find an ordinal in IDataReader for a field with the same name. If a match exists, a new PropertyOrdinalMap is created an added to the PropertyOrdinalMappings List. If a match doesn’t exist, no PropertyOrdinalMap is created. The result is that we just skip any properties in our DtoType that the reader doesn’t have a value for. That’s a key design decision. We don’t throw an error if a mapping doesn’t exist, we just skip that property and get the data that we can map.

Once the mappings are created it’s all pretty straight forward. We return to GetData(). We use System.Activator() to create a new instance of our DtoType called dto (short for Data Transfer Object). Then we iterate over our PropertyOrdinalMappings and use reflection to set the value of each dto property that we have a mapping for. At that point we have a brand new object of whatever type was passed in to the constructor and it’s been populated with the data from the current record in the IDataReader. We’re done! We can just return the dto.

So that’s it. A little object oriented design, some super-simple reflection code, and I have a GenericMapper that allows me to never right another line of data mapping code (as long as I’m willing to use the same name for my model properties and data table columns). I haven’t spent any time optimizing this. In fact, I haven’t really looked at it since I first wrote it. If anyone has suggestions for improvements please let me know.

Type Argument Specification is Redundant

noreply@blogger.com (Rudy Lacovara) — Mon, 03 Jan 2011 17:16:00 +0000

One of the things I like about ReSharper is that it teaches me things I didn’t know. Recently I was writing a property that calls a generic helper method that I’d created and ReSharper gave me the little popup below letting me know that I might want to refactor that line of code.

“Type argument specification is redundant”, what the heck does that mean? So I looked into it and learned something interesting that I didn’t know before. To understand what’s going on let’s first look at my NewIfNull() method.

NewIfNull() is a simple generic helper method that takes an object of any type that has a parameterless constructor, it then checks to see if that object is null. If it is null, the helper returns a new instance of that object’s type. If the object isn’t null, it just returns the object. It’s kind of like lazy loading for type instantiation. It’s useful in situations where I have properties for things like data services that I don’t want to instantiate until the first time I call them (if they’re called at all). Here’s the NewIfNull() code:

You can see that NewIfNull() is a generic method that takes a type parameter T which represents the type of the object that I’m passing in. A typical call to NewIfNull looks like

This isn’t too bad, but it’s a little annoying that I have to pass _workflowService as a parameter and then I also have to specify WorkflowService as my type inside of the angle brackets. Well apparently that part is completely unnecessary. That little helper message from ReSharper was trying to tell me that since T is just the type of my parameter obj, .Net can infer the type of T from parameter obj. Passing type T in the angle brackets is actually redundant. Brilliant! Now my calls to NewIfNull() look like this.

Much cleaner. No angle brackets or generic type parameters needed. I’ve been writing and using generic methods for quite a while and I never knew .Net would infer types like this. Thank you ReSharper. If you want to learn more, the MSDN documentation on Type Inference is pretty good. Check out Section 26.3.3 Type Inference.

Muffin Top architecture is killing us

noreply@blogger.com (Rudy Lacovara) — Mon, 02 Aug 2010 17:00:00 +0000

I want to share a story. I recently encountered a company that had somehow navigated the many hazards that face technology startups and managed to create a solid, successful business. They had gone public and now after years of struggling with a collection of applications that consisted largely of simple markup and script pages that directly accessed database queries, they were in the process of transitioning everything over to a real architecture, a single unified “Framework”.

The “Framework” was a thing of architectural beauty, object oriented design as far as the eye could see. It had been designed by some incredibly bright developers. So bright in fact that they had progressed beyond the point where they were even involved in writing code for a shipping application. Some were so smart, that they skipped the mundane part of a programmer’s career where they spend years writing and maintaining application code, and instead they moved directly into the architecture world.

The core of the “Framework” was the data access module which was based on LINQ to SQL, but hid all of the entities as well as the data context behind interfaces. These interfaces would of course need to be updated whenever changes were made to the LINQ data model, but they made the “Framework” feel a lot more object oriented and provided the additional benefit of hiding any functionality that the architecture group didn’t think regular application developers should have access to.

The LINQ data model also benefited from some extra architectural love. Mapping directly to tables was out of the question, so every LINQ entity mapped to a collection of sprocs. After all, the idea was to create an entity model, not just map directly to table data. No, every entity was carefully planned and modeled, every field the application developers might need was anticipated and aggregated using those sprocs. Now this did create yet another point (like the interfaces) at which data mapping logic could go wrong and throw errors, but that wasn’t a problem because the architecture team was able to use Visual Studio to generate a massive collection of database unit tests that would exhaustively test every possible crud scenario for every sproc and assert that it worked as expected. Of course the massive collection of db unit tests and the sprocs themselves would need to be rewritten every time even the smallest change was made to the data model, but architecturally speaking it was just the right thing to do.

Finally the day came when the “Framework” was pushed out to the unwashed masses of developers who would benefit from it’s glorious design. But an unexpected thing happened. New application development done on top of the “Framework” slowed to a crawl. It was taking 3 to 5 times longer for application developers to build code with the “Framework”. To make matters worse, the code that was produced, ran slower, sometimes much, much slower than the old legacy code that accessed the database directly. Also, the application developers kept coming back to the architecture team requesting that additional fields be added to the entities. They said these fields were required in order to provide features that were already available in the legacy applications. It was as if these guys had never been part of the endless modeling meetings where everyone worked out in advance every field each entity would ever need. The problems just kept piling up, and application ship dates slipped further and further. Eventually the architects had to face up to an unpleasant conclusion… the company’s application developers just weren’t very good. I’m not kidding, that really was their conclusion. I actually heard that straight from the mouth of one of the architects. (BTW, in case you’re wondering the application developers were actually some of the most skilled that I’ve ever run into. They were very good developers.)

The point of this story is that the problems we face as programmers in the .Net world are rapidly changing. It used to be that the most common problem I ran into was horribly constructed spaghetti code that was then copied and pasted into 20 different pages of the application. No code reuse, poor logic structure, and of course direct access to the database in the code behind or even in the markup pages. The problems were largely the result of us as .Net developers just not being skilled enough to properly design our code. But .Net is a developer-friendly technology and most of us were able to hack together code that worked even if it wasn’t pretty. Note that I did say “us”. Some of my early code is horrific.

Now, increasingly I see a much different problem. As a community we’ve learned a lot more about real software development. We understand all of those architectural things that our Java buddies used to talk about. We’re using design patterns, object oriented design, automated testing, ORMs, true layered architectures, dependency injection, component architectures, class inheritance all over the place, interfaces, contract based programming, service oriented architecture and a bunch of stuff that sounds like it should enable us to ship more reliable code, and to ship it faster. The problem is that in my experience, most of the time it doesn’t. Don’t get me wrong, I'm not saying that the practices listed above are bad, or that we should go back to writing simple markup pages with SQL queries written directly in our scripts. What I am saying is that in our eagerness to flex our new-found coding muscles, we’re just throwing these things into our applications because we think they’re a “best practice” and the result is a lot of over-architected, impossible to maintain, fat, sprawling code. I just get a sense that many very intelligent .Net developers are throwing this stuff into their applications because this is what a real architecture looks like, not because they are getting some measurable benefit. At the end of the day it’s all about shipping code, not creating an architecture that makes us feel like real architects.

A smart guy that I met at the unspecified company mentioned earlier put a name to it. He called it Muffin Top Architecture. I didn’t know what “muffin top” meant until he explained it to me. If you’re confused do a search on Google images for muffin top and you’ll figure it out. Hint, it doesn’t refer to the images of real muffins. Anyway, I’m starting to see quite a bit of Muffin Top code. The company I told the story about is by no means atypical. I could go on for pages and pages. I’ve even encountered companies who are unable to make changes to their own code, that they wrote. There are just so many layers of architecture and class inheritance that changes are incredibly painful even for the original developers.

I don’t have an easy answer, but I do think we need to start producing code that is simpler, easier to maintain, and quicker to produce. .Net is a great stack and it isn’t difficult to work with, but we’re developing this reputation of “.Net is a solid platform HOWEVER it takes so long to build on that it’s not appropriate for many projects, and definitely not for a startup project”. I don’t think that’s true. The problem isn’t .Net, the problem is that we’re over-architecting, and in many cases Microsoft is more than happy to help us further down the path to destruction with new persistence frameworks, designers, and component architectures. I think we should be looking more and more at technologies like Ruby on Rails and trying to figure out how they achieve the remarkable efficiencies that are possible on that platform. It’s certainly not through a designer. I also think that the answer lies in less code that is simpler and easier to understand, not piles of generated code and component architectures that make it impossible to figure out what code is executing when.

Another big part of the problem is the emergence of the non-coding architect. This is the person who doesn’t write any shipping code themselves, they just tell everyone else how they should be writing code. I think the role of architect is tremendously important. I see the architect as the lead developer, the mentor, the overall system designer. But let me make this clear, an architect who doesn’t write code isn’t a real programmer, they are an obstacle to real programmers. It doesn’t matter how skilled that person is, if the architect isn’t writing shipping code then their incentives are not aligned with the other developers. The real programmer’s number one priority in life is to ship working code. The non-coding architect’s priority is to create beautiful architecture.

If you have any thoughts please don’t hesitate to post them. Personally, I think that Muffin Top Architecture is killing us. Is there a solution or am I just destined to go be a Rails developer like Mike Moore, Jeff Cohen, and Scott Bellware? Check out Herding Code 84 for some interesting discussion by the .Net expatriates.

AAPL Part 6: Building a DataMapper and DataMapperFactory

noreply@blogger.com (Rudy Lacovara) — Tue, 27 Apr 2010 08:50:00 +0000

I recently decided that I needed to take a fresh look at how to build a persistence architecture that would provide the flexibility of an ORM, that would embrace change instead of resist it (making maintenance code easier), and would use ADO.Net. I started building from the ground up, threw away any best practices that resulted in friction, and Agile ADO.Net Persistence Layer is the result. Part 1 in the series can be found here: Agile ADO.Net Persistence Layer Overview.

In this post, we’ll get into the DataMapper. If you’ve been waiting to see some real ADO.Net code, this is the class you’ve been waiting for.

Where the DataMapper classes fit in

Let’s review our solution structure. Core is our main business layer project (Note: in previous posts and in the sample code Core is actually the BAL project. I never liked the name BAL so I’ve renamed it Core. Same code, just a different name. Sorry for any confusion.). Core contains Service classes which are our one stop shop for all persistence (see AAPL Part 3 and AAPL Part 4 for more on how the service classes work). The Service classes use the SqlDao for all communication with the database. The SqlDao uses the DataMapper classes to handle all mapping between the DataReader returned by executing a query, and the desired data shape. Here’s what our project looks like:

Last post we covered the SqlDao class and saw how it’s data access methods take a query wrapped in an SqlCommand, execute that query, then return either a Single<DTO> (where DTO is our desired data shape), List<DTO>, DataPage<DTO>,or a scalar value. The data access methods in SqlDao handle all the boilerplate logic for getting a connection, executing a query, stuff like that, but the logic that is specific to a data shape, things like get the value from this column in my reader put it in this property on my returned data shape, that stuff get’s delegated to a DataMapper class. This is what that SqlDao code looks like:

SqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
    reader.Read();
    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
    dto = (T)mapper.GetData(reader);
    reader.Close();
}

What does the DataMapperFactory do?

You’ll notice in the code above that we get our IDataMapper object from a DataMapperFactory. If we need to get the DataMapper for type User, we just call the DataMapperFactory.GetMapper method, pass type(User) as a parameter, and GetMapper returns the appropriate concrete DataMapper (Note that the code example above uses generics, so we use generic T instead of a specific type like User, but the logic is the same). DataMapperFactory is a simple factory class with one method, GetMapper. The code looks like this:

class DataMapperFactory
{
    public IDataMapper GetMapper(Type dtoType)
    {
        switch(dtoType.Name)
        {
            case "ListItemDTO":
                return new ListItemDTOMapper();
            case "User":
                return new UserMapper();
            default:
                return new GenericMapper(dtoType);
        }       
    }
}

Whenever we create a new IDataMapper for a specific type, like User, we need to add it to this factory method. The method is just a switch statement that uses the type.Name to look for a type-specific implementation of IDataMapper. If it finds one, it returns it. If it doesn’t find a type specific IDataMapper, it returns an instance of GenericMapper, which is our reflection based DataMapper that can be used to handle mappings for any query / data shape combination. We’ll cover GenericMapper in the next post.

What does the DataMapper do?

The behavior of a DataMapper class is defined by the IDataMapper interface. The interface requires DataMappers to implement two methods.

interface IDataMapper
{
    // Main method that populates dto with data
    Object GetData(IDataReader reader);
    // Gets the num results returned. Needed for data paging.
    int GetRecordCount(IDataReader reader);
}

The GetData method is the important one. That’s the method that our SqlDao calls whenever it needs to map the current record in an SqlDataReader to a specific data shape. Remember, this method doesn’t do any mapping of an SqlDataReader to a List, or DataPage, or anything like that. That’s handled by the SqlDao data access methods. The purpose of the DataMapper GetData function is to take an SqlDataReader that’s already open and on a record, read that record, put the column data into an object (DTO) of the desired data shape, then return that single DTO.

Note that GetData returns an Object, so the SqlDao has to cast the GetData result to the proper data shape. I tried all kinds of designs to get around this cast, I used interfaces, generics, base classes, but in the end all of these options added complexity to the code that I just couldn’t accept. So, I opted to stick with GetData returning an object. I think it was the right tradeoff, but if anyone has a better solution please let me know.

Here is a diagram of our IDataMapper interface and a couple of concrete DataMapper implementations.

Inside a DataMapper class

So let’s take a look inside the UserMapper class and see what an IDataMapper implementation looks like. The user class is a data shape that represents, wait for it… a user. The class looks like this.

So the job of UserDataMapper is to take a SqlDataReader, get the values from it’s columns, and put those values into and instance of the User class above. The DataMapper implementation that I use accomplishes this task in a very structured way that is easy to copy and reproduce for new data shapes.

Before we look at the code, I should mention that I write a fair amount of extra code to get the ordinals for each data field instead of just using the string field names. The reason for this is that using the ordinals allows me to use strongly typed methods on the SqlDataReader to get each column. This allows me to avoid the cast to and from Object that occurs when you get a column by name which results in boxing and unboxing for value types. It also gives me a little type safety. If the query is returning a column called UserGuid that has a type of Int32, when I call reader.GetGuid() on that column, my code will throw an error and alert me that there is a problem with my query. For more on the benefits of using ordinals take a look at High Performance Data Access Layer Architecture Part 3.

Now, with no further ado, here’s the UserMapper implementation.

class UserMapper : IDataMapper
{
    private bool _isInitialized = false;
    private int _ordinal_UserGuid;
    private int _ordinal_UserRole;
    private int _ordinal_CreatedUtc;
    private int _ordinal_ModifiedUtc;
    private int _ordinal_CompanyGuid;
    private int _ordinal_FirstName;
    private int _ordinal_LastName;
    private int _ordinal_Email;
    private int _ordinal_Password;
    private int _ordinal_PhoneWork;
    private int _ordinal_PhoneMobile;
 
    private void InitializeMapper(IDataReader reader)
    {
        PopulateOrdinals(reader);
        _isInitialized = true;
    }
 
    public void PopulateOrdinals(IDataReader reader)
    {
        _ordinal_UserGuid = reader.GetOrdinal("UserGuid");
        _ordinal_UserRole = reader.GetOrdinal("UserRole");
        _ordinal_CreatedUtc = reader.GetOrdinal("CreatedUtc");
        _ordinal_ModifiedUtc = reader.GetOrdinal("ModifiedUtc");
        _ordinal_CompanyGuid = reader.GetOrdinal("CompanyGuid");
        _ordinal_FirstName = reader.GetOrdinal("FirstName");
        _ordinal_LastName = reader.GetOrdinal("LastName");
        _ordinal_Email = reader.GetOrdinal("Email");
        _ordinal_Password = reader.GetOrdinal("Password");
        _ordinal_PhoneWork = reader.GetOrdinal("PhoneWork");
        _ordinal_PhoneMobile = reader.GetOrdinal("PhoneMobile");
    }
 
    public Object GetData(IDataReader reader)
    {
        // This is where we define the mapping between the object properties and the 
        // data columns. The convention that should be used is that the object property 
        // names are exactly the same as the column names. However if there is some 
        // compelling reason for the names to be different the mapping can be defined here.
 
        // We assume the reader has data and is already on the row that contains the data 
        //we need. We don't need to call read. As a general rule, assume that every field must 
        //be null  checked. If a field is null then the nullvalue for that  field has already 
        //been set by the DTO constructor, we don't have to change it. 
        if (!_isInitialized) { InitializeMapper(reader); }
        User dto = new User();
        // Now we can load the data
        if (!reader.IsDBNull(_ordinal_UserGuid)) { dto.UserGuid = reader.GetGuid(_ordinal_UserGuid); }
        if (!reader.IsDBNull(_ordinal_UserRole)) { dto.UserRole = (UserRole)reader.GetInt32(_ordinal_UserRole); }
        if (!reader.IsDBNull(_ordinal_CreatedUtc)) { dto.CreatedUtc = reader.GetDateTime(_ordinal_CreatedUtc); }
        if (!reader.IsDBNull(_ordinal_ModifiedUtc)) { dto.ModifiedUtc = reader.GetDateTime(_ordinal_ModifiedUtc); }
        if (!reader.IsDBNull(_ordinal_CompanyGuid)) { dto.CompanyGuid = reader.GetGuid(_ordinal_CompanyGuid); }
        if (!reader.IsDBNull(_ordinal_FirstName)) { dto.FirstName = reader.GetString(_ordinal_FirstName); }
        if (!reader.IsDBNull(_ordinal_LastName)) { dto.LastName = reader.GetString(_ordinal_LastName); }
        if (!reader.IsDBNull(_ordinal_Email)) { dto.Email = reader.GetString(_ordinal_Email); }
        if (!reader.IsDBNull(_ordinal_Password)) { dto.Password = reader.GetString(_ordinal_Password); }
        if (!reader.IsDBNull(_ordinal_PhoneWork)) { dto.PhoneWork = reader.GetString(_ordinal_PhoneWork); }
        if (!reader.IsDBNull(_ordinal_PhoneMobile)) { dto.PhoneMobile = reader.GetString(_ordinal_PhoneMobile); }
        return dto;
    }
 
    public int GetRecordCount(IDataReader reader)
    {
        Object count = reader["RecordCount"];
        return count == null ? 0 : Convert.ToInt32(count);
    }
}

You’ll notice that the first thing we do is declare an _ordinal_FieldName member for each property in our User class. These local members are used to store the ordinal location for the column in the SqlDataReader that maps to that User field. We also create an _isInitialized flag that let’s us know whether we’ve set our _ordinal_FieldName values yet. This is important when the UserMapper is being used to get more than one User from a DataReader. The SqlDao code will get a single IDataMapper instance and will then call that mapper over and over for each record that exists in the SqlDataReader. We only want to find the ordinals once, so we create our _isInitialized flag that can be set to true after we set all the ordinal values.

Next is the InitializeMapper method. This just calls our PopulateOrdinals method and then sets that _isInitialized flag, that we just discussed, to true.

PopulateOrdinals is the method that takes our SqlDataReader, finds the ordinal location of each field we need, then stores those ordinal values in our local _ordinal_FieldName members.

Finally we arrive at GetData, the heart of our class. We’ve structured our code in such a way that GetData is actually pretty simple. We check to make sure the _isInitialized flag has been set. If it hasn’t we call PopulateOrdinals. We then create a new instance of our User class called dto. After that it’s just a matter of repeating the same line of code for each field.

if (!reader.IsDBNull(_ordinal_UserGuid)) { dto.UserGuid = reader.GetGuid(_ordinal_UserGuid); }

This one line of code performs a null check on the ordinal, if the column exists it uses a strongly typed Get method to get the value of that column and assign it to the dto, if the column is null then we do nothing because the constructor for User already set each property to it’s null value on creation. And that’s it, after setting each field value (or not setting it) we just return our dto and we’re done. Simple maintainable, easy to copy and reproduce code.

The last method is GetRecordCount. We need this method to support the DataPage<DTO> shape. Our SqlDao needs to know the total record count if it’s returning a data page. If it’s not returning a data page this method isn’t used.

Conclusion

That’s it. Now that I look at the DataMapper code its a little anti-climactic because it’s really very simple stuff. But then that’s the idea, break a complex operation into smaller pieces so that each one is easy to understand, and if needed, reproduce. To add a new DataMapper we just copy and paste the existing UserMapper and change the field names. Easy. We don’t need to touch any of the code at the SqlDao level that handles getting Single<DTO>, List<DTO>, or DataPage<DTO> return types. We also don’t touch any code that affects any other data shape. All of our code is encapsulated in this one easy to handle class.

Also, keep in mind that the real power of this architecture is that we don’t need to create a type specific DataMapper at all. For the vast majority of tasks, the GenericMapper is going to work just fine. In practice I never create any type-specific DataMappers during development because it seems like at that early stage data shapes are always in flux, and when a data shape changes, the last thing I want to do is go back and update a bunch of mapping code. I look at the type specific DataMappers as something that I never want to use unless an app has gone into production and it’s experiencing performance issues related to the GenericMapper. If that happens then I have a hook in the framework that I can use to create a simple DataMapper for that data shape that contains what I think is the fastest ADO.Net code possible.

Next time we’ll go over the GenericMapper.

AAPL Part 5: How to Write a Data Access Object (DAO)

noreply@blogger.com (Rudy Lacovara) — Mon, 19 Apr 2010 12:32:00 +0000

In this post, we’ll get into what is probably the most useful class in the framework, the Data Access Object (DAO).

Where does the DAO fit in?

Let’s backtrack a bit and take a quick look at the design of core. Below we see a snapshot of Core’s folder structure. We have a Persistence folder that contains our SqlDAO (a DAO written specifically for SqlServer), as well as our DataMappers and Persisters (we’ll go into these later). The Rules folder contains business rules and validation rules for different entities / data shapes. Security has a little plumbing code for business layer authorization. Finally, the Services folder contains the service classes that we’ve focused on during the previous few posts.

The service classes are the one stop shop that our UI code uses to get and save data. A typical data access method looks like this.

public User GetUserByEmail(string email)
{
    string sql = @"SELECT * 
                  FROM [User] 
                  WHERE [Email] = @Email";
    SqlDao dao = SharedSqlDao;
    SqlCommand command = dao.GetSqlCommand(sql);
    command.Parameters.Add(dao.CreateParameter("@Email", email, 100));
    return dao.GetSingle<User>(command);
}

You’ll notice that this is a pretty simple method. There’s no real Ado.Net code. We’re just defining a query, wrapping it in an SqlCommand, and then passing both the SqlCommand and our desired data shape <User> to our DAO. This mystery DAO object does all the heavy lifting for us and even helps us create our query parameter in very efficient, single line of code. This is a very intentional design. After writing data access code for a few years, I finally noticed that the only things I really change from one data access method to the next are: the query; the parameters; and the return data shape. Everything else is just boilerplate plumbing code that gets repeated over and over again. So, I took all that data access boilerplate code and extracted it out to a helper class called SqlDao. The main functions of the DAO are:

The DAO is the single access point to our database. All communication with the database goes through the DAO. No code goes around it.
The DAO encapsulates common ADO.Net tasks like getting a connection, creating query parameters, executing queries, etc.
The DAO defines methods that take an SqlCommand as a parameter, execute it, and return the data in one of our standard formats, a single DTO, as List<DTO>, a DataPage<DTO>, or a string value.

There’s a lot of good stuff in there that makes writing data access method almost frictionless. I can’t go over everything so I included a complete listing for the SqlDao at the end of this post.

The GetSingle<T> and GetList<T> methods

Now let’s look at something a little more interesting. The SqlDao.GetSingle<T> method takes an SqlCommand as a parameter and returns the restults of that command as a single T where T is the class of our desired datashape. So, the line dao.GetSingle<User>(command); will execute command and then return an object of type User. If no User was found then the method just returns null. Here’s how it works.

// GetSingle
public T GetSingle<T>(SqlCommand command) where T : class
{
    T dto = null;
    try
    {
        if (command.Connection.State != ConnectionState.Open)
        {
            command.Connection.Open();
        }                
        SqlDataReader reader = command.ExecuteReader();
        if (reader.HasRows)
        {
            reader.Read();
            IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
            dto = (T)mapper.GetData(reader);
            reader.Close();
        }
    }
    catch (Exception e)
    {
        throw new Exception("Error populating data", e);
    }
    finally
    {
        command.Connection.Close();
    }
    // return the DTO, it's either populated with data or null.
    return dto;
}

The code is pretty simple. We declare a new object of type T called dto (Data Transfer Object) and initialize it to null. We then check the connection on our command and make sure it’s open. We call ExecuteReader on the command which gets us an SqlDataReader. If the reader has any rows then we go to our DataMapperFactory and get an IDataMapper for the type T. A cool thing that is happening behind the scenes is that the DataMapperFactory checks to see if we have an IDataMapper that has been created specifically for type T, if we don’t it returns a GenericMapper that uses reflection to map SqlDataReader fields to fields on our type T. Once we have a mapper it’s just a matter of calling the mapper.GetData(reader) method and returning the dto.

Now let’s take a quick look at GetList<T>. This method is very similar to GetSingle<T>, but it returns a generic list of objects of type T instead of a single object of type T. If no results are returned when we execute the SqlCommand passed to GetList<T>, we don’t return null, instead we return an empty list. Here’s the code.

// GetList
public List<T> GetList<T>(SqlCommand command) where T : class
{
    List<T> dtoList = new List<T>();
    try
    {
        if (command.Connection.State != ConnectionState.Open)
        {
            command.Connection.Open();
        }   
        SqlDataReader reader = command.ExecuteReader();
        if (reader.HasRows)
        {
            IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
            while (reader.Read())
            {
                T dto = null;
                dto = (T)mapper.GetData(reader);
                dtoList.Add(dto);
            }
            reader.Close();
        }
    }
    catch (Exception e)
    {
        throw new Exception("Error populating data", e);
    }
    finally
    {
        command.Connection.Close();
    }
    // We return either the populated list if there was data,
    // or if there was no data we return an empty list.
    return dtoList;
}

It’s really the same as GetSingle<T>. We just have a different return type List<T>, and a while loop that get’s our individual DTOs and adds them to our return dtoList. The main logic for getting an IDataMapper from the DataMapperFactory and calling mapper.GetData(reader) to parse the reader results into an object of type T remains the same.

Conclusion

At this point you may be thinking, “That’s it???, you didn’t answer anything! All you did is get some mapper object and call the GetData method on it.” That’s true to some extent. Our GetSingle<T> and GetList<T> methods just contain the logic that is always going to be common to those tasks regardless of the data shape passed to them. Anything that is data shape specific, things like how do I get data from this field in my SqlDataReader and put it in this field in my returned data shape, are delegated to the IDataMapper. This separation of logic means that it’s very easy to maintain our code. We can add DataMapper logic for new data shapes and never touch any of the logic for GetSingle, GetList, and GetPage. In fact, our logic for a specific data shape is encapsulated in a single IDataMapper class that touches nothing else in the application. That means it’s easy to change and maintenance friendly. Plus, don’t forget that we have the reflection-based GenericMapper, so we never have to create a new IDataMapper if we don’t want to.

Next time we’ll take a look at a data shape specific IDataMapper and the DataMapperFactory. In the meantime, the full source for SqlDao is included below. Have a look at the implementation for GetDataPage<T> and see if it makes sense now that we’ve gone over GetSingle<T> and GetList<T>. The CreateParameter methods are worth a look too. I went with a design where I just overload the method CreateParameter for each type of parameter that I need to create, so the usage syntax for CreateParameter(Guid value) and CreateParameter(int value) are almost identical. Each CreateParameter method was designed with the target of enabling that single line syntax that I use: command.Parameters.Add(dao.CreateParameter("@Email", email, 100)); so each method contains automatic null value checking. You’ll also see some methods that weren’t in my original spec. While using this framework on different projects I’ve found it convenient to have methods like GetSingleInt32(command) and GetStringList(command).

Finally you can download a small, one page sample app that uses aapl at http://aapl.codeplex.com/.

using System;
using System.Linq;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.Data;
using System.Configuration;
using hf.Common;
using hf.Common.DataShapes;
using hf.Core.Persistence.DataMappers;
 
 
namespace hf.Core.Persistence
{
    public class SqlDao
    {
 
        #region "Database Helper Methods"
 
 
        // Connection
        private SqlConnection _sharedConnection;
        public SqlConnection SharedConnection
        {
            get
            {
                if (_sharedConnection == null)
                {
                    _sharedConnection = new SqlConnection(ConfigurationManager.ConnectionStrings["hf"].ConnectionString);
                }
                return _sharedConnection;
            }
            set
            {
                _sharedConnection = value;
            }
        }
 
 
        // Constructors
        public SqlDao() { }
        public SqlDao(SqlConnection connection)
        {
            this.SharedConnection = connection;
        }
 
 
        // GetDbSqlCommand
        public SqlCommand GetSqlCommand(string sqlQuery)
        {
            SqlCommand command = new SqlCommand();
            command.Connection = SharedConnection;
            command.CommandType = CommandType.Text;
            command.CommandText = sqlQuery;
            return command;
        }
 
 
        // GetDbSprocCommand
        public  SqlCommand GetSprocCommand(string sprocName)
        {
            SqlCommand command = new SqlCommand(sprocName);
            command.Connection = SharedConnection;
            command.CommandType = CommandType.StoredProcedure;
            return command;
        }
 
 
        // CreateNullParameter
        public  SqlParameter CreateNullParameter(string name, SqlDbType paramType)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.ParameterName = name;
            parameter.Value = DBNull.Value;
            parameter.Direction = ParameterDirection.Input;
            return parameter;
        }
 
 
        // CreateNullParameter - with size for nvarchars
        public  SqlParameter CreateNullParameter(string name, SqlDbType paramType, int size)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.ParameterName = name;
            parameter.Size = size;
            parameter.Value = DBNull.Value;
            parameter.Direction = ParameterDirection.Input;
            return parameter;
        }
 
 
        // CreateOutputParameter
        public  SqlParameter CreateOutputParameter(string name, SqlDbType paramType)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.ParameterName = name;
            parameter.Direction = ParameterDirection.Output;
            return parameter;
        }
 
 
        // CreateOuputParameter - with size for nvarchars
        public  SqlParameter CreateOutputParameter(string name, SqlDbType paramType, int size)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = paramType;
            parameter.Size = size;
            parameter.ParameterName = name;
            parameter.Direction = ParameterDirection.Output;
            return parameter;
        }
 
 
        // CreateParameter - uniqueidentifier
        public  SqlParameter CreateParameter(string name, Guid value)
        {
            if (value.Equals(NullValues.NullGuid))
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.UniqueIdentifier);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.UniqueIdentifier;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateParameter - int
        public  SqlParameter CreateParameter(string name, int value)
        {
            if (value == NullValues.NullInt)
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.Int);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.Int;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateParameter - bool
        public SqlParameter CreateParameter(string name, bool value)
        {
            SqlParameter parameter = new SqlParameter();
            parameter.SqlDbType = SqlDbType.Bit;
            parameter.ParameterName = name;
            parameter.Value = value ? 1 : 0;
            parameter.Direction = ParameterDirection.Input;
            return parameter;
        }
 
 
        // CreateParameter - datetime
        public  SqlParameter CreateParameter(string name, DateTime value)
        {
            if (value == NullValues.NullDateTime)
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.DateTime);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.DateTime;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateParameter - nvarchar
        public  SqlParameter CreateParameter(string name, string value, int size)
        {
            if (String.IsNullOrEmpty(value))
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.NVarChar);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.NVarChar;
                parameter.Size = size;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
 
        // CreateTextParameter
        public SqlParameter CreateTextParameter(string name, string value)
        {
            if (String.IsNullOrEmpty(value))
            {
                // If value is null then create a null parameter
                return CreateNullParameter(name, SqlDbType.Text);
            }
            else
            {
                SqlParameter parameter = new SqlParameter();
                parameter.SqlDbType = SqlDbType.Text;
                parameter.ParameterName = name;
                parameter.Value = value;
                parameter.Direction = ParameterDirection.Input;
                return parameter;
            }
        }
 
        #endregion
 
 
 
        #region "Data Projection Methods"
 
 
        // ExecuteNonQuery
        public void ExecuteNonQuery(SqlCommand command)
        {
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                command.ExecuteNonQuery();
            }
            catch (Exception e)
            {
                throw new Exception("Error executing query", e);
            }
            finally
            {
                command.Connection.Close();
            }
        }
 
 
        // ExecuteScalar
        public Object ExecuteScalar(SqlCommand command)
        {
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                return command.ExecuteScalar();
            }
            catch (Exception e)
            {
                throw new Exception("Error executing query", e);
            }
            finally
            {
                command.Connection.Close();
            }
        }
 
 
        // GetSingleValue
        public T GetSingleValue<T>(SqlCommand command)
        {
            T returnValue = default(T);
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    if (!reader.IsDBNull(0)) { returnValue = (T)reader[0]; }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnValue;
        }
 
 
        // GetSingleInt32
        public Int32 GetSingleInt32(SqlCommand command)
        {
            Int32 returnValue = default(int);
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    if (!reader.IsDBNull(0)) { returnValue = reader.GetInt32(0); }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnValue;
        }
 
 
        // GetSingleString
        public string GetSingleString(SqlCommand command)
        {
            string returnValue=null; 
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    if (!reader.IsDBNull(0)) { returnValue = reader.GetString(0); }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnValue;
        }
 
 
        // GetGuidList
        public List<Guid> GetGuidList(SqlCommand command)
        {
            List<Guid> returnList = new List<Guid>();
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    returnList = new List<Guid>();
                    while (reader.Read())
                    {
                        if (!reader.IsDBNull(0)) { returnList.Add(reader.GetGuid(0)); }
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnList;
        }
 
 
        // GetStringList
        public List<string> GetStringList(SqlCommand command)
        {
            List<string> returnList = new List<string>();
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    returnList = new List<string>();
                    while (reader.Read())
                    {
                        if (!reader.IsDBNull(0)) { returnList.Add(reader.GetString(0)); }                    
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return returnList;
        }
 
 
        // GetSingle
        public T GetSingle<T>(SqlCommand command) where T : class
        {
            T dto = null;
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }                
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    reader.Read();
                    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
                    dto = (T)mapper.GetData(reader);
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            // return the DTO, it's either populated with data or null.
            return dto;
        }
 
 
        // GetList
        public List<T> GetList<T>(SqlCommand command) where T : class
        {
            List<T> dtoList = new List<T>();
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }   
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
                    while (reader.Read())
                    {
                        T dto = null;
                        dto = (T)mapper.GetData(reader);
                        dtoList.Add(dto);
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            // We return either the populated list if there was data,
            // or if there was no data we return an empty list.
            return dtoList;
        }
 
 
 
 
        // GetDataPage
        public DataPage<T> GetDataPage<T>(SqlCommand command, int pageIndex, int pageSize) where T : class
        {   
            DataPage<T> page = new DataPage<T>();
            page.PageIndex = pageIndex;
            page.PageSize = pageSize;
            try
            {
                if (command.Connection.State != ConnectionState.Open)
                {
                    command.Connection.Open();
                }
                SqlDataReader reader = command.ExecuteReader();
                if (reader.HasRows)
                {
                    IDataMapper mapper = new DataMapperFactory().GetMapper(typeof(T));
                    while (reader.Read())
                    {
                        // get the data for this row
                        T dto = null;
                        dto = (T)mapper.GetData(reader);
                        page.Data.Add(dto);
                        // If we haven't set the RecordCount yet then set it
                        if (page.RecordCount == 0) { page.RecordCount = mapper.GetRecordCount(reader); }
                    }
                    reader.Close();
                }
            }
            catch (Exception e)
            {
                throw new Exception("Error populating data", e);
            }
            finally
            {
                command.Connection.Close();
            }
            return page;
        }
  
        #endregion
 
    }
}

Agile ADO.Net Persistence Layer Download is on CodePlex

noreply@blogger.com (Rudy Lacovara) — Fri, 29 Jan 2010 19:29:00 +0000

I’ve have quite a few requests for sample code so I pulled the AAPL code out of my app, put it in a stripped down ASP.Net MVC sample app and published it on CodePlex. You will need Visual Studio 2008, ASP.Net MVC Framework version 1.0, and Sql Server 2008 to run the sample app. A db backup file is included in the zip and in the source code. You can download the code at:

http://aapl.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=39653

Agile ADO.Net Persistence Layer Part 4: Writing data access for new data shapes

noreply@blogger.com (Rudy Lacovara) — Mon, 25 Jan 2010 16:17:00 +0000

What’s a new data shape?

Has this ever happened to you? You have a persistence architecture that works beautifully. It’s effortless to get and save entities, everything works great. Then you get a requirement for a search results page that will display data from your entity, fine. But then they also want the grid to display fields from another entity entirely. It’s not even in the same aggregate! You find yourself thinking things like what a stupid requirement, why would anyone want to do that. But the real problem isn’t with the requirement. The problem is that your beautiful architecture is inflexible. It works great as long as you’re dealing with nicely segmented entities, but as soon as you need to handle data that is a composite of fields from two different entities, things fall apart. The worst part of it is, in the back of your mind you’re thinking “it would be so easy to just write this in TSQL”.

Another scenario you once again have your beautiful architecture. This time you need to add a couple of fields to an entity. It’s a pretty simple change, but you’re dreading it because you know that in order to make this simple change, first you have to modify all of the CRUD sprocs, then you need to modify the method signatures on every data access method for this entity in the DAL, then you need to modify the parser (or data mapper methods), then you need to modify the BAL methods that call the modified DAL methods, then you need to modify the entity class itself. Once again, the architecture is inflexible. Changes are painful. I call this friction.

Changes like this happen all the time. The majority of our time as developers is spent coding changes to existing systems. Perversely, we usually design architectures to be really easy to use for green field projects, but really hard to use when we need to make changes to those same systems. Making it easy to deal with change was my main objective when coming up with this architecture. In this post I’m going to go over solutions to the two scenarios above using Agile ADO.Net Persistence Layer.

Scenario 1: Adding fields to an existing entity

Just to refresh your memory, our solution looks like the screenshot below. Common contains a DataShapes folder that contains the class definition for every DTO (Data Transfer Object) used in our BAL. For this scenario we’re going to be working with the Category DTO. Our BAL contains a Services folder that contains a service class that provide business logic and persistence logic for each aggregate (that’s service in the DDD sense, not a web service). We’ll be working with the CategoryService.

So, our Category data shape is a straight DTO. It has no methods, just properties. It looks like this.

public class Category
{
    public Guid CategoryGuid { get; set; }
    public string CategoryName { get; set; }
    public string CategoryKey { get; set; }
 
    public Category()
    {
        CategoryGuid = NullValues.NullGuid; 
        CategoryName = NullValues.NullString;
        CategoryKey = NullValues.NullString;
    }
}

Our CategoryService class contains business logic and data access methods. Each data access method does three things, it defines a query, it selects a data shape to return, then it passes both the query and data shape to our DAO (Data Access Object) which executes the query and maps the results to the data shape. Our data access methods in the Category Service look like this. You can see both methods returns a List<Category>.

public List<Category> GetAllCategories
{
      string query = @"SELECT * 
                    FROM Category 
                    ORDER BY CategoryName";
      SqlDao dao = SharedSqlDao;
      SqlCommand command = dao.GetSqlCommand(query);
      return dao.GetList<Category>(command);
}
 
 
public List<Category> GetAllCategoriesInUse
{
     string query = @"SELECT DISTINCT c.*
                    FROM Category c
                    JOIN PostCategory p on p.CategoryGuid=c.CategoryGuid
                    ORDER BY CategoryName";
     SqlDao dao = SharedSqlDao;
     SqlCommand command = dao.GetSqlCommand(query);
     return dao.GetList<Category>(command);
}

Now for our simple change. We forgot to add a CreatedUtc field to our Category entity to store the date when a category was created. Crap. We already have a bunch of code written that uses these classes. How hard is it going to be to make this change without breaking any of our data access methods and UI code. Well the first step is obvious, we need to add CreatedUtc to the database table. So we do that. Now for the tough part. Do we need to modify any sprocs? No. Do we need to modify any DAL methods? No. How about the data access methods in our service class, we need to change those right? Nope. The queries defined in the data access methods use “SELECT *” so they’ll pick up any new fields added to the Category table. How about the DAO, surely we need to change some mapping code there? Nope, it uses a little reflection to automatically map query results to a data shape based on field name. As long as we use CateogryUtc for both our database column name and our Category DTO field name, no code changes are needed. So what do we need to do?? Just one thing, we need to add CreatedUtc to our Category data shape like this.

public class Category

{

public Guid CategoryGuid { get; set; }

public DateTime CreatedUtc { get; set; }

public string CategoryName { get; set; }

public string CategoryKey { get; set; }

public Category()

{

CategoryGuid = NullValues.NullGuid;

CreatedUtc = NullValues.NullDateTime;

CategoryName = NullValues.NullString;

CategoryKey = NullValues.NullString;

}

Done. You have to admit, even for a simple change, that was pretty easy. Just add the new field to the Category DTO, add the new column to the table, and the persistence layer handles the rest. Now I know some of you are thinking “But what if I don’t want to use the same names for my columns and DTO fields” or “No way would I ever use reflection in my DAL, it’s too slow”. Those are valid concerns, and I have an easy way to tune the DAO to address both of those issues, but the nice part is that if you’re willing to work within these constraints, your development can go quickly and with very little friction.

Scenario 2: Adding a new data shape

Now let’s look at the case where we need a new composite data shape that consists of data from two different aggregates. Our site has Categories, Blogs, and BlogPosts. The customer tells us that we need to display a list of the the most popular categories along with a count of how many BlogPosts are in each Category. This doesn’t really fit with how we did our initial modeling for this project. A BlogPost contains a collection of Categories not the other way around. Still, the data model will support it, we just have a simple join table between the Category table and the BlogPost table. There’s no reason we can’t write a query that uses the relationship in the other direction.

So, our solution is to add a new data shape called CategoryWithPostCount that will contain the category data plus a count of blog posts that are using that category. Because we’re using an architecture that anticipates and embraces change, adding this new data shape will be a simple 2 step process. First we create the CategoryWithPostCount DTO and add it to our Common/DataShapes folder.

public class CategoryWithPostCount
    {
        public Guid CategoryGuid { get; set; }
        public string CategoryName { get; set; }
        public string CategoryKey { get; set; }
        public int PostCount { get; set; }
 
        public CategoryWithPostCount()
        {
            CategoryGuid = NullValues.NullGuid; 
            CategoryName = NullValues.NullString;
            CategoryKey = NullValues.NullString;
            PostCount = NullValues.NullInt;
        }
    }

Second, we open up our CategoryService class and add a new GetTopCategoryList data access method. Remember that our framework does most of the heavy lifting for us. All we need to do is define the query, pick a data shape, and then pass both to the DAO which will automatically map them by field name. Writing the method is easy. We fire up Sql Server Management Studio, write and test our query, copy one of the other data access methods, and paste our new query into it. Here’s the result.

public List<CategoryWithPostCount> GetTopCategoryList(int pageSize)
{
    string query = string.Format(
                    @"SELECT TOP {0} c.*, q1.PostCount
                    FROM Category c
                    JOIN (
                        SELECT c.CategoryGuid, count(pc.PostGuid) as PostCount
                        FROM PostCategory pc 
                        JOIN Category c ON c.CategoryGuid = pc.CategoryGuid 
                        Group By c.CategoryGuid
                        ) q1 ON q1.CategoryGuid = c.CategoryGuid
                    ORDER BY q1.PostCount DESC
                    ", pageSize);
    SqlDao dao = SharedSqlDao;
    SqlCommand command = dao.GetSqlCommand(query);
    return dao.GetList<CategoryWithPostCount>(command);
}

That’s it, we’re done. A brand new data shape complete with data access method ready to use in minutes. When it’s this easy to add new data shapes, it really frees you up to work in a much more agile way. You don’t need to worry about getting entity classes designed exactly right the first time, or missing pieces of data like a count or date created. If you do, the architecture is designed to make it painless to add that stuff as it’s needed. And none of this is hard, none of it is magic, we’ve just made some decisions and accepted some limitations (like DTO field names and table column names will be the same) that allow us to take the friction out of design changes.

Next time we’ll get into the DOA and the associated DataMapper classes.