Jonathan Crossland's Talkware

First Look at VS.NET 2010 (for me)

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 06 Nov 2009 02:09:00 -0800

I don't have time for a large introduction to this post, rather I'll jump straight in and tell you about the things that I picked up on.

- The start page, with its now big button, lots of padding approach, seem wasted space, especially on any wide screen monitors. the start page for VS.NET 2008 wasn't anything great, but you had the recent projects, some news and other bits all accessible with one click. A minor thing, but its not a better start page.

- The skinned look. This was a complete waste of time, in my opinion and in fact makes the experience in visual studio worse. With the extra padding, now around all the toolbars (the same width as the splitters), it makes everything seem smaller, and it is. The dragging of the toolbars are much cleaner, smoother than 2008.

I don't like the way the treeview lines and arrows disappear when it has no focus. If it is a setting, I could not find it, but now I cannot at a glance see if a folder has something in it vs one that does. A small point, but means you have to move the mouse and wait for the lines to fade in.

- The code editor now has filtering on what you type in the intelli-sense. Before it simply took you to the correct item in the intelli-sense window, now it removes entries that do not match. This is a massive negative point, as I use the intelli-sense as a browser as well.

- The code editor, firstly which is a major plus, is the selection highlight. Now it is not an inversion of text colour, with blue selection background, it is a translucent colour, which keeps the text colour as it is, untouched. This simple thing is a massive improvement.

- Unfortunately there is a microsecond delay now over 2008 when typing. Each character you type appears that little more slowly, which actually took me a while to get over. It was actually annoying to me. I know I am quite sensitive about it, but I think others will also have picked up on it.

- The skin colour of dark blue with white text on the tabs, now distract me. I always have a black windowed code editor, not the usual white. But now, the toolbar tabs for each file is more visible in my peripheral vision as I type in the editor. This is also distracting.

- The round break point icons in the margin is now that fraction smaller, which I don't like, but can live with.

- Overall, the skin has introduced too much contrast. It is much more stand out as an interface, which is better to look at, but worse to code in.

Why does Visual Basic appear on the root, and c# in 'Other Languages' - on new Project Dialog? Is this a VB editor, then if we are lucky some other languages? I thought .NET was about multi-language.
1. Visual Basic
2. Other Languages
3. Other project Types
4. Test projects

The Agile pendulum has swayed too far

jonathan@lucidocean.com (Jonathan Crossland) — Mon, 19 Oct 2009 09:12:00 -0800

Being an agile developer has meant a lot to me. Over the years, I have tried to get those around me, more agile, more iteratively focused, more quality, result driven and so on, but I fear it is growing askew.

Agile is on the cusp of becoming so customer, service and result focused that quality, reuse, robustness and many other facets are getting less emphasis.

Agility can be a weakness too.

What is Production Code and how do

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 11 Sep 2009 08:36:00 -0800

... you know when you have it, or not?

You see it written in code samples: No warranty, please take care if you use in production environment. This is not production code, but what does production code really mean?

Firstly production code can be any code that is in use. But that does not refer anything about quality. Just because software is on the shelf, does not mean its good quality.
Production code must have some sort of quality level.

Amusingly the DSDM pocket book, (Atern) says that "it must be good enough" - which makes me laugh, then cry.

Quality is subjective yes, and quality costs money. So what should production code be?

The following is a list of heuristics for referring to code as production code.

More than 40% of your application has Unit Tests
At least 3 or more people have spent time actively seeking bugs, for longer than 30% of the time it took to originally write.
At least 3 or more people have been actively seeking bugs, in at least 5 different configurations/environments.You have mixed the environment (different server, different database name, altered culture/region, etc).
Bugs have been solved with SolvingDesign (A lasting solve)
Code Regression is reduced, practically to a stand-still. A ratio of 50 bugs solved per 1 reintroduced is a good ratio. 5 bugs solved for 1 reintroduced is a terrible ratio.
Users have used the code for at least 40% of the development time taken, reporting bugs, and the system recompiled and redeployed at 5 times during this period. (preferably at intervals, so input can be recalculated)

Influence that will present itself, and cause you into thinking its production code, before its not.

Deadline and other pressures
Reactionary mindset, rather than pre-emptive
Bad company software creation intelligence (what else can you call it?)
Human traits, inexperience, carelessness etc

An ASP.NET web root link handler

jonathan@lucidocean.com (Jonathan Crossland) — Thu, 20 Aug 2009 02:56:00 -0800

In order to get a useful, all purpose, all use client-side href link back to root, from anywhere down the web folder structure, and paying attention to IIS6, IIS7, Cassini and other Visual Studio internal web hosting scenarious and of course the odd bug within ASP.NET and your own web site code, I created a handler for ~/.

Here is the html I wanted.


<a href="~/">Some link to root of site</a>

public class RootHandler : IHttpHandler
    {
        #region IHttpHandler Members

        public bool IsReusable
        {
            get { return true; }
        }

        public void ProcessRequest(HttpContext context)
        {
            context.Response.Clear();
            string toUrl = context.Request.ApplicationPath;
            context.Response.Redirect(toUrl);
            context.Response.End();
        }

        #endregion
    }

    <handlers>
      <remove name="root"/>
      <add name="root" verb="*" path="~/" type="Namespace.RootHandler, Assembly"/>   
    </handlers>

Then with a little more code in the ProcessRequest method, you could find the ~/ part of the url, strip it down and modify the url.


        public void ProcessRequest(HttpContext context)
        {
            context.Response.Clear();
            
            string toUrl = context.Request.ApplicationPath + "/";
            string url = context.Request.RawUrl;

            int idx = url.LastIndexOf("~/");

            url = url.Substring(idx + 2);
            
            url = VirtualPathUtility.Combine(toUrl, url);

            context.Response.Redirect(url);
            context.Response.End();
        }

It is only with .NET Framework 3.5, where HttpHandlers have gotten to an acceptable bug free level for use.

Warning: this is a simplistic example for purpose of this post, please be careful in production scenarios.

ASP.NET Framework Should Haves

jonathan@lucidocean.com (Jonathan Crossland) — Thu, 23 Jul 2009 05:53:00 -0800

ASP.NET should have been a plug-in from the start. The IIS Handler, should have been split into (.NET with ASP.NET) and (.NET without ASP.NET). The idea of a config file and a WebConfigurationManager is not and should not have been specific to ASP.NET.

One cannot help but feel that ASP.NET was designed with Web Server Controls as the flagship feature. "Create that 3rd party market for web". No real thought paid into how people actually use the web on simple terms.

What I am writing here, probably resembled the paper David Heinemeier Hansson scribbled on when thinking and designing Ruby on Rails, but I have no intention of mapping features, just stating the obvious.

Get Server returns html
Post Server *saves data and returns html
Querystring may change what code is executed, and change the "context"
Url may require changing, remapping, extension changes etc

Now, to do this in ASP.NET is relatively straight forward?
Yes it is, simple enough to do each thing, but it gets more complicated when you want all these ideas together.

There is also a grey area as to what we should call easy. Yes it's easy to do x, but is it simple and will it remain simple as time goes by.
The minute you start asking some questions, ASP.NET starts to give varying levels of success. What if it was posted from a different server? What if the html had to be static and not dynamic? what if you need to create your own physical files and return those to the browser? What if you did not want an aspx page at all? What if your new Module wants to create the Page (HttpHandler)

Asp.NET is far too cluttered, and muddled. The design from, Web Server to a response, should have been completely extensible and pluggable.

The problems that I have come across, over the last few months (again), have all been relating to ASP.NET not being able to let go. ASP.NET has so many tenuous links to what you write, its everything but impossible to break away.

A framework should always be designed in layers. A framework should not be a large foundational block, written to sit at the bottom only.

if I am not entirely clear, perhaps a few links into oddities will help.
Actionless Form
Scenarios where switching viewstate off like this does not actually work.
and why caching problems like this does not work properly all the time.
Double postback scenarios
The ridiculous IsPostBack code and scenarios
and it does not end. Friendly adaptors, bad html to asp.net naming, lack of rudimentary support for url manipulation, deflected posts and more, makes ASP.NET a framework look like this.

In my humble opinion, on design/framework notions here are some ASP.NET should haves.

The HttpRuntime and associated server objects should have been better formulated and allow for better involvement. Allow extensibility.
The runtime should have made Pages (HttpHandler) an optional unit of packaging and code, not the complete target.
Better fine grained Page type. Split it into more pieces, separating cache, viewstate and more from the design. So we can use at any of the levels of interest.
HttpHandlers should have been designed as a clear open and extensible list of handlers. The pattern should be more workflow like.
The compilation process of asp.net syntax and html should have been an exposed set of objects, also extensible. (build providers)
Expressions should have been designed much better, and give us control of more of the syntax.
It should have been easy to create an aspx file and not have code behind, and include your own syntactical sugar.
Asp.NET should have been a plug-in to the .net engine for the web, and not soo tightly coupled.
Classic ASP should have been a continued 'available' and compatible unit within the framework.
Should have reduced the gap between Server and Client side code. not server controls writing client code out, but rather a whole new better solution
All that time it took to develop usercontrols and pages and server controls, could have been better spent on using existing syntax and an html object model for controls. It would have been compliant, less error prone and significantly better.

Personally, I cannot wait for the rewrite of ASP.NET. Unfortunately with all the man hours, money and face they have put into ASP.NET - they wont be brave to throw it away. Yes folks you will be stuck with it, and its even going to version 4.

I like the .NET framework as a whole, I love the c# language, but ASP.NET is a beast. Why on earth they built MVC on top, I'll never know. If it was not for Visual Studio supporting ASP.NET, it would be a disaster.

ASP.NET just should not have!

How I fight with ASP.NET as a Framework

jonathan@lucidocean.com (Jonathan Crossland) — Wed, 08 Jul 2009 07:57:00 -0800

Bad, bad, Bad..

I have soo much to write about this topic, I think I will leave it there....

What I am building now

jonathan@lucidocean.com (Jonathan Crossland) — Tue, 23 Jun 2009 06:33:00 -0800

I have to build Forms. Web Forms, digital versions of paper forms. You know the kind. You get probably hate filling them in.
Currently there are a few forms, but expectations are, that many, many more forms are to follow. Forms for Edit, Forms with Permissions (part of forms visible to some, editable by others), forms saved as pdf, as html. Each form can expect largely varying workflows. Data from forms to varying database schemas. A complete nightmare to implement generically, but there is a need for a lot of it to sit on top of something familiar and useful across domains and clients.
Some of the goals include:

- easier to maintain, add forms etc, than previous implementations.
- easier to edit and deploy to different sites.
- customizable workflow
- customizable processing.
- Notifications, manual override of process, some manual flow points.

This is kind of like ruby on rails meets Windows Workflow foundation. But alas that wont work out very well, as a combined technology and WCF is overkill in some areas, and might be worth doing on another level. However this is a .NET team, with no WCF experience. Would a quick Activity based object model not suffice?

I have been sitting with this for a few weeks now, and I have 90% of my plan of attack. However, I have started developing already. You guessed it, I have already built a small Framework of my own that will suit this development.
If you have bright ideas, let me know.

Lack of MetaData on code and .NET Framework libraries

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 22 May 2009 04:59:00 -0800

Although .NET Reflecton offers a decent look at the Types you build, it is pretty generic in its offering. For example, it does not yet provide us with capabilities to control and secure how Reflection can inspect our Types. If you do not want a Type to be Instantiated via Reflection, or if you would like to restrict certain Types from being visible at all. Further there is no idea for extension or change. One cannot simply plug into Reflection and override behaviour.

High abstractions and higher flexibility

Many more wrappers are needed, at a higher level, which would sit on top of .NET Reflection. For example, architectural dependencies. Agile mechanisms, if code had enough metadata attached, one could alter the code via reflection/refactoring in a much more dynamic way. Consider the following two web service methods in two different companies.

  public class PersonService
    {
        public void AddPerson(string lastname, string firstname, 
            string birthdate)
        {
        }
    }

    public class PersonWebService
    {
        public void AddNew(string firstname, string surname)
        {
        }
    }

You can see that they are completely different contracts. Interface and contract speaking they are like apples and sand. However, to your mapping, calculating, schema, metadata, tagging and understanding brain, its the same thing.

What we need, is to remove ourselves from Contracts to something more like guidelines.

  public class PersonService
    {
        [Operation(Schema:Person, "Add")]
        public void AddPerson([schema(LastName)]string lastname, 
             [schema(FirstName)]string firstname, 
             [schema(DateOfBirth)] string birthdate)
        {
        }
    }

    public class PersonWebService
    {
        [Operation(Schema:Person, "Add")]
        public void AddNew([schema(FirstName)]string firstname, 
             [schema(LastName)]string surname)
        {
        }
    }

Reverse Composite

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 22 May 2009 04:19:00 -0800

Not sure if there is a Design Pattern relating to this kind of need in another pattern catalogue, but it seemed simple enough to reverse the GOF composite. A class with a collection of leaves, that iterate, recurse its way up to the root. This needs work in terms of combining or actually making a variation on the composite. I think that this is perhaps only a .1 degree of separation.

Problem

You have a list of Items, that may or may not be related. There may be 1 or more trees. You have a bag of unsorted or related objects as a signle array. You need to maintain a complete list of leaves, but also need the relationship/heirarchy to become apparent. You want to express the heirarchy in a Composite Pattern, but it would exclude certain leaves has it has no part in the particular root that the composite gets built on.

Context

You need a collection of leaves, from across the composite tree. You need to have access to the parent and/or recurse to the root.

Types are stripped from its branches.

Structure

A class that works with a collection of leaf classes, that each point to it's parent. In effect a composite in reverse. Keep all the leaf objects in a collection and build the heirarchy from the bottom up.

One of the main reasons behind this is multiple paths to a set endpoint. The root becomes the set endpoint where by we can plot a course from a leaf. The Leaf though is where we start from and full knowledge of all leaves should be known at the outset.

By adding a Parent on the Composite (GoF), you can achieve a similar result, but you will not have a flat list of leaves.

VS.NET Add-ins worth looking at

jonathan@lucidocean.com (Jonathan Crossland) — Wed, 13 May 2009 04:53:00 -0800

Run VS.NET as Administrator on Vista.

Export your settings using Tools | Import and Export Settings, don't let another windows reinstall take it all away from you.

Get Regionerate
- but make sure you use the "Remove All regions" command, before trying to apply others. It wont replace your existing regions.

Get Studio Tools, for File manager and the tear off editor.

Download Dev /efor - not sure about the name, but it works nicely.

Snippet Editor, useful, for changing the way the existing snippets work, as well as new ones.

Carl J has a large list of add-ins.

Why I use Twitter

jonathan@lucidocean.com (Jonathan Crossland) — Mon, 11 May 2009 04:51:00 -0800

It is amazing how these sorts of tools divide opinion.
I usually blog about technology, especially framework oriented things, but I have been thinking a lot about business again. How can it work for business.

For example, a company I have here in my midst, Oxford Creative, arose from building sites for the car manufacturer Seat. (They are searching for clients, so get on board quickly, as they are a small but effective team.)

I have been talking about twitter and many of my friends do not see the point.

But twitter, is less evasive, more rapid, more dynamic than any other social media tool.
I do not get hounded by it. I switch it off, its off. I switch it on, I have access to some great minds.

I found this How To Use Twitter Effectively for Business and Advocacy, which says a lot about its power, amongst many other posts, which have been floating around twittersphere.

I use it, because I can meet some interesting people, like @unclebobmartin, and @kentbeck, @bigballofmudd (Brian Foote). I like twitter because its less intrusive than phones, emails and facebook, but its powerful enough to communicate.

From Software Interfaces to

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 08 May 2009 08:59:00 -0800

Eventually Interfaces will be replaced with magnets!

The Every Project Framework - Part 2

jonathan@lucidocean.com (Jonathan Crossland) — Wed, 06 May 2009 05:25:00 -0800

In The Every Project Framework - Part 1, I spoke of partitioning, and abstracting structure and infrastructure. Now Let's talk about how we do that.

Over the last decade, the most prolific and available Architectural Pattern introduced and adopted by business developers is the Layers pattern. Call it by another name, but the concept is around everywhere. Most enterprise systems have open or closed Layers. They have many names, but Data Access Layer, Business Layer and so on, are often used. This a good thing towards raising the level of abstraction, increasing the infrastructure and providing more structure. However, having layers, does not automatically mean your application is on a good footing.

Raising the bar

If you are very skeptical, read the 'Why' section below first.

What do you think of your objects? How do you think of them? Are they transportation, containers, db table wrappers or other? Majority of business applications I see, have objects as a 1 to 1 mapping to a database table and it effectively is a container for a row of data. Some then wrap it into a collection of these rows and you have a table. Is that really what you need? is that efficient and good practice? Does it raise the bar high enough?

The first thing to say is that wrapping your table into an object is not entirely bad, as it is a level of abstraction away from the table.
However, if its at the highest level of abstraction you go to, then you will still have miles of code that you need to place elsewhere. Code such as Context, Security, Permissions, work flow and so on. If you manage all of your Types as a 1 to 1 mapping to the database tables, are you not merely writing code against the database?

If its UI (form) accessing and using -> the DataType Person, then your level of abstraction to the data is almost zero. You have abstracted the concept of a database table and replaced it with a more crude version, which you are populating.

Then the relationships and structure between all these singular Types will slowly come together within your UI, behind your button, in some method of some obscure class. Instead, the singular classes should be a little more structured and the bar should be raised higher, providing your UI with more closer, ready and easier to use objects.

To repeat in more a programmatic way, consider a table named Person.

If you have your UI --> Types --> Table

effectively you have abstracted a Row and Column concept away from the UI and replaced it with a PersonType and a PersonTypeCollection.
That means the UI, must understand how to deal with these Types.

To raise the bar, you would have more abstractions between UI and database
perhaps more like so

UI --> PersonCompositeType --> transaction(PersonDataType1/PersonDataType2) --> Table1/Table2

So PersonCompositeType is at much high level.

Please note:I am not saying that your architecture must look exactly like this, what I am suggesting is that more levels of abstraction can be achieved between the UI and your 1 to 1 mapped Type, that is so commonly found.

Why?

Have you ever had the following in your last or current project?

The UI is a lot more work than writing the "engine" or "business layers"!
Well of course it is, because the level of abstraction for the business layer is so low.

Your project starts quickly, progress seems to be made at speed, and then it slows right, right down....
It would, because, laying out database tables, getting your SQL and business objects in a 1 to 1 mapping is easy. Tying these low level abstraction together into a working application is harder.

Are you using a workflow engine? Are you using a business rules engine? Do you have isolated, stateless services? Do you have wrappers, commands, actions?
If not, then all the workflow, business rules, services are tucked away, most likely in your user interface. If you don't think you need these things, your large projects are most likely failing, running late and/or getting harder and harder to maintain.

Closing thoughts

If you think in "the ways of frameworks". Not as a framework that will be released to the world. Not as a time consuming lets focus on the framework, but rather build it, partition it and expand it within the project, your application will benefit from having more structure and more infrastructure.

Even having one of those is vastly beneficial. That is why a company with their own libraries (infrastructure), can get an application up quicker, even if its not a framework. One or both infrastructure and structure is vital for reuse and time to market. Yet, there is very little training going on at university, or courses in this area.

We should be training developers from their earliest years in software to think about reuse, to understand patterns, and putting rudimentary frameworks together.

Don't let the fact, that there are no perfect frameworks out there, lure you into thinking frameworks are bad. They are just diamonds in the rough. Most importantly, the "good" aspects of frameworks can be used within your application to high rewards. But how do you do it? -- stay tuned...

The Every Project Framework - Part 1

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 01 May 2009 07:11:00 -0800

In The Every Project Framework - Introduction, I spoke of structure and infrastructure, lets now move a little deeper.
[more]

Structure is just like Pronutro

When you create an Application, you are immediately adding complexity and dependencies. How well you manage these is how long your application stays 'clean' or 'partitioned', but nevertheless your Application is getting structure with each method calling another, with each class depending on another.

If you don't think about a class, its name, where it should reside, or if you choose incorrectly it affects everything to come. You may not be aware of it entirely, but every decision has a consequence. Perhaps in the future we will have tools to help us deal with 10 or more steps of future planning, so we can verify our decisions a little better, but for now, we only rely on experience and intuition. (Like the movie Next with Nicholas Cage)

btw. Pronutro is a black hole for milk!

We have all heard of 'spaghetti code', but code does not need to be that bad, before it can start becoming a strain to continue with. Just one configuration file setting in 1 tiny config file, can cause masses of damage if its not there or if its incorrect. Yet for the majority of the time, developers put things into configs and rely on the fact that its there.

Convention over Configuration should be adhered to first as a rule.

Partitions
One of the most important aspects for agility within any structure is partitioning. Dividing lines between x and y. Effectively its all about low coupling. Read Grasp patterns regarding Cohesion and Coupling. But also how clean are the partitions? A clean break is when indirectly what you rely on works even if its not available. For example, I used TidyHtml to neaten some html before parsing it. If the TidyATL.dll is not registered in the registry, do I die a painful death or do i simply use the html and proceed to the next step without tidy? How you design and implement code with that sort of thought is what makes it a clean partition. Unfortunately at the extreme ends of things, our tools make this tremendously time consuming to get right for all scenarios. It is easy for example using Tidy, but a lot harder for most critical aspects of an application.

Giving Structure is a fundamental part, but below it on the "layers diagram", is the infrastructure.

Raising the Infrastructure

Today it is pretty simple to use your current modern language to create a simple calculator. the level of abstraction is almost perfectly aligned to the requirements of the small Application. For example, the textbox in .NET is perfectly sufficient, and the Types float, decimal, double, int etc are all at the 'right level' for a calculator.
However its not at the right level for a Stock Management Application or other more complex business software. This is where we need to raise our infrastructure to a more suitable level.
And most of us do, we create Loggers, Data layers and so on, to aid us in reaching the Application from a higher vantage point. But are you properly aware of what is structure and what is infrastructure? And are you raising the level of abstraction high enough?

Next post, I will discuss these levels of abstraction, Architectural Design Patterns such as Layer, from the strict vantage point of structure and infrastructure.

Generic Object Suffixes for Naming your Classes

jonathan@lucidocean.com (Jonathan Crossland) — Thu, 30 Apr 2009 05:03:00 -0800

When creating a large project, it is sometimes extremely hard to find names for certain things. So I have a small list. I mix and match ActivityItem, PropertyItem, ElementFactory and so on. Helps with those generic classes you cant find names for!
you probably already using most of them, if not all ..... (got more for me?)
[more]

Single Item Activity Action Command Item Element Base Unit Node Property Column Row Value	Arrays of Items Array Composite Collection List Set
Contextual Info Context State Status Format Mode Level Type Scope	Controllers/Managers/Services Runtime Environment Executor Handler Engine System Controller Provider Publisher Designer Manager Factory Builder Converter Inspector Service Utility Helper Message Process Thread Reader Writer Serializer Action Command
Extension Adaptor Extension Facade Proxy Wrapper Dynamic Static
Configuration Attribute Configuration Setting	User Interface Style View Component Control Editor Widget

The Every Project Framework - Introduction

jonathan@lucidocean.com (Jonathan Crossland) — Mon, 27 Apr 2009 03:55:00 -0800

Do you think of a Framework as a large 'hole, to be avoided'? Are they too complicated to develop within your project, for everyday development?
I take the view, that a Framework can very easily be part of every project you build, and with great benefit. You just have to understand how.

A Framework has two very important characteristics to give to Applications. A Framework provides structure and infrastructure.

[more]

Structure

An application without structure is just text. An Application without structure contains no discernible patterns. If your Application contains no patterns, it becomes complicated. If you have structure and patterns, you have organization, categorization and commonality. An Application must have structure and I know that every single Application ever written has a structure. Every system ever built has patterns. Even the smallest console driven application will have a set way of dealing with arguments, and other code structure.

As developers we are inclined to think in patterns, that is why Design Patterns are familiar and majority of developers can agree that whether they use a Pattern from a book or their very own Design Patterns, a Pattern is a valid helpful tool.

Infrastructure

An Application that does not have infrastructure, is voltage. All Applications sit on levels of abstraction, whether its the assembly language, a runtime, programming libraries and/API, all Applications rely on infrastructure. The infrastructure of a given Application differs between projects, because as developers we can develop our own levels of abstraction, or rely only on those that are provided, or mostly somewhere in between. Even the smallest application will have some sort of API, or common shared library. We rely on the infrastructure provided at the bottom and the layers of abstraction all the way to our user interface.

Structure and Infrastructure are at the heart of Frameworks. Yes, a Framework can be other things as well, but at its core, a Framework offers structure and infrastructure to an Application.

Within all Applications...

there lies a Framework.

The developers may not have consciously created it, nor was it perhaps designed, but its there. It may be difficult to see, dirty, mixed up within all your code, but it is there!. The trick is to identify it and evolve it as you develop your Application. When you can separate the Application from the Framework (even if the framework is small), means you have separated along the two large dividing lines of change that will occur in your Application. Even if your Framework is not perfect and cannot be used outside of the current project, having it separate aids in the agility and life-time of the Application.

It is a craft, do not expect to build reusable complete Frameworks the first time, but as you build and think in the language of Frameworks, your skills will improve and it will become easier and easier.

"a class is to object, as a Framework is to Application"

Refactoring complex classes using Composition Part 3

jonathan@lucidocean.com (Jonathan Crossland) — Mon, 20 Apr 2009 02:55:00 -0800

To conclude the previous two posts on using composition for refactoring complex classes

Refactoring complex classes using Composition Part 1
and Refactoring complex classes using Composition Part 2

Why should complex classes be refactored? To make code smaller and more manageable. To separate code that changes at different rates, and which are not concerned with the same things.

Composition vs Inheritance? Composition = "has a" relationship, Inheritance "is a" relationship. Inheritance quickly makes you feel like you create Object-Oriented code, but its one of those devices that should be used for a real reason, not just off the cuff.

Reading
- Some background from C2 Wiki - Composition Instead Of Inheritance
- A good principle from Evolve aggregations from inheritance heirarchies - Brian Foote and William F. Opdyke
- Look at Adaptor, Facade, Proxy from Design Patterns (GoF) and also the Stategy, which is mentioned here by Erich Gamma A Conversation with Erich Gamma, Part III
- Refactoring Catalog - Martin Fowler
- Refactoring posts - Martin Fowler

Refactoring complex classes using Composition Part 2

jonathan@lucidocean.com (Jonathan Crossland) — Thu, 09 Apr 2009 04:07:00 -0800

Please Note: Please make sure you have proper unit test coverage on your classes, as you refactor.

From Part 1, I showed you how I initially split a large class into a few more, splitting the complexity of having the code in one place. However, you will still find that for a really complex class, there will still be quite a lot to do. Here are a few more composition based things to consider for continuing the refactoring process.

[more]

Extract surrounding Exceptions

You will have to decide where you do this carefully, according to your own code, but I try to pull out the larger surrounding Try/Catch blocks into the wrapper, like Start method below. I now can remove the code from RuntimeExecutor for exceptions on as many methods as possible.

There may well be places you do not want to do this, but if your RuntimeExecutor class is private and only used by this wrapper, you should be able to remove most Exception handling in the wrapped class.
However, if the class is complex in this area, you may need an Exception Handling strategy instead. If your wrapped class, throws a complex amount of Exceptions in weird and wonderful areas within the code, you will have to think about it harder, but for most classes you can get away with extracting it.

//abbreviated version of the class.
    public class Runtime : IActionExecutor
    {
        public void Start()
        {
            try
            {
                _Executor.Start();
            }
            catch (Exception ex)
            {
                LastException = new RuntimeException("Execute Failed. See inner exception for details.", ex);
                Status = RuntimeStatus.Errored;
            }
        }

        public RuntimeException LastException
        {
            get
            {
                return _Exception;
            }
            protected set
            {
                _Exception = value;
            }
        }
    }

Splitting by lifespan and age

Think of your class in three main stages, Constructing, Running, Deconstructing and refactor based on it. For example, move Initialization code from RuntimeExecutor to Runtime. My class has an Initialize method and some code in its constructor. I now refactor this into the wrapper Runtime. I also move initilization checks to the wrapper.

    //abbreviated version of the class.
    public class Runtime : IActionExecutor
    {
        public void Start()
        {
            try
            {
                if (_IsInitialized)
                    _Executor.Start();
            }
            catch (Exception ex)
            {
                LastException = new RuntimeException("Execute Failed. See inner exception for details.", ex);
                Status = RuntimeStatus.Errored;
            }
        }

        private void Initialize()
        {
            try
            {
                //Initialization code, sets RuntimeState (see Part 1)
                _IsInitialized = true;
            }
            catch (Exception ex)
            {
                LastException = new RuntimeException("Execute Failed. See inner exception for details.", ex);
                Status = RuntimeStatus.Errored;
            }
        }
      
    }

I then do the same with any deconstruction code, so that they too exist in the Runtime class, and extracted from the RuntimeExecutor

Conclusion

The Utility class we created to move those more stateless methods into, can be expanded a little more, by moving even more methods from the main class into it. If you need to, you can even pass the RuntimeState instance via the constructor through to the class, so it can now have state. This will allow you to move even more methods into the Utility class. Remember this class is private and owned by the RuntimeExecutor. If some of these methods are more generic than that, you need to move it out into other classes.

We created four classes from one bulky one. Here is an example in code, of what we did.

 /// 
    /// We refactored UP, and created a Wrapper
    /// We moved wrapper methods, exceptions to this class
    /// 
    public class MyTypeWrapper
    {
        MyType _MyType;
        MyTypeState _MyState;

        public MyTypeWrapper()
        {
            _MyState = new MyTypeState();
            _MyType = new MyType(_MyState);
        }
    }

    /// 
    /// This is the original Bulky Class
    /// Some of its members moved UP (Wrapper)
    /// 
    internal class MyType
    {
        MyTypeState _MyState; //all fields now wrapped in MyTypeState
        MyTypeUtility _MyUtility;

        public MyType(MyTypeState myState)
        {
            _MyState = myState;
            _MyUtility = new MyTypeUtility();

        }
    }

    /// 
    /// All the fields moved from MyType to here
    /// 
    internal class MyTypeState
    {
        public MyTypeState()
        {
        }
    }

    /// 
    /// Methods moved here are stateless
    /// and/or if you add MyState, it can also be stateful.
    /// 
    internal class MyTypeUtility
    {
        public MyTypeUtility()
        {
        }
    }

Cleaning up

You now need to make sure that all is tidy. I usually do a mixture of the following.

Rename methods to suit the new classes
Split large methods at least into two
Rename classes, to make sure they make sense
Make RuntimeState, RuntimeExecutor classes private and check every method is private if its not called outside of its surroundings. If it is, I attempt to move the method to the utility class.
See if the Interface we made in Part 1, is still required (most of the time it wont be, only useful for refactoring)
If there are still large classes, I attempt it all from the start (Part 1)

Refactoring complex classes using Composition Part 1

jonathan@lucidocean.com (Jonathan Crossland) — Wed, 08 Apr 2009 04:09:00 -0800

Please Note: Please make sure you have proper unit test coverage on your classes, as you refactor.

Using Design Patterns which deal with compositional power, such as Wrapper, Facade, Adaptor, Bridge, Proxy and others are extremely useful for breaking large classes down. Rather have 10 smaller classes than 1 large one.

Well here is a very simple start to how to get your refactoring of that large class under way. Please be sure to compile your project after each step, to fix any errors, before moving to the next step.

[more]

Step 1

Split the class into two and create a wrapper.

Pick a very large, complex class. Mine is called "Runtime"
Refactor the name of the class. I will call mine "RuntimeExecutor". Add any suffix for the moment, you can always come back to it. (Naming classes becomes harder and harder as you divide classes.)
Add a new class with the "old name". Runtime.
Extract the Interface of ActionExecutor. In VS.NET you simply right-click on the RuntimeExecutor class, and select Refactor | Extract Interface
For this exercise, simply select all the members.
A new class file with IRuntimeExecutor interface is created.
Now implement the interface on the Runtime class.
And create an Instance of the RuntimeExecutor from within the constructor of Runtime

It will look something like the code below

//abbreviated version of the class.
    public class Runtime : IActionExecutor
    {
        RuntimeExecutor _Executor; // this is the original class, renamed and now included as variable

        public Runtime()
        {
            _Executor = new RuntimeExecutor(); // we create the instance
        }

        #region IRuntimeExecutor Members

        public void Start()
        {
            _Executor.Start(); //we call the executor
        }

        public void Stop()
        {
            _Executor.Stop(); //we call the executor
        }

        #endregion
    }

Step 2

In the code above, you can see that I have added _Executor.Start() to the Start method. So in effect, Runtime is simple wrapping the Start of RuntimeExecutor. Now the next few steps are harder, but will increase the rewards.

Create a new class suffixed with State. I called mine RuntimeState
Take all the fields and properties for those fields from RuntimeExecutor, and cut/paste into State.
If you only have Fields, refactor them so that you now have Proeprties for those fields
Add a private field for the RuntimeState in RuntimeExecutor and
Modify the contructor of RuntimeExecutor by adding "RuntimeState state" as a new parameter, mapping the parameter to the field.

    //new state class
    internal class RuntimeState
    {
        private string _Field;
       
        public string Field
        {
            get { return _Field; }
            set { _Field = value; }
        }

    }

    internal class RuntimeExecutor : IRuntimeExecutor
    {
        
        private RuntimeState _State; //new field, replaces all the other fields that were here

        public RuntimeExecutor(RuntimeState state) //new param
        {
            _State = state; //set it and use it

            //now wherever it used the old fields, map it to the _State.NewField
           _State.Field = "some value";

        }
     }

So at this point, instead of 1 class, you have three. We refactored the class into two, the original and the State version. And we wrapped it higher up, by renaming it and substituting it for the new wrapper.

RuntimeExecutor (renamed original)
Runtime (new, a wrapper for RuntimeExecutor)
RuntimeState, (new split all properties and fields from RuntimeExecutor)

Step 3

We now need to split the original class further. This time instead of wrapping it, or refactoring UP, we now need to refactor down. Inspect the original class, and locate any methods that can be isolated and split from the class with minimal fuss.

Locate methods that are only looking at RuntimeState only, and move them, cut/paste into the RuntimeState class.
Make these methods internal, and point RuntimeExecutor to _State.MethodName
Locate methods that are utility functions to the class, such as getConfiguration, or IsRunning, and that can be moved to another class. If you have such functions, create a new class for it, and move it to the new location

So now you should have 4 classes from the original 1.

RuntimeExecutor (renamed original)
Runtime (new, a wrapper for RuntimeExecutor)
RuntimeState (now contains state related functions)
Utility class containing helpful functions

Conclusion

I follow this pattern for refactoring large classes that does not present itself clearly as refactorable. It gives me a set way of doing things, which then offers up more refactoring choices as I go along.
For example. When I find a lot of state, perhaps I refactor that too, into even smaller state classes.
Importantly though as you go along, you should be on the lookout for common classes. Perhaps another large classes State was extremely similar.
I then follow a pattern using inheritance for refactoring, and perhaps break things down even smaller.

In Part 2, I will show you how to utilize the wrapper for even more refactoring loveliness.

Smegging Code Metrics

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 03 Apr 2009 07:05:00 -0800

Traditional Code Metrics such as LOC an CC is absolutely rubbish. It is pretty much as useful as Flesch-Kincaid is for reading. A C# code file measures quite readable.

[more]

Also, how strange that doing a search on Wikipedia for Code Metrics takes you to what can only be described as a Microsoft Advert. Its dying... It's dead... Only three or four add-ins for visual studio? Not much competition.

Does anyone think that Code metrics in the traditional form is helpful at all?

The problem is we can't measure code yet. We cannot make judgements on quality. or can we?

Perhaps we can look at code, from architecture. A Service Oriented Architecture has a different quality belief system than Object-Oriented. We know what formulae to create on either. We know that less state in a SOA application is more quality?

or what about patterns. If well known Design Patterns can be found within code, is it better than non well known Patterns, or no discernible pattern?

I think these sorts of questions can begin to get answered properly now. Someone will create that new system/formulae soon. We need a system, and it will be provided, eventually.

How much should you design Upfront?

jonathan@lucidocean.com (Jonathan Crossland) — Mon, 30 Mar 2009 07:22:00 -0800

FeelGoodFactor

I follow the FeelGoodFactor principle. Requirements at first glance can be cloudy and messy. It can ambiguous and abstract, or it could just be plain daunting. However, you need to list what you can, and the attempt 'solves'. By solves, I mean a logical route to a solve, a prototypical solution to that problem, or at least common consensus by a few good men (cough). You need to get most of the areas in your head sorted, and then commit a bit of that solution in your head to paper and a review by your peers.

[more]
Here are some example of how I have conducted the last few projects, in the context of FeelGoodFactor.

Case Studies

Document Generator

Client wanted PDF and or possibly html versions of contracts and documents, downloadable from their website. They wanted the documents to be dynamic, so that it included data, their development team would inject into it.

There should be an editor for the mail merge like features, manage data templates (data fields) via XML, manage documents and map documents to data templates. A programmatic object model for adding data to the documents and rendering it out to pdf or html.

The FeelGoodFactor came once, I had established which PDF library, I was to use, and the approach I was to take. The approach was to create a parser, mapping expressions to data, and rendering to pdf, html or whatever the DocumentWriter was required. The extensible parts where DataFields (They would constantly change, or be added to), and the format the document would be required in, PDFDocumentWriter, HTMLDocumentWriter.

I was not concerned with the website, the document management side, as I knew I would quickly create a set of tables and UI of it, there was nothing to design for the early upfront stage of design.

I first wrote the XML Schema for the DataFields, and an ObjectModel to read/write the XML, and then moved to create a parser for html, to find my custom schema expressions.

So after 2 days, of writing code, aiding my design thoughts, and prototyping 3rd party technology, I quickly decided on a few Design Patterns that would coalesce into the base design, which I sketched in UML on paper and then started coding. Within 12 days, I finished the system, complete with +-70% test coverage, and example code in the form of recipes and Reference Help file for the small Framework. I then spent two days creating a few tables, and an API for Document Management on the server, using the Framework, ready for our website team, to use and integrate within the various sites and feature points.
They created a web service to wrap some of the functions and call it offsite, and it was done.

So the project had 2 days of thought, design, and prototyping, and 10 days developing. I developed in iterations of 2 days, and therefore had 5 iterations. iteration 1 and 2 was development and Unit Tests, 3 and 4 was refactoring and developing. Iteration 5 was testing and documenting.

Publishing Framework

For internal movement of files, website images, document management, it was required to move files of certain types configurable to different servers, folders, ftp and other locations. It also had to work work and accept ONIX request for books and publishable content, with managing files, such as TOCS, covers etc

After deciding on the architecture, Windows Services and Windows Communication Foundation, I created a small design prototype. I set to work creating the Web Service and Windows Service. Once that was communicating, sending any size files to the server, and not corrupting it or losing data, or anything else, I was in the FeelGoodFactor. I could then happily design each section within iterations as the project went on. The project also contained Onix publishing schema, and Onix Product publishing, with adaptors for DotnetNuke and many other facets.

This was developed over 7 iterations of 12 days each. I lead the design for 3 developers including myself. I spent the first 3 days, by myself, outlining some ideas, investigating Onix and other aspects. We had 2 meetings discussion ideas for the project. I spent 2 days creating an initial project, with hooks and areas for the team to continue on with, and gave them each a separate area to develop, with isolated objectives and isolated functionality. We integrated our work in iteration 2, and continued to code together in team system for iteration 2. We had regular scrums, not always in the morning and probably more every 2 days, than everyday.
I then did iteration 3 myself, as they worked on other projects. This gave me an opportunity to delete projects in team system (they get so messy), recreate projects, split and refactor projects, drag and drop classes aroundm change Namespaces, and create a more fine grained beast. With a more fine grained set of projects and solutions, it looks a lot better, and I got that FeelGoodFactor. At this stage with had 3 separate Visual Studio Solutions with around 6 projects in each.

In Iteration 4, I was rejoined by the two developers and we set to work creating unit tests, refactoring, and adding extra rounding features. Iteration 5, we started with a day or so of design meetings and requirements gathering, and set to work expanding the solution. I continued to design a new area, while the team continued developing new functionality.
Iteration 6 and 7, I was alone again, developing the last functions, refactoring, testing and making the design stick. I also documented various bits and pieces of the 'framework' portion of the project.

Iteration 8, which is the very next iteration, has been postponed, until our client, can meet and review. We shall not continue further, as specialization, further business rules and so on are required.

So the project has totalled 84 days, of which 10 days has been spent on design, thought and prototyping. However, I probably spend about 1 hour per day, refactoring and moulding the design, while adding new features.

The database, may get built slowly as we develop, but the focus is never on the database. I treat it as a throw away resource for these smaller projects. Both projects use reusable company Data Utilities, and can relatively easily be implemented in another database platform (depending on features, but we avoid platform specifics)

Design is not dead

Design is needed, you need to sit down and evaluate things. But the key is quiet time. Design must solve the problems it set out to solve. It must be a reachable design, within the time frame. You can reach a deadline easier, if your design is within iterations and gradually getting bigger, with the project. When you design upfront, one tends to add more things that are required. Designing for problems, evaluating solutions to problems, should be done upfront. Identify the problems within the system, and solve those, by careful thought, prototyping and any tool, such as UML, if it suits your needs, but don't design every object within your object model.

It would be a wasted of time, outlining the complete UML drawn object model, instead draw packages and the very important classes. Then create a set of rules/standards for the developers to follow, when creating the rest of the model. Also set a rule, that if you have to 'think twice', communicate with the team lead, to verify your thoughts. The same goes for when you cannot concentrate, perhaps never had breakfast. Get your team leader to pair program with you for a little bit. I call it The 'PushStart'. All of us need it from time to time.

So design until you feel that FeelGoodFator, and then develop an iteration, until it gets 'cloudy', then design until you get that FeelGoodFactor again. Now that is being Agile!

A Simple Software Design Methodology

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 27 Mar 2009 04:35:00 -0800

Hi, after you read this post, don't miss my podcast Interview with Robert C. Martin and Kent Beck.

No matter what software development methodology you adopt, you will always be faced with 'design' that is between good and bad, between necessary and unnecessary, between complicated and over-complicated. Design, even if it is technically good, can have disastrous effects on the project, if implemented incorrectly, or creates discord within a team, misunderstanding. In fact there are so many reasons, it would be better just to tell you what good design is. That's if I could, but I can't!.
However, I can say that Design is one of the most important factors for a project, it influences time lines more than anything else. Underdesigning, or overdesigning, neglecting to think ahead far enough, not including certain implicit requirements can all add days and weeks to a project.

Most of the time, its in your hands

A project requires thought and understanding. It takes a wide range of thinking outside of the box, thinking realistically, but yet being extremely open to ways of improvement. Here are three activities that will definitely aid your methodologies, skydiving and chess and music.

[more]

Skydiving

Skydiving, you jump out the plane, the ground is the deadline, and you simply ride the wind, and try to get finished before you hit the ground.

Often we are creating the projects, setting up a design, and coding it, within minutes or hours.
The first 'design' that sounds plausible, remains relatively unchallenged. We build some classes, think a little and then start coding. Most of the actual design is fleshed out as we go.
Unless you have a team, where design does get challenged, a lot. You get teams of developers debating direction and throwing pros and cons, which would be good, if there was a result. However, I have seen too many meetings end in stalemate, because it gets tiring. Not everyone thinks on the same wave length at the same time.

You cannot solve all the problems in one meeting. You cannot solve a design, or build some UML for the whole system on the whiteboard, using everyone's ideas, and thoughts. You will be there for days, weeks and many will get irritated and tired. It sounds obvious to let someone lead the design, but many projects, in todays flatten work environments want to get teamwork functioning and get the team to debate and meeting the designs. This will never work, unless it is very small, focused and within a time-frame, knowing that it may not be solved there and then.

software methodology principle number 1. Stop Skydiving.

Chess

Although we have refactoring tools, it takes real skill refactoring complex code, without breaking it, and still making it better. A large system can take weeks to refactor. In fact, most of the time, it doesn't get fixed properly. Maybe a little, but not completely.

So between "designing everything upfront" and "afterthough and refactoring", we need to consider more, and learn to read ahead. Like chess we should actually be stretching ourselves, practising to think ahead. In chess, its relatively easy to think of a few possibilities, but to really take advantage of the board, you have to be thinking ahead, further than your opponent.
In development, we need to get our read/think ahead skills better honed.

In chess, players think about the pieces, where each piece can move to, where it conflicts with moves of other possibilities. They construct a sort of tree, an algorithm, that each player might tailor to their own abilities and skills. They build this 'possibility tree', and then asses the tree for 'patterns', set moves, known strategies, and then further they inflate and deflate nodes in their minds eye, as to each nodes are more important, are more risky, or more possible to give them maximum rewards.

software methodology principle number 2. Play Chess.

Music

Rhythm, timing, tuning and arrangement, are the pinnacles of a good band. Without it, the best songs are rendered as noise.

The first thing is to find the rhythm section. Most importantly this will be your drummer and bass player. They have to be tight. They are providing the timing and the overall rhythmic feel. Then you need the rhythm guitar, who follows the drummer and bass, he has to add a little more flare at times, but overall, he is simply giving the treble feel. The Vocalists, and backup singers are providing the stage presence, the user interface and the meaning.
Developers who understand their position and work within their area, listening to the other band members and adjusting themselves to eachother.

Playing as a band, takes a little planning. Sometimes jamming is good, but most of the time, jamming will result in a rough product, perhaps with some awesome sections, and amazing solos, but as a record, it should be well rehearsed, tight and arranged with care. Then you get quality.

By the way, you also need someone to help you get into time, practice and coordinate things. There is always a band leader. There are also many many egos in a band, and the good ones learn how to do with that, so that the focus becomes the music, not the people.

software methodology principle number 3. Play as a Band.

ExtensionMethods repository

jonathan@lucidocean.com (Jonathan Crossland) — Wed, 25 Mar 2009 07:15:00 -0800

Oh what joy. Fons Sonnemans and Loek van den Ouweland have created a repository for ExtensionMethods. Go on, you know you want to. The site ExtensionMethod, allows you to submit your own creations to the community.

Now I had this idea, just around 10 minutes before I found the site, so they beat me fair and square. (By a mile). So therefore here is my top three ideas for the site, which I had, and hope they think about.

1. Grade it
Allow us to vote for it. However more than that, flag it with Compiled, Tested. Little icons that the site show next to each method.

2. Open Source the ExtensionMethod
Allow the community to edit the code, which will improve it over time. If I use it, and it fails because of something small, allow me to edit it, and solve the problem for everyone. Add original author, and contributor names to the list. Perhaps allow an 'Editor' per Type, to look after it and make sure its tested and complete.

2. Give us an Assembly
Run a build, once a month, compile all the snippets, which could be done automatically, if you get the correct data from the submitted snippet.
As part of the grading, is whether your automated compilation worked, when including the snippet. (snippet compiler like, compile on the fly and validate the method).

Now we can all download an Assembly for all these community driven ExtensionMethods. Updated: I found this nice, yet strangely odd set of ExtensionMethods for boolean.

Future Proofing your Application

jonathan@lucidocean.com (Jonathan Crossland) — Tue, 24 Mar 2009 09:06:00 -0800

Software lifespan types

StaggeredRelease
We develop versions and patches and roll them out periodically.
ContinuousLeak
We fix and create mini patches, live alteration, component replacement, live into the environment. Typically its developed by in-house staff, developers and/or support.
OneHitWonder
An App created and used, more or less with no problems or errors. Seems to be around for a while, and then fades away. Typically a utility, which has very little value, and soon gets replaced or irrelevant. Typically a quick utility written by someone on inspiration, but with little time to spend on its up keep.
RewriteEvolution
An App released again and again, but always with the knowledge that it will be rewritten and/or always seems to get rewritten. Typically where there seems to be little justification on spending a lot of time on it.
ConcreteBeast
An App, that is not liked or really wanted at all, but seems to be impossible to get rid off as too many things are dependent on it. Typically a piece of software installed before you got there.

[more]

Future proofing can be expensive, but there are some easy things to do, to increase your odds by a few percentage points. So in order to future-proof it, what should you do? First of all realize what kind of application its going to be, what characteristics will it have. You can use the quick list above, or some better scale.

Depending on less, not more

Loose coupling is an important factor in our making of agile software. Separating interface from implementation and other Patterns that promote loose coupling are very important for everyday development.

Use generalizations if you are not sure

Use a generalized, abstracted wrapper. Talk to your own Document object, and snap in with a Pattern of some sort, the actual real MS Word Document. This extensible model is vital, for allowing your Application to continue on through changes that happen around it. Actually this should be called use good Design Principles.

Control of Source and Components

Even though one tries to depend on less, we may still use many components, user interface controls and so on, which is good for reuse, but bad for dependencies. But try to avoid using reusable components, especially those you do not own, which is in direct conflict with increasing reuse and everything I stand for, but nevertheless.

Choose the right platform

Sometimes the platform or framework may dissipate. Choose your platform wisely. Think of Linq2SQL. Sometimes you need to go with something, like DAO, because, well, what else is there, that you could trust more? Now its ADO.NET which us currently 7? years old.

Know your RoadMaps

Get all the roadmaps of technologies that you depend on. Understand the manufacturer, their product cycles, and history. Learn a lot, read a lot and get your intuition firing.

Ask your manufacturer

How long do you expect to support this component? How long? Its ok if its only 3 years, if it coincides with your application lifespan. Sometimes they can be very helpful, and sometimes you will find subtle aspects that will help your choice, so just ask.

Avoid the Latest thing

Unless, your intuition says otherwise. Its sometimes great being one of the bleeding edge, but its very risky.

Frameworks and the MSDN Library

jonathan@lucidocean.com (Jonathan Crossland) — Mon, 23 Mar 2009 08:02:00 -0800

We all know that the learning curve for Frameworks are quite high. We all know that Frameworks demand commitment, and learning time. We know this because we struggle with it everyday, whether its Java or .NET, Swing, or MFC.

1991-1993, Ralph Johnson and others wrote about Recipes and Cookbooks. There have been papers on documenting frameworks for more than 15 years. MSDN has some of the flavours but none of it is actually polished, nor tailored to the audience and the way in which people seek information.

My D4 Software Frameworks Pattern Language points you to some Patterns for documenting Frameworks:

WritingReference
You must provide a reference of interfaces.
WritingRecipe
You need to provide examples of isolated features of the framework that a pure reference cannot show.
WritingCookbook
You need to combine a large amount of recipes.
SelfDescribing
Using tools and generators, you can generate documentation.

All of them are self explanatory, except let me add, amazing that blogs seem to be the number 1 source for this kind of information, as well as google tends to find things on MSDN better than MSDN finds on their own material?.

The Solution to the MSDN Dilemna

The last one (SelfDescribing) is where I am particularly keen. Not GeneratingReference but SelfDescribing Imagine if a method on a class called MethodA() had been marked with an attribute called TemplateMethod()

Well a Tool, would be able to highlight it for you. If you had a tree of all Types, TemplateMethods and Hooks, you would now better understand the way in which the types where constructed. You would now know that MethodA() called Method(B) then MethodC(). The internals of MethodA() ois not known, shown or cared for. But from a documentation point of view it is crucual for certain designs.

It is this, that is missing from MSDN, because it is missing from all frameworks. Documentation needs to be better, but cannot, until the compiled code, can be examined in a much more thorough way. Developers of Frameworks, must declare their intentions of methods, class and patterns by attributing metadata onto it.

So the solution, is to mark the .NET Framework Types with documenting metadata. Allow all Framework developers to utilize this metadata descriptions for their frameworks, and provide a tool to inspect it.

D4 Software Frameworks

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 20 Mar 2009 09:03:00 -0800

Effectively, here is an Index to the pattern language in draft.

A Pattern Language for Designing, Developing, Documenting and Deploying Object-Oriented Software Frameworks

[more]

D1: Designing Patterns

Designing Patterns are concerned with the Designing process and compliment Developing Patterns for implementations. They can be utilized as standalone patterns but usually combined for best effect. They can also be combined with design methodologies and design principles.

Designing Patterns discussed here (not Design Patterns [GOF]), are a category of Patterns that relate to the design of a Software Framework. Designing Patterns compliment and point to Design Patterns [GOF] but are different in nature.

DesignGeneralized
Find what is inherently or adherently common

DesignAbstraction
Find what is common via abstraction.

DesignRefinement
Refine abstract and generalized concepts.

CoalescentPatterns
Use one or more Design Patterns (GOF) to achieve goal.

ApplyAmmersePrinciples
Apply good design principles such as AMMERSE

UseIterativeDesign
Start with DesignGeneralized, then DesignAbstraction and lastly DesignRefinement

D2: Developing Patterns

Developing Patterns are used to provide Patterns for implementing Software Frameworks. Developing patterns are architecturally minded, and they serve to realize Designing Patterns by their implementation techniques.

DevelopHotspot
Discover where your hotspots should be. Choose DevelopWhitebox or DevelopBlackbox

DevelopWhitebox
You want less DesignRefinement or require some form of DesignAbstraction or DesignGeneralized to remain.

DevelopBlackbox
You want less DesignAbstration or DesignGeneralized and want complete DesignRefinement.

ConfigurationTool
You have used DesignBlackbox and require more flexibility and customization.

CodeGeneration
You have used DevelopWhitebox and want to allow for easier and quicker DesignRefinement.

SoftwareFactory
You have ConfigurationTool and CodeGeneration but want more power and flexibility.

D3: Documenting Patterns

Documenting Patterns are used to describe the Software Framework and aid the Deploying Patterns.
They facilitate and maximize understanding and therefore protential reuse.

WritingReference
You must provide a reference of interfaces.

WritingRecipe
You need to provide examples of isolated features of the framework that a pure reference cannot show.

WritingCookbook
You need to combine a large amount of recipes.

Self-Describing
Using tools and generators, you can generate documentation.

D4: Deploying Patterns

Deploying Patterns realizes the Software Framework itself, and looks at ways that maximize the Frameworks reuse potential.

DeployWithSupport
You need maximum facilitation of the framework.

DeployWithSampleApplication
You need to show off and demonstrate complex capabilities that WritingRecipe and WritingCookbooks do not show.

DeployWithTraining
Although Documenting Patterns are applied you need to further adoption.
DeployAsIntegration
Although Documenting Patterns are applied, Your framework requires easy and quicker implementation and adoption.

Bibliography

Design Patterns: Elements of Reusable Object-Oriented Software
- Gamma, Helm, Johnson, Vlissides (Addison Wesley)

Implementing Application Frameworks
- Fayad, Schmidt, Johnson (Wiley)

Pattern-Oriented Software Architecture: A System of Patterns [Vol 1]
- Buschmann, Meunier, Rohnert, Sommerlad, Stal (Wiley)

Pattern-Oriented Software Architecture [Vol 2]
- Schmidt, Stal, Rohnert, Buschmann (Wiley)

Designing Reusable Classes
- Johnson, Foote

Business Component Factory
- Herzum, Sims (Wiley)

Software Factories: Assembling Applications with Patterns, Models, Frameworks and Tools
- Jack Greenfield, Keith Short, Steve Cook, Stuart Kent

Evolving Frameworks: A Pattern Language for Developing Object-Oriented Frameworks
Don Roberts, Ralph Johnson
Available at: st-www.cs.uiuc.edu/users/droberts/evolve.html

A "Framework" for Object Oriented Frameworks Design
- Parsons, Rashid, Speck, Telea
Available at: portal.acm.org/citation.cfm?id=832937

Design, implementation and evolution of object oriented frameworks: concepts and guidelines
- J. van Gurp, J. Bosch
Available at: publications.jillesvangurp.com/spejvg.pdf

A point about AgileDesign

jonathan@lucidocean.com (Jonathan Crossland) — Thu, 19 Mar 2009 07:00:00 -0800

Two comments came through to me regarding AgileDesign. It was pointed out that Agile is a loaded term, and perhaps I should look to rename it. Another aspect mentioned to me in an email, was that Extensible and Agile are probably the same or too similar to be different principles. The thing though, is that my AgileDesign, is exactly "Agile" Design, and although Extensible and Agile are both concerned with change, there is a very important difference between them.
[more]

Agile has a few definitions.

Moving lightly, briskly and nimbly
Speed of operations as a result of a request
Dealing with change
Ready to move

You are agile, when you can change quickly and efficiently.

AgileDesign is a principle which highlights the approach design from an agile perspective. This is what comes from using Design Patterns, following good heuristics, and having an agile mind to cope with change better.

In the AgileDesign approach it contains a list of patterns, from TemplateMethod (GoF) to Pair Programming (XP), which are both aids to accomplishing the goal of an AgileDesign.

When you Design a software solution, it is not just about 'how good you are', or how elegant a solution is. It is not about how 'awesome' or not the code is. What is very important is the amount of agility, and where the agility lies.

In Framework texts, you find mention of FrozenSpots and HotSpots (Pree). AgileDesign covers both areas. AgileDesign covers mutable, sometimes even seemingly immutable portions. However, ExtensibleDesign is the adding of something new, and extending. Therefore ExtensibleDesign and AgileDesign are concerned with change, but offer a different set of solutions.

Software: LackOfEvolution

jonathan@lucidocean.com (Jonathan Crossland) — Wed, 18 Mar 2009 05:11:00 -0800

The majority of Software is quite multicellular. This sounds quite good, but in fact, its not even a small insect yet. or one could argue they are at the bug stage. Software tends to be rude, uncultivated, and unevolved, mainly due to LackOfEvolution.

One of the significant aspects of this, is that focus is granted to the development of the initial project, and then a handful of people (or if you are unlucky, just you), who are burdened with supporting it afterwards. This basically means "bug fixes until death" and lots of phone calls and emails later, you lose the lack of thought and action, but the software does not evolve. Software suffers from LackOfEvolution

Why do I say this
I say this because the vast majority of software only undergoes one full life-cycle. It may then get a smaller cycle of a patch or update, but most do not go to version 2 with the same code base. Version 2 tends to have very little of the original team members, and the code may very likely have completely rewritten areas. Version 3 even less of anything original. It may even have a new technology or language that it uses. Developers also enjoy writing their own code, rather than using other code, so this makes it easier to rewrite.

solution?
The problem is that the software does not get time to age, mature and be cultivated.
We need that extra time. An extra cultivation iteration is needed in all software teams, on any project. Create a planned version 1.1. Get your full team to conduct a special maturing cycle, where they add a few features, refactor code, shuffle things around. Plan it properly just as you would the first life-cycle.

LackOfEvolution for Teams

Software Teams can suffer from LackOfEvolution also. Developers can sometimes get caught in a project and not learn new things while they are busy on a project. You get to the end of a two year development and realize a few things have passed you by.

solution?
It is vital to stimulate debate about code, to have a trainer in for a day, to get show and tell sessions by members of the team, and other useful activities to counter act the stagnant waters. If you don't do SCRUM, have a SCRUM week, where you follow SCRUM for the week and evaluate it. Have a CompetencyFramework fro all developers with a plan on their evolution within a project and technology. We all need evolution, of character, skills and understanding, it is how we get better.

Understanding Framework Types

jonathan@lucidocean.com (Jonathan Crossland) — Tue, 17 Mar 2009 07:03:00 -0800

Traditionally, you read about the following two types.

Vertical Frameworks
or an Application Framework that covers all the elements top to bottom. It may be narrow, or relatively wide. However, it is most likely to be specific to a type of Application and not concerned with cross application reuse. For example: Employee Benefits, Batch Manufacture, Ledger and things of that nature. It would contain its own logging, errorhandling, data access, auditing. It would allow extensibility, configuration, but from a vertical perspective.

Horizontal Frameworks
or Infrastructure Framework is concerned with the wider services, and does not dictate the higher layers of an Application. So it would contain a wider range of services, but offer less vertical specializations.

[more]

A Vertical Framework is an Application

Ultimately, both types would be striving for the shape of the other. A tall narrow vertical framework, would be striving for more 'width', and likewise a Horizontal Framework would be striving for 'height'.

Both these kinds of Frameworks can be implemented with the same tools and concepts, but with a different focus.

If you focus on delivering an Application, you are concentrating on application requirements, usability, and not a framework's requirements. Therefore, I believe that a Vertical Framework is just an Application that has a 'built-in' framework, which is being worked on while developing the application. That does not exclude it from the term framework, instead, I believe that all Applications should be built in this way.

Everyday Framework Development

So what does it take? I don't recommend that without experience you go ahead and develop Frameworks within your current project, but do learn about it and try small Framework like things in your applications.

If we understood, exactly what a Framework looked like, we could tell exactly what to build in our applications that would make it more Framework like and thus more reusable.

In my previous post, I requested 140 characters of a framework definition, which although only a few replied, it is still somewhat revealing.

It is interesting because, most have the correct concept, but when you analyse descriptions, not all terminology is used with the same definition. For example. API is now used almost synonymous with anything you call that has a method, but a Framework is not an API (Application Programming Interface). A Framework is not an Application, nor is it just an Interface. A Framework is more than stateless calls. A Web Service is kind of like an API. Most of the time these are stateless. At least its best practice to have it stateless. Contrary, a Framework should contain state, and is much more than an interface. An API is often referred to as a Library, which is a collection of API. On the state-full side, you are looking at an Object Model, but just a Model is not necessarily pluggable or extensible.

So Object Model, API, Library and software assets of that sort are only 'one portion' of a Framework, and you can use them without it ever being part of one.
So if you are building an Application that contains an API, or library, then you have a useful reusable set of methods or objects. You do not have a framework.

Is it a part solution to a problem? Indeed a Framework can have solutions, and in fact the more the better. But you can create code that solves problems within an Application, not just a Framework. So it cannot be the only answer.

Is it an Independent System of cohesive software. This is true, because the Framework must remain independent from the Application, to maximize reuse. But so too, is a component. Uncle Bob mentions one extra thing in his description, which is absolutely vital, and that is "performs a general function that controls user specified functions". This is vital, as did Paul Tiseo eluded to it also when he mentioned application-specific code. The correlation there is generalization and specialization controlled by the Framework. The Generalized provided by it, and the Specialization executed by it. This polymorphic activity is a vital aspect of a Frameworks definition.

However, both definitions and my first definition also, do not mention MetaData. MetaData is also vital, as it is the very thing that declares the changes, specializations and completes the picture.

So in fact, we can easily conclude that if we build an Application and keep a "subset of cohesive software, library + application-specific code, with generalization and specialization, configurable, extended by means of Framework code using MetaData", we are in fact building an Application Framework within the Application itself. It must be able to be separated out though, to finalize the categorization requirement.

If you are still interested, have a read on "The Quest to Define a Framework"

Framework Definition in less than 140 characters

jonathan@lucidocean.com (Jonathan Crossland) — Fri, 13 Mar 2009 04:24:00 -0800

What is a Framework? Its a difficult one, since most of the time, we all have terminology that differs in meaning for us.

My usual line is this "a class is to object" as "framework is to application", but it doesn't speak of the relationships between classes, or the architecture or structure.

I am trying to compile a list of definitions from you, so we can look at how we think of Frameworks today, right now. Over time, I wish to discuss each definition, break it apart. Look at what is good and bad, misleading, wrong, or right. Debate the definitions and then hopefully we can all understand it better. So please add me to your twitter and send me your definition.

A Framework is: [more]

CoalescentPatterns and AggregatedPatterns which exposes mutable and extensible areas, for specialization and change. - @jcrossland

"An independent system of cohesive software that performs a general function that controls user specified functions." - @unclebobmartin

"dumb soundbite: a framework is a program you don't mind finishing." - @KentBeck

"Code that attempts to solve a problem that hasn't been shown to exist yet" - @benfulton

"Framework is designed to support dynamic websites, apps, and services, while alleviating overhead assoc with common activities" - @JRewald2000

"A framework is an API that sits above a language in order to aid developers usually in a specific area of expertise, e.g. web" - @arumbrindy

"Not a solution. But the concept how to create one." - @BanDoga

"Framework is a painkiller in the way it lets you focus on what really matters. Good for production, not learning purposes." -@josemotanet

"Framework = application - (language + libraries + application-specific code)" - Paul Tiseo (see comments)

"An abstract scaffolding of a solution for a certain problem, and yet hard to implement on it's concrete form." - @nobodybutca

"A framework is a collection of solved problems that, when used, allows for increased focus on unique features and functionality." - @irobinson

Please send me your definition! never too late, I will post it up here.