Reddnet Scribbles

Going Postal on FedEx, Socialism, and Healthcare

Stephen M. Redd — Wed, 16 Sep 2009 15:37:00 GMT

It has been a while since I’ve gone off on one of my famous rants... so go refill your coffee and enjoy ?

The national healthcare debate here in America has continued well past the typical levels of American political absurdity. It's a mess, and a real solution hasn't appeared anywhere near the table yet.

But the biggest victim of this political fist-fucking so far has been the U.S. Postal Service.

Anytime someone wants to scare you away from a government solution to the terrifying state of healthcare in this country, they dig up the specter of the post office!

"What? Do you really want healthcare to be run like the post office?”

Most people think then of long lines, mail gone missing, and incompetent postal workers... and they answer, "hell no"!

But I say "hell yeah"! Give me a post office for healthcare PLEASE!

For many, the post office is their prime example of a failed socialistic government institution. They often claim that everything is better when the private sector runs it, and some of them go so far as to tell me that the post office is un-American!

These people are either completely ignorant, or obtuse to a degree that borders on criminally insane.

The socialist-commie plot to destroy America through badly implemented mail delivery is written directly into the Constitution of the United States itself. The post office as an organization was put into motion by the very first sitting U.S. congress only 4 months after the adoption of the Constitution.

Did you know that?

I don't want to frighten you... but...

BOOO!

For a really good meat-weight overview of post office history, there is a fantastic article over at about.com that tells the story pretty well.

This country was founded by post-office loving socialists!

Rather than being an example of how government run organizations don't work, the U.S. Postal Service is an example of something that ONLY a government organization could have achieved.

Back when this nation was first founded, few could seriously question the benefits of postal services. The colonies under Britain’s rule had already had a postal system of some sort for over 100 years. The benefits had been proven beyond rational doubt.

With a newfound nation as geographically large as this one, the need for fast and efficient post was absolutely essential to the prosperity of the nation and it’s people.

The task of providing postal services across the vast and untamed wilderness was far beyond the abilities of any regular private business of the time. The risks of establishing a wide-spread postal network were enormous, the costs staggering, and the potential for profit laughable.

There were a number of private courier companies at the time, but they could not operate at the scale and uniformity necessary to bring the full promise of the post to reality.

For the majority of its 230 year history, the post office has been nothing short of amazing, and the overall contributions to both individuals and business have been immeasurable.

Just on the technology side alone, the post office has been essential to nearly every major improvement in transportation that has ever been made. Many organizations private and public alike were involved along we way. Together they built roads, bridges, rail systems, canals, boats, automobiles, trucks, airplanes and everything else that goes along with them.

The post office backed these projects financially, protected them politically, and encouraged the insane pace of growth and innovation. It would not have been possible for a bunch of rebellious east-coast dirt-farmers to become the most powerful nation on earth in less than 200 years without the vital role of the post office.

All through our history, if there was any project related to transportation, large scale logistics, or distance communication you’d find the post office right in the middle of it all... forever pushing for more, better, faster, and cheaper mail delivery.

Some will argue that the superior private sector would have built those things themselves without the evil government’s help... I'm pretty sure that is true too.

But how much longer would it have taken private companies to decide there was a profit potential in building the great rails through the impossible terrain of the wild west absent those lucrative postal contracts?

How long before the great bridges across the mighty Mississippi would be a "sound business investment" on their own?

How many inventions would have died on drawing boards for lack of private funding?

Many of those post office sponsored projects were so far beyond crazy that ONLY a government could possibly have been deranged enough to try them. I mean come on! They even tried to invent missile mail!

Yeah... that’s exactly what it sounds like... letters delivered via missiles! That's the kind of nuts that only government can aspire to.

Without the post office, we might be as far along as the train and early automobiles by now... maybe....

You can argue the benefit vs. drawback of almost any specific government program from income tax to Medicare if you want... But to argue that the U.S. Postal Service is an example of one of the failures is just plain ignorant.

The post office of today has very real problems. But the causes of most of those issues are also recent. The rise of modern electronic communication systems have eroded the underlying need that made the post office the essential service it once was.

The post office still gets by, but it is primarily a small parcel and letter carrier; precisely the kind of enterprise hit hardest by phone, fax, and email.

Even if you have no respect for today’s post office, at least have the decency to give it the credit it deserves for the two centuries of dedicated service that was so essential to the people of this nation!

A lot of people extolling the virtues of capitalism will contrast the post office with FedEx. FedEx is their poster-child for why capitalism rocks.

The story of FedEx is also quite amazing; an inspiring tale of the absolute best that capitalism has to offer A few people with their own absolutely crazy idea that go on create one of the largest business empires on the planet in just a few years.

But! And here is the part where I start to get truly angry...

Anyone that can seriously argue that FedEx is an example of why we don't need government organizations is either too ignorant to be running their mouth or so fucking retarded that I wonder how they avoid drowning when it rains.

FedEx is probably the best example anyone could find anywhere for EXACTLY why government programs and organizations are essential to the very existence of that healthy modern capitalistic free-market everyone is so damned proud of!

Who do you think built the highways and roads that FedEx trucks use to affordably pickup and deliver those packages? The interstate highway system was a hotly debated socialistic government program.

The United States Military, the Borg Collective of socialistic government organizations, has been responsible for literally EVERY significant aerospace engineering advancement since the airplane was invented... they even funded the Wright Brother themselves. How would FedEx compete without that massive fleet of modern high-tech jet planes?

How about the airports? Where would those FedEx planes land without them? The government, largely prompted by the U.S. Post Office, was responsible for establishing the nationwide network of coordinated high-capacity civilian airports.

What about the GPS system that lets FedEx track where their trucks and planes are? How would FedEx efficiently handle logistics at that scale without free use of the Air Force's network of positioning satellites?

When FedEx needs to know where to send a package, whose database do you think they use to look up the addresses? Who do you think it is that maintains that massive index and keeps it reasonably up to date? Yeah! The lowly post office does that.

What about the internet that FedEx uses to provide that amazing customer service experience? The internet itself was funded and managed by the defense department before it was freely given over to the public in the early 90's.

Consider the massive phone network upon which FedEx's voice and data systems rely. Would AT&T have been able to build it had they not been a government protected monopoly? Part of Ma Bell's deal with the government was "universal access"... either they expand to provide phone service everywhere or bye-bye monopoly! Would they have ventured so much so fast if they faced the risks of a competitive free market? Could they have even negotiated the necessary land rights needed in order to lay the wires and place their equipment if the federal government wasn’t backing them up?

Where would FedEx be today if it weren't for all these government programs and organizations?

If you break down FedEx's entire business, can you name me even one single significant part that doesn't owe its very existence to some government run organization or program somewhere?

FedEx isn't successful despite the government... it is successful BECAUSE of the government!

Ok, enough about FedEx. I’m not nearly angry enough yet, so let’s get back to healthcare so I can lose my damned mind!

As far as "public services" go, healthcare is a young concept. A hundred years ago the notion of healthcare as we understand it today didn't even exist.

Most of the things that we expect governments to deal with have been around a very long time. It is a long list, but here are some highlights...

Military and police
Fire brigades
Postal and communication services
Water, sewer and waste removal
Transportation and roads

Nearly all modern civilizations deem these services so critical, that providing them is the exclusive responsibility of their governments. In most places, access to these services is a fundamental right of the people and they are smart enough to know that only a non-profit public organization that represents "the people" can be entrusted to provide for them.

But all of these services started out as special privileges reserved solely for the wealthy and powerful.

Healthcare is similar to those other services, except that it is still a very new concept. Here in America, healthcare is still only provided as a commercial service for those who can afford it. But the rest of the world understood fifty years ago that the good health of everyone was essential to the prosperity of anyone...

America missed the boat on this obvious concept though. But now that our prosperity is undeniably threatened by our own lack of common sense, we are finally being forced to look for a way to solve the healthcare problem.

The need for healthcare in the U.S. today is a lot like the need for a postal service was back in 1776. Like the postal service then, we've had about 100 years of practical experience with some form of healthcare... enough to have proven the benefits beyond a reasonable doubt. We also have ample proof that private interests can not be trusted to meet the full need.

The founding fathers could have decided not to take on a national postal service. They could have left it to private couriers. But they understood that a profit motive wouldn’t get the job done. Instead they appointed the Postmaster General, created the Post Office, and they got the job done.

So here we are now... faced with this undeniable need for universal healthcare and no free-market solution in sight.

We can fuck-about and let the free market continue to fail, or we can unite the hands of the Surgeon General, establish a real U.S. Heathcare Office, and get the job done already.

But no one is talking about that... all they keep trying to do is sell me fucking insurance! As if a piece of paper from a company whose entire job is selling pieces of paper is some kind of a healthcare solution.

When I have a heart-attack, I don't want a policy motherfucker! I want a doctor!

Healthcare is not a "regular" commercial service.

When people get sick, they are not able to make an "informed consumer choice". You aren't buying a fucking box of cereal here! You are sick and you need help RIGHT NOW! .

Healthcare is not a consensual business transaction between willing participants and it certainly is not a fair trade nor an equitable exchange of goods or services. The person needing healthcare is scared, sick, and possibly incapacitated. Hell... they might even be dead already!

The seller of healthcare service has ALL of the advantages. You WILL buy whatever they are selling or you WILL fucking die!

Is that your idea of the free market at work?

And stop telling me insurance is the consumer product that will save the day here!

Fuck you!

Us regular Americans... we do NOT have any say about which health insurance policy we get. We get whatever policy our employer chooses. They dictate what insurance is offered, how much it costs, and which services will be covered.

Do you really think that MY interests are involved in that bullshit free-market exchange in any way at all?

Are you really that damned dense?

And what is with this fucked up idea that the free-market can bring me "competitive choice" of provider and policy options? It hasn’t so far, but even if it did... what the fuck!?!?

Have you actually tried to read an insurance policy?

Even if I could decipher the actual terms of the contract, which is tilted to give every advantage away to the insurer of course, and even if there was a large selection of competing policies for me choose from... I still have NO FUCKING IDEA what my medical needs are or what they will be in the future!

I fail to see how a competitive choice helps anyone. Unless the policy covers every possible healthcare service I might ever possibly need, it isn’t a solution.... and if a policy could provide total coverage at a reasonable rate why would we need competitive choice? Everything is fucking everything and it doesn't matter if I get it from Paul or from Bob does it?

No... what these ass-hats are proposing is just a mother-fucking game of Russian-roulette!

Maybe you guess correctly and pick a policy that just happens to meet your future medical needs... or maybe you guess wrong, your claims get denied, and you fucking die!

Thanks free-market!

Maybe we should have tried insurance back in 1776... Postal insurance companies!

How about that!

You'd pay a Postal Maintenance Organization (or PMO) and they'd give you a policy. Then when you need to have a letter delivered to someone, they'd check to make sure you used the words from the PMO approved word list.

Then you'd have to go find a private letter carrier, but only one that was "in-network" at your PMO of course.

Assuming this carrier is willing to send someone running across 500 miles of Indian and predator infested wilderness... in the rain and snow... AND the carrier doesn’t charge more than the insurance company allows... THEN... maybe... you get your letter delivered.

But only as long as you don't have a pre-existing fucking clue as to what the hell is going on!

Considering how much more important healthcare is compared to mail delivery, I am at a loss to understand how it is that we are still arguing if the government should provide healthcare as a public service.

What we should be arguing about is why we have to wait 45 minutes in line at the local Health Office just to get a flu shot... Or waste our time choosing between prescription labels with the traditional U.S. flag printed on them or the new ones with the cute kittens...

But if I hear you giving the post office any more shit... I might just go postal for real! Hope your health insurance premiums are paid up!

The case of the non-enforced foreign key relationship

Stephen M. Redd — Sat, 12 Sep 2009 07:06:00 GMT

While working with the TicketDesk 2.0 MVC project, I came across a really unusual situation within the relational database that TicketDesk uses.

The resulting voyage of discovery lead to me finding my first legitimate use of a "non-enforced foreign key relationship" in a relational database... The situation required an explicitly defined foreign key relationship, but also prohibited the server being able to enforce the relationship.

Here is how it happened...

I'm trying to make as few changes to the TicketDesk 1.x data model as possible. The focus for the 2.0 project is the shift to the new MVC platform and sticking with the old data model makes it easier to support upgrades. Plus the existing model is pretty decent as it is.

In TicketDesk, attachments are stored in the database rather than the file system. This prevents the need for write access to the web server's file system. It also allows you to copy or move ALL of the data by just moving the database. Simpler all-around.

We could have a huge discussion about the drawbacks and advantages of this design. The wisdom of using a RDBMS to store binary file data is very debatable. But the end-analysis here is that TicketDesk isn't expected to handle a lot of files, they are not likely to be large files, and the advantages of having all the ticket data in the database outweighs the disadvantages in most environments.

Anyway, the big problem in TicketDesk 1.x is that the client-side file upload mechanism is terrible. It does gets the job done, but it requires that files be uploaded together in a single HTTP POST. This can be slow, the process is subject to request timeouts and size limits, and it uses a LOT of resources on the server. Additionally, if the server rejects the submission then the user has to re-upload everything all over again.

For TicketDesk 2.x I wanted to switch to using a flash based uploader. I personally HATE flash, but when it comes to file uploading plain HTML and javascript fail to provide any good solutions.

The flash uploader will send the files up to the server using separate out-of-band connections to the web server. This means that the server has to "put" the files somewhere until the user has finished submitting the rest of the form.

For new tickets, the files will get uploaded before there is a ticket in the database to associate them with. For existing tickets, the uploads need to be held in a pending status until the user has finished supplying the meta-data (descriptions, file names, and comments for the activity history log).

I really didn't want the server to dump pending attachments to the file system or hold them in memory, so I needed to modify the database to provide a place for the server to put the uploaded files as soon as they arrived.

The original data model looked like this:

As you can see, there are two tables. Tickets and TicketAttachments. The contents of the file are directly stored in TicketAttachments along with meta-data (file size, name, description, file type, etc). The old primary key was on both TicketId and FileId... not my best primary key definition ever, but it made sense at the time. TicketId is foreign keyed to the TicketId column in parent Tickets table.

Not a remarkable design really.

At first I toyed with creating a new "PendingFiles" table. But there was already a table with the right kind of structure in the database. It seemed to be a waste to duplicate table structure and I've never been a fan of "staging" tables anyway.

So I started by figuring out how to use the existing TicketAttachments table to store the "pending" files as well as "real" files.

For pending attachments to an existing ticket, the table is already pretty good. All that is needed is a column to flag files as pending instead of real. So I added an "IsPending" bit column to the table. New attachments have IsPending flagged. Once the user commits the changes, we just flip the bit to make the file a real attachment.

Here is the revision.

Now, to handle attachments for new tickets. I figured that I could put the files in the same table, but we'd need to leave the TicketId column null... we don't know the TicketId yet.

I didn't like the primary key as it was anyway. FileId is an identity column and can serve as a primary key all by itself.

So, I changed the primary key to FileId then marked TicketId column to allow nulls and got this:

Great! Except...

Because of the foreign key relationship, we can't add a row to TicketAttachments with a null TicketId value unless there is a row in the Tickets table that also has a null TicketId. We can't make TicketId in the Tickets table nullable because it is the primary key (and an identity field to boot).

Ok fine... the DBA in me screamed bloody murder, but I deleted the foreign key relationship. I've been doing web development long enough to know when the advantages of denormalization outweigh good relational design. I hate having to manage relationships purely in code, but sometimes that really is the best way.

So we get to this:

But! LINQ to SQL needs the foreign key to generate the relationship between the entities. I could manually define the relationship in the LINQ to SQL mapping file, but I *know* that this is going to bite me in the ass later.... it always does. Combine that with the uncomfortable absence of a foreign key in the database itself and I just couldn't bring myself to let that slide.

Then I remembered that there is this odd setting in the SQL Management Tools. The designer for editing foreign keys has a true/false flag for "enforce relationship". I never paid it much attention... I mean, what crazy bastard would go to all the effort to define a foreign key, then tell the server to ignore it?

That just never made any sense to me. Until now...

So I did some testing and research. If you set a foreign key up in SQL, but tell it not to enforce it (this is called the NOCHECK option) then the SQL server will behave as if the foreign key doesn't exist. Thus, we can add rows to TicketAttachments that don't correspond to a row in the parent table in direct violation of the foreign key.

Tools that read the database schema do see that there is a relationship though, and so LINQ to SQL will still generate a relationship in the mapping file automatically. From the point of view of external tools it is as if there actually is a regular foreign key relationship in place.

So the final design ended up looking like this:

The foreign key shown here is setup with the NOCHECK option and is not enforced by the server.

So now I know why you'd use the NOCHECK option... I don't expect to need it often, but now I know why such a bizarre option exists and have finally, after years of working with databases, had a good reason to use it.

TicketDesk 1.2.3 Released on CodePlex

Stephen M. Redd — Wed, 09 Sep 2009 05:11:00 GMT

I've formally packaged a new version of TicketDesk at Codeplex.

TicketDesk 1.2.3 is a minor update. The most significant change is that the HTML editor has been replaced with the markitUp! editor using the markdown syntax.

This should hopefully work around some problems many users were reporting with adding comments in IE8 as well as some display problems that could occur when a user cut/paste content from a word processor or other sources with embedded rich formatting.

TicketDesk 2.0 MVC - alpha demo now available

Stephen M. Redd — Thu, 03 Sep 2009 04:07:08 GMT

As I've mentioned before, I'm working on the next major version of TicketDesk 2.0 on the ASP.NET MVC framework. The project is still a little early in development, but is starting to resemble a real application now.

I've put demo site up to give the public a preview...

The demo of TicketDesk 2.0 MVC alpha is now online. I'll be updating it from time to time as I reach different milestones. When the project gets closer to a beta state I'll likely check it into source control over at the TicketDesk codeplex project, but for now I'm working offline on it.

Implemented so far:

TicketCenter with the default list views
TicketEditor displays ticket information
- The visual formatting is very rough
- Attachments cannot be downloaded (this is on purpose... my demo is not an FTP server for the public's warez :P)
New Ticket feature should be fully functional
TicketEditor activity panel should support most activities
- "edit ticket" and "add attachments" remain incomplete.
Account management is borrowed mostly from the "sample" MVC app, but has been customized for ticketdesk.

Stuff that isn't done:

Notifications and RSS are not implemented
Visual Styling and formatting is very "stock MVC sample" for now
- I like the general layout, but the text formatting needs lots of work
Ticket Search is absent
None of the admin tools are complete

Go poke at the demo and see what you think.

I welcome any comments, observations, or questions you might have, but don't go reporting bugs yet...this is an alpha demo so I already know it doesn't work very well yet :)

Enjoy...

Fixing markitUp! 1.1.5 - bug in IE8 when closing preview iframe

Stephen M. Redd — Thu, 27 Aug 2009 18:29:00 GMT

I've been working with the wonderful markitUp! editor by Jay Salvat. Specifically, I'm using markitUp! as a markdown editor for TicketDesk and a few other apps I'm working on. I'll probably post more about using markitUp! as a markdown editor later, but for now I wanted to address a specific bug that markitUp! exhibits in IE8.

By default, markitUp! uses an iframe element for a preview window.

In IE 8, closing the preview iframe will cause IE 8 to try to close the entire hosting window or tab. IE 8 will prompt the user before it does this, but if you click yes when prompted it will indeed kill the window/tab... which is not good.

After some digging, I've located the problem...

Here is the relevant code with the part that is called when closing the preview window in bold:

 
// open preview window
function preview() {
	if (!previewWindow || previewWindow.closed) {
	    if (options.previewInWindow) {
		previewWindow = window.open('', 'preview', options.previewInWindow);
	    } else {
		iFrame = $('<iframe class="markItUpPreviewFrame"></iframe>');
		if (options.previewPosition == 'after') {
		    iFrame.insertAfter(footer);
		} else {
		    iFrame.insertBefore(header);
		}
		previewWindow = iFrame[iFrame.length - 1].contentWindow || frame[iFrame.length - 1];
	    }
	} else if (altKey === true) {
  	    if (iFrame) {
 		iFrame.remove();
 	    }
 	    previewWindow.close();
 	    previewWindow = iFrame = false;
	}
	if (!options.previewAutoRefresh) {
	    refreshPreview();
	}
}

What is supposed to happen is that the code removes the iframe element, then calls the close method on previewWindow variable. That variable would normally have a reference to the content window within the iframe (you can see where that is set in blue in the code excerpt above). So calling close on the variable would normally just try to close that sub-window... or maybe it would do nothing at all because the iframe containing the sub-window would have already been removed. The behavior is internal to the browser and I suspect that specific mechanics are probably a little different from one browser to another. But either way, this works fine on all the browsers I've tested with except IE 8.

With IE8, this code appears to invoke the the close() call on the containing window instead (which is your page's main window in most cases). If you make the close call before the iframe is removed, IE8 will behave like the other browsers do, but when the close call happens after the iframe is removed IE 8 starts asking you if you want to close the whole browser window. for some reason, the contents of the previewWindow variable change after you remove the iframe.

Not a problem! my solution was to simply alter the code to only call the close method when there isn't an iframe being used. That way, close is called for cases where you are using the pop-up window, but doesn't get called when you are using an iframe.

 
 // open preview window
 function preview() {
 	if (!previewWindow || previewWindow.closed) {
 	    if (options.previewInWindow) {
 		previewWindow = window.open('', 'preview', options.previewInWindow);
 	    } else {
 		iFrame = $('<iframe class="markItUpPreviewFrame"></iframe>');
 		if (options.previewPosition == 'after') {
 		    iFrame.insertAfter(footer);
 		} else {
 		    iFrame.insertBefore(header);
 		}
 		previewWindow = iFrame[iFrame.length - 1].contentWindow || frame[iFrame.length - 1];
 	    }
 	} else if (altKey === true) {
     	    if (iFrame) {
   		iFrame.remove();
   	    }   	    else {
   		//SMR - else block added here to prevent this call when preview is in iframe
   		//      IE8 incorrectly tries to close the hosting window if you call it when using iframe
   		previewWindow.close();
   	    }
     	    previewWindow = iFrame = false;
 	}
 	if (!options.previewAutoRefresh) {
 	    refreshPreview();
 	}
}

This seems to fix the problem in IE 8, while not breaking anything in the other browsers I'm testing with (chrome, firefox, etc). I could have just moved the close call so it happened before the iFrame gets removed (which also seems to work), but I'm a little concerned that closing the window before removing the iframe might have unexpected results in browsers I'm not testing with... but it would probably be fine either way.

If you want, you can download my modified versions of the markitup! source. I have included a standard and minified one both.

Please note that this version also contains my own killPreview() function. By default markitUp! has only one toolbar button for the preview. If you click it, the preview opens and if you ALT + Click it the preview closes. But in my own implementations I prefer to have a separate toolbar button for closing the preview... users don't magically "know" about the ALT+Click trick and I get tired of people reporting "I can't close the preview window" as a bug in the apps that use markitUp!.

TicketDesk 2.0 and the ASP.NET MVC Framework

Stephen M. Redd — Mon, 03 Aug 2009 20:25:00 GMT

Now that the ASP.NET MVC Framework is out, I've decided to tackle learning the new platform the same way I usually do... by writing a real application for the new platform.

TicketDesk 1.0 was originally just a playground application to help me get up to speed during the last round of new-tech releases from Microsoft... so it seemed natural to explore the MVC Framework with a re-write of the same application. TicketDesk is just small enough to be workable by a lone part-time programmer, and it is just big enough to provide a decent proving ground for the new technologies.

So let's discuss MVC and how it relates to TicketDesk 2.0...

One of the ironies of my life is that I've been primarily an ASP.NET developer ever since it was first released and I've also been working with MVC and MVC-like development patterns nearly that entire time too.

MVC patterns just makes sense for web apps seeing as the nature of HTTP itself matches that pattern so cleanly. In other environments, MVC has been a formally accepted pattern for years and years.

But ASP.NET Webforms was initially designed to make programming for the web feel more like windows programming with an event driven programming model. Microsoft excels at event driven programming techniques, and the resulting webforms framework was a fantastic adaptation of the pattern into web development space. Webforms allows you to mostly ignore all that messy HTTP stuff and code pages just like you would in a persistent windows environment.

But like most abstractions, webforms tends to break-down when you try to do stuff at the edges. So it wasn't uncommon for platform developers to find problems that just didn't map well to the abstractions provided by webforms. So many of us ended up spending amazing amounts of time hacking into the gap between the webforms model and the raw HTTP pipeline itself.

If you look at the architectures behind most of the larger and more successful ASP.NET application platforms (sharepoint, the 1.x starter kits, IBuySpy, DotNetNuke, CommunityServer, etc.) you will usually find elaborate examples these kinds of hacks. All of them are just variations on a theme... use MVC-like patterns to gain some control over the HTTP request/response pipeline.

With the rise of modern AJAX techniques and technologies, the need for a new approach has become very apparent. Ajax mucks around with the request pipeline in ways that the webforms framework does not tolerate elegantly. If you've tried to do any significant Ajax stuff in webforms, you've probably noticed how quickly things get messy.

Fortunately Microsoft recognized this and decided to formally embrace the MVC pattern. The result is the ASP.NET MVC Framework which was delivered a few months ago.

Which brings me back to TicketDesk....

I originally built TicketDesk 1.x as a way to experiment with .NET technologies that were new at the time (Ajax, EF, and LINQ to SQL). So I thought it would be fitting to do the same thing again now to get my hands dirty with the Microsoft MVC Framework.

I didn't port the existing TicketDesk 1.x code though. Instead, I've started with a clean solution and am re-implementing the same set of features as TicketDesk 1.x using all fresh code written for the MVC framework.

I suspected all-along that TicketDesk would probably map very well to the MVC Framework, and I'm no stranger to the MVC design pattern itself. I had also hoped that the MVC design pattern might eliminate many of the obstacles I had encountered especially with the Ajax parts of TicketDesk 1.x.

The experiment is about 3 months old now, and has been very challenging. The MVC Framework itself has a lot of room for improvement, but is a solid foundation on which to start. Some of the most obvious drawbacks are the slim Visual Studio IDE support, sparse documentation, and poor examples of how to do ASP.NET MVC "the right way".

The biggest challenge for me has been the steep learning curve. I've been writing web apps for over 12 years, most of that working with ASP.NET, but the ASP.NET MVC framework really requires an entirely different way of thinking. I'm also just now learning my way around JQuery too which has further slowed me down.

Currently Microsoft is providing only basic Ajax functionality within the MVC framework, but they have encouraged the use of JQuery. JQuery gives you a rich and very successful source for all those fancy UI components that Microsoft doesn't provide on the MVC framework. While the ASP.NET MVC Framework doesn't help you much with JQuery, it also doesn't interfere any. Future versions of the framework promise to further embrace JQuery head-on. I've not been impressed with Microsoft's own ability to deliver decent Ajax libraries so far, but JQuery has a very large 3rd party community developing high quality code... and most of it is some kind of open source to boot.

While I've found that writing against the MVC Framework takes significantly longer and requires much more effort, the quality and usability of the resulting application is many orders of magnitude better.

So I've formally decided to re-write the official TicketDesk application on the ASP.NET MVC Framework.

The initial 2.0 release will not contain very much new functionality compared to 1.x, but I hope to provide a significantly better user experience and a much more compartmentalized code-base.

Currently TicketDesk 2.0 is targeting the ASP.NET MVC Framework 1.0 on the .NET 3.5 stack. I did experiment with the RTM release of the Entity Framework this time, but I still find that EF is just not ready... so I'll be sticking with LINQ to SQL for a while longer. I'm confident that I can switch back to EF should the next version resolve my remaining concerns. It is likely that the next version of the MVC Framework will be released before I am done with TicketDesk 2.0, so it will likely shift to target that version before the final release.

Here are some early goals for the TicketDesk 2.0 project:

Implement 100% of the functionality from TicketDesk 1.x
Upgrade Tools for 1.x to 2.x migration
Improve Application Settings and Administration (more and better online admin tools)
Improve formatting for RSS and Email notifications
Enable full functionality for browsers without JavaScript
Enable smoother Ajax UI features for browsers that do support JavaScript
Use a Markup editor instead of a WYSIWYG HTML editor (too many problems with raw HTML data entry). I'm currently working with MarkItUp! using Markdown syntax.
Unit testing for controllers and business/entity logic (using VS Test Project)
A cleaner separation between the web application and model/business/entity logic
Fully W3C compliant XHTML 1.0 Strict Output

I have no real time-frame for a 2.0 delivery as this is still a part-time project for me. Currently I have implemented most of the functionality for the TicketCenter, new ticket creation, and have just started work on the Ticket Viewer/Editor.

I have not marked out a potential 1.3 upgrade of the older code-base either, but my primary focus will be on the 2.0 MVC version.

TicketDesk - Design Philosophies Explained

Stephen M. Redd — Thu, 11 Jun 2009 06:45:37 GMT

It has been just over a year since I introduced TicketDesk over at CodePlex. While it hasn't taken the world by storm or anything, it did generate a lot more interest than I would have expected. There are several companies using TicketDesk in production environments, and there have been a few thousand downloads from other people that may be using it too.

While TicketDesk isn't generating the kind of download numbers that I'd want to base a software startup on, for an open source project it is what you might call "wildly successful".

If there was a major failure on my part with bringing TicketDesk to the public, it would be that I didn't do a good job explaining the ideas behind the overall design. So let me take a stab at explaining the philosophy behind TicketDesk.

The general idea behind TicketDesk was to take my 15 years or so of experience, much of it spent being frustrated by help desk issue trackers, and use that experience to design a different kind of help desk system; one that avoids those problems.

And believe me, I have a very long list of complaints with help desk systems!

I suppose the best way to explain it is to discuss the fundamental design idea then illustrate how TicketDesk implements them.

TicketDesk is an issue tracker for help desks... and that is all:

The help desk at most organizations will have many considerations aside from issue tracking. There are internal rank structures, chains of command, political issues, business practices, and financial considerations of all kinds. Unfortunately, the help desk is deeply involved in all of these things.

The mission of TicketDesk is to allow the help desk keep track of issues, and that is all it does.

TicketDesk does not attempt to understand your org chart.
It doesn't recognize user rank, status, or departmental affiliations.
It doesn't act as a time tracker.
It doesn't do billing.
It doesn't do project management.
It doesn't manage your inventory.
It doesn't handle your business process.
It doesn't do inner-departmental accounting.
And it absolutely does NOT care about your internal politics.

TicketDesk is made for internal help desks:

TicketDesk was designed exclusively for use by help desks supporting users within the same organization. It assumes there is a decent level of trust between all participants.

TicketDesk can be used in other environments, and there are plans for future versions to better enable external user scenarios.

You should carefully evaluate TicketDesk's features before attempting to use it in a customer-facing capacity. Also, you may find the features insufficient for organizations performing contracted support for users external to your organization.

Have as few data fields as possible for any given ticket:

This the most basic design ideas behind TicketDesk.

In most help desk systems there are just too many fields, and few of them turn out to be useful. During planning management is hyped-up about the advantages all those fields will bring, but it doesn't take long for the staff to learn that the free-text description field is the only reliable source of information (and even that is a dubious assumption).

So I've spent a lot of time thinking about the various fields common to similar systems.

There are many reasons why different fields fail, but it boils down to just three overall trends:

The fields may not relate to the user's specific problem. For example, Questions like what OS are you using aren't useful when the user is reporting a problem with their phone.
The end user is incapable of answering some questions. It isn't their fault, they aren't IT professionals so they just don't know the answers, especially to the more technical and detailed questions like "what is your OS version?" or "what is the printer model number?".
The end user is not qualified to answer some questions. This isn't a lack of skill, but just a lack of enough information. The classic example here is the priority field, which users cannot provide a meaningful answer. They don't know how their problem stacks up in relation to other issues; only IT can provide a useful answer here.

After exploring the problems I came to the conclusion that these just cannot be solved by software and it is unlikely that training, threat, or corporate policy would help either. The only solution is for the system to expect these problems, embrace them, and concentrate on helping the humans work around them on a case by case basis.

TicketDesk follows important philosophies:

Avoid asking any question where the user cannot be reasonably expected to answer with 100% accuracy no matter what kind of problem they are reporting.
Avoid asking questions that don't apply to nearly every possible situation being reported.

Thus TicketDesk asks as little from the user as possible. The system expects that the only useful field will be the free-text details field. Other fields do exist, but they are designed to be general in nature, optional, or are answered by the staff rather than the end-user.

Tickets should evolve as a natural conversation between the help desk and the end-user:

As discussed above, TicketDesk does not attempt gather a lot of detailed and quantifiable information up front. Instead it expects that help desk may have to ask for additional information.

Tickets are designed to be an ongoing two-way conversation with the user by borrowing heavily from web 2.0 and social networking concepts.

The activity area of tickets acts as a forum-style discussion board combined with an activity and history log. Every action that can be performed with a ticket solicits additional comments that also become part of the ongoing conversation.

The notifications system (and RSS feeds) ensures that both staff and users remain informed as the ticket progresses to completion. And TicketDesk makes it very simple to perform actions or add comments which encourages the staff to actually make frequent updates as they work through an issue.

The result should be a constant stream of information flowing between the user who submitted the ticket and the help desk staffer assigned to deal with it. Either party, as well as interested 3rd parties, can jump in at any time to add to the conversation.

Avoid Workflow & Routing Hell:

This is one of the more controversial of TicketDesk's design philosophies.

Most help desk systems have customizable and dynamic workflows with rule-based routing. This allows for a lot of control over how a ticket moves through the system.

There is no inherent "problem" with this kind of system in my experience. I have had the misfortune of working with system where the workflow customizations were insanely over-engineered to create horridly inefficient routes with many unnecessary steps, but when used wisely these features don't exactly present a "problem" directly.

Avoiding advanced workflow and routing is a design philosophy based mostly on technical considerations.

Workflow and routing is a nightmare to code, especially for a small development team with limited resources. The advantage of this kind of feature set though is rather limited. Other than making managers happy by having the system act as a policy-cop, there isn't much added value to the feature set.

Additionally, TicketDesk is designed to collect a very minimal set of fields, and doesn't expect end users to necessarily fill them in meaningfully so in TicketDesk there aren't many fields that can participate usefully with advanced workflows.

Instead I designed TicketDesk to use a static state-based workflow that should be valid in just about any organization. While simple, it is also unobtrusive and frictionless for the most part.

There have been some requests for workflow options that require only simple workflow customization options or a limited set of pre-defined optional rules. I plan to explore those ideas for inclusion in future versions of TicketDesk, but I have no plans to introduce a full-featured workflow customization or rule-based routing engine.

Allow organic categorization:

Most issue tracker systems provide the end user several with cascading category lists with context sensitive sub-categories. The options in sub-cats adjust according to previous selections to produce granular categorizations. As described before though, this just doesn't work that well because users don't get these selections right very often or the selections themselves are incomplete or outdated.

By omitting detailed categorization in TicketDesk, the searchability of tickets does become a little degraded and it can be more difficult to locate related tickets.

To give TicketDesk decent searchability without re-producing all the problems of traditional over-categorization; TicketDesk includes a web 2.0 style tagging mechanism. This allows users and staff both to organically add keywords to tickets as they desire.

Anyone can tag, but it is only really successful as a substitute for categorization if the help desk takes it on themselves to ensure that tickets are tagged well before being resolved. This takes some discipline and effort, but the up-side is that it produces a degree of searchability that can far exceed traditional categorization mechanisms. And best of all, there isn't a lot of administrative overhead to tagging since the system evolves and adapts all by itself over time.

Tagging is optional though, and many shops (mine included) choose not to make much good use of it. That's OK as TicketDesk doesn't rely on tagging, and a lack of it doesn't degrade the system's ability to perform the primary mission.

Email Notifications should not spam users:

This is a major problem in a lot of different software systems. There is a need to keep users informed of changes in a timely manner, but if you send notifications too frequently the system will overwhelm users.

When this happens people tend to ignore notifications and the important ones get lost in the noise.

To combat this problem, TicketDesk puts an enormous amount of effort into reducing the number of notifications sent to ensure that notification always conveys useful new information.

Here are the basic rules behind the email system:

Do not notify users about changes that they have made themselves. You know what you just did right?
Wait a few minutes before sending a notification to see if additional events involving the same ticket happen. If so, wait until changes slow down a bit, then consolidate the events into a single message.
Convey all of the information about the ticket in the message so users do not have to log in to see what is going on.
Attempt to guarantee delivery by supporting an intelligent re-try mechanism.

Depsite the fact that this system took a while to get implemented, it has proven good at keeping down the number of messages sent as well as eliminating unnecessary notifications.

The actual format of the notification message is still a little rough around the edges, but that will be worked out in future releases.

TicketDesk will not provide performance reporting:

This is also a controversial philosophy, but one that is absolutely essential to the success of the system.

TicketDesk will not implement any reports or data collection features assist management in measuring employee performance, or that could be used this way.

Anytime the issue tracker becomes a tool by which management measures employee performance the system ceases to have value. Instead it becomes an enemy of the users. Users will manipulating the data in the system to protect themselves and inflate their performance numbers. Anything that would make them "look bad" will be deliberately obscured or omitted from the system.

Researchers call this "management dysfunction", and it is a well established and thoroughly vetted reality. Despite that though, managers around the world still insist on attempting to automate the measurement of employee performance... which is ironic. If they were successful what would be the point in having managers on staff?

Your help desk is probably staffed by very smart people. People that love figuring things out and whose job is to be very good at figuring things out. How long will take them to learn how to game the system?

Even if you have some honest staffers that don't manipulate the system... it will punish those honest users while rewarding users that do manipulate the data to their advantage.

The purpose of TicketDesk is to facilitate honest and open communication between users and help desk. If the system is used to gather performance metrics then it cannot provide honesty nor openness and fails the primary mission.

To complete the failure, any performance metrics you "thought" the system was gathering turn out to be inaccurate and distorted, resulting in a system that can't measuring actual performance nor perform the other tasks it is designed for.

I first learned about this issue from Joel Spolsky, creator of the popular FogBugz bug tracking system, but have witnessed this same phenomenon in nearly every help desk environment I've ever worked with. .

You can read Joel's take on the issue yourself if you wish, he explains it better than I can.

Now... there are ways to do useful reporting in a way that doesn't lead to management dysfunction. But it takes very careful design where you deliberately create reports that cannot be used to show individual or group performance metrics. That is a slippery slope, and I have not yet had time to do the design for such reports yet.

I do have plans to add some reporting in the future, but the reporting will be carefully designed to prevent such abuses.

Read how Redd is Seeing Red Over Red Flag Bullshit!

Stephen M. Redd — Wed, 06 May 2009 04:34:00 GMT

So... I've moving to a new place, but I need to keep electricity turned on at the old place for a while.

Not an unusual case for a lot of people.

If you are selling a house for example, you might move before the house is sold off. You don't want to cut the power though. Or maybe you are moving and just want to take your time and keep both places for a while. Or maybe you just bought a vacation home and want to keep power at both locations permanantly... whatever...

So I was suprised when I called Duke Energy today and they refused to turn on power at my new place unless I also put in a disconnect order for the old place.

WTF?!?!

In my particular case, I am just getting my house repossessed, but I don't quite know when exactly yet. I didn't want to get caught flat-footed when the foreclosure went to the next step -- the "get the fuck out" step. So I've already moved to an apartment and need power turned on there.

But I don't want the old house to get damaged by the extreme south carolina summer before it is sold... afterall, the better it sells on the auction block the less I'm likely to get nailed for in the end. Plus it is kinda rude to let a perfectly fine house go to shit.

So I was calling Duke Energy to get new power service at my apartment, and was suprised that they were completely unwilling to connect power at both places. The rep said it had something to do with the "red flag laws that were passed in November".

And I'm like... the fucking what laws?

She gives me some poorly informed speach about a law that was supposed to prevent identity theft being passed and that to protect its customers from identity theft they no longer allowed concurrent service unless the second property was a rental property.

I asked exactly which law this was and if it was a state or federal law, but she apparently didn't know anything more about it.

I could have taken that as an answer, but at this point I'm horridly curious. There have to be a LOT of people that pay for power in more than one residence at the same time.

So I'm asking... so what do I do if I own two houses?

And she says, well you'll have to disconnect power from one of them unless one of them is rental property.

I asked about how exactly this was supposed to help protect my identity, but the answer was incoherent (clearly the rep had no idea).

er....

I got cut-off from this customer service rep (cell phones rock that way!).

So I called back and got a different rep... and I went though it again.... and got the same answers again. And this rep was also unable to cite me the exact law or even tell me if it was federal or state law.

The only thing new I learned was that they could turn on power as long as they had a disconnect order for the other place, but I didn't have to disconnect right away. I could set the disconnect date in the future and run both concurrently for a while. And if I needed to change the disconnect date later, I could just call back and let them know.

So I just set a disconnect date for 6 weeks from now at the old place and she was more than happy to turn on service at the new place.

Before she finished up with me though, I followed up with a final question. I told her that I was thinking of buying a condo that was for sale and renting it out, but I'd want keep the power in my name. Since she'd mentioned rental propertiy I asked if there was anything special I needed to do when I needed to turn on service at that property.

She said, no... just call us and let us know it is a rental property and we can turn on your service within 72 hours.

WTF?!

ARRRRRGGGGggggg!

After I got off the phone, I went digginng to find out what fucked up law is responsible for this completely idiotic polilcy.

Eventually I found it.

It is a set of FTC "rules" (not exactly a law, but may as well be) passed back in 2003 that took effect in November 2008.

The intent of the rules are to get certain institutions to do stuff to keep people from using someone else's identity. Mostly it was aimed at financial institutions, but covers some other industries... like my power company apparently. The idea is that the instituations have to have mechanisms (which are poorly defined) in place to detect and prevent people from signing up for accounts using someone else's identity.

Seems harmless... and in 2003 it was pretty far-thinking regulation actually.

From what I can gather, the intention was for the institutions to actually check your ID out more thouroughly if a "red flag" situation is encountered (thus the street name of the rule). Red flags are situations that are suspisious and could be the result of an attempted identity theft.

OK!

As well intentioned as the FTC's idea is though, it has failed.

Duke Energy is either too incompetent to actually check your ID, or the FTC rules are simply too difficult to adhere to without risking penalties from the FTC... so Duke simply decided that the best way to deal with it was to unilaterally deny any request that would result in one of those red flag situations in the first place.

I can see that a red flag would get raised when the power company gets a request for new power service from a residential customer who already has an account for another location. I've seen it happen more than a few times. Someone kicks out a roomate or gets a divorce and the other party turns on power at the new place using their former co-resident's identity.

Sure... fine... nice of the FTC to want to discourage that crap.

But instead of just checking my ID like the FTC rules intended, the power company just feels safer not letting me have power at two locations. I guess that's cheaper than having to pay the FTC crap-loads of fines if they accidentally failed to protect an ID somewhere.

And even better, the FTC's new rules do absolutly nothing to stop that same aggreved roomate from going to another city with a diffrent power company and still using someone else's identity to hook up their power anyway.

And even more ironic... you can setup power using a stolen identity just by telling the rep on the phone that your new location is rental property. Apparently there are no requirments to prove that you are renting it out or anything... you just have to tell them you are and you can setup as many accounts as you want.

Good job guys! Thanks for looking out for us there!

ASP.NET MVC - After RedirectToAction call, the target action fails to render a partial view correctly

Stephen M. Redd — Wed, 25 Mar 2009 07:57:24 GMT

Another undocumented "feature" I ran into when playing with the MVC.

I have an ajax action link that sorts a list on the page. To accomplish this, the link performs an ajax request to the controller's Sort action and in the end redraws the part of the page contianing the list's data with the new sort settings.

No biggie... except that when using Mozilla Firefox, it never worked right. The list would be redrawn, but would always contain the entire page's content including the menus, headers, and all the other stuff.

This was another one of those things I had hoped would magically go away with the RTM of the MVC framework... but when it didn't I had to go figure it out.

The process here was a little more complex than what you'd see in the examples and tutorial apps, but not by much. Here is how it worked.

The controller has two actions: List and Sort

The List action reads the user's profile and gets data based on the user's sort and filter preferences. It will render a partial view containing just the list's data if the request is an ajax request, otherwise it renders the entire page view.

The Sort action simply updates the user's profile with their new sort preferences. Then it uses RedirectToAction to tell the browser to call the controller's List action again.... and the list action would do the actual list building using the updated settings.

Simple enough...

After some debugging though, I found that some browsers don't resend the necessary headers on a subsequent ajax request when redirected this way. So the list action would think the request was a non-ajax request and it would thus render the whole page view.

There was a lot of discussion about some late beta changes to the MVC framework's IsAjaxRequest() method, so I had hoped it was just a beta bug... but apparently not.

In order to get this to work reliably, I had to have the sort method add a setting to TempData if it was called from ajax, then have the list method check both the TempData setting AND the IsAjaxRequest() method.

Annoying, but at least there is a workaround.

ASP.NET MVC - Ajax partial update fails in IE when updating a table's contents

Stephen M. Redd — Mon, 23 Mar 2009 18:17:00 GMT

I chased my tail for a while last week after the final release of the ASP.NET MVC framework went RTM. I'd been having a few "issues" getting my app working right, especially in IE 8.

I'd been hopeful that the RTM releases of either IE 8 or the MVC framework would magically fix these problems for me, but after both went to RTM last week I discovered that I was going to have to tackle the problem myself.

The main problem was that, in IE 8, I was unable to update a table when I was using Ajax to fetch a partial view containing the table's contents. I had been planning to use this technique to handle paging and sorting of the table's data and for an auto-refresh of the data periodically.

This worked fine on other browsers, but with IE 8 this always threw an unusually crytpic and unformative Javascript error (an "unknown exception") and the update would not complete.

I did finally get to the bottom of the problem...

As it turns out it has nothing to do with MVC or the beta of IE 8. As it turns out, I was getting bitten by a very old limitation that all versions of IE back to IE 4 share in common.

The Ajax mechanism that performs the partial update operates by simply replacing the innerHTML value of the target elment with whatever it fetches back from the server... in my case it would replace the table's contents with new rows it got from the server.

But IE doesn't support using the innerHTML method on tables directly. After some digging I actually came across an explaination for this limitation from the guy who actually wrote the innerHTML method. If you aren't interested in why, just know that he had really good reasons.

Once I knew that it was the innerHTML property of the table that was the problem, it wasn't too much trouble to fix. I just wrapped the table in a div tag, and set the MVC application's ajax call up so it would targeted the div instead of the table.

Not a new problem, but the fact that I was working with both a new browser and the new MVC framework made it hard to know I was dealing with a more general problem in the first place.