Tales from the Evil Empire

Etching sketches with a Netduino Go

Bertrand Le Roy — Wed, 02 May 2012 01:40:00 GMT

Netduino Go is designed for much more than toy projects, but one has to recognize that toy projects are fun and also great learning experiences. Today, I want to take you through the process of reproducing the behavior of a famous toy that you will surely recognize. That toy, that has helped many of us realize what poor motor skills we possess, has two knobs that control the horizontal and vertical coordinates of a pen that draws into the dust on the back of a simple screen. It’s a great mechanical device that is part of western culture.

Our version will use two potentiometers, a 320x240 LCD display, a button to erase the screen, and of course a Netduino Go. The hardware setup looks like the following:

We connected the button to port 2 (but it doesn’t matter as there’s only one), the two potentiometers to ports 3 and 4, and the display to port 5.

We can initialize this setup with the following code:

private Potentiometer Vertical = new Potentiometer(GoSockets.Socket3);
private Potentiometer Horizontal = new Potentiometer(GoSockets.Socket4);
private Button Button = new Button();
private VirtualCanvas Canvas = new VirtualCanvas();

To initialize the display, I wrote this method, which initializes a VirtualCanvas object on socket 5, sets the orientation of the display to landscape, fills the screen with white and flushes the command buffer:

public void InitCanvas() {
    Canvas.Initialize(GoSockets.Socket5);
    Canvas.SetOrientation(Orientation.Landscape);
    Canvas.DrawFill(white);
    Canvas.Execute();
}

The VirtualCanvas object can be seen as the driver for the display. It’s a simple vector API, but a crucial point is that these vector drawing primitives are not executed on the Netduino. Instead, they are serialized, sent to the display and executed there. This keeps communication between the microcontroller and the screen to a minimum, and frees the Netduino main board computing power. Essentially, you have a graphics coprocessor bundled with that screen. This is also why you need to call Execute when you want a command batch to be sent to the display and drawn.

To do the actual drawing, all we need is to read the values from both potentiometers and convert them into screen coordinates:

public int ReadY() {
    return (int)(Vertical.GetValue() * Canvas.Height);
}

public int ReadX() {
    return (int)(Horizontal.GetValue() * Canvas.Width);
}

And then render that on the screen:

Canvas.DrawLine(currentX, currentY, x, y, black);
Canvas.Execute();

I’m using DrawLine here and not DrawPixel so that the drawing looks continuous even if the potentiometers are rotated very fast. This requires remembering the previous coordinates, which is fairly simple to do.

All this is pretty easy really. You can see it in action in the following video. To keep things exciting, I threw in some debugging and live command drawing…

The source code for this can be downloaded here:
http://weblogs.asp.net/blogs/bleroy/Telecran.zip

All parts used in this demo can be purchased from Nwazet.

Electronics for developers with Netduino Go

Bertrand Le Roy — Fri, 20 Apr 2012 22:59:00 GMT

Wouldn’t it be great if you could build your own stuff? Microcontrollers let you do that, but they usually require dealing with a lot of complexity and unknowns. If you’re a software developer, chances are you don’t really know how to use a capacitor, or how a transistor works, even though it is at the heart of all computers.

Well there is a way now to develop awesome hardware projects without knowledge of electronics, just by connecting modules together. Netduino Go is a platform that allows for that without constraining the types of projects you can build. Today the number of available modules is still limited (the platform was released only this month), but it is expected to grow fast.

In this post, I’ll introduce you to the platform, with the assumption that you know programming but not electronics.

So let’s get started. What you need to follow along this post is the following:

A Netduino Go
A potentiometer module
A RGB LED module
A version of Visual Studio (C# Express is enough)
The .NET Micro Framework 4.2
The Netduino SDK for 64-bit or 32-bit depending on your version of Windows

First we’ll create a new project, selecting the Netduino Go Application project type:

This creates a new project with references to assemblies managing the .NET Micro Framework, the Go Bus and the Netduino’s ports:

We still need to add specific support for the two modules we’re going to use, the RGB LED and the Potentiometer. For that, let’s right-click on References and add the two assemblies we need:

Once that is done, we can start creating objects representing our modules:

using System;
using NetduinoGo;

namespace FirstStepsWithNetduinoGo {
    public class Program {
        public static void Main() {
            var led = new RgbLed();
            var pot = new Potentiometer();
        }
    }
}

This code works without specifying a port because Netduino Go knows to take the first module it finds that identifies as the right module type. Each constructor has an overload that takes a GoSockets enumeration so we could have written:

using System;
using SecretLabs.NETMF.Hardware.NetduinoGo;
using NetduinoGo;

namespace FirstStepsWithNetduinoGo {
    public class Program {
        public static void Main() {
            var led = new RgbLed(GoSockets.Socket2);
            var pot = new Potentiometer(GoSockets.Socket1);
        }
    }
}

We now have objects representing our module, but we still need to do something with them. Let’s start by turning on that LED. But before we do, let’s review our hardware setup:

The potentiometer is connected to socket #1 on the Netduino, and the RGB LED is on socket #2. Now we need to connect a micro-USB cable between the computer and the Go. This will both power the whole thing and give us the channel we need to deploy and debug our code. The first time you do that, it will also install the driver for the Go on your computer.

Let’s code:

led.SetColor(255, 212, 42);

In order to deploy that code, we need first to make sure that the project properties are set to deploy on USB:

Once this is done, we can hit F5 and see that LED turn on…

And… You should have gone blind by now. That LED is incredibly bright. Our next step is going to add some code so that the potentiometer can control the intensity of that light:

var led = new RgbLed();
var pot = new Potentiometer();
int red = 255, green = 212, blue = 42;
while (true) {
    var intensity = pot.GetValue();
    led.SetColor(
        (byte)(red * intensity),
        (byte)(green * intensity),
        (byte)(blue * intensity));
}

Deploying this code gives the following result:

And of course, this is visual Studio, so we have full debugging support, as can be seen in the video.

And that’s it for today. This should get you started with Netduino Go. And we didn’t have to solder anything… Next time, we’ll play with a touch screen.

Overriding the Pager rendering in Orchard

Bertrand Le Roy — Sat, 31 Mar 2012 04:41:00 GMT

The Pager shape that is used in Orchard to render pagination is one of those shapes that are built in code rather than in a Razor template. This can make it a little more confusing to override, but nothing is impossible.

If we look at the Pager method in CoreShapes, here is what we see:

[Shape]
public IHtmlString Pager(dynamic Shape, dynamic Display) {
    Shape.Metadata.Alternates.Clear(); 
    Shape.Metadata.Type = "Pager_Links";
    return Display(Shape);
}

The Shape attribute signals a shape method. All it does is remove all alternates that may exist and replace the type of the shape with “Pager_Links”. In turn, this shape method is rather large and complicated, but it renders as a set of smaller shapes: a List with a “pager” class, and under that Pager_First, Pager_Previous, Pager_Gap, for each page a Pager_Link or a Pager_Current, then Pager_Gap, Pager_Next and Pager_Last.

Each of these shapes can be displayed or not depending on the properties of the pager. Each can also be overridden with a Razor template. This can be done by dropping a file into the Views folder of your theme. For example, if you want the current page to appear between square braces, you could drop this Pager-CurrentPage.cshtml into your views folder:

<span>[@Model.Value]</span>

This overrides the original shape method, which was this:

[Shape]
public IHtmlString Pager_CurrentPage(HtmlHelper Html, dynamic Display,
    object Value) {
    var tagBuilder = new TagBuilder("span");
    tagBuilder.InnerHtml = Html.Encode(Value is string ?
        (string)Value : Display(Value));
            
    return MvcHtmlString.Create(tagBuilder.ToString());
}

And here is what it would look like:

Now what if we want to completely hide the pager if there is only one page? Well, the easiest way to do that is to override the Pager shape by dropping the following into the Views folder of your theme:

@{
    if (Model.TotalItemCount > Model.PageSize) {
        Model.Metadata.Alternates.Clear();
        Model.Metadata.Type = "Pager_Links";
        @Display(Model)
    }
}

And that’s it. The code in this template just adds a check for the number of items to display (in a template, Model is the shape) and only displays the Pager_Links shape if it knows that there’s going to be more than one page.

ZenGallery: a minimalist image gallery for Orchard

Bertrand Le Roy — Sat, 31 Mar 2012 04:13:00 GMT

There are quite a few image gallery modules for Orchard but ~~they were not invented here~~ I wanted something a lot less sophisticated that would be as barebones and minimalist as possible out of the box, to make customization extremely easy. So I made this, in less than two days (during which I got distracted a lot).

Nwazet.ZenGallery uses existing Orchard features as much as it can:

Galleries are just a content part that can be added to any type
The set of photos in a gallery is simply defined by a folder in Media
Managing the images in a gallery is done using the standard media management from Orchard
Ordering of photos is simply alphabetical order of the filenames (use 1_, 2_, etc. prefixes if you have to)
The path to the gallery folder is mapped from the content item using a token-based pattern
The pattern can be set per content type
You can edit the generated gallery path for each item
The default template is just a list of links over images, that get open in a new tab
No lightbox script comes with the module, just customize the template to use your favorite script.
Light, light, light.

Rather than explaining in more details this very simple module, here is a video that shows how I used the module to add photo galleries to a product catalog:

Adding a gallery to a product catalog

You can find the module on the Orchard Gallery:
https://gallery.orchardproject.net/List/Modules/Orchard.Module.Nwazet.ZenGallery/

The source code is available from BitBucket:
https://bitbucket.org/bleroy/nwazet.zengallery

Drawing transparent glyphs on the HTML canvas

Bertrand Le Roy — Sat, 17 Mar 2012 06:38:59 GMT

The HTML canvas has a set of methods, createImageData and putImageData, that look like they will enable you to draw transparent shapes pixel by pixel. The data structures that you manipulate with these methods are pseudo-arrays of pixels, with four bytes per pixel. One byte for red, one for green, one for blue and one for alpha. This alpha byte makes one believe that you are going to be able to manage transparency, but that’s a lie.

Here is a little script that attempts to overlay a simple generated pattern on top of a uniform background:

var wrong = document.getElementById("wrong").getContext("2d");
wrong.fillStyle = "#ffd42a";
wrong.fillRect(0, 0, 64, 64);
var overlay = wrong.createImageData(32, 32),
    data = overlay.data;
fill(data);
wrong.putImageData(overlay, 16, 16);

where the fill method is setting the pixels in the lower-left half of the overlay to opaque red, and the rest to transparent black.

And here’s how it renders:

As you can see, the transparency byte was completely ignored. Or was it? in fact, what happens is more subtle. What happens is that the pixels from the image data, including their alpha byte, replaced the existing pixels of the canvas. So the alpha byte is not lost, it’s just that it wasn’t used by putImageData to combine the new pixels with the existing ones.

This is in fact a clue to how to write a putImageData that works: we can first dump that image data into an intermediary canvas, and then compose that temporary canvas onto our main canvas. The method that we can use for this composition is drawImage, which works not only with image objects, but also with canvas objects.

var right = document.getElementById("right").getContext("2d");
right.fillStyle = "#ffd42a";
right.fillRect(0, 0, 64, 64);
var overlay = wrong.createImageData(32, 32),
    data = overlay.data;
fill(data);
var overlayCanvas = document.createElement("canvas");
overlayCanvas.width = overlayCanvas.height = 32;
overlayCanvas.getContext("2d").putImageData(overlay, 0, 0);
right.drawImage(overlayCanvas, 16, 16);

And there is is, a version of putImageData that works like it should always have:

My Orchard comment notification rule

Bertrand Le Roy — Fri, 02 Mar 2012 06:02:53 GMT

The Rules module in Orchard enable you to set-up “if this then that” types of rules. The system is fully extensible in terms of what “this” and “that” are of course, but Orchard comes with everything you need out of the box to set-up comment notifications. Let’s create this rule:

Before you begin, you need to make sure you have the rules features installed and enabled.

For comment notifications, “this” is: “when content with type comment is published”, and “that is: “send an e-mail”.

In order to reproduce this, create a new rule named “Content Notification”. Add an event, choose “Content Published” and select “Comment” as the published type:

Now we will add a “Send e-mail action”. As the recipient of the e-mail, I usually set “Site Admin”, but many people will want the mail to be sent to the owner of the blog. To get that result, you can select “Other” with the following pattern:

{Content.CommentedOn.Container.Author}

As the subject of the e-mail, I use this pattern:

nwazet.com: A new comment has been posted by {Content.CommentAuthor}

Finally, as the body, I use the following:

<p>New comment by {Content.CommentAuthor} on {Content.CommentedOn.DisplayText}</p>

<p>{Content.CommentMessage}</p>

<p><a href="http://nwazet.com/admin/comments">Moderate</a></p>

Once you’ve saved all that, don’t forget to enable the new rule, and you should now get an e-mail every time someone posts a comment. Oh, one last thig: you need e-mail settings to be correctly set-up for this to work, naturally.

Leaving the Evil Empire

Bertrand Le Roy — Thu, 01 Mar 2012 06:14:40 GMT

TL;DR: I’m leaving Microsoft to found my own company. My involvement in Orchard continues unchanged.

It’s been almost nine years. Nine years at Microsoft, first as a developer, and then as a program manager. I’ve worked on ASP.NET versions 2.0 to 4.0, ASP.NET Ajax, Ajax Control Toolkit, then on integrating jQuery into the Microsoft developer’s toolbox, and of course, on Orchard. I’ve made mistakes, errors of judgment, but I like to think I’ve learned a lot along the way, that my blog has helped many who were learning or struggling with the technology, and that I’ve built a few really cool things.

Most of all, I’ve changed.

When I got hired by Scott Guthrie et al., Microsoft was for me the Mecca of software development, the place where the amazing, superbly integrated software that I loved was made. I thought ASP.NET was the best group at Microsoft, and the most innovative. It was the zenith of my career. So I thought. During those nine years, I saw how the sausage was made, and I understood that innovation was and should be less centralized, and that open-source and community-driven software was a lot more exciting and promising than anything that a single actor could come up with, no matter how big it was. Maybe because it was so big.

At the same time, the world changed.

Almost all sectors of human activity have started to move away from a hierarchical, top-down model, to a distributed one. It takes the form of open source, of peer-to-peer, or of social networks.

I have a conviction that this is only the beginning of a profound change in our societies, that innovation is considerably more likely to occur in cooperative networks of small structures than in large competing corporations.

All this, plus a number of ideas that I can’t possibly develop at Microsoft, led me to the decision to quit my job and to found my own company.

So what does this mean for Orchard?

Not much. Let me explain what Orchard means to me. Before Microsoft hired me, I had spent two years building a CMS on ASP.NET, that was very targeted at rationalizing our web site development business. I knew at the time that most web sites are first and foremost content-driven. In the ASP.NET team, the focus was very different: the technology was built for business applications, not really web sites. From the start, I advocated for an approach that gave contents a more prominent place, ideally through a new content-centric framework. Eventually, others came to similar conclusions from different starting points, and the time became right for Orchard to begin. I jumped in enthusiastically.

In other words, Orchard is something I’ve wanted to do for a very long time, and I care about it very much. We still have a lot of work to do on the platform, and I’m going to be a part of that.

Orchard is also one of the first projects started by Microsoft to make such a full transition to a community-driven project. From our beginnings where 100% of the code was produced by a team of a dozen Microsoft employees, we are now able to produce amazing releases such as Orchard 1.4 with only two full-time employees, because the community has stepped in to produce the new features. Governance of the project has been transferred to an elected steering committee.

For me to leave Microsoft and become a part of this community only makes sense.

My new company, Nwazet, is focused on Open Source software and hardware. The hardware part won’t surprise you much if you’re a regular reader of this blog. With my business partner and friend Fabien Royer, we’re in the process of building amazing new products around Netduino microcontrollers.

And… That’s about it. I’m very excited about this change. If you or your company need help on Orchard projects, custom electronics or both, hire us!

More than one driver for a single Orchard part

Bertrand Le Roy — Fri, 24 Feb 2012 07:01:10 GMT

Drivers in Orchard are responsible for taking content parts and using them to generate shapes for the rendering engine to transform into HTML. A little known fact is that there can be more than one driver for any given part. You might be wondering what this can be used for: one shape per part seems like a reasonable assumption.

I’ll show one case where this ability to add a driver to the one that already exists came in really handy.

One of the sites I’m working on is making heavy use of taxonomies. The customer wanted to add management screens for the order of terms in any given taxonomy, that had some interesting specific features. We wanted a couple of links to that additional screen right inside the existing UI for a taxonomy field. I could have created a new part, but it would have never made sense unless a TermPart was already present. Orchard is so extensible in fact that extensions are extensible, so in a completely separate module from the original taxonomies, I created a new ContentPartDriver<TermPart>:

protected override DriverResult Editor(TermPart part, dynamic shapeHelper) {
    return ContentShape("Parts_Taxonomies_Term_OrderLink",
            () => shapeHelper.EditorTemplate(
                TemplateName: "Parts/Taxonomies.Term.OrderLink",
                Model: part,
                Prefix: Prefix));
}

Both drivers will run, and they will both emit their own shape. Each of those shapes can in turn be moved around using placement.info. This driver and the template to render the new shape is all I had to write. Here is the end result:

This is a pretty nifty way to enrich existing admin UI for any content type, don’t you think?

About Orchard Governance and Microsoft

Bertrand Le Roy — Mon, 30 Jan 2012 08:29:03 GMT

Back in September, we did something with Orchard that is kind of a big deal: we transferred control over the Orchard project to the community.

Most Open Source projects that were initiated by corporations such as Microsoft are nowadays still governed by that corporation. They may have an open license, they may take patches and contributions, they may have given the copyright to some non-profit foundation, but for all practical purposes, it’s still that corporation that controls the project and makes the big decisions.

That wasn’t what we wanted for Orchard. We wanted to trust the community completely to do what’s best for the project. This is why we organized elections for our new Steering Committee and had five members elected. Anyone who had ever posted anything on our forums that wasn’t spam could be a candidate, and had five votes to cast.

We got 9 candidates, and after a week of vote casting, we had our committee elected (see http://orchard.codeplex.com/discussions/271355 for the details of the vote). Our five members are elected for a year and they each get a vote in any major decision. Two out of those five are Microsoft employees, which means the company doesn’t even have a majority (even though I have a veto right that I never used, as the elected “benevolent dictator” of the group). We have public online meetings every week.

The committee’s role is to give the project’s strategic direction, and to decide what feature areas are going to be in the next releases. It is not to make technical decisions or build the product (more on that in a moment). The weekly meetings are used to give status on current development, do bug triage and make decisions as needed.

This was not just the right thing to do. It did have a very visible impact on the involvement of non-Microsoft developers. Shortly after the election, we started calling for teams to form around our 1.4 release’s features. Feature teams are being given a feature scope by the committee, and they are then free to make their own decisions on technical design. They regularly interact with the committee, which gets to decide when the features make it into the main branch.

This was quite successful as we are preparing to launch what will be the first release of Orchard where entire features have been written by the community. In fact, 3 out of 4 of the new feature areas of 1.4 have been written mostly by non-Microsoft developers.

To my knowledge, this makes Orchard the most open of all Microsoft-initiated Open Source projects. It now belongs entirely to the community, and absolutely everything is not only done publicly but also in a way where community participation is actively sought after and encouraged.

My hope is that as the project becomes more and more successful, our model gets copied by other corporate-initiated projects, at Microsoft and elsewhere. I would love to hear your thoughts, dear readers, on this.

What’s the deal with Orchard and NuGet? Does Orchard suck?

Bertrand Le Roy — Wed, 07 Dec 2011 02:34:02 GMT

Damn, Rob saw right through the PR lies of Phil Haack about the new NuGet gallery and figured it all out. Kidding, kidding. For your enjoyment and context, here is Rob’s post:
http://wekeroad.com/2011/12/06/nuget-and-orchard/

Phil made a thoughtful answer to the post where he gives the rationale behind the decision:
http://wekeroad.com/2011/12/06/nuget-and-orchard/#comment-380571113

What I want to address here is what that means (or not) about Orchard.

“Why wasn’t it fast before?”

Because by default, back when the NuGet gallery was first built, Orchard was grossly under optimized. We made a lot of progress, but it still is under optimized out of the box --in a number of ways-- which is why we’re having this discussion for the 1.4 release:
http://orchard.codeplex.com/discussions/274921

One thing you cannot accuse the Orchard team of is premature optimization… But seriously, there are options available today: installing the cache module, or the combinator module. Or custom, in-site optimizations.

So could the NuGet team have optimized its gallery instead of mobilizing several engineers over a few months to rewrite it? Sure they could have, but it so happened that they didn’t, for reasons Phil explained better than I could.

As a matter of facts, our own gallery site, which just passed the million-download mark, still uses that same codebase that NuGet was using before. The new NuGet gallery doesn’t display total download stats (because the home page is now static html) but I believe it’s around 3 million so we’re looking at a similar scale here. Just visit http://gallery.orchardproject.net/ and you’ll see that it’s performing quite well. So what’s the deal here?

Well, we did spend the time to apply a few optimizations:

We profiled the code and fixed the biggest bottlenecks. Those fixes, of course, now benefit all Orchard users.
We profiled the database queries (using Ayende’s excellent NHProf) and optimized the biggest offenders.
We activated output caching.

All optimizations that NuGet never saw applied. And well, see for yourself, it’s not too bad. We’re not quite done, there is a whole lot more we can and will do, with what limited resources we have (hint: a lot less than NuGet).

We’re not the only ones: more and more big sites are going online with Orchard, with good performance.

But Rob also says something interesting and expresses a perception that I’ve seen many times:

“I’ve deployed Big CMS’s before as a solution for clients and every single time we decided to move away. They’re great for getting off the ground – but after a while there’s just too much friction.”

That is true of some CMS, but not all. In the case of Drupal or Orchard (not what Rob used, I checked), my own experience and that of many others is that because the content model is factored at precisely the right level of abstraction, it’s actually sometimes harder to get off the ground, but the friction diminishes over time. Where a more traditional application requires you to pile functionality on a relatively monolithic data model, modern CMS allow you to add functionality in a completely decoupled and unobtrusive way that puts the composition exactly where it needs to happen, somewhere between individual properties and full types.

Source-controlled database backups

Bertrand Le Roy — Mon, 05 Dec 2011 01:31:00 GMT

I’m not very satisfied with traditional database backup solutions. It seems like almost no progress was made since SQL Server 6.5 (at least for the built-in tools). They are still outputting monolithic binary backup files that only do the job if the only thing you expect from backups is the ability to recover from catastrophic failures. I expect more, but before I explain, we need a disclaimer:

I am no expert of database backup and may very well miss some crucial points or some aspects of the current state of the art. The solution exposed in this post is the result of my own experiments and has no pretense at exhaustiveness or even reliability. It’s something I use on my own simple projects, and no more. Database backups are serious business and your data is valuable. Use this at your own risk.

What I wanted from my database backup solution can be summarized as follows:

Works on Azure databases (can you believe that as I’m writing this there is no valid built-in backup solution on Azure?)
Backups are not stored in the same place as the database
Has more than one state to rollback to
Can be automated
Easy to restore
Can make diffs between backup states
Can merge backup states

The last two, diffs and backups, can seem a little peculiar and require some explanation. I work on several Orchard projects, where a team of developers collaborate. The team is virtual, with people in the US, and in various places in Europe. Because Orchard is a CMS, the database contains more than just data: it also stores the configuration of the site, something that developers modify as part of their routine job. Since it is impractical to work off a central database because of latency, and also because one developer can break everyone’s instance with a bad change, we need multiple copies of the database, and a way to reconcile them.

What works in a distributed fashion, resolves conflicts, and stores history? Why, a Distributed Version Control System of course! This is what gave me the idea of storing my database backups in a private repository on BitBucket, using Mercurial as the DVCS.

In order to be able to do diffs and merges, we need a text-based persistence format. I chose to use the bcp command-line utility to emit that format because my past experience of it has been that it’s fast and reliable. The code could in principle use the bulk copy SQL commands, but having never worked with them I went for bcp. If someone wants to modify the code and make a pull request using bulk copy that’s fine by me. Bcp produces flat text files for each table that are very easy to diff and merge.

My solution, HgDBackup, comes with two command-lines that automate bcp, one for backup and one for restore. HgDBackup.exe enumerates the tables in the database and spits out one text file per table. HgDRestore.exe enumerates the tables, tries to find a text file with the same name, and if found, drops all rows in the table before importing the contents of the file into the table.

The schema of the database is not backed up, so you need to do that as a separate operation every time it changes, or as a separate automated task.

In the distributed development environment I described above, the process to take a new version of the database will be as follows:

execute HgDBackup
commit local changes
pull and update from the repository
execute HgDRestore
run tests
commit if a merge was necessary
push to repository

Both commands have built-in help that you can bring up with hgdbackup /? or hgdrestore /?.

The backup command responds to the following flags:

/Server or /S: the address of the server to backup (e.g. tcp:myazuredb.database.windows.net or .\SQLExpress)
/Database or /D: the name of the database to backup (e.g. Orchard)
/User or /U: user name to use on the database
/Password or /P: password to use on the database
/Output or /O: output directory where the tables will be dumped (e.g. C:\backup)
/Template or /T: connection string template

The restore command responds to the same flags, except that Output is replaced with Input.

The default connection string pattern is SQLExpress with integrated security (so no password has to be on the command-line).

For example, here is what I use to backup one of my Azure databases:

HgDBackup.exe
  /S:tcp:something.database.windows.net /D:Orchard
  /U:mysubscription@weirdstuff /P:**********
  /O:C:\Projects\nwazet.Backup\Tables
  /T:Server={0},1433;Database={1};User={2};Password={3};
     Trusted_Connection=False;Encrypt=True;

And here is what I use to import that backup into my local SQLExpress instance:

HgDRestore.exe
  /S:.\SQLExpress /D:Orchard
  /I:C:\Projects\nwazet.backup\Tables

That nwazet.backup directory is a clone of a private repository on BitBucket. It’s of course important that the repository is private because databases contain secrets.

The code is available on BitBucket:
https://bitbucket.org/bleroy/hgdbackup

You may also download the already compiled commands:
http://weblogs.asp.net/blogs/bleroy/HgDBackup.zip

And remember, use at your own risk…

From ScrewTurn Wiki to Markdown

Bertrand Le Roy — Wed, 26 Oct 2011 05:56:03 GMT

I'm in the process of moving the Orchard documentation site from ScrewTurn Wiki to a Mercurial + Markdown system, where revisions are managed through source control tools instead of a fully online wiki. We see quite a few advantages in doing that, but that's a story for another post.

Today, I just want to post on that quick and dirty tool that I've built to translate the ScrewTurn Wiki markup into Markdown. It's not a masterpiece in any way, it's not exhaustive, but I thought it may be useful to others, so here it is…

https://bitbucket.org/bleroy/screwturn2markdown/src

State of .NET Image Resizing: how does imageresizer do?

Bertrand Le Roy — Sat, 22 Oct 2011 22:18:00 GMT

I've written several times before about image resizing in .NET and how the various built-in solutions (GDI, WPF and WIC) compare in terms of quality, speed and size. I'll put the links to my previous articles at the end of this post for reference.

Several readers have since pointed me to the imageresizer library, which is pure .NET and thus has no problems running in medium trust. Medium trust is an issue that has plagued existing options, preventing many people from using the best available approach. I was doubtful though that a purely managed library could come anywhere near the native Windows libraries in terms of performance. The best way to find out, of course, is to run a benchmark. Fortunately, I still had the code for my previous benchmarks so I just had to add imageresizer code to it.

I excluded IO from this test by measuring the processing time for each library working on binary streams. This also levels the playing field as some of them start processing while they are reading a file, making it harder to separate file reading overhead from image processing. By working with streams, the comparison is more meaningful. The timing code is wrapped around only the meaningful code.

The first problem that I hit with imageresizer is that it has a strong dependency on System.Web. This is really unfortunate and very probably unjustified. The library should be split between the pure image processing bits, which should have no System.Web dependency, and the request handling and caching bits, which can depend on System.Web all they want.

Since I'm mostly interested in web applications, I changed my command-line code into an ASP.NET Pages application, which probably saved me some configuration headache. The code is linked to at the end of this post. One thing I have to say is that the API feels good, natural and reasonably concise, a nice change when compared with the noisy Interop-like code we had to write before. It's nice to have something that is native .NET rather than a thin layer on top of Windows APIs that stink of C.

var settings = new ResizeSettings {
    MaxWidth = thumbnailSize,
    MaxHeight = thumbnailSize,
    Format = "jpg"
};
settings.Add("quality", quality.ToString());
ImageBuilder.Current.Build(inStream, outStream, settings);
resized = outStream.ToArray();

I really liked that I don't have to do size calculations myself, and I can just give it a square or rectangle inside of which it has to fit. You have many options to customize that. It's unfortunate that the quality setting has to be set this way instead of being promoted to a 1st class property like other settings but I can live with it. Notice how the library can work with streams, which is very important for web apps. One thing I really didn't like though was that some APIs, such as Build, take untyped object parameters and then decide internally at runtime what to do with them. Why the author didn't use strongly-typed overloads here is a mystery to me.

So how does imageresizer do then? Well, mostly as expected. In terms of size, I found it to be a little greedier than its peers, but really nothing dramatic:

The quality of the images is where imageresizer shines. See for yourself (quality 85% for those images):

imageresizer	WIC

The images look sharper (in particular the hummingbird one), and a little bit more saturated. Because of that, some moiré effects seem more visible (the shot before last makes that the most obvious. Still, the resizing is overall the best I've seen so far.

Now of course, while managed code is usually quite fast, image processing is heavy in the sort of operation where native code has a definitive advantage. That is reflected in the results I got in terms of processing speed:

Imageresizer is clearly the slowest. That was expected, but now we know by how much. It is 31% slower than GDI (I excluded the 50 quality point here), and almost 4 times slower than WIC.

Of course, unless your application is doing lots and lots of resizing operations, speed may not be that important, in particular if you are doing appropriate disk caching. Imageresizer shines in that department by making disk caching super-easy. The library, after all, is very web centric and tries hard to do everything right for the web scenarios. We should not underestimate how hard it is to get web image resizing right.

That leaves us with the unfortunate conclusion that we still don't have an image resizing silver bullet. I'll try to summarize all that we know in a table that will hopefully help you make a decision in the context of your specific application:

	Imageresizer	WIC	WPF	GDI
Speed	*	****	****	**
Quality	*****	****	****	***
Medium trust	Yes	No	No	Yes
Supported	By author	By MS	No	No
.NET friendly	****	*	****	***
Web centric	Yes	No	No	No

Benchmark code:
http://weblogs.asp.net/blogs/bleroy/ImageResizeBenchmark.zip

Results:
http://weblogs.asp.net/blogs/bleroy/ImageResizerBenchResults.zip

Previous articles:

The imageresizer library:
http://imageresizing.net/

So what's new in Orchard 1.3?

Bertrand Le Roy — Wed, 05 Oct 2011 00:52:22 GMT

Orchard 1.3 was released yesterday night with some really neat features that I will outline in this post. I will come back in depth on some of those with full-length posts. Let's start with the simple but super-useful ones…

1. Draft Preview

Once you've created and saved a content item, there will now be a "Preview" link next to its summary:

Previously, it was impossible to view unpublished contents. This was one of the most requested missing features, and there it is.

2. Delete Content Types and Parts

Speaking about features high on the wish list, you can now delete the custom content types and parts that you built.

3. Markdown

Not everyone likes editing contents in Wysywyg editors, for various reasons. There has always been the possibility in Orchard of switching the flavor of the Body part, but the only available alternative to the default "html" so far has been "text", which was only giving you a plain text box. Now there is a new option, "markdown".

Markdown is a neat text syntax that has the advantage of being simple, rich and human-readable.

I especially like the code sample convention of starting a line with four spaces. Very handy for writing documentation. Oh, about that, did I mention that we are planning on switching the Orchard documentation from the current wiki to Markdown?

4. Title Part

Until now, the title of Orchard content items has been kept in the Route part. That was an inconvenience when you wanted to give a title to a content type that did not have a vocation to have its own URL, for example widgets.

Title is still there in the Route part, but what we did is introduce a new interface, ITitleAspect, that the Route part implements.

public interface ITitleAspect : IContent {
    string Title { get; }
}

In all the places where we were using the old title, we are now using the interface instead. If you need to add a title, but not a URL to your own content types, you can either make one of your parts implements ITitleAspect, or you can add the new Title part, that already implements the interface in a ready to use form.

5. Rules

Rules are a truly wonderful feature that enables you to add event-driven behavior to your site from the admin UI. Let's create for example a new "Comment Notification" rule and add a "content created" event that we'll configure to be triggered on the Comment content type:

Once this is done, we can add an action to be triggered when that event fires. We'll choose "Send e-mail", which will raise the following form, where we can specify who will receive the e-mail, and what will be the subject and body:

The subject and body of the e-mail can use tokens, which are another new feature that I'll get to in a minute.

Here is how the content creation event was wired in a handler:

OnCreated<ContentPart>(
  (context, part) =>
    rulesManager.TriggerEvent("Content", "Created",
    () => new Dictionary<string, object> { { "Content", context.ContentItem } }));

When a piece of content is created, an event is triggered on the rules manager, giving it the content item being created as the context. The event itself is implemented in a class that implements IEventProvider. I'll write more on creating your own events in a future post.

Actions are classes that implement IActionProvider. Both event and action providers can expose their own UI forms for their configuration, which brings us to the next feature…

6. Forms

So far, to expose admin UI, a module writer had to create controllers or drivers, view models and views. Starting with this release, they can use a much easier forms API. This is what rule events and actions are using to expose their UI. The forms API makes it possible to describe a form in a nice, compact and extensible form that is using, you guessed it, dynamic shapes.

Shape.Form(
    Id: "ActionEmail",
    _Type: Shape.SelectList(
        Id: "Recipient", Name: "Recipient",
        Title: T("Send to"),
        Description: T("Select who should be the recipient of this e-mail."))
        .Add(new SelectListItem { Value = "owner", Text = T("Owner").Text })
        .Add(new SelectListItem { Value = "author", Text = T("Author").Text })
        .Add(new SelectListItem { Value = "admin", Text = T("Site Admin").Text }),
    _Subject: Shape.Textbox(
        Id: "Subject", Name: "Subject",
        Title: T("Subject"),
        Description: T("The subject of the e-mail."),
        Classes: new[] { "large", "text", "tokenized" }),
    _Message: Shape.Textarea(
        Id: "Body", Name: "Body",
        Title: T("Body"),
        Description: T("The body of the e-mail."),
        Classes: new[] { "tokenized" }
        )
    );

What you see here is the form that is in the e-mail action picture above. Of course, each of the shapes in there will be rendered by a template that can be overridden (as usual). The advantage of such a description is that it can be enriched dynamically. A comments form for example could have a captcha inserted dynamically by another module, just by injecting a new shape in the form. I encourage you to dive into existing implementations to understand how you can use this new API, until we get the chance to write some good documentation.

7. Tokens

Tokens are system-wide variables whose values can vary depending on the context. They can be used in a variety of places, one of which is rules, as you can see from the example above (the subject of the e-mail that is sent when new comments are added uses a token to describe the name of the commenter). I've blogged about tokens before, so I encourage you to read these posts to know more:

http://weblogs.asp.net/bleroy/archive/2011/07/22/future-orchard-part-1-introducing-tokens.aspx
http://weblogs.asp.net/bleroy/archive/2011/07/27/future-orchard-part-2-more-tokens.aspx

8. New Content Manager Methods

Building contents by composing small parts is an extremely flexible approach, but it makes it more difficult to build efficient database queries. There is always caching, but we also needed to optimize things a bit. We did that by adding some new methods to the content manager, such as GetMany, which is taking multiple content ids in one query. We also added the Aggregate attribute that can be used to specify records that should be eagerly loaded. For example, the Tags property of the TagsPartRecord and the TagRecord property of ContentTagRecord are marked with the attribute, because when the tags are included with a content item, you don't want the act of actually getting the tags associated with the content item to be lazily executed (that is a SELECT N+1 condition). If you want the tag part, there is a very high chance that you'll want to actually access the tags themselves: the part in itself has nothing else of interest. With the attribute, the tags will be fetched at the same time and from the same query as the item itself.

9. The rest

There is a lot more in this release, such as RSS for lists (it's just there, not much to say about it), some localization improvements, the task lease API that enables better synchronization across web farms, and many, many bug fixes and performance improvements (in particular, if you use multi-tenancy, you should feel the difference). If you want to see everything that was fixed, take a look at the list on CodePlex:

http://orchard.codeplex.com/workitem/list/advanced?keyword=&status=Closed&type=All&priority=All&release=Orchard%201.3&assignedTo=All&component=All&sortField=Votes&sortDirection=Descending&page=0

What's next?

The current plan for the near future is to quickly release a 1.4 version with the focus on the new Projector feature that will enable you to query your contents in any way you want directly from the admin UI. This, I can assure you, is going to be a killer feature.

PIX-6T4, the DIY console, has an Orchard site

Bertrand Le Roy — Sun, 18 Sep 2011 05:54:29 GMT

If you're a regular reader of this blog, you may have noticed the electronics posts lately. With my friend Fabien, we've been building a little handheld console from readily available parts. It's also programmable in C#, using a library that we built. It's great fun really. Our plan is to build a kit and a book out of it, to help people learn the basics of digital electronics and C# programming.

We've built a site for the project over a few nights last week, using Orchard and hosting on Azure. I thought I'd throw in a little return on experience and talk about the modules we used.

The site can be found here: http://www.pix6t4.com

The first thing I did to build the site was to build the theme. I started from the built in Theme Machine, and just added the markup and styles for the header. I then changed the fonts from the stylesheet and layout files, added a few custom styles such as the atomic bullet lists or the round corner menus, some transparency for the body and that was pretty much it. The Theme Machine is extremely flexible and let me get exactly what I wanted without changing much markup at all. Most of the changes are CSS.

I then packaged and deployed my Azure image onto our Azure instance. We are using multi-tenancy in order to host multiple small sites on a single Azure instance, so I just created a new tenant for our new domain name. After the second or so it took me, I was able to go through setup for the tenant, and pointed the site to the same SQL Azure connection string I'm using for the main tenant, but using a difference table prefix.

Here are the modules we're currently using (I installed those on my dev machine and added them to the Azure solution so that they get uploaded with the image, as you can't install modules directly on Azure):

Google Analytics: very simple to use, just register the new site on Analytics and copy the provided script into the module's admin page. You are done, Google starts accumulating stats on your site from this moment.
Content Types: as you add modules to your Orchard site, you will often want to extend existing types with new parts. You need the content types feature for that.
Import Export: the Azure story for backups is, well, lacking at this point in time. We'll use export to work around this and download the contents of the site on a regular basis. It's not the same as proper automated backups but it's way better than nothing.
Vandelay Industries: well, I built this one. I has several useful features, such as favicon settings, Feedburner, tag clouds, etc.
YouTube Field: We added a Youtube field to the page content type, to make it super-easy to add video to our pages. I had to make a couple of modifications to the module (nice that all modules come with source code): I overrode the template so that it uses the iframe YouTube markup rather than object and embed. As a side note, the default markup has a Lady Gaga video hard-coded into the archaic <object> tag. Nobody noticed I suppose because no recent browser uses that nowadays. I also modified the driver so that it only returns a shape if a video has been configured and null otherwise. I will contribute back those changes of course.
Contact Form: works as advertised. You can create a contact form page, and every time someone fills the form, we'll get an e-mail notification. The module also has a report page where you can see all the contact requests. I did have to fix a bug in the driver to prevent double entries in the database. I will also contribute that change back.

And well, that is it really. It's a simple site, and it was a simple job to set it up, as it should be. :)