• A new version number represents a new set of resources. If you have to create a new version to introduce a breaking change in one resource, that change expands to all the resources.

For example. You have two resources /orders and /customers. You need to introduce a new version to accommodate an schema change in orders. That implies adding a new version number in the URL for v1/orders and v1/customers. Although customers is still the same resource, it’s now referenced as a new resource v1/customers.

• It’s hard to introduce backward compatibility changes. You might want to introduce improvements or changes that new clients can use without affecting existing ones. You can create a new version number for this, but it will represent some unnecessary overhead. Existing clients won’t be affected by the change so creating a new version does not seem to be right. Also, you will not want to keep the same version number as you will want clients to know which specific version they are targeting.

• It does not go along with the idea of introducing incremental changes. A new version number usually represents a major release. If you want to make those changes public as they become available, you need a new version number. However, you won’t want to create v1, v1.1, v1.2 for the overhead discussed in #2.

A better approach for versioning.

Use an http header to specify version. If no http header is specified in the request message, stick to the latest version.

1
2
3

/orders
accepts-version: 1.0
content-type: application/json

The “accepts-version” header represents the version the client can understand. If some changes were introduced in the resource representation that won’t affect the client, the service might be able to return it. Let’s say that you now have a new version 1.3 for /orders, which only contains backward compatibility changes. The server can return a header to inform that.

1
2

/orders
version: 1.3

The client will know a new version exists, which is also compatible with 1.0 so it can optionally upgrade to it. This approach also works for fine for dynamic languages or schema-less types like json.

For embedded URLs or browser support, the http header can be replaced by an optional query string parameter ?accepts-version or ?v to make it shorter.

As part of the project where I was working on, we had to automate many of these steps in the installation process so we came up with a set of utilities classes that call the underline Win32 APIS for generate the certificate and also do the required registrations for the namespace and port. The process for doing this with these classes is pretty straigforward as it is shown below,

1
2
3
4
5
6
7
8
9
10
11
12
13

var cert = X509Util.CreateSelfSignedCertificate(Environment.MachineName);

//Register a namespace reservation for everyone in localhost in port 9010
HttpServerApi.ModifyNamespaceReservation(new Uri("https://localhost:9010"),
  "everyone",
  HttpServerApiConfigurationAction.AddOrUpdate);

//Register the SSL certificate for any address (0.0.0.0) in the port 9010.
HttpServerApi.ModifySslCertificateToAddressBinding("0.0.0.0",
  9010,
  cert.GetCertHash(),
  System.Security.Cryptography.X509Certificates.StoreName.My,
  HttpServerApiConfigurationAction.AddOrUpdate);

All the code is now available for you in github SelfHostUtilities.

The first thing is to install AppFabric Caching. Wade Wegner wrote a very useful post describing all the required steps here.

Once AppFabric Cache is installed, you need to start the cluster and configure a new cache that will be used by our extension. Open up the Cache PowerShell console (Caching Administration Windows PowerShell in programs). Run the following command in the PowerShell console:

Start-CacheCluster

Create a new cache for our extension. Run the following command in the PowerShell console:

New-Cache OutputCache

At that point, we are ready to jump into the implementation of the extension. Before writing any code, we will use the AppFabric Caching client library, which is available as a Nuget package. You can find it in the repository under the name of “ServerAppFabric.Client”. The library written by Filip provides an extension point for the persistence providers called IApiOutputCache, so we will have to implement that interface.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

public class AppFabricCachingProvider : IApiOutputCache
{
    readonly static DataCacheFactory Factory = new DataCacheFactory();
    const string Region = "OutputCache";
    readonly DataCache cache;

    public AppFabricCachingProvider(string cacheName)
    {
        this.cache = Factory.GetCache(cacheName);
        this.cache.CreateRegion(Region);
    }

    public void Add(string key, object o, DateTimeOffset expiration, string dependsOnKey = null)
    {
        var exp = expiration - DateTime.Now;

        if (dependsOnKey == null)
        {
            dependsOnKey = key;
        }

        cache.Put(key, o, new[] { new DataCacheTag(dependsOnKey) }, Region);
    }

    public bool Contains(string key)
    {
        var result = this.cache.Get(key, Region);
        if (result != null) return true;

        return false;
    }

    public object Get(string key)
    {
        var result = this.cache.Get(key, Region);
        return result;
    }

    public T Get<T>(string key) where T : class
    {
        var result = this.cache.Get(key, Region) as T;
        return result;
    }

    public void Remove(string key)
    {
        this.cache.Remove(key, Region);
    }

    public void RemoveStartsWith(string key)
    {
        var objs = this.cache.GetObjectsByTag(new DataCacheTag(key), Region);
        foreach(var o in objs)
        {
            this.cache.Remove(o.Key, Region);
        }
    }
}

This implementation is pretty straightforward and uses the AppFabric client library for getting or storing data in the cache. All the data is stored as part of a region, which groups all the entries together to facilitate management. The name of the cache is passed as part of the constructor, so it will be provided at the moment of instantiating and configuring this extension.

The following code shows how the extension is configured,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

var config = new HttpSelfHostConfiguration("http://localhost:999");
config.Routes.MapHttpRoute(
      name: "DefaultApi",
    routeTemplate: "api/{controller}/{id}",
    defaults: new { id = RouteParameter.Optional }
);
var server = new HttpSelfHostServer(config);

config.CacheOutputConfiguration().RegisterCacheOutputProvider(() => new AppFabricCachingProvider("OutputCache"));

server.OpenAsync().Wait();

Console.ReadKey();

server.CloseAsync().Wait();

The name of the cache must be the same that we used before in the powershell console with the New-Cache command.

Finally, output caching can be configured in any existing controller using the [OutputCaching] attribute distributed with the Filip’s library.

1
2
3
4
5
6
7
8

public class TeamsController : ApiController
{

    [CacheOutput(ClientTimeSpan = 50, ServerTimeSpan = 50)]
    public IEnumerable<Team> Get()
    {
        return Teams;
    }

The implementation of this provider is available as a separate project in GitHub.

As part of this post, I will show how you can use that browser.js library to make Ajax calls to a ASP.NET Web API that authenticates the calls with Hawk.

The first thing is to define the Web API to call from javascript. For the sake of simplicity, we will use a very simple “Hello World” controller that returns the name of the authenticated user.

1
2
3
4
5
6
7

public class HelloWorldController : ApiController
{
  public string Get()
  {
    return "hello " + this.User.Identity.Name;
  }
}

As second step, we will configure the Hawk filter globally to authenticate the calls. This filter is part of the HawkNet integration project with Web API.

1
2
3
4
5
6
7
8
9
10
11
12
13

public static void Register(HttpConfiguration config)
{
  config.Filters.Add(new RequiresHawkAttribute((id) =>
  {
    return new HawkCredential
    {
      Id = "dh37fgj492je",
      Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
      Algorithm = "hmacsha256",
      User = "steve"
    };
  }));
}

Once we have the Web API configured, we will write the javascript to make the call, which will use browser.js to generate the hawk header on the client.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<script src="~/Scripts/browser.js"></script>
<script>
$(function () {
    var url = 'http://localhost:28290/api/HelloWorld';

  var credentials = {
        id: 'dh37fgj492je', // Required by Hawk.client.header
        key: 'werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn',
        algorithm: 'sha256',
        user: 'Steve'
    };

  $("#helloworld").click(function() {
              
      var header = hawk.client.header(url, 'GET', { credentials: credentials });

      $.ajax({
          dataType: "json",
          headers: { authorization: header.field },
          url: url,
          success: function (error, response, body) {
  
              $("#response").html(body.responseText);
          }
      });
  });
});
</script>

As you can see, the part that matters in the sample above is how you generate the hawk header by calling the “header” method in the client object provided by the browser.js library. That method receives the url, the http method and the credentials. The result of that call is the hawk authentication header, which is attached to the authorization header in the ajax call.

The complete sample can be found in the HawkNet github project

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

exports.index = function(req, res){

  var returnResponse = false;
  var data = {};

  getName(function(err, name) {
      if(err) return callback(err);

      data.name = name;

      if(returnResponse)
          res.render('index', { title: 'Express', data: data });

      returnResponse = true;
    });

  getPhone(function(err, phone) {
      if(err) return callback(err);

      data.phone = phone;

      if(returnResponse)
          res.render('index', { title: 'Express', data: data });

      returnResponse = true;
    });
};

The code above calls two functions to retrieve a name and a phone from different places asynchronously. The results of the calls are temporarily stored as members of the “data” variable. The ugly part is how the “returnResponse” variable is also used to notify that each function has done its part and a response can be returned. This contains duplicated code, it’s error prone and it can easily get more complicated as the number of async calls increase.

As interesting fact, there isn’t any built-in or native functionality to handle an scenario like this in Node.js. That’s where an module like “async” comes in place. “Async” is a swiss knife for async work in Node.js. It provides a ton of functions for coordinating asynchronous tasks such as the ones shown in next examples.

One of the functions you will find useful for an scenario like this is parallel. The parallel function receives an array of async functions to call, and invokes a final callback when all the functions have completed their part.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

exports.index = function(req, res){

  async.parallel([
      function(callback){ getName(function(err, name) {
          if(err) return callback(err);

          callback(null, name);
      })},
      function(callback){ getPhone(function(err, phone) {
          if(err) return callback(err);

          callback(null, phone);
      })}
  ], function(err, results) {
      
      var data = {
          name : results[0],
          phone: results[1]
      }

      res.render('index', { title: 'Express', data: data });
  });
}

Every function in the Array passed to the parallel function receives a callback argument, which must be called after the work is completed. That callback receives two arguments, an error if something went wrong and a result. All the collected results and errors are later passed to the final callback.

Another useful function is “each”, which is similar to “parallel”, but it iterates over an array and invokes a function representing the async work for every element in that array.

1
2
3
4
5
6
7
8
9
10
11

exports.index = function(req, res) {
  async.each([1, 2, 3], function(item, callback) {
          getDataForItem(item, function(err, data) {
              callback(null, data);
          });

  }, function(err, data) {

       res.render('index2', { title: 'Express', data: data });
  });
}

In the code above, the function is executed three times with the values “1”, “2” and “3”. A callback is also executed when the work is completed. The final callback is executed after all the functions invoked the callback.

This is a good way to coordinate async work in node.js, but you also have promises, which is not widely adopted yet in the platform as a way to represent async tasks. It’s implemented by a lot of modules out there, but it’s not something standard yet. A promise is an object that represents an asynchronous task. Among other things, this object represents a way to manipulate the task, determine when it is completed or to chain other tasks for example. The following example illustrates how a promise looks like in the Moongose module (example included in the Moongose docs),

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

promise = Meetups.find({ tags: 'javascript' }).select('_id').exec();
promise.then(function (meetups) {
  var ids = meetups.map(function (m) {
    return m._id;
  });
  return People.find({ meetups: { $in: ids }).exec();
}).then(function (people) {
  if (people.length < 10000) {
    throw new Error('Too few people!!!');
  } else {
    throw new Error('Still need more people!!!');
  }
}).then(null, function (err) {
  assert.ok(err instanceof Error);
});

Meetups is a Moongose object for executing queries against a MongoDB collection. “exec” is a method in the returned promise to start the call, and “then” is another method to chain other promises when the execution is completed. In this example, another promise to find people is executed after all the meetups have been found.

For some other scenarios in which your applications or services rely on JWTs for doing authentication/authorization, the Thinktecture Authorization Server represents a very nice alternative, which integrates really well with ADFS. The Authorization Server can act as a brige or broker between client apps that use OAuth 2.0 to get a JWT, and ADFS, which is used for authenticating the users.

The configuration of the Authorization Server is very simple. Once you get Authorization Server deployed in IIS, it uses Entity Framework code first to automatically generate the backend database for you on the first use. After that, you only need to change the configuration files to trust ADFS as it is shown bellow,

[Authorization Server Folder]\Configuration\identityModel.config

1
2
3
4
5
6
7
8
9
10
11

<issuerNameRegistry type="System.IdentityModel.Tokens.ValidatingIssuerNameRegistry, 
  System.IdentityModel.Tokens.ValidatingIssuerNameRegistry">
  <authority name="http://[your ADFS server]/adfs/services/trust">
      <keys>
          <add thumbprint="[signing cert thumprint]"/>
        </keys>
        <validIssuers>
          <add name="http://[your ADFS server]/adfs/services/trust" />
        </validIssuers>
  </authority>
</issuerNameRegistry>

[Authorization Server Folder]\Configuration\identityModel.services.config

1
2
3
4
5
6

<system.identityModel.services>
  <federationConfiguration>
      <wsFederation passiveRedirectEnabled="true"
 issuer="https://[your ADFS server]/adfs/ls/" realm="urn:authorizationserver" />
  </federationConfiguration>
</system.identityModel.services>

urn:authorizationserver is the Relying Party identifier I used for configuring Authorization Server in ADFS. In ADFS, you only to configure a new relying party with a WS-Federation endpoint, and set the URL of the Authorization Server. Make also sure the Relying Party identifier matches the one you configured in Authorization Server.

The following code shows how to get a JWT from Authorization Server using the Resource Owner Flow (The user is authenticated by Authorization Server in ADFS)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

private static string GetToken()
{
    ServicePointManager.ServerCertificateValidationCallback =
      (o, cert, chain, ssl) => true;

    HttpClient client = new HttpClient();

    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post,
      "https://[your auth server]/[App namespace in Authorization Server]/oauth/token");
    request.Headers.Add("Authorization",
      "Basic " + Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(
          string.Format("{0}:{1}", "[client id]", "[client secret]"))));

    request.Content = new FormUrlEncodedContent(new Dictionary<string, string>
    {
         {"grant_type", "password"},
         {"username", "[account]"}, //Windows account name without domain
         {"password", "[password]"},
         {"scope", "All"}
    });

    HttpResponseMessage response = client.SendAsync(request).Result;
    response.EnsureSuccessStatusCode();
    JObject json = JObject.Parse(response.Content.ReadAsStringAsync().Result);

    string accessToken = json["access_token"].ToString();
    string refreshToken = json["refresh_token"].ToString();

    return accessToken;
}

You can find more details in the Dominick’s blog

1
2
3
4
5
6
7
8

<microsoft.identityServer.web>
  <localAuthenticationTypes>
      <add name="Forms" page="FormsSignIn.aspx" />
      <add name="Integrated" page="auth/integrated/" />
      <add name="TlsClient" page="auth/sslclient/" />
      <add name="Basic" page="auth/basic/" />
  </localAuthenticationTypes>
<microsoft.identityServer.web>

Those handlers are for Forms Authentication (Username/Password in an html form), Integrated Windows Authentication, Client Cert Authentication, and finally Http Basic Auth Authentication. The order in this list determines the priority by default unless the priority has been changed in the execution context. The order in the execution context can be changed in multiple ways. For example, when you are doing WS-Federation, the relying party can pass an additional “WAuth” query string parameter with the expected authentication type such as “urn:oasis:names:tc:SAML:1.0:am:password” for Forms Authentication or “urn:ietf:rfc:2246” for Client Cert authentication, which overrides the priority in the list set on the configuration. The same thing can be done for SAML 2.0 by changing the authentication context in the SAML Request message or changing the URL.

However, as I said before, in one of the handlers run, you will not have a chance to run any of the other handlers unless you implement some workaround. For example, you might want to add a checkbox in the html form for Forms authentication to allow the user to authenticate with a client certificate. When the checkbox is clicked, you do an http redirect to ADFS but changing the URL this time to use this new authentication method. If the RP is using WS-Federation, that means a new redirect that includes the WAuth query string variable with the value “urn:ietf:rfc:2246”, or a redirect to “auth/sslclient/” in the case of SAML 2.0.

If you want to avoid this manual step and detect the client certificate automatically, more work is involved, and it is what we are going to discuss next.

1. Enable Forms Authentication as default in the configuration file. The “Forms” handler must be on top of the list.
2. Create a new virtual directory in the ADFS IIS, and configure that virtual directory to require HTTPS and Accept Client Certs.

1. Configure an ASP.NET Generic handler in that virtual directory to check if the certificate is present or not

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

using System;
using System.Web;

public class CertificateHandler : IHttpHandler
{

  public void ProcessRequest(HttpContext context)
    {
        context.Response.ContentType = "application/javascript";

        if (context.Request.ClientCertificate.IsPresent)
        {
            context.Response.Write("if(typeof(certificateCallback) == 'function') 
              { (certificateCallback) (true); }");
        }
        else
        {
            context.Response.Write("if(typeof(certificateCallback) == 'function') 
              { (certificateCallback) (false); }");
        }

    }

    public bool IsReusable
    {
        get
        {
            return false;
        }
    }

}

This code checks if the client certificate is present and invokes a javascript callback with “true” or “false” based on that condition. This code does not check the authenticity of the cert or anything else, and it is only for determining if the certificate is present. The rest of the validations will be done by ADFS.

1. Configure this handler in the web.config of that virtual directory.

1
2
3
4
5

<system.webServer>
  <handlers>
    <add name="certificate" type="CertificateHandler" path="Script" verb="GET"/>
  </handlers>
</system.webServer>

1. Include a CertCallBack.js javascript file into the ADFS folder with the code required for the callback,

1
2
3
4
5
6
7

window.certificateCallback = function (cert) {
    var url;
    if (cert) {
        url = "auth/sslclient/" + window.location.search;
        document.location.href = url;
    }
};

This code is very simple. It simply does a redirect when the cert is present (the callback was called with a “true” value). The redirect in this sample is only valid for SAML 2.0

1. Modify the FormsSign.aspx page to include the following code at the end of the Page_Load event,

1
2
3
4

string script = System.IO.File.ReadAllText(Server.MapPath("CertCallBack.js"));

this.Page.ClientScript.RegisterClientScriptBlock(this.GetType(), "callback", script, true);
this.Page.ClientScript.RegisterClientScriptInclude("certauth", "/CertDetection/Script");

This code injects our CertCallBack.js script in the page and it also includes the script located in the virtual directory configured to accept client certs (/CertDetection/Script invokes the ASP.NET Handler basically)

That should be all. If the browser detects a client certificate when the script in the CertDetection virtual directory is resolved, the callback will be executed with a true value, making the form to do a redirect to authenticate the client with a Client Certificate instead. Otherwise, the client will see the Form to enter the username and password. We are adding a new virtual directory for not interferring with the existing authentication methods in the ADFS virtual directory.

The code for the handler and the javascript callback is included in the attached zip file

That content is included in the generated Azure package, and it is deployed in the AppRoot folder when the package is finally published in VM in the cloud.

One of the problem with this feature is that you might want to include content with an structure that changes often or content with thousands of folders/files, which requires some tedious manual work in Visual Studio to keep that content updated in the project.

Good news is that you can use a MSBuild task to inject that custom content automatically when the package is being generated. You have to include a custom Target “BeforeRoleAddContent” right after the declaration of the “Microsoft.WindowsAzure.targets” as it is shown bellow,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

<Import Project="$(CloudExtensionsDir)Microsoft.WindowsAzure.targets" />

<Target Name="BeforeAddRoleContent">
  <ItemGroup>
    <AzureRoleContent Include="..\Solr\Solr">
      <RoleName>SolrMasterHostWorkerRole</RoleName>
      <Destination>Solr</Destination>
    </AzureRoleContent>
    <AzureRoleContent Include="..\Solr\jre6">
      <RoleName>SolrMasterHostWorkerRole</RoleName>
      <Destination>jre6</Destination>
    </AzureRoleContent>
    <AzureRoleContent Include="..\Solr\SolrFiles">
      <RoleName>SolrMasterHostWorkerRole</RoleName>
      <Destination>SolrFiles</Destination>
    </AzureRoleContent>
    <AzureRoleContent Include="..\Solr\Solr">
      <RoleName>SolrSlaveHostWorkerRole</RoleName>
      <Destination>Solr</Destination>
    </AzureRoleContent>
    <AzureRoleContent Include="..\Solr\jre6">
      <RoleName>SolrSlaveHostWorkerRole</RoleName>
      <Destination>jre6</Destination>
    </AzureRoleContent>
    <AzureRoleContent Include="..\Solr\SolrFiles">
      <RoleName>SolrSlaveHostWorkerRole</RoleName>
      <Destination>SolrFiles</Destination>
    </AzureRoleContent>
  </ItemGroup>
</Target>

The example above injects several content folders in two different roles, “SolrMasterHostWorkerRole” and “SolrSlaveHostWorkerRole”. The “Include” attribute specifies the source folder, and the Destination folder within the AppRoot is specified in the Destination element.

As Solr is mainly a java implementation, you only have a few alternatives to run in on Windows Azure. You can deploy it as a worker role together with the java runtime machine, or you can deploy it in a VM. As any solution in the cloud, the state persisted in the worker role or VM goes away when the VMs are replaced or they go down. As Solr persists the indexes in disk, you need to make sure it is stored in a permament storage like Azure Drive or the storage service. If you decide to use a worker role, this requires some additional work to make Solr to store the indexes in Azure drive for example, which is a VHD stored in the storage service that can be mounted by the VM as a local disk. Good news is that MS Open Tech has already done this for us. They have a created a template that deployes Solr with a fail-over configuration(master-slave) in two worker roles, one role for the master node, and another role for the slave node. The slave node replicates from the master node, so in case you lost one of them, you still have the other node available. . In addition, it configures a web role with an MVC application that acts as a admin dashboard for doing basic management stuff. This solution is hosted in Github as part of this project Windows-Azure-Solr. The Github site also provides instructions to get the solution deployed in Windows Azure.

The template that you download from GitHub imports data into Solr by crawling some URLs. That’s part of the data-config.xml file that you can find in the configuration folder of the master and slave nodes (SolrMasterWorkerRole\SolrFiles\data-config.xml and SolrSlaveWorkerRole\SolrFiles\data-config.xml). Solr supports the idea of data importers, which can be used to import data from different sources such as existing web sites, files in disk or even a database.

In this case, we will modify that data-config.xml file to use a data importer that pulls data from an existing SQL Azure instance. The following example shows how this data importer configuration looks like,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<dataConfig>
  <dataSource type="JdbcDataSource" name="ds1"
    driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
    url="jdbc:sqlserver://[your server];database=[your database];user=[your user];password=[your user];encrypt=true;hostNameInCertificate=*.database.windows.net;loginTimeout=30;"
    readOnly="true"
  />

  <document name="articles">
    <entity name="article" dataSource="ds1" pk="id"
  query = "SELECT id, title, description, tags, author, lastupdated from Articles"
  deltaQuery="select id FROM Articles WHERE LastUpdated &gt; '${dataimporter.last_index_time}'"
  deltaImportQuery="SELECT id, title, description, tags, author, lastupdated from Articles where id = '${dataimporter.delta.id}'"
      >
      <field column="id" name="id" />
      <field column="title" name="title" />
      <field column="description" name="description" />
      <field column="tags" name="tags" />
      <field column="author" name="author" />
      <field column="lastupdated" name="lastupdated" />
    </entity>
  </document>
</dataConfig>

First of all, we have defined a dataSource element that points to our SQL Azure instance. This is a Jdbc data source that uses the SQL Server driver and sets the connection string in the url attribute. Secondly, we have defined a document, which specifies one or more entities that are mapped from the SQL Azure database with a select statement. The fields section maps the different fields in the document to the fields returned by the select statements. This document is what Solr stores in the index using Lucene. As you can see, three queries have been defined. The first one with the “query” attribute is used by the data importer when a full import of the complete database is done into Solr. The other two queries are used for supporting a delta or partial import scenario. These two are optional and useful only in scenarios where you have frecuent updates and a lot of data to import, which will make the full import considerably slow.

Since this data importer uses the SQL Server driver for the Jdbc data source, you will have to download that package from the Microsoft website and copying it in the folder where Solr looks for the external libraries (SolrMasterWorkerRole\Solr\dist and SolrSlaveWorkerRole\Solr\dist).

We have defined so far the mapping of a document against one or more tables in the database, but Solr still requires the definition of those fields, which are part of the schema. The schema definition can be found in the schema.xml file (SolrMasterWorkerRole\SolrFiles\v44\schema.xml and SolrSlaveWorkerRole\SolrFiles\v44\schema.xml). The following example shows how the schema is modified to include the fields used by the data importer.

1
2
3
4
5
6

<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="title" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="description" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="tags" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="author" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="lastupdated" type="date" indexed="true" stored="true" required="true" multiValued="false" />

This is enough to get Solr configured to pull data from SQL Azure, so we are in conditions to deploy Solr into an Windows Azure subscription also using the tool provided by MS Open Tech. If you have installed all the pre-requisites and followed the instructions in the wiki page of the Github website, the following command should be enough to perform that deployment.

Inst4WA.exe -XmlConfigPath “SolrInstWR_V4.4.xml” -subscription “[subscription name]” -location “[location]” -DomainName “[cloud service name]”

If you are lucky enough to get the command to work in the first instance, it will open a new browser instance when the deployment is complete, and it will also redirect you to the MVC dashboard running in the web role.

You are in conditions now to execute a few commands to import the data into Solr. You will see the location and port where the Solr master and slave nodes are running as part of the home page in the dashboard. For doing a full import into the master role, you have to copy that location and port, and append “/dataimport?command=full-import”. For example, “http://mysamplesolr.cloudapp.net:21000/solr/dataimport?command=full-import”. That will start a full import that runs asynchronously, so you can use this other command to check the status of the import “/dataimport?command=status”. For doing a partial import, you only change the command to this one “/dataimport?command=delta-import”. Once the import is complete, you can do a search to verify that everything looks ok. That can be done with the following command “/select?q=[query]”. For example, “http://mysamplesolr.cloudapp.net:21000/solr/select?q=azure”

So you have Solr indexing all your data now, but what happens with the security ?. This thing is open to the world. Anyone can do anything with your Solr instances as everything is public. There are some ways to secure Solr pages by changing some settings in the web server, which is Jetty by default. However, the Solr documentation recommends to put Solr behind a reverse proxy that filters the requests. There are several reverse proxy implementations for Solr in Github, but I will use a different approach for Windows Azure here. Given that the MS Open Tech template includes a web role for the MVC dashboard, and two worker roles for running the Solr master and slave instances, we can make the Solr instances available in internal endpoints only and use the MVC application as a facade or reverse proxy to forward all the requests to these instances. The only public and visible face will be the MVC application in the web role. All the requests for Solr must go through this MVC application first, which can filter any request that looks malicious or any request that can damage the existing indexes. A simple way to do this is to allow only get operations and filter the rest. This is the approach I’ve been taken and implemented as part of a fork created from the MS OpenTech project in Github. This fork is available here.

The following method in a controller works to describe the problem,

1
2
3
4
5
6
7

public HttpResponseMessage Post(User model)
{
  var response = Request.CreateResponse(HttpStatusCode.Created, user);
  var link = Url.Link("DefaultApi", new { id = id, controller = "Users" });
  response.Headers.Location = new Uri(link);
  return response;
}

The Request propery is being used to generate a content negotiated response, and the Url property to infer the new location for the resource. If you use ASP.NET Web API as it is today, you will have to write a lot of custom code to initialize the configuration instance as it is shown below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

var controller = new UserController();
controller.Configuration = new HttpConfiguration();
var route = controller.Configuration.Routes.MapHttpRoute(
         name: "DefaultApi",
         routeTemplate: "api/{controller}/{id}",
         defaults: new { id = RouteParameter.Optional }
);
var routeData = new HttpRouteData(route,
       new HttpRouteValueDictionary
       {
          { "id", "1" },
          { "controller", "Users" }
       }
);
controller.Request = new HttpRequestMessage(HttpMethod.Post,
  "http://localhost:9091/");
controller.Request.Properties
  .Add(HttpPropertyKeys.HttpConfigurationKey, controller.Configuration);
controller.Request.Properties
  .Add(HttpPropertyKeys.HttpRouteDataKey, routeData);

Glenn Block wrote a very nice extension as part of the Web API Contrib project for making this configuration much more simpler. This extension is available here, https://github.com/WebApiContrib/WebAPIContrib/blob/master/src/WebApiContrib.Testing/ApiControllerExtensions.cs.

However, we can left all this in the past with the new ASP.NET Web API vNext release. The introduction of the IHttpActionResult interface (Equivalent to ActionResult in ASP.NET MVC) has simplified a lot the unit testing story for controllers. A controller method can return now an implementation of IHttpActionResult, which internally uses the Request or the UrlHelper for link generation, so the unit test only cares about the returned IHttpActionResult instance.

The following code shows the same controller method using an instance of IHttpActionResult.

1
2
3
4
5
6
7
8
9
10
11
12

public CreatedAtRouteNegotiatedContentResult<UserModel> Post(UserModel user)
{
  user.Id = 1;

  var result = new CreatedAtRouteNegotiatedContentResult<UserModel>(
    "DefaultApi",
    new Dictionary<string, object> { { "id", user.Id } },
      user,
      this);

  return result;
}

CreatedAtRouteNegotiatedContentResult is an implementation also included in the framework for handling this scenario. A new resource is created and the location is set in the response message.

The code for the unit test is much simpler too.

1
2
3
4
5
6
7
8
9

[TestMethod]
public void ShouldCreateUser()
{
    var controller = new UserController();
    var result = controller.Post(new UserModel { Name = "foo" });

    Assert.IsInstanceOfType(result.Content, typeof(UserModel));
    Assert.IsNotNull(result);
}

The UrlHelper class has also been modified to make most of its methods virtual, so they can be mocked or overriden as part of an unit test. You can use this in case you still need to rely on the Url instance in the controller. For example, if you have a controller method like this one.

1
2
3
4
5
6
7
8
9
10
11
12

public HttpResponseMessage Post(UserModel user)
{
  user.Id = 1;

  var response = new HttpResponseMessage(HttpStatusCode.Created);
  var link = Url.Link("DefaultApi",
    new { id = user.Id, controller = "User" });

  response.Headers.Location = new Uri(link);

  return response;
}

You can create a custom class that derives from UrlHelper and overrides the Link method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

[TestMethod]
public void ShouldCreateUser()
{
  var controller = new UserController();
  controller.Url = new MyUrlHelper();

  var result = controller.Post(
    new UserModel { Name = "foo" });

  Assert.IsNotNull(result);
}

public class MyUrlHelper : UrlHelper
{
  public override string Link(string routeName, object routeValues)
  {
    return "http://example.com/user/1";
  }
}

Running NancyFx as a OWIN handler is pretty straightforward, and discussed in detail as part of the NancyFx documentation here. After run the steps described there and you have the application working, only a few more steps are required to register the additional middleware services.

The example bellow shows how the Startup class is modified to include Hawk authentication.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        app.UseHawkAuthentication(new HawkAuthenticationOptions
        {
            Credentials = (id) =>
            {
                return new HawkCredential
                {
                    Id = "dh37fgj492je",
                    Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
                    Algorithm = "hmacsha256",
                    User = "steve"
                };
            }
        });

        app.UseNancy();
    }
}

This code registers the Hawk Authentication Handler on top of the OWIN pipeline, so it will try to authenticate the calls before the request messages are passed over to NancyFx.

The authentication handlers in Katana set the user principal in the OWIN environment using the key “server.User”. The following code shows how you can get that principal in a NancyFx module,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

public class HomeModule : NancyModule
{
  public HomeModule()
  {
    Get["/"] = x =>
    {
      var env = (IDictionary<string, object>)Context.Items[NancyOwinHost.RequestEnvironmentKey];

      if (!env.ContainsKey("server.User") || env["server.User"] == null)
      {
          return HttpStatusCode.Unauthorized;
      }

      var identity = (ClaimsPrincipal)env["server.User"];

      return "Hello " + identity.Identity.Name;
    };
}

1

}

Thanks to OWIN, you don’t know any details of how these cross cutting concerns can be implemented in every possible web application framework.

Microsoft.Owin.Security is also the core project where you can find the base classes for writing a new authentication handler, and the ones that all the these implementations use.

At first glance, the core class that you will use to create a new authentication handler is Microsoft.Owin.Security.Infrastructure.AuthenticationHandler<T>, where T is class that derives from AuthenticationOptions and contains all the properties for initializing the handler.

AuthenticationHandler<T> derives from Microsoft.Owin.Security.Infrastructure.AuthenticationHandler, which provides the following definition.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

public abstract class AuthenticationHandler
{
   protected OwinRequest Request;
   protected OwinResponse Response;

   protected AuthenticationHandler();

   protected virtual Task ApplyResponseChallenge();

   protected virtual Task ApplyResponseCore();

   protected virtual Task ApplyResponseGrant();

   protected abstract Task<AuthenticationTicket> AuthenticateCore();
}

This class basically provides access to the current request/response OWIN messages, and also a few methods that can be overridden as part of the implementation.

• ApplyResponseChallenge: This method can be overridden to send a authentication challenge when a middleware service in the upper layer denies the execution (e.g 401 status code). For example, this can be useful for basic authentication.
• ApplyResponseCore: This method by default calls ApplyResponseChallenge and ApplyResponseGrant. It can be changed to make some additional processing of the Response message.
• ApplyResponseGrant: This method is only useful for authentication methods that needs to implement sign-in/sign-out concerns such as OAuth or Forms authentication. You won’t be using this for some authentication options such as basic or hmac authentication.
• AuthenticateCore: This is the more important method, and the one where all the main implementation of the authentication handler lives. The AuthenticationTicket will contain the identity of the authenticated user or null if the user couldn’t be authenticated.

The AuthenticationOptions base class only contains the following structure,

1
2
3
4
5
6
7
8
9
10

public abstract class AuthenticationOptions
{
  protected AuthenticationOptions(string authenticationType);

  public AuthenticationMode AuthenticationMode { get; set; }

  public string AuthenticationType { get; set; }

  public AuthenticationDescription Description { get; set; }
}

• AuthenticationMode. It’s an enumeration with two possible values Active/Passive. It basically identifies the http flow for the authentication implementation. Some authentication methods like Basic or HMac are Active, so it can be used directly against the current request without requiring additional http redirections. Some other methods like OAuth or Forms are not, so those have to identified as Passive. This setting will affect the authentication middleware behaves, so you have to set the correct value for it.
• AuthenticatinType. It represents the authentication scheme. For example, Basic.
• AuthenticationDescription. It can be used to provide more information about the authentication method.

Only these two classes are used so far to create a new authentication handler. However, you will also need a another class that acts as a factory, and it is used to inject the handler into the OWIN pipeline. That class is Microsoft.Owin.Security.Infrastructure.AuthenticationMiddleware.

1
2
3
4
5
6
7
8

public abstract class AuthenticationMiddleware<TOptions> : OwinMiddleware where TOptions : Microsoft.Owin.Security.AuthenticationOptions
{
  public AuthenticationMiddleware(OwinMiddleware next, TOptions options);

  public TOptions Options { get; set; }

  protected abstract AuthenticationHandler<TOptions> CreateHandler();
}

This class basically receives the Options that must be passed to the AuthenticationHandler, and also contains a method CreateHandler where the handler is actually created.

I will use the rest of this post to describe the implementation of a handler for doing HMac authentication using Hawk.

The first step in the implementation is to create a custom AuthenticationOptions class.

1
2
3
4
5
6
7
8
9
10
11

public class HawkAuthenticationOptions : AuthenticationOptions
{
  public const string Scheme = "Hawk";

  public HawkAuthenticationOptions()
       : base(Scheme)
  {
   }

  public Func<string, HawkCredential> Credentials { get; set; }
}

The Credentials property is a callback that the authentication handler implementation will use to resolve the user/key to verify the HMAC received in the authorization header. In a Basic Authentication implementation, you could potentially have a callback or a reference to a repository to verify the username and password received in the authorization header. I am also passing “Hawk” as part of the constructor to set the authentication scheme associated to the handler.

The next step is to implement the AuthenticationHandler.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

public class HawkAuthenticationHandler : AuthenticationHandler<HawkAuthenticationOptions>
{
  private readonly ILogger logger;

  public HawkAuthenticationHandler(ILogger logger)
  {
      this.logger = logger;
  }

  protected override Task<AuthenticationTicket> AuthenticateCore()
  {
     AuthenticationHeaderValue authorization = null;

     if (Request.GetHeader("authorization") != null)
     {
         authorization = AuthenticationHeaderValue.Parse(Request.GetHeader("authorization"));
      }

     if (authorization != null &&
          !string.Equals(authorization.Scheme, HawkAuthenticationOptions.Scheme))
     {
          this.logger.WriteInformation(string.Format("Authorization skipped. Schema found {0}",
               authorization.Scheme));

          return Task.FromResult(EmptyTicket());
      }

      if (authorization == null ||
         string.IsNullOrWhiteSpace(authorization.Scheme))
      {
          this.logger.WriteWarning("Authorization header not found");

          return Task.FromResult(EmptyTicket());
      }
      else
      {
          if (string.IsNullOrWhiteSpace(authorization.Parameter))
          {
              this.logger.WriteWarning("Invalid header format");

              return Task.FromResult(EmptyTicket());
          }

         if (string.IsNullOrWhiteSpace(Request.Host))
         {
              this.logger.WriteWarning("Missing Host header");

              return Task.FromResult(EmptyTicket());
          }

          try
          {
              var principal = Hawk.Authenticate(authorization.Parameter,
                   Request.Host,
                   Request.Method,
                   Request.Uri,
                   this.Options.Credentials);

              var identity = (ClaimsIdentity)((ClaimsPrincipal)principal).Identity;
              var ticket = new AuthenticationTicket(identity, (AuthenticationExtra)null);

              return Task.FromResult(ticket);
           }
           catch (SecurityException ex)
           {
                this.logger.WriteWarning(ex.Message);

                return Task.FromResult(EmptyTicket());
           }
        }
    }

    protected override Task ApplyResponseChallenge()
    {
         if (Response.StatusCode != 401)
         {
              return Task.FromResult<object>(null);
          }

         var ts = Hawk.ConvertToUnixTimestamp(DateTime.Now).ToString();
         var challenge = string.Format("ts=\"{0}\" ntp=\"{1}\"",
                ts, "pool.ntp.org");

         Response.AddHeader("WWW-Authenticate", HawkAuthenticationOptions.Scheme + " " + challenge);

        return Task.FromResult<object>(null);
     }

     private static AuthenticationTicket EmptyTicket()
     {
         return new AuthenticationTicket(null, (AuthenticationExtra)null);
      }
 }

The implementation is Active, so only overrides the AuthenticationCore and ApplyResponseChallenge methods. The AuthenticationCore implementation tries to locate a Authorization Header with a format that follows the Hawk specification to authenticate the call. If an Authorization header is not found or it can not be parsed, an AuthenticationTicket instance with an empty identity is returned. Otherwise, the identity is set in the ticket and returned as part of the task. This implementation also uses the logging facilities provided by Katana.

The last step is to implement the AuthenticationMiddleware class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

public class HawkAuthenticationMiddleware : AuthenticationMiddleware<HawkAuthenticationOptions>
{
    private readonly ILogger logger;

    public HawkAuthenticationMiddleware(
        OwinMiddleware next,
        IAppBuilder app,
        HawkAuthenticationOptions options) : base(next, options)
    {
         this.logger = app.CreateLogger<HawkAuthenticationHandler>();
     }

     protected override AuthenticationHandler<HawkAuthenticationOptions> CreateHandler()
     {
         return new HawkAuthenticationHandler(this.logger);
     }
 }

This implementation only overrides the CreateHandler method to return a new HawkAuthenticationHandler instance. As that instance relies on the Katana logger, the logger instance is first created from the IAppBuilder instance injected in the constructor.

Once you have the AuthenticationMiddleware implementation completed, you will want to inject it in the OWIN pipeline to use it in an existing application. An extension method can be provided to make this task easier for the developer.

1
2
3
4
5
6
7
8
9

public static class HawkAuthenticationExtensions
{
    public static IAppBuilder UseHawkAuthentication(this IAppBuilder app, HawkAuthenticationOptions options)
    {
        app.Use(typeof(HawkAuthenticationMiddleware), app, options);
        app.UseStageMarkerAuthenticate();
        return app;
     }
 }

The following example shows how this extension is used to register the HawkAuthenticationMiddleware service in an application that uses Web API with OWIN

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

public class Startup
{
     public void Configuration(IAppBuilder app)
     {
         app.SetLoggerFactory(new ConsoleLoggerFactory());

         var config = new HttpConfiguration();
         config.Routes.MapHttpRoute("Default", "api/{controller}");

         app.UseHawkAuthentication(new HawkAuthenticationOptions
         {
             Credentials = (id) =>
             {
                 return new HawkCredential
                 {
                     Id = "dh37fgj492je",
                     Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
                     Algorithm = "hmacsha256",
                     User = "steve"
                 };
              }
         });

        app.UseWebApi(config);

     }
 }

All the code provided as part of this example for Hawk can be found in the HawkNet github project.

The .NET ecosystem offers today a lot of alternatives for developing web applications. You can either use any of the frameworks supported by Microsoft with ASP.NET such as Forms, MVC or Web API, or any other open source alternative like FubuMVC, ServiceStack, NancyFx or OpenRasta to name a few. From an architecture standpoint, all these frameworks have three main layers in common (in spite of the difference with the implementation details), hosting, middleware, and application.

The hosting layer is responsible for managing the underline process, where the http connections are established and managed, and also to materialize those connections into request/response objects that are sent to the upper layers. For example, the hosting layer could be ASP.NET running in IIS, or it could also be a console application that uses the http listener directly.

The middleware layer provides common infrastructure services, which runs at low level in the http stack, such as security, caching, or any other concern that can be handled at this level.

The application layer is where the applications or services are implemented using the features available in the framework.

However, the distinction between the middleware and application layers is not always clear, which causes that many of the middleware services end up being implemented in the application layer. For example, you could implement middleware services for security in ASP.NET MVC using filters, which rely on features specific to the application level in that framework. This make almost impossible to reuse all these services across all the available frameworks, so you will find different implementations of the same services for the different frameworks. To give an example, basic authentication is something that could be implemented as middleware service and reused, but today is implemented differently in each framework.

Someone would suggest that a ASP.NET Http Module could be a good option for implementing a middleware service, but that’s not necessarily true as it would only work when the hosting layer uses ASP.NET.

This exact problem is what OWIN specification tries to address by providing a common abstraction for any http-aware service or application with minimal dependencies on existing frameworks or implementations.  At very core level, OWIN defines a handler, which is represented as an application delegate,

using AppFunc = Func< \         IDictionary<string, object>, // Environment \         Task>; // Done

The idea is to represent the three layers discussed above as a set of handlers using the “chain of responsibility” pattern. The environment argument is a dictionary that the different handlers use to exchange values, and the next handler in the chain is represented with the task argument. Having said that, the first handler in the chain would be responsible for the hosting layer, and it should populate the environment with keys that describe the received http request message, such as scheme, port, request path, headers, etc. All those common headers are discussed as part of the specification.

Katana is a Microsoft implementation of the OWIN specification, which includes handlers for the hosting and middleware layers. It’s an open source project, which can be found in this location. All the investment that Microsoft has done so far for ASP.NET in the different application frameworks (i.e. MVC or Web API), it’s now being refactored as OWIN handlers that can be reused across different application frameworks. For example, cookie authentication or even OAuth are being implemented as middleware services that can be reused not only by ASP.NET web apps but also for other application frameworks in the open source world.

The application layer is also implemented as another OWIN handler, which is responsible for setting up all the infrastructure required to handle the request/response at the application level. You will find application handlers for ASP.NET MVC, ASP.NET Web API, SignalR, or NancyFx to give an example. 

Gettting Started with OWIN and Katana

The easiest way to get started with OWIN is to use the assembly distributed as nuget package. You will find a single nuget package for this called “OWIN”. This assembly only contains an interface IAppBuilder, which can be used to configure all the handlers that you want to use in your web application. This is a very low level interface that you won’t have to implement yourself if you use Katana.

In addition to the “OWIN” assembly, you will find two additional nuget packages

• Owin.Extensions: Provides a set of very useful extensions methods for the IAppBuilder interface that you can use to configure the handlers.
• Owin.Types: Provides a set of wrappers to represent the environment dictionary as a typed object representing the request and response messages.

You can implement your first OWIN handler using these three packages (You will not able to run it without the hosting handlers provided by Katana).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

public class MyMiddleware
{
  Func<IDictionary<string, object>, Task> next;

  public MyMiddleware(Func<IDictionary<string, object>, Task> next)
  {
    this.next = next;
  }

  public Task Invoke(IDictionary<string, object> env)
  {
    var headers = (IDictionary<string, string[]>)env["owin.RequestHeaders"];

    var url = string.Format("{0}://{1}{2}{3}",
                (string)env["owin.RequestScheme"],
                headers["host"].First(),
                (string)env["owin.RequestPathBase"],
                (string)env["owin.RequestPath"]);

    if ((string)env["owin.RequestQueryString"] != "")
      url += "?" + (string)env["owin.RequestQueryString"];

    Console.WriteLine("The received URL is {0}", url);

    return this.next(env);
  }
}

This is a low level handler that sends the request url to the console. As you can see, it relies on some existing properties in the environment, which should have been set by Katana in the hosting layer. The handler also receives the next handler to call in the chain as part of the constructor.

The entry point in any Katana application must be a class that receives the IAppBuilder interface as argument to configure all the handlers, as it is shown below,

1
2
3
4
5
6
7

 public class Startup
 {
    public void Configuration(IAppBuilder app)
    {
       app.UseType<MyMiddleware>();
    }
 }

UseType is one the extension methods provided by “Owin.Extensions” to configure a new handler in the pipeline. You can also other extension methods to define a handler on the fly using delegates as this one,

1
2
3
4
5

app.UseHandlerAsync(async (req, res) =>
{
    res.ContentType = "text/plain";
    await res.WriteAsync("Hello World");
});

There is one more nuget package that you will find useful to create OWIN handlers, which is “Microsoft.Owin”. This is a package provided as part of Katana (It’s still a pre-release), which extends the Owin Request and Response classes with more properties and methods, and also helper methods to deal with these classes.

Hosting an application with Katana

Katana provides three options to host a web application.

​1. OwinHost.exe. It’s a command tool that you can use to run the application. You can use multiple command arguments to specify the assembly with the entry point for configuring the IAppBuilder or other configuration settings such as the port number or tracing. This is tool is available as part of the nuget package “OwinHost”.

​2. .NET application. There is a WebApp class that you can use to self-host a web application in any regular .NET application such as console app or a windows service. This class is available as part of the “Microsoft.Owin.Hosting” assembly.

1
2
3
4
5

using (WebApp.Start<Startup>(new StartOptions { Port = 5000 }))
{
   Console.WriteLine("Press Enter to quit.");
   Console.ReadKey();
}

You need to pass the startup entry as argument to the static method “Start”.

​3. ASP.NET. You need to reference the nuget package “Microsoft.Owin.Hosting.SystemWeb” (Pre-Release) from your web application project, and include the Startup class in that project as well.

In ETW, you have applications publishing events in queues (or providers) and other applications consuming events from those queues in real-time through ETW sessions. When an event is published in a provider, it goes nowhere unless there is session collecting events on that queue. (The events are not persisted).

The adoption of ETW in .NET application was pretty low as it was very hard to configure and use. However, things might change now that .NET 4.5 supports a new EventSource class for publishing events easily in any .NET application. Another way to publish events in the past was to use .NET diagnostics infrastructure with the trace listener for ETW, “EventProviderTraceListener”.

About this last one, it can be easily configured and associated to a trace source as it is shown bellow,

1
2
3
4
5
6
7
8
9
10
11
12

<system.diagnostics>
  <sources>
    <source name="MyConsoleApp" switchValue="Verbose">
      <listeners>
        <add name="ETWListener" type="System.Diagnostics.Eventing.EventProviderTraceListener, System.Core, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
               initializeData="13D5F7EF-9404-47ea-AF13-85484F09F2A7" />
      </listeners>
    </source>
  </sources>

  <trace autoflush="true"/>
</system.diagnostics>

The trace listener is associated to a GUID through the initializeData attribute. That identifier is used by ETW and associated to the provider, so it’s the one you will use to collect the events in a session.

Once the listener is configured, you can start publishing events in that provider through a the trace source.

1
2
3
4
5
6
7
8
9

TraceSource myTraceSource = new TraceSource("MyConsoleApp");

myTraceSource.TraceEvent(TraceEventType.Error, 1, "Tracing Error Message.");
myTraceSource.TraceEvent(TraceEventType.Warning, 2, "Tracing Warning Message.");
myTraceSource.TraceEvent(TraceEventType.Information, 3, "Tracing Information.");
myTraceSource.TraceEvent(TraceEventType.Verbose, 4, "Tracing Verbose Message.");
myTraceSource.TraceEvent(TraceEventType.Critical, 5, "Tracing Critical Message.");

myTraceSource.Close();

Listening for those events in a ETW session is a different story. The standard procedure is to use a set of tools for starting an ETW session and dump the events into a binary file with a proprietary format (these are “etl” files), that can be imported in the event log viewer or an specific tool for this kind of file. You also have a tool for converting “etl” files to more user readable formats like xml or cvs. However, you can also use the ETW unmanaged API to start a new session and subscribe to the events as it was done with the EventTraceWatcher implementation in this sample.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

var providerId = new Guid("13D5F7EF-9404-47ea-AF13-85484F09F2A7");

using (EventTraceWatcher watcher = new EventTraceWatcher("MySession", providerId))
{
  watcher.EventArrived += delegate(object sender, EventArrivedEventArgs e)
  {
    if (e.Error != null)
    {
      Console.Error.WriteLine(e.Error);
      Environment.Exit(-1);
    }

    // Dump properties (key/value)
    foreach (var p in e.Properties)
    {
      Console.WriteLine("Event", "\t" + p.Key + " -- " + p.Value);

    };

    // Start listening
    watcher.Start();

    Console.WriteLine("Listening...Press <Enter> to exit");
    Console.ReadLine();
}

You use the EventTraceWatcher to subscribe to events generated by an ETW provider, which are received as part of an event. What’s really interesting about this code is that it runs out of process. You could be collecting the events in a completely different application in the same host and doing whatever you want with those events like persisting them in a database for example. The performance of the application generating the events would not be affected at all.

Now, imagine that you have a service running in one of your servers, which can be used to start a new ETW session and publish those events via SignalR. In that way, you can connect with a browser to the SignalR hub created by that service and get the events realtime, which represents an interesting way to see what’s going on your server at a given time.

You can do an slight change to the code using the EventTraceWatcher to publish the events to a SignalR hub as it is shown bellow,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

class Program
{
    static void Main(string[] args)
    {
        string url = "http://localhost:8080";

        using (WebApplication.Start<Startup>(url))
        {
            Console.WriteLine("Server running on {0}", url);

            var hubConnection = new HubConnection("http://localhost:8080/");
            var serverHub = hubConnection.CreateHubProxy("EventsHub");
            hubConnection.Start().Wait();

            var providerId = new Guid("13D5F7EF-9404-47ea-AF13-85484F09F2A7");

            using (EventTraceWatcher watcher = new EventTraceWatcher("MySession", providerId))
            {
                watcher.EventArrived += delegate(object sender, EventArrivedEventArgs e)
                {
                    if (e.Error != null)
                    {
                        Console.Error.WriteLine(e.Error);
                        Environment.Exit(-1);
                    }

                    // Dump properties (key/value)
                    foreach (var p in e.Properties)
                    {
                        serverHub.Invoke("PushEvent", "\t" + p.Key + " -- " + p.Value).Wait();
                    }
                };

                // Start listening
                watcher.Start();

                Console.WriteLine("Listening...Press <Enter> to exit");
                Console.ReadLine();
            }

            Console.ReadLine();
        }
    }
}

class Startup
{
    public void Configuration(IAppBuilder app)
    {
        // Turn cross domain on 
        var config = new HubConfiguration { EnableCrossDomain = true };

        // This will map out to http://localhost:8080/signalr by default
        app.MapHubs(config);
    }
}

public class EventsHub : Hub
{
    public void PushEvent(string message)
    {
        Console.WriteLine("Event: " + message);

        Clients.All.PushEvent(message);
    }
}

This uses SignalR hosted in a console application with OWIN, so it could be literally moved to a windows service as well. Every time an event is captured from the ETW session, it is published in the SignalR hub.

On the other side, you can also use SignalR to subscribe to the same hub and receive the generated messages at real time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

<script type="text/javascript">
    $(function () {
        var messages = $('#messages');

        var connection = $.hubConnection('http://localhost:8080');
        connection.start()
            .done(function () {
                console.log("Connected, transport = " + connection.transport.name);
            })
            .fail(function () { console.log('Could not connect'); });

        var proxy = connection.createHubProxy('EventsHub');

        proxy.on('PushEvent', function (message) {
            messages.append(message + "<br>");
        });
    });
</script>

The code is available at Github, https://github.com/pcibraro/RealtimeTracing

This is relatively simple to implement in a message handler,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

public class IPThrottlingMessageHandler : DelegatingHandler
{
  IIPRepository repository;
  int maxRequestsHour;

  public IPThrottlingMessageHandler(IIPRepository repository, int maxRequestsHour = 150)
            : base()
  {
    this.repository = repository;
    this.maxRequestsHour = maxRequestsHour;
   }

   public IPThrottlingMessageHandler(HttpMessageHandler inner, IIPRepository repository, int maxRequestsHour = 150)
            : base(inner)
  {
     this.repository = repository;
     this.maxRequestsHour = maxRequestsHour;
   }

   protected override System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
  {
     var ip = GetClientIp(request);

      if (ip == null)
      {
         return ToResponse(request, HttpStatusCode.Forbidden, "The client ip couldn't be found");
       }

       if(this.repository.Increment(DateTime.Now.Hour, ip) > this.maxRequestsHour)
       {
          return ToResponse(request, HttpStatusCode.Forbidden, "Quota exceeded");
        }

       return base.SendAsync(request, cancellationToken);
  }

  private string GetClientIp(HttpRequestMessage request)
  {
     if (request.Properties.ContainsKey("MS_HttpContext"))
     {
        return ((HttpContextWrapper)request.Properties["MS_HttpContext"]).Request.UserHostAddress;
      }
     else if (request.Properties.ContainsKey(RemoteEndpointMessageProperty.Name))
     {
        RemoteEndpointMessageProperty prop;
        prop = (RemoteEndpointMessageProperty)request.Properties[RemoteEndpointMessageProperty.Name];
        return prop.Address;
      }
      else
      {
          return null;
       }
   }

   private static Task<HttpResponseMessage> ToResponse(HttpRequestMessage request, HttpStatusCode code, string message)
   {
       var tsc = new TaskCompletionSource<HttpResponseMessage>();

       var response = request.CreateResponse(code);
       response.ReasonPhrase = message;
       response.Content = new StringContent(message);

       tsc.SetResult(response);

        return tsc.Task;
    }
 }

This handler uses a repository to store the number of calls with a given IP in one hour. If the number of requests per hour exceeds the quota, an error response is returned to the client.

Keys is the first scenario and probably the simplest one. Every client application is identified with a simple and fixed application key. This authentication mechanism is perhaps a bit weak, but the data that the service has to offer is not sensitive at all. The data is available for everyone with a key, and it’s pretty much used for public services such as Google maps or a search for public pictures in Instagram for example. The only purpose of the key is to identify clients and apply different SLA (service level agreements) such as api quotas, availability, etc.

HMAC is typically used for consuming sensitive data that is only consumed by his owner and not shared with anyone else. This kind of authentication is typically used in multitenant applications, where a tenant is the owner of the data. This model fits real well with cloud computing where a vendor such as AWS or Windows Azure use a key for identifying the tenant and provide the right services and private data. No matter which client application is used to consume the services and data, the main purpose of the key is to identify the tenant. Hawk is new specification born in this area to standardize how HMAC authentication.  

OAuth is last one and probably the most complicated one. It was born with the idea of delegating authorization in the web 2.0. The service who owns the data can use OAuth to share that data with other services or applications without compromising the owner credentials.

The analogy given by Eran Hammer Lahav in this post “Explaining OAuth” is very close to what the specification tries to address,

“Many luxury cars today come with a valet key. It is a special key you give the parking attendant and unlike your regular key, will not allow the car to drive more than a mile or two. Some valet keys will not open the trunk, while others will block access to your onboard cell phone address book. Regardless of what restrictions the valet key imposes, the idea is very clever. You give someone limited access to your car with a special key, while using another key to unlock everything else.”

This kind of authentication makes a lot of sense in social media services like Twitter, Facebook, Windows Live or Google to name a few, where the service owns some private data like contacts or pictures that can shared with other applications without putting the user credentials into risk.

OAuth assigns a key to every different client application allowed to consume the data, so the access can easily be revoked by disabling the key associated that client application.

I already implemented this feature in my Hawk port for .NET. A bewit token can be generated as it is shown below,

1
2
3
4
5
6
7
8
9
10
11
12

var credential = new HawkCredential
{
     Id = "dh37fgj492je",
     Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
     Algorithm = "hmacsha256",
     User = "steve"
 };

var bewit = Hawk.GetBewit("localhost",
  new Uri("http://localhost:8091/Api/HelloWorld"),
  credential,
  60000);

The GetBewit method expects the following arguments,

• The host name
• The complete request URI
• The Hawk credentials with information about the key and algorithm to use
• A time-to-live setting in seconds for the token

That token is an string representation that you can add as a additional query string in the Web API call.

1
2

new HttpRequestMessage(HttpMethod.Get,
  "http://localhost:8091/Api/HelloWorld?bewit=" + bewit);

In that way, you can share a link to your Web API with a limited access for a period of time to someone without having to share any security credentials.

On the service side is as simple as configuring the HawkMessageHandler as part of the Web API configuration,

1
2
3
4
5
6
7
8
9
10
11
12

var handler = new HawkMessageHandler((id) =>
{
     return new HawkCredential
     {
           Id = id,
           Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
           Algorithm = "hmacsha256",
           User = "steve"
       };
 });

config.MessageHandlers.Add(handler);

The handler will automatically detect a bewit token in the query string, and it will performed all the required validations.

The first mechanism is the error policy. You can configure the error policy preferences as part of the configuration object (HttpConfiguration) in the IncludeErrorDetailPolicy property. This is just an enum that instructs Web API about how to deal with exceptions.

The possible values for this enum are,

• Default: It’s uses the customErrors configuration settings if you are using ASP.NET as host or LocalOnly for self-host.
• LocalOnly: Only includes error details for local requests
• Always: Always includes error details
• Never: Never includes error details

When an exception happens, Web API will check the value on this setting for including details about the exception in the response message or not. For example, if Always is enabled, Web API will serialize the exception details as part of the message that you get as response.

The second mechanism is Tracing. Tracing is a service that you can inject as part of the configuration object as well. The default implementation does do anything.

1
2
3
4

public static void Register(HttpConfiguration config)
{
    config.Services.Replace(typeof(ITraceWriter), new MyTracer());
}

MyTracer is a custom implementation of the ITraceWriter service, which Web API uses for tracing purposes. This is a general tracing mechanism, so Web API will call it for logging everything and not just errors.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

public class MyTracer : ITraceWriter
{
    public void Trace(HttpRequestMessage request, string category, TraceLevel level,
        Action<TraceRecord> traceAction)
    {
        TraceRecord rec = new TraceRecord(request, category, level);
        traceAction(rec);
        WriteTrace(rec);
    }

    protected void WriteTrace(TraceRecord rec)
    {
        var message = string.Format("{0};{1};{2}",
            rec.Operator, rec.Operation, rec.Message);
        System.Diagnostics.Trace.WriteLine(message, rec.Category);
    }
}

If any of these two work for you, you can still use an Error Filter.  Tugberk has written a blog post about how to integrate ELMAH with an Error Filter in Web API here.

1
2

protected internal abstract Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
 CancellationToken cancellationToken);

In most cases, a message handler does something and delegates some other work to the rest of the handlers configured in the pipeline. For example, a handler for security checks the Auth Http header, and delegates the call the handlers configured out of the box by Web API, which eventually will call a controller method. The framework also provides a base class to make delegation implicit, DelegatingHandler, which receives the next handler to call as part of the constructor.

The following example shows a message handler implementation for basic authentication,

1
2
3
4
5
6
7
8
9
10
11

public class BasicAuthHandler : DelegatingHandler
{
   Func<string, string, IPrincipal> auth;

   public BasicAuthHandler(HttpMessageHandler innerHandler, Func<string, string, IPrincipal> auth)
       : base(innerHandler)
   {
       this.auth = auth;
    }

    protected override System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,

1
2
3
4

System.Threading.CancellationToken cancellationToken)
    {
         if (request.Headers.Authorization != null &&
             !string.Equals(request.Headers.Authorization.Scheme, "basic",

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

StringComparison.InvariantCultureIgnoreCase))
         {
              return base.SendAsync(request, cancellationToken);
          }

          if (request.Headers.Authorization == null ||
              string.IsNullOrWhiteSpace(request.Headers.Authorization.Scheme))
          {
              return ChallengeResponse(request);
           }

          var authToken = request.Headers.Authorization.Parameter;
          var decodedToken = Encoding.UTF8.GetString(Convert.FromBase64String(authToken));

          var username = decodedToken.Substring(0, decodedToken.IndexOf(":"));
          var password = decodedToken.Substring(decodedToken.IndexOf(":") + 1);

          var principal = this.auth(username, password);

          if (principal == null)
          {
              return ToResponse(request, HttpStatusCode.Unauthorized, "Invalid credentials");
           }

          Thread.CurrentPrincipal = principal;
          if (HttpContext.Current != null)
             HttpContext.Current.User = principal;

          return base.SendAsync(request, cancellationToken);
   }

   private static Task<HttpResponseMessage> ChallengeResponse(HttpRequestMessage request)
   {
       var tsc = new TaskCompletionSource<HttpResponseMessage>();

       var response = request.CreateResponse(HttpStatusCode.Unauthorized);
        response.Headers.WwwAuthenticate.Add(new AuthenticationHeaderValue("basic", "realm=localhost"));

       tsc.SetResult(response);

       return tsc.Task;
     }

     private static Task<HttpResponseMessage> ToResponse(HttpRequestMessage request,

1
2
3
4
5
6
7
8
9
10
11
12

HttpStatusCode code, string message)
     {
        var tsc = new TaskCompletionSource<HttpResponseMessage>();

         var response = request.CreateResponse(code);
         response.ReasonPhrase = message;

         tsc.SetResult(response);

         return tsc.Task;
      }
 }

A good thing about Message Handler is that you can configure them globally or per route. In this case, if you want to enable basic authentication for some routes only, it’s a matter of configuring this handler in the routes you want to have protected.

1
2
3
4
5
6
7
8
9

config.Routes.MapHttpRoute(
   "BasicAuth",
   "MyController",
   new { controller = "MyController" },
   null,
   new BasicAuthHandler(new HttpControllerDispatcher(config), (u, p) =>
   {
      return new GenericPrincipal(new GenericIdentity(u, new string[] {}));
   }));

As you can see, the Inner Handler is a built-in handler provided by Web API, HttpControllerDispatcher, which does all the magic for processing the request and pass it over to the controller action. You can also inject any other dependency as part of the constructor. One thing to consider is that message handlers are singleton if you configure them this way, so make sure to inject the dependencies in the right way for avoiding memory leaks (If you have to use a repository for example, you might want to inject a factory or pass a delegate for resolving the dependencies from a DI container).

In most cases, a domain prefix is good enough to identify tenants (e.g. mytenant.xxxxx.com). That approach typically relies on CName records for mapping the prefixes or tenants and the domain name to an URL where the application is actually hosted. Many cloud providers support the idea of mapping custom domains to their web hosted services, so this approach with CName works just fine.

An evident problem is that you would require a different CName record per prefix or tenant. If you have to create those records manually, this approach simply does not scale as the number of tenants increase.

An alternative is to use a wildcard CName, and route all the requests that match that wildcard to the hosted application in the cloud. For example, *.xxxxxx.com to your web application url.

Many DNS servers don’t support wildcards in CName such as the ones offered by free for GoDaddy or NameCheap. However, AWS Route 53 supports wildcards and also an API to manage almost everything in the DNS tables.

Configuring AWS Route 53 is relatively easy. Assuming that you already have a AWS account, you need to create first a hosted zone, which represents the association of a domain name with a set of name servers provided by Route 53. Once you specified the domain name (e.g cibrax.me), and the hosted zone is created. Route53 will show you a list of name servers you need to use. If you already own a domain in other place like GoDaddy or NameCheap, you need to go there and update the list of name servers associated to that domain.

Afterwards, you need to create a resource record set, which is basically the CName record containing the wildcard or prefix you want to use. Here, you can specify the CNAme record and the mapped URL. For example, *.cibrax.me goes to www.xxxxxxxx.com.

That’s all from the point of view of DNS configuration. The rest is part of the implementation of your web application.