Sverre's blog

First experiences with using WinRM/WinRS for remote deployment

2011-09-29T11:37:00.000+02:00

What is WinRM/WinRS?

Windows Remote Management (WinRM) is a remote management service which was first released with Windows 2003 R2.

WinRM is a server component, while Windows Remote Shell (WinRS) is a client which can be used for executing programs remotely on computers which run WinRM.

The following example shows how to remotely list the contents of the C:\ folder on a computer with host name Server01:

WinRS –r:Server01 dir c:\

Using WinRM for remote deployment

My first encounter with WinRM/WinRS was to execute some PowerShell scripts for automatic remote deployment of a test environment. The commands were executed from an MSBuild script in a CruiseControl.Net build.

The scripts would first uninstall any old versions of the components, and then renew databases and install new component versions. Finally a set of NUnit tests would be executed on the environment.

WinRS failing to execute remote commands due to limited quotas

It was very easy to get started with WinRS, and in the beginning everything seemed to work fine. But now and then the execution failed with System.OutOfMemoryException or with the message “Process is terminated due to StackOverflowException.”.

The reason for these problems was not obvious since there was no mention of quotas in the error messages, but after some investigation it turned out that they were caused by a too low memory quota on the server. The default memory quota is 150 MB, and can be changed by executing the following command on the remote server (will set memory quota to 1 GB):

WinRM set winrm/config/Winrs @{MaxMemoryPerShellMB = "1000"}

Multi-Hop configuration

In one of my scripts i tried use a UNC path to access a remote share from the target computer, but got “Access is denied”. It turned out that the Credential Security Service Provider (CredSSP) had to be configured on the client and on the server in order to achieve this: http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=VS.85).aspx

Resources

Configuring WinRM

Quota Management for Remote Shells

Using Gendarme with CruiseControl.Net for code analysis

2011-08-04T10:53:00.001+02:00

Gendarme is being developed as a part of the Mono project and is a tool for code analysis. It comes with a wide range of predefined rules and can easily be extended with you own custom rules which you can write in C# or other .Net languages.

Configuring the CruiseControl.Net buidl task

CruiseControl.Net has been delivered with the Gendarme task since version 1.4.3. However, the Gendarme executable must be downloaded and installed separately. The binary can be downloaded from this link: https://github.com/spouliot/gendarme/downloads

Gendarme is designed for processing the build output assemblies in ONE directory. I.e. it does not support recursive search for assemblies, which fits well if you have one CruiseControl.Net build project per service/application, but in my case I wanted to generate a report for an entire product branch with multiple services and applications.

This can be achieved by using the <assemblyListFile> configuration element, which lets you specify a file that contains the full path to each assembly which should be analysed.
In order to generate the file, I execute the following PowerShell command:

Get-ChildItem -Path 'D:\SomeDir\Work' -Recurse -Include MyCompany*.dll -Exclude *.Test*.dll,*Generated.dll | sort -Property Name -Unique | sort -Property FullName | foreach {$_.FullName} | Out-File -FilePath 'D:\SomeDir\Artifact\AssembliesForCodeAnalysis.txt' -Width 255

The PowerShell command above will recursively scan through the directory “D:\SomeDir\Work” and include all DLL files starting with “MyCompany” excluding those which ends with “.Test.dll” or “Generated.dll”. Next it will select distinct files regardless of paths (in order to filter out shared assemblies which are duplicated), before it sorts by full path name and write the output to file.

Using the PowerShell command as an executable step, the project configuration in ccnet.config turns into this:

   1:        </msbuild>

   2:        <exec>

   3:          <executable>powershell</executable>

   4:          <buildArgs>-Command "Get-ChildItem -Path 'D:\SomeDir\Work' -Recurse -Include MyCompany*.dll -Exclude *.Test*.dll,*Generated.dll | sort -Property Name -Unique | sort -Property FullName | foreach {$_.FullName} | Out-File -FilePath 'D:\SomeDir\Artifact\AssembliesForCodeAnalysis.txt' -Width 255"</buildArgs>

   5:        </exec>

   6:        <gendarme>

   7:            <executable>C:\Program Files (x86)\Gendarme\gendarme.exe</executable>

   8:            <assemblyListFile>D:\SomeDir\Artifact\AssembliesForCodeAnalysis.txt</assemblyListFile>

   9:            <baseDirectory>D:\SomeDir\Work</baseDirectory>

  10:            <limit>2000</limit>

  11:            <severity>medium+</severity>

  12:            <confidence>high</confidence>

  13:            <quiet>FALSE</quiet>

  14:            <verbose>TRUE</verbose>

  15:            <failBuildOnFoundDefects>FALSE</failBuildOnFoundDefects>

  16:            <verifyTimeoutSeconds>600</verifyTimeoutSeconds>

  17:        </gendarme>

  18:      </tasks>

  19:      <publishers>

  20:        <merge>

  21:          <files>

  22:            <file>D:\SomeDir\Artifact\test-results\*.xml</file>

  23:            <file>D:\SomeDir\Artifact\gendarme-results.xml</file>

  24:          </files>

  25:        </merge>

  26:        <statistics />

Configuring the Dashboard

The stylesheets which are needed for showing the formatted reports in the CruiseControl.Net dasboard are included with the CruiseControl.Net installation, and just need to be referenced in dasboard.config:

   1:       <buildPlugins>

   2:        <buildReportBuildPlugin>

   3:          <xslFileNames>

   4:            <xslFile>xsl\gendarme-summary-ccnet.xsl</xslFile>

   5:         </xslFileNames>

   6:         <xslReportBuildPlugin description="Gendarme Report" actionName="GendarmeBuildReport" xslFileName="xsl\gendarme-report-ccnet.xsl"/>

   7:        </buildReportBuildPlugin>

Resources

Gendarme home page: http://www.mono-project.com/Gendarme

Gendarme CCNet task configuration: http://confluence.public.thoughtworks.org/display/CCNET/Gendarme+Task

Intellisense for CruiseControl.Net configuration files

2010-12-19T00:09:00.000+01:00

Editing the CruiseControl .Net configuration file ccnet.config may be a cumbersome process. The XML configuration elements are documented at http://ccnetlive.thoughtworks.com/ccnet/doc/CCNET/Configuring%20the%20Server.html, but it would be more convenient to have intellisense available when editing the configuration file.

Intellisense for CCNet configuration files can be added to Visual Studio by using the schema definition file ccnet.xsd. Unfortunately this file is not distributed by the CCNet installation package, but it is included in the source distribution. For the current version the file is located at “\project\ccnet.xsd” in the downloadable source distribution zip file.

Your can also get it from the the source code repository at SourceForge (link is to version 1.5).

Adding the XSD schema to Visual Studio

Once you have gotten your hands on the ccned.xsd file, it must be copied to the schema folder of your Visual Studio installation, e.g. to “C:\Program Files (x86)\Microsoft Visual Studio 10.0\Xml\Schemas\”.

Note: Copying the file to the folder “Microsoft Visual Studio 10.0\Common7\Packages\schemas\xml” will not have any effect!

Configuring the namespace

Which namespace should be used for the CCNet configuration files? A namespace must be specified in order for Visual Studio to know which schema to use for intellisense.

CCnet.xsd defines the target namespace "http://thoughtworks.org/ccnet/1/5":

<?xml version="1.0" encoding="utf-8"?>
<xs:schema targetNamespace="http://thoughtworks.org/ccnet/1/5"
    elementFormDefault="qualified"
    xmlns="http://thoughtworks.org/ccnet/1/5"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="cruisecontrol">

… which means that the following namespace must be defined in the CCNet configuration files:

<cruisecontrol xmlns="http://thoughtworks.org/ccnet/1/5">
<project name="foo">
<webURL>http://localhost/ccnet</webURL>
<modificationDelaySeconds>10</modificationDelaySeconds>

The schema file seems to favor using XML elements instead of attributes for many configuration options, which contradicts many of the example configurations which are distributed with CCNet, but I don’t consider this as being a big issue.

Presentation: “Introduksjon til NServiceBus”

2010-05-26T20:31:00.001+02:00

On 25th of May I presented NServiceBus at NNUG Oslo. The presentation is now available at Slideshare (in Norwegian):

Introduksjon til NServiceBus

View more presentations from Sverre Hundeide.

EDA, messaging and NServiceBus at NNUG Oslo 25th of May

2010-05-11T23:51:00.001+02:00

On the next Norwegian .Net User Group I will give a talk on NServiceBus. Topics which will be covered include an overview of the architecture and capabilities of NServiceBus and configuration options provided by the framework.

Ole-Marius Moe-Helgesen will give an introduction to Event Driven Architecture and messaging and he will also share experiences from a project at an insurance company which made use of NServiceBus.

Jan Ove Skogheim Olsen will share project experiences with NServiceBus from Call Norwegian.

More information available here: http://www.nnug.no/Avdelinger/Oslo/Moter/Brukergruppemote-25-mai-2010/

Persistence with Command and Query Responsibility Segregation

2010-02-21T20:01:00.001+01:00

Command and Query Responsibility Segregation (CQRS) is a pattern where reading of data and commands for updating the domain model are separated into separate services. Architectures for distributed systems built on the CQRS pattern offers high scalability and reliability and has gained in popularity during the last couple of years.

Greg Young visited this month’s javaBin meeting in Oslo for a talk on CQRS based architectures, and in this blog post I will share some of the new insight I got into CQRS.

Event sourcing and eventual consistency are two essential concepts which fit well together with a CQRS based architecture, and previously I considered these two concepts to be mandatory in order to make the architecture scalable and reliable. However, the complexity introduced to a system by using these two concepts may scare many brave developers away from building real production systems which makes the most out of an architecture built on the CQRS pattern.

Most developers feel more comfortable with using well-known architectures built on a relational model stored in a RDBMS supporting ACID capable transactions. The mind shift required when changing to event sourcing and eventual consistency may seem too big and risky.

Figure 1, typical architecture utilizing the CQRS pattern

In the javaBin talk Greg Young actually advised against using eventual consistency when starting to implement a new system and rather gradually introduce the concept in parts of the system as it evolves and scalability issues appear. This will simplify the initial implementation and make it easier to get started with CQRS.

The simplest alternative: No event sourcing and no eventual consistency

This is the simplest option for handling consistency and concurrency because the domain model and denormalized read model can be updated in a single transaction.

The domain model and the denormalized read model can either be stored in the same database server or in different servers. A distributed transaction is required if updates are made on different threads or to different database servers, which will have an impact on the performance. An ORM is typically used for persistence of the domain model.

The read model can even be implemented as views on top of an existing schema modeled for OLTP.

Figure 2, Domain model and read model updated in a single transaction

Solving scaling issues as they arise: Event sourcing and no/partial eventual consistency

Greg Young prefers using event sourcing rather than a relational schema when persistence of the domain model. To quote his thoughts about ORMs: “Using an ORM is like kissing your sister!”

The events can for example be stored in a RDBMS, an object database, a document database or be serialize to flat files. The event store must support transactions.

As the system evolves and scalability issues surface, an event queue (and hence eventual consistency) for updates to the read model can be party introduced.

Figure 3, event store and read model is updated in a transaction.
An event queue and eventual consistency is introduced in areas where scaling issues arise.

The most complex and powerful alternative: Event sourcing and eventual consistency

In this alternative the domain events are stored in an event storage, and a queue is used to update the read model.

Two different queues can be used when updating the read model. The most traditional architecture is to publish the event to a separate queue in the same transaction as updates the event store. Tools like NServiceBus are typically used when publishing to the queue.

The second alternative was described in a recent blog post by Greg Young and uses the event storage as a queue. This means that there are no requirements for distributed transactions, as the only write happening when processing a command is to the event store. The read model is updated from the events in the event store and not from a separate queue. This has the advantage that there is only one version of the truth; it’s not possible to publish events which have different content from the ones stored in the event storage.

Figure 4, event store and event queue

Figure 5, using the event store as a queue

Conclusions

There is a wide range of options available for how to design persistence in a CQRS based architecture. The most important thing to consider is that the persistence requirements for the domain model on the command service usually will not conform well to the data retrieval requirements for the read service (think OLTP vs. OLAP).

Other factors which must be taken into consideration when designing the persistence models are the cost requirements, is it a greenfield or a brownfield project, the skills and competency of the developers, SLAs and enterprise architecture guidelines for the organization.

Domain-Driven Design: Strategic design

2010-01-22T19:33:00.000+01:00

Eric Evans visited the January meeting of Oslo XP Meetup for a talk about Domain-Driven Design, and this post is a summary of his talk.

Context mapping

Generic subdomains

A standardized domain which can be bought off the shelf, e.g. an accounting module.

Supporting subdomains

The parts of the system which is required, but which is not important enough do make or break your business.

The core domain

The core domain is typically built by 5-10% of a software system and is the areas and features of your software which are so important that they differentiate your business from your competitors’ businesses. The business should put all efforts into getting this part of the system as good as possible. The core domain will depend on the supporting domains.

Example:

The star rating of books at Amazon helps the customer get the right book. But rating is not strictly required for customers to buy books. The rating functionality is thus a part of Amazons supporting domain.

EBay also has a star rating system. This system doesn’t rate the product, but how trustworthy the seller is. Since trust is essential for a customer to buy something at EBay, their star rating system is a part of the company’s core domain.

“The hackers” and the core domain

Why do often the irresponsible, lesser skilled programmers who care nothing about good software design become the heroes in the organization?

From the customers’ and the leadership’s perspective, the heroes are the people who build the most valuable and useful features, and these features are often the core domain.

On the other hand, the skilled developers often focus on “platform” related architecture and features instead of the core domain. This may be fatal for the company if the “hackers” are allowed to build a badly design core domain which ends up in a Big Ball of Mud.

Good design has business value! Eric suggests putting your most responsible and skilled developers on the core domain and fire the bad ones.

Bounded Context - A system will always have multiple models

The enterprise model is a bad idea. Rather each team should build their own model with a clearly defined bounded context. This will result in clean, well defined models within each context instead of Big Ball of Mud models which try to do everything.

Strategic design and legacy systems

The conical dilemma regarding old legacy software is: Keep the existing system or build a new one from scratch?

Eric describes three different strategies:

1. Rebuild from scratch

This strategy will almost always fail and will certainly take much longer time than estimated. Eric advises against this strategy, unless there are really good reasons (most classical reasons often heard are simply not good enough!).

2. Refactor the existing system

Might work, but most likely the system will never reach the desired level of quality. Lesser skilled developers will continue hacking the system the same way as before, and degenerate the newly shiny refactored parts.

3. Continue hacking on the old system

This is the fate of most systems.

So what to do then? Eric suggests using anticorruption layers to isolates new well-designed domain models from the old legacy system.

Questions from the audience

Two different teams work on an almost identical domain model. Should they use a shared domain model or create on domain model per team?

Eric suggests that each team build their own domain model, even if they overlap and some code will be duplicated. The alternative will lead into creating a Big Ball of Mud.

My own comments to Eric’s talk

I’m currently working on a project where the goal is to replace several old mainframe systems with a new common system on a new platform. According to Eric, this project should have been doomed to fail. In it’s forth year of development and 2,5 years in production the project is still going strong, albeit it has gone through several challenging and difficult periods.

So why hasn’t this project failed then? I think the reason is that the organization I work for previously has made a couple of failing attempt to replace the old legacy systems. This made them learn which costs and efforts are needed to create a new system which replaces the old ones.

The new system will also give a competitive advantage with it’s flexibility in defining new product and price structures, and this added business value is also a motivation for continuing the project.

LEAP Conference – day 3

2009-06-18T01:39:00.002+02:00

A summary of day 3 of the LEAP conference in Redmond, Seattle

Sync: Why, what and how

With Lev Novik

Why Sync?

Facilitates rich clients

Faster response, richer UX

Legacy applications can be migrated to use the Cloud as a data storage by using Sync
…

General Sync Challenges

Granularity of changes
Change (non-) Reflection

Using a timestamp. Use locking until synchronization is finished?

Conflicts

Not detecting conflicts will result in data loss
Complex algorithms for conflics detection exists, which don’t require storing the history of all changes

Loops

Multiple devices synchronizing data to multiple servers at the same time
Can result in duplicated data

Hierarchical data

The order of synchronization is important
Eg. one endpoints adds an item to a folder, while another endpoint deletes the entire folder

Item filtering

Optimization by syncing parts of the data more frequently

“Column” filtering

Select parts of the data
Challenge: Can’t do conflicts detection, since one of the endpoints don’t have the complete version of the data

Errors and interruptions

Not all conflicts can be solved automatically

Doing so will result in loss of data
Must wait for a human to resolve them

Microsoft Sync Framework

What does MS Sync Framework do?

Makes it easy to sync participating endpoints

Build in endpoints for

V1: File system, relational databases
V2: SQL Data Services, Live Mesh, ++

The Sync Session

Data stores implements a Sync Provider
The Sync application has a Sync Orchestrator which communicates with the endpoints’ sync providers
Synch Framework Runtime

Metadata

Versioning

Runtime

Algorithms to solve sync problems

Metadata Store

For those who can’t store the metadata themselves

Simple Provider Framework

Makes writing providers easy

How do customers use the sync framework?

Write sync applications

Implement synch orchestration

Write sync providers in order to support sync

Declare an object identifier
Declare versioning
Enumerate changes

Sync Participants

Sync endpoints

Stores metadata
Can be many kinds of devices, and the sync logic should not be implemented for each of them

Sync providers

Does most of the sync work
Operates on the endpoints’ meta data

Sync application

Has the Synch Orchestrator

The sync logic can be placed in different locations (eg. on the client or in a web service) for differenc scenarios.

Sync Framework on MSDN: http://msdn.microsoft.com/sync/

Visual Studio Team System: ALM as we do it at Microsoft

With Stephanie Cuthbertson

Some facts about Microsoft Development

TFS usage at MS

VS 2008

13 000 users
2 570 000 work items
40 100 000 source file

Planning and tracking

Feature planning and prioritizing in the development of VSTS 2010

Value props prioritizing

Voting and weighting/prioritizing of features in an Excel sheet
Work items are then imported to TFS

Generate MS Project GANTT from TFS

VS 2010 demo

Simple task editing integration with Excel and MS Project
Improved forecasting statistics and status reports
User requirement tracking

Can edit requirements through a web interface

Requires a separate (new) licence

Can link requirements to test cases

Branching

In the development of VSTS 2010, branching per feature is used
Feature must pass “Quality Gates” before merging into active branch

Feature complete
Test complete
All bugs fixed
Static code analysis
Localization testing
etc

Tracking and reporting in VSTS 2010

Better SharePoint integration
Web dashboard

Extensive statistics and analytics possibilities

Always Responsive Apps in a World of Public Safety

With Mario Szpuszta

A case study for a system for ship tracking and tracing delivered to Frequentis.

Who is Frequentis AG?

Provides systems for

Air traffic
Ship tracking & tracing
Coordination systems for police offices

Terms

MCS- Maritime Communication System

Ship – Ship, Ship – Land, Land – Land
Usually hardware interface

CAD – Computer Aided Dispatching

Collaborative Incident Management
This is the kind of software made in this case study

TnT – Tracking and Tracing

CAD and MCS Solution from Frequentis

Tracking & Tracing Architecture

GUI in WPF

Several modules
Complex requirements

Lots of information and operations available for the users

Could not use CAB, Prism or similar frameworks since the GUI would then run in one process and one app domain. The entire system should not go down if one module crashes.
Each GUI module runs in a separate process. A separate shell was created in order to achieve this.

Communication with Maritime Communication System with .NET remoting
GUI communicate with the services through a message bus
Server

WCF service modules
Windows 2008 and SQL Server 2005

The Service Bus

Complex communication

Everyone communicate with everyone

Failure of one system may not affect others
Challenges

Not every harbour can pay for the required infrastructure, like huge server farms
Failure of single entity may not affect others

Classic architecture

Keep it simple

Lightweight
Reliable

Loosely couples
Many-to-many communication

Solution

Created custom Message Subscription Database
Use WCF Peer-to-Peer channel for communication

Issue: Max. 700 msg/sec limitation due to slow serialization
No Duplex-bindings, no MSMQ

Just leverage NetTcp-bindings

Tech-hints for WCF

NetDataContractSerializer will include assembly info – serialization will fail if endpoints have different assembly versions, even though the contracts are compatible
DataContractSerializer enables loosely coupling

Creating a responsive user interface

The application may never hang at any time
Encapsulate logic in “autonomous” tasks
Set of jobs executed based on commands
Core rule: Everything executed asynchronously

Thread pool with queue and queue manager

Commands, Jobs and Queues

Business logic encapsulated into Jobs (and ONLY there)
Commands executed autonomously without side effects

Results from Async Jobs

Modules implements INotify interface

Passed into the constructor of a job
Job calls back to module through INotify

Communication with other systems

Create yet another job
Job talks to IConnectionPoint

Tasks, Jobs – Tech Hint

CCR (Concurrency Coordination Runtime, originally from The Robotics Studio)

Simplified execution of concurrent tasks
Has now been released as a separate toolkit separated from Robotics

WPF-based client

Why WPF?

Huge amount of information needed to be presented
Frequentis hired a separate UX-research team

Different alternative UX-stories were investigated

Advanced requirements for alternative visualizations of data

Presentation Model Pattern

Separate UI from code

Green Computing through Sharing

With Pat Helland

Introduction

In 2006, 1,5% of the electricity in US was consumed by Data Centers

This is more than what is consumed by TVs
Projected to double every fifth year

Sharing resources vs. dedicated resources

Shared resources may not be available when you need them
Dedicated resources are expensive and have less utilization

Sharing through

Virtual machines
Cloud computing

The evolving landscape of data centers

Power Usage Effectiveness (PUE)

PUE = Total Facility Power / IT Equipment Power
Typical factor is 1.7

Power and cooling is expensive

Infrastructure and energy cost are both more expensive than the server cost

Redundancy

Represents more than 20% of the data center cost
All servers require

Dual power paths
Dual network

“Chicago Data Center”

Highly efficient data center with PUE = 1,2
Servers are located in isolated steel containers, each containing 2 000+ servers

Individual servers are never maintained

Over-Provisioning versus Over-Booking of Power

Power Provisioning

Total power consumption for a server is typically 200W
Power consumption typically peaks at about 90% for a data center

Theoretical max power consumption is seldom used, eg. because disk usage prevents 100% CPU utilization
This means that it is possible to add more servers than the theoretical max limit in order to utilize the available power

Services and Incentives

Amazon’s Server Oriented Architecture

One page request typically use over 150 services

Service Level Agreements (SLAs)

Example: 300ms response for 99.9% of requests with 500 requests per sec

What does this mean for developers?

Factories are more efficient than hand-crafted manufacturing

LEAP Conference – day 2

2009-06-17T01:43:00.001+02:00

A summary of day 2 of the LEAP conference in Redmond, Seattle

Pharma in the Cloud

With Eugenio Pace

Windows Azure Primitives:

Code host
- WCF
- ASP.Net
- Worker
  - Similar to a windows service in an on-premises application
Persistence
- Table
- Blob
- Queue
  - Events can be published to the queue, and Workers can handle these events
- SQL Data Services
  - Supports most of the functionality of the regular SQL Servers
  - Funtionality has been significantly extended since the PDC08 demo
Application Services
- ACS (access control)
- ServiceBus
- Workflow

Cloud vs. on premises and build vs. buy

Building a multi-enterprise collaboration application in the cloud for “BigPharma”

Requirements:
- De-centralized management
- Fine grained access control
  - Org –> Row –> Field
- Leveraging existing Identity and AuthZ infrastructure
  - Using Active Directory (local users/groups used in demo)
  - Support Single Sign On

Demo: http://pharmacloudcatalog.com/catalog/Provisioning

Identity & Access Control
- Using claims-based identity
  - Both for the web service and for the web site
- Using MS Geneva Framework
  - Identity providers for ASP.Net exists which support this framework
- Custom Security Token Service (STS)
  - Inheriting from SecurityTokenService
  - Starter kit available at: http://startersts.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=27842#ReleaseFiles
- Mapping tokens to permissions can be done in the web interface of .Net Services Access Control Service
ServiceBus
- Enables communication from the server to the client without requiring an inbound connection to the client (all connection from client/server are outbound – to the service bus)

Download & study sample for Azure (note: a different samle than the one demonstrated in this session) http://www.codeplex.com/azureissuetracker

Microsoft Dynamics CRM

With Girish Raja

Dynamics CRM 4.0 demo

The Outlook CRM add-in client
- Appears as a separate folder in Outlook
- Data available offline
Flexibility
- Accessing by browser, Otulook or mobile
- Hosting as software or as service
Extensibility Toolset – customization tools for
- System Administrators
- Developers
- Business Analysts
Configurable entity model
- Create entities (similar to database tables) from the Customization screen in Dynamics CRM Online
- The asmx web service endpoints are automatically updated with the custom entities
- Configurable role based access with high granularity
Workflow editor
- Uses Windows Workflow internally
- Activities can be created in the web interface

.NET Service Bus

With Clemens Vasters

Demo application where a website in the cloud communicates with an application running on premises (on Clemens’ laptop) through the .NET Service Bus
- No need to configure firewalls
- Security kept intact
Why .NET Service Bus?
- Enable bi-directional connectivity
  - Not depended on the kind of device or the location of the device
  - Without having to open inbound firewall/NAT ports
- Provide federated naming and discovery
The first version of the service bus (will be released with Azure in November 2009), will use Windows Workflow from .NET 3.5, with an DSL on the top for supporting migration to .NET 4.0
NetTcpBinding is the preferred one for optimal performance
Service Bus Naming
- Hierarchical structure, similar to DNS
- Updates takes effect immediately
- Naming scheme: scheme://solution.servicebus.windows.net/name/…
What’s wrong with DNS?
- High latency for updates
- Names hosts, not services
Service Registry
- A registry for service endpoints
- Services can be categorized (eg. printers can be organized into a separate category)
Service Bus Messaging
- Based on WCF
- Not supported:
  - Atomic transaction flow
  - Protocol level transport authentication

More information about Service Bus on MSDN: http://msdn.microsoft.com/en-us/library/dd582728.aspx

SQL Data Services – Under The Hood

With Gopal Kakivaya

Motivation

Database as a service
- Pay-as-you-go model
- Guaraneed SLA
- Familiar relational programming model
- Leverage existing skills and tools
  - This is new compared to the PDC08 version
- Full control of the logical database administration
- The physical aspects of the database administration is handled by the service provider

Concepts

Database Provisioning Model
- Account
  - Each account as one or more servers
- Server
  - Has one or more logins
- Database
  - Users
Connection Model
- Clients connect directly to a database
Security Model
- Uses regular SQL security model
  - Username + password
- Future: AD Federation, etc

Architecture

Components
- Master node
- Data Nodes
  - SQL Server
    - Replication Agent
    - Local Partition Map
  - Fabric
    - Reconfiguration Agent
    - PM Location Resolution
    - Failure detector
    - Ring Topology

One partition is set as partition, and one or more secondary partitions are located on other data nodes.

Partitioning
- Provides better fault tolerance
- Failed partions can be rebuilt faster
  - Eg. if the database is divided into 10 partitions, it’s much faster to rebuild the failed partition than the entire database
Fault tolerance
- Security built into the software
  - Signed data, eg. will detect if the network card has corrupted the data
- Can be used on cheap hardware
  - If anything fails (eg. a disk), the faulted hardware will automatically be shut down
    - This is made possible by the use of replica sets
Replication
- Reads are completed at the primary
- Writes are replicated to all nodes
  - The primary partition will wait for acknowledges from the secondaries
  - All writes, both to the primary and to the secondaries are part of the transaction
- The replication factor may be configured, based on the customer’s demand
  - “Replication factor of 4” means that there are 1 primary and 3 secondaries
Reconfiguration
- As machines die, new machines must take their place
- Types of reconfiguration
  - Primary failover
  - Removing a failed secondary
    - Might be temporary, eg. because of an update made to the machine.
    - The secondary will not be replaced immediately, since it might be temporarly down
  - Adding recovered replica
  - Building a new secondary

LEAP Conference – day 1

2009-06-16T01:43:00.001+02:00

A summary of day 1 of the LEAP conference in Redmond, Seattle

Keynote

With Scott Guthrie

Rich web

AJAX & HTLML4/5 and Silverlight
Silverlight 3
- Ships in July 2009
- Runs inside and outside the browser
Demos (http://www.iis.net/media/)

Smooth streaming – adaptive bitrate
The streaming server is free
Supports pre-recorded and live content
Content is cached on servers local to the user – one webserver can serve a large number of clients
Demo client/server application created with Silverlight template

Using navigation template

“Out-of-browser” settings
- Can use GPU acceleration
- Support for context menus (right-clicking) will be added to Silverlight 4
Expression Blend 3
- Ships in July 2009
- New feature SketchFlow for sketching/prototyping UI
  - Use multiple sources like scanned images, pictures etc
  - Create workflows
  - Wiggly Styles
  - Separate skin which looks like a hand-drawn image
  - Without colors – focus on functionality and usability
  - Looks similar to Balsamiq Mockups
- Photoshop import
  - Supports selecting layers
- Sample data
  - Creating, editing and styling
  - The designer person can get the application working with testdata without being dependent on the developer
  - Disigner import for Silverlight will be significantly improved in VS 2010
Web platform installer
- http://www.microsoft.com/web/
- Supports install of IIS Media Server 2.0
- Search engine optimization tool
  - Generate site analysis report

Multi-core

How can developers take advantage of multi-core CPUs?
- Shift from how to do things to what to do:
  - Using LINQ and lambda expressions
  - The framework can then utilize multiple CPUs by partitioning the execution into different chunks which can be run in parallel on different CPU cores
The ASP.Net core has been configured to support multi-core parallelism as default in .Net 4
.Net Parallel Extensions
- Net parallel task debugger window in VS 2010
  - Easier navigation between threads and tasks

Automated Testing

VS 2010 has better support for TDD
- New mode for working with classes which hasn’t yet been created (switch into this mode by pressing Ctl + Alt + Space).
- Automatic generation of classes and methods based on the test
- Demo – Scott referring to the AAA (Arrange – Act – Assert) pattern (which is good!)
Manual test tools

Cloud

The same .Net runtime binaries are used on-premise and in cloud
- ASP.Net
- SQL Server
- WCF / Workflow

SharePoint

Built-in support in VS 2010
Features include:
- Projects
- List
- Web-Parts
- Workflows

SharePoint Patterns and SharePoint futures

With Paul Andrew

Development patterns and practices

http://www.microsoft.com/spg

The SharePoint Development Lifecycle

Memory management

Use the SPDisposeCheck utility
SharePoint APIs return IDisposable objects

Deployment

VSeWSS 1.3
- Extension to VS
- Simplify deployment with the “Package” feature
  - Creates the .wsp file in one command
- Automatic file renaming
- Deployment conflict resolver
- Deploy additional assemblies

Futures

This information was confidental until the Sharepoint conference in Las Vegas in October 2009, and will not be covered in this post.

Patterns & Practices roadmap

With Eugenio Pace

P&P FY09 Programs

Client development
- Prism (WPF and Silverlight)
- Web Client
- Mobile Client
Server development
- SharePoint Guidance
- Services Development
- Web Service Security Guidance
- Enterprise Service Bus
- Web Service
Solution Development Fundamentals
- Enterprise Library
- Application Architecture Guide
- Testing Patterns & Guidance
- Data Access Guidance

What’s coming in FY 10?

Client
- Prism 3, expected March 2010 (WPF 4.0 / SL 4.0)
- Web Application Guidance (ASP.Net, MVC, jQuery, Dynamic Data)
Server
- SharePoint Guidance, April 2010 ()Internet Scale, Silverlight, LOB, Office 14)
Services
- Cloud Identity Management Guidance, November 2009 (Geneva, Azure Services, LiveID)
Fundamentals
- Enterprise Library 5.0, March 2010
- Data Access Guidance, March 2010 (Domain Driven Design, EF 2.0, Astoria, .Net RIA Service)
- Application Architecture Guide 2^nd Edition, November 2009
- Acceptance Testing Guide, November 2009

CloudLib

Reuses existing blocks for
- Exception handling
- Validation
Extensions to existing blocks for
- Security
- Log
New blocks added for the cloud
- DataAccess (SDS)
- Tables
- Config
- Blob
- Queue
- Worker

How to work with the P&P team?

http://www.microsoft.com/practices/
http://www.codeplex.com/
Affect the P&P backlog by getting involved in the community

Rich Internet Applications

With Ian Ellison-Taylor

Silverlight 3

3D support
“Out of browser” demo
- Select “Desktop shortcut” when installing
- Still hosted in the browser internally
  - This is not visible to the user
  - Not possible to access the browser’s XML DOM
New capabilities
- Media
  - Smooth streaming
  - More format choices (like H.264)
    - More efficient decoding – uses less resources
  - Fullscreen HD playback
  - Extensible media formats
  - Content protection
Graphics
- Perspective 3D Graphics
- New Bitmap API
- Enhanced Control Skinning
- Bitmap Caching
  - Performance improvements
- Themed App Support
  - Supported in Blend 3
- Improved Text Rendering
  - Crisper text
  - Faster rendering
  - Support for more languages (about 30 in total)
  - Better layout algorithms
Dev Productivity
- Controls (60+)
  - Datagrid
- Search Discoverability
  - Control which information to make available for search robots
- .Net RIA Services Framework
- Improved Performance
  - Targeted for big applications
- Advanced Accessibility
Out of Browser
- Run Apps Out of Browser
- Desktop & Start Menu
- Safer & More Secure
  - Still running inside a sandbox, same security access as when running in the browser
- Smooth Installation & Auto Update
- Windows Integration
  - Better support for Windows 7, including touch and new start menu
- Connectivity Detection
  - Detects network connection status
Design Tooling
- Prototyping w/ SketchFlow
  - Sketch out random ideas
  - Link them together using workflows
  - Real controls are used under the covers
    - The scetch skin makes the user focus on the functionality – not on visual details like colors and fonts etc
- Visual Design Workflow
- Accessibility Interactivity
- Design w/data
- VSTF Integration
- Design Surface Extensibility

Windows Forms

Will still be supported for many years and will be continued to be developed

A simple and compact style for BDD specifications

2009-06-01T10:56:00.001+02:00

When appropriate I prefer to use the testcase-class-per-fixture style for writing BDD style contexts/specifications.

However, when testing small systems where there is only one specification per context, the testcase-class-per-fixture syntax becomes overwhelming and cumbersome to user.

For this reason I’m sometimes using a more compact format for the specifications:

        1: public void $Method$_Given$Context$_Should$ExpectedBehaviour$()

$MethodName$	The system under test (SUT), will often be a method name
$Context$	The situation/scenario
$ExpectedBehaviour$	The expected outcome of the given context

Example specifications:

       1: namespace BDDSimpleStyleExample

       2: {

       3:     [TestClass]

       4:     public class DateTimeSpecifications

       5:     {

       6:         [TestMethod]

       7:         public void AddDays_GivenTodayIs1stOfJanuaryAndAddingTwoDays_ShouldReturnJanuary3rd()

       8:         {

       9:             //Arrange (setup up context / fixture, usually multiple lines of code)

            var today = new DateTime(2009, 1, 1);

  

            //Act (one line of code, invoke the System Under Test)

            DateTime result = today.AddDays(2);

  

            //Assert (Usually ONE assert / specification)

            Assert.AreEqual(new DateTime(2009, 1, 3), result);

        }

  

        [TestMethod]

        public void AddDays_GivenDateIsLastDayOfMonthAndAddingOneDay_ShouldReturnFirstDayOfNextMonths()

        {

            //Arrange 

            var today = new DateTime(2009, 1, 31);

  

            //Act

            DateTime result = today.AddDays(1);

  

            //Assert

            Assert.AreEqual(new DateTime(2009, 2, 1), result);

        }

    }

}

The disadvantages of using this style is that the method names in the test class may get very long, and the test results output isn’t formatted as good as it would have been when using one test case class per fixture:

The following ReSharper live template can be used to quickly add new specifications/tests:

       1: [TestMethod]

       2: public void $Method$_Given$Context$_Should$ExpectedBehaviour$()

       3: {

       4:     //Arrange 

       5:     $body$

       6:  

       7:     //Act

       8:  

       9:     //Assert

      11: }

LEAP part 3 - SOA

2009-03-29T19:59:00.001+02:00

The third master class: Loosely Coupled Business Systems: SOA on the Microsoft platform

Udi Dahan - The Software Simplist had been hired to present the third LEAP master class in Oslo. He is an well known international expert on enterprise software architecture and design, and is the author of the open source messaging framework nServiceBus.

The entire class was based on discussion and interaction with the audience, and the only Power Point slide used was the one showing the agenda.

He started out with sketching a naive traditional n-tier application (big ball of mud), and based on suggestions from the audience we explored different solutions which might improve the solution. Whatever suggestions we threw at him, he always had a thoroughly considered answer describing pros and cons with the suggested solution. He obviously has a lot of experience with real world enterprise SOA applications.

The goal was to create autonomous services - standalone services with loose coupling to other services. The system should be scalable and reliable and use as few resources as possible.

Topics discussed

Coupling

Avoid coupling by slicing independent logic into separate vertical autonomous boundaries / services
- Example of services: Order, inventory management, billing, shipping
How should services communicate with each other?

Where to put the orchestration/workflow logic?

Layer on top of the business layer components?
In an Enterprise Service Bus on the side of the services?
In the GUI?

When to use...

Fire and forget
Messaging (async / synchronous)
Events

Duplicate data in order to remove dependencies and keep services autonomous?

How to synchronize the data?
- Publish / subscribe pattern can be used. E.g. when an address is updated in the customer service, the customer service can publish the updated address.
- Context of the update is important. Why did the change happen? This is information which is available close to the user and the business process (i.e. NOT at the data base tier), and is usually triggered from the UI.
- Versioning - published events have to be backwards compatible with subscribers
Duplication of data is usually considered a bad practice, but for SOA it may have advantages:
- Avoid service calls (e.g. retrieve customer address RPC style when needed)
- Services become more reliable and autonomous. What if the customer service is down? The the shipment service won't be able to do it's work since the address can't be retrieved.
- The duplicated data can be considered a local cache for the service

Publish/subscribe vs. request/response

Doesn't both solutions create dependencies between services?
- Request/response creates design-time dependencies
- Pub/sub makes it possible to create services which are both design-time and run-time autonomous
  - Run-time autonomous: Will continue to work even when other services goes down
  - Design-time autonomous: Has no direct references to other services
Pub/sub advantages:
- Better performance
  - Local cache (duplicated data) in each service
  - No blocking transactions across service boundaries
  - Easier to run processes in parallel - scales better
  - Fire and forget gives faster response in the UI
- Better resource utilization
  - Messages can be queued up and processed when resources get available
Pub/sub disadvantages:
- Systems might be harder to design correctly (requires untraditional thinking)
- Systems get harder to understand and debug
Request/response advantages:

Explicit service calls / dependencies makes the system easier to implement, understand and debug

Request/response disadvantages:

Synchronous blocking architecture requires more resources

Ties up more resources over longer time periods, ie. the system will not scale well.

Choose the right architecture for the right place, there is no silver bullet.

Achieving autonomous services by letting each service have it's own UI

Consider a shipment service which requires a shipment address. The traditional design would be to let the shipment service call the customer service. Another option would be to let the shipment service collect the shipment address through it's own GUI, and thus keeping the service autonomous.

LEAP part 2

2009-03-03T00:14:00.001+01:00

The second master class: The Microsoft Data Platform and Business Intelligence

The second master class of LEAP Norway was presented by Jon Jahren from Microsoft Consulting Services Norway.

This master class gave an overview of the Microsoft products and architectures for Business Intelligence (BI).

Master class summary

BI introduction

Competitors
How to introduce BI to a company
Microsoft's vision and strategy for BI
Demo of current Sharepoint BI solution
Demo of data mining in Excel 2008 SP2

Retrieve data from Analysis Services
Generate forecast

Microsoft's BI stack

Office, Sharepoint Server and SQL Server

BI technical architecture

ETL (Extract, Transform and Load) -> Presentation Server -> Client
Kimball method fundamentals for Data Warehousing
The Star Model
Biztalk vs. SSIS

Biztalk is message-based
Both products have very similar features, but SSIS have much better performance
In the latest versions, both can share the same adapters (WCF based)

SSIS demo
OLAP demo

Create Data Source View
Create Cube
Deploy
MOLAP with realtime proactive caching
Microsoft's strategy is to use ONE cube for all perspectives

A "perspective" is a view of the cube for a specific report
All fact and dimension tables in ONE cube

Metadata

Use Sharepoint for metadata
MDM - Master Data Management

Avoid duplicate entities
Estimated 5 - 15% of all data is duplicated in large enterprises
A new MDM application platform is to be released by Microsoft
Business Data Hub

In the cloud

PAXOS

Algorithm for fault-tolerant distributed computing

The future of BI

Load -> View -> Model

Create model automatically based on usage, instead of creating the model first

IMDB

In-memory storage for performance
Is the underlying engine for self-service BI
SQL Server Gemini

Add-in to Excel 14 for self-service BI
Supporting millions of rows

Next master class: SOA on the Microsoft platform with Udi Dahan

Technorati Koder: LEAP

Arrange, Act and Assert syntax for testing

2009-02-21T22:09:00.005+01:00

This post is a summary of a lightningtalk I held for some of my colleagues at Objectware back in August 2008.

What is Arrange, Act, Assert (AAA)?

AAA is a way to organize your tests. The contents of the tests are divided in to three parts:

Arrange: Do the necessary setup required for running the test. Usually multiple lines of code.
Act: Execute the code which should be tested. Should be one line of code.
Assert: Verify that the code behaved as expected. Should contain only one assertion.

Example of a test without AAA

[TestMethod]
public void GetCustomerById()
{
// Arrange
const int CustomerId = 5;
var repository = new CustomerRepository();

// Act & Assert
Assert.AreEqual(CustomerId, repository.GetCustomerById(CustomerId));
}

The same test written in AAA style:

[TestMethod]
public void GetCustomerById()
{
// Arrange
const int CustomerId = 5;
var repository = new CustomerRepository();

// Act
Customer customer = repository.GetCustomerById(CustomerId);

// Assert
Assert.AreEqual(CustomerId, customer);
}

The example above is simple, but the value of using AAA style becomes more apparent for complex tests.

Why using AAA syntax

The tests become more robust, easier to read and to maintain
Being able to use AAA syntax is essential when writing Behaviour Driven Development style specifications/tests

AAA and mocking

In the following example, which parts of the code are arranging, acting and asserting?

The record/playback style should not be used, since it results in tests which are very hard to read and understand.

The same test as above written in AAA style:

[TestMethod]
public void ShouldGenerateCreditInvoiceDocumentWhenNegativeAmountNew()
{
// Arrange
ICustomerService customerService = MockRepository.GenerateStub<ICustomerService>();
IDocumentService documentService = MockRepository.GenerateStub<IDocumentService>();
double amount = -1000;
Customer customer = new Customer() { Id = 12345, Name = "Name1" };
customerService.Stub(stub => stub.GetCustomer(customer.Id)).Return(customer);
 InvoicingProcess invoicingProcess = new InvoicingProcess(customerService, documentService);

// Act
 invoicingProcess.Invoice(customer.Id, amount);

// Assert
documentService.AssertWasCalled(stub => stub.CreateDocument(customer, DocumentType.CreditInvoice));
}

Mocking frameworks supporting AAA syntax

The following mocking frameworks supports AAA:

Rhino Mocks 3.5
Moq
NUnit
Typemock Isolator 5.0

LEAP part 1

2009-01-22T22:55:00.001+01:00

Technorati Koder: LEAP

About LEAP

Earlier this week I attended the first out of five classes of Microsoft's new Lead Enterprise Architecture Program (LEAP). The goal of LEAP is to educate a group of Microsoft partners and system integrators on the Microsoft platform. During the spring, 5 Master Class training sessions will be held at Microsoft's office in Oslo, and in June LEAP participants from all over the world will meet for one week in Redmond, Seattle.

This is the first time LEAP is organized in Norway, and if successful, it probably will be repeated later on.

The first master class

The topic of the first master class was N-tier and N-layer architecture and the class had about 40 participants.

Fortunately, Microsoft did not just give a presentation in the traditional drag and drop create-an-application-in-five-minutes style. Instead, the well known software architect Jimmy Nilsson had been engaged to run the show. He's best known for being the author of the excellent Applying Domain Driven Design and Patterns.

Instead of focusing on products, he advocated Test Driven Design/Development, Domain Driven Design, code quality and object orientation. Much of the time was spent on group discussions.

The following topics were covered:

Layering
- Focus on the Domain Model pattern
Data access
- O/R mapping, entity framework
- Suggested to use EF POCO Adapter for those who would like to start using Entity Framework v1 and have a migration path to EF v2
UI patterns
- MCV, MVP, MVVM
Case study
- Worked together in groups to suggest the most appropriate architecture for a specific case

Mounting ISO images in Windows 7 beta 1

2009-01-17T16:36:00.005+01:00

Technorati Koder: Windows 7

Yesterday I installed Windows beta 1 on the laptop which I use for training.

My employer provides me the Visual Studio installation CD as an ISO image, but when getting to the stage of installing it, I discovered that none of my regular ISO mounting tools (like DEAMON Tools) worked on Windows 7.

Fortunately, I found this small, non-intrusive and free mounting tool called Prismo File Mount. It installed in a second, and is really easy to use.

Using “Quick mount” on the ISO file transformed it into a browsable directory from which I could install Visual Studio, similar to the Compressed Folders shell extension for zip files built into Windows Explorer.

Using the “Mount…” option, it is possible to map the image as a drive, if desired.

UPDATE: Prismo File Mount does not work for 64-bit editions of Windows 7. MagicDisc is a good free alternative in case you are running 64-bit Windows 7.

My podcast subscriptions

2008-12-30T01:04:00.002+01:00

Technorati Koder: Podcasts

Podcasts are a great way to keep yourself updated on topics of your interests. I usually listens to podcasts when sitting on the bus back an forth between home and work and when exercising at the fitness centre.

For .Net developers, there exists several good podcast publishers. My favourites audio podcasts are:

.Net rocks: http://www.pwop.com/feed.aspx?show=dotnetrocks&filetype=master3
Hanselminutes: http://feeds.feedburner.com/HanselminutesCompleteMP3
Alt.Net Podcast: http://feeds.feedburner.com/altnetpodcast
Herding Code: http://feeds.feedburner.com/HanselminutesCompleteMP3
Polymorphic Podcast: http://feeds.feedburner.com/PolymorphicPodcast

I'm also subscribing to a few non- .Net podcasts. For these ones, I put some more efforts into filtering out non-relevant shows.

Agile Toolkit Podcasts: http://agiletoolkit.libsyn.com/rss
Software Engineering Radio: http://feeds.feedburner.com/se-radio?q=rss
Ruby on Rails Podcast: http://feeds.feedburner.com/rubyonrailspodcast