Smoothfriction.nl

Taking your startup to the next level: meet the Joel Test

Erik van Brakel — Sun, 01 Jan 2012 22:13:13 GMT

Tomorrow, I will start my second year at my current employer. The company is fairly small, when I started there were two developers and the managing director. Effectively it was just the two of us though. Additionally, the managing director actually has a technical background, and trusts our judgment. This left us with a lot of room to make our own decisions on what’s important. There’s around 6 of us now, and things are going nice and steady.

As one of my responsibilities I’ve decided on making the whole development process as painless as possible. I could of course try to think of all sorts of things to do myself, but luckily, there’s already a pretty good list out there already. Joel Spolsky, one of the people behind Stack Overflow (the Q&A site for and by software developers) wrote an article about this back in August 2000. Ages ago in software development terms. However, the list is still valid, and my aim was going to be to meet as many of the requirements I could.

Do you use source control?
Can you make a build in one step?
Do you make daily builds?
Do you have a bug database?
Do you fix bugs before writing new code?
Do you have an up-to-date schedule?
Do you have a spec?
Do programmers have quiet working conditions?
Do you use the best tools money can buy?
Do you have testers?
Do new candidates write code during their interview?
Do you do hallway usability testing?

The Joel Test has 12 requirements. According to Joel, a proper software development team should score at least a 10 out of 12. Being a startup company, it’s probably expected that you wouldn’t be there right away. Additionally, doing mainly green-field development, some of these are not directly applicable. However, with some creativity it’s quite possible to find a solution for each one.

For completeness sake, here’s the scoring before we started working on it:

Do you use source control?
Yes. We have a subversion server up and running, and we use it actively.

Can you make build in one step?
No.

Do you make daily builds?
No.

Do you have a bug database?
Yes. We use Mantis for that, and keep it as up-to-date as possible.

Do you fix bugs before writing new code?
No. Not by rule.

Do you have an up-to-date schedule?
No. We only have the main project deadlines to go by.

Do you have a spec?
No. We just wing it.

Do programmers have quiet working conditions?
No. We share an office with a consulting company. Lots of phone calls.

Do you use the best tools money can buy?
Yes. As far as money allows.

Do you have testers?
No. We test our code ourselves.

Do new candidates write code during their interview?
Yes. Or at least show code from pet projects and explain it.

Do you do hallway usability testing?
No. We simply don’t have people to do that between the two of us.

Adding everything up this gives us an appalling 4 out of 12. Plenty of room for improvement I’d say!

For my next posts I will explain how we’ve gone from a 4 out of 12 to somewhere between 8 and 10 out of 12.

Building your own Reflector with Mono.Cecil

Erik van Brakel — Sat, 12 Feb 2011 20:04:32 GMT

After writing my previous post, Building your own Reflector with Mono.Cecil and the CodeDom, I decided to find other initiatives to writing a ‘new Reflector’. I’ve now joined the Monoflector effort, working on filling the gaps in the current decompiler. It seems a good fit, because I already knew a few things about it after my small spike.

We’re working with the decompiler which is part of the Cecil project. However, there are quite a few bugs or just lacking features which makes reading the generated code quite frustrating. We’ve forked Cecil and are working on adding the features we need.

If you want to check out the project, I invite you to go to our github page at http://github.com/jcdickinson/Monoflector. Feel free to fork the project and get your changes in, but keep in mind that at the moment of writing, the GUI is under quite heavy construction. It might not be a good idea to work on that, because it will be quite hard to integrate your changes into the main branch.

Building your own Reflector with Mono.Cecil and the CodeDom

Erik van Brakel — Mon, 07 Feb 2011 01:23:05 GMT

Sparked by the news that Red Gate’s Reflector will not be available for free anymore starting early March (see announcement on the redgate site), I decided to try how hard it would be to build your own reflector. Turns out, not too hard if you use the tools at hand. There’s actually already a nice effort going on based on the Mono.Cecil.Decompiler, but I only found out after I started my trials.

What I basically did was get Mono.Cecil to read an assembly, hand me the MSIL statements and simply build an AST (Abstract Syntax Tree) out of that using the .NET CodeDom available in System.CodeDom.Compiler. Combined with the CSharp or VB code generators, available in Microsoft.CSharp and Microsoft.VisualBasic respectively, I managed to more or less decompile System.Object into valid C# code.

The code very rough, and not tested more than just reading and converting the MSIL by hand and using that to check. However, I did find that for most things, a decompiler like this is not very hard to build. Just very time consuming. You can check out the code at my github repo, here. Most of the work is in the ILToCSharpConverter.cs file, which simply converts most of the MSIL from the System.Object class into an AST, which then gets fed through the code generator.

Note: I excluded the GetFieldInfo method, because I ran into an infinite loop and didn’t want to try to fix that before publishing. Any other classes are NOT tested, and therefore will probably break. Also, the code generated is hardly complete or optimal, but it should more or less convey the meaning!

If you want, have fun hacking away. Might actually be fun to get this off of the ground, no?

PowerShell fun: Project Euler #3 - The primary factors are…

Erik van Brakel — Sun, 30 Jan 2011 16:52:11 GMT

Onwards, to Project Euler problem #3. The problem is defined as below:

The prime factors of 13195 are 5, 7, 13 and 29.

What is the largest prime factor of the number 600851475143 ?

So, basically, we need to find out which factors make up a certain number. The easiest way to do this is simply by trial division. A very naïve implementation would be something like this:

function Get-PrimeFactors ([long]$number)
{
    while ($number -gt 1)
    {
        for($i = 2; $i -le $number; $i++)
        {
            if($number % $i -eq 0)
            {
                $number = $number / $i;
                $i;
                break;
            }
        }
    }
}

To be honest, for a number of this magnitude even the simplest of solutions will work. I’ve been looking at different integer factorization algorithms (using my trusty pal Wikipedia), but none of them is simple enough to actually fit the problem. Most of them are actually really suitable for the problem, if the number was much, much bigger.

However, there are a few optimizations to be made. For instance, we can assume that we only have to check for divisors up to the square root of the number in question, because anything higher than that has a pairing divisor which is lower than the root. For instance, let’s look at 42. $\sqrt{42} \approx 6.5$, so we only have to check up to (and including) 6. If we look at all possible solutions for $a \times b = 42$, we’ll see that none of them have two factors above 6:

$2 \times 21$
$3 \times 14$
$7 \times 6$
$6 \times 7$
$14 \times 3$
$21 \times 2$

Besides that, we need to only look for prime factors. We know that all primes (except for the first one, being ‘2’) are odd. Adding all of these optimizations in, we’ll get something like this:

function Get-PrimeFactors ([long]$number)
{
        while($number % 2 -eq 0)
        {
            2;
            $number /= 2;
        }
        for($i = 3; $i -le [Math]::Sqrt($number); $i+= 2)
        {
            while($number % $i -eq 0)
            {
                $number /= $i;
                $i;
            }
        }
        $number
}

This is more or less a good enough solution for this problem. Using the number from the problem description, it will run take around 50ms to get all factors of that integer.

As I’ve said, I’ve looked into a bunch of factorization algorithms, but they were all pretty much overkill for the problem at hand. As you see, the solution is found in 50ms. Even bigger numbers like 600851475143123, which is the product of two quite substantial primes, takes less than 30 seconds. I think this function might be useful for later Project Euler problems, so I’ll keep it in the library for now.

PowerShell fun: Project Euler #2 - Fibonacci summed up

Erik van Brakel — Mon, 24 Jan 2011 23:37:45 GMT

The first few Project Euler problems are actually pretty trivial to brute force, but sometimes they have a smart solution. All in all, coming up with a semi-smart title for the post is usually harder than the problem itself. But I don’t want to leave anything out, so I’ll just continue on with problem #2:

Each new term in the Fibonacci sequence is generated by adding the previous two terms. By starting with 1 and 2, the first 10 terms will be:

1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms.

The Fibonacci sequence is a quite classic integer sequence, named after Leonardo of Pisa, aka Fibonacci. Apparently they’re used in the analysis of financial markets and much more, but I have never used it in the wild so far.

Besides that, because this solves so blazingly fast (even for numbers up to 10²⁰). This is simply because the Fibonacci sequence gets quite big, quite fast. So I just solve this using a very simple bit of PowerShell code.

function Get-Fibonacci ($max)
{
    $current = $previous = 1
    $previous;
    while($current -lt $max)
    {
        $current;
        $current, $previous = ($current + $previous), $current
    }
}

$measure = Get-Fibonacci 4000000 | where {$_ % 2 -eq 0} | measure-object -sum
Write-Host $measure.Sum

One thing I did find was that a recursive solution doesn’t really work, at least not in PowerShell. The default call depth is set to 100 recursive calls at most, which you will hit eventually. Not in this case though, but when testing for higher numbers it would fail. However, I think that while a calculating a Fibonacci sequence is a nice exercise to learn about recursion, it isn’t a good example of when to use recursion in the first place.

PowerShell fun: Project Euler #1 - Gauss being smart

Erik van Brakel — Sun, 23 Jan 2011 21:50:44 GMT

I am not at all sure how far I will get with this, but I am going to try. So far I have solved the first 7 problems from Project Euler, using PowerShell. I will, whenever I solve a problem, take my time to write about my solution and what I’ve learned about math and PowerShell. It should be a good read for anyone not particularly a wizard at math, because I don’t have a math major either. Everything I write here should be in reasonably understandable terms, because I need to understand it too!

Either way, let’s start with the first problem, defined as below.

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

This is easily solvable using a simple formula, but first, a naïve script to calculate this:

(1..999)
    | where {$_ % 3 -eq 0 -or $_ % 5 -eq 0}
    | measure -sum
    | select Sum

So basically, what I do here is create an array ranging from 1 to 999, select all items which are divisible by 3 or 5, and add them together. Then, for good measure, I select the ‘Sum’ property of the measure info object.

What I learned here is basically how to create a range (similar to Enumerable.Range in C#), and how to measure a variable in PowerShell.

This is not the optimal solution. This problem is actually best solved simply by using a simple formula for calculating the sum of a range of numbers:

$$\sum_{i=1}^n i = {n^2 + n \over 2}$$

Basically, for any range of numbers with a regular interval you can find the sum with this formula. Something even more interesting is that it is possible to also find the sum for a sequence of $i^2$, $i^3$, and so on. This will be useful for one of the next problems, so I will elaborate on that when I get there.

It’s easy to prove how this formula works. First, we take the sequence and put the reverse sequence below it, just for visualization. For the example I’ve taken {1,2,3,4,5}, it’s a bit shorter:

1 2 3 4 5
5 4 3 2 1

What’s so special about this? Well, every pair of the top and bottom row is equal to exactly 6, or (n+1). How many pairs are there? Well, that’s easy. In this case, 5, but in a general case, we could say there are ‘n’ pairs. So the sum of these pairs is equal to: $(n+1)n$ which can be reduced to $n^2 + n$. However, because we’ve duplicated the sequence, our sum is now twice the amount we’re looking for. Simply divide by two (= remove the duplicate sequence) and voila: we have our formula!

However, we’re not looking for {1,2,3,..,n}. We’re looking for all terms divisible by 3 or 5, or {3,6,9,..,n} and {5,10,15,..,n}. Luckily, we can just multiply the sequence for {1,2,3,..n} by 3 and 5 to get those. We just need to find the upper bound for both, to make sure the $3n$ and $5n$ won’t be greater than 1000. In other words, less than or equal to 999. It’s not much trouble calculating this:

999/5 = 199.8. Round down to 199.
999/3 = 333. No rounding needed.

But wait! we also need to make sure we’re not adding values twice. Every time a value is divisible by 3 and 5, we’ll add it twice if we simply add the two sequences. So we’ll need to make sure that the values divisible by 15 (= 3 * 5) are removed from the sequence once. Again, let’s find the upper limit:

999 / 15 = 66.6, round down to 66.

Adding all these values gives us the following equations:

$$\sum_{i=1}^{333} 3i + \sum_{i=1}^{199} 5i - \sum_{i=1}^{66} 15i = 3{{333^2 + 333} \over 2} + 5{{199^2 + 199} \over 2} - 15{{66^2 + 66} \over 2}$$

Problem solved, with and without programming! As for the title, I’ll leave it as an exercise for you as a reader to look up the story about little Carl Friedrich Gauss, and how it relates to this problem and it’s solution.

PowerShell series: Project Euler

Erik van Brakel — Sun, 23 Jan 2011 16:04:25 GMT

In the past week I’ve been trying to solve Project Euler projects using PowerShell. So far so good, most problems are quite doable (although sometimes a bit slow). I’m going to write at least three posts about this (I have two drafts already), for which I will need some way to properly format mathematical equations.

For this, I installed MathJax (http://www.mathjax.org/), which converts formulae in LaTeX of MathML format into the proper representation. This post is more or less me trying out a few options, and showing off the pretty math.

Sum of all terms of f(x²) where x = 1..n

$$\sum_{x=0}^{n}{x^2} = {{2n^3 + 3n^2 + n}\over6}$$

The quadratic formula

$$x = {{-b} \pm \sqrt{b^2 - 4ac} \over 2a}$$

And, because I really have to have an Euler reference, Euler’s formula

$${e^{ix} =\cos x +i\sin x}$$

The only thing so far which doesn’t work too well is the Sigma with bounds. They should appear below and above the sigma sign, but they don’t. I’ll just ignore that for now, unless someone comes to me with a brilliant solution ;-)

Update

Seems like I was simply not doing it right! I enclosed the formulae in single dollar signs, while I should’ve used double dollar signs. Single dollar signs are for inline display, double dollar signs are for blocks. Inline display renders differently, which is why the bounds were not properly aligned. Fixed it now, thanks to this thread: http://groups.google.com/group/mathjax-users/browse_thread/thread/c7ac90a0d90e18fb

PowerShell fun: Detecting installed .NET versions

Erik van Brakel — Tue, 18 Jan 2011 15:54:44 GMT

Here’s a scenario: you are writing install scripts which configure your IIS instance, or a deployment script that deploys an application which depends on a certain .NET version. Ideally, you’d want to detect the absence of your required framework and have an option to install said framework. Here’s a start on how to get this done using PowerShell.

First off, we need to find out how to detect which frameworks are installed. Luckily, StackOverflow came to the rescue. THIS question, and in particular the first answer, details on how to detect which frameworks are installed, by querying the registry.

The registry is the "official" way to detect if a specific version of the Framework is installed, but which registry keys are needed change depending on the Framework version you are looking for.

Framework Version  Registry Key
------------------------------------------------------------------------------------------
1.0                HKLM\Software\Microsoft\.NETFramework\Policy\v1.0\3705 
1.1                HKLM\Software\Microsoft\NET Framework Setup\NDP\v1.1.4322\Install 
2.0                HKLM\Software\Microsoft\NET Framework Setup\NDP\v2.0.50727\Install 
3.0                HKLM\Software\Microsoft\NET Framework Setup\NDP\v3.0\Setup\InstallSuccess 
3.5                HKLM\Software\Microsoft\NET Framework Setup\NDP\v3.5\Install 
4.0 Client Profile HKLM\Software\Microsoft\NET Framework Setup\NDP\v4\Client\Install
4.0 Full Profile   HKLM\Software\Microsoft\NET Framework Setup\NDP\v4\Full\Install

PowerShell offers, by default, an option to query the registry. Simply set HKLM: or HKCU: as your location, and you’re set. So my first thought was to simply use ‘Test-Path’ to check for the existence of these keys:

$installed1 = Test-Path “HKLM:\Software\Microsoft\.NETFramework\Policy\v1.0\3705”

However, this ALWAYS returned false. Turns out, you can’t test if a registry key exists by using Test-Path, because the actual keys are properties on the parent hive. So we’ll have to check if the hive exists, and if it exists check the property. This is typically something to wrap in a function, so I did:

function Test-Key([string]$path, [string]$key)
{
    if(!(Test-Path $path)) { return $false }
    if ((Get-ItemProperty $path).$key -eq $null) { return $false }
    return $true 
}

It really does just what I stated: ‘Test-Path’ on the path to the hive. ‘Get-Property’ on the key. Now, putting this together with the list of registry keys for installed frameworks, we get something like this:

function Get-Framework-Versions()
{
    $installedFrameworks = @()
    if(Test-Key "HKLM:\Software\Microsoft\.NETFramework\Policy\v1.0" "3705") { $installedFrameworks += "1.0" }
    if(Test-Key "HKLM:\Software\Microsoft\NET Framework Setup\NDP\v1.1.4322" "Install") { $installedFrameworks += "1.1" }
    if(Test-Key "HKLM:\Software\Microsoft\NET Framework Setup\NDP\v2.0.50727" "Install") { $installedFrameworks += "2.0" }
    if(Test-Key "HKLM:\Software\Microsoft\NET Framework Setup\NDP\v3.0\Setup" "InstallSuccess") { $installedFrameworks += "3.0" }
    if(Test-Key "HKLM:\Software\Microsoft\NET Framework Setup\NDP\v3.5" "Install") { $installedFrameworks += "3.5" }
    if(Test-Key "HKLM:\Software\Microsoft\NET Framework Setup\NDP\v4\Client" "Install") { $installedFrameworks += "4.0c" }
    if(Test-Key "HKLM:\Software\Microsoft\NET Framework Setup\NDP\v4\Full" "Install") { $installedFrameworks += "4.0" }   
    
    return $installedFrameworks
}

function Test-Key([string]$path, [string]$key)
{
    if(!(Test-Path $path)) { return $false }
    if ((Get-ItemProperty $path).$key -eq $null) { return $false }
    return $true 
}

It’s really simple, literally just checking each entry in the list hardcoded. It returns an array of strings, which will tell you which frameworks are installed. When I run it on my dev PC, I get the following output:

PS C:\Users\Erik> Get-Framework-Versions
2.0
3.0
3.5
4.0c
4.0

It’s pretty easy to incorporate this into your deployments scripts. Just pipe the array to a where call, and check for your required framework!

Powershell fun: Newest tag in SVN

Erik van Brakel — Thu, 13 Jan 2011 22:35:14 GMT

When I have to use SVN for version control (I am more of a Git fanboy nowadays), I use my tags to identify releases. And if we take a look at the Subversion red bookI’m not the only one:

Another common version control concept is a tag. A tag is just a “snapshot” of a project in time. In Subversion, this idea already seems to be everywhere. Each repository revision is exactly that—a snapshot of the filesystem after each commit.

However, people often want to give more human-friendly names to tags, such as release-1.0. And they want to make snapshots of smaller subdirectories of the filesystem. After all, it's not so easy to remember that release 1.0 of a piece of software is a particular subdirectory of revision 4822.

It’s a logical way of working with tags, really. I can’t think of a better way to use tags than to use them to give your releases a friendly name. Now, how cool would it be to actually be able to automatically fetch the latest tag from your repository and deploy that somehow? In other words, once I tag something in my version control, I want THAT snapshot to actually be published in someway. As I’m toying around with powershell a little bit, I figured I’d share this little nugget with you.

For explanatory purposes, we’ll use the sourceforge repository for the ever famous NHibernate project. The URL for the repository is found on the project page:

     https://nhibernate.svn.sourceforge.net/svnroot/nhibernate
 
  If we browse this repository, and navigate to the tags directory, we’ll quite a list. Dating back all the way to before version 1.0, with no easily sortable names, we can safely say that this is a good test subject.
  First of all, you’ll need to have SVN installed so you can actually execute some command line commands. Most notably, we’ll use ‘svn list’ and ‘svn export’ to respectively get a list of the tags, and download the code (without SVN noise) to our preferred location. The full script (those few lines!):
  $REPO = "https://nhibernate.svn.sourceforge.net/svnroot/nhibernate"
$OUTPUT = "nhibernate"

$xml = [xml](svn list $REPO/tags --xml)
$entries = $xml.lists.list.entry | Sort-Object {[int]$_.commit.revision} -Descending
$latest = $entries[0].name;
Write-Host "Latest tag is $latest, exporting it now."
svn export $REPO/tags/$latest $OUTPUT


  
It’s not that complicated at all. Basically, these steps are taken:


  fetch the entries the tags directory in xml format, store that as an xml datatype.

  select all xml/lists/list/entry elements, and sort them on the revision attribute in the ‘commit’ child element. Descending, else we’d get the lowest revision (so the earliest tag) first. We have to cast $_.commit.revision to an integer, because we want a numeric sort opposed to an alphanumeric sort.

  fetch the first element in the resulting array, and store the name. This is the name of the latest tag.

  export that tag.


With the resulting files you can then do whatever you want: you can build the files into a set of binaries and upload them to an FTP somewhere, or simply zip the files for backup purposes. Either way, this is my first little snippet of powershell, and it helped me solve a problem. I hope it helps others too!



Claiming an unused twitter account
Erik van Brakel — Tue, 11 Jan 2011 21:59:35 GMT
A few days ago I decided to chase the twitter name I really want, especially with my domain name (and coincidentally my business name) in mind. I’ve been checking the http://www.twitter.com/smoothfriction page regularly, to see if there was any action. Nothing happened: no tweets whatsoever, hardly any followers, just an idle account. An account I WANTED!
  I decided to hit google, to find out my options. At first, I was quite optimistic. I found a blog post by ‘Zee’ which explains which steps I should take:
     Email username@twitter.com with the following information:
    • The username you want      
• Your existing username, if you have one       
• Whether you want to change your username, or start a new account with the username you’re requesting
 
  Apparently, this person had the name he wanted literally an hour later. Hurray! … or so I thought. Apparently, early last year Twitter changed their policy, which basically meant no claiming of unused twitter names until they had their new system set up. So I had to find this new system (I assumed they would have something like that done within a year).
  So, without further delay, THIS is what you should do:
     Log in to your/an existing twitter account
    Go to the support page (http://support.twitter.com)
    On the top right you should see your account name with a little arrow. Hover over it and select ‘Check your tickets’
    Find the ‘new ticket’ link, follow it
    Select ‘Everything else’
    From this point on, it should be obvious
 
  I filled in the form, with a detailed description of why I thought the username should be released. Within a day (maybe two) I got a response from Twitter support. They checked the account, verified it was inactive and deleted it for me. I was now able to claim this name, and from this day on, I have @smoothfriction!