Marko Lahma's Blog

Running docker-compose against WSL2

2019-08-01T11:50:00.000+03:00

Just adding a quick note for myself here. I wanted to run against docker host on WSL2 and docker compose connects to DockerDesktopVM instance by default (the full virtual machine). What I wanted to try is to run Docker Compose experience without the virtual machine utilizing the new WSL2 capabilities.

Scott Hanselman has a good write-up about this feature.

Prequisities

Windows 10 insider build from the fast ring (at the time of writing), build 18932 or newer
Enable-WindowsOptionalFeature -Online -FeatureName VirtualMachinePlatform
Install Ubuntu 18.04 from Windows Store, ensure it has the correct WSL version

wsl --set-version Ubuntu-18.04 2

To make life easier, make the new version default

wsl --set-default-version 2

Install Docker Desktop WSL 2 Technical Preview

Run docker-compose targeting the Linux WSL2 docker host

The key is to use the -H parameter, when you are hosting both the Virtual Machine and lightweight WSL2 version.


   docker-compose -H "npipe:////./pipe/docker_wsl" up -d

And now my containers managed by compose are running under the lightweight WSL2 model. The full-blown Docker virtual machine can be shutdown while using in this WSL2 mode.

If you want permanently target the WSL2 engine, you can also set environment variable DOCKER_HOST to point to this npipe protocol address.

Optimizing Jint part 7: Rewriting the interpreter

2018-12-06T22:59:00.001+02:00

So when you start to hit the wall trying to get things faster using the current infrastructure, you can always level the playing field by re-thinking the problem and finding a better way to approach the problem. A recent big change was to refactor how the JavaScript AST tree is being interpreted.

Read more »

Optimizing Jint part 6: DictionarySlim is coming to town

2018-11-15T12:00:00.000+02:00

The past weekend I read about an interesting pull request which is bringing a new type of dictionary with great interest. The idea of creating a new, simpler and enhanced version, now called DictionarySlim, original spun from the famous k-nucleotide benchmark where .NET had some trouble performing well with the built-in Dictionary. What is this new member in *Slim family?

Read more »

Optimizing Jint part 5: Optimizing Javascript primitives

2018-11-02T12:00:00.000+02:00

Javascript primitives in Jint

Jint has abstraction for each primitive Javascript type. Having these fundamental types work and interact with the engine in efficient manner is crucial for good performance.

In this blog post we are investigating some of them and problems I found while profiling.

Optimizing Jint part 4: Interfaces are good, virtual calls better, direct field access the best

2018-11-01T12:00:00.000+02:00

I'm probably going to break all the rules of OO encapsulation and some SOLID principles here. Be warned.

We all have had our share of the strategy pattern, abstract factory, you name it. Abstracting targets behind interfaces and abstract base classes. They all are fine and aim to produce better software and they usually do. But when it comes to performance, you should sometime bend the rules a bit.

Read more »

Optimizing Jint part 3: Banishing LINQ

2018-10-30T00:00:00.000+02:00

So you want your code go fast? Just remove all occurrences of

using System.Linq;

and make the code compile again without it and be done with!

In all seriousness, LINQ is a powerful tool, but it also has it's pitfalls. It really makes a difference if you use LINQ in the wrong places, one being the hot paths of a Javascript interpreter.

Next I'm going to show examples of better performing code - it's neither prettier nor more idiomatic than the LINQ counterpart. So be warned. You should again weigh your use case and whether anything here is really needed.

Jint optimization: usage of LINQ in Jint

Jint used LINQ in some places and made the code quite easy to read. Jint being an interpreter, it made things unnecessarily slow however. When scripts are run there can be millions of iterations in loops, choosing the correct way of looping and collecting data can make a big difference.

Removing LINQ is quite the easy optimization for libraries that have hot paths and intensive calls.

What's wrong with LINQ

LINQ is a powerful abstraction that allows you to create idiomatic and logical constructs, usually easy to read code. It however, does this with the price of performance. Every abstraction you add, adds performance penalty.

Here's a classic example, that brings out most of the bad things you can think of. Let's filter and transform an array using LINQ and then compare that with boring old procedural c# code that's somewhat optimized for the case and is not using anything fancy.

So the LINQ version is quite idiomatic and gets the job done in much less lines of code, what about the numbers?

I'm deliberately using .NET Framework here, the LINQ version will run a lot faster under .NET Core, but the full framework is still the case we all need to consider as a runtime environment.

Can you spot the difference? There's a closure allocation because outer context is being referenced (the forbidden variable), the ToArray has trouble determining the wanted size and grows with the flow. Then there a are those little things like .Where taking a IEnumerable which requires a wrapper creation as array does not implement IEnumerable. Fun times.

OK it only takes 2.5x the time with LINQ and the overall time is a drop in the big ocean of run times of your software. But the thing is, these things add up. When we are talking about interpreter, there's a lot of loops and they all add up. If you need to call the logic 5000 times it's either 3 seconds or 5 seconds of wall clock run time.

Context-capturing closures

When LINQ captures context, like variables outside of it's lambda functions, LINQ needs to build a instance of a class to hold to the context state. Above example showed this via the forbidden lookup. Converting to regular loop gives you better performance when a lot of data is processed.

LINQ over arrays

It might not be directly evident, but LINQ needs to create a wrapper to allow you use the LINQ methods when you access an array. Array does not directly implement IEnumerable so tricks are needed. Let's examine how foreach and LINQ-based functions to return whether array of strings contains a non-null element differs.

LINQ against an array

The LINQ version produces famous <>c classes that you might have seen in task traces. So looking at the following example you can see how the machinery to hold state is produced by the compiler.

Regular loop against an array

Compare the earlier to a version that doesn't use LINQ, see how much less code and state to handle.

Don't optimize too early (or too late)

I don't believe too much in "don't optimize early" mentality. I believe in:

Understand the use case, common usage patterns, input variance and size - and then optimize or skip optimization accordingly.

If the code is in hot path, called a lot and affects the overall system performance - just make it fast to begin with.

There are however those many places where writing LINQ makes code a lot cleaner and idiomatic. When you know performance won't be an issue, just endorse it. It's a good tool as long as you know its limits.

Things to remember and consider

Non-LINQ version is usually/always faster
LINQ can capture context and throw memory allocations through the roof
Prefer concrete (collection) types when using LINQ extensions
Array might be a bad source in hot paths as it requires a wrapper

Optimizing Jint part 2: Tools of the trade

2018-10-29T12:00:00.000+02:00

I think making performance optimizations is a great example of true engineering task, there should be numbers and evidence, always. You cannot state that something has improved unless you can show hard numbers about either reduced time to run or reduced memory usage (ideally of course both). Otherwise it's just good old feels faster.

In this post we'll examine some tools and checklists when doing performance analysis.

Read more »

Optimizing Jint part 1: Setting the stage

2018-10-26T12:00:00.000+03:00

This is a story that started a year ago, but I finally had the time and motivation to write something down. I hope this story about a performance journey might give some insights to performance optimizations and how I analyze and optimize code.

Read more »

Quartz.NET is now CLSCompliant!

2009-09-11T08:17:00.001+03:00

Quartz.NET trunk was updated with Common.Logging 2.0 which was made CLSCompliant and now it was also possible to make Quartz.NET obey the requirements of Common Language System compliance with a few tweaks to the source code.
At the same time I also dropped support for VS2003 and VS2005 solutions. Using VS2008 for now allows targeting .NET 2.0 and source code finally gets the sweet language features of C# 3…
Maybe next little issue to dabble with could be to make Quartz.NET run under medium trust…

Getting Started With Quartz.NET – Part 2

2009-09-03T14:03:00.000+03:00

In the previous post we did a short evaluation whether Quartz.NET would be a viable choice for a project. In this post we’ll go through the steps to get Quartz.NET up and running.

Getting the distribution

You will find the latest download information on Quartz.NET’s download page. Just follow the links and choose the latest binary distribution. In your case that is Quartz.NET 1.0.1. After you have have download the zip file you should extract it to some location so we can explore the distribution a bit further.

What’s inside the package

Quartz.NET is distributed ready to run and source included the package does not contain documentation at the moment as the latest API documentation and the tutorial are available on the project’s website.

The bin folder

This folder contains the actual binaries that you will need to use Quartz.NET in your application. There are separate subfolders for 1.1, 2.0 and 3.5 respectively.

The database folder

This folder contains the create scripts for Quartz.NET database. There’s a script for each supported database.

The lib folder

This folder contains the external dependencies of Quartz.NET. Quartz.NET requires Nullables library for the 1.1 build to get nullable-like features and the Common.Logging framework for all builds. The folder contents are needed when building from the source.

The server folder

This folder contains the Quartz.NET sample Windows Service. It acts as a sample how to build a windows service for running of jobs and it can be used as-is if ne customizations are required. This server will be discussed in future blog posts.

The src folder

This folder contains the source the Quartz.NET.
You can also find the the solution files,license, readme and change log information in the root of the distribution.
Now that we have had a quick tour around he distribution we should be ready to create our first little sample application in part 3 of our series.

Getting Started With Quartz.NET – Part 1

2009-08-30T14:02:00.001+03:00

This is the first blog post on the series of posts about getting familiar with Quartz.NET. In this blog posts we dig in to the requirements and common things we might want to decide before proceeding.

Evaluating whether Quartz.NET is the thing for you

First of all, it’s always a good thing to evaluate whether the tool is suitable for one’s needs. It goes without saying that like any other tool, Quartz.NET is no silver bullet (but a good golden hammer I hear). The easiest way to nail it is to quote the project’s home page:

Quartz.NET is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems

Well big words but I believe that Quartz.NET can live up to it. Quartz.NET was ported from the famous Java Quartz and so If you’re familiar with the Java one, you should have no trouble grasping the concepts of Quartz.NET. Quartz.NET is distributed with same license as Java version, Apache 2.0, which in short gives plenty of breathing room including hassle-free commercial usage.
Quartz.NET is for you if…

You need a flexible scheduling framework written in pure .NET
You need failover or clustering support for job scheduling/execution
You want the best bang for the buck, it’s free as in beer

Quartz.NET might not be for you if…

You have a “no open source” policy ;)
You are trying to handle workflows or batch processing (well you can bend it but it’s just not a natural fit)
You prefer Visual Studio wizards and shrink-wrapped products – Quartz.NET will have it’s learning curve like any other framework

If you decided yes (a good call) we can proceed!

Quartz.NET requirements

Quartz.NET 1.0 is written in C# and targets framework 1.1 and above (you probably would get it to run with framework 1.0 quite effortlessly). So the requirements in a nutshell:

.NET Framework 1.1 or above, builds supplied for 1.1, 2.0 and 3.5 separately
Database if fail-safe persistence required, supported databases include MS SQL Server, Oracle, MySQL, PostgreSQL, SQLite and Firebird

As you can see, Quartz.NET is quite self-contained and requires little from you application. If you got interested with the framework then the next post is for you as were going to get started with Quartz.NET!

Start Your Engines

2009-01-18T16:31:00.001+02:00

This is the famous first post! I'm starting to blog about mostly technology related things (.NET in particular), probably with special focus on my pet project, Quartz.NET.