The first step is to create the interface that will be used.

public interface ICache
{
    void SetString(string key, string objectToCache, TimeSpan? expiry = null);
    void Set<T>(string key, T objectToCache, TimeSpan? expiry = null) where T : class;
    string GetString(string key);
    T Get<T>(string key) where T : class;
    void Delete(string key);
    void FlushAll();
}

This interface gives primary operations that can be execute against Redis (or any other cache system). It’s possible to enhance this interface with more methods, This is the basic operations that is required to run a cache. The first two methods are to set a value inside the cache. One set a simple string, the second take a class of type T. The second one will be mostly used to take an object and serialize it. The next two methods are to get from a key the unserialized data. The next two methods is to delete. One use a key to delete a specific object and the other one delete everything from the cache.

A second interface is used. This one will allow us to get some status about if the cache is enable and if the cache is running properly.

public interface ICacheStatus
{
    bool IsCacheEnabled { get;}
    bool IsCacheRunning { get;}
}

The difference between IsCacheEnable and IsCacheRunning is that the first one is controlled by us. Normally from the web.config, you should have a key to turn on and off the cache. In case you notice a problem with the cache, it is always a good option to be able to turn off. The Second property is about getting the status of the caching server, Redis. If this one become inactive, it’s interesting to get the status from an administration panel for example.

Despite this interface, we need to have an abstract class with shared logic for any cache system (not only Redis). This is where we will have the serialization process, the error logging and the handling of the on/off mechanism. This is where the Circuit Pattern could also be used. I will discuss about it in a future article. Keep in mind for the moment that

public abstract class Cache : ICache, ICacheStatus
{
    private readonly bool isCacheEnable;

    public Cache(bool isCacheEnable)
    {
        this.isCacheEnable = isCacheEnable;
    }

    public void Set<T>(string key, T objectToCache, TimeSpan? expiry = null) where T : class
    {
        if (string.IsNullOrEmpty(key))
        {
            throw new ArgumentNullException("key");
        }
        if (this.isCacheEnable)
        {
            try
            {
                var serializedObjectToCache = JsonConvert.SerializeObject(objectToCache
                     , Formatting.Indented
                     , new JsonSerializerSettings
                     {
                         ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
                         PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                         TypeNameHandling = TypeNameHandling.All
                     });

                this.SetStringProtected(key, serializedObjectToCache, expiry);
            }
            catch (Exception e)
            {
                Log.Error(string.Format("Cannot Set {0}", key), e);
            }
        }
    }

    public T Get<T>(string key) where T : class
    {
        if (string.IsNullOrEmpty(key))
        {
            throw new ArgumentNullException("key");
        }
        if (this.isCacheEnable)
        {
            try{
                var stringObject = this.GetStringProtected(key);
                var obj = JsonConvert.DeserializeObject<T>(stringObject
                     , new JsonSerializerSettings
                     {
                         ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
                         PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                         TypeNameHandling = TypeNameHandling.All
                     });
                return obj;
            }
            catch (Exception e)
            {
                Log.Error(string.Format("Cannot Set key {0}", key), e);
            }
        }
        return null;
    }

    public void Delete(string key)
    {
        if (string.IsNullOrEmpty(key))
        {
            throw new ArgumentNullException("key");
        }
        if (this.isCacheEnable)
        {
            try{
                this.DeleteProtected(key);
            }
            catch (Exception e)
            {
                Log.Error(string.Format("Cannot Delete key {0}",key), e);
            }
        }
    }

    public void DeleteByPattern(string prefixKey)
    {
        if (string.IsNullOrEmpty(prefixKey))
        {
            throw new ArgumentNullException("prefixKey");
        }
        if (this.isCacheEnable)
        {
            try
            {
                this.DeleteByPatternProtected(prefixKey);
            }
            catch (Exception e)
            {
                Log.Error(string.Format("Cannot DeleteByPattern key {0}", prefixKey), e);
            }
        }
    }

    public void FlushAll()
    {
        if (this.isCacheEnable)
        {
            try{
                this.FlushAllProtected();
            }
            catch (Exception e)
            {
                Log.Error("Cannot Flush", e);
            }
        }
    }

    public string GetString(string key)
    {
        if (string.IsNullOrEmpty(key))
        {
            throw new ArgumentNullException("key");
        }
        if (this.isCacheEnable)
        {
            try
            {
                return this.GetStringProtected(key);
            }
            catch (Exception e)
            {
                Log.Error(string.Format("Cannot Set key {0}", key), e);
            }
        }
        return null;
    }

    public void SetString(string key, string objectToCache, TimeSpan? expiry = null)
    {
        if (string.IsNullOrEmpty(key))
        {
            throw new ArgumentNullException("key");
        }
        if (this.isCacheEnable)
        {
            try
            {
                this.SetStringProtected(key, objectToCache, expiry);
            }
            catch (Exception e)
            {
                Log.Error(string.Format("Cannot Set {0}", key), e);
            }
        }
    }
    public bool IsCacheEnabled
    {
        get { return this.isCacheEnable; }

    }
    
    protected abstract void SetStringProtected(string key, string objectToCache, TimeSpan? expiry = null);
    protected abstract string GetStringProtected(string key);
    protected abstract void DeleteProtected(string key);
    protected abstract void FlushAllProtected();
    protected abstract void DeleteByPatternProtected(string key);
    public abstract bool IsCacheRunning { get;  }
}

As you can see, this abstract class will delegate all methods into a protected abstract methods which contains the cache implementation code. This one does not know about concrete implementation, just how to handle general caching knowledge. It also abstract a single method which save a string. This mean that the implementer does not need to care about anything other than string. However, the one that will use the class has access to a Set method that allow to pass a string or an object. The next class, is the one that does the real job. Here is a simple Redis implementations of this abstract class.

public class RedisCache : Definitions.Cache
{
    private ConnectionMultiplexer redisConnections;

    private IDatabase RedisDatabase {
        get {
            if (this.redisConnections == null)
            {
                InitializeConnection();
            }
            return this.redisConnections != null ? this.redisConnections.GetDatabase() : null;
        }
    }

    public RedisCache(bool isCacheEnabled):base(isCacheEnabled)
    {
        InitializeConnection();
    }

    private void InitializeConnection()
    {
        try
        {
             this.redisConnections = ConnectionMultiplexer.Connect(System.Configuration.ConfigurationManager.AppSettings["CacheConnectionString"]);
        }
        catch (RedisConnectionException errorConnectionException)
        {
            Log.Error("Error connecting the redis cache : " + errorConnectionException.Message, errorConnectionException);
        }
    }

    protected override string GetStringProtected(string key)
    {
        if (this.RedisDatabase == null)
        {
            return null;
        }
        var redisObject = this.RedisDatabase.StringGet(key);
        if (redisObject.HasValue)
        {
            return redisObject.ToString();
        }
        else
        {
            return null;
        }
    }

    protected override void SetStringProtected(string key, string objectToCache, TimeSpan? expiry = null)
    {
        if (this.RedisDatabase == null)
        {
            return;
        }

        this.RedisDatabase.StringSet(key, objectToCache, expiry);
    }

    protected override void DeleteProtected(string key)
    {
        if (this.RedisDatabase == null)
        {
            return;
        }
        this.RedisDatabase.KeyDelete(key);
    }

    protected override void FlushAllProtected()
    {
        if (this.RedisDatabase == null)
        {
            return;
        }
        var endPoints = this.redisConnections.GetEndPoints();
        foreach (var endPoint in endPoints)
        {
            var server = this.redisConnections.GetServer(endPoint);
            server.FlushAllDatabases();
        }
    }

    public override bool IsCacheRunning
    {
        get { return this.redisConnections != null && this.redisConnections.IsConnected; }
    }
}

The Redis connection get its setting from the web.config. The instantiation of the Redis object is done by using the ConnectionMultiplexer that come from the StackExchange API. This one is thread save and this is why the Cache will be a singleton from the dependency container.

    container.RegisterType<RedisCache>(new ContainerControlledLifetimeManager()
                                                                , new InjectionConstructor(
                                                                        Convert.ToBoolean(ConfigurationManager.AppSettings["IsCacheEnabled"])
                                                                )); //Singleton ( RedisCache use thread-safe code)
    container.RegisterType<ICache, RedisCache>(); //Re-use the singleton above
    container.RegisterType<ICacheStatus, RedisCache>(); //Re-use the singleton above

This is how to register the cache with Microsoft Unity. The first one register the RedisCache class with a new object object shared by every queries to the cache, thus every requests. The two next registrations associate the two interfaces to that cache instance.

From there, it’s possible to use anywhere the interface. It’s also easy to unit test since you can mock the ICache interface which is the only interface that you need to pass through all your code. About what need to be used, it’s clear from the dependency injection code that we use ICache as the interface to use and not the concrete RedisCache class. The cache shouldn’t be used in the controller class, neither in your service class or in your repository class. This belong to the accessory classes which are between your service and repository class. Here is the a graphic of the layers that is recommended to have when using a cache system and a database.

The idea is that that the only layer to know about the cache is the accessor. The service layer does not know about the cache or the database — it only know about to get and set from the accessor. The repository does not know about caching, it’s responsibility is to get from the persistence storage the data. This can be with Entity Framework (or any other ORM) or directly with Ado.Net. On the other hand, the cache does not know about the database, it only know how to store data in a fast access way. This mean that the accessor class is the only one to get the cache injected. Here is a small example.

public class ContestAccessor: IContestAccessor
{
	private readonly IContestRepository contestRepository;
	private readonly ICache cache;
	public ContestAccessor(IContestRepository repository, ICache cache)
	{
		//...
	}
}

This class can have methods to get specific information. Here is an example to get a contest by id.

public Contest GetById(int id)
{
    var key = string.Format("contest_by_id_", id);
    var contestObject = this.cache.Get<Contest>(key);
    if (contestObject == null)
    {
        contestObject = this.contestRepository.GetById(id);
        this.cache.Set(key, contestObject);
    }
    return contestObject;
}

This is a basic example, that get the contest from the cache, if this one does not find it, than get it from the repository and store it inside the cache for the next call. Every time, we return the object whatever where it comes from. The service layer uses the injected accessor (the interface IContest for example). It does not know anything about the repository or the cache — the service just knows about getting its object by an id.

The first thing to do is to install Redis on your machine. It is created for Linux but has a simple installation for Windows. In fact, Microsoft has a open source implementation where you can download the installation from this GitHub page. I hear a lot of good feedback about it. Most say that Microsoft try not to inject any Microsoft’s flavor into it and keep it just as in implementation from the Linux ones — which is great. I also believe that this will be keep up-to-date since Microsoft offers a Redis service in part of Azure. Once you download the file and install, you will have a running service in your machine.

Once it’s running, you can do a quick test with the Redis-cli console. You can set and get a value with a simple set and get command.

It’s also possible to get all keys by using the keys * command. Once everything work as expected, you can delete everything with the flushall delete command.

Next step, is to be able to set and get from the Asp.Net C# application. This is where it can be tricky. There is multiple clients available. Do not waste your time with the servicestack one. Even if a lot of documentation is available, that library became not free since version 4. The version 3 is still available to download via Nuget but it’s more than 1 year old, the documentation does not fit quite well with the version 4 and it requires a lot of tricky hack to make everything work with all dependency packages. After wasted 3 hours I decided to use the stackexchange version one. The name is similar so do not get confuse.

Once installed, you’ll be able to access Redis with almost no effort for basic command like setting and getting. Here is a short example of possible use.

public class RedisCache : ICache
{
    private readonly ConnectionMultiplexer redisConnections;

    public RedisCache()
    {
        this.redisConnections = ConnectionMultiplexer.Connect("localhost");
    }
    public void Set<T>(string key, T objectToCache) where T : class
    {
        var db = this.redisConnections.GetDatabase();
        db.StringSet(key, JsonConvert.SerializeObject(objectToCache
                    , Formatting.Indented
                    , new JsonSerializerSettings
                    {
                        ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
                        PreserveReferencesHandling = PreserveReferencesHandling.Objects
                    }));
    }


    public T Get<T>(string key) where T :class 
    {
        var db = this.redisConnections.GetDatabase();
        var redisObject = db.StringGet(key);
        if (redisObject.HasValue)
        {
            return JsonConvert.DeserializeObject<T>(redisObject
                    , new JsonSerializerSettings
                    {
                        ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
                        PreserveReferencesHandling = PreserveReferencesHandling.Objects
                    });
        }
        else
        {
            return (T)null;
        }
    }

From here, you can inject the ICache with your IOC and use RedisCache. You can get and set any object. Voila! Of course, this class is not ready for any production code. The real ICache should have more methods like deleting and you should not hardcode “localhost”, but this should give you enough to get started with Redis and .Net.

First of all, I am using JSON.Net for serialization and deserialization. It is the common serializer library for a long time, it’s even the one Microsoft is using now for all its new projects. Second, I was already using some configurations to handle references. The problem was that one of the object had a collection of an interface which it was not deserialized correctly because JSON.Net did not know to which concrete type to instanciate.

To fix this issue and have during serialization the concrete type added to the serialization, you need to use JsonSerializerSettings with the TypeNameHandling property to All. The result look like the following code:

var serializedObjectToCache =  JsonConvert.SerializeObject(objectToCache
                       , Formatting.Indented
                       , new JsonSerializerSettings
                       {
                           ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
                           PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                           TypeNameHandling = TypeNameHandling.All
                       });

The result of the serialization is a JSON attribute named “$type” which has the complete namespace and class name. You can find more detail on JSON.Net documentation about TypeNameHandling.

The deserialization contains also the same JsonSerializerSettings.

var obj = JsonConvert.DeserializeObject<T>(serializedObject
                        , new JsonSerializerSettings
                        {
                            ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
                            PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                            TypeNameHandling = TypeNameHandling.All
                        });

With the three settings used in this article, you can serialize complex object with recursive references, with abstract type or interface type and not have to worry about serializing an Entity Framework object. So, why this is not the default configuration? Because this add a lot of information in the JSON. This shouldn’t be a huge problem if you are using it with Redis, more a problem is you transfer the information through an Ajax call.

Still tools you picked up along the way you still can use them today. I’m a backbone/grunt guy, I did not get into any other framework bandwagon and until now my tools served me well.

The way I see it that’s all about to change, with ES6, ES7 and Babel, the language is about to be transformed drastically. With frameworks like Angular2, Aurora & React taking advantages from the new language features and new front-end programming paradigm to better handle UI states, I feel that yet again we are leaving another era and the community will shift to new high.

Can you feel it? There’s something in the air and that’s not necessarily a bad thing, to the next 5 years!

Here is what you can see if your decompile Asp.Net Identity assembly. You will see a resource file with defined strings.

The problem is that you cannot just define your own resource file and setup the string you desire. You must use one of Microsoft’s package for your language. Here is the one for French.

Install-Package Microsoft.AspNet.Identity.Core.fr

The problem with that solution is that you must be lucky to have your language available and you must like the string they chose for you. Some other options available is to create your own UserValidator and PasswordValidator and by inheriting the Identity ones can still have the benefit of Identity. Overall, there is some workaround but still is time consuming for nothing. Better solution like just having to drop a resource file of your own would have been way easier and easily custom for solution. Nevertheless, it starts to get in the right direction after more than a decade trying to create a better Identity framework.

[Microsoft.SqlServer.Server.SqlFunction]
public static SqlString HashPassword2(string plainPassword)
{
   var hasher = new Microsoft.AspNet.Identity.PasswordHasher();
   return hasher.HashPassword(plainPassword);
}

The code is way simpler, and reuse the Microsoft Asp.Net code which is interesting. The problem is that SQL project does not allow to use NuGet. It’s not possible to get the reference from a NuGet package. So, we need to select the DLL from the browse window of the reference. The simple way to do it is to select the one in the Bin folder of you Asp.Net MVC project. Once done, we still need to do something to allow SQL to read it. If you do not, it will compile but not publish.

Creating [MySqlToMsSql]...
(63,1): SQL72014: .Net SqlClient Data Provider: Msg 6503, Level 16, State 12, Line 1 Assembly 'microsoft.aspnet.identity.core, version=2.0.0.0, culture=neutral, publickeytoken=31bf3856ad364e35.' was not found in the SQL catalog.
(63,0): SQL72045: Script execution error.  The executed script:
CREATE ASSEMBLY [MySqlToMsSql]
    AUTHORIZATION [dbo]
    FROM 0x4D5A9000...
An error occurred while the batch was being executed.

To fix that issue, we need to go in the property of the third-party DLL added to the project.

The property about Model Aware, Is Visible and Generate Sql Script must be set to true.

First thing to do is to create a SQL Server Project.

Second task is to add a file that will have the code. You can right click the new project and add a new item. Select user defined function.

This will create a partial class named “UserDefinedFunctions”. You can rename the file with a custom name — what is important is to have the class as partial.

Finally, you write your own method. Mine was returning a single string so I had to return a SqlString.

public partial class UserDefinedFunctions
{
    [Microsoft.SqlServer.Server.SqlFunction]
    public static SqlString HashPassword(string plainPassword)
    {
        byte[] salt;
        byte[] buffer2;
        using (var bytes = new Rfc2898DeriveBytes(plainPassword, 0x10, 0x3e8))
        {
            salt = bytes.Salt;
            buffer2 = bytes.GetBytes(0x20);
        }
        byte[] dst = new byte[0x31];
        Buffer.BlockCopy(salt, 0, dst, 1, 0x10);
        Buffer.BlockCopy(buffer2, 0, dst, 0x11, 0x20);
        return new SqlString (Convert.ToBase64String(dst));
    }
}

This is the code I used. It does the hash of a plain password and return the hashed password. This algorithm is compatible with Entity Framework so it was a way to convert legacy password into Entity Framework format. You can see that not only that class is partial to a specific class, but it also has an attribute. The Microsoft.SqlServer.Server.SqlFunction is required to specify the type of the method for SQL. It also allows to add additional information like if the function is deterministic.

The last step is to publish. You need to right click the project, select Publish. This will raise a dialog where you need to select the database server, the database name, and the script to generate. This will build (if not already done) the project. The bin folder will contains the code (dll) but also a dacpac and a SQL file.

The dacpac is what is installed by Visual Studio into Sql Server. This is a file that contains SQL objects. It is used not only by C# code but could contains any SQL object like Sql Procedure for example. The format cannot be read by NotePad because it is a compressed file. You can unzip it if you are curious. And we are curious! So here is the code above dacpac unzipped file structures.

The interesting file is the model.xml. It contains a reference to the DLL file but also the function metadata we created and the source of the code inside a CDATA.

That said, if you used Visual Studio, the function is already been installed for you. You can see it under the Scalar-valued Functions.

From here, you can just use a simple select, pass your parameter and you are all set.

If you cannot have you database working with CLR, it maybe because it is disabled. To enable, you need to do the following command once.

sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'clr enabled', 1;
GO
RECONFIGURE;
GO

The problem I ran into is the performance. This is very slow compared to any native SQL code. It is also slower than execute the same code directly inside a simple Console application that loop through all the data. For example, removing that call take 7 seconds. Even if it wasn’t converting the password and keeping this one plain, the time was reasonable. Using the CRL SQL Server User-Defined Function the time raise to 11 minutes. Doing the loop in a console by reading the data and updating the value with ADO.Net took half of the time. The time is getting high because of the code itself which is not a simple task. Even if it takes few milliseconds to execute in a console, the hashing process is still little expensive to do. Also, CRL in SQL add a layer of slowness because the SQL engine need to stub inside your SQL query Microsoft intermediate language (MSIL) a bridge to the function. This generated stub is compiled into optimized code for the SQL Server you are executing this one to.

That said, the performance was quite slow from few seconds to several minutes. I had a huge script that was converting a lot of data, gigs and gigs of data across a lot of tables from one database to another. The overhead was not something I wanted to paid during the conversion and could be executed afterward by a small console job later. I could also use the CLR function in a second script later because even if the performance was several minutes, creating the console application wasn’t saving much more. That said, during the creation of the CLR function for SQL I got curious about performance and I learned that this is far from being a subject that is black or white. Most of the time, it depends of the operation executed to know if it worth it or not to use a C# CLR function. I personally believe it should be limited. Even if it is possible to debug with Visual Studio those function remotely, I believe that business logic should remain in the application tier –t he database should be used only for persisting the information. Nevertheless, special case like using Regex on string, doing special data manipulations can be not possible with SQL only. This is a great tool to have on your belt and should be used wisely.

On the other hand, doing that mean that the user can navigate directly into your webpage by the url in the browser. If you changed the URL by just changing a hash tag that define a specific action in your website, you may have that action trigged by the click event but not when the user access the webpage directly from the url.

For example, you have several link for action like “open item1” and “open item2”. You change the url with window.location.assign because you want the user to be able to use the back button — use the history. So the history looks like this:

• http://yourwebsite.com/page1
• http://yourwebsite.com/page1#item1
• http://yourwebsite.com/page1#item2

If someone go directly to the item2 than you need to execute the code behind this hash tag. To have your code executed you need to use the window event called onhashchange.

    window.onhashchange = function() {
        //Get the hash value from the url and execute your code       
    };

The onhashchange become a hub if all time the url change which can be convenient to have a central place has a content controller. This method removes the need to have action handled in every click that has hash url to just have the click event of these links to change the url and let the onhashchange handles which action to execute.

Here is an example:

<a id="report">Current Order Report</a>
<a id="edit">Edit Current Order</a>

$('#report').click(function(){
    window.location.assign("#open-current-report");
});

$('#edit').click(function(){
    window.location.assign("#edit-order");
});

window.onhashchange = function(){
    if(window.location.hash) {
        var hashFragment = window.location.hash.substring(1); // Get hash value (without #)
        switch(hashFragment) {
            case "open-current-report":
                //Do your action
                break;
            case "edit-order":
                //Do your action
                break;
        }  
  }
};

This example use JQuery to attach the click event but this is not required for the location assign, neither for the onhashchange — they are both pure JavaScript method and event.

Here is the button’s Html. The container is not required and is just there for the purpose of having this button sticky at the bottom-right of the window. The button element is the real button. This one will grows when hovering. It contains two texts: one when collapsed and one when extended.

<div class="right-corder-container">
    <button class="right-corder-container-button">
        <span class="short-text">+</span>
        <span class="long-text">Add Me</span>
    </button>
</div>

The css does 90% of the job. The remaining 10% is that we need to have a hover JavaScript to add and remove a class to add the long text visibility. The first CSS style is about fixing the container at the bottom-right of the browser’s window.

.right-corder-container {
    position:fixed;
    right:20px;
    bottom:20px;
}

The next CSS style is what round up the button into a perfect circle. This is also where we setup the transition effect timing.

.right-corder-container .right-corder-container-button {
    height: 62px;
    width: 62px;
    border:none;
    background-color:#6FB583;
    border-radius: 62px;        /*Transform the square into rectangle, sync that value with the width/height*/
    transition: all 300ms;      /*Animation to close the button (circle)*/
    box-shadow:2px 2px 5px rgb(25, 73, 15)  ;
    cursor:pointer;
}

The span specify the plus sign to be placed in the middle. We use absolute position to have fined precision about where to place the text. Also, this is easier to ensure that even during the effect that this one remains at the same place.

.right-corder-container .right-corder-container-button span {
    font-size: 72px;
    color:white;
    position: absolute;
    left: 10px;
    top: 16px;
    line-height: 28px;
}

The hover is where we specify the size of the final stage of the button. We also change the right side to be less round. This way, the button appears to be less than an oval, more to a label.

.right-corder-container .right-corder-container-button:hover {
    transition: all 400ms cubic-bezier(.62,.1,.5,1);      
    width:200px;                                            
    border-top-right-radius: 5px;                           
    border-bottom-right-radius: 5px;                        
}

The next style block is about the long text effect. We want this text to appear slowly. We need to tweak some properties to be sure that the vertical scroll-bar does not get affected.

/*
    Long text appears slowly with an animation. That code prepare the animation by hidding the text.
    The use of display is not there because it does not work well with CSS3 animation.
*/
.right-corder-container .right-corder-container-button .long-text {
    transition: opacity 1000ms; /*Only the text fadein/fadeout is animated*/
    opacity:0;                /*By default we do not display the text, we want the text to fade in*/
    color:white;
    white-space: nowrap;      
    font-size: 0;             /*Set to 0 to not have overflow on the right of the browser*/
    width: 0;                 /*Set to 0 to not have overflow on the right of the browser*/
    margin:0;                 /*Set to 0 to not have overflow on the right of the browser*/
}

This is how we show the long text. We need to add a delay in the transition opacity because we do not want to start showing the text too early. Mostly because the contain size is animated and not big enough to receive the text yet.

/*
    Animation to have a text that appear progressively. We need to play with a delay
    on the width and the font-size to not have the browser have the text appears on the right
    side of the browser view port. This has the side-effect of having an horizontal bar.
*/
.right-corder-container .right-corder-container-button .long-text.show-long-text{
  transition: opacity 700ms,
              width 1ms linear 270ms,       /*two thirds of the animation on the container width*/
              font-size 1ms linear 270ms;   /*two thirds of the animation on the container width*/
  opacity:1;
  margin-top: 2px;                          /*Center the position vertically*/
  margin-left: 65px;                        /*Center between the + and the right end side*/
  font-size: 20px;                          /*Text size, cannot be defined initially without moving the scrollbar*/
  width: auto;                              /*Required to be set with a delay (see animation) to not have scrollbar. Delay is to wait container to size up*/
}

The JavaScript is minimal and you do not have to use JQuery for a simple task as this. However, I had this one already available to here is the code with JQuery.

$(".right-corder-container-button").hover(function() {
    $(".long-text").addClass("show-long-text");   
}, function () {
    $(".long-text").removeClass("show-long-text");
});

This code is available in JSFiddle.

Français:

Il est maintenant une tradition que la rencontre du mois d’août soit un souper dans un restaurant. Le rendez-vous est au McKibbins sur Saint-Laurent à 18h30.

Quand: 18 août à 18h30

Où: McKibbins sur Saint-Laurent – 245 Sherbrooke Ouest

Qui: Tout le monde!

Combien: L’entrée est gratuite, mais la nourriture et les consommations ne sont pas inclus.

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez Sophie

English:

It is now a tradition that the meeting of August is a dinner in a restaurant. The meeting place is the McKibbins on Saint-Laurent street at 6:30pm.

When: August 18th, 2015 at 18:30

Where: McKibbins on Saint-Laurent street – 3515 Boulevard Saint-Laurent

Who: Everybody!

How much: Free, but food and drinks are not included

Register: No need, just come, you’re welcome

Questions?: Contact Sophie

I think an example is better before continuing about how to divide code to help creating more efficient unit test. Here is a snippet that illustrate many problem. We have an Order class that has a list of item that can be bought. At some point, we want to pass a transaction so we have a method called Purchase which return the list of item that we can really purchase based on different business logic. For example, we allows only item that is between 0$ and 100$ to be bought, only item that is not discontinued, we can only buy 1 item of any kind per transaction, not more than 3 items at the same time can be in the basket and finally nothing above an order of 50$ is allowed (so we only add item to be purchase under 50$).

public class ShoppingBasket
{
    public List<Item> Items { get; set; }

    public List<Item> Purchase()
    {
        var canBeBought = new List<Item>();
        foreach (var item in Items)
        {
            bool canBuy = true;
            if (item.Money < 0 || item.Money > 100)
            {
                canBuy = false;
            }

            if (item.IsDiscontinued)
            {
                canBuy = false;
            }

            if (!this.IsUnique(item.Id))
            {
                canBuy = false;
            }

            if (this.IsShoppingBasketFull())
            {
                canBuy = false;
            }

            if (this.HasReachMaximumAmountPerOrder(canBeBought))
            {
                canBuy = false;

            }
            if (canBuy)
            {
                canBeBought.Add(item);
            }
        }
        return canBeBought;
    }

    private bool IsUnique(int id)
    {
        return this.Items.Count(d => d.Id == id) > 1;
    }

    private bool IsShoppingBasketFull()
    {
        return this.Items.Count >= 5;
    }

    private bool HasReachMaximumAmountPerOrder(List<Item> canBeBought)
    {
        return canBeBought.Sum(d=>d.Money) > 50;
    }
}

public class Item
{
    public int Id { get; set; }

    public double Money { get; set; }

    public bool IsDiscontinued { get; set; }
}

If you want to test if the Lambda expression is well formed for the IsUnique method, you will have to create a list of items and pass it to the ShoppingBasket class. So far, this is fine. However, you cannot call the method IsUnique because this one is private. You have two choices. You break the encapsulation of the ShoppingBasket by changing the visibility from private to public or you refactor your class to something more testable, more cohesive and more object oriented. Changing the encapsulation is fine. In fact, in that case it does not really matter because it does not disrupt the system. Someone could call IsUnique any time and it does not change any operational flow. However, is it really the responsability of the ShoppingBasket to handle the unique?

Let’s start trying to unit test this actual code to illustrate how it is not a good idea to test through the public method to reach a private method.

[TestMethod]
public void TestIfIsUniqueMethodWorksWhenListHasNotUniqueItem()
{
    // Arrange
    var shoppingBasket = new ShoppingBasket();
    var item1 = new Item() {Id = 1, IsDiscontinued = false, Money = 5};
    var item2 = new Item() {Id = 2, IsDiscontinued = false, Money = 10};
    var item3 = new Item() {Id = 3, IsDiscontinued = false, Money = 50};
    var item4 = new Item() {Id = 1, IsDiscontinued = false, Money = 5};
    shoppingBasket.Items = new List<Item> {item1, item2, item3, item4};

    // Act
    var itemThatWeCanPurchase = shoppingBasket.Purchase();

    // Assert
    Assert.AreEqual(2, itemThatWeCanPurchase.Count);
}

You will tell me that this is fine. In fact, it is. In that case the problem is not the amount of preparation. It is often the case, but not that time. The problem is that if tomorrow the threshold of 50$ go down to 10$ that this unit test will fail. Right now, the item id 2 and 3 are returned but if this limit goes from 50$ to 10$ than the id 2 is out which will return only one item. The unit test about unique fail because of the maximum amount per order. This is wrong.

A unit test must test a unit. It has to test a single thing. It has to be atomic. This is why we need to refactor that code.

The first thing to do is to ask who is the expert of the field for each logic. The first logic concern the logic about that we can only buy item with money between 0$ and 100$. This raise multiple questions.

1. Is it a shopping basket logic?
2. Does that logic is universal on the software we are building or just for specific case? Does this range may change during the lifetime of this software?
3. Do we have different kind of Item?

The first question make sense. Maybe depending of the type of shopping basket that logic can change. For example you could have “SmallShoppingBasket” and a “BigShoppingBasket” and depending of the basket the threshold change. This is why we cannot build a software without knowing the specification: the code changes depending of what are the rules. Creating a generic system for all case increase the time to build this one by a huge magnitude of time. That said, for that first condition, we need to extract that code out of the loop in all case. The reason is that this condition is independent of all items — it is a condition on a single item. Since we are not planning to have multiple basket and the amount belong to an item, for that use case, it makes sense to set the condition inside the Item class. This change result having one more method in Item and the unit test to call that method.

public class Item
{
    public int Id { get; set; }

    public double Money { get; set; }

    public bool IsDiscontinued { get; set; }

    public bool IsPriceLegitForPurchase()
    {
        return this.Money < 0 || this.Money > 100;
    }
}

The second condition is checking if the item is discontinued. The call is already on the item which is great. However, already have a condition on the item done one line above since our last refactoring on the price.

foreach (var item in Items)
{
    bool canBuy = true;
    if (!item.IsPriceLegitForPurchase())
    {
        canBuy = false;
    }

    if (item.IsDiscontinued)
    {
        canBuy = false;
    }
    // ...

Instead of calling twice the item we should think. These two operations has something in common: it validates the item for purchase. We should have a method in the item that does that for us. Not only it will self-document what it is going on but that logic could be reused later without having to duplicate both calls. We end up having a single validation method for purchase in item.

public bool IsValidForPurchase()
{
        if (!this.IsPriceLegitForPurchase())
        {
            return false;
        }

        if (this.IsDiscontinued)
        {
            return false;
        }
    return true;
}

The test is also clearer.

//...
if(!item.IsValidForPurchase())
{
   canBuy = false;
}
//...

The third condition was to check if the shopping basket is full. The logic belong to the basket because this one is the one who know its size, which is 5. The problem is that when we will test that logic, we do not want to have to create items that necessary pass condition for “IsValidForPurchase” method. We just want to create a list of item and test the count. That said, this is a perfect case that this method should be public.

The next condition is about if the list of item that is valid to purchase has still some place to add additional item. This is an interesting case. This logic does not belong to item, it belongs to the list. It means that this list is an entity which some properties, states, etc.

public class ItemValidToPurchase : List<Item>
{
    public bool HasReachMaximumAmountPerOrder()
    {
        return this.Sum(d => d.Money) > 50;
    }
}

The condition in the Shopping Basket is just to call the method.

if (canBeBought.HasReachMaximumAmountPerOrder())
{
    canBuy = false;
}

So the final code looks like this for the test about unique.

private ShoppingBasket GetShoppingBasket()
{
    var shoppingBasket = new ShoppingBasket();
    var item1 = new Item() {Id = 1, IsDiscontinued = false, Money = 5};
    var item2 = new Item() {Id = 2, IsDiscontinued = false, Money = 10};
    var item3 = new Item() {Id = 3, IsDiscontinued = false, Money = 20};
    var item4 = new Item() {Id = 1, IsDiscontinued = false, Money = 5};
    shoppingBasket.Items = new List<Item> {item1, item2, item3, item4};
    return shoppingBasket;
}

[TestMethod]
public void TestIfIsUniqueMethodWorksWhenListHasNotUniqueItem()
{
    // Arrange
            var shoppingBasket = this.GetShoppingBasket();

    // Act
    var isUnique = shoppingBasket.IsUnique(1);

    // Assert
    Assert.IsFalse(isUnique);
}

[TestMethod]
public void TestIfIsUniqueMethodWorksWhenListHasUniqueItem()
{
    // Arrange
    var shoppingBasket = this.GetShoppingBasket();

    // Act
    var isUnique = shoppingBasket.IsUnique(2);

    // Assert
    Assert.IsTrue(isUnique);
}

And the classes code look.

public class Item
{
    public int Id { get; set; }

    public double Money { get; set; }

    public bool IsDiscontinued { get; set; }

    public bool IsPriceLegitForPurchase()
    {
        return this.Money < 0 || this.Money > 100;
    }

    public bool IsValidForPurchase()
    {
            if (!this.IsPriceLegitForPurchase())
            {
                return false;
            }

            if (this.IsDiscontinued)
            {
                return false;
            }
        return true;
    }
}
public class ItemValidToPurchase : List<Item>
{
    public bool HasReachMaximumAmountPerOrder()
    {
        return this.Sum(d => d.Money) > 50;
    }
}
public class ShoppingBasket
{
    public List<Item> Items { get; set; }

    public ItemValidToPurchase Purchase()
    {
        var canBeBought = new ItemValidToPurchase();
        foreach (var item in Items)
        {
            bool canBuy = true;
            if(!item.IsValidForPurchase())
            {
                canBuy = false;
            }

            if (!this.IsUnique(item.Id))
            {
                canBuy = false;
            }

            if (this.IsShoppingBasketFull())
            {
                canBuy = false;
            }

            if (canBeBought.HasReachMaximumAmountPerOrder())
            {
                canBuy = false;
            }

            if (canBuy)
            {
                canBeBought.Add(item);
            }
        }
        return canBeBought;
    }

    public bool IsUnique(int id)
    {
        return this.Items.Count(d => d.Id == id) == 1;
    }

    public bool IsShoppingBasketFull()
    {
        return this.Items.Count >= 5;
    }
}

As you can see, the condition has changed on the IsUnique method. Because and error was present and only could have been detected with a valid set of unit test on that method. We could go even further by refactoring even more this code. I’ll write a second post about what we could have done with a more flexible code that allow to have dynamic validation using other design pattern.

You can find the initial code and the final code in this Git Repository.

To keep it simples, let’s start with a division that will hold all our Html elements.

<div id="container">
</div>

Next, we need to create our three main controls. The first one is the input where the user will write any information that will be saved. The second one is the toggle button that expand or collapse the input box and the third one is the save button that will be displayed only when the user save.

var $controlContainer = $('#container');

var $inputBox = $("<input>")
	.attr('id', 'inline-editor-textbox')
        .attr('type', 'text')
	.attr('value', '')
	.attr('maxlength', 20 )
	.attr('style', 'display:none')
	.attr('placeholder', "This is some watermark")
	.addClass('hide-input-inline')
	;
$inputBox.appendTo($controlContainer);

var $buttonToggle = $("<button>")
	.attr('id', 'toggleButton')
	.attr('title', 'Open')
	.attr('type', 'button');
var $spanIcon = $('<span>')
	.addClass("glyphicon glyphicon-plus");
$spanIcon.appendTo($buttonToggle);
$buttonToggle.appendTo($controlContainer);

var $buttonOk = $("<button>")
	.attr('id', 'okButton')
	.attr('style', 'display:none')
	.attr('title', 'Ok')
	.addClass('hide-ok-button')
	.attr('type', 'button');
var $spanIconOk = $('<span>')
	.addClass("glyphicon glyphicon-save");
$spanIconOk.appendTo($buttonOk);
$buttonOk.appendTo($controlContainer);

The next step is to add events. The toggle will open and close the input. It will also change the icon of the button from a “+” to a “x”. The okay button will save the information and collapse the input if everything is fine. The keyup is there to show the save button if the text box contains text.

$buttonToggle.click({ input: $inputBox, buttonToggle: $buttonToggle, buttonOk: $buttonOk},onToggle);
$buttonOk.click({ input: $inputBox, buttonToggle: $buttonToggle, buttonOk: $buttonOk},onOk);
$inputBox.keyup( { input: $inputBox, buttonOk: $buttonOk }, onKeyUp);

The toggle function has reference to some of our controls. This is a great way to pass reference without having to communicate through JQuery selector. Depending of the state of the toggle, which is the class “hide-input-inline” we make it appears or not. We add a delay because we have CSS3 animation and we want everything to be smooth together.

function onToggle(jqueryEvent) {
	var data = jqueryEvent.data;
	var $input = data.input;
	var $buttonToggle = data.buttonToggle;
	var $buttonOk = data.buttonOk;

	var isHidden = $input.hasClass("hide-input-inline");
	if (isHidden) {
		$input.fadeIn();                            
		$input.removeClass("hide-input-inline");    
		$input.addClass("show-input-inline");       
		$input.val('');                             
		$buttonToggle.addClass("add-to-close");     
		$buttonToggle.attr('title', 'Close');
	}
	else {
		$input.removeClass("show-input-inline");   
		$input.addClass("hide-input-inline")      
			.delay(600)
			.fadeOut();                             
		$buttonToggle.removeClass("add-to-close");  
		$buttonOk.addClass("hide-ok-button");     
		$buttonOk.removeClass("show-ok-button")
			.delay(600)
			.fadeOut();                             
		$buttonToggle.attr('title', 'Open');
	}
}

The key up is to check the length but we could also check for “Enter” to be pressed and invoke the Ok code. The code is adding or removing classes, with a delay to let CSS animation to be executed.

function onKeyUp(jqueryEvent) {
	var data = jqueryEvent.data;
	var $input = data.input;
	var $buttonOk = data.buttonOk;
	if ($input.val().length === 0) {
		$buttonOk.removeClass("show-ok-button");
		$buttonOk.addClass("hide-ok-button")
			.delay(600)
			.fadeOut(); 
		htmlElementWithIconState($buttonOk, true);
	} else {
		$buttonOk.fadeIn();
		$buttonOk.addClass("show-ok-button");
		$buttonOk.removeClass("hide-ok-button");
		htmlElementWithIconState($buttonOk, false);
	}
}

This helper method is used to mark the save icon into a disable state. At this moment, this is not really used but if validation is added we could still have the button add without having to remove it until the user fix the problem.

	
function htmlElementWithIconState($element, disabled) {
    $element.prop("disabled", disabled);
    $element.find('.glyphicon').css('opacity', (disabled === true) ? .4 : 1);	
}

Finally, we have the okay button hit which close up the text box and every buttons. In real life, a callback into a real save or okay code would be executed. If successful than close, otherwise keep everything open for the user to adjust his or her input.

	
function onOk(jqueryEvent) {
	var data = jqueryEvent.data;
	var $input = data.input;
	var $buttonOk = data.buttonOk;
	
	onToggle(jqueryEvent);
}

#toggleButton,
#okButton
 {
    padding:0 2px 4px 2px;         
    margin-left:3px;               
    height: 22px;                  
    width: 22px;                   
    vertical-align: top;           
}


#toggleButton span.glyphicon
{
    transition: all 1000ms ease;
}


#okButton {
    transition: all 1000ms ease;
}


.hide-input-inline {
    width:0;
    transition: width 1000ms ease-in-out;
}


.show-input-inline {
    width:210px;
    transition: width 1000ms ease-in-out;
}


.add-to-close span {
    transform: rotate(135deg);
}


.hide-ok-button {
    opacity:0;

}


.show-ok-button {
    opacity:1;
}

	
#inline-editor-textbox::-webkit-input-placeholder { /* WebKit browsers */
    color:#7f7f7f

}
#inline-editor-textbox {
    border: none;
    background-color: #E2E2E2;
    color:#7f7f7f;
    padding-left:4px;
    height:22px;/*To fix the size of the button since we do not have any border*/
}

#inline-editor-textbox:focus {
    outline: none; 
    background-color:white; 
}

Français:

Pour le meetup de Juillet, on fait une pause des présentation et on se rend directement au Benelux (18h30 comme d’habitude). Ce sera beaucoup plus amical et détendu comme soirée.

Quand: 21 Juillet à 18h30

Où: Benelux – 245 Sherbrooke Ouest

Qui: Tout le monde!

Combien: Gratuit

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez Sophie

English:

July’s meetup will take place at the Benelux (6:30 PM as usual). There won’t be any presentation to have a relax and friendly evening together.

When: July 21th, 2015 at 18:30

Where: Benelux – 245 Sherbrooke West

Who: Everybody!

How much: Free

Register: No need, just come, you’re welcome

Questions?: Contact Sophie

Email renders differently in every email client. Gmail or Outlook? iPhone or Android? Email on Acid preview your email in 42 different email clients and mobile devices in minutes.

Email on Acid also decided to sponsor Inker development, a big thank you for that!

Design is a bit of another story, it’s much harder to learn to be a good designer online & results may vary greatly.


Recently I was contacted by Designlab to try their online course. For 2 years now I have moved into product management, which changed a bit how I usually works with UX & UI designers, I thought it was the right time to deepen my knowledge into user experience design.

The Premise

For me the really interesting thing about Designplab is their mentor list. They have designers coming from Facebook, Amazon, Pinterest, Yahoo, Lyft & etc.

Needless to say if you are building a startup just having these kinds of external opinions about your app is worth the price of admission.

While in the course

The designlab courses are built in a way where each week one course unit is unlocked, I would say that I spent about 6 hours per week on the course. This is followed by a session with your mentor where you review your work. This is one of those things where the more work you put into it, the more you will get out, especially with your mentor.

On my side, I was looking to gain a better knowledge of all the “steps” a user experience designer walks through when building an application. On that side I was not disappointed, You start from an Empathy map & Design-lab walk you through every steps so that the last week you are able to a/b test your prototype.

The courses are made up of a series of lectures & projects.These projects can be directly commented on by your mentor & other students. Unfortunately I did not have much interaction with the other students, it felt weird to me to add comments to someone else’s submission, even seeing them, sometimes you see horrible things & you think oh my..

However I also saw much stronger submissions than mine, which pushed me to rework my projects.

Would I recommend?

At $300 a piece I really liked the course, obviously if you are already a designer this make no sense, but if you are a developer or a product manager looking to better understand the process these guys go through it is a very good course.

In the end what really sells it up is the 4 sessions you will get with your mentor. You can also request a mentor on their profile page, if you signup I would do that.

Design-lab is running a free course on responsive design written by Patrick Wong a product designer at Lyft. Check it out!

Français:

Ce mois-ci, nous aurons une présentation qui va intéresser les développeurs de nombreuses sphères. Amenez vos collègues et amis!

Intro to Riak par Florencia Herra-Vega

Riak est un système de base de donées NoSQL distribué très tolérant aux pannes. L’idée derrière Riak est que vous pouvez dormir tranquilement même si un noeud es littéralement en flammes, et de rendre facile le remplacement et la réparation du cluster. Nous parlerons de l’approche de Riak à la scalabilité et à la redondance de données, en expliquant un peu de théorie sur les systèmes distribués. Nous parlerons aussi des différentes configurations de consistence dans Riak, et de comment utiliser la magie mathématique des types de données “CRDT” pour éviter les conflits et bien structurer les modèles de votre application.

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Quand: 16 juin 2015 à 18:30

Où: Notman House – 51 Sherbrooke Ouest, 3ème étage

Qui: Tout le monde intéressé par Ruby et/ou Ruby on Rails

Combien: Gratuit

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez Sophie ou Martin

English:

This month we will have a presentation that will interest developers many areas. Bring your colleagues and friends!

Intro to Riak par Florencia Herra-Vega

Riak is a NoSQL datastore built to run as a highly fault-tolerant distributed cluster. The idea behind Riak is that you can literally set a node on fire and the cluster will be able to function reliably while you sleep on, and bounce back to normal when the node is replaced. This presentation will go over Riak’s approach to replication and scaling, covering a tiny bit of distributed systems theory in the process. We will also go over Riak’s consistency configurations and the mathemagically conflict-free CRDT datatypes and how you can create sensible data models for your application while making the most of these options.

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

When: June 16th, 2015 at 18:30

Where: Notman House – 51 Sherbrooke West, 3rd floor

Who: Everyone who is interested in Ruby on Rails or Ruby

How much: Free

Register: No need, just come, you’re welcome

Questions?: Contact Sophie or Martin

Here is an awesome story from England involving my Fibonacci Clock project. A few days ago I received a message from Mr. Elliot Malkin, a mathematics teacher at Churchill Academy & Sixth Form in Bristol UK . He decided to offer his 13 years old students some questions regarding the clock. They had to watch the videos, read about it on Kickstarter and then pick one of the proposed questions. He kindly sent me some of the student’s responses I wanted to share it with you.

Thanks to Mr. Malkin for sharing this with us and best of luck to all his lucky students!

[See image gallery at basbrun.com]

I present to you the Fibonacci Clock, a clock for nerds with style. Beautiful and fun at the same time, the clock uses the famous Fibonacci sequence to display time in a brand new way.

How do I tell time?

The Fibonacci sequence is a sequence of numbers created by the Italian mathematician Fibonacci in the 13th century. This is a sequence starting with 1 and 1, where each subsequent number is the sum of the previous two. For the clock I used the first 5 terms: 1, 1, 2, 3 and 5.

The screen of the clock is made up of five squares whose side lengths match the first five Fibonacci numbers: 1, 1, 2, 3 and 5. The hours are displayed using red and the minutes using green. When a square is used to display both the hours and minutes it turns blue. White squares are ignored.

To tell time on the Fibonacci clock you need to do some math. To read the hour, simply add up the corresponding values of the red and blue squares. To read the minutes, do the same with the green and blue squares. The minutes are displayed in 5 minute increments (0 to 12) so you have to multiply your result by 5 to get the actual number.

Often, there are multiple ways to display a single time. To add to the challenge, the combinations are picked randomly from all the different ways a number can be displayed. There are, for example, 16 different ways to display 6:30 and you never know which one the clock will use!

Circuit

I built the Fibonacci clock using an Atmega328P micro-controller using Arduino. You could buy an Arduino board and a DS1307 real-time clock break-out board and build a custom shield for your circuit but I preferred to build my own circuit board. That allows me to keep size small and price low.

Buttons

The three buttons attached to Arduino pins #3,#4 and #6 are used together to change time. Button on pin #3 can be used alone to change the color palette of the LEDs. An extra button is attached to pin #5 to change between different modes of the clock. Two modes are lamp modes and the default mode is the clock. All buttons are connected to the Arduino pins with a 10K pull-down resistor in parallel.

Real-Time Clock

The real-time clock chip DS1307 is connected to the Arduino analog pins 4 and 5 with two 2.2K pull-up resistors. The clock pin 5 (SDA) is connected to the Atmega328P pin 27 (Arduino A4) and the clock pin 6 (SCL) is connected to the Atmega329P pin 29 (Arduino A5). To keep time while unplugged the DS1307 chip needs a 3V battery connected to pint 3 and 4 of the chip. Finally, the real-time clock is driven by a 32KHz crystal connected on pins 1 and 2. A 5V power is applied on pin 8.

LED Strip

I am using LED pixels built on top of the WS2811 drivers. These microcontrollers allows me to set the color of each individual LED with a single output on the Arduino microcontroller. The Arduino pin used to control the LEDs in this project is pin #8 (Atmega328P pin #14).

Atmega328P

You will find all the details on how to connect the Atmega328P to make an Arduino clone on my post “Build an Arduino Clone“. I added a new feature in this project, an FTDI port to program your Arduino microcontroller directly on this circuit. You connect pin one to the reset pin of the Arduino trough a 0.1uF capacitor to synchronize your uploader with the chip bootup sequence.

Pin 2 (RX) of the FTDI port connects to pin 3 of the Atmega328P(Arduino 1-TX) and pin 3 (TX) of the FTDI connector connects to pin 2 of the Atmega328P (Arduino 0 – RX). Finally FTDI pin 4 goes to 5V and 5 and 6 to ground.

The enclosure

The following video presents all the steps for the construction of the Fibonacci clock enclosure. The idea is to create 5 square compartments in the clock, two inches deep, matching the size of the five first terms of the Fibonacci sequence, 1, 1, 2, 3 and 5. The LEDs are distributed in all squares and connected in the back of the clock to the circuit board.

The enclosure is built out of birch plywood. The frame is 1/4″ thick and the back panel is 1/8″ thick. The separators are 1/16″ thick and can be made of any opaque material. The dimensions of the clock are 8″x5″x4″. The front of the clock is a piece of 1/8″ thick semi-transparent plexiglass. The separators are marked using a Sharpie pen.

The wood finish is a water based varnish applied after a good sanding using 220 sand paper.

Pick your colors

There are 5 different color palettes built in the clock. The default RGB mode, a Mondrian mode with red, blue and yellow and some more contemporary color palettes I would not be able to name!

Fibonacci Clock with the Mondrian color palette

Make it a lamp

The Fibonacci clock can also be converted into an ambiant lamp! The code published already support two lamp modes. Simply push the mode button to toggle between the three modes. The code is open for you to hack, feel free to implement your own modes!

The source code

You can find my source code on my github account:
https://github.com/pchretien/fibo

A deeper look at email rendering engines

Prior to 2007, coding emails wasn’t actually so bad. Depending on the type of campaign you would be doing you could even use divs and floats. Yes, you heard me. You could absolutely use anything IE6 had to offer at that point in time.

Then, something catastrophic happened. Microsoft decided to replace the IE engine from Outlook with the Microsoft Word engine. As you can guess, the community went crazy. Multiple blog posts and campaigns were launched to try to convince Microsoft to take a step back. Unfortunately, the opposite happened. Microsoft confirmed their choice of engine and published a blog article “The Power of Word in Outlook” (that is still available on archive.org). The blog article can be summarized in one simple quote:

Congrats on that ugly 3rd bar chart, guys… And just like that, the email development landscape changed completely.

Outlook 2000 and 2003 use the IE version installed on your computer for rendering emails, so you could actually use Outlook 2003 with IE 11 right now. However, that might not be the best idea. There are quite a few bugs because IE 11 was not built for that.

Also, something else to know that is quite depressing- there was no amelioration from 2007 to 2013 in the rendering engine. It’s virtually the same thing with a new coat of paint. Now that’s all said and done, let’s get past Outlook and take a more general view of the email landscape right now.

.

Yahoo, Gmail, Hotmail- they all use a custom engine that mashes up your html and spits out a “secured” version of your content. They also remove any style tags. Since they load the email directly into the interface instead of into an iframe they cannot risk you breaking their interface with your custom CSS.

The mobile landscape is quite peculiar. On one side, you have the default IOS and Android mail that are using webkit. It’s generally quite amazing what you can do with them. On the other side, you have the gmail app and inbox that are still using the same crap that the desktop gmail version uses. I’m not sure why Google went that way; one thing is for sure, they don’t care much about responsive email. If you absolutely must, there is a technique to achieve responsive email in gmail, but be ready to lose countless hours to testing. Plus, it’s more of a hack than it is gmail allowing you to make responsive email.

Now you can understand why your devs go insane when your marketing director comes back to you with some small spacing issues.

Optimizing development time

There is a real issue with the time spent coding emails. Even an experienced dev can lose an atrocious amount of time testing all of the possible configurations, time that he could be spending doing things more useful to the company. And I’m not even talking about when you give this task to an intern (which happens a lot); not only will he spend days trying to figure this thing out, he has no idea where he is going and the code will probably be of very poor quality.

So today I will review for you the best tools that you can give to your team to achieve two goals, one of saving time, a lot of it, and the second of keeping a clean code base, that respects standards and is something you can build on top of when you have this new transactional email to code.

Zurb Ink CSS framework

Ink is a breath of fresh air in an environment of despair. Ink is a battle-tested responsive email framework that mainly handles your email layout. It should really be your starting point for any email project.

Ink is, however, not a silver bullet. For one thing, it has no sass and less file structure. You are stuck with one mangled CSS file. It also requires some decisions about margin and paddings that will conflict with your design. It might also increase the number of tables you are using. Don’t get me wrong; it is still the best CSS framework around for email. It’s just not as drop-in as we would like it to be.

The magic behind the Ink responsive system

Ink uses a row system with a TD wrapper, shown up here, and when the window width is smaller than 600px it puts it at display:block. This changes the column’s layout to stack on top of each other. The expander td is used to fix some bugs to make the table go to a full width in mobile layout.

As explained before, this won’t work in gmail, but it is the best future proof technique available right now.

Premailer, or how to keep your code clean

Premailer is not a new tool. Three years ago I was using it at CakeMail to inline CSS files to HTML. However, what changed in the last 3 years is that it is now much more usable. For a project that seemed dead at the time it is now running at full steam, and the support is extremely good. It will even fix stuff you don’t know about to make your email more compatible. There is a grunt plugin, so it should be easy to integrate into your current stack. You can now keep your HTML files clean and generate CSS inline versions on the fly.

If you have a guy that goes onto a website and copy/pastes his code to get an inline version, please automate that process because otherwise it wastes so much time.

Automating tools

There are a couple of Javascript tools that can help you test your email’s code faster. For one, you have litmus with a grunt plugin. With a simple command you can push a new test to litmus.

// Send a specific template to litmus
grunt litmus:dist/output/sidebar_hero/index.html

Then there is grunt nodemailer, which makes it easy to send a test email to anyone.

// Send a template to any email address
grunt email --fileSrc=dist/output/example.html

Team workflow

Time and time again I have seen email being miss-managed with a tangle of non-reusable HTML tags. Email templates beg to be used with a templating engine with an architecture component. The more you abstract the ugly, the faster your devs will be able to implement new templates based on previous work.

The reality is that when starting a new project, your typical team has no transactional templates completed. The guy decides to code them and they generate an output version of their templates. They then plug their sendgrid library on to it. They forget they even exist.

But then the company grows, more teams start sharing templates, and they find they are not always using the same languages or templating engines. They start customizing all of their own templates, and you end up with three sets of very similar email templates. Then what happens is what was bound to happen- one day, the marketing team wants to change the email header, and now you are stuck re-doing the work for three different teams and trying to coordinate the releases. There is frustration about something that should be so simple, and a lot of time is lost.

In reality, they should have all extended the same template, coded with the same templating engine. If you use a lib for sending emails, when they give data they receive the HTML code. Then, to modify the header, you just need one guy to modify the base templates and all of the apps will use the same header at the same time. That way you can also build a library of components and hide those ugly tables behind it. This will enable other devs to build new emails faster, with fewer errors.

Personally, I use Nunjucks to handle my email HTML components. It enables me to do things like this:

// Cross compatible button for email services
{% macro button(label='default', link='#', class='', align='left') %}
 <table class="button {{class}}" align="{{align}}">
  <tr>
    <td>
      <a href="{{link}}">{{label}}</a>
    </td>
  </tr>
 </table>
{% endmacro %}

// import component in template
{% from "/html-components/component.button.html" import button %}

//Usage
button('Google', 'http://www.google.com', 'button-green', 'left');

Being smarter about transactional email delivery

Chances are, right now you are currently using a library given to you by your provider to deliver transactional emails for your app. That’s a good first step. However, you need to think about what is going to happen when the provider goes down (and it will).

A delivery service should be more than just sending and receiving a response. There are three core things to look for when integrating transactional email delivery for your application:

1. Back-up

You should not be locked in with any one provider. When a provider goes down, there should be a way to re-route that request to another provider. Especially since most of them offer a couple of thousand free emails, this can be a life-saver. Your “forgot password” email should not be dependent on only one service.

Also, while some offer a “templating engine” please don’t lock in to it. Use the one your company uses and then send the template through their API, that way you do not destroy reusability.

2. Asynchronous

You should not have to wait for any delivery service to respond. If you can build a service that responds in 100 ms vs. the 200 ms from any email service provider you should. You won’t know if it is sent, anyway.

3. Logs

You should log any errors that you encounter. You also need to be alerted when your provider goes down so you can react with your back-up provider.

I’m still here, what’s next?

There is a tool that does everything I just talked about. It’s called Inker. It handles your coding stack to your delivery server. If you are wondering how you can do email better, it should probably be your first stop.

Inker Features:

Coding

• Built on top of Zurb Ink
• Sane CSS components structure with sass
• Sane HTML components structure with Nunjucks
• Localization
• Auto generate template to HTML documents with inline CSS
• Auto deployment on litmus for testing
• Auto deployment to any email address for testing

REST API Transactional Email Delivery service (nodejs)

• Asynchronous for a warp speed response (you can use it in sync mode too)
• Generate emails with custom data on the fly
• Integrate all major email delivery providers
• Back-up, when one goes down we redirect the request to another provider
• Logs! hipchat, slack, logtenries, winston, push notifications with push bullet, are all in there, but you can easily add your own too

Français:

Railsconf cette année a été fantastique! Avec l’annonce de Rails 5 et de ses nouveaux features par DHH, Sandi Metz qui sait toujours nous éblouir avec du Ruby si pur, clair et magnifique et Aaron Paterson travaillant fort pour améliorer les performances de RubyGems, Rails et Minitest, on a beaucoup à discuter. Pour ceux qui y étaient, vous pourrez partager ce qui vous a marqué le plus. Pour ceux qui n’ont pas eu cette chance, préparez vos questions!

Nous aurons aussi une excellente présentation qui saura intéresser tout le monde, des hobbyists aux gurus de Ruby:

IoT/IoE…and some ruby to rule them all by André Courchesne

Vous avez toujours rêver de contrôler le monde physique a partir de votre code Ruby?
Peut-être lire un senseur de température et l’afficher sur un graphique dans une application web ?
Contrôler l’arrosage de vos plantes en vous basant sur les données d’un senseur d’humidité ?
Mais ou commencer? Si vous étés un ingénieur logiciel et/ou avec passer votre vie a faire des site web, l’idée de brancher un senseur de température qui utilise un protocole 1-wire peut sembler complexe.
Plus maintenant ! Le un des marcher le plus en demande dans présentement sont les appareils connecter (IoT/IoE) et le mouvement DIY/Maker le rend très abordable et facile.
Dans cette courte présentation je vais vous parler des 3 plateforme les plus populaire sur le marcher: Arduino, RaspberryPi et BeagleBoneBlack.
La présentation va inclure:

– Qu’est-ce que les IoT/IoE
– Comment est-ce que je peut embarquer ?
– Quel carte/plateforme choisir
– Bien entendu un peut de Ruby pour contrôler tout ca
– Un exemple réel
– Démo d’une application amusante

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Quand: 19 mai 2015 à 18:30

Où: Notman House – 51 Sherbrooke Ouest, 3ème étage

Qui: Tout le monde intéressé par Ruby et/ou Ruby on Rails

Combien: Gratuit

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez Sophie ou Martin

English:

RailsConf this year was fantastic! With the announcement of Rails 5 by DHH and its new features, Sandi Metz who always knows to dazzle us with so pure, clear and beautiful Ruby and Aaron Paterson working hard to improve the performance of RubyGems, Rails and Minitest, much has to discuss. For those who were there, you can share what marked you most. For those who have not been so lucky, get your questions ready!

We also have an excellent presentation that will interest everyone, from hobbyists to Ruby gurus:

IoT/IoE…and some ruby to rule them all by André Courchesne

Ever wanted to control something physical from your Ruby code?
Maybe read some temperature sensor values and display them on a graph?
Control the watering of your garden based on soil humidity ?
But where to start? If you dedicated your life to software engineering and/or web design, the idea of connecting a temperature sensor to some board and reading from a sensor that uses a 1-wire protocol may sound far fetched and beyond reach.
Not anymore ! One of the hottest trends in today’s DIY/Maker market is making internet connected devices (IoT/IoE for shorts) and it’s easy as Pi !
During this short presentation I’ll introduce you to the world of Arduino, RaspberryPi and BeagleBoneBlack, the 3 most popular platform for hardware hacking and making your own IoT/IoE.
Presentation will include:
- What is IoT/IoE
- How can I get in on it ?
- Which board to choose depending on your needs
- And off course how can Ruby be used in this
- A real world example
- Live demo just for fun

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

When: May 19th, 2015 at 18:30

Where: Notman House – 51 Sherbrooke West, 3rd floor

Who: Everyone who is interested in Ruby on Rails or Ruby

How much: Free

Register: No need, just come, you’re welcome

Questions?: Contact Sophie or Martin

Fibonacci Clock

The following video is the making of the clock, from woodworking to electronics. All the code and schematics will be published under the GPL licence at the end of the campaign.

Français:

Ce mois-ci, le meetup sera animé par Gary puisque les organisateurs habituels seront tous à l’extérieur du pays. Ce meetup ayant lieu en même temps que Railsconf 2015, nous vous offrons une présentation digne de ce nom!

Abstract Feature Branch by Andy Maleh

Combien de fois vous êtes-vous retrouvés près de la ligne d’arrivée de la mise en prod d’un nouveau feature excitant, mais retirés à la dernière minute par les gestionnaires pour régler des problèmes plus urgents? Que faites-vous avec le code incomplet de votre feature si une bonne part est fonctionnelle et déjà intégrée dans Master? Faut-il l’extraire dans une branche à part? Faut-il commenter le code qui n’est pas prêt à être exposé au client? Et qui faire des améliorations de design, amenés par le nouveau feature qui sont déjà utiles à l’équipe aujourd’hui? Est-ce qu’on laisse les améliorations de design sur la glace jusqu’à ce que le feature reviennent en priorité?

Ces décisions sont généralement prises par l’équipe au cas par cas et viennent souvent avec des gros compromis sur la qualité et sur la livraison.

Consultant pour Sears en 2009 et travaillant chez Groupon.com en 2012, j’ai travaillé dans des équipe qui ont adoptés une solution efficace et peu couteuse au problème appelée ”Branch by Abstraction“, une technique popularisée par Martin Fowler, auteur de UML Distilled and Patterns of Enterprise Application Architecture.

Dans cette présentation, je vais expliquer Branch by Abstraction et démontrer comment l’appliquer à Ruby en utilisant un gem que j’ai créé appelé Abstract Feature Branch, partiellement inspiré par les librairies internes utilisées par Sears et Groupon. Ce gem a au dessus de 50000 téléchargements sur rubygems.org et a été utilisé dans une multitude de mes projets, incluant le lancement de EarlyShares.com et l’amélioration de Factor75.com.

À la fin de la présentation, l’audience aura une compréhension de base des concepts de Branch by Abstraction et suffisamment de connaissances de Abstract Feature Branch pour être en mesure d’explorer le concept dans ses propres projets.

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Quand: 21 avril 2015 à 18:30

Où: Notman House - 51 Sherbrooke Ouest, 3ème étage

Qui: Tout le monde intéressé par Ruby et/ou Ruby on Rails

Combien: Gratuit

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez  Sophie ou Martin

English:

This month, the meetup will be hosted by Gary since the usual organizers will all be out of the country. Since the meetup will occur at the same time as the Railsconf2015, we offer a presentation worthy of its name!

Abstract Feature Branch by Andy Maleh

How many times do you find yourself near the software release finish line on an exciting new feature only to be pulled off by management in the last minute to sneak in some higher priority emergency fix? What do you do with the unfinished feature’s code if parts of it were already functional and merged into master? Do you pluck it out and move to a remote feature branch? Do you simply comment out the code not ready for exposure to the customer? What about some of the nice software design improvements that were done as part of the feature and are being put to good use by the team today? Do you also put design improvements off till the feature is deemed by management high priority again?

These questions typically get decided on by the team on a case by case basis, and often with big compromises affecting both project delivery and code quality.

While consulting for Sears in 2009 and working at Groupon.com in 2012, I happened to work in teams that adopted a very effective and inexpensive solution to the problem called “Branch by Abstraction“, a technique originally popularized by Martin Fowler, author of UML Distilled and Patterns of Enterprise Application Architecture.

In this talk, I aim to explain Branch by Abstraction and demonstrate how to apply in Ruby using a gem I wrote called Abstract Feature Branch, partly inspired by the internal team libraries used at Sears and Groupon. The gem has already had over 50,000 downloads from rubygem.org and has been utilized in several of my newer projects, including launch of EarlyShares.com and enhancements of Factor75.com.

Attendees should walk out with basic understanding of Branch by Abstraction and enough knowledge of Abstract Feature Branch to be able to explore further in their own projects after the talk.

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

When: April 21th, 2015 at 18:30

Where: Notman House - 51 Sherbrooke West, 3rd floor

Who: Everyone who is interested in Ruby on Rails or Ruby

How much: Free

Register: No need, just come, you’re welcome

Questions?: Contact  Sophie or Martin

Gucumber supports most of the basic Gherkin syntax, including features, scenarios, scenario outlines, docstring and tabular data. In addition, the CLI and step definitions also support tag filters and before-after hooks.

Installing

Installing with Go should be as simple as:

$ go get -u github.com/lsegal/cmd/gucumber

Usage

If you’ve used Cucumber before, the library and CLI are fairly straightforward and similar to other language implementations. You can see an example in the features directory of the Gucumber package itself.

Given a .feature file:

# features/basic/basic.feature
Feature: Basic support
  This package should support the use of scenarios.

  Scenario: Basic usage
    Given I have an initial value of 3
    And I add 3 to the value
    Then I should have 6

And a step definition file:

# features/basic/step_definitions.go
package basic

import (
    . "github.com/lsegal/gucumber"
    "github.com/stretchr/testify/assert"
)

func init() {
    var initVal int

    Given(`^I have an initial value of (d+)$`, func(val int) {
        initVal = val
    })

    And(`^I add (d+) to the value$`, func(val int) {
        initVal += val
    })

    Then(`^I should have (d+)$`, func(val int) {
        assert.Equal(T, val, initVal)
    })
}

You can run gucumber to run these tests, which executes from the features directory by default, but can any path can be passed to the CLI:

$ gucumber
Feature: Basic support
  Scenario: Basic usage                 # features/basic/basic.feature:3
    Given I have an initial value of 3  # features/basic/basic.feature:4
    And I add 3 to the value            # features/basic/basic.feature:5
    Then I should have 6                # features/basic/basic.feature:8

Finished (3 passed, 0 failed, 0 skipped).

Writing Step Definitions

Gucumber by default pulls in all .go files in the features directory being executed. You might have noticed a few things about these files:

1. The Given, When and other step definition functions are used without the gucumber package prefix. In my examples, I choose to import gucumber into the global namespace to simplify definitions. While this is not required (you can just as easily import in a more traditional way), I would recommend it for simplicity in these files.
2. The global (also can be prefixed) T value, which seems to represent a test context and is passed into test assertions. I use testify/assert as a personal choice, but Gucumber should support most testing libraries, including Go’s own testing.T style. If the assertion library you use is not working, please open an issue.

Future Work

Gucumber is not yet complete, but it is also in a fairly usable state. Error reporting of test failures and other runner summary information can be improved, and patches would be accepted to bring this implementation up to speed with others. Check it out and let me know how it works for you.

Get It Now!

You can get Gucumber and/or more information on GitHub!

Français:

À la suite d’une présentation à Confoo, Leonard a accepté de nous donner un talk qu’il a originalement écrit pour les employés de Ballistiq. Pour limiter leurs coûts en CDN, ils ont décidé d’implémenter leur propre serveur à partir de l’interface Rack. Cette présentation sera une excellente suite à la présentation de Marc-André Cournoyer de janvier: Re-implementing Rails (ActionController & ActionDispatch).

How to build your own CDN and save tons

Lors de cette présentation, Leonard Teo de Ballistiq montrera comment ils ont écrit leurs propre CDN pour ArtStation, le plus gros site d’artistes de jeux et de films, et sauvé >90% de leur coût sur Cloudfront.

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Quand: 17 Mars 2015 à 18:30

Où: Notman House - 51 Sherbrooke Ouest, 3ème étage

Qui: Tout le monde intéressé par Ruby et/ou Ruby on Rails

Combien: Gratuit

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez  Sophie ou Martin

English:

After a presentation at Confoo, Leonard accepted to give us a talk meant for Ballistiq employees. To limit their CDN costs, they decided to build their own server from the Rack interface. This presentation is a nice follow up to Marc-André Cournoyer’s January presentation: Re-implementing Rails (ActionController & ActionDispatch).

How to build your own CDN and save tons

In this talk, Leonard Teo from Ballistiq shows you how they wrote their own CDN for ArtStation, the world’s largest site for game and movie artists, and saved > 90% on costs over using Cloudfront.

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

When: March 17th, 2015 at 18:30

Where: Notman House - 51 Sherbrooke West, 3rd floor

Who: Everyone who is interested in Ruby on Rails or Ruby

How much: Free

Register: No need, just come, you’re welcome

Questions?: Contact  Sophie or Martin

Without even probably realizing it Google Drive have become an integral part of my day to day toolset, everyone in my company use it to share documents across teams & departments.

For a tool to be useful It is important that our workflow is as painless as possible, forgetting to share a document to someone or having a coworker unable to find a document can impacts the company productivity.

This get me to last week when one of my coworker asked me where was all the documents shared by the company on the drive, I was a bit baffled. Personally I always hit the “shared with me” button to look at company documents. But that tab was gone, he had the new interface. Goodbye “shared with me” tab, please welcome the incoming tab.

Unfortunately Incoming is much less useful than it’s predecessor. Instead of being a document drive it is a timeline of your sharing activity.

Worst, to have all those documents in one place you have to “move” them. Forget about moving all your documents at once, you will have to do it one by one & subject yourself to one quirky popup behaviour.

Considering you can have years of documents in there, that’s just an awful process to get through.

Why did google changed the shared with me tab?

What was so wrong with it? One possibility is that Google found out that in general people will look for the last item shared by co-workers & then the longer it has been shared, the more it become irrelevant.

That seems to be the behaviour that drive the new ui. You can also now choose items you want to move into your personal drive as opposite of having a bunch of things you are not using anymore.

However this is no excuse, Google should have a process to either import everything, or at least push a walkthrough the first time you open the new Incoming ui that let you import what you want, also explaining how this is gonna work for now on.

Any front-end dev will tell you that email template integration is one of the worst part of their job. It is often a task given to interns or to one of those cheap third party companies, & for good reasons.

It takes experience/time to build solid email templates, testing in different browsers/email clients is also hard & costly. If you use a senior dev it costs even more money, oh and you will also probably piss him off.

I understand this too much, not only did I work for web agencies, I also worked a couple of years at CakeMail building 2 wysiwyg email editors. If you think building an email template is hard, try building a good wysiwyg email creation tool.

A beacon of hope, ink

When Zurb launched Ink I was ecstatic, a solid grid system for building emails, could it be possible? Ink is the starting point for a new generation of sane email templates, unfortunately it only solve 1 issue, having a solid css framework for building email.

You are still stuck with a shitty workflow to make all the magic happen.

Enter Inker

Inker solves the issues ink does not. For one it incorporates ink slapped with a sass structure instead of plain css for better encapsulation. Then it solves the template html management problem with Mozilla Nunjucks, it give you a sane html component architecture that can be rendered into one html file. Then Inker inline your css for you & can send it to litmus or an email address for testing.

What does it mean in the end? Inker is the glue that makes everything easier while developing emails. In one command you can generate an email template from a sane CSS & HTML markup that can be deployed on litmus. Saving you time, money & frustration.

An API to rule them all

Inker also comes with a basic email sending server. This server is a rest api that can handle two things, for one it can return the template with parsed custom variables. It will also, guess what, send emails to any smtp service you want to (ex: Mailgun, sendgrid, etc).

In general, a company having multiple applications will handle emails separately for all its apps. This means multiple ways of building the templates, multiple templating engines, multiple ways of sending the emails.

Inker optimizes all this, you have one clearly define way of building templates, that has been tested & is not error prone. You have one simple email server that handle all sending requests, it can handle multiple providers if one should go down, or if you get a better deal somewhere else.

Inker API is easy to implement in any application you build, removing the SMTP nonsense from your apps & helping you focus on more important things.

The bullet point

• Built on top of Zurb Ink
• Sane CSS components structure with SASS
• Sane HTML components structure with nunjucks
• Auto generate template to HTML documents with inlined CSS
• Auto deployment on litmus for testing
• Auto deployment to email for testing
• Basic internal email sending server

Download | Documentation

Special thanks

Thanks to Litmus for the free email client testing for this project.

Quite a bit has changed since then:

• Our network is larger
• The backend code has been completely refactored (moving from thousands of requests per day to millions will do that to you)
• We’ve managed to acquire a bunch of great customers.

Two other major adjustments have changed how I look at Where’s it Up:

1. We have customers running more tests, faster, than I ever imagined. Where’s it Up users now create enough network traffic to represent a real number of dollars.
2. We’ve expanded the number and type of tests we’re running, such that some are orders of magnitude more bandwidth- and CPU-intensive than anything we considered at launch. Compare: generating a screenshot of a web page, compressing it, and uploading it to AWS S3 (our shot test, powering ShotSherpa), to a single DNS lookup.

I tried to make the old pricing model work. We tiered some of our job types, limiting them to customers on even more expensive plans. Then we capped our plans at a total number of tests per month. Ultimately, the flat-rate model had two problems: customers couldn’t use some of the test types unless they were giving us $200/month, and running a million screenshots from our expensive Alaska server would cost us more than $200.

New Model

Our new pricing model is to sell credits, and charge a different number of credits for different test types. Running a quick DNS costs one credit, whereas a screenshot costs ten. This model allows us to offer every test type to every user, and to bill users more accurately for their usage.

Rather than giving every user one free month when they sign up, we’ll give them 10,000 free credits (at least until my accountant finds out). This is enough to:

• Confirm that one HTTP endpoint is accessible 10,000 times; or
• Confirm that one HTTP endpoint is accessible on every continent, every hour, for 35 days; or
• Check DNS results for your domain from 70 countries, daily, for the next 142 days; or
• Take a screenshot of your website every morning for the next 2.74 years

Conclusion

It’s too soon to call this a success, but I’m more happy with the credits model than I have been with anything else we tried or considered. It allows every customer to execute every job type, while ensuring we don’t sell more than we can support.

Doing email marketing is special, you have a lot of tools available but at the same time you are dealing with a very outdated technology. Here is my top 6 things an email marketing manager should know.

Generic marketing image

Supporting 20 email clients is not going to be easy

You are having a hard time getting your website glitches fixed on internet explorer 8?

Now imagine that on top of web browsers like chrome, firefox & IE, you need to support each of those on Gmail, hotmail, yahoo & then you need to test on outlook 2003, 2007, 2010, 2013, apple mail. Having responsive design? alright add IOS & android to the mix.

Yup that makes for a lot of testing & that’s no easy testing unless you have something like litmus, & even then. Litmus provides you with a screenshot of the different email clients but you have no way to inspect that code, that makes for a lot of back & forth, a lot of waiting in between tests & a lot of frustration.

Coding email templates is an art (& design too)

Thinking of giving your superb new design to your web developer intern, be careful. Coding emails is not like coding a website, you need to employ a lot of techniques (using table over divs being one of the most important) to make your design work.

Just to give you an idea, here are some important guidelines:

• Use tables over divs like in 1996.
• Background images do not work on outlook 2007,2010,2013 (unless you do some VML voodoo)
• You must inline all your css directly on the html tags
• Don’t ask about flash (no flash)
• Don’t ask about javascript (no javascript)
• It’s possible to use simple responsive design
• You can use gifs, you can kinda use video.
• CSS support is a mess, if you are going to ask a dev if you can do this & that why don’t you refer yourself to campaign monitor css chart before (because that’s what your dev is going to look at anyway)

More information:
The Foundations of Coding HTML Email
Email Design Guideline

If your devs do not deliver quality, use a third party

Integrating emails sucks, I get it. You need to test on 20 email clients, that’s a given you are going to have some small discrepancies between platforms & it is going to take an enormous amount of stackoverflow time to fix them all. If you see your developers can’t deliver good quality go AWOL.

There is a lot of third party companies that will do it for you & they know every tidbits about email templating. I will name 2 companies here that I know deliver good quality from my time at CakeMail, Email Monk & Mail Bakery. Look around, compare prices, there is a ton of them.

Shop your email marketing platform

Campaign monitor, Mailchimp, CakeMail, MadMini, GetResponse, you get it, there is also a ton of them. Don’t read reviews, test them. Most have a free tier & pricing are generally similar, if you have a big email list you can deal your price too, don’t be afraid to try.

Excited about a/b split testing? why don’t you start by testing that on your subject, sender name & sent time.

Thinking about optimizing your campaign? Don’t start with changing your call to action button from blue to red, it’s not only one of the most costly change, it is probably one of the least profitable. If your clients do not open your campaign there is nothing to test.

Your subject line directly affects how many people will see your campaign, try changing the keywords to get people attention.

Test:

• Short & long subject lines,
• Adding numbers - 6 reasons why you will love our chocolate & 10% off
• Personalize with name or city – Mike how did your like your experience shopping at Walmart Edmonton?

Your sender name can also have a huge impact on your open rate. First avoid the noreply@ approach, this is a big teller that your email is just marketing content. It is also generally agreed that a combination of personal & corporate name is the way the go, exemple: Cedric Dugas, CakeMail.

Your sending time will also affect your open rate. Your campaign is received at the same time that those 10 other marketing campaign? Big chances it’s not going to be opened. Also please choose a platform where your email can be sent separately by timezone over a 24 hours period. Meaning if you schedule it to be launched at 4am, it must be 4am everywhere in the world.

* Side note, Mailchimp offer a feature that optimize your sending time by itself & they are pretty good at it.

Do your homework. You’re not the first doing a/b split test on a email marketing campaign.

There is a ton of articles & papers on email marketing optimizations, put your house in order before going to war.

• Email Marketing Best Practices In Action
• 25 Little Email Marketing Experiments That Deliver BIG Results
• The “From” Name: Perhaps Your Most Important Email Marketing Decision

Français:

Ce mois-ci, nous avons deux présentations qui (comme toujours) s’annoncent captivantes! Nous sommes de retour à la Maison Notman.

Présentation de Ember.js

Par : Jean-François Couture

Fatigué du spagetti JQuery pour ajouter de nouvelles fonctionnalités côté client? Ajoutez cette présentation à votre calendrier!

 

Outside-In agile testing

Par : Michael Elfassy

Michael nous fera la démonstration d’une méthode pour tester nos applications Rails qui a pour but d’être la plus intuitive, efficace et fiable possible. Tout en étant amusante pour le développeur. Challenge accepted!

 

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Quand: 17 Février 2015 à 18:30

Où: Notman House - 51 Sherbrooke Ouest, 3ème étage

Qui: Tout le monde intéressé par Ruby et/ou Ruby on Rails

Combien: Gratuit

Inscription: Pas besoin, tout le monde est le bienvenu

Questions?: Contactez moi

English:

This month we have two presentations and they will be captivating (as always)! Montreal.rb will take place in the Notman House.

Ember.js Introduction

With : Jean-François Couture

If you are tired of jquery spaghetti when adding functionality browser side, mark your calendar!

 

Outside-In agile testing

With : Michael Elfassy

Michael will demonstrate a method for testing our Rails applications that aims to be as intuitive, efficient and reliable as possible. As well as being fun for the developer. Challenge accepted!

 

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

When: February 17th, 2015 at 18:30

Where: Notman House - 51 Sherbrooke West, 3rd floor

Who: Everyone who is interested in Ruby on Rails or Ruby

How much: Free

Register: No need, just come, you’re welcome

Questions?: Contact me

1) Marc-André Cournoyer – Re-implementing Rails (ActionController & ActionDispatch).

2) Gem Club – Présentez un Gem que vous aimez aux autres participants

Ce mois-ci, RPM Montréal va nous recevoir. Ils sont situés au 420 Guy.

 

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

We are at the first Montreal.rb of 2015. This time, the meet-up will be at RPM Montreal.

1) Marc-André Cournoyer – Re-implementing Rails (ActionController & ActionDispatch).

2) Gem Club – Present a Gem you like to the other participants

This month, RPM Montreal will host Montreal.rb. There address is: 420 Guy.

 

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

1) Simon Génier – MRuby

2) Étienne Barrié – Kwargs

3) Christian Blais – 15 slides in 15 minutes

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Here we are, last ruby meet-up of the year! This time again, Montreal.rb will take place in the Notman House Café.

1) Simon Génier – MRuby

2) Étienne Barrié – Kwargs

3) Christian Blais – 15 slides in 15 minutes

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

I tried pinging the server’s hostname to ensure it was successful. No problem. I tried connecting to the server using the mongo command line on the primary server (`mongo server.example.com:27017`). No problem.

Eventually I copied the mongo hostname from the replica set configuration, and tried to use it to connect to mongo on the new secondary. No dice! As it turns out the NAT hadn’t been configured to allow that hostname to work locally. A few new entries in our /etc/hosts file later, and our server was joined successfully.

MongoDB Tip:

When MongoDB has trouble connecting, try the connection strings listed in the replica set config from all relevant hosts.

From simple header to whole website scrolling effects, the parallax must be one of the longest trend to happen in web design. In fact I never had one product manager in 5 years that did not ask at some point or another to add a parallax effect to that text, or content block, or header. Why? My guess is that they want to spice up their app or website but they don’t really know how, so adding parallax is something simple to think about that “add” to a page. Hey everyone has one, me too!

Linkedin, Medium, I’m surprised how big companies still use it a lot. Me? I hate parallax, it slows down my computer & every time I see one I only see a product manager saying..

But the real interesting question is, does parallax hinder user experience?

Unfortunately not enough studies have been done on this subject, so it’s really hard to have a real understanding on the effects parallax have on user experience. The only study I found was done by a student of the Purdue University. He found out that there was no significant differences between the 2 groups he tested in terms of website usability, except that the group having the parallax found it fun…..

It’s quite an interesting read & there is a very good summary of the study (if you do not want to pay) here.

So anyway

I guess I’m stuck with your stupid parallax for still some time until it fades away, I look forward for the time where we will be all laughing about the parallax trend. Until then, yes I will use that plugin to do that parallax thing you want, can we move on to something more important?

// Rant mode off

1) Guillaume Malette – How to handle uniqueness constraints in Rails

2) David Lazar – Fork you gem, I am gonna fix yer wagon

Comme d’habitude, vous êtes tous conviés au Bénélux ensuite pour continuer les discussions autour d’une bonne bière!

Shopify commandite la bière et la pizza comme les derniers mois. Ils cherchent toujours des développeurs pour leurs bureaux à Montréal. N’hésitez pas à les contacter.

Here it is, the first snow of the year. Even with the snow, Montreal.rb will take place in the Notman House Café.

1) Guillaume Malette – How to handle uniqueness constraints in Rails

2) David Lazar – Fork you gem, I am gonna fix yer wagon

As usual, you’re all invited for a beer at Benelux!

Shopify is sponsoring beers and pizzas like in the past months. They are still searching developers for their Montreal office. Don’t hesitate to contact them.

At SSENSE we recently had to build a CMS for our content team, this project was interesting because our content team creates custom layouts for each article & this is not something you can easily do using a platform like Wordpress for example. Unfortunately like too much internal project, time was short. We had to build something fast, iterate fast & deliver it as soon as possible.

A perfect project to be done using scrum & include the design in the process!

About designing one sprint ahead

Couple of months ago I spoke about how hard it was for us to find a balance between having the design done directly in the sprint, with demanding stakeholders we had no choice but to move the design one sprint ahead.

However with the CMS we were reign free, I built very strong user stories, very driven on the acceptance criteria & we started our sprint without design.

Our workflow

Front-end does basic wireframe integration based on our grid system
↓
Designer applies styling with css/rework base integration
↓
Integrator adds functionalities
↓
Designer applies styling/rework integration
↓
The team iterate together if any user experience issues popup or we find better solutions.

Obviously that means our designer had to use more his CSS/HTML chops than its Photoshop skills, if your designer is completely unable to use CSS that’s another thing. However we were very fortunate to have a designer than can also code very well & with a good CSS framework, all is possible.

Is it really better?

For us this workflow allowed to find UX issues sooner & iterate faster than our usual workflow, in the end it made us more agile you could say : P.

I really liked how this worked out with the team & everyone seemed happier with the workflow as wells. Now how can we replicate this workflow on all our projects?

Looking at other teams in the wild that do design ahead they are generally not happy with it, it’s not a lack of trying to integrate it, but it’s hard to find solutions to that problem since design involve generally more than the scrum team & stakeholders have generally some control over design & processes.

More ressources about scrum & design processes:
The Sprint Ahead — Sprint Behind Tango
Designing Ahead: The Good, The Bad, and The Ugly
UX working a Sprint ahead is full of FAIL. Work as a single team instead.

I just finished building a CNC kit from Shapeoko and started cutting stuff. I needed a simple project that would be a bit more challenging than engraving my name on a piece of scrap wood. I decided to give a try to the Wobbly Circle presented by Matt Parker on the Numberphile Youtube channel.

In this video, Matt Parker explains the mathematics required to assemble two disks perpendicular one to the other in such a way that their combined center of gravity always remains at the same height, making the assembly behave like a ball when rolling on a plane surface.

I wanted to use Free and Open Source software as much as possible for this project. I designed the profile in DraftSight, a free CAD program from Dassault Système. I then exported the drawing in DXF and converted it to SVG using Inkscape. This conversion is not optimal because of the way Inkscape handles line thicknesses. Finally I opened the SVG in Makercam.com, an online CAM software to generated the G-Code for the milling machine.

Inkscape is great to produce art pieces and engraving but not really usable as a CAD program. I am still searching for the best free and Open Source solution out there, if you have any suggestion please let me know in the comment section.

All the files for this simple project are available on my Github account: https://github.com/pchretien/centerofgravity

Having this flag allows us to handle a few things gracefully:

• Large value contracts
  By marking them as managed, they don’t expire because someone was on vacation when the account expired. They stay happy, the revenue continues, the expiry date remains accurate.
• Contracts with tricky billing processes
  The majority of our contracts pay us with PayPal or Stripe. A selection of contracts however have complex hoops involving anti-bribery policies, supplier agreements etc. This gives us some time to get this ironed out.
• Contracts where we’ve failed to bill well
  We occasionally make mistakes when billing our clients. When we’ve screwed up in the past, this helps ensure that there’s time for everything to resolve amicably.

This doesn’t mean the system has been without flaw. We currently get daily emails reporting on new signups, expired accounts, etc. It mentions all accounts that were not expired because of the managed flag.

Like most features, that was added after mistakes were made: we’d left some managed accounts unpaid for months. With better reporting, though, now, we couldn’t be happier with it.

Shift Registers & Prototyping

The 7 segment display I am using doesn’t provide any logic and must be driven directly by the micro-controller. Since it would require more output pins than what is available on the Trinket, we will use two shift registers.

I am using the 74HC595 shift register to drive each segment and the TPIC6B595 to sink whole digits. Because sinking a digit potentially requires 7 times more current than driving a segment, we use the TPIC6B595 high power shift register. The popular 74HC595 will be more than adequate to drive each segment.

To control one or more shift registers you need three outputs on your micro-controller, one for the clock, one for the data and one for the latch. The latch pin is used to clear the register memory and write to the outputs when the data has been written using the clock and data pins. To send a bit of data you sink the clock, set the data pin to the value you want to write and pull the clock back up.

I did not used the ShiftOut library but used the schematics presented on their page to wire my circuit. You can find this very well written tutorial at http://arduino.cc/en/tutorial/ShiftOut.

I soldered all the parts for this project on an Adafruit breadboard prototyping board.  All the Trinket code for this project can be found on my Github account at https://github.com/pchretien/scorecounter. The code comments contains info on which Trinket pin to use for each part.

[See image gallery at basbrun.com]

Woodworking

To build the enclosure any box would do. For this project I decided to make a nice looking wooden enclosure using pine and mahogany. The technique used here is almost the same as for building bandsaw boxes.

First step, glue a stack of wood together and flatten the face that will become the bottom of the box. Once the blank has cured, cut the excess wood and make all faces flat and square.

It is now time to cut the front and back of the box on the band-saw. Cut a 1 inch piece for the front so you have enough room to fit the seven segment display. For the back, remove about 1/4 inch of material. This will be the door to access the internal circuit.

It’s now time to cut the interior of the box on the band-saw. Make the entry cut on the bottom of the box so it will not show on the final project. I suggest using a maximum of 6 tooth per inch (tpi) blade to make the cut since a lot of wood needs to be cleared when cutting a piece this tick.

Now, do the same on the front panel to fit the seven segment display. Use the display to mark the dimensions of the hole and start cutting at the bottom to hide the entry cut. Make the hole a bit smaller and finish it with a file to get a snug fit.

The last cuts you have to make are the two holes for the buttons and the hole in the back for the USB cable. You may want to install all the electronic inside the box before to drill the hole for the USB cable to make sure it aligns with the Trinket connector.

Cutting the interior of the body and the front panel with the band-saw leaved two entry cuts you now have to close. Simply apply some wood glue in the cut and close it using clamps. Let dry for about an hour.

Finally sand and apply some finish on the box and screw the back in place using #4 wood screws.

[See image gallery at basbrun.com]

Final Product

You are done.

[See image gallery at basbrun.com]

MongoDB Changes

I blogged about our switch to MongoDB for Where’s it Up a while back, and we’ve been pretty happy with it. When designing our schema, I embraced the “schema free world” and stored all of the results inside a single document. This is very effective when the whole document is created at once, but it can be problematic when the document will require frequent updates. Our document process was something like this:

• User asks Where’s it up something like: Test google.com using HTTP, Trace, DNS, from Toronto, Cairo, London, Berlin.
• A bare bones Mongo document is created identifying the user, the URI, tests, and cities.
• One gearman job is submitted for each City-Test pair against the given URI. In this instance, that’s 12 separate jobs.
• Gearman workers pick up the work, perform the work, and update the document created earlier using $set

This is very efficient for reads, but that doesn’t match our normal usage: users submit work, and poll for responses until the work is complete. Once all the data is available, they stop. We’ve optimized for reads in a write heavy system.

The situation for writes is far from optimal. When Mongo creates a document, under exact fit allocation, it considers the size of the data being provided, and applies a padding factor. The padding factor maxes out at 1.99x. Because our original document is very small, it’s essentially guaranteed to grow by more than 2x, probably with the first Traceroute result. As our workers finish, each attempting to add additional data into the document, it will need to be moved, and moved again. MongoDB stores records contiguously on disk, so every time it needs to grow the document it must read it off disk, and drop it somewhere elsewhere. Clearly a waste of I/O operations.

It’s likely that Power of 2 sized allocations would better suit our storage needs, but they wouldn’t erase the need for moving entirely, just reduce the number of times it’s done.

Normalize, slightly. Our new structure has us storing the document framework in the results collection, and each individual work result from the gearman worker is stored in results_details in its own document. This is much worse for us on read: we need to pull in the parent document, then the child documents. On write, we’re saved the horrible moves we were facing previously. Again, the usage we’re actually seeing is: Submit some work, poll until done, read complete dataset, never read again. So this is much better overall.

We had some slight work to manage to make our front end handle both versions, but this change has been very smooth. We continue to enjoy how MongoDB handles replica sets, and automatic failover.

Worker Changes

We’ve recently acquired a new client which has drastically increased the number of tasks we need to complete throughout the day: checking a series of URLs using several tests, from every location we have. This forced us to re-examine how we’re managing our gearman workers to improve performance. Our old system:

• Supervisor runs a series of workers, no more than ~204. (Supervisor’s use of python’s select() limits it to 1024 file descriptors, which is allows for ~204 workers)
• Each worker, written in PHP, connects to gearman stating it’s capable of completing all of our work types
• When work arrives, the worker runs a shell command that executes the task on a remote server using a persistent ssh tunnel. Waits for the results, then shoves them into MongoDB.

This gave us a few problems:

• Fewer workers than we’d like, no ability to expand
• High memory overhead for each worker
• The PHP process spends ~99.9% of its time waiting, either for a new job to come in, or for the shell command it executed to complete.
• High load with 1 PHP process per executing job (that actually does work)

We examined a series of options to replace this, writing a job manager as a threaded Java application was seriously considered. It was eventually shot down due to the complexities of maintaining another set of packages, and due to the reduced number of employees who could help maintain it. Brian L Moon’s Gearman Manager was another option, but it left us running a lot of PHP we weren’t using. We could strip down PHP to make it smaller, but it wouldn’t solve all our problems.

Minimizing the size of PHP is pretty easy. Your average PHP install probably includes many extensions you’re not using on every request, doubly so if you’re looking at what’s required by your gearman workers. Look at ./configure --disable-all as a start.

Our solution involves some lesser-used functions in PHP: proc_open, stream_select, and occasionally posix_kill.

We set Gearman to work in a non-blocking fashion. This allows us to poll and check for available work without blocking until it becomes available.

Our main loop is pretty basic:

1. Check to see if new work is available, and if so start it up.
2. Poll currently executing work to see if there’s data available, if so record it.
3. Check for defunct processes that have been executing for too long, kill those.
4. Kill the worker if it's been running too long.

Check for available work
Available work is fired off using proc_open() (our work tends to look something like this: sudo -u lilypad /etc/wheresitup/shell/whereisitup-trace.sh seattle wonderproxy.com). We save the read pipe for later access, and set it to non-blocking. We only allow each non-blocking worker to execute 40 jobs concurrently, though we plan to increase the limit after more testing.

Poll currently executing work for new data
Check for available data using stream_select() using a very low poll time. Some of our tests, like dig or host, tend to return the data in one large chunk; others, like traceroute, return data slowly over time. Longer running processes (traceroute seems to average 30 seconds) will have their data accumulated over time and appended together. If the read pipe is done (checked with feof()) close them off, and close the process (proc_close)

Check for defunct processes
We track the last successful read from each process, if it was more than 30 seconds ago we close all the pipes then recursively kill the process tree.

Kill the worker
Finally, the system keeps track of how much work it’s done. Once it hits a preset limit, it stops executing new jobs. Once its current work is done, it dies. Supervisor automatically starts a new worker.

Site Changes

Our original site wasn’t actually hugely problematic. We had used a large framework, which often left me feeling frustrated. I’ve lost days fighting with it, on everything from better managing 404s to handling other authentication methods. It was the former that lead to me rage-quitting the framework and installing Bullet. That said, we’re not afraid to pull in a library to solve specific problems, we’ve been happily using Zend_Mail for a long time.

We’ve re-written our site to use the Bullet framework. Apart from appreciating micro frameworks in generally, they’re particularly apt for delivering APIs. We’ve had great success writing tests under Bullet, and being able to navigate code in minutes rather than hours is an everlasting bonus. This leaves the code base heavily weighted to business logic, rather than a monolithic framework.

Here is Palmerston, a nice street with pretty street lights (much prettier than ours):

Here is Markham, with the sports field effect:

I took those two pictures on the same camera, with the same settings. I exported them both using the same white balance value. Full resolution options are posted at smugmug. I have emailed Toronto 311, who has forwarded me on to Toronto Hydro. In the meantime I'm going to need upgraded blinds, and to start wearing my sunglasses at night.

During the rebuild, the raid controller was doing its best to monopolize all the I/O operations. This left the various systems hosted on that server in a very constrained I/O state, iowait crested over 50% for many of them, while load breached 260 on a four core vm. Fun times.

To help reduce the strain we shut down all un-needed virtual machines, and demoted the local Mongo instance to secondary. Our goal here was to reduce the write load on the constrained machine. This broke the experience for our users on wheresitup.com.

We’ve got the PHP driver configured with a read preference of MongoClient::RP_NEAREST. This normally isn’t a problem, we’re okay with some slightly stale results, they’ll be updated in a moment. Problems can occur if the nearest member of the replica set doesn’t have a record at all when the user asks. This doesn’t occur during normal operations as there’s a delay between the user making the request, and being redirected to the results page that would require them.

Last night, with the local Mongo instance so backed up with IO operations, it was taking seconds not ms for the record to show up.

We shut that member of the replica set off completely, and everything was awesome. Well, apart from the 260 load.

That makes for some headache, that is why product managers will often opt to put design one sprint ahead. It’s just easier to manage, but at what cost?

On one side you get people that say you should never split design from development, all in one sprint.

1. Your user stories take two sprints for being done instead of one, big iterations are slower.
2. Design & development get split a bit, less collaboration on design creation, may miss some important answers.
3. Developers takes less accountability of the design
4. Designers get caught up between two guns, must deliver next sprint user stories but also help developers do mini iterations when the design does not work as expected.

Then on the other side, keeping both separate has also great advantages:

1. Most teams already worked somewhat like that, ease workflow between design & development
2. Easier to manager front-end devs workflow, always work to do
2. Make sure devs understand the user stories well, better estimation
3. Easier to manage designs flow, can get inputs from users & stackholder before going to development stage

For me..

I’m frankly not sure one option is always better than the other one. I guess it depends a lot on the team, the company, the culture you are in.

For now I opt to do design one sprint ahead, on some projects it makes sense to do it in the same sprint, but in a lot of projects, user stories can have more development than design, so figuring out how to make all that work in every sprint is a big headache.

I still integrate the designers in the agile team, they are there at the daily scrum, the devs know what the designers are working on, it’s a base to build upon, it all can change in the development sprint but at least we have some foundation we can start working with, and iterating on.

The first problem I ran into with the 3D TVs was the glasses, they didn’t work. I tried on the first pair (which had an ugly, heavy, security cable attaching them to a podium), no dice. I tried on the second pair, no luck there either. Looking at the glasses in more detail, I found a power button! Pushed, tried them on, nothing. Repeated the process on the other pair, still nothing. There I was, big electronics department, with a range of 3D TVs in front of me, and the glasses didn’t work.

If I was going to describe the target market for 3d televisions in 2010, I might have included a picture of myself. Male, 30, into technology, owns a variety of different entertainment products and consoles, decent disposable income. As far as I could tell I represented the exact person they were hoping would be the early adopters on these.

While wandering off I finally encountered a sales person, I mentioned that the glasses for the TVs didn’t work. He told me they were fine, and motioned for me to follow him. It turned out that you had to push and hold the power button down for a few seconds in order to turn them on. As I put them on the sales person walked away, and I got to enjoy a demo video in 3D.

Well, sort of. First the glasses had to sync with the television, then I was all set, great demo video on directly in front of me. Of course, I wasn’t in a room with a single television, there was several along the wall to the right and left of the central television, and since my glasses had sync’d with the one directly in front of me (not the others), the other televisions had essentially been turned into strobe lights. Incessantly blinking at me. When I turned my head towards the blinking lights the glasses re-sync’d with a different television, a disorienting procedure that allowed me to view it properly, but turned the one directly in front of me into a strobe light.

So, after requiring aid to put on a pair of glasses that were practically chained down, I was being forced to view very expensive televisions adjacent to a series of strobing pictures with an absentee salesman.

Despite all of the issues, this was really cool! This was 3D, for my living room! No red-blue glasses either, this was the real thing! Maybe I could get one, then invite Allison over again for a 3D MOVIE! Clearly owning impressive technology was the way to any woman’s heart. While those thoughts were racing through my mind I caught my own reflection in a mirror, the glasses were not pretty. If you picture a pair of 3d glasses in your head right now, you’re probably imagining the modern polarized set you get at the movie theatre. Designed to fit anyone, over prescription glasses, rather ugly so people don’t steal them. Those glasses were years away back in 2010, and these were active shutter glasses. Rather than just two panes of polarized plastic, they each lens was a small LCD panel capable of going from transparent to opaque and back many times a second. They looked a little like this, but in black:

As I put the glasses back on the presentation pedestal and rubbed my sore nose I realized: There was absolutely no way I could try to kiss a girl for the first time wearing a pair of those. I left the TVs behind, I think I picked up some apples I could slice and flambé to serve over ice cream instead.

The kissability test: When considering a product, could you imagine kissing someone you care about for the first time while using it.

When I listen to him on the radio, as his calming voice meanders through the lives in his stories, I often picture him. In my mind, he’s sitting in a library on overstuffed leather chair, with a tweed coat laid over the arm, a side table with a lamp and glass of water beside him… perhaps a large breed dog dozing at his feet. As each page came to an end he leisurely grasps the corner, sliding his fingers under the page, gently turning it, just as an avid reader moving through a treasured novel. This is the man I picture when I hear the measured voice regaling me with tales from the mundane to the fantastic.

This could not be further from the truth.

The man I saw at the Sony Centre for the Performing Arts has nothing in common with the man I pictured but the voice. As his story began, as he lapsed into those measured tones, his feet never stopped moving. He danced around the microphone like a boxer, stepping closer to make a point, jumping to the side in excitement, waving his arms in exclaim, always ready to strike us with another adverb. When he reached the bottom of each page, he’d frantically reach forward and throw it back, as eager as a child on Christmas morning. It’s easy to fall under the spell of a great storyteller, to stop seeing and only listen, but his fantastically animated demeanour shook that away, and spiced the story in ways I couldn’t have imagined over years of radio listening.

Listen to his podcast for a while, then go see him live, he’s an utter delight.

Motion Detector Lamp

Woodworking

The enclosure is made out of yellow birch and mahogany. The technique is a derivative of the band-saw box technique used to build small boxes out of a piece of solid wood. The first step is to make a solid block of wood by gluing several pieces together. I decided to sandwich a piece of yellow birch between two pieces of mahogany. Clamp the pieces together and let dry for about an hour.

Next, remove the clamps and square the block on a table-saw or using a jointer. Once the block is squared, cut the two side panels and carve the interior of the box on the band-saw. Make sure you leave enough material for the screws that will hold the side panels. Finally, put it all back together, sand all visible surfaces and apply the finish of your choice.

Tip: Make sure you drill some pilot holes when screwing the side panels to avoid splitting the wood.

[See image gallery at basbrun.com]

Electronic

Your project box is now done, it’s now time to fill it with electronic gear. For my project I’ll use a motion detector, 6 neo-pixels, an LED and a push button. The motion detector sits on top of the box and the six neo-pixels will be equally spaced on two sides of the box.

This is the build of material you’ll need to build this simple project:

• (1)Arduino Board
• (1) Adafruit Prototyping Shield
• (6) Adafruit Neo-pixels
• (1) Push button
• (1) Motion detector

The motion detector is quite simple to use. You power it using the Vcc/Grnd of the Arduino board and when movement is detected it sets the signal pin to 1 (5v). You should then use the ReadDigital() function in your Arduino sketch to read the state. There are two potentiometers on the motion detector, one to set the sensibility of the detector and the other to set the delay the signal pin stays up when movement is detected.

The Neo-pixels are even easier to use. I’ll not go into many details since Adafruit provides tons of samples on their Learning System . All you have to know is that only one output pin is required to drive as many Neo-Pixels as you want.

The button input is monitored using an interrupt. I am using an interrupt because some of the lamp mode sequences are not listening for inputs while executing. The debounce is made in the interrupt callback function. Make sure to read the Arduino spec to know what pin of your board is associated with the interrupts you want to use. For the Leonardo board interrupt 1 is associated to the pin #2 of the board.

Finally, I added a 3mm LED to indicate that the lamp is powered on. Not really necessary since the product is a lamp. The LED is driven through a 1K resistor connected directly to the Vcc pin of the Arduino.

If you want to use the project sketch with no modification connect the devices to the following pins. Note that you can change the pins you use simply by updating the values in the Arduino sketch.

• The IR receiver on pin #4
• The Neo-Pixels data line on pin #3
• The button on pin #2 (For the Leonardo board)
• The power LED on the Vcc pin with a 1K resistor

[See image gallery at basbrun.com]

That’s it … What I love about these little project is that I can use it as sandbox for other projects. I make sure the USB connector is accessible so I can reprogram the board whenever I want to.

You will find all the source code on my github account at: https://github.com/pchretien/motionbox

[See image gallery at basbrun.com]

I’m writing this with mixed emotions. Atom is an editor that feels very much like Sublime Text in pretty much all respects. They even borrowed most of the default keyboard shortcuts, so if you’ve been using Sublime, you will likely be able to easily migrate to Atom. That said, the “feel” is where the similarities end. Instead of a Python core, Atom is based on Node.js and WebKit, using JS (CoffeeScript), HTML, and CSS to customize and implement most of the features. Generally speaking, this is a good foundation— the problem is, Atom seems to have gotten it all wrong when it comes to laying down a strong architecture for both the plugin API and core components. If I could summarize this editor in one sentence, it would be: “a promising and extensible editor sitting on a very shaky technical foundation”.

Because Atom is being marketed as a “hackable” editor, I want to focus most of this article on the API and actual functionality that can be customized. I spent my last week both using Atom and writing a handful of plugins, 3 of which I published (atom-runner, go-format, and rst-preview), so I would say I’ve had a good run-through of the current functionality of the editor and its API.

But before we get there, I do want to cover some of the basics.

Feature Check? Check.

First, let’s start with the easy stuff. On Atom’s landing page, GitHub bills this editor as “extensible”, “web native”, “modular”, and “full-featured”. It’s hard to argue with the first 3, because giving someone untethered access to a WebKit browser (it actually seems to be running node-webkit) and displaying the entire UI inside of a DOM implicitly gives you web nativity and pretty extensive control over the entire application. Extensibility, specifically extensibility provided through the HTML, CSS, and JS web technologies that most developers (and some non-developers) know and use every day, is basically the strongest selling point this editor has. Say what you want about using HTML in a desktop application, but exposing your plugin architecture as a bunch of HTML/CSS is something even some of my non-tech friends could pick up and hack on. Foundationally, GitHub made smart technology choices.

That leaves us with the last point: “full-featured”. This is the one I take issue with. Although Atom already has pretty decent parity with an editor like Sublime, it’s definitely missing quite a lot of the behavior you would expect from a polished code editor. And yes, I know this is beta, and I know things will change, but things only change if there is feedback, so here it is:

The Irony of Git Support

So let’s take a step back and remember that this is GitHub publishing this editor. GitHub, as in, creators of the canonical social Git hosting site that most of us use. And yet, clearly lacking in this editor is any sense of actual Git support. Sure, there are line diffs and some basic diff / branch status listings in the status bar, but come on, not even a built-in “Git Commit” command? For what it’s worth, there is a cool “view [history/blame] on GitHub” command, which is alright, but I imagine for those of us hacking on planes or on non-GitHub-backed projects, that’s not really going to… fly.

Runner? Builders?

TextMate came with some pretty good script running support, a feature I’ve always liked more than Sublime’s variant of “Build Systems”. That said, both of these other editors have pretty good functionality for running code in the editor. I was shocked to find out that there was basically no such built-in support inside of Atom, which is what prompted me to write atom-runner. A final version of Atom must support some kind of equivalent, and it should be built-in.

Miscellaneous UI Fails

Call me spoiled, but these are features I expect every editor to have, if they claim even partial support for:

1. Tab completion is almost useless in its current state. There is no intelligent memory of the last / most used completion in the document like Sublime manages.
2. Ditto for file switching (Cmd+T in Sublime/TextMate). There is no memory of the last file, so you can’t hit the quick Cmd+T – Enter keystroke to switch between two files. You have to fuzzy-find from scratch each time— and the fuzzy-finder is extremely loose in its matching policy.
3. Middle-clicking does not close tabs. — Edit: This looks to have been added into a recent build (as of 03/11/2014). Nice!
4. Unless I haven’t found it, no keyboard shortcuts to split windows or move windows between tab groups. — Edit: Cmd+K, ArrowKey is how you do it. Thanks @bryanl.
5. Finally, this one is probably a bug, but history support is fairly weak and often times simply switching tabs or windows will completely clobber your history. History should be the most stable part of any text editor. If I can’t trust the history management, it’s kind of a deal breaker to adoption, for me.

Performance Woes

I can’t stress this one enough. Atom is extremely slow, and this is extremely disappointing. Performance-wise, running Atom today feels a lot like running Eclipse back in 2006; slow startup, visible lag when switching tabs or loading files, scroll and typing lag. To quantify this with an example, starting up the editor from the command-line in a reasonably sized project (~8 root directories, ~200 files) can take a good 7-8 seconds before the main window comes into view. Similarly, switching between tabs when clicking in the file view or tabs at the top has a noticeable lag on larger files— I’m talking up to 500ms to 1sec. Scrolling those files shows similar lag. Granted, it’s not horribly bad, but it’s enough to feel the pain when switching between Sublime and Atom.

My biggest fear with Atom’s performance problem is that it does not seem conceivable to me that things can get better. Given that Atom is still not “complete”, the “1.0” build will likely only have more code and more DOM nodes to render; and when more plugins are published, you will only have people running more of them at a time hogging up all that processing power. Unless there is some seriously inefficient core code running under Atom, the performance problems really seem like a fundamental architectural limitation, not just a lack of optimization. Just like Eclipse learned, you can’t optimize yourself out of bad architecture, and Moore’s law no longer moves fast enough to be able to save you from scaling CPU intensive applications— especially when your architecture is threadless and single-core-bound.

The only real solution here is to go the route that Chrome went and use separate processes for each tab / component. This would give you the core-scalability that CPU intensive apps need. Unfortunately, shoehorning in such a fundamental piece of infrastructure into a sizeable codebase is not an easy task— if it’s even technically possible (since everything is displayed in a single DOM, I don’t know if it is). This is really something that has to be built from the ground up, but it wasn’t. This might be the most costly mistake GitHub has made with Atom.

Shaky Foundation: The API Is On One Too

A common theme running through my time with Atom is that much of the core infrastructure seems half-baked or completely lacking. It’s almost as if they decided they were going to write a text editor app half way through implementing the text editor component. Much like the lack of a strong performant foundation, there are parts of the API that seem inconsistent, unnecessarily evolutionary, and just generally under-engineered. Here are a handful of wat moments:

Documentation Where Art Thou?!

The documentation ranges from decent to misleading to lies to simply not there. It’s almost impossible to rely on the documentation as is, and I had to resort to simply reading other core Atom plugins instead of looking at docs. At the time of posting, the Editor.getPath() method is still missing from the docs— arguably the most important method if you’re doing anything with the files, and necessary for all 3 of the plugins I published. Similar inconsistencies and notable missing things arise, like lack of event documentation (again, necessary for each of the 3 plugins I published), and lack of explanation for seemingly similar methods.

I’m a little biased when it comes to docs. I happen to believe that the best way to audit an API design is by writing good documentation around it, effectively dogfooding your documentation as the API contract. GitHub seems to have missed the mark here in thinking that docs are an optional artifact that you build up after you’ve finished coding. It doesn’t surprise me, at least, to see that attitude translate into weak APIs and inconsistent use of those APIs across different core plugins in the editor. When there is no document guiding usage, it’s not surprising that you end up with a smorgasbord of implementations.

There Are 4 Ways To Use Events

Searching for a way to register events in my plugins, I ended up reading a handful of different core Atom plugins that do event registration. A came across 4 distinct ways to actually register events:

1. You can use atom.subscribe(), which I found by accident. This was the easiest one, since it didn’t require any magic mixins.
2. You can use the Subscriber module which you can use some fancy syntax to mix into your classes. This, of course, only works if you have a “class”, something Atom does not generate for you by default (it just throws you a bare Object returned in module.exports).
3. You can just do your event registration in the View class, if you have one, which has Subscriber already mixed in.
4. Or, finally, you can use Node.js style events and just call buffer.on 'event' instead of going through a Subscriber.

This begs the obvious question: why 4 separate ways? It also begs the more subtle but subversive question: why would you ever register an event from a view?

Again, the lack of documentation means everyone is going to find their own slightly different way to register events. After a while, you end up with a really messy API. If you missed it— events have no docs. I guess based on that fact, it’s not too surprising that the above happened.

The Plugin Architecture Is Not MVC, It’s Just “V”

Atom has a strong concept of a View, which is good and expected. Less expected, though, is the lack of any other kind of architectural components. Basically, the only thing you ever operate on/with are views— until you are not. There is a “View” superclass, but there is no “Controller” superclass. That’s why you have to use a View to subscribe to events, which Views shouldn’t even be doing.

Another great example is that in order to register a plugin command you actually have to call a method on workspaceView. Not the Workspace object, but rather, its associated view. The weird part is that the commands are not registered from views, they are actually registered from the pseudo-controller thingies, but they are registered on views. Wat.

Other Various Mishaps And Crazies

1. You can get from an EditorView to its associated editor (getEditor()) but you cannot get back to the view. Generally in a DOM like arrangement, you should be able to get back to the parent of a child node (especially since these things actually are DOM nodes). Of course, this is absolutely necessary when some events are only emitted with an editor handle and not the view (like atom.project‘s editor-created event), because you need the view to actually do anything useful (remember, it’s all in the “V”).
2. atom.workspace has a getActiveEditor(), but workspaceView does not have getActiveEditorView(). Instead the latter only has getActiveView() (which may not be an editor).
3. There is a (undocumented, of course) configDefaults hook and a atom.config.setDefaults command. Why both?
4. There are magical “attach” and “detach” methods associated with the main module but it’s unclear how they differ from “activate” and “destroy”.
5. Serialization is apparently handled by the “controller”, but views can also serialize/deserialize themselves. It’s unclear which you should use and when.
6. There is a gutter API (the git-diff package uses it) but it is completely unexposed and subject to churn. Again, dogfooding is the answer, here.

But I Want To Like It

I really tried hard to like Atom. I still really do like it. It’s beta, and I’m willing to bet that it will get (functionally) better. I am still, however, seriously concerned about performance, and although I thought I wouldn’t mind it too much, I found myself getting extremely frustrated with the lagginess by the end of the week, to the point where I launched Sublime a couple of times when I knew I was going to touch larger files.

Atom has a ton of potential. The power of using WebKit to render a UI opens a ton of doors, because it literally lets you customize every single pixel in the window. That’s power I don’t even think Vim or Emacs have. There are so many plugin ideas I have with that kind of power, including local Fiddle support, live updating game IDE features (you can use a Canvas to live-render your game) like Processing, live Brackets-style web editing, and more.

I want Atom to work, I really do, but it’s not quite working, yet. If GitHub can resolve performance problems, lay down a solid foundation for an API, document it, and fix some of the minor UI glitches, they would have something I’d be happy to switch to. Until then, I’m probably going to play around with it for a little longer and then inevitably switch back to Sublime.

Just after Christmas Allison was kind enough to give me a hand knit sweater she’d been working on for over a year. It’s fantastic. Around the same time she’d commented that she was jealous of my neck warmer. An idea stuck! I’d knit her a neck warmer!

Small problem: I’ve never knit a thing in my life.

I’ve never let not knowing how to do something stop me before, and this didn’t seem like the time to start. I headed up to Ewe Knit, where Caroline was able to administer a private lesson. First I had to use a swift to turn my skien of yarn into a ball. They appear to sell yarn in a useless format to necessitate the purchase of swifts, a good gig if you can get it.

Once I had my nice ball of yarn I “casted on” a process I promptly forgot how to accomplish. The process mostly consisted of looping things around one of my knitting needles the prescribed number of times. That number was 17 according to the pattern. Once I’d casted on the regular knitting started, each row involved an arcane process where I attached a new loop to the loop on the previous row. The first few rows were quite terrifying, but eventually I slipped into a rhythm, and was quite happy with my progress by the time I’d made it to the picture shown below.

Just a few rows later I made a terrifying discovery: I’d invented a new form of knitting. Rather than knitting a boring rectangle, I was knitting a trapezoid, and there was a hole in it. My 17 stitch pattern was now more like 27. There was nothing for it but to pull it out and basically start over.

Several hours, and many episodes of The Office later, I’d slipped into a great rhythm, and developed a mild compulsion to count after every row to ensure I had 15 stitches. The neck warmer was looking great! Just another season of The Office, and some serious “help” from the cat, and I’d be finishing up.

I headed back to Ewe Knit for instructions on casting off, where I tied off the loops I’d been hooking into with each row. Then I sewed the ends together, and wove in my lose ends. A neck warmer was born!

This felt like a success before I’d even wrapped it. I’d spent a lot of time working on something she’d value, I’d learned more about one of her hobbies, and gained new appreciation for the sweater she’d knit for me. She just needed to unwrap it.

She loved it.

Samus is Open Source and available on GitHub. You can read a lot more about how Samus works in the README, but read on if you want more example usage.

Samus Goals

There are a couple of problems I’m trying to solve with Samus. I noticed these problems when releasing versions of other OSS projects I maintain

1. Automate the obvious release-time tasks like bumping your library version, adding a version tag, building the RubyGem, and more. Although tools like npm have language specific ways to handle some of these things, Samus gives me a generic solution I can use on pretty much all the code I release, from C, to JS, to Ruby code.
2. Simplify credential management so that you know your machine has the right keys before you try to push code. I release code from different machines. I usually have Git setup on these machines, but API keys for package managers is occasionally a different story. Samus allows me to share credentials in a way that is as secure as I need it to be
3. Package an archive of the release so I can batch releases. If I’m doing a release of multiple libraries at once, I usually want them all staged before I push them out. Having a separate “stage” step lets me make sure everything is ready and working before I start a release. More on this at the end of the post.

Quick Start

Here’s a simple example of how I published Samus, today:

First, I created a file ~/.samus/personal/credentials/lsegal.rubygems with my RubyGems access key. The contents of the file are:

Secret: MY_API_KEY

Samus looks inside ~/.samus for custom commands and credentials. The sub-directories can be local-only directories, or they can be Git-backed (and optionally private) repositories that I can share with other maintainers. Organizing this is mostly up to you, so there is a lot of power here.

Next, I ran samus build 1.0.0, which used the samus.json manifest file to run the tasks needed to build the release-v1.0.0.tar.gz file.

Finally, as the last step, I publish with: samus publish release-v1.0.0.tar.gz.

That’s it. In short:

mkdir -p ~/.samus/personal/credentials
echo "Secret: API_KEY" > ~/.samus/personal/credentials/lsegal.rubygems
samus build 1.0.0
samus publish release-v1.0.0.tar.gz

Why Separate Build & Release Steps?

One big change from the normal work flow is separating “staging” a release from the release itself. Typically, when you release code, you go into your project, update version, build the gem, push code, and push the gem. This all happens from your workspace. Although this works for a single project, it does not work quite as well when you are releasing multiple versions of the same project (like, say an RC candidate and a stable release, or multiple libraries that depend on each other) at the same time. A lot of build time tasks can take a while, like running test, building docs (if necessary), occasionally compiling code, so if I’m going to release multiple things at once I want to make sure everything is good to go before I hit the button. It’s sort of the same way “git commit” and “git push” are separate tasks. You often want to make sure everything is setup locally before you “push-to-live”.

That said, Samus could very well include some simplistic “build-and-push” single-command syntax, if it were useful.

Samus TODO & Contributing

Although Samus is designed to be a simplistic tool, there is still a few extra features it could use. Here are a handful of ideas:

1. Samus could use more build and publish commands. For example, a command to update your changelog at build-time, more control over Git workflow management, and integration with more package managers (only RubyGems and npm are currently supported) would be great.
2. Improving the credential management to support different setups. Currently Samus does rely on SSH keys being available on the machine for Git commands. I would like to fix that.
3. Self-contained releases could be useful. Right now you still need some local setup to use Samus, like credentials on your machine, and occasionally some binaries (custom commands, if you use them). Being able to build a self-contained archive with all the custom commands and credentials in the zip would make publishing much more reliable (though a little less secure if that archive got leaked).

Extra commands are always welcome, so if you have an idea for a generic command, feel free to submit a pull request to have it included in Samus.

If you think Samus would be useful, feel free to give it a spin and let me know what you think. I just updated YARD to use Samus by adding the manifest file in the Quick Start above– for most Ruby projects, this would be enough to get you started.

I then published the post on reddit.com. The post generated about 75 more visits in the hours following the post. That was cool but nothing that deserve a complete blog post.

A few days later, for the second time in the short  life of this blog, I have been published on hackaday.com. In less than 2 hours I got more than 500 visits. This is quite unusual for my blog! :) Of course, that is far from what has been described as the slashdot effect, capable of killing your web server for hours, but I was pretty proud of that appearance.

A week later, I published my article on the DIY site instructables.com. This site shows the number of visits on the project page. The page view was going up pretty fast when I got featured on the front page of the site. From that moment on the visits exploded. After five days on the site I have more than 10 000 views. This is more than 10 times the number of visits from hack a day.

An other thing I learned this month is that you can share links with other websites. Eeweb.com offered me to publish my website as the website of the day. I thought that sounds a bit 90’s but I decided to give it a try. After two days featured as the site of the day I received 19 visitors from their website. That’s not a lot of visibility for my projects and now I have to put a link to their website on mine. Done.

But then, can we compare visits on my blog to page views on instructables … I don’t think so. It is much harder to bring visitors on your website than to get page views on a site like instructables. The question is, what ratio of the hack a day visitors actually click on the link to my website. For most visitors the text published by the hack a day staff is enough and they don’t feel the need to visit my blog.

Some people will argue that we must also consider the “quality” of the visitors. That would mean that the 1000 visitors from hack a day are of “less quality” because they come from a site that only gives an overview of the post. That would also mean that  the 300 or so visitors from instructables are “high quality” visitors because they come from a site that publishes the complete article, which means they visit the blog to learn more about your work.

Well, I don’t really know much about all this web mathematics but since it gets real close to marketing I have a tendency to believe this is … bullshit. I like as much my 75 extra visitors from Reddit than the 10 000 viewers of my instructables project. I write a blog to share my project and ideas, the more I am read, the more I am happy.

RGB / RFID Lamp Project

You can change the color of the lamp by placing a colored disc into the circular recess in front of the lamp body. When no disc is placed in the receptacle, the lamp changes color randomly. Place the white disc to produce a white light and the black disk to turn off the lamp. You can also turn the lamp into a stroboscope or produce other special effects by combining different sequences of colors.

The body of the lamp and the RFID disks are made out of scrap wood I had in my shop. The shade is made from a transparent drawing tube and tracing paper.

[See image gallery at basbrun.com]

The material

There are many different ways this project can be built. The following list only includes the essentials electronic components required for the project.

• RGB LEDs strip
• 3 N-Channel Mosfet transistors
• RFID reader and tags.
• One 3mm or 5mm LED
• Arduino micro-controller.
• Piezo speaker (optional)

 The Circuit

The circuit for this project has many uses:

• Drive the three color channels (RGB) of the LED strip
• Read data sent by the RFID reader
• Drive the “ready” LED
• Send tones to the piezo speaker if any

Since all this is controlled by an Arduino micro-controller, building the circuit as an Arduino shield would make it more compact and reusable. The arduino shield PCB design will be explained in the next section.

RGB / RFID Lamp Circuit (Breadboard View)

The RGB channels are driven by the Arduino pulse with modulation (PWM) output pins 3, 5 and 6. The three Arduino logic pins are connected to the gate of three N-Channel Mosfet transistors. You can control the intensity of each channel by writing values from 0 to 255 on any of these pins.

The RFID reader communicates with the Arduino micro-controller through a serial connection. I wanted to be able to re-program the lamp without having to disconnect the RFID reader so I decided to use the SoftwareSerial library. This library allows you to use the IO pin of your choice for serial communication, freeing the pins 0 & 1 for communication with the computer. I used pins 8 and 9 of the micro-controller to communicate with the RFID reader.

The power is provided by a 12v 600mA wall transformer. The 12v pin of the transformer is connected to the power wire of the LED strip and to the Vin pin of the Arduino.

I added a small LED to indicate when the RFID reader is ready to read a new tag. This LED is connected on pin 7 of the Arduino micro-controller. Finally, an optional piezo speaker can be attached to pin #10 of the Arduino bord to integrate sounds in the project. The “ready” LED and the piezo are optionals.

[See image gallery at basbrun.com]

The Arduino Shield

This project was a perfect fit for a custom Arduino shield. I designed the shield using Fritzing, an open source application that allows you to build your circuit and PCB at the same time. The circuit has been designed as a single side PCB shield. You can find the Fritzing project on my Github repo at: https://github.com/pchretien/rgbrfid

[See image gallery at basbrun.com]

Woodworking

I love woodworking and I love electronics so mixing both into a single project was pure fun. I used Stketchup to do draw the lamp and test the design. The plans are available on my Github account at https://github.com/pchretien/rgbrfid.

RGB / RFID Lamp Sketchup Drawing

To build the body of the lamp I cut two pieces of 1/4″ plywood and four pieces of 3/4″ pine. To cut the shape of the lamp I printed a 1:1 template of the lamp using a TOP / Wire-frame view in Sketchup and glued it on top of the boards. To make sure the boards stayed aligned during cutting, I put some carpet tape between all pieces. (this trick from the best woodworking show on Youtube woodworkingformeremortals.com).

Once the shape of the lamp is cut, remove the two plywood boards and trace a line 3/4″ inside the outline of the lamp. Again, using the band-saw, cut the inside of the lamp. It’s now time to sand all the pieces and apply several coats of spay paint.

The six wooden disks are cut from a piece of 1/4″ plywood. Each disk is made gluing two 1/4″ disks together. Do not forget to place the RFID tags between the two pieces of plywood. I used a 1 1/2″ Fostner bit to make a shallow recess to fit the RFID tags. Sand and paint the disks with the colors of your choice.

Finally, I used a PVC pipe to hold the LED strip. I painted it white, cut a hole on top of the lamp and placed the pipe into the hole with epoxy glue.

[See image gallery at basbrun.com]

Assembly

I assembled the lamp with my youngest kid… Here is an overview of the process.

1. Wrap the LED strip around the PVC pipe and mark where it should be glued to cover the whole length of the pipe
2. Glue the LED strip onto the pipe and pass the wires through a hole at the bottom of the pipe
3. Attach the RFID reader under the disk holder with small wood screws
4. Attach the Arduino to the bottom of the lamp
5. Glue the small LED in the center of the disk receptacle through a small hole
6. Strip the transformer connector, pass it through a hole on the back of the lamp and make a knot
7. Connect all the wires and connectors to the Arduino shield
8. Screw the bottom of the lamp and install the rubber pads
9. Et voilà!

[See image gallery at basbrun.com]

The Code

You will find the Arduino code, the Sketchup drawing and the Fritzing circuit schematics on my github account at: https://github.com/pchretien/rgbrfid

Our provider asked us to pay them by wire transfer, so I dutifully walked to the bank, stood in line, then paid $40CAD in fees & a horrible exchange (to USD) rate to send them money. Not a great day, so I grabbed a burrito on the way home. A few days later we were informed that some intermediary bank had skimmed $20USD off our wire transfer, so our payment was $20USD short. Swell.

In order to send them $20USD, I’d need to go back to the bank, stand in line, hope I got a teller who knew how to do wire transfers (the first guy didn’t), buy $20USD for the provider, $20USD for the intermediate bank, and pay $40CAD for the privilege. $80 to send them $20. Super.

Luckily XE came in to save the day again. Using their convenient online interface I was able to transfer $40USD for only $63CAD, including my wire fee. I paid a much better exchange rate, lower wire fees, and didn’t have to put pants on. The only downside was a lack of burrito. Bummer.

If you’re dealing with multiple currencies and multiple countries, and these days it’s incredibly likely that you are, I’d highly recommend XE.

Using GeoIP:

1. How to use it

Deciding how you’ll use GeoIP information will affect how much you end up spending for a GeoIP database, how often you’ll renew, and what safeguards you’ll need to put in place. Country level granularity is relatively easy to come by, city level within the US and Canada however tends to be much more expensive.

Our integration goal is to support a nice slogan, for us that’s “You’re in .., Your customers aren’t. WonderProxy: Test Globally”. We opted for Region level as opposed to City, or Country level. We felt like “Ontario” or “Texas” was more impressive than “Canada” and “United States”, but were also wary of the lower accuracy level with city rate (Telling someone they’re in Brooklyn, when they’re really in Manhatten wouldn’t inspire confidence).
2. Acquire a database

There’s several options available. This step was easy, we bought ours from MaxMind. We feel relatively familiar with the GeoIP data provider marketplace, and MaxMind has seemed both quite accurate and responsive to updates throughout WonderProxy’s existence. IP2Location is another provider with downloads available.

MaxMind also provides API access to its data. We’ve been leveraging this for a long time in our monitoring systems (we check all our servers to ensure they’re geo-locating correctly), but they’re all batched processes. Waiting for a remote API to return during page load, in particular for a landing page is folly. IP2Location also offers an API, as does InfoSniper. APIs work really well in batched process, or anything somehow detached from page loads.

3. Rebuild Apache & PHP

Our initial build only required the Apache module, this way additional superglobals were provided in PHP. I can grab <?=$_SERVER['GEOIP_REGION_NAME']; ?> get someone’s location, it’s really easy. We later installed the PHP module (using the same database) to support arbitrary IP lookup within our administration systems. We also encoded the MaxMind ISO 3166 data into our application to convert country codes to names.

If you’re taking the API approach life should be easy, there’s plenty of code examples for every major API provider. If you’re using an API you also have the ability to choose different levels of granularity on the fly, full data some of the time, minimal data most of the time to save on credits.

4. Handle edge cases

Not every IP will have a result, it’s important to catch these and handle correctly. We’ve simply decided to test the variable and replace with “Your Office” when the lookup fails.

On the API front It’s worth spending a few minutes to make a request fail on purpose and ensure your code handles it well. I’ve had a few important daily reports fail because the API we were using was unavailable, frankly it’s embarrassing.

We've been really happy with how easy the integration has been. I've already added several new integration points throughout our administrative system (providing lookups on banned users, the IP associated with transactions, etc.). For us the integration is really supporting the slogan and looking nice, but there's plenty of practical uses like estimating shipping charges, localizing prices, and adjusting content.

This was a huge mistake

As I worked on the redesign for WonderProxy (Original vs Redesign) I added a few basic admin features almost by accident, and all our lives got remarkably easier. I added a few more, and things got easier still. Tasks that used to be a chore (like setting up a free trial) almost became fun. Researching account history is just a few easy clicks, with nice graphs using nvd3, and pretty data tables. Editing accounts in place, with code that understands 30GB = 32212254720 bytes.

Saying things “creating trial accounts is fun” may sound like gross exaggeration, but it’s not. I’m pretty happy with the code i’ve got there, which may add to it. The form supports pasting in an address like Paul Reinheimer <paul@preinheimer,com>, parsing it out to its component parts, and generating an available username. Then in for expiry, I leverage php’s strtotime() function so I can enter in something simple like “+2 weeks” or “next thursday” and have that parsed and work properly.

The speed at which we’re both willing and able to resolve requests has greatly increased. Trial accounts (which convert with great regularity) are easy to do in a minute, rather than 10, so we’re more likely to do them when they roll in, rather than waiting until we’re on the server for another reason. When it comes to customizing accounts, getting an accurate history, and being able to quickly modify accounts has helped everyone. I’m lacking a basis for comparison, but our revenue has also been climbing nicely since the change, having a dashboard to find clients exceeding their plan limits has certainly helped.

If you’re looking at the next big thing to improve for your team, I’d strongly suggest taking a harder look at your internal tools.

• Cheque

These are easy, we open a filing cabinet, pull out a cheque, sign it and put it in the mail. Then the contractor waits for a week or so for the cheque to arrive, deposits in the bank, receives a horrible exchange rate, and waits two or more weeks for it to clear.

• Wire Transfer
This requires me to go to the branch in person, wait in a line, pay a lot in fees, then the money shows up in a day or three, possibly with additional fees being deducted along the way.
• PayPal
These are easy, open up our PayPal account, make a transfer, log out out again. The recipient ends up paying like 3.5% in fees, receives a mediocre at best exchange rate, then waits longer for the money to appear in their account.

So, everything sucks, bad rates, fees, and possibly involving me going to a bank in person.

Eventually we got frustrated with the money we were effectively losing with crappy exchange rates, and took a look further. We came across XE's currency trading services. It took a fair amount of effort to sign up (various forms to be scanned and sent in), but it’s been fantastic. I log on to execute a trade, enter that I would like to buy USD with $1000CAD, and get a spot rate on the transfer. I choose to execute the trade, with the USD funds being deposited in the recipients US account. I pay XE through my bank’s online bill payment services, then about a week later the US funds arrive in the recipient’s account.

I get to do it all from my desk, we get a fantastic exchange rate, and the transfer service makes its money on the spread, so there’s no extra fees.

If you’re paying across borders, I would heartily recommend investigating XE to see if they can meet your needs.

Around the same time, at my son’s summer camp, the instructor was telling them scary stories. That gave him the idea to convert the stuffed monkey into a scary monkey with red bright eyes and playing a spooky melody! This post details the building process. I hope you’ll enjoy!

“Scary” Stuffed Monkey Project

The plan

A small pressure button will be placed in one hand of the monkey. When pressed, the  monkey’s eyes, two red LEDs, will turn on and a spooky melody will be played.

Since we had more IOs at our disposal, we decided to add a light sensitive resistor at the top of the head that would control a green 3mm LED in the nose. When the light is turned off or when you place your hand on the head, the LED starts blinking.

Finally, I decided it would be nice not to have any power button. Kids always forget to turn these off anyways. This implies some sort of sleep mode for the toy to save power.

The Micro-Controller (08M2)

The project circuit is built around a PICAXE 08M2 micro-controller. This micro-controller has four inputs and outputs. Several reasons have led to the choice of this micro-controller:

1. Size The PICAXE 08M2 is one of the smallest through hole micro-controller on the market. An Arduino microcontroller board would have been too large for the project.
2. Power I wanted the circuit to be powered by three 1.5V button cells and run for days beforereplacing the batteries. This micro-controller consumes very little power and offers a sleep mode that almost totally shuts down the chip. Using this great feature I managed to build a circuit that can run for weeks on three LR44 batteries.
3. Simplicity This microcontroller is very robust and simple to integrate into a circuit without complex power regulation circuit and no external crystal is required to drive the chip clock.
4. Price Price is always a concern when you build something. The PICAXE 08M2 chip cost around 3.00$ CN. Compared to the 25$ Arduino board this is a bargain!

PICAXE 08M2 micro-controller

Inputs / Outputs

The circuit must drive three different components. Two red LEDs for the eyes, one green LED for the nose and one piezo for the scary melody are controlled by a pressure button and a photo-resistor.

Scary Monkey components

The two red LEDs are connected in parallel to the same digital output pin of the micro-controller. Each LED is connected to a 220 Oms resistor to reduce the current driven through the LED.

The green LED is connected to an other digital output of the micro-controller. As for the red LEDs, a 220 Oms resistor limits the current passing through the LED.

The piezo needs an output that supports PWM (Pulse With Modulation) to play the different tones of the melody. The PICAXE API offers the function tune() to play music on pin #2 of the 08M2 micro-controller.  The PICAXE IDE offers a simple wizard to  help you program the melody.

Two different inputs are used to interact with the stuffed animal. A pressure button in one hand of the monkey triggers the melody and lights up the eyes. The LED in the nose of the stuffed animal is controlled by a photo-resistor located in the head. When the lights are turned off or when you place your hand over the photo-resistor, the nose of the monkey blinks green.

Since I am using standard electronic components I had to find a way to stitch it to the stuffed animal. I used small pieces of perforated prototyping board and stitched through the holes around the components. Next time I’ll use the Adafruit Wearable Electronics!

The Circuit

The circuit for this project is very simple. Because we are not using motors and because the PICAXE micro-controller is pretty resistant to small power fluctuations, I decided to power the circuit directly from a 4.5V stack of button cell batteries. No need for power supply or power regulation.

Scary Monkey Circuit

At the exception of pin #2 (Serial In) , all the micro-controller pins are used. As recommended by the manufacturer, I connected pin #2 to the ground through a 10KH resistor.

Pin #3 is used as a digital output to control the eyes of the monkey. Both red LEDs are connected in parallel to the micro-controller pin through a 220H resistor to limit the current flowing in.

The micro-controller pin #4 is home for the push button. This is the button that will be fitted in the hand of the stuffed monkey. A 10 KOms pull-down resistor is keeping the input pin to zero when the button is not pressed. When pressed, the button connects the 4.5V trace to pin #4.

The piezo is attached to the PWM output on pin#5 of the micro-controller. Pin #5 is mapped to input #2 of the PICAXE 08M2, the only output supporting pulse with modulation (PWM) required to play music tones.

To detect the level of light we hooked a photo-resistor to pin #6 of the 08M2. In conjunction with a 10 KOms resistor, the photo-resistor acts as a voltage divider connected to pin #6 of the PICAXE.

Finally, the green LED located in the nose is connected to output pin #7 of the micro-controller. As for the eyes LEDs, a 220 Oms resistor is connected in series to LED to limit current.

To power the micro-controller, connect the positive wire of your battery pack to pin #1 of the chip. The negative pole of the batteries goes to pin #8 of the PICAXE 08M2.

 The Printed Circuit Board (PCB)

As mentioned earlier, size was critical in this project. I didn’t want to end-up with a bulky circuit squeezed into the stuffed animal. To minimize the size I decided to make my own PCB.

Custom PCB

Building a PCB requires a lot of preparation and many steps to complete with success. I used the open source application Fritzing to design the PCB. You will find the schematics on my Githug account at: https://github.com/pchretien/julie.

The first step is to print the circuit onto the copper board. There are many tools and products out there to do that but I used the simplest and cheapest way: toner transfer. Toner transfer consists of printing the circuit on a glossy paper using a laser printer and then using an iron to transfer the circuit onto the copper board. This is made possible because the glossy paper will not absorb the toner. When heated with the iron, the toner can be “transferred” to an other surface.  Here are a few things to remember when doing toner transfer:

1. Use a glossy and thick paper. I used an empty Avery sticker sheet to build this circuit.
2. Clean the copper surface before to applying the toner. I used soap and acetone. Do not touch the surface with your fingers.
3. Stick the piece of paper onto the copper board so they remain aligned. Make sure the toner side touches the copper board.
4. Use an iron at max temperature and apply pressure on the entire surface of the circuit. This will take several minutes to complete.

After a while, the toner will be transferred to the copper surface. The result will vary from one time to the other. Complete the traces with a Sharpee marker where the toner has not been transferred properly.

The next step is to remove the copper layer where there is no toner or Sharpee. I used ferric chloride to do this. Place your circuit board into a plastic container and cover it with ferric chloride. Gently move the container from one side to the other for a couple of minutes. It may take 10 to 15 minutes for the copper to completely dissolve.

You’re almost done. All that’s left to do is to clean the circuit to remove the toner and drill holes. I used acetone to clean the circuit for this project but I would not recommend it. It spreads the toner all over the circuit surface and makes it look dirty. I would recommend using a brush and soap. It takes longer but the final result is nicer. To drill the holes I used a 1/32″ drill bit. You may have to use a larger bit for some components like a power connector or a USB connector.

Assembly

It’s now time to assemble the circuit and to fit it into the monkey. You must remove the existing stitches using a utility knife or a proper seam ripper. Start with the back where you will insert the circuit board and wires. You will also need to rip the hand stitches to insert the button. You should be able to insert the eyes and nose LEDs and the photo resistor from the back. If not, remove more stitches.

This part of the project is where you can involve younger kids. Stitching is pretty easy and you don’t really care about the look of the final result. Placing the components in place is something any kids can do and it really gives them the impression that they actually build the whole thing by themselves.

Assembly Of The Scary Monkey

The Code

The PICAXE micro-controller comes with a free to download programming IDE . You connect your computer to a development board using a special cable connected at one end to the development board and at the other end to a USB port of your computer.

The language used is good old Basic. As a software developer I must admit I would have preferred a C compiler as with the Arduino but, hey, to light up a few LEDs and play music, Basic will do the job just fine.

You can download the code from my Github account at: https://github.com/pchretien/julie

It’s open and free

This project is published under the GPL2 license. You can use, make and duplicate this project as you which …

Besides all of the performance and cache memory improvements made between Atmosphere 1.0, version 2.0 comes with a host of improvements and new features. Probably the biggest improvement that our users will notice is much better memory usage under high load and improved WebSocket/fallbacks transport performance. This changes alone makes it worth upgrading!

Read the announcement here!

Follow me along and read my new post about Vim: How to Configure Syntastic.

Until then, I’ve decided to post some of my early audio tests with the ruby-audio gem. Installing it is pretty easy, but it relies on libsndfile which you will need to brew install (or other package manager). This worked for me:

$ brew install libsndfile
$ gem install ruby-audio

Then it’s just a matter of using the API, which is not very heavily documented but not too hard to figure out. I managed to create a simple sine wave with the following code:

require 'ruby-audio'
include RubyAudio

sample_rate = 48000
len = 10   # 10 seconds
freq = 440 # Play the C note
amp = 0.9  # Almost full volume

# The sample_rate dictates how many samples (or data points) make up
# ONE SECOND of audio. To get 10 seconds of data points we multiply
# our duration by the sample_rate.
buf = Buffer.float(sample_rate * len)
buf.size.times do |i| # for each data point
  sine_rad = ((freq * Math::PI * 2) / sample_rate) * i
  buf[i] = amp * Math.sin(sine_rad)
end

# Create the .wav file and write it with our buffer
format = FORMAT_WAV | FORMAT_PCM_16
info = SoundInfo.new(channels: 1, samplerate: sample_rate, format: format)
snd = Sound.new('out.wav', 'w', info)
snd.write(buf)
snd.close

After reading Robin Schmidt’s tutorial on Sinusoids and the Frequency Domain, I was recalled back to my algebra days when we talked about sinusoidal periods, at which point I remembered that the sine function has a natural frequency of 2π (6.28…). That means if we want to generate a sine wave at 440Hz we first have to normalize the period to 1, and that is done by multiplying the frequency by 2π. Robin explains it better than me. Unfortunately, the part about sample rates was omitted from that document, and we need to take that into account too. Multiplying by 2π normalizes the function so that a period occurs from t=0..1, t=1..2, etc. ("one second"), but our one second is actually at sample_rate on the x axis, so we have to re-normalize for that by dividing. That’s what our sine_rad variable refers to.

Writing the wave file is fairly straightforward. You give it a sample rate, number of channels (our buffer only contains one channel, which means the file is mono, not stereo). The only somewhat confusing part is the format, which I took from the documentation.

For reference, here is how a square wave could be created:

buf.size.times do |i|
  buf[i] = amp * (-1) ** (freq * i.to_f / sample_rate).floor
end

Using (–1)^x is a nifty little trick to oscillate from [-1,+1] as long as x is an integer (the value is a complex number if x is not an integer, and that’s complicated!).

The next post will hopefully have more progress on the VST plugin.

So here we go. Down the rabbit hole.

Resources

Robin Schmidt’s Music Engineering Tools — The first important resource I came across happened years ago when I made attempt number one to learn about how digital audio worked. It was then that I came across probably the most important beginner’s resource to digital audio anybody could possibly find. I found this resource again a few days ago while researching the same basic principles and was instantly reminded why this site is so important: the articles provided here are well written with just enough math to understand the concepts but without going into crazy abstract mathematical details. In short, this is probably the closest to a set of "explain to me like I’m 5" articles without the impractical analogies… and maybe a little bit more math. I don’t think you will find any simpler explanations of these concepts anywhere else on the internet, though.

The ruby-audio gem — It’s a little dated and extremely limiting, but the ruby-audio gem let’s you quickly create .wav files with a bunch of data fairly quickly. I’ve been using this to experiment with simple sine waves (I’ll show this in the next post) to get a practical understanding of the concepts I’m reading about.

The First Goal: A VSTi Plugin

One of the things I want to do is make it really easy to prototype sounds and filters I’m implementing, and Ruby is the obvious choice when it comes to quick prototyping. The ruby-audio gem works alright, but being forced to write a wav file to disk and load it up into an audio editor makes my workflow slow. Being able to stream audio signals directly to a DAW (Digital Audio Workstation, i.e., Ableton Live, Reason, Pro Tools, Cubase, etc.) would make things much simpler, but this means I need to write a VST plugin in order to get that audio into those programs.

The problem is, VST’s are pretty much only developed via a C++ SDK that Steinberg has tight control over, and there is very little documentation on how to roll your own VST plugin without the SDK (there are open source implementations, but I won’t waste my time reverse engineering this stuff). Furthermore, the Ruby landscape is pretty much void of anything that even comes close to any audio libraries (ruby-audio is one of the very few exceptions), so I’m working in the dark here.

Fortunately I have experience embedding a Ruby interpreter into C applications, so my next step will be to take a really simple VST plugin base and bind all of the hooks down to a Ruby script running in the background that I can change at runtime, or at least load up without recompiling. I already compiled the AGain VST sample that comes with the SDK and have a good idea how to hook this up to a Ruby interpreter, but there are two things I haven’t yet figured out:

1. A VST host (DAW) has to load a VST plugin as a DLL (I say DLL to disambiguate "library", which might be confused with a RubyGem or some plaintext .rb file; these DLLs are compiled down to machine code with exported symbols to be loaded by a process) file on disk, and a plugin does not control the process, so I can’t just "create a plugin" by launching a Ruby process and requiring some library code. I have to create a single C++ VST plugin (by compiling a DLL) that calls out to Ruby from the hooks provided by the SDK’s API. The problem is that I don’t really know how I’m going to load arbitrary Ruby code if I can’t control the process, and more importantly, how I’m going to change the code on the fly, since I won’t have an interactive console. I’ll probably have to point to a file on disk that gets reloaded on modification, unless I can spawn up a mini web-server from the plugin that evaluates code passed in via some fake irb console. DRb won’t work, since I can’t pass Procs across the wire, and that will definitely be necessary.
2. The AGain sample is a "VST" plugin. This is different from "VSTi" in a way that I am not fully aware of. Ultimately I want a VST plugin that produces new audio samples, not filters existing audio samples, and that is what a VSTi (or VST instrument) plugin is for. The problem here is that I’m not sure if I need to use a different API to create a VSTi. I’ll have to keep digging for a sample on this. It might be pretty easy to make, but I just don’t yet know how.

Once the VSTi plugin is finished, I should be able to load up Ableton Live and play my data directly into a buffer, record it, and analyze the differences… as well as hear it live. It will also give me a chance to create a VST library that seems to be missing in the Ruby ecosystem.

That’s it for now.

wAsync is really simple to use. For this blog, let’s write a simple chat using Node.js. First, let’s define a super simple Node.js WebSockets server with static files support. I’m dropping the entire code here assumining you know how Node.js works.

Now let’s write a wAsync’s Android client that can communiate with this server. Graphically, it will looks like

Pretty simple Android application

As with for the Node.js server, I assume you are familiar with Android development.

Now let’s explain the code. First, we create a new client (line 19)

Client client = ClientFactory.getDefault().newClient();

Next, let’s create a Request to send:

RequestBuilder request = client.newRequestBuilder()
   .method(Request.METHOD.GET)
   .uri(serverIpAddress)

This demo is using JSON, and we want to use Jackson for marshalling/unmarshalling Java POJO. So just we define a really simple Encoder:

.encoder(new Encoder<Data, String>() {
           @Override
           public String encode(Data data) {
              try {
                return mapper.writeValueAsString(data)
              } catch (IOException e) {
                throw new RuntimeException(e);
              }
           }
})

This code will be invoked before the message is sent to Node.js and will covert the POJO to JSON. Next we define a decoder that will handle Node.js’s JSON response. We only decode messages, we aren’t interested by other events wAsync propagates:

.decoder(new Decoder<String, Data>() {
           @Override
           public Data decode(Transport.EVENT_TYPE type, 
                              String data) {
 
             if (type.equals(Transport.EVENT_TYPE.MESSAGE)) {
                try {
                   return mapper.readValue(data, Data.class);
                } catch (IOException e) {
                }
             }
             return null;        
})

Next, we create our Socket to Node.js (line 49)

final org.atmosphere.wasync.Socket socket = client.create();

And define two functions, one for handling the message, one for handling unexpected exceptions, and finally open the Socket

socket.on("message", new Function<Data>() {
           @Override
           public void on(final Data t) {
               uiHandler.post(new Runnable() {
               @Override
               public void run() {
                 Date d = new Date(t.getTime());
                 tv.append("Author " + t.getAuthor() + "@ " 
                   + d.getHours() + ":" + d.getMinutes() + ": "
                   + t.getMessage() + "\n");
                  }
              });
            }
            }).on(new Function<Throwable>() {
                @Override
                public void on(Throwable t) {
                    tv.setText("ERROR 3: " + t.getMessage());
                    t.printStackTrace();
                }
        }).open(request.build());

Finally, we just display what we are receiving on the Android screen:

  bt.setOnClickListener(new OnClickListener() {

           String name = null;
           public void onClick(View v) {
               try {
                 EditText et = (EditText) findViewById(R.id.EditText01);
                 String str = et.getText().toString();
                 if (name == null) {
                   name = str;
                 }
                 socket.fire(new Data(name, str));
                 et.setText("");
                 Log.d("Client", "Client sent message");
               } catch (Throwable e) {
                 tv.setText("ERROR " + e.getMessage());
                 e.printStackTrace();
                }
            }
       });

That’s all. If you want to chat between Android and a browser, you can install this HTML file, which will use atmosphere.js, the Atmosphere Framework’s client side to communicate with Node.js. You can use the Javascript WebSocket API directly as well, but with atmosphere.js, you get transports fallback support for free.

Now let’s write the same client, but this time using the Atmosphere Framework instead of Node.js. wAsync supports natively the Atmosphere protocol with fallback transports like long-polling, so we can tell wAsync to use long-polling in case the server doesn’t support websockets. For the server, instead of Node.js, we use the NettoSphere server.

The full code can be browsed here. The two differences are:

AtmosphereClient client =
   ClientFactory.getDefault().newClient(AtmosphereClient.class);

Here we create a specialized client, which will allow use to set some Atmosphere’s protocol specific property:

RequestBuilder request = client.newRequestBuilder()
   .method(Request.METHOD.GET)
   .uri(serverIpAddress)
   .trackMessageLength(true)
      [...]
   .transport(Request.TRANSPORT.WEBSOCKET)
   .transport(Request.TRANSPORT.LONG_POLLING);

The track message lenght feature make sure two JSON messages are never delivered to the function as a single message. To supports that with Node.js, we would have needed to install Socket.IO.

What’s next with wAsync? Add support for both Socket.IO and SocksJs protocol. Want to contribute? Fork us! For more information, ping me on Twitter or follow the Atmosphere Framework!

The wAsync developpement is powered by Async-IO.org, the company behind the Atmosphere Framework!

Take a look at Building Games using WebSocket, Server-Side Events and Long-Polling with the Netty Framework and Atmosphere, from the Async-IO.org team.

All of that work was done under several companies umbrella: Sun Microsystems, Ning, Sonatype and more recently Reverb. In all cases, I was allowed to work on those frameworks because they were used internally. For example, I’ve spent the last 18 months working at Reverb, significantly improving the Atmosphere Framework and even deploying it on Wall Street Journal, one of the biggest/live deployment I’ve ever made so far supporting WebSockets and Internet Explorer 6/7/8/9 at the same time. Phew!!!

During that 18 months at Reverb, I was finally able to release the 1.0 version of Atmosphere, which was a major accomplishment for me after working on the framework for the last 4 years. Then, the popularity of the framework sky rocked and everything jumped: download are almost 200% up every months, Github “watcher” doubled, mailing list is now close to 500 users, Twitter account reached 1100 followers and emails traffic are stable months after months. More important, in December I’ve started getting requests for professionnal services with Atmosphere, contract support, etc.

So, with the current Atmosphere success, I’ve had a hard decision to take: work at Reverb and let the Atmosphere’s community grow and “leave by itself”,  or resign from Reverb and try to build a company to support Atmosphere. Last January was difficult for me as I was constantly under fire by having to work on Atmosphere and at the same time help scaling the Reverb’s platform. Not a nice situation!

So, I’ve decided to tale a leave on absent from Reverb and try to build my own company around the Atmosphere Framework, called Async-IO.org. The website is live and the traffic is already passing 500 unique hit per day, so this is positive…but, It’s a major challenge for the following reasons:

• I cannot stay unemployed for several months. Hence I allocate myself 5 months to succeed with my crazy decision. I know it’s a very  small amount of time, but that’s all I can allocate.
• I’m based in Canada (in Québec), not in California or in a city like London or Berlin where I can have funds from Venture Capitals Companies. Montréal is unfortunately stuck in the early 200o where IBM and Microsoft rules the city, so no luck on that side as well!
• I want to try to do everything all by myself, from designing the website (thanks Github) and learn from existing open source projects that succeeded. I’m event sketching the official logo this week!
• I don’t want to spend large amount of money on infrastructure. Instead, I would like to reward people that have been working with me in Atmosphere as soon as I can. But I need a solid foundation before I can do that.
• I don’t have a large corporation financing my work. Since Atmosphere compete with framework like Socket.IO/Node.js, Vert.x and Play 2 (they are all powered by a big company), it’s a real challenge!
• Atmosphere work with all browsers, mobile browsers and Java WebServers, and from version released sometimes 5 years ago. The testing matrix here is really complex and right now I  strongly need to rely on the community to help, and in return I’ve adopted a short release cycle to make every body happy. That way too much time consuming so I need to improve that situation.
• Continue being active with the community like I have been for the last 4 years!

Now, after reading this you can call me crazy and I would fully understand! Interested to help? Pretty simple:

• Follow Async-IO.org, Atmosphere or myself on Twitter
• Follow/Watch/Start the Atmosphere Framework on Github. The project is on page 2 for most popular framework, and being on the first would help.
• If you are using Atmosphere, consider buying an annual subscription or get in house training with Async-IO.org.

So, for the next 5 months I will use this blog to share my experience with this colossal adventure! Stay tuned!

DI is just for testing (?)

The entire arguments, both for and against, are based on the assumption that because there are different ways to test things in Ruby, DI is awkward to use; therefore you should never use it. This is shown through the framing of all of the above posts as though DI only exists to answer the question of, "okay, now how do I test it?".

Unfortunately this premise is faulty and wrong, and thus the arguments fall flat on both sides. Yes, we don’t need to use DI to test in Ruby, but that does not mean DI has no value in the language. Quite simply, DHH is attempting to throw out the baby with the bath water on this issue.

DI is not a magical concept

Let’s start from the top with a simple revisiting of what dependency injection is:

DI is a way to expose relationships made through composition in order to influence (or control) those composed objects. Put simply, DI is basically when you pass objects into a constructor or method instead of creating them inside the method itself. That’s it. It sounds insanely simple, right? It is.

We actually do this all the time, and it’s not special. DI is just a fancy name for this concept when the compositions we influence happen to be special, like databases, or dates, or, you know, heavy external components that need to be stubbed out in testing. From the start, this does sound very useful for testing in statically typed languages where you cannot hook into any method that’s already been defined. However, that behaviour you immediately think about is a side-effect of DI, not its inherent purpose.

DI is for reuse

The real purpose of DI is that of reuse. DI allows us to take a component’s logic and substitute different compositions so that we can reuse the logic in different scenarios. Some of those compositions we might substitute might be mocks or stubs, but they might also be used in production code too, like:

1. A different UI backend. A good example of this is testing frameworks (not tests themselves) which might have different test runners for different UIs—a console runner, a JUnit style XML output runner, a fancy GUI runner, and more. Because this is typically abstracted inside of a generic global "run my tests" controller class/method, there has to be a good way to inject our new backend into this controller (if we’re looking at this from MVC). Yes, we could explicitly pick our backend and run that directly with our tests, but now we’ve inverted our API; the UI is no longer a detail of the controller, it’s the primary interface with which we access the program logic. In other words, instead of asking the Controller setup the View, we’re asking the View to setup the Controller.
2. A different transport backend. Tim Bray talks about the example of substituting Net::HTTP with FakeWeb in his testing example, but this may as well be an example of a real world application and not just a testing scenario. Maybe we want to take our monolithic Request class and instead run our requests over some HTTP-like protocol over UDP, or maybe even a serial port. If calling Net::HTTP.new is buried somewhere in an internal #send method, there’s not much we can do without DI. We can monkeypatch or redefine methods directly on an instance, but these are not thread-safe or elegant solutions to the problem. DI is the elegant solution, here.
3. A different database? I’m not sure why this example was lost on DHH, of all people, as DI is being used inside of his own framework and he completely forgot about it. Arguably, I’m no expert on the implementation of ActiveRecord, but conceptually speaking, almost any database-agnostic library like AR handles selecting the DB backend through DI. I’ve roughly tracked this behaviour down to AR’s #establish_connection method, which allows you to inject, via arguments, values that will influence which backend database drivers are being used in AR. That is basically DI, right there (it’s not a great form of DI, since you can only influence, not fully control, the composition, but I’m sure deeper down in the framework is a method that exposes more control).

TL;DR: you can’t hate DI because you already use it

If you use ActiveRecord and have a bunch of settings in your database.yml file, you are making use of dependency injection. You are influencing a composition through arguments passed into the connection manager. You probably use this all over your own code, too—whenever you defer the creation of an object and allow that object to be overridden by something passed in via arguments. These are all cases of DI.

In other words: DI is not magical. Don’t hate it.

But…

If you’re in Ruby-land, or any dynamic language that lets you swap out method implementations, you shouldn’t use DI solely for testability of your APIs. And that, my friends, is really the only real point about DI being made in these posts.

To qualify the argument a little more: you should be thinking about using DI in your APIs if there are interesting compositions you want to be able to influence or substitute out—specifically large components, like databases, UIs, etc.

Here’s how it looks if you toggle it on and off:

In my dotfiles setup, it’s turned on by default.

In most cases, whitespace characters do not matter. But sometimes they do. For example, Go and Python are picky about spaces. Also, a mix of spaces and tabs will look different for different people and their editors — because nobody can agree how many spaces at tab character is (4 or 8?!). This is something I’ve talked about before.

This week, for the second time in a couple of weeks, I lost a bunch of time on another whitespace: character 160, the non-breaking space. To make a long story short, it was a copy and paste from Skype, and the JSON parser I was using choked on it.

It seems spacehi hasn’t been touched in (almost exactly) 10 years. I found its mirror on github. But it didn’t handle non-breaking spaces, so I forked it.

I’m willing to maintain it, it a very simple Vim plugin. Bug reports and improvements are welcomed.