git manual. I’m sharing the details because even after using git for 15 years, I was mildly surprised by the behavior. Luckily, it does make sense now that I understand the underlying reasons.
Anyone using git for version control is likely familiar with the .gitignore file where you can specify which file name patterns git will ignore by default when you perform various actions, such as git status, git diff and git add. It’s very useful to ignore generated files and temporary files. You can add a line that starts with a ! to make it NOT ignore files with that pattern. Patterns are evaluated top to bottom, and the last pattern that matches a file will determine if it’s ignored or not.
We wanted to add a simple .gitkeep file to preserve the tmp/pids directory in our project on all machines that checked out the project. This arguably trivial operation did not work and took me a while to figure out why it did not work.
1 2 3 4 5 | |
So why is the file we just told git we want to NOT ignore still getting ignored? It’s because our .gitignore already contained this handy line that ignored all temporary files (because they have no place being checked in!)
1
| |
This line makes git ignores all files and directories inside tmp. It’s also the direct reason why our earlier change did not give the result we want. After some reading of the manual, I found this relevant bit (emphasis mine):
An optional prefix “!” which negates the pattern; any matching file excluded by a previous pattern will become included again. It is not possible to re-include a file if a parent directory of that file is excluded. Git doesn’t list excluded directories for performance reasons, so any patterns on contained files have no effect, no matter where they are defined.
It really makes sense from a performance perspective to not recurse into children of ignored directories to see if they happen to be not ignored.
TL;DR: you need to re-include an ignored parent directory if you want to customize rules for its contents.
Let’s update .gitignore using our new knowledge. The git diff after my changes:
1 2 3 4 | |
The order of operations here is:
/tmp/* and all its children/tmp/pids back.pid files in tmp/pids (because those are the only ones generated there).gitkeepAlternatively, you can use git add --force tmp/pids/.gitkeep to add it while ignoring .gitignore rules. Arguably that’s faster than checking the manual and thinking about it, but it also prevents you from learning why it failed to work in the first place.
This worked for me ™, so I hope it helps you as well. If you run into issues, feel free to reach out to me via Twitter or email. I’m an experienced Ruby software developer with a focus on back-end systems and an obsession with code quality. I even have a few Ruby on Rails maintenance services that I offer. If you still need to upgrade to Rails 6, then grab the handy free checklist or reach out to have me do it for you.
]]>This post describes some of the obstacles I ran into getting proper JS-supported system tests working in my up-to-date Rails app, Infinity Feed.
It used to be complex to get Capybara working with a JS-enabled headless browser for your integration tests. This has gotten significantly easier with Rails 5.x and got another boost with 6.x.
If your knowledge about Rails frontend testing still stems from the Rails v4 or v5 era, you might be pleasantly surprised by how easy it can be now.
For this blog post, the relevant bits of my stack are:
You’ve got the usual suspects here that test Rails with RSpec. I use unit tests for models and other plain old Ruby objects (POROs), there are controller tests for specific interactions and there was a feature test intended to test my frontend interactions. This is done differently in the modern era by using system tests.
I use Faraday for HTTP calls, so VCR + Webmock is part of my test stack to intercept & record HTTP calls so they can be replayed without hitting the network during my tests. This makes tests more consistent and faster. Win-win!
I had a feature test written with out-of-the-box Capybara which appeared to work just fine, until I wanted to test that Hotwire Turbo was doing its magic to make automatic JS-powered HTTP calls to replace my HTML with new HTML. The JS did not get executed in my tests!
So, I had to figure out how to enable JS. Remembering how much of an ordeal this used to be, and how often the best practices changed, I started searching the internet for how it is done in 2021.
Since Rails 5.0/5.1 there are system tests, which are similar to the old feature tests we had, but now Rails handles all of the overhead you used to have to do yourself. You can stop futzing with DatabaseCleaner and configuring Puma to run in-process. It’s all taken care of by Rails now.
Since Rails 6, integration with browsers for testing happens via the webdrivers project, which handles downloading and updating the browser for you. It just works! Just beware of unexpected VCR interactions (see below).
It’s really easy:
spec/features/ to spec/system/type: :feature to type: :system if needed for these filesAdd this:
1
| |
To the top of your system test files to pick a driver that supports JS. You can use this to configure different browsers and screen sizes.
And now you have working system tests!
There’s a few extra things left to configure if you need them.
If you use Devise to handle your authentication, you should register its integration test helpers to be used in feature and system tests:
1 2 3 4 5 6 7 | |
This enables the very useful sign_in helper function, so your tests can focus on testing actual features instead of always having to simulate a login.
VCR helps you intercept HTTP calls during tests, as I mentioned earlier. The thing is that Webdrivers automatically checks if the latest version of the browser is installed and downloads it if needed. As you can imagine, this did not play well with VCR.
My solution is to force the webdriver to check for an update before I configure VCR. VCR also needs to be told not to interfere with system test calls.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | |
This worked for me ™, so I hope it helps you as well. If you run into issues, feel free to reach out to me via Twitter or email. Besides being close to launching a smart RSS reader over at Infinity Feed, I’m a freelance Ruby software developer with a focus on back-end systems and an obsession with code quality.
]]>When you do launch, everything changes. Suddenly it’s not only yours anymore, it’s also theirs. Your users (who preferably turn into your customers) will start to use it. They develop attachments to what is there. They get frustrated with what they don’t like. They will, collectively, present you with conflicting requests to make changes.
The worst outcome is that nobody cares, and all that’s left is you and your disappointment.
Not launching, just yet, in order to fix that one bug, to tweak that one thing… it’s so tempting! You get to keep it for yourself, just for a little while longer.
The problem is that this mindset is similar to analysis paralysis: you can get stuck in it without ever launching. There is always something more you can do. Something to add, something to fix. It will never be perfect, but you can trick yourself into trying to achieve it. In reality, you’re just avoiding the potential for disappointment that could come from an unsuccessful launch.
I’ve read somewhere that if you are not at least a little ashamed of what you have launched, you have waited too long. It always sounded wise and useful, but now that I’m getting closer to launching Infinity Feed, my mind is spinning and grasping at all kinds of excuses for why I should wait just a little longer.
How I’ve been dealing with it? I’ve started talking to folks about what I’m building. I’ve hinted yesterday in the footer of my blog post that it will go live soon-ish. I’ve put up a marketing placeholder page (which read okay at the time, but now I just want to rip it off and replace it with my new one). Basically: I’ve started building momentum that should result in me launching the thing I’ve been announcing.
Now if you’ll excuse me, I have a few more things to do before I hit the “deploy” button for real. Launching really is scary, but this post pushed me another step closer to doing it.
Curious? Want to say something? Feel free to reach out to me via Twitter or email. Besides being close to launching a smart RSS reader over at Infinity Feed, I’m a freelance Ruby software developer with a focus on back-end systems and an obsession with code quality. I’ve also decided to look at cryptocurrencies again, so expect me to mention that again in the future.
]]>Addition on 8 September 2022: note that I never published a follow-up for this due to how obvious it is that a lot of crypto is a scam, and how ethically dubious a lot of practices are. As a layperson, assume you will get fleeced when you enter the ecosystem. The amount of tinfoil and conspiracy theories in this scene promote Fear/Uncertainty/Doubt (FUD) which aims at getting new people to add money to the crypto ecosystem in order to prop up prices for folks already invested. Sound familiar? Yep, it’s a huge pyramid scheme! See also: Line goes up, a good video that breaks down the ecosystem and issues.
Two of my friendly colleagues disappeared into a crypto-shaped black hole over the last three months. They re-emerged, talking with great enthusiasm about it. They have doubled their seed money, and talk about all the potential for the technologies in the space. I must admit, they have broken down some of my long held skepticism, so it’s time to look at it again myself and see if their hype is real.
For context: I’m coming at this as a crypto skeptic software developer. I looked at Bitcoin when it was very new (5+ years ago?) and did not get it. It seemed like speculation on a virtual coin without intrinsic value or purpose. It smelled a lot like a pyramid scheme. Association with black market trade did not help. In the last few years the negative environmental impact of Bitcoin has not made me a fan either.
I’ve looked into speculative trade before. I’ve had a good look at ForEx in 2012-2013, doing paper trading, lots of reading, and some programmatic stuff, before deciding the speculative stuff was not really for me. Similarly I’ve read up on stock market investment, including Benjamin Graham’s classic book.
The crypto world has developed a lot in the last few years. It’s not just Bitcoin anymore, so let’s dive back in to see what’s up with it now.
In case it’s not obvious: I’m not an expert, this is not financial advise, etc. Feel free to point me in the direction of more solid info where I can learn.
It’s good to clarify what your goals are when you start with something. It helps you direct your search for information, and might help a bit against temptations.
My high level goals:
First goal: I want to understand how things work in crypto in 2021. Below is my understanding so far. It will probably be wrong on a lot of accounts, but that’s why it’s nice to write down so I can correct it later.
Which pieces are there on the market? What types of coins/tokens/contracts/etc are there? My very limited understanding right now is:
I know Bitcoin has miners who burn CPU/GPU/FPGA time to calculate hashes in order to process transactions and create new bitcoins in the process. Transactions are relatively slow and expensive (a Google search led me to this which claims 1MB per 10 minutes of transaction bandwidth, currently you pay 3-10 USD per transaction depending on how much of a hurry you are in (10-60 min delay). That’s relatively little if if your trade is for thousands of dollars, but a lot for a cup of coffee.
Look into how/why Bitcoin has a 4-year periodicity in its prices. There’s something about regularly scheduled restrictions in how many new coins are mined, so total new supply decreases and as such value doubles. This makes huge waves and affects all coins, apparently. My friends keep talking about how the next few months will have huge gains and then it’ll dry up. Why?
How do the other coin types do this? There are Proof of Stake coins (vs Bitcoin’s Proof of Work) where there’s a finite supply of coins which are setup via smart contracts, have a buy-in ahead of time and then freely trade once they go live. There’s a thing about burning coins as part of the protocol so they get more valuable over time. There’s also a thing about staking claims, where you freeze your coins temporarily in exchange for a percentage of the transaction fees? Time periods here are daily or weekly, so it’s all incredibly rapid compared to the yearly rates traditional banks pay.
Analogy to the stock market, where each stock represents a piece of ownership of a company. Shares earn dividends for you merely holding on to them. There’s also profit/loss based on the resale value of the share. Companies have revenue, profit, equity, capital reserves, etc. A lot of metrics you can contrast to the share performance.
Based on this you can choose to go long on a share (i.e. you buy it and just hold on to it), avoid it, or do risky things such as shorting (that is, you borrow a share to sell it now and promise to buy it back later; if it works you pocket the profit, if you fail your loss has no hard limit other than the share price). While at it: don’t every borrow money to invest. Only play with what you can afford to lose.
Do things like stop/loss orders exist in crypto? What are the new options that have no analogy in the classic markets?
How can you use it, right now? Better question: how is crypt used right now? How mature are the products? What are common pitfalls, and how have those been mitigated?
How user friendly / approachable is crypto? How close are we to my parents (who are tech late adopters) using this for something useful? How close are we to them being able to explain it to someone else?
My friends mentioned that there’s a lot of rough edges and opportunity for quality of life tools. What would be useful? Is there a reason these don’t exist yet? Is there an opportunity to monetize some of this?
To manage risk you have to be aware of risks. What are the obvious ones?
Crypto exchanges / brokers can run away with your coins, and apparently that’s just a risk you have to accept. Crypto is dog-eat-dog world, like the game Eve Online. Don’t trade with what you can’t afford to lose. How can you identify trustworthy ones, besides just going with what everyone is doing? If you can’t do this, how can you identify the (obviously) untrustworthy ones?
Wallets, ledgers, accounts. Somewhere to store your coins/tokens/etc. In my mind it’s close to how your SSH keys also control your bank account. Public/private key encryption. Is this model accurate? Be cautious and careful. What are good practices for keeping things safe?
Who are the big fishes in each pond, and what are their interests? When they move, they can generate huge waves. Stop/loss orders which normally might make sense could get triggered hard due to big fish movements. My friends mention that folks trading on margin tend to have their contracts enforced at the end of the day/week/month/quarter. The quarterly ones were unexpected to them due to the volatility they caused. Knowing what/when happens means you can anticipate it.
There is the obvious loss of value, loss of trade volume/liquidity, risk of the huge fishes in the market making splashes, problems with underlying things (i.e. if one coin is connected with others, they have a relation).
Writing this down helped me get a picture of what I think I know. Plenty of questions and uncertainty, so next is filling in the gaps and double checking myself, then checking with my friends if my understanding matches theirs.
I’m going to work the high-level goals from top to bottom, I think.
Feel free to reach out to me via Twitter or email. Besides my foray into cryptocurrencies, I’m a freelance Ruby software developer with a focus on back-end systems and an obsession with code quality. I’m also developing a smart RSS reader over at Infinity Feed (beta should go live soon-ish).
]]>This weekend I experimented with streaming my own Diablo III gameplay on my Twitch TV Channel. This was my first time recording and broadcasting. It was an interesting experience.
Streaming a game is definitely an different way to play a game. Having the microphone on and your gameplay recorded feels a little bit like giving a presentation. For me, this meant that I felt very nervous, awkward and self-conscious at first. It made me lose my train of thought a couple of times, etc. After a while it got more comfortable, so the side-effects of being nervous went away a little.
Another thing to get used to is multi-tasking between the game and commentating on what you’re doing.
After streaming with zero watchers, I wanted to watch my recordings to see how I did. Twitch kept pausing the video to buffer every 30 seconds, despite my bandwidth being more than sufficient. I figured it was a limitation of Twitch. Luckily, Twitch makes it easy to export streams to my Youtube channel, which does not have issues with bandwidth.
For my setup, I started with Nvidia Shadowplay (which comes as part of the video card drivers) to record my gameplay, downscale the footage from my 1440p screen to a 720p video and stream it straight to Twitch TV. I recorded my voice-over with my gaming headset while playing the game.
Shadowplay was a very easy and low-barrier way to get started. The default settings are very conservative, so the result looked horrible to my very spoiled eyes. I had to increase the output resolution and bitrate to make it look a little bit better, but the Twitch stream output was nothing like watching Youtube videos in the same 720p resolution.
Looking for a better way to record and broadcast my video, I followed Twitch’s recommendation of Open Broadcaster Software (OBS). They have a good guide on how to configure it.
The visual quality went up, but I kept pushing too close against the maximum bitrate of 3500 kbit in order to make the stream look as good as possible. In fact, 3500 kbit is not enough to make the video look decent to me. Twitch starts getting buffering issues above 3000 kbit, but looked noticeably worse when I lowered my bitrate that far.
So: I was not happy with the quality of my stream on Twitch, yet my plan was to also get the footage on Youtube for later watching. Having zero subscribers to your (new) streaming channel means nobody sees it anyway. So I decided to skip Twitch and upload straight to Youtube. Uploading directly to Youtube allows me to record higher content at a (much) higher bitrate. Heck, I could upload a 1440p video in a crazy high bandwidth.
To determine what bitrate would be possible, what would be good and what would be a good baseline, I recorded a couple of short clips at different bitrates.
The Youtube video processing process is also interesting to look at. You upload the video, then they start processing it. Once this is done, the lowest quality version of the video (360p) is available. Invisibly in the background they keep processing increasingly higher quality versions (480p, 720p, 1080p and finally 1440p). Once the video list shows (HD) after the title’s name, the 720p version is available. This background process will take a couple of hours, depending on the length of the video.
I settled on uploading videos at night as unlisted, then making them public the next morning.
Oh, yes. I was recording playing a game.
I was playing Diablo III: Reaper of Souls, which has been out for over a year now. Last week Blizzard released the 2.1.2 patch in preparation for the Season being reset in early February.
At the launch of Season 1, I briefly tried it. At the time there were ways to exploit the game to get a maximum level character in two hours. For me this took the fun out of it. How can you compete on a ladder of sorts if you’re up against cheaters?
Blizzard hotfixed the ways to cheat out reasonably soon, but I had given up on it at that point. Now that a new season is coming up and it looks like people will actually be racing to reach the top, I’m interested in participating again.
So to see what my chances would be, I created a brand new hardcore character in Season 1. Hardcore characters stay dead once they get killed in game. You only live once. I like the added challenge of keeping a character alive.
The first night I played 6 hours and reached level 50 by playing the game in adventure mode. The second night I played about 3 hours to reach level 60. The third night I played another 3 or so hours to reach level 70 (the maximum) and reach my goal. Total time taken: 12.5 hours.
The key is to not push the difficulty level to extreme from the get-go, and to play in Adventure mode instead of the campaign. You have a more focused leveling experience and it’s nice not having to go through the low level parts of the campaign yet again. The gameplay is dumbed down quite a bit because this part serves as a tutorial of sorts. In adventure mode there is no tutorial, because you only unlock it after completing the campaign once.
]]>http://devblog.avdi.org/2014/09/17/boolean-externalities/
In his post he asks the question: if a predicate returns false, why does it do so? If you chain a lot of predicates, it’s hard to figure out why you get the answer you get.
Consider this example. It implements simple chained predicate logic to determine if the object is scary.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | |
Following the chain of logic, something is scary if it’s either a ghost or a zombie. They are both not alive, but a ghost has regrets and a zombie is hungry for brains. This is the code as I would probably write it for a production app. It’s simple and very easy to read.
The downside is that if you want to know why something is scary, you have to go and read the code. You can not ask the object why it arrived at its conclusion.
The following is a logical next step in the evolution of the code: I have modified the code so it can explain why a predicate returns true or false, though there is a tremendous “cost” in length and legibility.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | |
Yes, that’s a lot more code. All composite predicates have a “why_[predicate]” and a “why_not_[predicate]” version. Now you can ask if something is scary and why (or why not).
There are a few problems with this approach:
scary?, where you would expect it.why_scary and why_not_scary. Don’t Repeat Yourself, or you will get logic bugs.Let’s see if we can make the code legible again, while preserving the functionality of “why” and “why not”.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | |
So far, so good. The code is very legibile, but there is a mysterious superclass EitherAnd.
Before we look at how it works, let’s look at what it allows us to do:
1 2 3 4 5 6 7 8 9 10 11 12 | |
For each predicate that uses either or all we can ask why or why not it’s true and the response is a chain of predicate checks.
If you want to make your code legible, there usually has to be some dirty plumbing code. In this example we have hidden this in a superclass, but it could have been a module as well without too much effort.
In order to keep the code easier to read, I have chosen to not extract duplicate logic into helper methods.
This class implements two methods: either and all.
Both methods have the same structure:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 | |
It is possible to provide traceability for why a boolean returns its value with less than 200 lines of Ruby code and minor changes to your own code.
Despite the obvious edge cases and limitations, it’s nice to know there is a potential solution to the problem of not knowing why a method returns true or false.
Blogspot served me well at first, but after a while I got tired of fighting with it. Having to write your posts in raw HTML to make them look decent is so 20th century. Because Octopress is Jekyll with all kinds of nice stuff on top, I can now write my posts in Markdown and be happy again. Deployment is easy as well, since Octopress just generates static HTML and uses scp to put it on my server. I have to admit, it feels good to have complete control over my own blog on my own domain.
The last blog post I imported was written a long time ago, in 2009, while I was still working for yoMedia. Oh boy how things have changed since then.
Since 1 September 2009 I am the proud owner of my one-man company, Narnach. My original plan was to split my time 50/50 between building my own projects and freelancing as a software developer. It took a month for the first freelance opportunity to present itself. While working with that client, another opportunity came along and I’ve been busy non-stop ever since. In all the freelance fun I completely forgot about doing my own projects, though. Funny how that goes.
2010 was a good year. For most of my projects, I worked together with Gerard de Brieder of Govannon as a two-men freelance army and that collaboration worked out well. In 2011, Gerard got himself a physical office that we moved into. Soon, Gokhan Arli of Sylow joined us as third man on the team.
2011 has been my busiest year yet. It went so well that for the first half year I averaged 60 hour weeks just to be able to keep up with all the work. The second half of the year I finally learned to say “NO” to new opportunities, so things quieted down a bit to “regular” 35 hour weeks. Still, I worked way more than I should have and in December I was displaying symptoms of what, according to Wikipedia, may be a Burnout. So I went on vacation on 20 December and I took almost three weeks off to relax and recover. It seems to have helped!
For 2012 I have decided to change two things.
First, I’m slowing down my freelance activities. This means taking on less new clients and letting some existing clients go. The result should be less stress and more energy to work for the clients I am keeping.
Second, I’m revisiting my original plan of developing my own project. Freelancing is fun, but you are always helping other people grow their company. I finally have an idea that I would like to develop, so I figure now is the best time to start. The change from my original plan is that this will strictly be an evening/weekend thing. “They” say it takes at least a year or two to achieve “overnight success”, so I figure I’d not quit my day job as a freelancer just yet.
]]>People like to communicate with each other. Centuries ago we wrote letters and sent them with the merchants, hoping they would arive. The telegraph was a revolution: we could send a message faster, cheaper and with more certainty of delivery. The telephone was even more revolutionary: direct communication over a long distance.
In the age of the internet, change happens even faster. E-mail has been around for “ages”, just like various forms of chat services have come and gone.
Broadcasting has gone through a similar change. It started with spoken announcements; proclamations from the king. At some point there were pamphlets and posters. Books can be seen as a way to broadcast a message. Newspapers are a periodic form of broadcast. Radio enabled long-distance audio broadcasting without the cost of creating a physical carrier for the message. Television added moving images to radio.
Now we have the internet, where we go through similar stages.
Static, web 1.0, websites are pamplets re-invented. Old concepts in an electronic shape. Ebooks are electronic books. Newspapers try to put their content on their own websites, updating them daily to bring their news to the masses. E-mail newsletters just scream “newspaper” to me. Radio can be found as streaming audio.
As we have become more familiar with the internet and with an increased access through broadband, cable and fiber, we have started to innovate with the new medium. RSS changed the direction of broadcast from push to pull. youTube may have started as a way to share existing videos, but it has since grown to a place where anyone can make themselves heard. That is many to many communication.
The internet made interactivity a lot easier than in the off-line world. Web forums and UseNet allow groups of users to interact with each other through written messages. Blogs allow everyone to have their own newspaper column. Instead of an opinions page in the newspaper that is the internet, every writer gets their own column and they all respond and refer to each other’s writing.
Twitter is the latest thing. It is a hybrid between instant messaging, e-mail and RSS feeds. People say it does not scale, yet the Twitter engineers keep making it better and more and more people are able to use it. Is there a limit to which it can scale? Is its centralized server model not going to be an important limitation on both scale and freedom later on? When there are six billion people using one and the same service, relying on it for an important part of their daily communications, how much can you trust on one company to take care of it?
Would it not be better to turn it into a distributed service? It has sucessfully scaled e-mail, Jabber/XMPP, the telephone network and internet itself. E-mail, as a world-wide service, has never gone down, even if individual servers might go down from time to time.
Traditional media has its problems. Paper flyers, mail-delivered advertising and commercials on the radio and television are a few examples. The relatively high price of print media or traditional broadcasting limits the amount of these forms of advertising. This makes it somewhat bearable. In contrast, spam via e-mail, blog comments, web forum posts and instant messages have no such limitations. It’s virtually free to broadcast your message to a million people, so it happens a lot and people really don’t like it.
E-mail spam can happen because there is a near-zero cost or risk for the sender. The same goes for other on-line communication. It can be interesting to consider the friend-of-a-friend model, such as is seen on Linked In and other social networks?
To send a message to someone, the whole connection chain between sender and receiver must be known, no matter how long it is. If someone spams, it means there is a chain of real people connecting them to you. This means the sender is traceable instead of anonymous. If you flag a message as spam, the whole chain is notified that they were part of a spam chain. This means people can choose to ban the spammer from using them as a connection in sending a message. It also means you can identify people who act as a gateway for spammers to send messages to other people.
The nice thing about treating communication as a social network activity is that it makes people more aware that they are dealing with people. By taking anonymity out of the equation, it lowers the tendency of people to act like a Total Fuckwad when they think nobody is watching them.
Are there other ways to look at communication, to turn it upside-down and re-investigate how it works? What is going to be the next Twitter?
]]>RubyGems has been around for ages and has made it relatively easy to distribute Ruby code. Not everyone uses it, though. Some prefer to use the Debian package manager, or whatever their OS provides, instead. This is very useful if a gem has external dependencies, but it is not as portable as RubyGems.
RIP was recently released (well it is only version 0.0.1, but still) as something to use complementary to RubyGems. It does not allow relative version requirements (<, <=, >=, >) for dependencies, only exact version requirements. It borrows the concept of virtual environments from the Python world. A different approach to package management out in the wild means people will gain new insights. What can we learn here? Where lies the right balance between having rigid, version specific, dependencies and open-ended dependencies?
Thinking along the dependency management line, why do we require exact versions or do we put an upper limit on accepted versions? The only reason I can think of is incompatibilities introduced in later versions, but is it right at all to introduce backward incompatibilities in your API? Can’t we learn something from functional programming here?
In FP, pure functions don’t have side effects. One of the implications is that the data they receive does not get altered. You don’t add a new item to an existing array; you return a new array with the new item appended to the existing array. Because of this, there is no problem when you have a multi-threaded program: there is no risk that two threads will try to modify a shared resource at the same time.
This means you don’t need mutexes to lock an object to one thread while it manipulates the object. No mutexes means no deadlocks or other headaches associated with threading.
What was I talking about? Ah yes, dependencies and how they relate to functional programming. Explicit version dependencies can be seen as mutexes: only one version is allowed to be used at once. Two versions of a library can not be loaded at the same time. This is good if the two versions are incompatible. It is bad if the newer version only adds new functionality to the library.
What if you would build your library in a way that resembles the pure functions of functional programming? No side effects in this case means there are no nasty surprises when upgrading. If your program works with version 1 of the library, it will work without changes with version 1000. Existing functionality is immutable.
To make this work, new versions should only introduce new behaviour, they can not change old behaviour. I think making bugfixes would be ok, but performance enhancements are not, as it might be that you introduce negative side effects in some edge cases, thereby breaking someone’s app. Maybe fixing bugs will actually break someone’s app if they depended on buggy behaviour. Hmmm….
This makes dependency management rather easy. You set a minimum version requirement for the libraries you use and you can just upgrade the libraries to newer versions as they become available. New applications that use new features can co-exist with old applications that use old features from the same library.
Under this model, if you plan to radically re-architect a project, you could fork it and release it under a new name. Rails v1., Rails2 v1, Rails3 v1. A downside is that forks can have a large shared codebase, but there will no longer be conflicts between versions of one project.
Has anyone ever explored the possibilities of library development along these lines? Did it work or were there problems that I have overlooked? What good features of the ‘current’ systems would you lose?
]]>After only a couple of us replied to Sander’s e-mail, we knew none of the heroes usual people would step forward to organize this. On friday, Julio Javier Cichelli (@monsieur_rock) sent me a direct message on Twitter about my thoughts on how the unconference should be organized. From there, we discussed how to get it organized, how to get speakers and what we were going to present. In short, we stepped forward to organize the side-event.
Next I tweeted to ask for presenters. The message got re-tweeted a number of times and almost within minutes there is feedback from multiple people willing to do a presentation. At the end of the day we have 6 people willing to speak. Using Twitter to organize something is a really quick and powerful way to do it.
As more and more people indicate they are willing to speak (thank you all!) the focus moves from finding speakers to handling the details of making it all work. At what time do we start? How many hours do we have? Is there wifi? Is there a beamer? The further you go, the more you discover there are things you should find out or arrange.
Next week we need to find a sponsor for the venue and we need to start thinking about the things that need to be done on the event day itself. We also need to confirm time and location with all presenters and announce the side-event.
The side-event is an unconference. I have never been to one, so I can only go by what is on the internet. A characteristic of unconferences is that there is no fixed agenda. There are no time slots. It is not about one person being an expert and bestowing wisdom upon the attendees, but about the attendees sharing wisdom with each other. I like that.
Everybody knows something other people can benefit from, so the more opportunities there are for everyone to contribute, the more everyone will learn. Any one of the attendees can decide on the spot they want to talk about something, show code or sing a song. I hope people will do this.
The Devnology meetings have impressed upon me the importance of interactivity at a gathering of people, so I hope we can give the unconference an interactive twist.
After a presenter is done speaking, we’ll try to get a group discussion started on the topic. Once the discussion starts to fade, or starts to run in circles, we can ask for the next speaker to get on stage and introduce a next topic.
After the last speaker, we can try to spark group discussions by encouraging people to approach the speakers and ask them questions. This, in turn, can create a number of smaller discussions, with the speakers being the center of interactivity. It’s a great way to get to know new people.
Between 6 more or less confirmed speakers, group discussions, short breaks and (I hope) spontaneous speakers, it looks like we will actually fill up the 5 hours we have available to us.
Unconferences tend to have a theme, this is so people can prepare themselves and to have some form of coherence between talks. This is trickier, as I did not really think about this until now.
Here is a list of topics that people have expressed they want to talk about:
If I do a bit of creative extrapolating, one topic that can be extracted from this is “The Future of Web Development (using Ruby)”. Let me explain by briefly looking at each topic:
The Future is an interesting topic that can be applied in a lot of ways. Here’s a number of ideas for presentations:
With VPSes and cloud computing becoming available everywhere, is there still a need to own your server hardware? With services like Heroku, Github webpages and Disqus, do you still need to even know how to install Ruby or how to configure Apache?
Even if you don’t use these services, by using tools like Capistrano, Ubuntu Machine, Deprec or Rudy, you can still simplify deployment and server management. Does simplifying these things bring new opportunities? What does a sysadmin do with the time saved by these tools and services? Are there new possibilities opened by freeing up sysadmin time? What are they?
I have addressed this in a separate blog post.
I have addressed this in a separate blog post.
Between asking Sander for info (via Twitter, of course), discussing things with Julio and discussing details with speakers, there is quite some communication going on. It’s exciting and a little scary at the same time.
I’m a natural introvert, so I tend to avoid communication when I can get away with it. It’s not often that I approach other people first. Taking an active role in helping to organize an unconference like this is therefore quite a bit outside of my comfort zone.
So why the heck am I doing this? One reason is because I want to see it happen. If nobody does it for you, do it yourself. The other reason is is that I want to expand my comfort zone. Someone wise once wrote: “If something does not scare you at least a little bit, it is not worth doing.” It might sound be a bit extreme, but the core idea is valuable nonetheless: a very good way to learn things is to do the things that scare you.
Helping to organize a side-event for KoC will most definitely be a great learning experience.
]]>Today I upgraded my Macbook to ruby 1.9.1 (patchlevel 129) as the main version of Ruby I use. It was not really intentional, but now that I have it, I’m kind of sticking with it. That’s the short story. There’s also a long story that involves a server, lots of logs and me not paying attention.
Earlier today I wanted to analyze 2.5GB of Rails log files. Because it is not such a good idea to do that on a live production server, I decided to use the one server that never really does anything: the backup server. It’s hidden all the way in the back of our server network, far away from the business of our webservers, so it is the perfect place to do some heavy number crunching. After sending the 2.5GB of Rails log files over with scp -C (-C stands for compress) I tried to install the request-log-analyzer gem, but there was no gem command.
A quick ruby -v resulted in bash telling me there is no ruby. My confused reaction went along the lines of: “No Ruby? What? We have a server without Ruby? How can this be?” I checked the server’s sources dir and there was actually a dusty tarball for ruby 1.8.6 sitting undisturbed. I immediately jumped on it and started to unpack it and ran ./configure.
While waiting for this to end, I thought about Ruby 1.9.1 and it’s promise of speed and the huge stack of logs I was planning to start working on. I never made it to the make && sudo make install part for Ruby 1.8.6.
After downloading the latest Ruby 1.9.1 tarball to my desktop and sending it through a chain of servers to the poor ruby-less backup server, a ./configure && make && sudo make install made it all happy again. It actually purrs if you listen close enough to your SSH session.
In the meanwhile I figured I’d upgrade my local Ruby 1.9.0p0 install to the latest patchlevel, so I also perfomed the ./configure && make && sudo make install ritual on my own machine as well. As a habit I always run a ‘-v’ check to see if the version did get installed, but I accidentally typed ruby -v instead of ruby1.9 -v and to my surprise it said:
ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-darwin9.7.0]
Oh, oh. That was not supposed to happen. That should have been Ruby 1.8.6! A check for ruby1.9 showed it was still the old Ruby 1.9.1:
ruby 1.9.1p0 (2009-01-20 revision 21700) [i386-darwin9]
Since the server was done installing as well, I jumped over to there. Unfortunately, request-log-analyzer did not like ruby 1.9:
$ request-log-analyzer log/production.log
Request-log-analyzer, by Willem van Bergen and Bart ten Brinke - version 1.1
Website: http://github.com/wvanbergen/request-log-analyzer
/usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer.rb:27:in `require': /usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer/output/fixed_width.rb:48: invalid multibyte char (US-ASCII) (SyntaxError)
/usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer/output/fixed_width.rb:48: invalid multibyte char (US-ASCII)
/usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer/output/fixed_width.rb:48: syntax error, unexpected $end, expecting '}'
... => { :horizontal_line => '━', :vertical_line => '┃', ...
... ^
from /usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer.rb:27:in `load_default_class_file'
from /usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer/output.rb:4:in `const_missing'
from /usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer/controller.rb:38:in `const_get'
from /usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/lib/request_log_analyzer/controller.rb:38:in `build'
from /usr/local/lib/ruby/gems/1.9.1/gems/request-log-analyzer-1.1.6/bin/request-log-analyzer:88:in `'
from /usr/local/bin/request-log-analyzer:19:in `load'
from /usr/local/bin/request-log-analyzer:19:in `'
So now we know Ruby 1.9 is strict on character encoding and does not like this particular version of the gem. My natural reaction is to switch to my local machine and do a github checkout of the source and build a new gem:
$ gh clone wvanbergen/request-log-analyzer
/usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/github/extensions.rb:11: warning: undefining `object_id' may cause serious problem
/usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/github.rb:149:in `module_eval': /usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/commands/commands.rb:40: syntax error, unexpected ')' (SyntaxError)
helper.tracking.sort { |(a,),(b,)| a == helper.origin ? -...
^
/usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/commands/commands.rb:40: syntax error, unexpected '|', expecting '='
...per.tracking.sort { |(a,),(b,)| a == helper.origin ? -1 : b ...
... ^
/usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/commands/commands.rb:40: syntax error, unexpected '}', expecting keyword_end
...rigin ? 1 : a.to_s <=> b.to_s }.each do |(name,user_or_url)|
... ^
from /usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/github.rb:149:in `load'
from /usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/github.rb:66:in `block in activate'
from /usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/github.rb:65:in `each'
from /usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/lib/github.rb:65:in `activate'
from /usr/local/lib/ruby/gems/1.9.1/gems/github-0.3.4/bin/gh:8:in `'
from /usr/local/bin/gh:19:in `load'
from /usr/local/bin/gh:19:in `'
Aargh! Another gem that does not play well with Ruby 1.9.
At this point I was kind of fed up with gems not working with Ruby 1.9, so I decided to use a script I wrote ages ago to do simple log crunching. It did not look as pretty as request-log-analyzer, but since it was my script, I felt it would be easiest to fix if it was wrong.
The script did needed a little tweaking to work on Ruby 1.9, but that went pretty ok. The script started crunching, and crunching, and crunching and grew to about 800M (not bad for holding about a gazillion URLs and their call times, standard deviations and more relevant numbers). It was mostly done, generating a new report to highlight different stats every minute or so. And then the Ruby process died. There was something about character encoding and UTF-8:
analyze_log.rb:232:in `split': invalid byte sequence in UTF-8 (ArgumentError)
from analyze_log.rb:232:in `block in '
from analyze_log.rb:231:in `each'
from analyze_log.rb:231:in `'
Luckily, there were enough reports done so that I could look at the numbers I wanted to see.
This experience has taught me two things:
Next up is checking/fixing my own Rubygems to guarantee they at least do work with Ruby 1.9.1p129. A better world starts with you doing your best to improve it.
]]>Devnology is a foundation that organizes meetings for software developers. Their goal is to bridge the gap between the many different communities that exist for the many programming languages and platforms that a software developer can choose to use. The background of the founders is mainly in the Windows world.
In order to learn from others, Devnology was started as a way to help developers share and learn tips and tricks, insights, tools, news and insights. In this respect, it is similar to how the Ruby User Group meetings work as a way to learn from each other. The difference is that the Ruby meetings are more heavily focused on socializing, while Devnology meetings are balanced more towards learning.
Ruby meetings tend to be fun beer nights: we gather in the pub, drink beer (or ice tea) and talk all night long about topics that can include Ruby, tools, new companies or projects, freelancing, pet projects and much, much more. Sometimes someone gets a laptop on the table and shows what they are working on, usually in the form of a live demo or by showing code. The key difference between Ruby meetings and Devnology meetings is the organized format of Devnology meetings. Devnology meetings have a limit on attendance, a pre-arranged location that is either rented or sponsored and they have a speaker or theme to provide enough fuel to keep the conversations going for the whole evening.
Getting to know new people is a chance to learn about new ways of thinking. The more different people you meet, the more you can learn. Within a single community, there tends to be an overlap in certain ways of thinking. As an example, let’s contrast Rubyists and Windows developers.
Rubyists, in my experience, tend to be young and eager to try out new things. There are quite a few people that learn on the job and don’t have a formal education. A lot of them work at smaller companies or as freelancer and they are pretty passionate about what they do. Due to Rails, a lot of work is in web development. Open Source is very important and everyone is more or less expected to have an account on Github to share their code. A lot of people have a Mac or at least run Linux on their laptop. I don’t know anyone who is serious about Windows as a developer platform for Ruby. There are some tools for Windows, but support is poor compared to Linux and Mac. Rubyists tend to be intimate with the command line and they tend to know at least a little bit of how to operate a Linux server. From front-end HTML/CSS design, through Rails code to a MySQL database design, from automated unit testing, integration testing and performance testing, a Ruby developer tends to know at least a bit about everything. Chances are, they will fulfill multiple or all of these roles on their projects.
In contrast, the handful of Windows developers I met at the last Devnology meeting tend to be older than Rubyists and appear a bit more formal. There are quite a bit of consultants working for larger companies. I actually heard people describe themselves as Software Architect and talk about corporate ladders as the most normal thing in the world. For a Rubyist to say that would be strange.
For me, these two groups are kind of opposites. Due to this, they would normally not interact much. This causes knowledge to be discovered independently and to be spread in their own communities. What is new in one group could be discovered a year later by the other group. If the two groups interact and share knowledge, the knowledge sharing might happen earlier. This is where Devnology can add a lot of value.
A good example of knowledge that is not universal is the Git distributed source code management system. For Rubyists, it is the SCM to use. Subversion is so 2007. In the Windows world, Subversion is still the way to go and Git is largely unknown. Git was developed by Linus Torvalds, of Linux kernel fame, to replace Bitkeeper after Bitkeeper stopped being free. Git has great support on Linux and Mac, but Windows support took a while to get going.
Being distributed, Git does not depend on a central repository to store all code. When you have a checkout of the code, you have the full repository. This includes all history and branches, tags, etc. Git supports a centralized workflow, where everyone pushes their changes to a single server and pulls their updates from there. Through git-svn, it is possible to even use git to interact with Subversion repositories.
The advantages of Git becomes more interesting in the Github model: everyone on the team has a public repository on Github and a private repository on their laptop. You work locally and then push your changes to your public repository. Then you pull changes from other people’s public repositories, work offline for a bit and push all changes back to your public repository. Other people can then pull in your changes and so on. This is almost an evolutionary approach to coding, where the best patches get pulled in by a lot of people and remain with the project. For closed source projects, you can use Github’s private repositories, which you can share with people of your choice. In the Ruby world, Github is becoming a social network for developers.
My personal “I am so glad I use Git instead of Subversion” moment came when the old server we use to keep a number of private repositories on had a HDD failure. We just put in a new HDD, uploaded my working copies of all repositories and got back to work. During the week we had no central server, we committed code to local branches and ran a built-in git server to share new patches over the network. With Subversion we would have needed to do extra work to regularly backup the central repository and we could not have created new patches during the week the server was down.
Last week wednesday I attended Devnology’s second meeting and had a great evening. There were about 18 people, mostly .NET and Java developers, but there were also two Smalltalkers, a Pythonist and me as a Rubyist. Though a lot of references and examples used the Windows platform and it’s tools and languages, the discussion went about fundamentals that apply to all platforms, languages and communities. The meeting type was Round Table discussion and it was divided into two parts, each with their own topics.
We gathered a number of topics and then voted on them. The two topics we would be discussing were: Generalist vs Specialist and Learning on the job vs Computer Science.
Generalist vs Specialist was interesting because it immediately became clear that everyone had a different opinion on what the terms meant. Is it specific to a language, a business domain, a platform, a role in the team or something else entirely? What is the scope in which you define these terms? It is interesting to question this. I always thought of myself as a Generalist, because I can do almost anything required in our company: be the sysadmin, be the software architect, design the database, write back-end code, write front-end code, test the system, plan the project and lead the team. On the other hand I am a Specialist, because all my knowledge is focused within the Ruby and Rails environment. I would be lost on Windows with a .NET project to develop a GUI application.
After a bit, the discussion flowed over to Learning on the Job vs Computer Science. Arguments that were put forth for CS are that it gives you a broad knowledge of different ways to solve a problem. Learning on the Job saves you four years and immediately starts to teach you what you need to know. You might not know the theoretical background about why things work the way the do, but you will be able to apply it.
The discussion also highlighted interesting perspectives of people who initially started to work and then later got their CS degree. Going this route gives you a lot more practical context to put the theory in. This is the opposite of the CS-first approach, where you first learn a lot of ‘useless’ theory that only later on becomes relevant when it gets a context in your job. A lot of theory might never get a proper context, I can imagine.
I initially started on a CS-ish route by studying Artificial Intelligence, but I did not find it challenging enough. For this reason I quit and found myself a job, where I did find a challenge and learned a lot of things. From time to time I do find myself hungry for the knowledge I could have gained at a CS course. The problem is that the school system is, in my opinion, very fake with grades as goals instead of knowledge as supreme goal. It’s been four years since I left university and I only have vague memories of most of the things I learned, while I was always among the best scoring students. On the other hand, I also tend to forget how to use certain software libraries I knew intimately half a year ago but never used since then. I think that the details of what you know will fade, but the general concepts you learn will probably expand your way of thinking and stay.
The question of this part was “What is/are your favorite…?” A couple of suggested things to list were people’s top-3 books, blogs, podcasts, tools.
My choice was the Pragmatic Programmers. They started as simple software developers turned authors, but they went on to build a publishing house for software books. Whenever I want to learn something new, I always check in there is a PragProg book or screencast available. They introduced the concept of beta books, which are books that have a beta version published as PDF while the author is still writing the book. This is great, as the author gets a lot of feedback to make the printed book better. If the book has source code that contains a bug, you can just click the ‘Report Errata’ button at the bottom of the PDF and submit a bug report and possible fix for the code on that page. Getting your hands on an early version of a book also means that you can read it way before the paper version even ships.
A second choice is Peepcode, which sells professional screencasts on a wide range of topics. Recently they also started publishing smaller eBooks as PDF. A lot of early work is focused on Ruby, but more recent work covers a wider range of topics. Non-ruby topics include Git, Emacs, Clojure, Objective-C, Productivity, Javascript and more.
Devnology is a great initiative that I intend to support by means of attending meetings and generating publicity in the Ruby communities I am part of. Sharing knowledge between previously unconnected communities is a good thing and I hope it will be a huge success.
]]>Array.new(5,1) # => [1,1,1,1,1] [1] * 5 # => [1,1,1,1,1]]]>
When deploying ruby gems or executing arbitrary commands, Capistrano does not really work for me. Don’t get me wrong, the upload:deploy task is great if you want to send an updated project file to your deployed project. Sending a gem or configuration file to an arbitrary directory on the server is not as easy. It is hard to break out of its project-sized box.
Our strategy so far has been to try and minimize our interaction with servers that were more than one hop away from our laptops. When we did have to restart a daemon or look at log files, we would have to do the SSH hop, hop, hop ritual and do our thing. Then exit, exit, exit until we’re back on our laptop’s bash shell. Having RSA key logins to a number of servers saves the trouble of having to enter passwords, but the manual SSH-hopping does get tedious after a while.
Being the kind of programmer who rather scripts the tedious things away rather than perfecting his typing speed to speed up repetitive tasks, I figured it must be possible to automate these things. Knowing I’m not the only one with this problem, Google led me to a solution. Net-ssh-gateway (NSG). Thank you, Jamis Buck!
NSG makes it possible to establish an SSH connection through a gateway server to the next gateway server behind it using port forwarding. Repeat this multiple times until you’ve connected to the final server. The code for this looks like this:
Since this is rather bulky and non-DRY code, let’s condense it into something that involves less repeating of code:
Capistrano uses NSG internally for its gateway connection code, but it seems kinda tightly coupled with Capistrano internals. Also, when I looked at it for the first time, I did not really get how it worked or what it did. To gain a better understanding, I decided to extract the relevant code into a script and rework it to be stand-alone.
The great thing about trying to do something with code you don’t really understand is that you will understand it once you have dissected it far enough. After that, you can work with it and adapt it to suit your needs.
This resulted in an early version of SimpleGate, my attempt at creating a wrapper library around NSG to make gateway chaining easier. The first version was simply the relevant Capistrano method reworked to work in isolation. The next version improved on the gateway chaining by making at as easy as calling SimpleGate.new.through_to(%w[foo bar baz]) to connect through foo and bar to baz.
SimpleGate also has a ServerDefinition to wrap a simple YAML configuration file that stores the actual server connection information. This is useful for cronjob-activated scripts and other non-interactive code when you have a password authentication server in the chain of gateways. For command line tools using SimpleGate, it saves typing.
Capistrano has a good support for SSH, its configuration files and the various authentication schemes. Passwords and RSA keys are both not a problem. SimpleGate currently only supports passwords, as that is what NSG supports out of the box. RSA key logins are a todo item. For configuration, it does the simplest thing that can possibly work: just store the connection info as a plain YAML file in ~/.servers.yml.
After two minor version bumps, I had something that was good enough to build a script that connected through multiple gateway servers to my target and request its uptime. The next step was to execute arbitrary commands, which was a small modification.
Here I discovered another hard to reproduce feature that Capistrano executes in a really nice way: sudo passwords. For some reason I still have to discover, SimpleGate does not let me enter a password when asked for it on the remote server: it just fails the password check and quits. I guess that is another todo item.
After discovering this I wanted to check up on another server that was hidden behind a number of hops and started to change the hard-coded gateway connection sequence in my test script to connect to the new server.
Woah! Wait! Full stop.
Hard-coded connection info is not good. The server name should be a command line option and the connection sequence should be figured out by the script, not by me. Since I was not interested in manually working out all possible connection sequences, I figured it was time to add a simple path-finding Router class to the project…
After a quick read of relevant sections in Bratko, to refresh my knowledge of the topic, I decided to model the search space as a directed non-cyclical graph and search through it using a simple depth-first recursive search algorithm. Support for cyclical graphs will be another todo.
For every node in the network, all its possible connections are described in a YAML file, that is just a Hash of Arrays with server name strings. A special ‘local’ node represents my laptop or any arbitrary internet-connected system. The search algorithm comes down to:
This results in a relatively naive path finder, but with only 15 nodes in my network, this is not much of a problem. Smarter algorithms can be added once I actually need them.
A version bump later I remembered by original goal: upload a gem, install it and restart the daemon associated with it. Restarting is all done in userland, so that is not a problem. Installation requires sudo, which is still a todo item. Uploading was still open.
NSG can open a normal net/ssh session. net/sftp can use this session to do file transfers. A quick copy-paste-adjust later I had a new executable for copying a single file to a single server, through an arbitrary number of hops.
Right now SimpleGate is at version 0.5.0 and it has the three items noted above as left to do. Then it has its core functionality and should be polished up for its 1.0 version.
The command line tools should get parameter support and a –help interface. Then the config files should both be documented and command-line editable. Once those things are in, multi-server support might be useful. In a Capistrano-like fashion connect to multiple servers (sequentially or in parallel) and execute commands on all of them or upload file(s) to them. The file uploader can get a better interface instead of mimicking the code.
]]>In computer science, futures and promises are closely related constructs used for synchronization in some concurrent programming languages. They both refer to an object that acts as a proxy for a result that is initially not known, usually because the computation of its value has not yet completed.
Yesterday evening a number of Rubyists from Amsterdam came together in the public library with our laptops to do cool computer stuff until the library closed.
Among the topics we discussed were new-ish programming languages such Ioke and Clojure. From here the discussion went to the languages that preceded them, like Self and IO.
I think that at some point someone asked “What can IO do that Ruby can’t do?”, so we started looking at the list of features of IO. There we discovered futures.
Futures look nifty: create an object that does not yet have a value and then move on and eventually do something with the object when you need its value. It will block if there is no value yet, which makes sense, but as long as you don’t need the value, you can just move on with your business. This can be useful to do heavy computations or to initiate asynchronous interaction with a remote API. As long as you don’t need the result immediately, a future can be useful.
Then I mentioned Ruby’s Thread#value, which was new to people. So a bit of code was created that looked similar to:
# sleeps for 3 seconds, then prints 615
t = Thread.new { sleep 3; 123}
puts t.value * 5
Explicitly creating a Thread and calling its value does the trick, but it is not very elegant. If you read it, you will probably end think, “What is that Thread doing? Oh, there’s a #value call there. Thread + #value is a future!”. Or you don’t make the connection and misunderstand what this code is doing.
How should it look then to be more elegant?
# sleeps for 3 seconds, then prints 615
f = future { sleep 3; 123}
puts f * 5
So, this is way more elegant! You see it is a future, because it says so! If you don’t know about futures, there is Wikipedia or Google and you might figure out what it does.
In order to make this happen, we need to define a future method that returns an object that when called returns Thread#value for that future. A proxy object. This leads to the next implementation:
class Proxy
def initialize(&block)
@thread = Thread.new(&block)
end
def method_missing(*args, &block)
@thread.value.send(*args, &block)
end
end
def future(&block)
Proxy.new(&block)
end
f = future { sleep 3; 123}
puts f * 5
We created a Proxy class to do the threading logic and use method_missing to capture method calls, which we re-direct to Thread#value.
This works as long as you don’t call methods defined on Object or Kernel, because you get those for free in every class. You need to either avoid using them or get rid of them. There is BlankSlate, but since we were just trying to make our idea work, we just un-defined most methods on the object. Then we had the crazy idea to drop the Proxy object altogether and make Thread function as its own Proxy object. Thread, say bye-bye to your methods!
Here is the final bit of code we produced:
Then the library closed so we went home, after which I polished it a bit and created a github project for it.
This morning my colleague, Filip Slagter, asked me if there wasn’t already a library to do futures in Ruby. After a bit of searching, I found there is one. And it looks nice! Still, I like to re-invent the wheel. It is the best way to learn how something works.
]]>Philipp Pirozhkov created Ryan, a web framework built on top of Reia and YAWS. For reasons I have yet to figure out, it does not want to build on my machine. At least there’s more code to look at to learn the language. Ryan also has a RSpec-like syntax for writing tests, which look interesting.
Starting with web development might be a bit too ambitious for me. I actually managed to mess up a simple “Hello, world!” example, so I’m starting slow. Note to self: methods in Reia have parentheses. It’s not Ruby, where you can omit them.
Wrong hello world:
puts "Hello, world!"
Proper hello world:
puts("Hello, world!")
Sincy my brain still thinks in Ruby, let’s start with a simple bit of Ruby code and convert it to Reia.
First, the Ruby code:
[1,2,3].each { |n| puts n }
This prints out three lines with 1, 2 and 3 on them. Now the same code in Reia:
[1,2,3].each { |n| puts(n.to_s()) }
The two obvious differences:
Just like Ruby, Reia has multiple ways to write this code. Another Ruby-esque way to write it is by using the ‘do’ block notation instead of curly braces:
[1,2,3].each do |n| puts(n.to_s())
Notice the puts() is indented and there is no “end”: that is the Python-style indentation at work. A third way to write the code is by (ab)using List Comprehensions:
[puts(x.to_s()) | x in [1,2,3]]
This is the strangest form for me, since Ruby does not have something similar. The way I interpret it is by reading from right to left: for each x in [1,2,3], do the puts thingy left of the pipe.
]]>My name is Wes Oldenbeuving and I’m a programmer from Amsterdam, the Netherlands. My hobbies include reading books and blogs, playing video games, watching tv series and programming.
My interests cover a wide variety of topics like computer science, economic theory, history, (melodic) metal music, politics, philosophy, personal growth and cooking. I’m not a great chef, but I do try to improvise on recipes every once in a while.
I have been programming (in Ruby) since March 2006 for the Dutch company yoMedia. Ruby on Rails is what brought me to Ruby, but it is not the only thing keeping me. Ruby as a language is really nice to work with and works for both simple scripts, to automate mundane tasks, and to build complex applications. The people in the local Ruby communities are also nice, smart and full of crazy creative ideas.
We have a number of regular events that I attend 99% of the time: Amsterdam.rb, Utrecht.rb and the soon-to-start Amsterdam.rb screen-peeking diner thingy.
I expect my blog posts to mainly cover programming-related topics, though they will not necessarily be all about Ruby. In fact, expect the first posts to be about the new Reia programming language.
]]>