Building a responsive graph

A responsive SVG graph should behave like any other styled element

• Expand to fill the containing element (if required)
• Keep line widths and text the same size and prevent distortion, no matter how big the graph becomes

SVG does allow percentages as a unit, but I haven’t personally seen them used much (at all, actually) and D3 uses px for its ranges.

Luckily SVG elements can also define their own viewports through the viewBox attribute, so its contents will always fill the same portion of the viewport, no matter how big or small to the outer element becomes (or whatever other transformations are applied to it).

W3C SVG viewBox example

(A related attribute is preserveAspectRatio, which I’ve set to none as I want the graph to fill the whole space. There are other options to force uniform scaling if you want your graph to maintain a fixed aspect ratio.)

<svg viewBox="0 0 1000 1000" preserveAspectRatio="none">
<g>
<!-- graph here -->
</g>
</svg>

This demo graph resizes nicely, but the text and line width can get horribly distorted – not really what I wanted.

Preventing the distortions

Graphical SVG elements support the vector-effect attribute, and setting this to non-scaling-stroke stops the lines from being affected by any transforms that have been applied. Unfortunately there isn’t an equivalent for text, but this JavaScript snippet from Gavin Kistner can help here by undoing the damage. Just set it to run on load and hook it up to fire when the window resizes as well.

(I’ve actually changed the code slightly as it uses getTransformToElement to, um, get the transform from the containing SVG element to its contents, but the transform matrix returned by the function doesn’t take into account transforms applied to the SVG element itself. So I’ve put an SVG element (with a custom viewBox) inside the outer SVG element and referenced that in the JS instead.)

Putting it all together

Demo graph

<svg>
<svg viewBox="0 0 1000 1000" preserveAspectRatio="none">
<g>
<!-- graph here -->
</g>
</svg>
</svg>

.line {
  fill: none;
  stroke-width: 3px;
  vector-effect: non-scaling-stroke;
}
.area {
  vector-effect: non-scaling-stroke;
}

$(function () {
	"use strict";
	var resizeTracker;
	// Counteracts all transforms applied above an element.
	// Apply a translation to the element to have it remain at a local position
	var unscale = function (el) {
		var svg = el.ownerSVGElement.ownerSVGElement;
		var xf = el.scaleIndependentXForm;
		if (!xf) {
			// Keep a single transform matrix in the stack for fighting transformations
			xf = el.scaleIndependentXForm = svg.createSVGTransform();
			// Be sure to apply this transform after existing transforms (translate)
			el.transform.baseVal.appendItem(xf);
		}
		var m = svg.getTransformToElement(el.parentNode);
		m.e = m.f = 0; // Ignore (preserve) any translations done up to this point
		xf.setMatrix(m);
	};
	[].forEach.call($("text, .tick"), unscale);
	$(window).resize(function () {
		if (resizeTracker) clearTimeout(resizeTracker);
		resizeTracker = setTimeout(function () { [].forEach.call($("text, .tick"), unscale); }, 100);
	});
});

Notes

There’s also transform ref() that applies transforms to an element relative to a specified SVG element rather than its parent. This would be ideal as I could ditch the JS, but unfortunately Opera seems to be the only browser that supports it at the moment.

If you want sharper lines you can disable antialiasing with shape-rendering=optimizeSpeed or shape-rendering=crispEdges.

Two useful StackOverflow questions on this.

The demo doesn’t seem to work with Opera or IE. I may look into it at some point.

Overview

• Shiny examples
• Theories of emergence overview
• My view of emergence and (briefly) what I did with it
• Some ideas for engineering emergence: solve another problem
• Speculative real world examples

If you’d like to take a look, I’ve put the presentation up here.

Source on Github Source on Github
(You need both, but the first will get the second via NPM.)

Client code

MediaTemple were nice enough to introduce an API that provides a load of stats about what your server is doing some time ago. This is the same info that you can see in the server status bit of the account centre. Unfortunately the interface there is a bit clunky and it also logs you out (sensibly) after a few minutes’ inactivity.

Inspired by some of the really cool stuff Mike Bostock has done with D3, (mt) Stats gives you the same info with a (hopefully) nicer interface, and maybe even doing quite well with Tufte’s data-ink ratio. You can add as many graphs as you like (if your CPU can take it at least), move them around by dragging and adjust the interval over which stats are shown with the range slider.

If you hover over a graph, you should see buttons to change the metric for the graph and use a log or zero-based scale. On my server at least, I often end up with some large peaks in the kb out graph, which can hide a lot of much smaller bumps when only a few kb was transferred, so a log scale can be a useful way to see them more clearly. Similarly, certain metrics like memory usage can work better with (or without) a zero-based scale.

The app settings are stored in your browser’s localStorage (under mt-stats), so if you close the tab they should be there when you revisit the site. Though obviously not if you use a different browser or clear the cache.

I used Backbone to look after (and modularise) the graphs collection and some jQuery UI goodness for the drag and drop and slider. The site uses Twitter’s Bootstrap, lightly restyled to give it a Metro-like appearance. (Much easier than last time as the look has now been rolled out beyond phones.)

Server code

The back end is written in Node. The reason there’s a server component at all is to hide the API key away from prying eyes, though I did end up abstracting away some of the API’s quirks too. (The same API key can be used to do all sorts of other things like reboot the server and change the root password. I wish it were possible to create keys at different privilege levels but it’s not.)

The stats API can get details of the server’s current state or its state for an interval in the past, in which case a resolution can be given. This resolution may or may not be respected by MediaTemple, e.g. there’s a minimum resolution of 15s and a request for 2 hour intervals will be returned at a resolution of 60 minutes.

The API server will also only serve up a fairly small amount of data at a given resolution – smaller than I’d like, so the server divides up the client’s range into several requests and combines them before returning. The server code’s currently set up to return up to a week’s worth of data at once; it will return more, but at a resolution that would hammer the MediaTemple server rather, so please add some more ranges to the mt-stats.json config file if you want to do that. All data are transferred using JSON.

Setting up (mt) Stats on your server

• Get an API key for your MediaTemple server. You’ll also need your service ID later, which you can get by visiting https://api.mediatemple.net/api/v1/services/ids.json?apikey=XXXXX (I only have one service ID as I have one server, but apparently you could see more.)
• Get a copy of Node.js. You can put (mt) Stats on your server if you want to access your stats from anywhere, or run it locally on your machine if you prefer. If you haven’t got Node already and you’re using Linux, there’s an installation guide here. If you’re running Windows or OSX then you can just download the installer from the Node site.
• Download (mt) Stats Viewer from Github and run npm install in the folder where you put it to get the packages it needs (list further down for reference).
• You should now have a copy of mt-stats. Rename the config file node_modules/mt-stats/config/mt-stats-sample.json to mt-stats.json and change the service ID and API key to match your server and key.
• Rename the config file config/app-sample.json to app.json and change the URL and port. localPort is the internal port on which the service runs and url is the external URL for the service. Also edit gaAccount and gaDomain to match your Google Analytics details if you want to track usage.
• Run app.js and you should be good to go.

mt-stats-viewer settings

Please rename app-sample.json to app.json.

• localPort – Local port on which the app is running
• url – Public URL for the app
• mode – development uses uncompressed and (many) separate JavaScript files, while production uses Google’s CDN for jQuery
• title – Page title
• description – Meta description contents
• graphRanges – Timespans that are sent to the client for the range slider. Currently 5 mins – 1 week in ms
• author – Meta author contents
• jQueryCdnUrl – Yup
• gaAccount – Your Google Analytics account
• gaDomain – Your Google Analytics domain

{
	"localPort": 3000,
	"url": "http://bits.meloncholy.com/mt-stats",
	"mode": "production",
	"title": "(mt) Stats",
	"description": "Visualise your MediaTemple stats with some gorgeous D3 graphs.",
	"graphRanges": [
		300000,
		600000,
		900000,
		1200000,
		1500000,
		1800000,
		2700000,
		3600000,
		5400000,
		7200000,
		10800000,
		14400000,
		18000000,
		21600000,
		32400000,
		43200000,
		54000000,
		64800000,
		75600000,
		86400000,
		172800000,
		259200000,
		345600000,
		432000000,
		518400000,
		604800000
	],
	"author": "Andrew Weeks",
	"jQueryCdnUrl": "http://ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js",
	"gaAccount": "UA-XXXXXXXX-X",
	"gaDomain": "meloncholy.com"
}

mt-stats settings

Please rename mt-stats-sample.json to mt-stats.json.

• serviceId – Your server’s service ID
• apiKey – Your MediaTemple API key
• mode – Doesn’t do anything here, but its presence gives me a warm, comforting glow
• rootPath – The root URL path for all API calls
• interval – Server polling interval for client. MediaTemple’s stats update every 15s
• ranges – For each range, the resolution at which to request data from the API (e.g. every 15 seconds) and the maximum span (step) to request in one go (to stop the API server objecting). Range is the maximum timespan at which to use that resolution and step
• metrics – Metrics supplied by the API. apiKey is the key name in JSON objects and niceName is the name to use on the graphs
• definedRanges – MediaTemple also supports some default intervals that can be requested with these URLs, e.g. this URL will serve up the last 5 minutes’ data. These are not used by the front end
• currentUrl – API server URL from which to get the current stats. %SERVICEID and %APIKEY are replaced with your service ID and API key
• historyUrl – API server URL to request stats going back for the past X seconds, e.g. this URL will also give the past 5 minutes’ data
• rangeUrl – API server URL to get stats covering a specified time range

{
	"serviceId": 000000,
	"apiKey": "XXXXXXXX",
	"mode": "production",
	"rootPath": "/api/",
	"interval": 15000,
	"ranges": [
		{ "range": 3600, "resolution": 15, "step": 3600 },
		{ "range": 43200, "resolution": 120, "step": 28800 },
		{ "range": 86400, "resolution": 240, "step": 57600 },
		{ "range": 604800, "resolution": 1800, "step": 432000 }
	],
	"metrics": [
		{ "apiKey": "cpu", "niceName": "CPU %" },
		{ "apiKey": "memory", "niceName": "Memory %" },
		{ "apiKey": "load1Min", "niceName": "Load 1 min" },
		{ "apiKey": "load5Min", "niceName": "Load 5 min" },
		{ "apiKey": "load15Min", "niceName": "Load 15 min" },
		{ "apiKey": "processes", "niceName": "Processes" },
		{ "apiKey": "diskSpace", "niceName": "Disk space" },
		{ "apiKey": "kbytesIn", "niceName": "kb in / sec" },
		{ "apiKey": "kbytesOut", "niceName": "kb out / sec" },
		{ "apiKey": "packetsIn", "niceName": "Packets in / sec" },
		{ "apiKey": "packetsOut", "niceName": "Packets out / sec" }
	],
	"definedRanges": ["5min", "15min", "30min", "1hour", "1day", "1week", "1month", "3month", "1year"],
	"currentUrl": "https://api.mediatemple.net/api/v1/stats/%SERVICEID.json?apikey=%APIKEY",
	"historyUrl": "https://api.mediatemple.net/api/v1/stats/%SERVICEID/%RANGE.json?apikey=%APIKEY",
	"rangeUrl": "https://api.mediatemple.net/api/v1/stats/%SERVICEID.json?start=%START&end=%END&&resolution=%RESOLUTION&apikey=%APIKEY"
}

Dependencies

If NPM doesn’t do it for you, you’ll need to get copies of

• Express
• Jade
• Konphyg
• Less
• (mt) Stats

Problems and stuff

It doesn’t work at all! I’ve pressed new and nothing happens!

IE and Opera aren’t supported (at least up to IE 9) – as I said, I wrote this principally to learn about D3 and for me, so supporting older browsers was not on my to do list. The biggest problem here is probably D3, which uses the browser’s SVG capabilities to draw graphs and doesn’t work with IE and Opera. (I know that IE (as of version 9) and Opera have some SVG support, but D3 doesn’t seem to like it, or at least not for the graphs I used. I haven’t looked into this further.) If this bothers you, I’d be more than happy to integrate a polyfill if you write one. :)

Why do the graphs jump sometimes?

The client requests an update every 15 seconds (the maximum resolution offered by MediaTemple) and it relies on this to slide the graphs along and create a smooth viewing experience. Unfortunately the updates from MediaTemple are not always that reliable: they can have a timestamp that’s off by a couple of seconds or, sometimes, by 30+ seconds. In these cases, the graphs will unfortunately jump as they recalibrate to the new timestamp.

What’s with the ugly loops in my graphs?

Particularly after loading the page, MediaTemple’s API will sometimes supply an update that’s timestamped 30-45s after the previous one, rather than the 15s the app was expecting. The line tension I’m using means that the graphs sometimes show little loops in these cases. If this bugs you, you can adjust it in mt-stats.graph.js.

Urg! My graphs just raced of the screen and vanished!

The graphs sometimes seem to get their knickers in a twist and fly almost entirely off the screen before new data points are added. I think this is because the API server sometimes returns very few data points (2 or 3) when the range is set to 5 mins, though I have seen it happen at other times too. But I’m not sure what’s going on here. Sorry.

The axes are weird / have disappeared

Yes, they seem to do that sometimes. This can happen if you’re using a log scale and the graph range is currently small and not zero-based (e.g. 88.5-88.6). In those circumstances, D3 finds it difficult to add suitable tick marks. But it happens at other times too, and in those cases I’m not sure if it’s my fault (probably) or a bug in D3. Refreshing the page could help, but if not I apologise.

I don’t want to run Node.js

I like Node so I used it here. That said, the back end is basically a specialist proxy server, and translating it into another language should be pretty easy if you want to use PHP, Ruby or something else. (Again, please let me know!)

It only works on MediaTemple

That’s true. But if you use another host that provides server stats in a JSON wrapper then translating it should be fairly trivial. MediaTemple sends status updates in these formats:

Ranged request

{"statsList":{"timeStamp":1343138160,"resolution":15,"serviceId":00000,"stats":[
{"timeStamp":1343138160,"cpu":12.99,"memory":83.24,"load1Min":0.0,"load5Min":0.0,"load15Min":0.0,"processes":48,"diskSpace":43.88,"kbytesIn":1.93,"kbytesOut":21.07,"packetsIn":11.4,"packetsOut":20.27,"state":1},
{"timeStamp":1343138175,"cpu":0.01,"memory":83.79,"load1Min":0.0,"load5Min":0.0,"load15Min":0.0,"processes":48,"diskSpace":43.88,"kbytesIn":0.85,"kbytesOut":33.61,"packetsIn":16.07,"packetsOut":22.6,"state":1},
...
]}}

Current status request

{"stats":{"timeStamp":1345595595,"cpu":1.76,"memory":103.39,"load1Min":0.0,"load5Min":0.02,"load15Min":0.0,"processes":72,"diskSpace":49.2,"kbytesIn":57.0,"kbytesOut":6.2,"packetsIn":72.0,"packetsOut":86.0,"state":1}}

I think the only thing you’d have to do to the front end is change this line in mt-stats.data.js to extract the stats from whatever object your server sends them in

for (var i = 0, len = data.statsList.stats.length; i < len; i++) {

You'll have to do a bit more fiddling around in mt-stats/index.js and mt-stats/config/mt-stats.json as that uses MediaTemple URLs and checks for specific return messages, but hopefully translating wouldn't be too much work.

En fin

Hope this is useful / interesting to someone other than me. If you end up using it, please let me know!

Source on Github Source on Github
(You need both, but the first will get the second via NPM.)

Legal fun

Copyright © 2012 Andrew Weeks http://meloncholy.com

(mt) Stats is licensed under the MIT licence.

try {
	return true;
} finally {
	return false;
}

The result is consistent in Java, JavaScript and Python. I wrote this snippet on a whiteboard here at Google. For the next couple of weeks engineers passing by would abruptly stop, transfixed by the paradox. Nerd sniping. I now use it during hiring interviews, if the candidate suffers a similar breakdown, I know we’ve got a keeper.

If you haven’t tried it yet, the answer is false. Which makes sense I guess (haven’t dug into the specs, but I’m sure it’s all there).

But what actually happens in the try? Interestingly the statement is executed, but the function just doesn’t return. Consider

var a = 2;
try {
	return a++;
} finally {
	return a++;
}

The answer here is 3.

Luckily I found a nice function to help me (updated here). The function contains a big array of basic glyphs (and digraphs, trigraphs, etc.) and regexes of the letter with diacritical marks / Unicode di- or trigraph, e.g.

{ "base": "b", "letters": /[\u0062\u24D1\uFF42\u1E03\u1E05\u1E07\u0180\u0183\u0253]/g },

I needed this array on the server (for storing stuff in the database) and on the client (to give snappy performance), but, as it’s quite large, I wanted to wait until the page had loaded before I requested it, which meant using JSON.

Sending a regex via JSON

Unfortunately stringifying a RegExp gives {}, which is not quite what I had in mind.

If an object has a toJSON function then stringify will use that instead of doing its default conversion, so I added a function that loops through the array and, if it comes across a regex, converts it to a string.

(I’ve assumed the array to send is called bigArray and that bigArrayJSON is cached outside this function.)

bigArray.toJSON = function () {
	var entry;
	if (bigArrayJSON.length !== 0) return bigArrayJSON;
	for (var i = 0, len = bigArray.length; i < len; i++) {
		entry = {};
		Object.keys(bigArray[i]).forEach(function (key) {
			entry[key] = bigArray[i][key] instanceof RegExp ? bigArray[i][key].toString() : bigArray[i][key];
		});
		bigArrayJSON.push(entry);
	}
	return bigArrayJSON;
};

Converting a regex string back into a real regex

Now all I needed to do was convert the regex strings back into regexes, and it turned out I could do this with a one-liner.

The regex strings sent from the server will have the form /pattern/flags, which can be split up using /^\/(.*)\/(.*)/ into [pattern, flags] (with a quick slice to restrict it to the parenthesised matches only) – almost perfect for making a new regex. Unfortunately the RegExp constructor expects two parameters, not an array, but apply can handle this.

function saveBigArray(data) {
	bigArray = data.bigArray;
	for (var i = 0, len = bigArray.length; i < len; i++) {
		// Convert string regex to real regex. Use apply as exec returns array.
		bigArray[i].letters = RegExp.apply(undefined, /^\/(.*)\/(.*)/.exec(bigArray[i].letters).slice(1));
	}
}

I'm a bit uncomfortable with the lack of new when calling RegExp. Testing suggests that it works just fine – RegExp("aa?b", "g") returns an identical new regex as new RegExp("aa?b", "g") – but I'm not happy with this. Any ideas on how to do a new with apply?

(It's possible to use eval on the string regex too, but I didn't want to do that. :)

MixCloud RSS Converter

The source is also on GitHub if you want it.

The bit with some reminiscing you can probably skip

While at York, I was involved with (and ran) Breakz for quite a few years. Breakz is a music society that stages drum & bass, dubstep and electronic music events, and has brought some pretty major acts to York, including Chase & Status, Benga, Skream, Brookes Brothers and Ed Rush.

One of the things I did, around 5 years ago, was move the website to WordPress. I ended up jumping in after the domain expired last week, and the site needed some more TLC generally. As I had some free time, I swapped it over to a new (cheaper and faster-so-far) host and changed the theme for one based on the one I did for Premier Tri.

I added a lot of Facebook links as everything happens there now, but I wanted to put some music back on the site too. (Once upon a time, the site hosted MP3s composed by some of the DJs and links to a few online radio stations, but this bit had long since bitten the dust.) Of course there’s been an explosion in online music over the past five years, and there are a huge number of sites out there for artists to host their tunes and network with others.

Two of the most popular, with this crowd at least, are SoundCloud and MixCloud, where most of the DJs have accounts. I needed to get the tunes from there and onto the site with as little work (on their part) as possible, which basically meant syndicating RSS feeds via a WordPress plugin.

The bit about which you care

Neither SoundCloud nor MixCloud offer RSS feeds, but fortunately Adam Schlitt’s already written a converter for SoundCloud. Unfortunately I was out of luck for MixCloud (they used to have feeds, but they don’t seem to work now). So I decided to write one.

MixCloud does have an API, so I was able to use this to get a recent uploads stream and then query their oEmbed API to get an embedded music player for each post, which I then added to an RSS feed. To make it pretty, I also made a frontend using the ubiquitous Twitter Bootstrap.

I really am standing on the shoulders of giants here as this required almost no code on my part (maybe 300 lines) thanks to the node-oembed and node-rss modules from Stephan Maka and Dylan Greene, and of course Express, Jade and Node.js.

If you use it and if you can, please set the update interval to something fairly big. The app doesn’t cache anything, and every request could query MixCloud’s API around 20 times. Actually I probably shouldn’t worry about MixCloud as they no doubt have very meaty servers, but I don’t, so please think of me too. :)

Here’s the Breakz music page, which syndicates feeds from a number of MixCloud (and SoundCloud) accounts.

MixCloud RSS Converter

The code

If you’re interested in getting your own copy, I’ve put the source up on GitHub too. As there’s not a lot to it, hopefully it will (almost) just work. After adjusting the settings you should be good to go.

Settings

Please rename /config/app-sample.json to app.json.

• mode — production loads minimized files and includes Analytics, while development doesn’t.
• localPort — The local port to use.
• url — Public URL for the site.
• title — Name of the site. Used in the browser title bar and elsewhere.
• author — Not really used, actually. Just appears in the meta header.
• description — Meta description content.
• rssDescription — Short description of the feed. %user will be swapped for the current account’s name.
• cloudcastStreamUrl — MixCloud API URL for getting profiles.
• cloudCastOembedUrl — MixCloud API URL for getting Cloudcasts.
• jQueryCdnUrl — Yup. Only used in production mode.
• gaAccount — Your Google Analytics UA number.
• gaDomain — Your Analytics domain.

Example

{
	"mode": "development",
	"localPort": 3000,
	"url": "http://bits.meloncholy.com/mixcloud-rss/",
	"title": "MixCloud to RSS converter",
	"author": "Andrew Weeks",
	"description": "Get an RSS feed for your (or anyone else's) Mixcloud mixes and podcasts.",
	"rssDescription": "Visit %user's profile on MixCloud. Listen to their recent and most popular Cloudcasts. It's free and really easy to use. MixCloud to RSS thingy hacked together by Andrew Weeks. http://meloncholy.com",
	"cloudcastStreamUrl": "http://api.mixcloud.com/%user/cloudcasts/",
	"cloudcastOembedUrl": "http://www.mixcloud.com/oembed/?url=http%3A//www.mixcloud.com%cloudcast&format=json",
	"jQueryCdnUrl": "https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js",
	"gaAccount": "UA-XXXXXXXX-X",
	"gaDomain": "meloncholy.com"
}

Dependencies

In case NPM doesn’t help you out for some reason, you’ll also need

• Express
• Jade
• Konphyg
• node-oembed
• node-rss

Source on GitHub

Legal fun

Copyright © 2012 Andrew Weeks http://meloncholy.com

MixCloud RSS Converter is licensed under the MIT licence.

There’s a load of bumf about the original PHP version here (mostly relevant) and some supplementary info about this version here, though probably the best thing to do is to try the demo site. Yay!

Source on GitHub

Nota bene

The front end code is (almost) identical to that in the PHP version. It could do with updating a bit, e.g. to use the HTML5 history API rather than #! and moving the jQuery events over to on. But other than that I think it works fine.

I redid an existing project so I could focus on how Node, and not the app itself, works. It was definitely a learning process for me, and I’m sure I could have done things better (and would probably do some bits better if I were starting again now). So if you have any feedback on how I could improve things, I’d love to hear.

With those health warnings out of the way…

Settings

You’ll need to set up a MySQL database first (see sample content). Then there are some configuration things to do

/config/app.json (Please rename app-sample.json.)

• localPort – Local port to use
• sessionSecret – Session secret for Express
• siteUrl – Public URL for your site
• mode – development or production. development gives more error messages, while production uses minimized, combined public JS files that load a lot faster.
• batchSize – Number of videos to load at once in infinite scroll.
• videoCount – Approximate number of videos in the database. Just used for bragging rights in a few places.
• cookiePath – Cookie path to use. Unless you want to restrict it to a subfolder, I’d suggest using /.
• siteTitle – Name of the site. Used in browser title bar and elsewhere.
• siteTitlePost – Text to stick in the title bar after the video name (where appropriate).
• siteDescription – Meta description content.
• maxRelatedVideos – If we haven’t come via a search (via a link or refresh), show this many related videos at the side of a video.
• maxSimilarVideos – Show this many similar videos at the bottom of a video. See video.js for more. This number should obviously work for the largest video size offered.
• prettyLink – Prepend this to each video’s URL for SEO, so if this were amazing-stunts, the URLs would be something like http://example.com/#!amazing-stunts/name-of-this-video.
• jQueryCdnUrl – In production mode, load jQuery from here.
• gaAccount – Your Google Analytics UA number.
• gaDomain – Your Analytis domain.

Example

{
	"localPort": 3000,
	"sessionSecret": "session secret",
	"siteUrl": "http://bits.meloncholy.com/node-video-gallery/",
	"mode": "development",
	"batchSize": 20,
	"videoCount": 300,
	"cookiePath": "/",
	"siteTitle": "Video Gallery",
	"siteTitlePost": " | Video Gallery",
	"siteDescription": "This Video Gallery is a web app with lots of jQuery and Node goodness. Yum.",
	"maxRelatedVideos": 19,
	"maxSimilarVideos": 7,
	"prettyLink": "video",
	"jQueryCdnUrl": "http://ajax.googleapis.com/ajax/libs/jquery/1.6.4/jquery.min.js",
	"gaAccount": "UA-XXXXXXXX-X",
	"gaDomain": "meloncholy.com"
}

/config/database.json

• host – Database host
• database – Database name
• user – Database user
• password – Database password

Example

{
	"host": "localhost",
	"database": "database",
	"user": "user",
	"password": "password"
}

Dependencies

To get it to work, you’ll also need

• Express
• Jade
• Konphyg
• VidStreamer.js

Sample content

If you want to get going quickly, you can use the sample videos, photos and database from the PHP Video Gallery on GitHub (placeholder and database folders resp.).

And try something like this for vidStreamer.json (in /node_modules/vid-streamer/config) to get it to serve up the sample content.

{
	"mode": "development",
	"forceDownload": false,
	"random": true,
	"rootFolder": "/path/to/express/public/",
	"rootPath": "node-video-gallery/",
	"server": "VidStreamer.js/0.1"
}

Thanks

The download includes Longtail Video’s JW Player, released under Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) licence. Commercial licences are also available. You’re of course welcome to swap the player for another one if you don’t like it, though you’ll need to change some of the source files that set up parameters to pass to the player.

The sample video footage comprises some short video clips and still images of these video clips by Catrin Hedström and released under a Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) licence. It’s gorgeous footage too – do check it out!

Source on GitHub

Legal fun

Copyright © 2012 Andrew Weeks http://meloncholy.com

Video Gallery is licensed under the MIT licence.

Includes some code written by others; see source for credits and licence details (though it’s all permissive).

As using a LAMP stack to support a Node app seemed wrong, I wrote a simple JavaScript video streamer. Most of the requirements came from looking at how xmoovStream Server works.

• Supports http pseudostreaming
• Streams Flash video and can resume streams
• Streams other video and file formats and can resume these streams too
• Works with JW Player’s bitrate switching

Source on GitHub

Using VidStreamer

To make a standalone video streamer, try something like this

var http = require("http");
var vidStreamer = require("vid-streamer");
var app = http.createServer(vidStreamer);
app.listen(3000);
console.log("VidStreamer.js up and running on port 3000");

And here’s an example of including it in an Express app

var app = require("express").createServer();
var vidStreamer = require("vid-streamer");
app.get("/videos/", vidStreamer);
app.listen(3000);

Settings

There are a few things to twiddle in /config/vidStreamer.json. (Please rename vidStreamer-sample.json.)

• mode – Not actually used right now. :)
• forceDownload – Tell the browser to show a save file dialog.
• random – Show a random file instead of the one named (see below).
• rootFolder – The root folder from which files are returned. Accessing subfolders is possible by putting them in the path.
• rootPath – The path assumed to match up with the root folder.
• server – Server string returned in the header.

Standalone example

{
	"mode": "development",
	"forceDownload": false,
	"random": false,
	"rootFolder": "/path/to/videos/",
	"rootPath": "videos/",
	"server": "VidStreamer.js/0.1"
}

Example for Express. (Note that rootPath should be relative to the root URL of your Express app.)

{
	"mode": "development",
	"forceDownload": false,
	"random": false,
	"rootFolder": "/path/to/express/public/",
	"rootPath": "",
	"server": "VidStreamer.js/0.1"
}

Serving random files

You can use VidStreamer to serve up random files instead of the actual file requested. This can be useful if you’re demoing an app that’s supposed to have hundreds of videos but you don’t want to go to the trouble of making them all.

• Which file to return is calculated from the requested file name, so the same one will be returned each time for the same name. This is useful if you want to switch between video bitrates for example.
• The file returned will be of the same type (same extension) as the requested file.
• The file returned will be from the same folder as the requested file.

Thanks

I hadn’t really thought about how to write video streamer before, so Devendra Tewari’s post and of course the xmoovStream Server source were very useful to me.

Source on GitHub

Legal fun

Copyright © 2012 Andrew Weeks http://meloncholy.com

VidStreamer.js is licensed under the MIT licence.

HRODC postgraduate training courses

ACCEPT JESUS, FOREVER FORGIVEN!

While both are, of course, fine examples of important web design maxims, surely their impact would be much greater if they could be finessed into one great web page? After all, who could resist combining the eloquent brevity of a 1.2MB page full of tables with the understated finesse of animated rainbow gifs and autoplaying music?

Turns out I couldn’t. And, having just rediscovered the result, I thought I’d stick it up here.

Aaarg!

To enjoy the full experience, including the music and a bird that follows your cursor around, please view in an old version of IE.

Download source

Update: this is also part of the Node.js Video Streamer I made. Though not in PHP, obviously.

I recently uploaded a Video Gallery web app I’d made (demo, code download). Unfortunately I don’t have the real videos and pictures that it was supposed to use, so I thought I’d cut some clips from the silent-but-gorgeous They Call Us Animals music video footage. But the Video Gallery has about 300 different videos in the database, and I really didn’t want to make 900 sample clips (each one in 720p, 480p and 360p). So I made about 10, which seems enough to give a little variety.

But I couldn’t just serve up any video. Using the HTTP pseudostreaming support in JW Player, the site lets users change video sizes while they’re watching, and the player will switch to a different resolution source file if one is available to improve the viewing experience. It looks good, but the effect was rather spoiled when the video selector returned a higher resolution version of a completely different clip.

So I wrote a little PHP script that would serve up an apparently random sample video consistently.

Using the code

First set $folder to point at the folder where the files you want to serve are located. It’s currently set up to use the folder where the file is uploaded.

The file selector will serve up jpegs, pngs and flv files from the box. If you want to allow others, add their file extensions and MIME types to the $ext_list array. There are examples for CSS, PDF and HTML files in the source. (On a side note, please don’t add PHP to the list and use $folder = '.' unless you don’t mind it returning itself sometimes.)

To get a random-but-consistent file, link to it as any other file

<a href="http://example.com/rotate.php?file=kittens.flv">link</a>

<img src="http://example.com/rotate.php?file=kittens.jpg" />

The file returned will be of that same type (flv or jpg above). It will not be kittens.flv, but you will get the same one every time you ask for kittens.flv.

The code

After performing a few checks on the file type and allowed extensions, I used crc32 to convert the supplied filename into an integer and ran it through the modulus operator to get a number in the range of the file count in the current folder. Then it reads the chosen file and we’re done. And that’s it.

$ext_list = array();
$ext_list['gif'] = 'image/gif';
$ext_list['jpg'] = 'image/jpeg';
$ext_list['jpeg'] = 'image/jpeg';
$ext_list['png'] = 'image/png';
$ext_list['flv'] = 'video/x-flv';
$file = null;
$vid = null;
$vids = null;
$content_type = null;
if (!isset($_GET['file'])) return;
$file = pathinfo($_GET['file']);
// Just in case there's other stuff in the folder too (like this file)
if (!array_key_exists($file['extension'], $ext_list)) return;
if (substr($folder, -1) != '/') $folder .= '/';
// Select a file based on the given extension.
$vids = glob($folder . '*.' . $file['extension']);
$vid = $vids[abs(crc32($file['basename'])) % count($vids)];
$content_type = 'Content-type: ' . $ext_list[$file['extension']];
header ($content_type);
readfile($vid);

Fun with .htaccess

Supplying random-but-consistent videos like this changes their path, as it’s now necessary to include (at least) consistent-random.php?file=. Not ideal, but a few lines added to .htaccess can point the original paths at the new placeholder files.

RewriteBase /video-gallery/
RewriteRule ^images/thumb/([-_a-zA-Z0-9\.]+)$ placeholder/images/thumb/consistent-random.php?file=$1 [L]
RewriteRule ^images/full/([-_a-zA-Z0-9\.]+)$ placeholder/images/full/consistent-random.php?file=$1 [L]
RewriteRule ^videos/([-_a-zA-Z0-9\.]+.flv)$ placeholder/videos/consistent-random.php?file=$1 [L]

And we’re done! The Video Gallery now thinks it’s supplying the videos and images as originally intended, not the placeholder ones it’s actually showing.

Download source