Cartographer.js screenshots: choropleth, area-scaled circles, and pie charts

Thematic maps describe statistical data about places. They come in many flavors: color-coded regions, area-scaled circles, pie charts, flow charts, heatmaps, and so on. By aligning statistical data to a geographic anchor, the underlying data can be made easier to understand.

The problem

It is difficult to build thematic maps for the web using standard HTML and Javascript. In the few maps that do exist, developers are re-creating basic thematic elements or using sub-standard visualization techniques. In particular, “red dot fever”—the ubiquitous red pushpins on Google Maps.

Our solution

Cartographer.js, a simple library for Google Maps that simplifies the development process and provides instant access to proven visualization techniques. Developers are freed from worrying about how to tweak the Google Maps API to build the thematic elements they need, and can focus on more important problems like data collection and normalization. Cartographer.js supports custom styling so it can integrate seamlessly into existing web applications.

Why HTML + Javascript?

Cartographer.js is built to leverage cutting-edge web technologies. Other formats, like Flash or Processing (Java Applet), are not supported on many mobile devices, and many developers are already familiar with HTML + JS. Also, many recent advances in the power and speed of Javascript (cf. V8) make it more desirable to build rich web applications in pure HTML + JS. We feel there is a very strong future for this set of technologies.

How do I use it?

Cartographer.js is a simple one-line include. It relies on the excellent Raphael.js vector-graphics library. Cartographer.js is currently in an early-stage Alpha release.

You can find out more by visiting the Cartographer.js website, where you can download the script, see examples, and read through the documentation.

Your feedback is greatly appreciated!

Should you walk or take the bus?

It is intuitive that when you have a short distance to travel it is always faster to walk than wait around for a bus. But exactly how long should you wait? We decided to do some calculations to find out.

Click here to see Walk or Bus? and discover for yourself how long you should be willing to wait for a bus.

If you remember this rule of thumb you ought to get along just fine:

If you have N blocks to travel, you should only wait for the bus if it is less than N minutes away.

Of course, if you have a bike, you should probably use that instead.

Feel free to leave your questions or comments.

Link: Walk or Bus?

The Ebstorfer mappamundi, map of the Christian World in the 13th century overlaid with an image of Christ. (click for high resolution)

The Ebstorf mappamundi was drawn in 13th century Saxony and depicts the Christian worldview within the body of a crucified Christ. The map illustrates both the “known world” as well as significant landmarks and points of interest for the curious pilgrim.

Christ’s head is in the East, at the top of the map, the direction of Paradise. His hands mark the northern and southern limits of the known world, and his feet are at Gibraltar where the Mediterranean meets the Atlantic. In the middle of the map we see Jerusalem, the spiritual center of Christendom, located at Christ’s navel. Europe is in the bottom left quadrant of the map, Africa in the bottom right, and Asia dominates the upper half.

In the East, near Christ’s head, is the Garden of Eden surrounded by mountains. Just west are the Chinese (note the two figures bent to gather silk) and the Indians. In the Indus Valley we see opium eaters, people who stare at the sun all day (gymnosophists), as well as that strange tribe who subsists only on the scent of apples. Alexander the Great is consulting the Oracle of the Sun and the Moon.

Details from the Ebstorfer Mappamundi. Left: Places in India, including Alexander the Great consulting the Oracle, the Opium Eaters in their poppy field, and a gymnosophist staring at the sun. Right: Mt. Sinai (oriented sideways), and Sodom and Gommorah covered by the Dead Sea.

At the center of the map, near the all-important Jersulaem, we can find the Tower of Babel, Bethlehem (marked with the Star of David), Sodom and Gomorrah, and Mt. Sinai.

Africa and northern Asia both are hinterlands illustrated with mythical creatures and legends. In Africa, a tribe of dwarfs rides crocodiles. In Asia, two Amazonian women guard their citadel.

Maps such as this may explain much of the surprise 15th and 16th century explorers felt as they sailed to the Americas and around the Cape of Good Hope. Africa was much bigger than this map indicates, and the invisible Americas were very much in the way of a direct sea route to India. We know that the ancient Greeks had discovered that the world was round, and circa 1400 AD Ptolemy had produced an accurate map of Europe and Asia based on a spherical Earth. But the Ebstorf map follows the Roman tradition of placing landmarks in relative positions, maintaining basic order and structure but not following rules for measurements or Cartesian accuracy. And the T-O structure of this map—the base of the T was the Mediterranean, the cross was at Jersualem; the O formed the surrounding oceans—was an idealized depiction of the world that was all too common in the late medieval era.

Despite its shortcomings as a navigational aid, the Ebstorf mappamundi is a beautiful map. It is both a sacred object glorifying the Body of Christ and also a tourist map of the strange and wonderful places that formed the background of medieval storytelling.

References:

• Art and Cartography by David Woodward
• The Ebstorf Mappamundi by J. Siebold

Markets are People, published 1930 by Printer's Ink

“Markets are People” is a beautifully drawn map. It uses area-distortion to show population at both the state and city level. This is the first map we have seen that uses the area-distortion method at two resolutions. The effect is visually accurate, highly readable, and offers deeper insight into the data than similar maps that offer only one level of detail. (For comparison, look at modern cartograms on Wikipedia.)

Notice how easy it is, for example, to estimate the percentage of urban population in any given state: Illinois is about 50% urban. Relative sizes of cities are also easy to estimate.

This map was found in Graphic Presentation, published in 1939 by Willard C. Brinton, p. 242. The caption under the map reads:

Printer’s Ink Publishing Co., Inc. Chart by Walter P. Burns and Associates, Inc. New York City.

A Distorted Map of the United States Showing Population of Each State and Cities of 50,000 or More in 1930.

The presentation of cities whose areas are proportional to their population is the outstanding feature of this map.

The entirety of Graphic Presentation is worth exploring. It covers a full range of topics from color theory to line graphs and “quantitative cartoons”. The book is out of print, but we found a wonderful leather-bound copy at the university library. A PDF of the full text is also available.

Thanks to Datavisualization.ch and Flowing Data for pointing us to this book.

Color sorting using RGB, HSV, and YIQ color spaces

We recently needed to sort the pixels in an image by their color. On first attempt, we sorted by simple RGB values: first sort on R, then on G, then B. But it turns out there are many other possibilities for sorting colors. Every permutation of the RGB letters could be considered (RBG, GBR, GRB, etc.), not to mention other color systems HSV and YIQ.

To see all the color sorting possibilities, we wrote a Python script to try every permutation against a common set of source images.  Click here to see every color sorting option. Note that we use a Hilbert curve in the top set of images to provide better grouping of similar color values. The bottom set of images is linear and wrapped: left to right, then top to bottom.

Our goal was to find a sorting mechanism that looked pleasing to the eye, but of course the results are completely subjective.

• For aesthetics, we prefer the YIQ sort. It highlights trends in value change, but information about hue variation is hidden and sometimes invisible.
• To communicate more information about hues, HSV sorting is by far superior, but it lacks the pleasant gradient-effect of the YIQ sort.
• RGB sorting proves inferior to HSV and YIQ, though the results can still be interesting.

Also worth noting: HSV sort is excellent for grouping colors of similar hues, but the IYQ and YIQ sorts also do this well and they can look cleaner than their HSV counterpart.

(with thanks to Steve Witham for his Hilbert curves in Python.)

Diagram of US airways using Holten and van Wijk's bundling algorithm

We recently encountered (via infosthetics) a paper from Danny Holten and Jarke J. van Wijk that demonstrates an algorithm for creating beautiful graph visualizations. The original paper is Force-Directed Edge Bundling for Graph Visualization, published in 2009.

In computer science theory, a graph is a collection of nodes and the edges between them. Graphs are useful abstractions for describing transit networks, computer networks, and relationships among people (such as in a social network). Over the years mathematicians and theorists have discovered that there are hundreds of additional problem types that can be rewritten as graph theory problems. So, understanding and visualizing graphs is important for a large number of problem domains.

While there are many ways to represent a graph visually, the most intuitive is to draw a set of points (nodes) with lines (edges) connecting them. The goal, of course, is to make the visualization intuitive to understand for a human observer. The visualization should be functionally readable, aesthetically appealing, and ideally should allow an observer to glean interesting information about the graph’s structure. For larger graphs, several issues regarding visualization immediately present themselves. First, nodes must be laid out such that they to not interfere with one another. Edges should be drawn so that they can be distinguished from one another, with minimal crossing, but should also connect two nodes with the shortest line possible and minimal curvature. Node size and edge thickness become important, and either nodes or edges are connected to real-world space, they should be placed in some analogous fashion.

Unprocessed visualization of a graph representing major US airways

Holten and van Wijk have introduced several novel ideas to graph visualization theory. Their main contribution is the grouping of similar edges using a clustering technique. Edges become flexible springs that can attract and repel one another. While other researchers have proposed edge-bundling techniques, Holten and van Wijk’s algorithm uses a “self-organizing” approach that does not require additional information in the form of control hierarchies or meshes as did previously published algorithms. By transforming the edges into springs, they are able to leverage existing algorithmic techniques for modeling real world systems.

Algorithm

Each edge is modeled as a spring, and a number of control points are placed along each edge. Each control point on a spring-edge is compared with its corresponding control point on every other edge that interacts with this edge. A global spring constant is used to inform localized (edge-specific) spring constants, and this determines the “stiffness” of the resulting diagram. Over many iterations, each control point is moved slightly according to the sum of all forces acting upon it. As control points move, they inform subsequent iterations and alter the forces acting on other edges in the graph. The number of control points is increased for later iterations to provide greater edge smoothing.

The authors discovered that this basic algorithm resulted in too much bundling. They therefore introduce several edge-compatibility measures that are used to adjust the forces acting between any two control points. Edges should exhibit greater bundling tendencies if they:

1. are parallel rather than perpendicular
2. are of similar lengths
3. are proximate in space
4. have similar “bands of sight” (such that the edges of a skewed parallelogram would not be considered candidates for bundling).

Additional anti-aliasing was performed on the final output to increase overall smoothing. The resulting graph diagrams are beautiful and very readable.

Everything In Its Right Place

Kid A

National Anthem

How To Disappear Completely

Treefingers

Optimistic

In Limbo

Idioteque

Morning Bell

Motion Picture Soundtrack

Visualizations show relative pitch strength across the duration of the song. Colors scheme is derived from the Kid A album cover. Song analysis by Echo Nest via the Echo Nest Remix toolkit. Rendering was done with PyCairo.

We built a tool to map your Twitter followers. It’s called Tweemap.

It is still a little rough around the edges, but the service is very functional. Tweemap shows up to 10,000 followers for any given user. The Google Maps Javascript API provides core mapping utilities, and the Twittervision API was used for geocoding.

How it works

The main problem we encounter building Javascript maps is performance. Javascript is getting faster and better all the time, but compared to a desktop mapping solution, or even to Flash, it is quite slow and requires a delicate touch. Our approach was to use a series of aggregation techniques to group similar data points and thereby limit the amount of rendering required by the browser.

When geolocating Twitter users, we group them based on latitude/longitude coordinates. These point groups (PGroups) definitely contribute to efficiency. For example, we are able all users—potentially hundreds or thousands—in a major cities as an individual entity. PGroups are fed into a larger point collection (PCollection) class, which manages the groups and renders graphics onto the map.

At each zoom level, the PCollection builds a uniform grid across the surface of the map. Grid cells maintain a constant pixel width and represent a “square” of lat/lon coordinates. Each PGroup that is visible within the current map viewport—any not within the viewport is temporarily discarded—is placed into its corresponding grid cell. The grid is passed to the PCollection’s render() method, which creates and places circles on the map to represent the total count of followers in that cell. If the map is moved, the grid is extended in the appropriate directions, and PGroups are likewise placed into the new sections of the grid. By merely extending the grid, rather than recreating it, we can cache any existing icons and save a full redraw cycle, while at the same time preventing the map from going temporarily blank.

Icons are scaled against other icons in the viewport, so smaller icons become larger when they are the only icons in view. We also cap the max icon size so that circles don’t become so cluttered as to make the map unreadable.

We had considered using CloudMade’s custom map tiles for this map, but having run into several issues with their map style editor, we resorted to Google and its powerful API. One thing we didn’t like about Google’s maps is that at higher zoom levels it became very difficult to see our circles against the backdrop of Google’s imagery. To solve this problem, we created a new Google Maps layer that sits atop the standard layers to darken the map and make our icons visible. This new map layer is quite dumb, for every tile it serves a static PNG with limited transparency. This kind of layer can be created as follows:

var modLayer = new GTileLayer( new GCopyrightCollection(""), 0, 15 );
modLayer.getTileUrl = function( tile, zoom ) {
    return image_root + "mod.png"; // a darkened, semi-transparent PNG
}
modLayer.isPng = function() { return true; }

// group the new "darkened" layer with the terrain map
var layerTerMod = [ G_PHYSICAL_MAP.getTileLayers()[0], modLayer ];
var mtTerMod = new GMapType( layerTerMod, G_PHYSICAL_MAP.getProjection(), "Dark" );

// add the darkened map type to the map
map.addMapType( mtTerMod );
map.setMapType( mtTerMod );

We have not compressed the Javascript source for this project with the hope that it might be useful to someone else. Feel free to dig around.

Now, have a look at Tweemap.

Steve Reich - "Pulses"

Steve Reich’s Music for 18 Musicians. The first section, “Pulses” is visualized above using custom Python scripts. Analysis of the music is provided by the Echo Nest API.

See also, Song Visualizations with Echo Nest.

Salavon's photograph series Homes for Sale. Clockwise, from top left: Seattle, Miami, Los Angeles, New York City, Dallas, Chicago.

We find the work of Jason Salavon highly inspirational. At the intersection of art and visualization, his images are distinctive for their surface and immediate graphical language while also communicating significant stories of process and history.

By merging dozens or hundreds images that share some common theme, Salavon helps us extract high-level information about common form, texture, and color. The details are sacrificed, but we gain understanding of patterns inherent in the larger series.

In the Homes for Sale s images, aggregation is done by place and time. The photographs, all from 2002, were taken by realtors. We might expect different results in different cities, years, or seasons. As it is, we can see the green grass of Dallas contrasting with the yellow lawns in Chicago, and the blue skies of Miama and Los Angeles stand out against the gray atmospheres in Seattle and New York.

Every Playboy centerfold, by decade. From the left: 1960s, 1970s, 1980s, 1990s.

In the Playboy series, aggregation is achieved by limiting the kind of image (only centerfolds) and differentiating by decade. We see artifacts of culture in composition and colors, warm to cool, with lighter skin and hair emphasized in later decades.

Hiroshi Sugimoto, Akron Civic Theater, Ohio (1980)

We can think of Salavon’s photos as 2D films, where all frames are merged into one static image. Similar work has been done by the brilliant photographer Hiroshi Sugimoto, who’s camera has captured the entirety of feature films with one long shutter (above). Sugimoto does not wish to tell us about the film itself—in fact, the film title is never announced—but rather about qualities of light, surface, shape, form, and so forth.

Most data visualization projects start with structured data, spreadsheets, XML, databases, APIs. We like Salavon’s projects that bridge data visualization with more traditional media, using a raster imagery as the data sources, extracting patterns and meaning from highly unstructured content. They are beautiful in their own right, and serve as an inspiration for future data visualization projects.