Stuff happens, and people tweet about it. Something major happens, and a lot of people tweet about it. Masters student Stanislav Nikolov and his adviser Devavrat Shah are working on ways to algorithmically detect the latter.

People acting in social networks are reasonably predictable. If many of your friends talk about something, it's likely that you will as well. If many of your friends are friends with person X, it is likely that you are friends with them too. Because the underlying system has, in this sense, low complexity, we should expect that the measurements from that system are also of low complexity. As a result, there should only be a few types of patterns that precede a topic becoming trending. One type of pattern could be "gradual rise"; another could be "small jump, then a big jump"; yet another could be "a jump, then a gradual rise", and so on. But you'll never get a sawtooth pattern, a pattern with downward jumps, or any other crazy pattern.

And with that, the algorithm compares current patterns to the ones above. If they look like a trending pattern, the algorithm marks something as a trend with some probability. In testing with past trending topics, the algorithm was able to pick correctly over 90 percent of the time.

The best part is that this method can be applied to other time series data. "We can try this on traffic data to predict the duration of a bus ride, on movie ticket sales, on stock prices, or any other time-varying measurements."

When you go to a conference, there are typically several talks going on at the same time, and you can always tell there's a popular paper coming up when you see people leave a bunch of rooms at once and head straight into one. There's also the unfortunate case when someone speaks, and there's only a handful of people in the room, all in the back staring at their laptops. Open Data City visualized this activity during the German internet conference re: publica.

Open Data City used MAC addresses and access point connections to keep track of where devices went. So a person might be in a room connected to the nearest access point, disconnects as he leaves, and then reconnects as he reenters another room, which provides the flow.

It's fun to watch the conference play out even if you didn't attend. Each dot represents an attendee, and as the animation plays the dots migrate from room to room. Click and drag over the dots to select specific people. [Thanks, Michael]

What does this mean for statistics as a discipline? Well it is great news in that we have a lot more people to train. It also really drives home the importance of statistical literacy. But it also means we need to adapt our thinking about what it means to teach and perform statistics. We need to focus increasingly on interpretation and critique and away from formulas and memorization (think English composition versus grammar). We also need to realize that the most impactful statistical methods will not be used by statisticians, which means we need more fool proofing, more time automating, and more time creating software. The potential payout is huge for realizing that the tide has turned and most people who analyze data aren't statisticians.

Yep.

Those who disagree tend to worry what might happen — what kind of data-based decisions will be made — by non-statisticians, and that should definitely be a priority as we move forward. Non-statisticians often make incorrect assumptions about the data, forget about uncertainty, and don't know much about collection methodologies.

However, as a statistician (or someone who knows statistics), you can shoo everyone else away from the data and gripe when they come back, or you can help them get things right.

Curious about how people use "geek" and "nerd" to describe themselves and if there was any difference between the two terms, Burr Settles analyzed words used in tweets that contained the two. Settles used pointwise mutual information (PMI), which essentially provided a measure of the geekness or nerdiness of a term. The plot above shows the results.

In broad strokes, it seems to me that geeky words are more about stuff (e.g., “#stuff”), while nerdy words are more about ideas (e.g., “hypothesis”). Geeks are fans, and fans collect stuff; nerds are practitioners, and practitioners play with ideas. Of course, geeks can collect ideas and nerds play with stuff, too. Plus, they aren’t two distinct personalities as much as different aspects of personality. Generally, the data seem to affirm my thinking.

Or maybe pop culture (geek) versus education (nerd).

It's just metadata. What can you do with that? Kieran Healy, a sociology professor at Duke University, shows what you can do, with just some basic social network analysis. Using metadata from Paul Revere's Ride on the groups that people belonged to, Healy sniffs out Paul Revere as a main target. Bonus points for writing the summary from the point of a view of an 18th century analyst.

What a nice picture! The analytical engine has arranged everyone neatly, picking out clusters of individuals and also showing both peripheral individuals and—more intriguingly—people who seem to bridge various groups in ways that might perhaps be relevant to national security. Look at that person right in the middle there. Zoom in if you wish. He seems to bridge several groups in an unusual (though perhaps not unique) way. His name is Paul Revere.

You can grab the R code and dataset on github, too, if you want to follow along.

A few years ago I downloaded speed dating data from experiments conducted by Raymond Fisman, et al. (2005), which represents about 8,000 dates by 551 people. On each date, people scored each other on attractiveness, intelligence, ambition, and some other things, along with a yes or a no to seeing the other person again on a regular date.

Fisman, et al. noted gender differences in mate selection, such as: "Women put greater weight on the intelligence and the race of partner, while men respond more to physical attractiveness." And this: "Men do not value women's intelligence or ambition when it exceeds their own." Seemed like data worth checking out.

(Side note: Do people even speed date anymore?)

Three sections:

1. How the speed dating worked
2. What women and men want in a partner
3. Dating up the social ladder

How the speed dating rounds worked

In case you're unfamiliar with the speed dating process, here's how it works. There are two groups. Typically one group is women and the other is men. The point of it all is to match every woman with every man for a short period of time so that by the end, every one has gotten a chance to quickly know each other. The assumption is that you can learn a lot about a person in a short period of time.

In these speed dating sessions, the women stayed seated, and the men shifted each round. The pairs chatted for four minutes and then the men shifted again.

People scored each other on a 1-to-10 scale and indicated whether or not s/he wanted to date the other. So a few things can happen:

• Man wants to date woman, but woman is not interested.
• Woman wants to date man, but man is not interested.
• Both are not interested.
• Both are interested, so information is exchanged.

This also presented interesting dating styles. I won't go too in depth here, but it's fun to take a quick look.

Some people said yes to almost everyone, casting a wide net, whereas others were more stingy with their yeses. Some got a lot of yeses but only returned the favor a couple of times. Some people were really likable and ended up with a lot of mutual yeses.

For example, here are the one-way connections for the first dating session:

These are the mutual connections from the same session:

What women want vs. what men want

So what made one person more dateable than another? We can look at the pre-date surveys that asked others what they looked for in a partner and what they thought the opposite sex looked for. It was a 100-point scale, and participants were asked to divide those 100 points between attractiveness, intelligence, fun, sincerity, ambition, and shares the same interest.

The chart on the right compares the medians of what women said they want and what men said they want.

Women weighted the attributes more evenly than the men did, with intelligence on top and ambition on the bottom. In contrast, men weighted attractiveness more heavily. Ambition was also weighted lowest by the men but a few points lower, which matches the results in the paper.

There's nothing unexpected here. Although I thought sharing the same interest would be higher.

The contrast between what one group says it wants versus what the opposite thinks the other group wants is interesting. For example, women think men place attractiveness much higher in priority at the expense of intelligence and sincerity. And men think women actually weigh attractiveness more highly, also at the expense of intelligence and sincerity.

This is just what people said they wanted though. Is that what they actually wanted? As you might expect, the higher the ratings for all attributes, the higher the yes rate (the proportion of daters who said yes at the end of a round).

The trend is most clear with attractiveness and fun, which are easier to judge than the others in four minutes. The yes rates kind of level off for ambition and sincerity towards the higher ratings.

Look at intelligence though. There was a slight drop in yes rate when someone was rated with a 9 in intelligence by their peers. I suspect this was partially due to the relatively low number of people with this rating (only 26 of them), and the small group of high-intelligence people collectively had lower attractiveness ratings.

The trends are roughly the same when you split the results by gender. Although I would have expected women's yes rates towards men to continue upward given women ranked intelligence higher than attractiveness. Instead, that's how the men's yes rates towards women look.

Dating up (and down) the social ladder

We see this in sitcoms and movies all the time. There's a character who is less (traditionally) attractive interested in someone more attractive. His or her friend who is a genius in relationships launches into a speech about how said character has no chance because he or she can't date up the social ladder. Some might say s/he is undateable.

How does this "rule" pan out?

In the previous distributions, people got higher yes rates when they were rated more attractive by their partners. Flip this around. The more attractive someone was, the more selective they got. It's like the dating pool decreased for an individual the more attractive s/he was.

This doesn't stop people from trying though.

We only really see the change in selectivity with attractiveness (and kind of with fun) when you look at the full distributions, but we see a little more when we compare dating up versus dating down. As shown below, for every attribute, the median yes rate was higher when daters scored their partners higher than themselves. For example, the yes rate was much higher given a dater thought the partner was more fun than her or him.

Again, the difference is most obvious with attractiveness and fun, which makes sense because those are easier to judge in four minutes. You can see the wider spread between the points. However, there's still a spread for intelligence, sincerity, and ambition.

You can also see that the women were more selective than men. It's hard to say from the data alone if this is because the women were actually more choosy, because the men were less desirable, or a little bit of both. I'm guessing it's the women being more selective.

If we go back to the pre-date survey, the actual dating for men is similar to what they said was desirable in a partner. For women though, the speed date results are fairly different from their pre-survey responses. Again though, I suspect the difference comes from the challenge of judging a person in four minutes. Or not. If the former, speed dating seems better suited for men, and if the latter, well, I'm not sure what to do with that, so I'll let the ladies weigh in.

Back to the original findings in the paper. It looks like women do put slightly more weight on intelligence than men, and men put slightly more weight on attractiveness. However, the chart above seems to go against the results that men don't value women's intelligence or ambition when it exceeds their own. If it didn't matter, the yes rates for less ambitious and more ambitious would be near equal. I'll have to dig a little more into the discrepancy, but I suspect we might see something closer to the results when you control for the other variables (mainly attractiveness).

In any case, it's definitely not a straightforward decision.

Another way to look at it is that we don't see any yes rates of zero in the chart above. At the end of the day, even if you are less attractive, less intelligent, less fun, and less ambitious, just remember: There's still a chance.

Damien Hirst is an artist known for a number of works, one of those being his large production of spot paintings. There are over a thousand of them painted by him and his assistants, varying in size, number of dots, density, and color. Amanda Cox of The New York Times plotted paintings sold from 1999 to present, topping out at $3.4 million. That's a whole lot of dottage.

"As a law-abiding resident of this nation, I have the right to do whatever I want without a shadowy organization recording my every move, unless of course it's part of an electronic campaign designed to figure out, based on all of my emails and phone conversations, what types of clothes, shoes, and houseware products I like. Then it’s fine." Sources later confirmed that Landler had posted a Facebook rant on the issue, which had generated a pop-up ad from a company that restores lost PC data.

It seems like the technical side of map-making, the part that requires code or complicated software installations, fades a little more every day. People get to focus more on actual map-making than on server setup. Map Stack by Stamen is the most recent tool to help you do this.

We provide access to different parts of the map stack, like backgrounds, roads, labels, and satellite imagery. These can be modified using straightforward controls to change things like color, opacity, and brightness. So within a few minutes you can have a map of anywhere in the world with dark green parks and blue buildings. You can get very precise with image overlays and layer effects, using layers as cut-out masks for other layers. Or just make a regular-looking map in the colors you want.

The idea is to make it radically simpler for people to design their own maps, without having to know any code, install any software, or even do any typing.

It's completely web-based, and you edit your maps via a click interface. Pick what you want (or use Stamen's own stylish themes) and save an image. For the time being, the service is open only from 11am to 5pm PST, so just come back later if it happens to be closed.

See here for a taste of what others have done so far.

OpenStreetMap, the free wiki world map that offers up high quality geographic data, has grown a lot in the past eight years. The OpenStreetMap Data Report shows all these changes. Says the report: "The database now contains over 21 million miles of road data and 78 million buildings."

There are various parts to the summary, but the star is clearly the map of edits. Green indicates an older edit, and white indicates more recent edits. Blue and pink represents everything else in between. Above shows the global overview, but it gets most interesting when you zoom in on cities.

New York:

London:

Activity in Tokyo is bustling:

Be sure to scroll to the bottom of the report to see live updates to the map.