Graham King

Treating the common cold

Graham King — Wed, 27 Jan 2010 06:59:08 +0000

Will Vitamin C really prevent or cure your cold?

What about Echinacea?

The best way to find out is to follow these simple steps:

Gather 2000 people who have the common cold

Split them randomly into two groups of 1000

Give vitamin C to one group, and a sugar pill (the placebo) to the other

Make sure the people receiving the pills and those dispensing the pills don’t know which is which (to make your trial double-blind)

Wait a bit

See if the Vitamin C group gets over their colder faster than the other group

You will of conducted a double-blind placebo controlled scientific trial.

Luckily for us, we don’t have to do it ourselves. Several of these trials have already been done. Enough trials, in fact, that one can do a Meta analysis, a statistical review and summary of all the trials.

A comprehensive meta analysis, by an unbiased organization, is the gold standard of scientific inquiry; it is our best chance of knowing the truth.

The Cochrane Collaboration is an international not for profit organisation set up 15 years ago to create transparent, systematic, unbiased reviews of the medical literature on everything from drugs, through surgery, to community interventions. And I have a cold. I read their review to find out whether Vitamin C or Echinacea would be effective in treating it.

Vitamin C

Here is the review of research on Vitamin C for preventing and treating the common cold.

Thirty trials involving 11,350 participants suggest that regular ingestion of vitamin C has no effect on common cold incidence in the ordinary population.

It reduced the duration and severity of common cold symptoms slightly, although the magnitude of the effect was so small its clinical usefulness is doubtful.

Nevertheless, in six trials with participants exposed to short periods of extreme physical or cold stress or both (including marathon runners and skiers) vitamin C reduced the common cold risk by half.

The failure of vitamin C supplementation to reduce the incidence of colds in the normal population indicates that routine mega-dose prophylaxis is not rationally justified for community use. But evidence suggests that it could be justified in people exposed to brief periods of severe physical exercise or cold environments.

Echinacea

Here is their review of research on Echinacea for preventing and treating the common cold.

Echinacea preparations tested in clinical trials differ greatly. There is some evidence that preparations based on the aerial parts of E. purpurea might be effective for the early treatment of colds in adults but the results are not fully consistent.

Beneficial effects of other Echinacea preparations, and Echinacea used for preventative purposes might exist but have not been shown in independently replicated, rigorous RCTs.

Get well soon!

Predictably Irrational, by Dan Ariely

Graham King — Sat, 16 Jan 2010 07:12:43 +0000

My short notes on Predictably Irrational, by Dan Ariely. An excellent book. Entertaining, and covers much fascinating ground from social psychology and behavioral economics. Some of the experiments Dan and his team designed are fiendish!

Value is relative

We only know what we want when we see it in context. The bike the Tour de France winner rides. A set of speakers compared to another.

We only know what something is worth, or how much we like it, when comparing to other similar things (purchases, partners, jobs, etc..

We tend to choose the middle option. A high price option on a restaurant menu increases average order price, because it makes the rest seem cheap in comparison.

If we have several difficult to compare items, but two are easily comparable and one of those two is clearly superior to the other, we will prefer that one to all the other options. We can introduce a ‘decoy’ version of one item, slightly less appealing, so that the target item is chosen. In A v B, introduce -A, and most people will choose B.

Video: The truth about relativity

Arbitrary coherence

When we seriously contemplate buying something at a given price, that price becomes our anchor. From then on we judge other similar items relative to that price. We are coherent with our first valuation of something, even if that price is now irrelevant, arbitrary.

Items on sale attempt to get you to anchor at the ‘pre-sale’ price, before introducing the sale price.

Video: Starbucks, Dunkin Donuts, and Anchors

Herding

Joining a queue because other people are in it. Preferring a busy restaurant. Assuming something is good (or bad) on the basis of other people’s behavior.

Self-herding is when you do what you did before, because, well, that’s what you do. (See also: commitment)

We are irrationally attracted to FREE(!) things. We often pay a high cost in time and opportunity cost for them. Discounting from cent to free has a much higher influence than 2 cents to 1 cent. Free is a whole different place than something that has a cost.

Social Norms vs Market Norms

Pay someone cheap, they don’t work much, because they use market norms and don’t feel it is worthwhile. Pay someone nothing, they work hard, because they use social norms.

Gifts stay within social norms, unless you mention the price of it, in which case people revert to market norms.

Thinking about money makes people more self-reliant, selfish, solitary, individualistic.

Counting money is a pain buffer, makes people feel more powerful.

If both are present, market norms displace social norms, and it is very difficult to get the social norm back.

With gifts, benefits, and a good atmosphere, companies try and create a social contract with their employees. The employees hence expect the other side, the company to be there for them when they needed. Social norms require a high level of commitment from the company. Google and lots of startups get this right.

Video: The Cost of Social Norms

Emotions

When not experiencing the emotion, we are bad at predicting our responses in a strong emotional state, such as sexual arousal, anger, hunger, excitement, jealousy, etc.

It is not easy to look from one emotional state to another.

Most people are aware that they procrastinate. Giving them tools to commit to a deadline helps them overcome it.

The best remedy for procrastination is an external force: Automatic payroll deductions for saving, teacher imposed assignment deadlines, exercising with a partner, etc.

Engineer the world to force you to achieve your goal.

Simpler schedules are easier to commit to and more likely to be followed.

Package actions up into something that feels important enough to avoid procrastination. The same way auto-dealers package up our servicing into large mileage increments, and do all of it at once.

Video: The Influence of Arousal

Endowment effect

We value what we own more than other people do. Sellers usually value their items more than buyers do.

The sellers focuses on the loss of the enjoyment of the item. The buyer focuses on alternate uses for the money. Seller and buyer expect the other to see things the same was as them.

The more work you put into something, the more ownership you feel. The Ikea Effect, because you assembled it yourself.

The endowment effect is why companies offer ‘trial’ rates and ‘money back guarantees’. Once we have something, own it, we value it more. A ‘virtual ownership’ applies to trying on clothes, and especially to online auctions.

The endowment effect applies to ideas too. We don’t want to let go of them, so they become a rigid ideology.

People dislike losing options, and will work quite hard to keep all their options open, eve if that means occasionally ignore their best option.

Often, we are presented with two or more very similar options. In this case the biggest cost is that of not choosing – the time spent deciding is time we are enjoying neither option.

Life usually conforms to our expectations, so you can influence results by setting expectations. Marketing sets expectations.

Video: Dating, doors, and loss aversion

Priming

Priming is putting thoughts into someones head.

Often, an experiment will ask people to unscramble a word, that word is chosen to make them think the priming thought.

Priming is a context effect: Being influenced by something irrelevant in our immediate environment.

Two parties in a conflict, having already strongly committed to one side, need a neutral third party to resolve their conflict.

Video: The Effect of Expectations

Placebos

Placebos are belief and conditioning. Many, maybe most, surgical procedures are not placebo tested.

Honesty

We internalize social virtues, so our conscience keeps us broadly honest.

Given the opportunity, most people will cheat a little bit. A greater potential and reward for cheating doesn’t increase the cheating. Being reminded of the Ten Commandments, or of honor systems, eliminates the cheating.

People are more likely to cheat for non-monetary items – steal a pen from work, but not the value of the pen from the petty cash box.

It seems that people will only cheat within the limits their conscience (super-ego) allows them. We cheat to the extent we can justify it to ourselves.

“It is difficult to get a man to understand something when his salary depends upon his not understanding it” – Upton Sinclair

Video: The Context of our Character

Dan Ariely maintains an excellent website with links to research papers from the book, and much more, here: http://www.predictablyirrational.com/.

Influence, by Robert Cialdini

Graham King — Wed, 16 Dec 2009 06:42:09 +0000

As an Amazon reviews says, “arguably the best book ever on what is increasingly becoming the science of persuasion.”

If you want to understand why you felt compelled to give money to a Hare Krishna devotee, how car salesman or realtor’s work, and much more, you should read this.

It’s also a very easy and enjoyable read. These are my notes. They cover all the content in the book, but don’t link to research. In the book, most of the statements have links to research papers to back them up.

Get Influence: The Psychology of Persuasion from your local library, this has sold so many copies they are bound to have some.

Heuristics

We can process incoming information cognitively in one of two ways:

Controlled responding, which is subjecting information to a thorough analysis. This is when we think a problem through, research it, etc. We only do this if we have the desire and the ability. It is intellectually taxing and time consuming.
Use judgmental heuristics such as:
- Price as surrogate for value. Applies particularly to items which are hard to value: Wine, jewelry, art, employee salaries, etc.
- Trust experts. This is why pseudo-science books always have ‘PhD’ or ‘MD’ after the author’s name.
- Because – we want reasons to do something, even bogus ones.

Contrast principle: Two different things presented together or sequentially will feel more different than they really are. Hence sell the expensive item first, as the other items will seem cheap after that. Good cop / bad cop is about creating a contrast between the two, to increase liking of the ‘good cop’.

A fixed action pattern is an automatic behavior pattern often triggered by a single item in the information. Also known as a heuristic.

Reciprocity

A Hare Krishna devotee presses a flower or a copy of the Bhagavad-gītā into your hand (that happened to me), or a store gives you free samples, and you feel awkward about taking it for free. You end up giving money or buying something you don’t want. That’s the norm of reciprocity at work.

We also reciprocate to concessions. Start witha large but not unreasonable request, which gets rejected, then ‘retreat’ to the target request. People who comply after this technique has been used on them feel more satisfied with their decision than those who agree straight away, because they ‘bargained’ for it. They shaped the outcome, so they feel ownership for it.

A recent research article I read (apologies, can’t locate link right now) supports this, by saying that when the seller of a product is the first to quote a price (typically quite high), the product sells for more than when the buyer opens the negotiation (with typically a low price).

Surprise increase the likelihood of the target using heuristics instead of controlled responding. A shaved headed person in orange robes shoving an ancient Indian text at you, that’s surprising.

Consistency / Commitments

We wish to think of ourselves, and appear to others, as being consistent in our behavior, we apply the consistency principle heuristic. We make up reasons to justify our behavior, to seem logical and in line with our previous behaviors.

Foot in the door technique: Start small and build. Extract a small inconsequential commitment, then a slightly bigger one, and so on.

Commitments are most effective when they are:

Public: Forces us to stick to it to look consistent. Especially effective for those whith lots of pride or public self-consciousness.
Active: Perform some action to take the commitment, such as writing, swearing an oath (also public), etc.
Effort-full: The bigger the price the bigger the commitment. Hazing / initiation ceremonies. Extract a bigger effort with foor-in-the-door technique.
Freely chosen:Avoid threats, or any external reward such as prizes. The commitment must feel freely chosen for the person to accept responsibility for it, it must feel internally motivated.

Caveat: If the commitment alone gets the same result as the goal, we may not pursue the goal. For example if you are going to attend law school so that people will think you are clever, and telling people you are going to attend gets the response and recognition desired, the motivation to actually attend is removed.

Commitments get internally justified, they grow their own legs. Car salesman will often low ball you, offer an artificially low price.
You gladly accept the low price. Time passes as you fill in some forms, test drive, etc. They take the paperwork to their manager for final approval. Manager comes out and explains the salesman made a mistake, the price is higher.
You would of refused the higher price originally, but now you are committed, and much more likely to accept the high price.

Social Proof

The greater the number of people who find any idea correct, the more a given individual will perceive the idea to be correct. This is the Social Proof heuristic.

We look to others for behavioral guidance when we are unsure, in an unclear, unfamiliar, or ambiguous situation. Often in such situations others do the same thing, which leads to pluralistic ignorance.

In an unclear emergency situation, a lone bystander will help, but a group looks to each other to decide how to act, and does nothing. This is the Bystander effect.

In an emergency situation you need to reduce the uncertainty experienced by bystanders:

Pick one and speak directly to him
Say you need help
Tell him how to help (”Call an ambulance”, “Help me out of my car”, etc)

Social proof is strongest when the others we look to for guidance are similar to ourselves. Hence Joe Average in television adverts.

Individuals in psychological pain look to others in similar situations to see how to deal with it. After a widely reported high-profile suicide, suicides increase, often using a similar method. Copycat crimes work in a similar fashion.

Books, products, feature One Million Sold or New York Times Bestseller on the packaging. Informing someone that many people like him (NY Times readers) have adopted a certain behavior makes them more likely to also adopt that behavior. Testimonials.

Liking

We prefer to buy from people we like. The strength of the social bond is twice as likely to determine product purchase as is preference for the product itself.

So, what makes us like people?

Physical attractiveness

More attractive people, all other things being equal, are:

paid higher salaries
less likely to be sent to jail
more likely to be helped in an emergency
more persuasive
seen as possessing more of the desirable personality traits
seen as having higher intellectual capacity

It’s pretty shocking, I know, but the evidence is plentiful.

Physical attractiveness triggers the Halo Effect, where one positive characteristic dominates the way that person is viewed by others.

Similarity

We like people like us, whether that ‘like us’ is because they share opinions, personality traits, background, or lifestyle. To influence, try and match the target in as many ways as possible:

Dress
Interests
Background
Age, religion, politics, specific habits
Posture, verbal style, mood, etc. See Mirroring.

Compliments

People want to be liked. Be nice, pay compliments, send a greeting card. There is evidence that positive comments make the giver more likable even when the receiver knows the compliment is not genuine.

Contact and co-operation

Positive familiarity. Familiarity caused by contact leads to liking, unless the contact occurs under unpleasant circumstances.

Here is a recipe for creating cross-group hatred: Split people into two groups. Name each one, to increase group identity (creating a minimum group). Wait. Then mix the two groups in competitive actions.

And here’s how to fix it: Successful joint ventures towards common goals.

Association

An innocent association with good or bad things will influence the way people think about that us

Showing a credit card logo to people with positive associations to credit cards (most people) made them tip more, even when paying cash.
Killing the messenger: An association with bad news, used to be bad news for the messenger.
Celebrity endorsments. Sponsorship of the Olympics, or whatever cultural event is popular, even if the connection doesn’t make sense.
Stand attractive people by your product. Their beauty somehow ‘rubs off’ to make your product more appealing.
Hometown team victory makes us feel good, even though we obviously had no part in the victory.
Reductio ad Hitlerum: “Hitler was a vegetarian, so vegetarianism is wrong.”

Taken to extremes this is a personality flaw, showing low self worth, exhibited by groupies, sports fans, etc.

Authority

We trust experts, we obey experts. Often the appearance (symbols) of authority are enough, symbols such as:

Titles: PhD, M.D., Dr., Professor.
Clothes: Medicine is advertised on TV by actors in white lab coats.
Trappings: Car, house, ‘lifestyle’.

The seminal research on obedience to authority, and probably the most famous study in social psychology, is the Stanley Milgram experiment.

Scarcity

Opportunities seem more valuable when they are less available. Scarcity heuristic applies to information and goods.

If something is in short supply, everyone else must of wanted it, so it must be good.
If something is in short supply, we are about to lose the freedom of acquiring it.

Reactance theory says that an attempt to restrict something makes us want it more, and hence assume it is more valuable and desirable than we previously thought.

Things that recently became scarce are perceived as more valuable than things that have been scarce a long time. We react to a freedom being taken away.

Competition for a scarce item increases perceived value of the item, beyond scarcity alone, or social proof alone. Hence: Auctions.

How we know what isn’t so, by Thomas Gilovich

Graham King — Fri, 27 Nov 2009 06:27:59 +0000

By Thomas Gilovich, social psychologist and CSI Fellow, this well written book explains some of the reasoning and deduction errors we make when trying to understand the world, and ways to avoid making those errors.

This is an easy and engaging read, and offers several straightforward techniques to avoid making common reasoning errors. I recommend you look up How We Know What Isn’t So: The Fallibility of Human Reason in Everyday Life in your local library, or get it second-hand from Amazon for less than a posh cup of coffee.

These are my notes / summary of the book.

I. Cognitive determinants of belief

2. Something out of nothing: The mis-perception and misinterpretation of random data

We are predisposed to see order, pattern, and meaning in the world, and we find randomness, chaos, and meaninglessness unsatisfying.
As a consequence we tend to ’see’ order where there is none, and we spot meaningful patterns where only the vagrancies of chance are operating.

Detecting patterns and seeing connections is very useful, leads to discovery and advance. But the tendency is so strong we sometimes do it when there was nothing to spot.

Example of hot hand in basketball. Why?

Availability heuristic: We remember the long streaks, forget the single shots. Can interpret near-misses as evidence of ‘hot’ or ‘cold’ player.
People have faulty intuitions about what chance sequences look like.

See: R. Falk (1981) The perception of randomness. In Proceedings, Fifth international conference for the psychology of mathematics education.

In a coin toss, people expect a near perfect alternation of heads and tails. In a series of 20 tosses, there is a 50/50 chance of getting 4 heads in a row, and 10% chance of six in a row. Clustering illusion.

St Louis Gateway Arch is one of the worlds largest optical illusion. It appears much taller than it is wide, yet height and base width are the same.

Representativeness heuristic: “Like goes with like”. We expect things that go together to look similar. We expect instances of a type (librarian) to look like the prototype (a cliche librarian). Complex effect stem from complex causes. Effects look like their causes: Jagged handwriting means jagged nerves, heartburn comes from hot/spicy foods, etc.

Representativeness usually very helpful. Occasionally mis-applied. The salient (’prominent or conspicuous’) feature of something is what we remember, and apply the representativeness heuristic to that. The salient feature of a random sequence is the even mix of all the outcomes.

Law of averages is, by statisticians, correctly called the law of large numbers. The even mix of outcomes is only true for very large random sample. There is no ‘law of small numbers’.

Stock market: Random walk, but chartists insist they see patterns in the randomness.

In any random distribution (e.g. x,y plots on a graph), there will be a way of segmenting it (along the axes, diagonally, in bands, etc) that will appear non-random. Carving up data after the fact is meaningless. If we think we see a pattern in the data, test that pattern on other independent sets of data.
Unfortunately, for most people hypotheses constructed on one set of data are considered proved by that same data. This is why medical trials announce their goal, the expected outcome, ahead of time.

Once we think we see order in randomness, humans are exceptionally good at justifying it with a post-hoc theory. Improv comedy teaches ‘jump and justify’; say something random then justify it, it’s surprisingly easy.
For example people let to believe they are above or below the average at some task can explain the difference quite easily, even when the experimenters assigned the above/below at random.
Experiments with split-brain patients, where they justify what the other part of the brain just did.

Once a person has (mis)identified a random pattern as a ‘real’ phenomenon, it will not exist as a puzzling, isolated fact about the world. Rather, it is quickly explained and readily integrated into the person’s pre-existing theories and beliefs. These theories then serve to bias that persons evaluation of new information. People cling tenaciously to their beliefs in the face of hostile evidence.

Regression to the mean: When two variables are related, but imperfectly so, an extreme value on one tends to be matched by a less extreme value on the other. A high number roll on the dice tends to be followed by a lower number, and a low number by a higher one. A companies disastrous years tend to be followed by betters ones. Students with exceptionally good grades in high-school tend to do slightly less well in college.

Regression can be understood than in any performance (game, test, financial year, etc) there is a part of talent and a part of chance. An very high score is more likely to be a good student with luck in their favor, rather than an extremely good student with luck working against them, simply because there are more ‘good’ students than ‘extremely good’ students. So a very high score is likely to be followed by a slightly lower one, because it is unlikely to get chance that much in favor two times in a row.

Most people understand regression, but make two mistakes:

Insufficiently ‘regressive’ when making predictions. The ‘next’ value after an extreme one is closer to the average than they tend to predict.
Regression fallacy: Fail to recognize statistical regression, and explain it away with a superfluous and often complex causal theory. Ad-hoc justification where none was needed.

The regression fallacy shapes people (parents and teachers, mostly) perception of the effectiveness of rewards and punishments. A good performance is likely to be followed by a less good one, and a bad one by a less bad one. If the good performance is rewarded, the reward will be perceived as ineffective because the good performance was not repeated. If a bad performance is punished the punishment will be perceived as effective because the bad performance will not be repeated.
That notwithstanding, psychologists have known for some time that rewarding desirable behavior is generally more effective in shaping behavior than punishing undesirable responses.
See: B. F. Skinner (1953) Science and human behavior.

3. Too much from too little: The misinterpretation of incomplete and unrepresentative data

Using empirical evidence as proof: “I’ve seen it happen”, “I know someone who did”, etc. If a phenomenon exists, there must be some positive evidence – ‘instances’ of its existence visible to us. So empirical positive statements are necessary for a belief to be true, but they are not sufficient. We also need to know what goes in the other boxes.

Many of the beliefs we hold are about the relationships between two variables (’takes vitamin C’ and ‘gets better’). Say we were investigating ‘vitamin C megadose’ on cancer patients. We notice some patients get better after taking mega-doses of vitamin C. That prompts us to start investigating. Positive outcomes under treatment are one box, we need to know the other three:

	Takes Vitamin C	Doesn’t take Vitamin C
Gets better	a	b
Doesn’t get better	c	d

If we stop at noticing some patients getting better, all we have is box a. All we have is the ‘illusion of validity’.

For vitamin C to be effective, the probability of getting better after taking it [a / (a + b)] must be higher than the probability of getting better after not taking it [c / (c + d)].

This is often difficult to do intuitively, because box ‘a’ is the most salient. We notice things happening, not things not happening.

We often seek only evidence to confirm our beliefs; we should also seek information to disprove them, and only hold our beliefs if such evidence is not available.

Often the evidence in the other boxes is not available, particularly in evaluating selection criteria. How do we test whether a companies interview process is effective? We are asking the relationship between ‘passes interview’ and ‘performs well at job’. But it’s very nature, we don’t have the data on job performance for people who didn’t pass the interview.

	Passes selection	Doesn’t pass
Performs	a	?
Doesn’t perform	c	?

The only data we have is a and c, we can only compare the success rate of those who pass the selection. If the base rate of success is high (most people could perform well, only the best performers apply, etc), a would be higher than c whatever the selection criteria, so we would spuriously conclude our criteria are good.

It is not possible to be confident about the selection process without the data on those who were not selected.

Furthermore, simply being accepted can give someone a competitive advantage. Being admitted to a better school, getting a research grant, or working with high-performing colleagues will all improve an individuals performance compared to someone of similar initial ability who didn’t get selected.

Effectiveness of public policy is similarly difficult to measure, because we can’t both set the policy and not set it.

Often the lifestyles we lead, the roles we play, and the positions we occupy in a social network, deny us access to important classes of information and thus distort our view of the world. Overcoming that bias is difficult: We must first recognize the existence of a class of information we have not been exposed to, and then accurately characterize what that information is like.

Self-fulfilling prophecies: Have got a lot of attention. For a prophecy to be self-fulfilling there must be a mechanism that translates the expectation into confirmatory action. They often serve to exagerate a belief that holds a kernel of truth.
Thinking a bank is in trouble (usually with good reason), creditors will withdraw their money, and the bank will really be in trouble. Behaving in an unfriendly and defensive manner because you think someone is hostile will often produce that very hostility.

True self-fulfilling prophecies are ones in which a persons belief elicits the very behavior originally anticipated.
A seemingly self-fulfilling prophecy is one which alters a persons world or limits their responses, in such a way to make it very difficult or impossible for that expectation to be proved wrong. If someone thinks that I am unfriendly, they will avoid me, so I will have no opportunity to prove them wrong. If a sports player is thought incompetent, he won’t get to play, so won’t be able to prove himself competent. The continued absence of positive contribution can easily be mistaken for an absence of talent, when it is simply an absence of opportunity.

Negative first-impressions are generally more stable than positive ones, because we keep interacting only with people who created positive first impressions.

4. Seeing what we expect to see: The biased evaluation of ambiguous and inconsistent data

Information consistent with our pre-existing beliefs is generally accepted at face value, whereas evidence that contradicts them is critically scrutinized and discounted. Our beliefs are much less responsive than they should be to the implications of new information. This is a sane and necessary strategy. If a belief has a lifetime of support, it is perfectly valid to be skeptical of evidence that contradicts it.
Well supported beliefs have earned their inertia. We need to be wary of the beliefs that don’t have a solid foundation, such as cultural stereotypes, social norms, and traditions.

Ambiguous information is usually perceived in a way that fits out expectations. We may not even be aware of the ambiguity – it gets resolved before reaching conscious awareness.

Gamblers tend to attribute their losses to outside forces (chance, the referee, a team injury), but their wins to themselves (knew the team well, studied the form). We re-write our history to discount our losses and bolster our wins.

By carefully scrutinizing information that does not fit our beliefs, we can usually find a way of either discounting or re-interpreting it.

Scientists use a set of formal safeguards to prevent their own erroneous thinking affecting their results:

Statistical measures to guard against the mis-perception of randomness
Control groups and random sampling to avoid drawing inference from incomplete or unrepresentative data
‘Blind’ observers to eliminate experimenter bias.
Precisely specify in advance the meaning of various outcomes, and objectively determine those outcomes. These rules are the ‘context of justification’, used when testing an idea. Idea generating is much more open. Science works by flashes of inspiration followed by rigorous testing. The rigorous testing is what differentiates scientific inquiry from everyday life.

Multiple endpoints: By not precisely specifying an expected outcome, we can pick any one and claim it as success. Psychics will use very vague descriptions, so that we can apply them to our lives and perceive them as true. If the subject is fuzzy (’personal well-being’ for example), we are likely to seize upon any subsequent measure of it that fits our initial beliefs.

Variable windows: By not specifying an endpoint to a prediction, it stays ‘open’ until it comes true. ‘Thing happen in threes’ because we could ‘things happening’ after the third one has occured, whenever that may be. The third event defines the time window for our prediction. With a wide enough variable window, beliefs can only be confirmed.

Multi-faceted expectation: For any two sufficiently complex entities, we can produce a mapping of one onto the other that will produce a certain amount of overlap, and allows us to claim they are similar in some way. Often used with the representativeness heuristic.

You have a strong need for other people to like you and for them to admire you. At times you are extroverted, affable, and sociable, while at other times you are introverted, wary, and reserved. You have a great deal of unused energy which you have not turned to your advantage. While you have some personality weaknesses, yo are generally able to compensate for them. You prefer a certain amount of change and variety and become dissatisfied whne hemmed in by restrictions and limitations. You pride yourself on being an independent thinker and do not accept others opinions without satisfactory proof. You have a tendency to be critical of yourself. Some of your aspirations tend to be pretty unrealistic.

If you see yourself in that description, you are not alone. It is multi-faceted, so most people will find the part that related to them the most salient, and the statements are so general (multiple endpoints) that they are bound to ring true. This is how horoscopes work.

One-sided versus two-sided events:

One-sided events are ones that are memorable only when they turn out a certain way: ‘The phone always rings when I’m in the bath’, ‘You wait forever for a bus and three come along at once’, etc. If the phone does not ring whilst you are in the bath, the non-event does not register.
Two-sided events are ones that register regardless of the outcome: Vacations, dates, gambling, buying a stock, etc. Both outcomes stand out from the stream of experience.

In two-sided events, often negative outcomes are remembered better than positive ones, as they require more rationalization to incorporate them into our self-image and understanding of the world.
In one-sided events, we are more likely to remember the ’side’ of events that has meaning to us, which is usually the one that reinforces our world-view. If I believe my dreams are prophetic, I will remember those much more than the other dreams. If I believe that strange things happen during a new moon, I will notice and remember the strange things, not the ordinary things (strange things not happening).

Many one-sided events only ‘exist’ when they are confirmed. If a fortune teller says you will have twins, and you have twins, you remember the fortune teller, link the two events, and form a strong memory. If you have one child, you probably won’t think of the fortune teller at all. And even if you did, the fortune is not contradicted, simple un-confirmed. You could have twins later in life.

One-sided events tend to be temporally unfocused, they have variable windows to be confirmed in. A ‘prophetic’ dream, a fortune, doesn’t have a fixed date, so only the positive outcome is acknowledged.
Two-sided events tend to have fixed windows: A sporting event, a vacation, a job interview. The closed window forces us to acknowledge both outcomes.

In two-sided events, both outcomes produce the same intensity of emotion (winning or losing a bet). In a one-sided event, only one of the outcomes has any emotional weight: It ‘always’ seems to rain when you forget your umbrella, because getting wet has an emotional attachment (hair ruined, clothes soaked, etc), whereas not-getting-wet doesn’t.

Definitional asymmetries: ‘American tourists in London are loud’, ‘I can always spot fake breasts’, ‘You need to hit rock-bottom to bounce back’. All of these are difficult or impossible to disprove: We only notice the loud people, we don’t notice fake breasts that we didn’t spot, and how can you say if someone has hit rock-bottom or not? Definitional asymmetries are all one-sided events.

One-sided events tend to have a ‘normal’ outcome with a high base rate, which we don’t notice because it is part of everyday life (’look, it’s not a full moon’), and an less usual more salient outcome (’o oh, it’s a full moon!’).

All of the above relate to the availability heuristic.

II. Motivational and social determinants of questionable beliefs

5. Seeing what we want to see: Motivational determinants of belief

Endowment effect: We value something more when it is ours. Ownership creates an inertia that prevents many potentially beneficial transactions from occurring.

The endowment effect applied to humans is the Lake Wobegon effect, or Illusory superiority. This is particularly true on ambiguous traits (’intelligence’, ’sensitivity’, ‘idealism’), and less true on more specific traits (’thriftiness’, ‘being well-read’). If a specific definition is given in the question, the above average effect tends to disappear: People are not lying or cheating, it’s just that the first thing that comes to mind when asked an ambiguous question is usually something they are good at, something salient in their lives.

We attribute success internally and failure externally. Thanks to their own resources, they succeeded. Because of the others/the environment, they failed.
Many psychologists hold that we do this to maintain self-esteem. There is also a cognitive explanation: Succeeding at something is at least partly due to our own effort, and thus warrants some internal attribution. Failing at something usually happens despite our best efforts, so often invovles an unfavorable external situation.

People are more likely to believe things they want to believe, but are constrained by objective evidence and the need to construct a justification that would presuade a dispassionate observer. We draw the desired conclusion only if we can muster up enough evidence to support it. It is in this sense that most people think of themselves are objective.
People often do not realize that their selection and interpretation of data is biased by their goals – the data could be interpreted a different way, and there is other data they ignored. They may well be able to justify opposite conclusions on different occasions.

How we ‘filter’ data to ensure it supports our goals:

Seeking only to confirm
Select who we consult. By judiciously choosing whom we consult on an issue, we can increase our chances of hearing what we want to hear.
Amount of information: If the initial results confirm out expectations, we stop looking. If they don’t, we keep looking.

We should not stop at Can I believe this?, but should progress to Must I believe this?.

These are some of the ways we skew the evidence in the world, frame it to support our beliefs. This is healthy. People who can’t frame effectively run the risk of depression.

Beliefs are like possessions: We acquire the ones we think will make us feel good, and cling tightly to the ones we have.

6. Believing what we are told: The biasing effects of secondhand information

Telling a good story: The speaker needs their message to be worthy of the listeners attention. For the listener the interaction must be worthwhile in some way. The message must be understandable (not assume too much knowledge of the listener) and yet not too detailed (not assume too little knowledge of the listener).

Sharpening and Leveling: What the speaker construes as the gist of the message is emphasized, ’sharpened’, where are details thought to be less important are de-emphasized, ‘leveled’.

One results of sharpening and leveling is that we often develop exaggerated or extreme views of people we have only been told about.

One way to make a message more entertaining or seemingly informative is to increase it’s immediacy. Instead of the story happening to a friend of a friend’s colleague, have it happen directly to friend or family member, or even better, you. Often such alterations are intended for the self-aggrandizement of the speaker. It places them closer to center-stage. Other times it is simply an effort to make the story more salient, more vivid and concrete.

Increasing immediacy makes it difficult for the listener to accurately gauge the reliability of your message:

Reducing the hops: We all know that the more hops a message has been through, the less reliable it is. Increasing the immediacy of a message for the sake of entertainment or self-aggrandizement also makes a message seem more reliable than it is.
Changing the origination (’my brother’, instead of ‘my brother’s friend from work’): Your brother might be trustworthy, so we trust the message, but your brother’s friend is an inveterate liar. The original source of the message has been obscured.

Presenting and accepting remote accounts as secondhand is misleading when estimating the prevalence of a phenomenon in the general population. If something happened to lots of your friends cousins, it is happening a lot. If lots of your friends cousins heard a story about someone else’s friends cousin, and passed it on, there might be a single case.

In attempting to be informative, we might level some of the qualifications in the original message. This is often the case when scientific findings are reported in the news media.
Sometimes the facts are stretched to ‘help’ their audience get the message. This results in hysterical public service campaigns, that elicit far more fear than the original risk warranted (drugs, ’stranger danger’, terrorism, etc). The facts are stretched beyond recognition to make a more compelling story.
Parents are often guilty of this distortion in attempting to motivate behavior in their children.

The desire to entertain often creates a conflict for the speaker between satisfying the goal of accuracy and the goal of entertainment. There is often a tacit agreement between speaker and audience that the truth may be stretched: Tabloids and ‘light entertainment’ news shows are granted this permission. “One of the most common sources of such inaccuracy is the dissemination of unfounded or fallacious claims by news and other media organizations that try and entice by their ability to entertain.” The demand for news has been met by and artificial increase in supply.

Plausibility: Inaccurate or fictitious stories are sometimes told and retold because they just seem so plausible, that we let our critical guard down. Our standards for plausibility are often very low, a decent irony is often enough (the creator of the song “Don’t worry be happy” committing suicide, for example, or someone at the patent office resigning because there was nothing left to invent).

Summary: As we have seen in previous chapters, the data from our own experience is often biased and incomplete. In this chapter we saw that data from others is also. It is therefore important to locate unbiased, complete sources of base-rate data (scientific inquiry mainly), and use those to asses how likely it is that our own perceptions or those of our social group are true. If the base rate data and our personal experience concur, then we are likely correct. If they don’t a reliable base rate should be our guide. If the base rate data is to reliable, at least we know that we don’t know.

How to assess secondhand information:

Consider the source: The New York Times or the National Enquirer? A rock star or a researcher? An actor who plays a doctor on TV, or a practicing medical doctor.
Trust facts, distrust projections: Predicting the future is hard, even for an expert. What looks like an exponential curve can turn out to be sigmoidal.
Watch for Sharpening and Leveling.
Be wary of testimonials. One striking human interest story does not tell you anything about prevalence or risk.

7. The imagined agreement of others: Exaggerated impressions of social support

What we believe is heavily influenced by what we think others believe. This is usually a good strategy. However we often exaggerate the extent to which others hold the same beliefs as us. We think we have more social support for our opinions than we really do.

This is the false consensus effect: This is a relative effect. We realize if our belief is in the minority. We underestimate by how much in the minority.

Reasons:

Motivational: A desire to maintain a positive assessment of our own judgment.
Social: We interact with people who agree with us
Context interpretation: We assume that given the same context, everyone will infer the same thing. As we see others in the same context, we assume they come to the same conclusions as us. As a result the false consensus effect is strongest for beliefs we attribute to external factors (buy stock in Ford or Google), and weakest for ones we attribute internally (name your son Jacob or Ian). The context is often ambiguous, and different people resolve the ambiguity in different ways, often without noticing it.

Inadequate feedback from others: People are generally reluctant to openly question another person’s beliefs (Adults, that is. Children do it all the time). Only intimate friends and relatives can be counted upon for honest feedback. More casual acquaintances often side-step the awkwardness of disagreement and thus leave us without essential corrective feedback. Because so much disagreement remains hidden, our beliefs are not properly shaped by healthy scrutiny and debate.

III. Examples of questionable and erroneous beliefs

(not summarized)

IV. Where do we go from here

11. Challenging dubious beliefs: The roles of social science

The real purpose of [the] scientific method is to make sure Nature hasn’t misled you into thinking you know something you actually don’t know. R. Pirsig, Zen and the Art of Motorcycle Maintenance.

To avoid erroneous beliefs it is necessary that we deveop certain habits of mind that can shore up various deficiencies in our everyday inferential abilities. Fortunately, there is reason to believe that these corrective habits of mind are not difficult to develop. Students familiar with the work on errors and biases readily apply the learning to their everyday lives.

Most important mental habit is realizing the folly of trying to draw conclusions from incomplete and unrepresentative data.

Ask: What do the other three cells look like?

Information presented as firsthand is often secondhand or more remote, and from a less trustworthy source. Be sure you know where you information originated before assessing it’s value.

Many of these habits are core to scientific research. Familiarity with that world helps. Gives you valuable exposure to uncertainty and doubt, a healthy skepticism, and the awareness of how hard it can be to really know something with certainty.

Exposure to the ‘probabilistic’ sciences (psychology, economics, social sciences, etc) may be more effective in teaching these habits than the ‘deterministic’ (physics, chemistry, etc) sciences.
Probabilistic sciences deal with phenomena that are not perfectly predictable, and with causes that are generally neither necessary nor sufficient. The death of a spouse is associated with deterioration in health. However not all partners health deteriorates (not sufficient), and people’s health deteriorates for other reasons (not necessary).
To compensate for this lack of determinism, probabilistic sciencists must be aware of statistical regression, sample bias, and the importance of control groups.

Social sciences deal with everyday phenomenon, so it easier for them than physical sciences to make the probabilistic tools of reasoning readily understandable and available.

Social scientists should no longer suffer from physics envy. They cannot match their explanatory power or predictive precision, but specifically because of this, they are better equipped to deal with the messy, complex phenomena of real life. Their tools and process, rather than their content, may turn out to be the social sciences most valuable discoveries.

Memcached: List all keys

Graham King — Tue, 20 Oct 2009 16:50:30 +0000

In the general case, there is no way to list all the keys that a memcached instance is storing. You can, however, list something like the first 1Meg of keys, which is usually enough during development. Here’s how:

Telnet to your server:

telnet 127.0.0.1 11211

List the items, to get the slab ids:

stats items
STAT items:3:number 1
STAT items:3:age 498
STAT items:22:number 1
STAT items:22:age 498
END

The first number after ‘items’ is the slab id. Request a cache dump for each slab id, with a limit for the max number of keys to dump:

stats cachedump 3 100
ITEM views.decorators.cache.cache_header..cc7d9 [6 b; 1256056128 s]
END
stats cachedump 22 100
ITEM views.decorators.cache.cache_page..8427e [7736 b; 1256056128 s]
END

Thanks to Boris Partensky in the Memcached group here

There you go!

Guy Kawasaki’s 10-20-30 presentation rule

Graham King — Thu, 01 Oct 2009 17:41:22 +0000

Funny, practical, and well worth 1 minute 50 seconds of your life:

via the BootUp Labs Blog.

I’m on identi.ca and Twitter

Graham King — Thu, 17 Sep 2009 16:08:35 +0000

I am sharing my thoughts, mainly about web technologies, on identi.ca and twitter.

The nature of the medium means those thoughts will be generally raw and truncated, but timely.

Social psychology in sales copy: Good copy writing

Graham King — Wed, 26 Aug 2009 18:03:35 +0000

I recently received an advert for an investment fund in which, as the amateur social psychologist that I am, I noticed illustrated a couple of psychological principles. The are both covered in the email title:

Last chance to invest in a firm favourite

They are covered again in more detail in this paragraph:

The x y z Fund only launched six months ago, but has already attracted considerable interest. To keep it small and flexible the number of units has been capped at 200 million. Last week they had reached two-thirds of that total and interest is intensifying. In the last two days alone they sold over 6 million units, so it is likely to close very soon.

Social Proof

The first principle is Social Proof. This fund is “a firm favorite” and “has already attracted considerable interest” and already “sold over 6 million units”. In other words, other people think it’s a very good idea to invest in this fund, so you should to.
Many of the top researchers in the field of influence (social psychology, behavioral economics, etc) worked with the Obama campaign, Their last minute get-out-the-vote message was “A Record Turnout Is Expected”. People are more likely to vote if they think other people will vote. The most beautiful part of social proof is that it is undetected – most people will deny that this is the case.

No-one know which fund to invest in, because most of them under-perform the index, and if the stock market really is a random walk, then there is no sane way to invest. It is in exactly that type of ambiguous situation where people look to other people to decide how to react.

Scarcity

We want what we may not be able to have in the future, that’s the Scarcity Principle. This is the “Last chance” to invest in this fund because it is “likely to close very soon”. Fear of losing out on something can be an extremely powerful motivator, and you see it every day when stores have end-of-line or closing-down sales, or when something is offer in ‘limited quantity’.

Invest with your calculator, not your heart

That email is an emotional appeal, not a rational one. Selling a new fund during a recession must be difficult, and there’s no reason to think this one is selling well. The language doesn’t promise anything concrete, so it doesn’t lie. It just influences. And that is good copy writing.

The seminal book on this topic is Robert Cialdini’s “Influence”, which I highly recommend. It’s a New York Times best seller, and Fortune Magazine lists Influence in their “75 Smartest Business Books.” You see what I did there? :-)

Choosing a message queue for Python on Ubuntu on a VPS

Graham King — Mon, 10 Aug 2009 05:05:13 +0000

More and more, my web apps need to run things in the background: Sending email, re-calculating values, fetching website thumbnails, etc. In short, I need a message queue in my toolbox.

Luckily for me, message queues are this years Hot New Thing, so there’s some good options. I looked at RabbitMQ, Gearman, Beanstalkd and StompServer.

I’d like the message queue to play nice with Python, with Ubuntu, and take almost no memory, as I’m on a Virtual Private Server, and I’d like it to stay up forever. I want small and solid.

Summary

	RabbitMQ	Gearman	Beanstalkd	StompServer
Language	Erlang	C	C	Ruby
In Ubuntu?	Yes: rabbitmq-server	Yes: gearman-job-server	No	No, it’s a Ruby gem
Python lib	amqplib	gearman	pybeanstalk	stomp-py
In PyPI?	Yes	Yes	No	No
Memory	9Mb	1.4Mb	0.5Mb	7Mb
Protocol	AMQP	Custom	Custom	STOMP
License	MPL	BSD	GPL	MIT

Memory size is the resident set size, obtained like so: ps -Ao pid,rsz,args | grep . If there is a better way of estimating memory please let me know in the comments.

RabbitMQ

An all-singing all-dancing “complete and highly reliable Enterprise Messaging system”. With language like that you’d expect horrible bloat and per-cpu licensing, but happily that’s not the case. It’s straightforward to setup and relatively lean.

The protocol, AMQP, comes from the financial world, and is intended to replace Tibco’s RendezVous, the backbone of most investment banks. There’s lots of documentation, lots of users, a healthy ecosystem, and it looks good on your CV.
I tried RabbitMQ first, and liked it so much I almost stopped my evaluation right there and deployed it.

The best tutorial for using it from Python is here: Rabbits and Warrens

Publisher

Quick Code



import sys

import time

 

from amqplib import client_0_8 as amqp

 

conn = amqp.Connection(host="localhost:5672", userid="guest", password="guest", virtual_host="/", insist=False)

chan = conn.channel()

 

i = 0

while 1:

    msg = amqp.Message('Message %d' % i)

    msg.properties["delivery_mode"] = 2

    chan.basic_publish(msg,exchange="sorting_room",routing_key="testke y")

    i += 1

    time.sleep(1)

 

chan.close()

conn.close()

Consumer

Quick Code



from amqplib import client_0_8 as amqp

 

conn = amqp.Connection(host="localhost:5672", userid="guest", password="guest", virtual_host="/", insist=False)

chan = conn.channel()

 

chan.queue_declare(queue="po_box", durable=True, exclusive=False, auto_delete=False)

chan.exchange_declare(exchange="sorting_room", type="direct", durable=True, auto_delete=False,)

 

chan.queue_bind(queue="po_box", exchange="sorting_room", routing_key="testkey")

 

def recv_callback(msg):

    print msg.body

 

chan.basic_consume(queue='po_box', no_ack=True, callback=recv_callback, consumer_tag="testtag")

 

while True:

    chan.wait()

 

#chan.basic_cancel("testtag")

#chan.close()

#conn.close()

Gearman

Gearman is a system to farm out work to other machines, dispatching function calls to machines that are better suited to do work, to do work in parallel, to load balance lots of function calls, or to call functions between languages.

Developed by Danga Interactive (essentially Brad Fitzpatrick, who brought us Memcached and Perlbal). Used by LiveJournal, Digg and Yahoo.

Ubuntu users: Make sure you install package gearman-job-server, which is the newer leaner C version of Gearman. Don’t install gearman-server, that is the old Perl version.

Client



import sys

import time

 

from gearman import GearmanClient, Task

 

client = GearmanClient(["127.0.0.1"])

 

i = 0

while 1:

    client.dispatch_background_task('speak', i)

    print 'Dispatched %d' % i

    i += 1

    time.sleep(1)

Worker



import time

 

from gearman import GearmanWorker

 

def speak(job):

    r = 'Hello %s' % job.arg

    print r

    return r

 

worker = GearmanWorker("[127.0.0.1]")

worker.register_function('speak', speak, timeout=3)

worker.work()

Beanstalkd

Beanstalkd is a fast, distributed, in-memory workqueue service. Its interface is generic, but was designed for use in reducing the latency of page views in high-volume web applications by running most time-consuming tasks asynchronously.

Developed for a very popular Facebook Application. The smallest memory footprint: after startup, connecting, sending a few messages, it’s resident memory size (rsz) was still only 0.5 Mb!

To install the server:

sudo apt-get install libevent-dev
wget http://xph.us/dist/beanstalkd/beanstalkd-1.3.tar.gz
tar xvzf beanstalkd-1.3.tar.gz
./configure
make (there’s no install step, it just generates the file ‘beanstalkd’)

To install the Python library:

wget http://pybeanstalk.googlecode.com/files/pybeanstalk-0.11.1.tar.gz
extract it
sudo python setup.py install

There’s a good tutorial here: http://parand.com/say/index.php/2008/10/12/beanstalkd-python-basic-tut orial/

Producer

Quick Code



import time

 

from beanstalk import serverconn

from beanstalk import job

 

def producer_main(connection):

    i = 0

    while True:

        data = 'This is data to be consumed (%s)!' % (i,)

        print data

        data = job.Job(jid=i,data=data, conn=connection)

        data.Queue()

        time.sleep(1)

        i += 1;

 

connection = serverconn.ServerConn('localhost', 11300)

#connection.job = job.Job

producer_main(connection)

Consumer

Quick Code



from beanstalk import serverconn

from beanstalk import job

 

def consumer_main(connection):

    while True:

        j = connection.reserve()

        print 'got work: %s' % j.data

        j.Finish()

 

connection = serverconn.ServerConn('localhost', 11300)

connection.job = job.Job

consumer_main(connection)

StompServer

StompServer is a lightweight pure Ruby STOMP server.

To install the server on Ubuntu:

sudo apt-get install ruby-dev rubygems
sudo gem install stompserver

To install the Python library:

wget http://stomppy.googlecode.com/files/stomp.py-2.0.1.tar.gz
extract it
sudo python setup.py install

There’s a good Python / Stompserver tutorial here: http://morethanseven.net/2008/09/14/using-python-and-stompserver-get-s tarted-message-q/

Sender

Quick Code



import time

 

import stomp

 

conn = stomp.Connection()

conn.start()

conn.connect()

 

i = 0

while 1:

    conn.send('Message %d' % i, destination='/queue/test')

    i += 1

    time.sleep(1)

 

conn.disconnect()

Listener

Quick Code



import time

import sys

 

import stomp

 

class MyListener(object):

    def on_error(self, headers, message):

        print 'received an error %s' % message

 

    def on_message(self, headers, message):

        print 'received a message %s' % message

 

conn = stomp.Connection()

conn.set_listener('', MyListener())

conn.start()

conn.connect()

 

conn.subscribe(destination='/queue/test', ack='auto')

 

while 1:

    time.sleep(2)

Results and Conclusions

I’d be happy working with any of these four. All four were easy to setup, fast, decent in memory consumption, and had good Python libraries.

RabbitMQ has the most mindshare (it is the only one which registers on Google Trends), but it took the most memory and is the most complex to use. It looks like a great product, but it’s Message Oriented Middleware, not an in-memory job queue, so it’s not what I’m looking for.

StompServer had the least documentation, and took several times more memory than Gearman and Beanstalkd. In seems the most immature project, but would probably be a good choice for someone working in Ruby.

Beanstalkd is great. I would like to see it in the Ubuntu repositories, and it’s Python lib in PyPI, but aside from that, I can’t fault it. I’m not choosing it, because Gearman is even better.

Gearman was designed for exactly the problem I have, takes almost no memory (1.4Mb), has a great pedigree (Danga), is widely deployed (LiveJournal, Digg, Yahoo), is in Ubuntu (almost), has a Python library is in PyPI, and someone helped me out on the #gearman IRC channel straight away. It even has queue persistence and clustering. So, Gearman it is.

Quote of the day – monkeys

Graham King — Thu, 30 Jul 2009 19:09:11 +0000

In response to monkeys stealing his coffee beans, an Indian farmer observes: If you start shooting monkeys, you’ll spend the rest of your life shooting monkeys.

via Bruce Eckel

On cellphone use in cars

Graham King — Mon, 20 Jul 2009 17:11:28 +0000

A very interesting article in the New-York Times on the research behind the risks of being distracted by a cellphone whilst driving:

http://www.nytimes.com/2009/07/19/technology/19distracted.html

Here’s some excerpts:

in a survey of 1,506 people last year by Nationwide Mutual Insurance, 81 percent of cellphone owners acknowledged that they talk on phones while driving, and 98 percent considered themselves safe drivers. But 45 percent said they had been hit or nearly hit by a driver talking on a phone.

That’s the Lake Wobegon effect, the tendency for overestimate their capabilities in relation to others.

…research, showing that multitasking drivers are four times as likely to crash as people who are focused on driving, matches the findings of two studies, in Canada and in Australia, of drivers on actual roads.

The highway safety administration estimates that drivers using a hand-held device are at 1.3 times greater risk of a crash or near crash, and at three times the risk when dialing, compared with others who are simply driving. The agency based its conclusions on research from the Virginia Tech Transportation Institute, which placed cameras inside cars to monitor drivers for more than a year. The study found cellphones to be the most common cause of driver distraction.

Research also shows that drivers conversing with fellow passengers do not present the same danger, because adult riders help keep drivers alert and point out dangerous conditions and tend to talk less in heavy traffic or hazardous weather.

The research shows that having a conversation on a hands-free sets is as dangerous as a conversation on a handheld phone – the problem is that, unlike a passenger, the person on the phone doesn’t stop distracting you when road conditions change, and they aren’t a second pair of eyes compensating for your distraction.

So if the research is so strong, there are so many lives to be saved, how come we haven’t solved this one yet. Read on:

Joe Simitian, a state senator in California, managed to get his hands-free legislation, an effort he began in 2001, passed in 2006. He argued, based on data collected by the California Highway Patrol, that drivers using cellphones caused more fatalities than all the drivers distracted by eating, children, pets or personal hygiene.

In each previous year, the bill was killed — after lobbying by cellphone carriers, including Sprint, AT&T and T-Mobile. Mr. Simitian said that in the first two years, he would visit the offices of his colleagues on the Transportation Committee on the day of the vote and “find three cellphone industry lobbyists sitting in the legislator’s office,” Mr. Simitian said. “They’d just smile.”

Django dynamic forms and formsets

Graham King — Tue, 16 Jun 2009 01:08:23 +0000

A couple of great posts which explain Django dynamic forms and advanced formset usage very clearly:

James Bennett: So you want a dynamic form
Malcolm Tredinnick: Advanced Formset usage in Django

How and Why to extend Firefox in Javascript

Graham King — Thu, 11 Jun 2009 03:29:22 +0000

I will be giving this talk on Friday 12th June, at Open Web Vancouver 2009.

View more Keynote presentations from Graham King.

Unix shared directory permissions: GUID and umask

Graham King — Wed, 10 Jun 2009 19:11:26 +0000

I setup my Mercurial repository in the same way we used to do CVS, then SVN: A directory owned by a group, with the GUID bit, and all users who need to commit are in that group.

The steps are, create the group and add relevant users to it:



    sudo groupadd topsecretgroup

    sudo usermod -a -G topsecretgroup graham

Change the project directory to be owned by that group, and accessible by no-one else:



    cd topsecretproject/

    sudo chown graham:topsecretgroup -R .

    sudo chmod g=u,o= -R .

Set the GUID bit on all the directories, so that new files and directories are created owned by the group:



    find . -type d | sudo xargs chmod g+s

Change the umask for everyone, so that new files are created with read and write permissions for the group:



    sudo vi /etc/profile

    change 'umask 022' to 'umask 002'

The last part, changing the umask, isn’t ideal. It works on Debian and Ubuntu, because every user has their own group. I would rather a more focused solution, just for that directory – suggestions welcome.

References:

Mercurial and permissions
Multiple Committers
Change Ubuntu global umask
Collaboration models

Legal 1 Usability 0

Graham King — Wed, 27 May 2009 18:55:46 +0000

The cooking instructions for my Tandoori Chicken Breast microwave lunch, are to cook…

…until internal temperature reaches 74C (165F).

How many office kitchens have a cook’s thermometer? Score nothing for usability.

Should you for any reason attempt to sue the manufacturer, it will rapidly become apparent that you didn’t follow the cooking instructions. Score one for legal.

OpenTTD: Trains and signals for beginners – a tutorial

Graham King — Thu, 21 May 2009 07:22:43 +0000

I have been playing Open Transport Tycoon Deluxe, or OpenTTD on and off for a while, but I confess I only understood train signals very recently. The game gets a lot more fun once you can have complex track layouts, so here’s a tutorial on train track layout and signaling for complete beginners.

Building tracks the wrong way

If you’re anything like I was, all your train layouts probably look like this:

You can only run one train on that track, but say you’re happy with that. When you need to connect another station, you might, unsuccessfully, try this:

Notice the three two-way signals. A signal locks an entire section of track from that signal until the next signal or the end of the line. These signals define four locks, color coded on this screenshot. If the train from Lundinghattan Ridge is in the Mardingbury station, it will have a lock on the yellow section, but not on the green section. The signal nearest Mardingbury will be red, but the other two signals will be green. The train from Marbourne will be able to acquire a lock on the green section, and stop at the signal nearest Mardingbury. We have a train stand-off. Not good.

To make that layout work, you’d need to remove the signal nearest Mardingbury, thereby merging the green and yellow sections. You remove that, and you have two trains sharing a station. OK, so now you add a third station to your network. Now things really start to break down.

The blue section is shared between the Lundinghattan train and the Chenningpool train. The Lundinghattan train is top left, just leaving Mardingbury station. It has the lock on the yellow section. Notice the two signals nearest it are red (actually all the signals in this picture are red, but focus on just those two). The train from Chenningpool acquired the lock on the blue section, but and this is the first important concept of this tutorial, once it got level with the depot it had a choice of two paths: Mardingbury, which is blocked by a red signal, and the depot, which isn’t. A train faced with a red two-way signal will always avoid that signal, even if that means going away from it’s destination. If instead of the depot we had a track running to the other side of the map, our Chenningpool would of happily headed down it, to avoid the red signal.

In practice this means our Chenningpool train will head into the depot, turn around, and head back to Chenningpool. It will never make it to Mardingbury. There is something very wrong with our approach, and the short answer is that we were using two-way tracks and two-way signals. We need to think one-way. Let’s start again.

The basic loading loop

Every shared station should have a one-way loading loop.

Notice the signals around the loop are all one-way. To place a one-way signal place a signal as normal, then click the signal again, once or twice depending on the orientation you want for your signal.

Now let’s connect our loop up to a town, and run two trains betweens those two towns.

We connected the shared track from Marbourne to our loading loop, with two short one-way sections. We can see the back of a one-way signal in the red circle, and the front of a one-way signal just to the right of the blue circle.

Let’s look at what’s going on in this picture. The train circled in red has the lock on the red section of track, and is held at the signal circled in red. It is waiting for a lock on the blue section of track. Notice that it could of kept going around the loop, instead of branching off and stopping at the red signal. Faced with a red one-way signal and a clear track going the wrong way, the train will stop at the signal, which is nearly always what we want. This is exactly the opposite to what would of happened with two-way signals.

The train circled in blue has the lock on the blue section of track, and is about to acquire the lock on the yellow section. As soon as it does, it will release the lock on the blue section, and the train circled in red will move forward. This is a layout that works.

Prefer one way tracks

Let’s connect up the other two towns, and not get blocked this time. The trick is to make all shared sections of track one-way.

The only two-way signals in this picture are circled in blue. All the others are one-way. The two-way signals are there to prevent a train on the two-way track from locking part of the one-way loop. If the left-hand two-way signal was not there, a train in Lundinghattan station would hold a lock on it’s two-way section of track, and the bottom part of the one-way section, up to the next signals. Remember, a lock is between two signals or the end of the track. Incidentally, stations don’t end a lock. If you had a station half-way along a track, the lock would run right through it until the next signal.

Pre-signals, the pro-layout

Mardingbury is getting quite busy now, we’d like to have two tracks in the station. Stop all the trains (or be quick!), bulldoze the station, and build a new, two track one. I moved mine back a square to allow space for the tracks to merge. and made the loading loop a little bigger. To control access to a multi-track station, you need pre-signals.

Pre-signals come in two types, entrance and exit. An entrance pre-signal will be red if all the exit pre-signals behind it are also red. The motivation for pre-signals is nicely illustrated here: Pre-signals on the OpenTTD wiki.

The entrance pre-signal is circled in blue. Notice that it has a horizontal white-bar, to show it is different. The exit pre-signals are circled in purple, and have vertical white bars. There is currently a train in the station, so one of the exit pre-signals is red. Because one of the tracks is free (green signal), the entrance pre-signal is green. The next arriving train will correctly go to the empty track. Even though they are pre-signals, we are still using one-way signals

What you can’t see on the picture, but which are very important, are the two normal one-way signals circled in black. They control station exit, by forcing a train wanting to leave the station to acquire a lock on the yellow section. This prevents two train leaving at the same time crashing into each other.

When a train is in the station, it still holds a lock on it’s section of track. The lock runs from the exit -pre-signal at the entrance to the station, to the regular one-way signal at the exit of the station.

Scaling it up

You now know all the key concepts, the rest is just more of the same. Here for example is what you would do if Lundinghattan got busy.

You give it a loading loop, and a multi-bay station. Pre-signals control station entrance, and regular one-ways control the exit. You can see the one-way’s at the exit much better on this station.

There’s one final change we need to make to allow lots of trains – we need to replace the two-way section highlighted in blue with two one-way sections.

We don’t have any more two-way signals. Each station has a loading loop, and one-way tracks connect the stations. In our first tries we had one track connecting the stations, and could only run one train between them. Now we have two tracks connection the stations, and in this picture alone there are eight trains, all serving Mardingbury. Now that’s more like it!

The stations are quite close together, so it might not be clear what is loading-loop and what is the tracks that connect them, so here’s an example with stations further apart.

Other stations would have their own loading loops, and as long as the one way tracks connect, you end up with a network spanning the world. Trains can run from anywhere to anywhere, and new stations just need plugging in to the network.

I have one final tip: Playing with virtual toy trains can be quite addictive, so remember to get some sleep :-)

Quote of the day: Why racists have bad graphic design

Graham King — Wed, 20 May 2009 16:34:19 +0000

Charlie Brooker on a television advert by the British National Party, England’s (very small) right-wing political party:

Extremist material of any kind always looks gaudy and cheap, like a bad pizza menu. Not because they can’t afford decent computers – these days you can knock up a professional CD cover on a pay-as-you-go mobile – but because anyone who’s good at graphic design is likely to be a thoughtful, inquisitive sort by nature. And thoughtful, inquisitive sorts tend to think fascism is a bit shit, to be honest. If the BNP really were the greatest British party, they’d have the greatest British designer working for them – Jonathan Ive, perhaps, the man who designed the iPod. But they don’t. They’ve got someone who tries to stab your eyes out with primary colours.

Read the article: Charlie Brooker on the BNP and their political broadcast.

Speaking at Open Web Vancouver 2009 in June

Graham King — Sun, 17 May 2009 21:30:59 +0000

I will be speaking at Open Web Vancouver on Thursday, June 11, 2009 and Friday, June 12, 2009.
That’s in Vancouver, B.C., Canada. There’s a very interesting speaker lineup, and the whole conference is reasonably priced, so come along, learn, interact, and enjoy Vancouver in the summertime.

My talk will be entitled How and Why to Extend Firefox in Javascript (and Thunderbird, Komodo, and Songbird). I will post the slides here in June.

See you there!

Micro-Zooids: A story

Graham King — Sun, 10 May 2009 22:23:13 +0000

When I was 16, I wrote a computer game, called Micro Zooides. It was called that partly because on Windows .EXE files all start with the two characters MZ, and partly because it was about small creatures. Micro-Zooides was going to be about humanity’s progress, it was going to be Civilization, which didn’t exist yet.

The game had a splash screen of a Far Side comic, then a short video of me tromping through the woods like a Neanderthal, which my Dad filmed and which I digitized with a very early video capture card.

In Borland’s Turbo C++ 3.0 I wrote a basic graphics engine to display the tiles of the world, and an event loop so I could move the main character around the world. I drew sprites for a proto-human (the micro zooid), dirt, rocks and sticks. He could walk around the world, and pick up and put down rocks or sticks.

Then I took a break to plan. I have a proto-human, rocks, and sticks. How do I get to civilization?

Turn on debug output in SVN

Graham King — Thu, 07 May 2009 23:02:38 +0000

After a desktop and server upgrade, my subversion client stopped working. I am using Digest authentication, and it kept asking me for the username and password. Wireshark showed me that the SVN client wasn’t sending the Authentication header. To find out more, I turned on Subversion’s debug output. Here’s how you do it:

Edit /etc/subversion/servers
Add this line at the end: neon-debug-mask = 511

That showed me this error: auth: '/' is inside auth domain: 0.

This means that the path I was requesting (the root of the repo) was not considered inside the AuthDigestDomain I had set in Apache.

It turns out that at some point in the upgrade of Apache, Subversion, or a library, the AuthDigestDomain requires a scheme. I had
AuthDigestDomain svn.myserver.com
whereas it should of been
AuthDigestDomain http://svn.gkgk.org.

So now you know.