Abstract: It’s not about you, not about your product, it’s about how awesome you make your users. If you sell digital cameras, make your users better photographers, through a better camera yes, but also teach them about exposure, light, etc.

(If the embed is broken, see Kathy Sierra at Business of Software)

Here are my notes:

Make the user awesome.

You want them to talk about the amazing thing they did (with your tool).

Your product is part of a higher purpose (digital cameras -> photography). Don’t talk about your product, instead blog / tweet / evangelize the higher purpose.

Make the user better. Teach.

Sense of belonging. Do people want to wear T-shirts with your logo? What does being your user SAY about your user? It shouldn’t say something bad, and it shouldn’t say nothing.

Get them interested

Trigger an emotional reaction (faces, sex, cute, fear, story) so people pay attention.

Talk in conversations, not formal language. Stories trump data.

Contrast your marketing material, with your user manual. Take care of people after they buy, so they stay excited.

Give super-powers

Instead of Word Of Mouth, which requires people telling other people, use Word Of Obvious, where they show other people what they did, or other people see it, it’s obvious that person is awesome now that they use your product.

Flip business advice:

• Exceptional customer service -> exceptional customers.
• ‘From good to great’ -> great customers.

Make your customers into the perfect customers you wish you had.

What super-powers do you give your customers? Help them get really really good at something.

Don’t build a better [x], build a better [user of x]. Say: We help [type of person] get better at [thing].

Wrong: What problem do you solve? What value to you add?
Right: What does having the solution mean to our users? What bigger, cooler thing is enabled?

Make it a rule that with your product, the user must do something cool within [x] minutes. Make a document listing 10 awesome things they can do, quickly, with your product, and how.

Help users justify acquiring your product to others. They are already sold, help them sell others.

Make good things easy / natural / obvious

Motivate / Inspire users. Motivate users to do things they wish they already did.

Peter Bregman, harvard business.org, on the easiest way to change behavior:

Right things should be easiest, most natural. Make doing the wrong thing very difficult. Usability matters.

Help your users practice

Make your users smarter. Shrink the 10,000 hours they need to become experts. Give users patterns they can learn. Experts are better at identifying patterns in the problem domain (the thing you are making your users super-heros at). Make their practice fun.

After 1 or 2 years, experience is a poor predictor of performance / expertise. Was it 10 years, or 1 year repeated 10 times?

Offer exercises, games, contests, tutorials that support deliberate practice of the right things. The things that make people awesome.

Those exercises etc should use cognitive pleasures to keep users interested and practicing. Game designers have identified a bunch:

• Thrill
• Discovery
• Challenge
• Self-expression
• Social
• Novelty
• Sensory.

The more deliberate practice [pdf] your users get, the better they get, so the more they enjoy using your product. And the more chance other people have of seeing them practice, so more Word Of Obvious.

Talk with users at their angriest

Add a “WTF?” button.

Help / FAQ are usually written for people who are intellectually curious, wanting to learn more. The “WTF?” button is for angry upset people who hate your product right now. Different feeling, different audience.

How you make users feel about themselves, drives how they feel about YOU.

If you want them to RTFM, make a better FM.

Where do you fit on their Hero’s Journey (also here and full detail here). What part do you play? Are you the helpful sidekick? The ogre guarding the bridge? Something else?

Community management

Model good behavior in your community.

Make a short document for moderators / active users telling them how to answer ‘dumb questions’, in a nice way. Provide alternative answers to guide them away from RTFM or “just google it”.

Encourage a culture of “there are no dumb answers”. Newbies are often the best people to answer other newbie’s questions, yet on most forums people don’t feel ready to answer until they are experts. Encourage early participation.

Don’t force inclusivity, don’t force your expert users to hang out and nurture the newbies. Hanging out with other experts is a lot more interesting. Give them a special space, where they can use jargon. Django, like many projects, has two mailing lists: django-users (beginners, normal folk), and django-developers (experts).

Evolution of awesome:

1. Awesome product (features)
2. Awesome people talking about the product (expert / influencer testimonials)
3. User testimonials (”I love this company”)
4. User benefits (”Look at what I did”)
5. I know Kung-Fu

Summary

In the words of Jamie Zawinski:

So I said, narrow the focus. Your “use case” should be, there’s a 22 year old college student living in the dorms. How will this software get him laid?

Your job is to make people better. Your tool is software.

I have done most of my work while anxious about whether I could do the job, and unsure that it would be enough to achieve the goal if I did.
But I tried anyway, because there was no one but me between the enemy and my city. Surprising myself, I have sometimes succeeded.

From this article about GNU HURD.

Website Optimization. Conversion Rate Optimization

The art and science of persuading your site visitors to take actions that benefit you.

Crediblity

Site must appear credible: Often a gut reaction.
Source credibility theory: Perceived expertise and trustworthiness.

• Professional web design
• Intuitive navigation (good usability)
• Clear unique selling point
• Loads fast
• Use ‘trigger words’ that users are scanning for
• Benefit-oriented content, that focuses on visitors needs and goals

Reassure users with:

• privacy policy link
• trust logos: Verisgn / Better Business Bureau / etc

Six psychological principles of persuasion, use them:

• social proof: Testimonials of people like you. Popularity (best selling xyz).
• reciprocity: Give first, initiate.
• consistency / commitement: Foot-in-the-door
• liking / similarity: develop user personas, speak their language
• authority
• scarcity: list how many products left, how long till sale is over

These come from Robert Cialdini’s Influence

Media

Brand is: Company name, logo, and tagline. Make them good.

Hero shot: Include one high quality prominent picture of your product, or of someone enjoying it. Put image on left and text on right for easier reading. Make it relevant. Make image clickable for larger view. Give it a strong caption – headlines and captions are most read part of page.

Interactivity increases conversions: Video, animation, live chat, something the user can interact with. Be subtle and tasteful.

Personas:

• Demographics: Age, gender, income, education
• Psychographics: Personality type
• Geographic: Local, national, international?
• Customer pain points and goals: What is the problem they are trying to solve? What is the single most important factor to them?
• Value proposition / unique selling points: Why you and not the competition?
• Benefit hierarchy: What benefits / features of your product are important to the customer? Order them by importance.
• Key frustrations and objections: What is their biggest frustration with companies in your industry and/or products like yours?
• Buying criteria: What factors do they weight when making the buying decision?
• Risk reversal: What risks are your customers taking when buying your product, and can you avoid those? 100% money back guarantee, free trial, free return shipping, etc.
• Keyword phrases: How do your customers refer to your product, what words do they use to search for it?

Unique Selling Proposition: “Oh I’ve heard of you, you’re the company that …”

Write for the web:

• Brief. Bold. Inject personality.
• Active voice, start with a verb.
• Front load with keywords, users scan the first one or two words of each title / link / bullet.
• Think newspaper headline.
• Use sub-headings.
• Write for scanning: Half the words. One idea per paragraph. Bullet lists. Highlight keywords.

Develop primary and secondary responses:

1. Primary response is your main call to action, typically ‘Buy Now’.
2. Secondary response is something for people who are not quite ready to buy now. Don’t lose them. Instead get them to sign up for newsletter, or download a PDF brochure, etc.

Show don’t tell. Encourage customer’s emotional mental imaging.

Links get highest clickthrough at 7 – 12 words. Repeat keywords that brought users to your site (ad words) in the links.

Avoid jargon, hype, and wimpy could should may might. Be positive – this will absolutely happen as soon as they give their credit card number!

Always be closing: Actually ask them to buy. Ask them to perform the call to action.

Metrics

Conversion metrics

What are your objectives – what does a ‘conversion’ mean?

• Sell product: Count completed orders, count additions to shopping basket.
• Ad clicks: For ad-supported site.
• Distribution / Downloads: PDF, software, etc.
• Informational: Read a page, view a video.
• Comments: A blog could make getting a user to comment a conversion.
• Sign-ups: Joining the site. Registering for the newsletter or for a beta invite, etc.

What to track, what to care about:

• Unique visitors.
• Average time on site. Pages per visit. These measure how engaged the user was in your site.
• Bounce rate: % of users who left within a short period of time, and without browsing to a different page on your site. High bounce rate pages need some work, or the campaign that lead them there incorrectly needs some work (wrong adwords being targetted for example)
• Conversion rates: Whatever you define a conversion to be.
• Entry points: Which pages do your users first land on? Also called ‘primary content consumption’.
• Exit points: Where do users stop browsing and leave? If it’s the ‘payment complete’ page, fine. If not, needs looking into.
• ROI (return on investment) by keyword and campaign, an cost per conversion: Track your advertising spend’s effectivness.

Business metrics you should be tracking anyway:

• Revenue per order
• Profit per order

Performance metrics:

• Number of HTTP requests
• Total size of data needed to view page
• Load times with empty cache and with full cache:
  • Server processing time
  • Start render – until then page is blank
  • Onload / document ready – JS starts
  • Fully loaded and executed.

Test in several browsers, as load times will vary. Test from several worldwide locations. webpagetest.org

In the nightly choice between reading a good book and watching a sitcom on television, we often [make the wrong choice] – although surveys show again and again that the average mood while watching sitcoms on television is mild depression.

Mean World Syndrome

People who watch a lot of television believe the world is more violent and intimidating than it actually is.

If you are growing up in a home where there is more than say three hours of television per day, for all practical purposes you live in a meaner world – and act accordingly – than your next-door neighbor who lives in the same world but watches less television.

Our surveys tell us that the more television people watch, the more they are likely to be afraid to go out on the street in their own community, especially at night. They are afraid of strangers and meeting other people.

George Gerbner

• George Gerbner articles on Mean World Syndrome
• Mean World Syndrome at Metafilter
• Mean World Syndrome at Wikipedia

Third-person effect

The belief that the mass media has a greater effect on others than it does on oneself. I like to call this the “adverts-don’t-affect-me” effect.

• Third Person Effect at Wikipedia

Television watching, one of the signature activities of our culture, correlates with brain problems.

How do we know this?

A recent study of more than 2,600 toddlers shows that early exposure to television between the ages of one and three correlates with problems paying attention and controlling impulses later in childhood.

For every hour of TV the toddlers watched each day, their chances of developing serious attentional difficulties at age seven increased by 10%.

Surely this only applies to children? No. Most of the book is about the recent discoveries that adult brains are also plastic, and how that is being used to treat stroke, alzheimer’s, and brain injuries.

Or you might be tempted to think that this applies only to children’s programming, and to MTV. Not so. In the words of Marshall McLuhan, “The medium is the message”:

Most people think that the dangers created by the media are a result of content. [..] Erica Michael and Marcel Just of Carnegie Mellon University did a brain scan study to test whether the medium is indeed the message. They showed that different brain areas are involved in hearing speech and reading it, and different comprehension centers in hearing words and reading them.

The pragmatic implication is that the medium is part of the message. [..] each medium creates a different sensory and semantic experience – and, we might add, develops different circuits in the brain.

Why and how does television alter your brain patterns?

It is the form of the television medium – cuts, edits, zooms, pans, and sudden noises – that alters the brain, by activating what Pavlov called the “orienting response”, which occurs whenever we sense a sudden change in the world around us, especially a sudden movement. We instinctively interrupt whatever we are doing to turn, pay attention, and get our bearings.

The result?

The response is physiological: the heart rate decreases for 4 or 5 seconds.Television triggers this response at a far more rapid rate than we experience it in life, which is why we can’t keep our eyes off the TV screen, even in the middle of an intimate conversation, and why people watch TV a lot longer than they intend.

Because typical music videos, action sequences, and commercials trigger orienting responses at a rate of one per second, watching them puts us into continuous orienting response with no recovery.

No wonder people report feeling drained from watching TV.

Yet we acquire a taste for it and find slower changes boring. The cost is that such activities such as reading, complex conversation, and listening to lectures becomes more difficult.

The Brain That Changes Itself is a fascinating book that will change the way you think about your brain, and how you go about your daily activities.

The best answers are given by security expert Bruce Shneier – why open wireless. The second best answer is that Tor hacker Jacob Applebaum also runs open WiFi.

Here are my answers, and the reasons why you should join us.

What’s changed?

The only difference between an encrypted wireless network and an open wireless network is that the part between your computer and the router is no longer encrypted by the router. Anyone can listen in, and anyone can connect to your router and access the big wide Internet.

The part between your computer and your router is only a tiny part of the journey your data takes between you and, say, your online bank. None of the part after the router has changed; if it was encrypted before, it still is, and if it wasn’t is still isn’t, and you should fix that. Read one for how.

Reason: It’s nice

Have you ever used anyone else’s open router? Maybe one named ‘dlink’, ‘linksys’, or a Mac AirPort? I bet you were happy that was there for you. When you move into a new home, when you have problems with your connection, or simply when you’re out and about, it’s great to have a free network. Opening up my network is my small way of giving back.

Myth: Bad people in your front garden

The first question most people ask about open wifi is this: What if someone uses my connection to do something bad?

To use your connection, they would have to be a direct neighbour of yours, or sitting outside your house.

Isn’t it more likely they would go that extra block to the warm and dry coffee shop?

Do you hear of coffee shop owners going to jail much? What about the ‘dlink’ and ‘linksys’ people? “Bad People” typically have their own Internet connection.

Real Risk: Eavesdropping

You might be worried that other people will steal your secrets. Again, they’d have to be sitting in your front garden.

There’s a much better place to steal people’s online banking data, than your front garden: in a coffee shop, at a conference, or, best of all, in an airport.

Reason: It makes you safer

You should setup your machine so that all your connections are encrypted, wherever you are. Opening your home wireless gives you that extra discipline.

• Secure your web browsing. When doing anything senstive, make sure you are using the ‘https’ protocol (your browser will be showing a padlock).

• Secure your email. If using webmail, make sure it’s over https. GMail, to Google’s great credit, has that as the default. If you are using regular email, makes sure you use the encrypted protocol – IMAPS or POPS.

• Secure you IM conversations. Google Talk is encrypted if your client supports it (Pidgin does). Skype is encrypted. As far as I could tell, Yahoo Messenger isn’t, so avoid it.

Weak Risk: Your neighbourhood hacker

If you’re using Windows, you should be running a personal firewall on your machine. I believe there is now one built-in to Windows. Make sure it is switched on.

Administration

Most computers will auto-connect to any available wireless network when they start up. If you notice the same machines on your network every day for a while, they are probably auto-connecting. You’ll need to add rules to your router to ban their MAC address, shunting them back to their own router.

The point of open wireless isn’t to make your neighbours Internet connection redundant, but to temporarily help people out. Your neighbours don’t want to use your connection, as their is typically faster for them (they are closer to their router).

Happy sharing!

• When a process dies.
• When a machine stops responding to network requests
• When your machine has too high load average, memory consumption, or CPU usage.
• When a file changes, hasn’t changed for a period of time, or grows beyond a certain size.

It can run a script of your choosing to attempt to fix the problem. It has an HTTP interface that shows you essential stats about the services you are monitoring. For detailed graphs, I recommend Munin.

Here’s how to get it working on Ubuntu:

Editing the config file

Edit the single line to startup=1.

The config file that comes with monit is well commented, but just in case here’s the breakdown.

Set Monit to check services every two minutes (120 seconds), and log to /var/log/daemon.log

Setup email alerts:

Switch on the HTTP interface, allow access from anywhere, and require a username and password. Make it a decent password, because the HTTP interface allows you to stop and start services.

Monitor the machine itself:

Then monitor all the services running on that box.

If you monitor the totalcpu resource, note that is a percentage of all CPUs. On a 4 CPU machine, 25% represents a process consuming 100% of one CPU.

For the Apache monitor, the PID file is defined in /etc/apache2/envvars, and is usually /var/run/apache2.pid.

Here are the service monitoring lines from my config:

Quick Code

Start monit, and query it:

You need the HTTP interface to use the ’status’ command.

Monitoring MySQL replication

To monit MySQL replication, create a script to touch a file if replication is still running. Put that script in cron. Get monit to check that file.

The idea comes from replication monitoring with monit, where they use Ruby.

I ported the script to Python, as a Django management command.

The crontab:

The shell script:

The Django command:

Quick Code

Add these lines to /etc/monit/monitrc:

Happy monitoring!

Here are my notes from setting it up, they are brief, but should help you get going.

All the monitored machines run a small daemon called munin-node. One machine is the central server. Every few minutes it gathers data from all the nodes (including itself), generates the graphs, and writes out some HTML files.

In my case I have a proxy running Nginx and acting as the Munin central server, and two nodes each with Apache and MySQL.

Everything that Munin monitors is in a separate plugin. Most of the ones you’ll need are bundled, but might need switching on.

If you only have one machine, that is your only node. Do both steps on that machine.

On each node

Edit /etc/munin/munin-node.conf and add these lines:

If your Apache is not on port 80, edit the plugin config file /etc/munin/plugin-conf.d/munin-node and change or add these lines:

Link the Apache plugin (it may already be linked, no problem)

Switch on Apache ExtendedStatus

Outside of the Location section, add ExtendedStatus On Quick Code

Install the Perl www packages (the Munin Apache plug needs them):

Bounce munin-node and apache

On the central server

Setup a crontab to run the data gathering task every five minutes

Add this line

Edit /etc/munin/munin.conf and set the nodes you want to gather data from, and their IP addresses.

Run it!

Look in /var/www/. You should see a munin directory.

Setup your web server to point to the HTML files. If like me you’re using Nginx, edit /etc/nginx/sites-available/mydomain.com, and add these lines

Create the htpasswd file, or see password protecting a folder with Nginx.

Try it out, by browsing to http://myserver.com/munin. myserver.com is of course the address your Nginx is serving.

You should see some graphs, but they will be blank, or display Nan instead of numbers. Be patient. In 10 – 15 minutes Nan will be replaced by numbers, and the graphs will slowly start appearing.

Add nginx monitoring

A bit of fancy footwork required here. This might of improved by the time you read this.

Download Munin plugins nginx request and nginx status.

Replace the very first line of each one, the one that says @@PERL@@ with this line

Comment out the rest of the header. Open the file in vim, hit Esc, and type

Note: I’m obviously missing an install step here. If you know, please help me in the comments

Copy the plugins into munin’s available plugins directory

Edit your nginx config to switch on status. Edit /etc/nginx/sites-available/mydomain.com and add these lines:

Restart nginx: sudo /etc/init.d/nginx restart

Link the plugins into the active plugins directory

And you should be set. Happy monitoring!

Run commands starting with $ on Unix. Run commands starting with mysql> in the MySQL client.

The easy way: Skip the problem

If you hit both databases at the same time, with the same INSERT, they will create their own record, and try and replicate to the other, which already has that record, causing a duplicate error.

In a simple case like that, you just want to skip the offending statement:

More details on skipping MySQL duplicate errors

Most of the time, you skip one statement, and replication breaks again straight away, because there’s a whole queue of problem statements coming up.

The right way: Rebuild

If you are not sure that you can skip the duplicate, or if replication has been broken long enough that your two servers are out of synch, pick one database to be the master, and rebuild the other from a copy of that master.

First make sure your site is only using the master server. Stop any processes that might modify data on the server you need to rebuild.

We have two database servers:

• Good Server: The good one, with the correct data.
• Rebuilding Server: The one we are fixing. All it’s data will be erased with the Good Server data.

You’ll need a unix command line and SQL command line on both servers. If you don’t already use it, after you’ve got this fixed, I recommend checking out Screen.

1. On the Good Server

Dump data from the Good Server. The master-data switch adds a statement at the end of the file to start replication.

The quick switch makes dumping large tables use a lot less memory, so on a VPS it’s much faster.

All tables will locked during the dump.

Replace ‘my_database’ with your database name.

2. On the Rebuilding Server

Copy the dump onto the Rebuilding Server. Replace ‘myuser’, ‘good-server’ and ‘my_database’ as appropriate.

Load the dump. This can take a few minutes for a large database.

You should see Slave_IO_Running: Yes and Slave_SQL_Running: Yes. The master-data switch to mysqldump, in step 1, started replication at the right place for us. How nice. I love MySQL.

The \G means show vertical instead of the usual horizontal. It works with any MySQL command.

Now you have statements flowing Good Server –> Rebuilding Server. Next we need to get data going the other way.

Make a note of the File and Position rows.

3. On the Good Server

Set the slave here to be in synch with Rebuilding Server. Use the file name and log position from the previous step.

4. On the Rebuilding Server

Start the processes that point to the Rebuilding Server’s database. Done!

Monitor replication

The database is done, but you’re not done yet.

If you don’t already have good monitoring, take 15 minutes to setup Monit on all your production servers.

Monit is easy to setup (a word away from the monster that is Nagios), and will save again and again.

Once you have Monit working, take another 15 mins to monitor MySQL replication with Monit.

Now you’re done :-)

What about Echinacea?

The best way to find out is to follow these simple steps:

1. Gather 2000 people who have the common cold


2. Split them randomly into two groups of 1000


3. Give vitamin C to one group, and a sugar pill (the placebo) to the other


4. Make sure the people receiving the pills and those dispensing the pills don’t know which is which (to make your trial double-blind)


5. Wait a bit


6. See if the Vitamin C group gets over their colder faster than the other group

You will of conducted a double-blind placebo controlled scientific trial.

Luckily for us, we don’t have to do it ourselves. Several of these trials have already been done. Enough trials, in fact, that one can do a Meta analysis, a statistical review and summary of all the trials.

A comprehensive meta analysis, by an unbiased organization, is the gold standard of scientific inquiry; it is our best chance of knowing the truth.

The Cochrane Collaboration is an international not for profit organisation set up 15 years ago to create transparent, systematic, unbiased reviews of the medical literature on everything from drugs, through surgery, to community interventions. And I have a cold. I read their review to find out whether Vitamin C or Echinacea would be effective in treating it.

Vitamin C

Here is the review of research on Vitamin C for preventing and treating the common cold.

Thirty trials involving 11,350 participants suggest that regular ingestion of vitamin C has no effect on common cold incidence in the ordinary population.

It reduced the duration and severity of common cold symptoms slightly, although the magnitude of the effect was so small its clinical usefulness is doubtful.

Nevertheless, in six trials with participants exposed to short periods of extreme physical or cold stress or both (including marathon runners and skiers) vitamin C reduced the common cold risk by half.

The failure of vitamin C supplementation to reduce the incidence of colds in the normal population indicates that routine mega-dose prophylaxis is not rationally justified for community use. But evidence suggests that it could be justified in people exposed to brief periods of severe physical exercise or cold environments.

Echinacea

Here is their review of research on Echinacea for preventing and treating the common cold.

Echinacea preparations tested in clinical trials differ greatly. There is some evidence that preparations based on the aerial parts of E. purpurea might be effective for the early treatment of colds in adults but the results are not fully consistent.

Beneficial effects of other Echinacea preparations, and Echinacea used for preventative purposes might exist but have not been shown in independently replicated, rigorous RCTs.

Get well soon!

My short notes on Predictably Irrational, by Dan Ariely. An excellent book. Entertaining, and covers much fascinating ground from social psychology and behavioral economics. Some of the experiments Dan and his team designed are fiendish!

Value is relative

We only know what we want when we see it in context. The bike the Tour de France winner rides. A set of speakers compared to another.

We only know what something is worth, or how much we like it, when comparing to other similar things (purchases, partners, jobs, etc..

We tend to choose the middle option. A high price option on a restaurant menu increases average order price, because it makes the rest seem cheap in comparison.

If we have several difficult to compare items, but two are easily comparable and one of those two is clearly superior to the other, we will prefer that one to all the other options. We can introduce a ‘decoy’ version of one item, slightly less appealing, so that the target item is chosen. In A v B, introduce -A, and most people will choose B.

Video: The truth about relativity

Arbitrary coherence

When we seriously contemplate buying something at a given price, that price becomes our anchor. From then on we judge other similar items relative to that price. We are coherent with our first valuation of something, even if that price is now irrelevant, arbitrary.

Items on sale attempt to get you to anchor at the ‘pre-sale’ price, before introducing the sale price.

Video: Starbucks, Dunkin Donuts, and Anchors

Herding

Joining a queue because other people are in it. Preferring a busy restaurant. Assuming something is good (or bad) on the basis of other people’s behavior.

Self-herding is when you do what you did before, because, well, that’s what you do. (See also: commitment)

We are irrationally attracted to FREE(!) things. We often pay a high cost in time and opportunity cost for them. Discounting from cent to free has a much higher influence than 2 cents to 1 cent. Free is a whole different place than something that has a cost.

Social Norms vs Market Norms

Pay someone cheap, they don’t work much, because they use market norms and don’t feel it is worthwhile. Pay someone nothing, they work hard, because they use social norms.

Gifts stay within social norms, unless you mention the price of it, in which case people revert to market norms.

Thinking about money makes people more self-reliant, selfish, solitary, individualistic.

Counting money is a pain buffer, makes people feel more powerful.

If both are present, market norms displace social norms, and it is very difficult to get the social norm back.

With gifts, benefits, and a good atmosphere, companies try and create a social contract with their employees. The employees hence expect the other side, the company to be there for them when they needed. Social norms require a high level of commitment from the company. Google and lots of startups get this right.

Video: The Cost of Social Norms

Emotions

When not experiencing the emotion, we are bad at predicting our responses in a strong emotional state, such as sexual arousal, anger, hunger, excitement, jealousy, etc.

It is not easy to look from one emotional state to another.

Most people are aware that they procrastinate. Giving them tools to commit to a deadline helps them overcome it.

The best remedy for procrastination is an external force: Automatic payroll deductions for saving, teacher imposed assignment deadlines, exercising with a partner, etc.

Engineer the world to force you to achieve your goal.

Simpler schedules are easier to commit to and more likely to be followed.

Package actions up into something that feels important enough to avoid procrastination. The same way auto-dealers package up our servicing into large mileage increments, and do all of it at once.

Video: The Influence of Arousal

Endowment effect

We value what we own more than other people do. Sellers usually value their items more than buyers do.

The sellers focuses on the loss of the enjoyment of the item. The buyer focuses on alternate uses for the money. Seller and buyer expect the other to see things the same was as them.

The more work you put into something, the more ownership you feel. The Ikea Effect, because you assembled it yourself.

The endowment effect is why companies offer ‘trial’ rates and ‘money back guarantees’. Once we have something, own it, we value it more. A ‘virtual ownership’ applies to trying on clothes, and especially to online auctions.

The endowment effect applies to ideas too. We don’t want to let go of them, so they become a rigid ideology.

People dislike losing options, and will work quite hard to keep all their options open, eve if that means occasionally ignore their best option.

Often, we are presented with two or more very similar options. In this case the biggest cost is that of not choosing – the time spent deciding is time we are enjoying neither option.

Life usually conforms to our expectations, so you can influence results by setting expectations. Marketing sets expectations.

Video: Dating, doors, and loss aversion

Priming

Priming is putting thoughts into someones head.

Often, an experiment will ask people to unscramble a word, that word is chosen to make them think the priming thought.

Priming is a context effect: Being influenced by something irrelevant in our immediate environment.

Two parties in a conflict, having already strongly committed to one side, need a neutral third party to resolve their conflict.

Video: The Effect of Expectations

Placebos

Placebos are belief and conditioning. Many, maybe most, surgical procedures are not placebo tested.

Honesty

We internalize social virtues, so our conscience keeps us broadly honest.

Given the opportunity, most people will cheat a little bit. A greater potential and reward for cheating doesn’t increase the cheating. Being reminded of the Ten Commandments, or of honor systems, eliminates the cheating.

People are more likely to cheat for non-monetary items – steal a pen from work, but not the value of the pen from the petty cash box.

It seems that people will only cheat within the limits their conscience (super-ego) allows them. We cheat to the extent we can justify it to ourselves.

“It is difficult to get a man to understand something when his salary depends upon his not understanding it” – Upton Sinclair

Video: The Context of our Character

Dan Ariely maintains an excellent website with links to research papers from the book, and much more, here: http://www.predictablyirrational.com/.

As an Amazon reviews says, “arguably the best book ever on what is increasingly becoming the science of persuasion.”

If you want to understand why you felt compelled to give money to a Hare Krishna devotee, how car salesman or realtor’s work, and much more, you should read this.

It’s also a very easy and enjoyable read. These are my notes. They cover all the content in the book, but don’t link to research. In the book, most of the statements have links to research papers to back them up.

Get Influence: The Psychology of Persuasion from your local library, this has sold so many copies they are bound to have some.

Contrast principle: Two different things presented together or sequentially will feel more different than they really are. Hence sell the expensive item first, as the other items will seem cheap after that. Good cop / bad cop is about creating a contrast between the two, to increase liking of the ‘good cop’.

A fixed action pattern is an automatic behavior pattern often triggered by a single item in the information. Also known as a heuristic.

A Hare Krishna devotee presses a flower or a copy of the Bhagavad-gītā into your hand (that happened to me), or a store gives you free samples, and you feel awkward about taking it for free. You end up giving money or buying something you don’t want. That’s the norm of reciprocity at work.

We also reciprocate to concessions. Start witha large but not unreasonable request, which gets rejected, then ‘retreat’ to the target request. People who comply after this technique has been used on them feel more satisfied with their decision than those who agree straight away, because they ‘bargained’ for it. They shaped the outcome, so they feel ownership for it.

A recent research article I read (apologies, can’t locate link right now) supports this, by saying that when the seller of a product is the first to quote a price (typically quite high), the product sells for more than when the buyer opens the negotiation (with typically a low price).

Surprise increase the likelihood of the target using heuristics instead of controlled responding. A shaved headed person in orange robes shoving an ancient Indian text at you, that’s surprising.

We wish to think of ourselves, and appear to others, as being consistent in our behavior, we apply the consistency principle heuristic. We make up reasons to justify our behavior, to seem logical and in line with our previous behaviors.

Foot in the door technique: Start small and build. Extract a small inconsequential commitment, then a slightly bigger one, and so on.

Commitments are most effective when they are:

• Public: Forces us to stick to it to look consistent. Especially effective for those whith lots of pride or public self-consciousness.
• Active: Perform some action to take the commitment, such as writing, swearing an oath (also public), etc.
• Effort-full: The bigger the price the bigger the commitment. Hazing / initiation ceremonies. Extract a bigger effort with foor-in-the-door technique.
• Freely chosen:Avoid threats, or any external reward such as prizes. The commitment must feel freely chosen for the person to accept responsibility for it, it must feel internally motivated.

Caveat: If the commitment alone gets the same result as the goal, we may not pursue the goal. For example if you are going to attend law school so that people will think you are clever, and telling people you are going to attend gets the response and recognition desired, the motivation to actually attend is removed.

Commitments get internally justified, they grow their own legs. Car salesman will often low ball you, offer an artificially low price.
You gladly accept the low price. Time passes as you fill in some forms, test drive, etc. They take the paperwork to their manager for final approval. Manager comes out and explains the salesman made a mistake, the price is higher.
You would of refused the higher price originally, but now you are committed, and much more likely to accept the high price.

The greater the number of people who find any idea correct, the more a given individual will perceive the idea to be correct. This is the Social Proof heuristic.

We look to others for behavioral guidance when we are unsure, in an unclear, unfamiliar, or ambiguous situation. Often in such situations others do the same thing, which leads to pluralistic ignorance.

In an unclear emergency situation, a lone bystander will help, but a group looks to each other to decide how to act, and does nothing. This is the Bystander effect.

In an emergency situation you need to reduce the uncertainty experienced by bystanders:

• Pick one and speak directly to him
• Say you need help
• Tell him how to help (”Call an ambulance”, “Help me out of my car”, etc)

Social proof is strongest when the others we look to for guidance are similar to ourselves. Hence Joe Average in television adverts.

Individuals in psychological pain look to others in similar situations to see how to deal with it. After a widely reported high-profile suicide, suicides increase, often using a similar method. Copycat crimes work in a similar fashion.

Books, products, feature One Million Sold or New York Times Bestseller on the packaging. Informing someone that many people like him (NY Times readers) have adopted a certain behavior makes them more likely to also adopt that behavior. Testimonials.

It’s pretty shocking, I know, but the evidence is plentiful.

Physical attractiveness triggers the Halo Effect, where one positive characteristic dominates the way that person is viewed by others.

Similarity

We like people like us, whether that ‘like us’ is because they share opinions, personality traits, background, or lifestyle. To influence, try and match the target in as many ways as possible:

• Dress
• Interests
• Background
• Age, religion, politics, specific habits
• Posture, verbal style, mood, etc. See Mirroring.

Compliments

People want to be liked. Be nice, pay compliments, send a greeting card. There is evidence that positive comments make the giver more likable even when the receiver knows the compliment is not genuine.

Contact and co-operation

Positive familiarity. Familiarity caused by contact leads to liking, unless the contact occurs under unpleasant circumstances.

Here is a recipe for creating cross-group hatred: Split people into two groups. Name each one, to increase group identity (creating a minimum group). Wait. Then mix the two groups in competitive actions.

And here’s how to fix it: Successful joint ventures towards common goals.

Association

An innocent association with good or bad things will influence the way people think about that us

• Showing a credit card logo to people with positive associations to credit cards (most people) made them tip more, even when paying cash.
• Killing the messenger: An association with bad news, used to be bad news for the messenger.
• Celebrity endorsments. Sponsorship of the Olympics, or whatever cultural event is popular, even if the connection doesn’t make sense.
• Stand attractive people by your product. Their beauty somehow ‘rubs off’ to make your product more appealing.
• Hometown team victory makes us feel good, even though we obviously had no part in the victory.
• Reductio ad Hitlerum: “Hitler was a vegetarian, so vegetarianism is wrong.”

Taken to extremes this is a personality flaw, showing low self worth, exhibited by groupies, sports fans, etc.

The seminal research on obedience to authority, and probably the most famous study in social psychology, is the Stanley Milgram experiment.

Opportunities seem more valuable when they are less available. Scarcity heuristic applies to information and goods.

If something is in short supply, everyone else must of wanted it, so it must be good.
If something is in short supply, we are about to lose the freedom of acquiring it.

Reactance theory says that an attempt to restrict something makes us want it more, and hence assume it is more valuable and desirable than we previously thought.

Things that recently became scarce are perceived as more valuable than things that have been scarce a long time. We react to a freedom being taken away.

Competition for a scarce item increases perceived value of the item, beyond scarcity alone, or social proof alone. Hence: Auctions.

By Thomas Gilovich, social psychologist and CSI Fellow, this well written book explains some of the reasoning and deduction errors we make when trying to understand the world, and ways to avoid making those errors.

This is an easy and engaging read, and offers several straightforward techniques to avoid making common reasoning errors. I recommend you look up How We Know What Isn’t So: The Fallibility of Human Reason in Everyday Life in your local library, or get it second-hand from Amazon for less than a posh cup of coffee.

These are my notes / summary of the book.

I. Cognitive determinants of belief

2. Something out of nothing: The mis-perception and misinterpretation of random data

We are predisposed to see order, pattern, and meaning in the world, and we find randomness, chaos, and meaninglessness unsatisfying.
As a consequence we tend to ’see’ order where there is none, and we spot meaningful patterns where only the vagrancies of chance are operating.

Detecting patterns and seeing connections is very useful, leads to discovery and advance. But the tendency is so strong we sometimes do it when there was nothing to spot.

Example of hot hand in basketball. Why?

• Availability heuristic: We remember the long streaks, forget the single shots. Can interpret near-misses as evidence of ‘hot’ or ‘cold’ player.

• People have faulty intuitions about what chance sequences look like.

  See: R. Falk (1981) The perception of randomness. In Proceedings, Fifth international conference for the psychology of mathematics education.

  In a coin toss, people expect a near perfect alternation of heads and tails. In a series of 20 tosses, there is a 50/50 chance of getting 4 heads in a row, and 10% chance of six in a row. Clustering illusion.

St Louis Gateway Arch is one of the worlds largest optical illusion. It appears much taller than it is wide, yet height and base width are the same.

Representativeness heuristic: “Like goes with like”. We expect things that go together to look similar. We expect instances of a type (librarian) to look like the prototype (a cliche librarian). Complex effect stem from complex causes. Effects look like their causes: Jagged handwriting means jagged nerves, heartburn comes from hot/spicy foods, etc.

Representativeness usually very helpful. Occasionally mis-applied. The salient (’prominent or conspicuous’) feature of something is what we remember, and apply the representativeness heuristic to that. The salient feature of a random sequence is the even mix of all the outcomes.

Law of averages is, by statisticians, correctly called the law of large numbers. The even mix of outcomes is only true for very large random sample. There is no ‘law of small numbers’.

Stock market: Random walk, but chartists insist they see patterns in the randomness.

In any random distribution (e.g. x,y plots on a graph), there will be a way of segmenting it (along the axes, diagonally, in bands, etc) that will appear non-random. Carving up data after the fact is meaningless. If we think we see a pattern in the data, test that pattern on other independent sets of data.
Unfortunately, for most people hypotheses constructed on one set of data are considered proved by that same data. This is why medical trials announce their goal, the expected outcome, ahead of time.

Once we think we see order in randomness, humans are exceptionally good at justifying it with a post-hoc theory. Improv comedy teaches ‘jump and justify’; say something random then justify it, it’s surprisingly easy.
For example people let to believe they are above or below the average at some task can explain the difference quite easily, even when the experimenters assigned the above/below at random.
Experiments with split-brain patients, where they justify what the other part of the brain just did.

Once a person has (mis)identified a random pattern as a ‘real’ phenomenon, it will not exist as a puzzling, isolated fact about the world. Rather, it is quickly explained and readily integrated into the person’s pre-existing theories and beliefs. These theories then serve to bias that persons evaluation of new information. People cling tenaciously to their beliefs in the face of hostile evidence.

Regression to the mean: When two variables are related, but imperfectly so, an extreme value on one tends to be matched by a less extreme value on the other. A high number roll on the dice tends to be followed by a lower number, and a low number by a higher one. A companies disastrous years tend to be followed by betters ones. Students with exceptionally good grades in high-school tend to do slightly less well in college.

Regression can be understood than in any performance (game, test, financial year, etc) there is a part of talent and a part of chance. An very high score is more likely to be a good student with luck in their favor, rather than an extremely good student with luck working against them, simply because there are more ‘good’ students than ‘extremely good’ students. So a very high score is likely to be followed by a slightly lower one, because it is unlikely to get chance that much in favor two times in a row.

Most people understand regression, but make two mistakes:

• Insufficiently ‘regressive’ when making predictions. The ‘next’ value after an extreme one is closer to the average than they tend to predict.
• Regression fallacy: Fail to recognize statistical regression, and explain it away with a superfluous and often complex causal theory. Ad-hoc justification where none was needed.

The regression fallacy shapes people (parents and teachers, mostly) perception of the effectiveness of rewards and punishments. A good performance is likely to be followed by a less good one, and a bad one by a less bad one. If the good performance is rewarded, the reward will be perceived as ineffective because the good performance was not repeated. If a bad performance is punished the punishment will be perceived as effective because the bad performance will not be repeated.
That notwithstanding, psychologists have known for some time that rewarding desirable behavior is generally more effective in shaping behavior than punishing undesirable responses.
See: B. F. Skinner (1953) Science and human behavior.

3. Too much from too little: The misinterpretation of incomplete and unrepresentative data

Using empirical evidence as proof: “I’ve seen it happen”, “I know someone who did”, etc. If a phenomenon exists, there must be some positive evidence – ‘instances’ of its existence visible to us. So empirical positive statements are necessary for a belief to be true, but they are not sufficient. We also need to know what goes in the other boxes.

Many of the beliefs we hold are about the relationships between two variables (’takes vitamin C’ and ‘gets better’). Say we were investigating ‘vitamin C megadose’ on cancer patients. We notice some patients get better after taking mega-doses of vitamin C. That prompts us to start investigating. Positive outcomes under treatment are one box, we need to know the other three:

If we stop at noticing some patients getting better, all we have is box a. All we have is the ‘illusion of validity’.

For vitamin C to be effective, the probability of getting better after taking it [a / (a + b)] must be higher than the probability of getting better after not taking it [c / (c + d)].

This is often difficult to do intuitively, because box ‘a’ is the most salient. We notice things happening, not things not happening.

We often seek only evidence to confirm our beliefs; we should also seek information to disprove them, and only hold our beliefs if such evidence is not available.

Often the evidence in the other boxes is not available, particularly in evaluating selection criteria. How do we test whether a companies interview process is effective? We are asking the relationship between ‘passes interview’ and ‘performs well at job’. But it’s very nature, we don’t have the data on job performance for people who didn’t pass the interview.

The only data we have is a and c, we can only compare the success rate of those who pass the selection. If the base rate of success is high (most people could perform well, only the best performers apply, etc), a would be higher than c whatever the selection criteria, so we would spuriously conclude our criteria are good.

It is not possible to be confident about the selection process without the data on those who were not selected.

Furthermore, simply being accepted can give someone a competitive advantage. Being admitted to a better school, getting a research grant, or working with high-performing colleagues will all improve an individuals performance compared to someone of similar initial ability who didn’t get selected.

Effectiveness of public policy is similarly difficult to measure, because we can’t both set the policy and not set it.

Often the lifestyles we lead, the roles we play, and the positions we occupy in a social network, deny us access to important classes of information and thus distort our view of the world. Overcoming that bias is difficult: We must first recognize the existence of a class of information we have not been exposed to, and then accurately characterize what that information is like.

Self-fulfilling prophecies: Have got a lot of attention. For a prophecy to be self-fulfilling there must be a mechanism that translates the expectation into confirmatory action. They often serve to exagerate a belief that holds a kernel of truth.
Thinking a bank is in trouble (usually with good reason), creditors will withdraw their money, and the bank will really be in trouble. Behaving in an unfriendly and defensive manner because you think someone is hostile will often produce that very hostility.

True self-fulfilling prophecies are ones in which a persons belief elicits the very behavior originally anticipated.
A seemingly self-fulfilling prophecy is one which alters a persons world or limits their responses, in such a way to make it very difficult or impossible for that expectation to be proved wrong. If someone thinks that I am unfriendly, they will avoid me, so I will have no opportunity to prove them wrong. If a sports player is thought incompetent, he won’t get to play, so won’t be able to prove himself competent. The continued absence of positive contribution can easily be mistaken for an absence of talent, when it is simply an absence of opportunity.

Negative first-impressions are generally more stable than positive ones, because we keep interacting only with people who created positive first impressions.

4. Seeing what we expect to see: The biased evaluation of ambiguous and inconsistent data

Information consistent with our pre-existing beliefs is generally accepted at face value, whereas evidence that contradicts them is critically scrutinized and discounted. Our beliefs are much less responsive than they should be to the implications of new information. This is a sane and necessary strategy. If a belief has a lifetime of support, it is perfectly valid to be skeptical of evidence that contradicts it.
Well supported beliefs have earned their inertia. We need to be wary of the beliefs that don’t have a solid foundation, such as cultural stereotypes, social norms, and traditions.

Ambiguous information is usually perceived in a way that fits out expectations. We may not even be aware of the ambiguity – it gets resolved before reaching conscious awareness.

Gamblers tend to attribute their losses to outside forces (chance, the referee, a team injury), but their wins to themselves (knew the team well, studied the form). We re-write our history to discount our losses and bolster our wins.

By carefully scrutinizing information that does not fit our beliefs, we can usually find a way of either discounting or re-interpreting it.

Scientists use a set of formal safeguards to prevent their own erroneous thinking affecting their results:

• Statistical measures to guard against the mis-perception of randomness
• Control groups and random sampling to avoid drawing inference from incomplete or unrepresentative data
• ‘Blind’ observers to eliminate experimenter bias.
• Precisely specify in advance the meaning of various outcomes, and objectively determine those outcomes. These rules are the ‘context of justification’, used when testing an idea. Idea generating is much more open. Science works by flashes of inspiration followed by rigorous testing. The rigorous testing is what differentiates scientific inquiry from everyday life.

Multiple endpoints: By not precisely specifying an expected outcome, we can pick any one and claim it as success. Psychics will use very vague descriptions, so that we can apply them to our lives and perceive them as true. If the subject is fuzzy (’personal well-being’ for example), we are likely to seize upon any subsequent measure of it that fits our initial beliefs.

Variable windows: By not specifying an endpoint to a prediction, it stays ‘open’ until it comes true. ‘Thing happen in threes’ because we could ‘things happening’ after the third one has occured, whenever that may be. The third event defines the time window for our prediction. With a wide enough variable window, beliefs can only be confirmed.

Multi-faceted expectation: For any two sufficiently complex entities, we can produce a mapping of one onto the other that will produce a certain amount of overlap, and allows us to claim they are similar in some way. Often used with the representativeness heuristic.

You have a strong need for other people to like you and for them to admire you. At times you are extroverted, affable, and sociable, while at other times you are introverted, wary, and reserved. You have a great deal of unused energy which you have not turned to your advantage. While you have some personality weaknesses, yo are generally able to compensate for them. You prefer a certain amount of change and variety and become dissatisfied whne hemmed in by restrictions and limitations. You pride yourself on being an independent thinker and do not accept others opinions without satisfactory proof. You have a tendency to be critical of yourself. Some of your aspirations tend to be pretty unrealistic.

If you see yourself in that description, you are not alone. It is multi-faceted, so most people will find the part that related to them the most salient, and the statements are so general (multiple endpoints) that they are bound to ring true. This is how horoscopes work.

One-sided versus two-sided events:

• One-sided events are ones that are memorable only when they turn out a certain way: ‘The phone always rings when I’m in the bath’, ‘You wait forever for a bus and three come along at once’, etc. If the phone does not ring whilst you are in the bath, the non-event does not register.
• Two-sided events are ones that register regardless of the outcome: Vacations, dates, gambling, buying a stock, etc. Both outcomes stand out from the stream of experience.

In two-sided events, often negative outcomes are remembered better than positive ones, as they require more rationalization to incorporate them into our self-image and understanding of the world.
In one-sided events, we are more likely to remember the ’side’ of events that has meaning to us, which is usually the one that reinforces our world-view. If I believe my dreams are prophetic, I will remember those much more than the other dreams. If I believe that strange things happen during a new moon, I will notice and remember the strange things, not the ordinary things (strange things not happening).

Many one-sided events only ‘exist’ when they are confirmed. If a fortune teller says you will have twins, and you have twins, you remember the fortune teller, link the two events, and form a strong memory. If you have one child, you probably won’t think of the fortune teller at all. And even if you did, the fortune is not contradicted, simple un-confirmed. You could have twins later in life.

One-sided events tend to be temporally unfocused, they have variable windows to be confirmed in. A ‘prophetic’ dream, a fortune, doesn’t have a fixed date, so only the positive outcome is acknowledged.
Two-sided events tend to have fixed windows: A sporting event, a vacation, a job interview. The closed window forces us to acknowledge both outcomes.

In two-sided events, both outcomes produce the same intensity of emotion (winning or losing a bet). In a one-sided event, only one of the outcomes has any emotional weight: It ‘always’ seems to rain when you forget your umbrella, because getting wet has an emotional attachment (hair ruined, clothes soaked, etc), whereas not-getting-wet doesn’t.

Definitional asymmetries: ‘American tourists in London are loud’, ‘I can always spot fake breasts’, ‘You need to hit rock-bottom to bounce back’. All of these are difficult or impossible to disprove: We only notice the loud people, we don’t notice fake breasts that we didn’t spot, and how can you say if someone has hit rock-bottom or not? Definitional asymmetries are all one-sided events.

One-sided events tend to have a ‘normal’ outcome with a high base rate, which we don’t notice because it is part of everyday life (’look, it’s not a full moon’), and an less usual more salient outcome (’o oh, it’s a full moon!’).

All of the above relate to the availability heuristic.

II. Motivational and social determinants of questionable beliefs

5. Seeing what we want to see: Motivational determinants of belief

Endowment effect: We value something more when it is ours. Ownership creates an inertia that prevents many potentially beneficial transactions from occurring.

The endowment effect applied to humans is the Lake Wobegon effect, or Illusory superiority. This is particularly true on ambiguous traits (’intelligence’, ’sensitivity’, ‘idealism’), and less true on more specific traits (’thriftiness’, ‘being well-read’). If a specific definition is given in the question, the above average effect tends to disappear: People are not lying or cheating, it’s just that the first thing that comes to mind when asked an ambiguous question is usually something they are good at, something salient in their lives.

We attribute success internally and failure externally. Thanks to their own resources, they succeeded. Because of the others/the environment, they failed.
Many psychologists hold that we do this to maintain self-esteem. There is also a cognitive explanation: Succeeding at something is at least partly due to our own effort, and thus warrants some internal attribution. Failing at something usually happens despite our best efforts, so often invovles an unfavorable external situation.

People are more likely to believe things they want to believe, but are constrained by objective evidence and the need to construct a justification that would presuade a dispassionate observer. We draw the desired conclusion only if we can muster up enough evidence to support it. It is in this sense that most people think of themselves are objective.
People often do not realize that their selection and interpretation of data is biased by their goals – the data could be interpreted a different way, and there is other data they ignored. They may well be able to justify opposite conclusions on different occasions.

How we ‘filter’ data to ensure it supports our goals:

• Seeking only to confirm
• Select who we consult. By judiciously choosing whom we consult on an issue, we can increase our chances of hearing what we want to hear.
• Amount of information: If the initial results confirm out expectations, we stop looking. If they don’t, we keep looking.

We should not stop at Can I believe this?, but should progress to Must I believe this?.

These are some of the ways we skew the evidence in the world, frame it to support our beliefs. This is healthy. People who can’t frame effectively run the risk of depression.

Beliefs are like possessions: We acquire the ones we think will make us feel good, and cling tightly to the ones we have.

6. Believing what we are told: The biasing effects of secondhand information

Telling a good story: The speaker needs their message to be worthy of the listeners attention. For the listener the interaction must be worthwhile in some way. The message must be understandable (not assume too much knowledge of the listener) and yet not too detailed (not assume too little knowledge of the listener).

Sharpening and Leveling: What the speaker construes as the gist of the message is emphasized, ’sharpened’, where are details thought to be less important are de-emphasized, ‘leveled’.

One results of sharpening and leveling is that we often develop exaggerated or extreme views of people we have only been told about.

One way to make a message more entertaining or seemingly informative is to increase it’s immediacy. Instead of the story happening to a friend of a friend’s colleague, have it happen directly to friend or family member, or even better, you. Often such alterations are intended for the self-aggrandizement of the speaker. It places them closer to center-stage. Other times it is simply an effort to make the story more salient, more vivid and concrete.

Increasing immediacy makes it difficult for the listener to accurately gauge the reliability of your message:

• Reducing the hops: We all know that the more hops a message has been through, the less reliable it is. Increasing the immediacy of a message for the sake of entertainment or self-aggrandizement also makes a message seem more reliable than it is.
• Changing the origination (’my brother’, instead of ‘my brother’s friend from work’): Your brother might be trustworthy, so we trust the message, but your brother’s friend is an inveterate liar. The original source of the message has been obscured.

Presenting and accepting remote accounts as secondhand is misleading when estimating the prevalence of a phenomenon in the general population. If something happened to lots of your friends cousins, it is happening a lot. If lots of your friends cousins heard a story about someone else’s friends cousin, and passed it on, there might be a single case.

In attempting to be informative, we might level some of the qualifications in the original message. This is often the case when scientific findings are reported in the news media.
Sometimes the facts are stretched to ‘help’ their audience get the message. This results in hysterical public service campaigns, that elicit far more fear than the original risk warranted (drugs, ’stranger danger’, terrorism, etc). The facts are stretched beyond recognition to make a more compelling story.
Parents are often guilty of this distortion in attempting to motivate behavior in their children.

The desire to entertain often creates a conflict for the speaker between satisfying the goal of accuracy and the goal of entertainment. There is often a tacit agreement between speaker and audience that the truth may be stretched: Tabloids and ‘light entertainment’ news shows are granted this permission. “One of the most common sources of such inaccuracy is the dissemination of unfounded or fallacious claims by news and other media organizations that try and entice by their ability to entertain.” The demand for news has been met by and artificial increase in supply.

Plausibility: Inaccurate or fictitious stories are sometimes told and retold because they just seem so plausible, that we let our critical guard down. Our standards for plausibility are often very low, a decent irony is often enough (the creator of the song “Don’t worry be happy” committing suicide, for example, or someone at the patent office resigning because there was nothing left to invent).

Summary: As we have seen in previous chapters, the data from our own experience is often biased and incomplete. In this chapter we saw that data from others is also. It is therefore important to locate unbiased, complete sources of base-rate data (scientific inquiry mainly), and use those to asses how likely it is that our own perceptions or those of our social group are true. If the base rate data and our personal experience concur, then we are likely correct. If they don’t a reliable base rate should be our guide. If the base rate data is to reliable, at least we know that we don’t know.

How to assess secondhand information:

• Consider the source: The New York Times or the National Enquirer? A rock star or a researcher? An actor who plays a doctor on TV, or a practicing medical doctor.
• Trust facts, distrust projections: Predicting the future is hard, even for an expert. What looks like an exponential curve can turn out to be sigmoidal.
• Watch for Sharpening and Leveling.
• Be wary of testimonials. One striking human interest story does not tell you anything about prevalence or risk.

7. The imagined agreement of others: Exaggerated impressions of social support

What we believe is heavily influenced by what we think others believe. This is usually a good strategy. However we often exaggerate the extent to which others hold the same beliefs as us. We think we have more social support for our opinions than we really do.

This is the false consensus effect: This is a relative effect. We realize if our belief is in the minority. We underestimate by how much in the minority.

Reasons:

• Motivational: A desire to maintain a positive assessment of our own judgment.
• Social: We interact with people who agree with us
• Context interpretation: We assume that given the same context, everyone will infer the same thing. As we see others in the same context, we assume they come to the same conclusions as us. As a result the false consensus effect is strongest for beliefs we attribute to external factors (buy stock in Ford or Google), and weakest for ones we attribute internally (name your son Jacob or Ian). The context is often ambiguous, and different people resolve the ambiguity in different ways, often without noticing it.

Inadequate feedback from others: People are generally reluctant to openly question another person’s beliefs (Adults, that is. Children do it all the time). Only intimate friends and relatives can be counted upon for honest feedback. More casual acquaintances often side-step the awkwardness of disagreement and thus leave us without essential corrective feedback. Because so much disagreement remains hidden, our beliefs are not properly shaped by healthy scrutiny and debate.

III. Examples of questionable and erroneous beliefs

(not summarized)

IV. Where do we go from here

11. Challenging dubious beliefs: The roles of social science

The real purpose of [the] scientific method is to make sure Nature hasn’t misled you into thinking you know something you actually don’t know. R. Pirsig, Zen and the Art of Motorcycle Maintenance.

To avoid erroneous beliefs it is necessary that we deveop certain habits of mind that can shore up various deficiencies in our everyday inferential abilities. Fortunately, there is reason to believe that these corrective habits of mind are not difficult to develop. Students familiar with the work on errors and biases readily apply the learning to their everyday lives.

Most important mental habit is realizing the folly of trying to draw conclusions from incomplete and unrepresentative data.

Ask: What do the other three cells look like?

Information presented as firsthand is often secondhand or more remote, and from a less trustworthy source. Be sure you know where you information originated before assessing it’s value.

Many of these habits are core to scientific research. Familiarity with that world helps. Gives you valuable exposure to uncertainty and doubt, a healthy skepticism, and the awareness of how hard it can be to really know something with certainty.

Exposure to the ‘probabilistic’ sciences (psychology, economics, social sciences, etc) may be more effective in teaching these habits than the ‘deterministic’ (physics, chemistry, etc) sciences.
Probabilistic sciences deal with phenomena that are not perfectly predictable, and with causes that are generally neither necessary nor sufficient. The death of a spouse is associated with deterioration in health. However not all partners health deteriorates (not sufficient), and people’s health deteriorates for other reasons (not necessary).
To compensate for this lack of determinism, probabilistic sciencists must be aware of statistical regression, sample bias, and the importance of control groups.

Social sciences deal with everyday phenomenon, so it easier for them than physical sciences to make the probabilistic tools of reasoning readily understandable and available.

Social scientists should no longer suffer from physics envy. They cannot match their explanatory power or predictive precision, but specifically because of this, they are better equipped to deal with the messy, complex phenomena of real life. Their tools and process, rather than their content, may turn out to be the social sciences most valuable discoveries.

Telnet to your server:

telnet 127.0.0.1 11211

List the items, to get the slab ids:

stats items
STAT items:3:number 1
STAT items:3:age 498
STAT items:22:number 1
STAT items:22:age 498
END

The first number after ‘items’ is the slab id. Request a cache dump for each slab id, with a limit for the max number of keys to dump:

stats cachedump 3 100
ITEM views.decorators.cache.cache_header..cc7d9 [6 b; 1256056128 s]
END
stats cachedump 22 100
ITEM views.decorators.cache.cache_page..8427e [7736 b; 1256056128 s]
END

Thanks to Boris Partensky in the Memcached group here

There you go!

via the BootUp Labs Blog.

• Graham King on identi.ca
• Graham King on Twitter

The nature of the medium means those thoughts will be generally raw and truncated, but timely.

Last chance to invest in a firm favourite

They are covered again in more detail in this paragraph:

The x y z Fund only launched six months ago, but has already attracted considerable interest. To keep it small and flexible the number of units has been capped at 200 million. Last week they had reached two-thirds of that total and interest is intensifying. In the last two days alone they sold over 6 million units, so it is likely to close very soon.

Social Proof

The first principle is Social Proof. This fund is “a firm favorite” and “has already attracted considerable interest” and already “sold over 6 million units”. In other words, other people think it’s a very good idea to invest in this fund, so you should to.
Many of the top researchers in the field of influence (social psychology, behavioral economics, etc) worked with the Obama campaign, Their last minute get-out-the-vote message was “A Record Turnout Is Expected”. People are more likely to vote if they think other people will vote. The most beautiful part of social proof is that it is undetected – most people will deny that this is the case.

No-one know which fund to invest in, because most of them under-perform the index, and if the stock market really is a random walk, then there is no sane way to invest. It is in exactly that type of ambiguous situation where people look to other people to decide how to react.

Scarcity

We want what we may not be able to have in the future, that’s the Scarcity Principle. This is the “Last chance” to invest in this fund because it is “likely to close very soon”. Fear of losing out on something can be an extremely powerful motivator, and you see it every day when stores have end-of-line or closing-down sales, or when something is offer in ‘limited quantity’.

Invest with your calculator, not your heart

That email is an emotional appeal, not a rational one. Selling a new fund during a recession must be difficult, and there’s no reason to think this one is selling well. The language doesn’t promise anything concrete, so it doesn’t lie. It just influences. And that is good copy writing.

The seminal book on this topic is Robert Cialdini’s “Influence”, which I highly recommend. It’s a New York Times best seller, and Fortune Magazine lists Influence in their “75 Smartest Business Books.” You see what I did there? :-)

Luckily for me, message queues are this years Hot New Thing, so there’s some good options. I looked at RabbitMQ, Gearman, Beanstalkd and StompServer.

I’d like the message queue to play nice with Python, with Ubuntu, and take almost no memory, as I’m on a Virtual Private Server, and I’d like it to stay up forever. I want small and solid.

Summary

Memory size is the resident set size, obtained like so: ps -Ao pid,rsz,args | grep <name>. If there is a better way of estimating memory please let me know in the comments.

RabbitMQ

An all-singing all-dancing “complete and highly reliable Enterprise Messaging system”. With language like that you’d expect horrible bloat and per-cpu licensing, but happily that’s not the case. It’s straightforward to setup and relatively lean.

The protocol, AMQP, comes from the financial world, and is intended to replace Tibco’s RendezVous, the backbone of most investment banks. There’s lots of documentation, lots of users, a healthy ecosystem, and it looks good on your CV.
I tried RabbitMQ first, and liked it so much I almost stopped my evaluation right there and deployed it.

The best tutorial for using it from Python is here: Rabbits and Warrens

Publisher

Quick Code

Consumer

Quick Code

Gearman

Gearman is a system to farm out work to other machines, dispatching function calls to machines that are better suited to do work, to do work in parallel, to load balance lots of function calls, or to call functions between languages.

Developed by Danga Interactive (essentially Brad Fitzpatrick, who brought us Memcached and Perlbal). Used by LiveJournal, Digg and Yahoo.

Ubuntu users: Make sure you install package gearman-job-server, which is the newer leaner C version of Gearman. Don’t install gearman-server, that is the old Perl version. Also install package gearman-tools to get the command line tool.

Client

Worker

Beanstalkd

Beanstalkd is a fast, distributed, in-memory workqueue service. Its interface is generic, but was designed for use in reducing the latency of page views in high-volume web applications by running most time-consuming tasks asynchronously.

Developed for a very popular Facebook Application. The smallest memory footprint: after startup, connecting, sending a few messages, it’s resident memory size (rsz) was still only 0.5 Mb!

To install the server:

• sudo apt-get install libevent-dev
• wget http://xph.us/dist/beanstalkd/beanstalkd-1.3.tar.gz
• tar xvzf beanstalkd-1.3.tar.gz
• ./configure
• make (there’s no install step, it just generates the file ‘beanstalkd’)

To install the Python library:

• wget http://pybeanstalk.googlecode.com/files/pybeanstalk-0.11.1.tar.gz
• extract it
• sudo python setup.py install

There’s a good tutorial here: http://parand.com/say/index.php/2008/10/12/beanstalkd-python-basic-tut orial/

Producer

Quick Code

Consumer

Quick Code

StompServer

StompServer is a lightweight pure Ruby STOMP server.

To install the server on Ubuntu:

• sudo apt-get install ruby-dev rubygems
• sudo gem install stompserver

To install the Python library:

• wget http://stomppy.googlecode.com/files/stomp.py-2.0.1.tar.gz
• extract it
• sudo python setup.py install

There’s a good Python / Stompserver tutorial here: http://morethanseven.net/2008/09/14/using-python-and-stompserver-get-s tarted-message-q/

Sender

Quick Code

Listener

Quick Code

Results and Conclusions

I’d be happy working with any of these four. All four were easy to setup, fast, decent in memory consumption, and had good Python libraries.

RabbitMQ has the most mindshare (it is the only one which registers on Google Trends), but it took the most memory and is the most complex to use. It looks like a great product, but it’s Message Oriented Middleware, not an in-memory job queue, so it’s not what I’m looking for.

StompServer had the least documentation, and took several times more memory than Gearman and Beanstalkd. In seems the most immature project, but would probably be a good choice for someone working in Ruby.

Beanstalkd is great. I would like to see it in the Ubuntu repositories, and it’s Python lib in PyPI, but aside from that, I can’t fault it. I’m not choosing it, because Gearman is even better.

Gearman was designed for exactly the problem I have, takes almost no memory (1.4Mb), has a great pedigree (Danga), is widely deployed (LiveJournal, Digg, Yahoo), is in Ubuntu, has a Python library in PyPI, and someone helped me out on the #gearman IRC channel straight away. It even has queue persistence and clustering. So, Gearman it is.

In response to monkeys stealing his coffee beans, an Indian farmer observes: If you start shooting monkeys, you’ll spend the rest of your life shooting monkeys.

via Bruce Eckel