Students’ Understanding of the Equal Sign Not Equal, Professor Says

When I saw the above headline, it automatically became in my mind a candidate for stupidest headline of the week. It turns out, though, that there’s some actual insight lurking in the study referred to in the article (find the summary here).

The example of an incorrect but common view of math is described by a simple example. 4+3+2=( )+2. Good math students say the answer is 7. Bad (but in the US very common) math students say the answer is 11. Given just that fact, I would have thought, “Huh? How the hell do you come up with 11? You have to be a moron!” It turns out, all you really need is to have been taught to solve problems without ever being taught the meaning of the arithmetic involved.

And to learn this, we go to an insightful video TED talk by a teacher named Dan Meyer (Math class needs a makeover) in which he talks about “paint by numbers” math (a wonderful phrase I found in the side notes but don’t remember him actually saying in the video). In paint by numbers math, kids learn how to plug numbers into a pre-existing equation and crank out a result. Meyer disparages this kind of teaching and the students who excel at it, which I think is quite unfair to those students. They’re just doing what they’re told and trying to do it well. And a good number of those students go on to learn real math. Okay, a bigger number of them go on to be assistant managers at Wendy’s but that’s not the point. The point is that if satisfying the teacher is that easy, blame the teacher.

(For the record, I was one of the students who knew there was an equation but if I didn’t remember it off the top of my head, wouldn’t bother to look it up because that’s BORING).

Anyway, back to the failure to get the truly obvious answer in the sample equation. Remember that? (4+3+2=( )+2) Those of us who continued taking math and science in high school even after reaching the point where it was no longer required understand that the equals sign (=) means “equal.” More specifically, it means what’s on one side of that sign is equivalent to what’s on the other side, even if it looks a little different. Therefore, the answer 11 makes no sense for the parentheses because it means one side (4+3+2) adds up to 9 and the other side (11 + 2) adds up to 13 and 9 is not equal to 13. How could anybody think those things are equal?

Go back to the tragedy of paint-by-numbers math, which does not teach the students that both sides of the equals sign must be equal. Instead, it teaches them that an equals sign means “fill in result on the right.” (This is my interpretation, not the reporter’s or researcher’s). So they add up the left, fill in the result in the parentheses and then add that to 2 to get 11. Simple. Straightforward. Wrong.

When I saw the explanation (what the researchers called a running equals sign because you had to add another one to come up with the final 11) I was stunned. It turns out, people who can’t do simple arithmetic may not be stupid after all! They just learned to do things a stupid way.

To be fair, I suspect that this “result goes to the right” attitude is almost hard wired into the brain. It’s the easy way to get by. In early experiences with arithmetic, it’s the minimum you need to learn to seem to succeed. It satisfies the teacher while putting very little stress on the brain. It may take real effort to push that sort of thinking aside and get people to think about the equality symbol in more abstract terms.

The researchers went to some effort to check textbooks for the way they taught the equals sign (Chinese textbooks. Don’t ask. It doesn’t need to make sense for someone to give a grant to study it), apparently looking for an easy way to fix the bad understanding of math. This was obviously a blind alley because no one learns math from a textbook. Think about it. CSPAN is the most boring channel ever invented (except maybe the Golf Channel but that’s another discussion), yet it exists. There is no Math Channel.

The whole point of looking at the way students see the equals sign is as an entry into more complex math. It’s a reasonable predictor of (lack of) success. If you never learn that the equals sign means that things are equal, the chances are pretty good you also won’t learn the limit of dy/dx as x approaches zero is an important number even though it’s not the answer (You see, the limit might be zero but that means x won’t be, though it’s close enough for government work and higher math. It’s very abstract. Trust me). In other words, if you don’t pick up the most basic concepts, you’ll never get the hard ones, either, /no matter how hard you try/.

The thing to remember here is that this is a failure of teaching. It may be self perpetuating. I remember reading somewhere that most high school math teachers are actually some other kind of teacher (English, Social Studies, whatever) who got pulled in to teaching math because there weren’t enough math teachers, not because it was what they wanted to do or what they were trained in. If these teachers had some of the same deficits in their own math skills, they might not even recognize that the “answer goes to the right” method is something students need to be trained out of.

Dan Meyer (see the video I linked above) thinks the answer is to use extended projects and let kids figure things out for themselves. The example he gives is brilliant (Don’t take my word for it. Watch the video). I have serious doubts about this as a model for math education beyond about the second grade. People figuring things out on their own took thousands of years to come up with advanced stuff like calculus. There’s even a book about the history of the concept of 0 (Zero: The Biography of a Dangerous Idea). Apparently, zero has not been a completely obvious concept for all mathematicians throughout history, though it does help in calculating my disposable income. This is an indication that we might want to actively explain some concepts to students before the end of their first thousand years of education, just to speed up the discovery process a bit. (But for, as Meyer says, teaching mathematical reasoning, some projects may be great.)

Those kinds of arguments aside, and ignoring the silly diversion into textbooks, the researchers who found this strange misunderstanding of the equals sign have shined a light on how math is learned and how early math education can go wrong. This does not make me any less impatient with people at the checkout line who can’t seem to make change without taking their socks off but hopefully it will give people who care about math education a clue about what can go wrong.

Paint-by-numbers math = bad. Understanding the equals sign = Less pain at learning real math.

Sometimes the mind wanders. Sometimes complete sentences come out.

1. Buying textbooks is a decent measure of the priorities of a school. Going to the bookstore is always an exercise in horror. Even if I have the money, I’m still shocked to find out how much they want for every title. Then when classes are done, there always seems to have been at least one book – costing anywhere from $65 to $200 – that you never even cracked the cover on. They didn’t care much about the class or the needs of the students, did they?

2. More and more schools are responding to the criticism above by offering e-books as an alternative. These cost maybe 20% less than the hard copy but they are invariably so loaded up with DRM  (http://en.wikipedia.org/wiki/Digital_rights_management)they are almost useless. For general reading, I often buy e-books. For classes, I spend the extra cash on a physical book. Either way, I curse the publishers, the school, the professors and my bank account, not necessarily in that order.

3. For a long time I thought that it was a sign of getting older when I would open a web page and instantly hit the hot keys to increase the font size. It happens a lot. It might also be a sign of setting my screen to a high resolution because there is NEVER enough real estate on the monitor (Even when I use 2, apparently). Either way, I’m starting to think that someone, somewhere should at least try to figure out if their pretty design is at all usable by people like me. The average age of the population keeps going up. This probably does not indicate that the average level of patience with tiny text is also going up.

4. In college programming courses, I learned almost nothing about testing. Come to think of it, I didn’t learn much about that in science classes either (believe it or not, labs don’t count. That’s rarely more than learning techniques, rather than learning how to test a hypothesis). This raises the musical question, how has civilization survived with so little understanding of how to tell if a thing works?

5. On a similar note (get it? note?), how do systems that clearly don’t work get to be institutionalized? See this fascinating Wired Danger Room article (that I found by way of the indispensable Slashdot) for at least one example of what I mean and a sign that there could be hope. Hope is good. There should be more of it.

6. The concept of “Minimum Viable Product” (http://en.wikipedia.org/wiki/Minimum_viable_product) can’t survive a committee. Every member has their own idea about “minimum” that may or may not have anything to do with what the project really needs. More likely it reflects what someone WANTS it to have (If this were not true, there would be no such thing as politics). So minimum becomes a moving target as we keep trying to cram more features into it. There’s a definite flaw in this approach but it seems to be very common.

7. I recently read a book on managing a project portfolio. I suppose managers would find it a wonderful how-to manual. Typically for today’s management texts it was almost fanatical in pushing teamwork and consensus. It also insisted that all decisions be well grounded in “adding value to the company.” But consensus and consistently adding value to the company are both principles that are deadly to bold decisions and new research. I worry about discouraging those things. Not that adding value to the company and consensus are bad. Of course they’re not. But if your vision goes no farther than that, your endeavor, whatever it is, is doomed and well it should be.

8. Never play leapfrog with a unicorn. It’s one of the only pieces of advice I have always followed.

Have a good summer.

Can an article – written by a professional journalist for a national news magazine – credibly claim that there’s a creativity crisis in America? Isn’t the act of writing the article itself creative? Doesn’t that mean something?

Well, no.

By way of Slashdot (here) I found a Newsweek article (here) that made the highly controversial claim that American children (6th grade and under) are less creative than previous generations and advocated project-based learning in the classroom as the “scientific” solution.

I really wish that people who write about science would try learning a little first.Really I do.

Let’s start with the setup: A longitudinal study by E. Paul Torrance (Wikipedia bio; obit; Books by Torrance on amazon) in which young children were tested for creativity, then followed for decades and their creative achievements recorded. The conclusion was that it was a good test, that people who scored high in creativity while very young, often went on to be highly creative adults. Longitudinal studies (Wikipedia definition here), by the way, are hard to do well but can lead to very rich data sets that can be useful for far more than originally intended.

So far we’re in “duh!” territory. The big take-away is that psychologists were thrilled and amazed to find out they could measure creativity. They may also have been wrong but we’ll get to that. Psychologists were also interested to find out that creativity and intelligence did not necessarily go together. Again, “Duh.” Anyone who’s ever seen an interview with Ozzy Osbourne (or any of a hundred others I could name – sorry Ozzy. You’re still great!) could have told you that. Of course, they are not mutually exclusive either. Frank Zappa proved that!

This test has been given lots and lots of times to students all over the world. A researcher, Kyung-Hee Kim (web page here) analyzed the data and determined that the test scores for American children have declined since about 1990. That’s kind of interesting but not, by itself, a crisis. I was disappointed when my grandson refused to wear the TV remote for a hat, even after I modeled it for him, but he’s still a good kid. He might even have some capacity for creativity. Just not in hats.

The Newsweek article, however, describes this decline in scores as a national crisis but, in fact, gives very short shrift to the question of how creative the kids are really or why their scores might be declining. It assumes without the slightest evidence that this is a permanent trend and that America has to do something right away. Then it goes on to describe the prescription: Project based learning in the classroom.

I’m not going to critique project-based learning. It sounds great when the advocates describe it. But the author’s agenda comes out, for example, in the use of the term “drill and kill” to describe rote memorization. “Drill and kill” is a pejorative term used by people who don’t like rote. The author accepts without question that it is bad. There is room for disagreement on this point.

Leaving aside issues of the relative merits of different teaching styles and the very contentious politics associated with them, I was struck by the implication that “science says” we have to start teaching American kids to be creative because they aren’t learning it and if we don’t change the schools to teach it to them, they’ll never learn it and the rest of the world will pass us by! The evidence to back this up just isn’t there.

One point that stood out to me was that the alleged decline in test scores (no reference is given to any validation of either the methodology or the conclusion, therefore I consider the point unproved) means there is also a decline in creativity among the very young. There is a vague mention of TV and video games as possibly suppressing creativity but no real explanation. But, if those things are really involved, how do we know that they aren’t causing children to express their creativity in a way the test isn’t very good at measuring? Would the test results change if the questions were given by bright cartoon characters on TV? What if the reason the kids are performing differently on the tests is because of increased education about “stranger danger?” Not trusting the tester will affect a lot of tests. Has anyone tested that?

What if creativity really is declining but TV and video games are not the cause? Ask any doctor how easy it is to cure a disease without a diagnosis. Treating the symptoms can buy you time but if you don’t have a good idea of the underlying cause, you may be completely helpless to keep the patient alive (Watch a couple episodes of House for a nice, if exaggerated, illustration of this. Most episodes have at least two attempts to cure the wrong problem, followed by the patient getting even worse, then House comes up with a miracle cure. Yes, it’s just TV. Take it with a grain or 12 of salt. I’m trying to make a point here, not prove a case in court!).

In this case, without any clear idea of a cause, we can’t even be sure there is a problem to cure. Psychological tests can be sensitive to cultural and, well, psychological factors that are not necessarily obvious to the researchers involved with them. And creativity, for all the research that’s been done, is still imperfectly understood. What if the (alleged) decline in creativity is due to changes in diet that change the balance of important brain chemicals? Or what if it’s just a passing thing, a statistical blip that will change in the next generation? This is why I get so annoyed with science “reporting” these days. Even a tiny bit of critical thinking would be better than what you get from most science related stories these days, especially any science related article with the word “crisis” in the headline!

I have nothing against project-based learning (though building an entire curriculum around it seems a bit much) and I’m certainly willing to believe that it’s a good thing to teach children to be creative. If nothing else, it makes for a fun childhood. But I don’t see a crisis and absolutely don’t see a basis in this for educational policy. I see opportunities for more research and for developing our understanding of creativity in childhood and even beyond. Let’s try that.

I had a slightly weird encounter yesterday with Google Social Search. This is a beta product (which in Google-land doesn’t really mean anything) that shows you results from your search that are found via your “social circle.” I ran a search and noticed this new and unusual thing at the bottom of the first page of results.

At first, I thought it was amusing. Then I thought it was creepy. Then I decided it was just annoying. Let’s examine the meaning of this service by going through each of these points in turn.

Amusing: My search was a catch all for material on an academic subject. It doesn’t matter which one. School’s out but I’ve been gong to school so long, sometimes my brain just gets in that mode. I had already tried searching Google Scholar and found some interesting stuff, and a lot of other stuff that I could not afford to buy. The ridiculous price of so many scholarly and scientific publications is a pet peeve of mine (I don’t mind them making a buck. I just mind that they jack up the prices so high that published research is effectively hidden from most of the world, especially me). So since I didn’t have hundreds of dollars to shell out for a very few articles that might or might not be relevant, I decided to broaden the search and see what regular Google would bring up.

There was a lot of useless cruft, as there generally is. But at the bottom of the page I noticed my boss’s name. That seemed odd to me. His blog (http://ribbonfarm.com) is well read but I didn’t think it was that popular or that relevant to my search that it would be on the front page of the search results! So I looked at the referenced post. It was 2 years old, I had read it when it was new, and it was completely irrelevant.

Then I noticed the header that said something about my social circle. Hmmm. Yes, my boss and I have chatted using Google chat. We don’t very often, partly because our work is covered by a non-disclosure agreement, so over an uncontrolled forum like Google chat (or anybody else’s chat) there’s a limit to what we can say without violating confidentiality. So we do most of our discussions over work-based email. We don’t do them in person because we work in different states and rarely see each other but that’s neither here nor there.

We come here to the question of the definition of “social circle.” My boss is a good guy and I consider him a friend, so I don’t have a problem with him being included in my social circle (This is an important thing. We’re both highly opinionated and a bit hard headed. If we weren’t on pretty friendly terms, we’d probably kill each other). I have had bosses in the past who would make me feel exactly the opposite. I work in the IT field, where 24/7 availability is more or less the norm. That means sometimes you use non-work channels to get in touch with people. This means that Google’s definition of a social circle may contain any number of inappropriate people. What about the times I’ve emailed tech support at some company, or complained about a product? They are decidedly NOT part of my “social circle!”

On Google’s page describing the social search (http://www.google.com/support/websearch/bin/answer.py?hl=en&answer=165228) which I found after some hunting, there is a description of how to remove irrelevant stuff like that from the circle of friends so it won’t be considered in social search results. It seems to be possible to just tell Google not to use someone when doing this search, though I haven’t tried it. What this means to me is that this feature I didn’t ask for puts the onus on ME to fine tune it to avoid seeing results I don’t want. Picture me banging my head against a wall at this point. I won’t actually do it because I hate pain but come on!

Actually, after looking over the options for how to remove such things from my social circle (and the little disclaimer that says it can take weeks for them to actually disappear from your search results. Way to be responsive Google! Thanks!) I’m thinking the only real option is to not use Google. For anything. Because they cull everything you use for social connections.

This brings us to my next emotion about social search: CREEPY.

The use case they describe in their documentation is getting a movie review. They say that movie reviews from your friends will be more relevant to you than movie reviews from some unknown professional reviewer somewhere. Well, there’s a good point there. I have no respect at all for professional reviewers. The other night a friend and I were watching a movie review TV show and I remarked that when the critics use terms like, “real emotion,” “honest” and “true to life” you couldn’t pay me enough to watch whatever it is they’re reviewing. Those may be fine artistic qualities but do not, in my experience, make the product very entertaining. Unless you’re the kind of person who thinks that emptying a box of Kleenex because you’re crying so hard is a lot of fun. That’s not me.

On the other hand, I have NEVER IN MY LIFE USED GOOGLE TO FIND A MOVIE REVIEW. Are there a lot of people doing that kind of thing? Why? The world is full of movie review sites. I would expect that anyone who is interested in movie reviews is already familiar with rottentomatoes.com or similar sites. There’s no need to search for that. I’d be interested to see what numbers Google has for those types of searches. In other words, does this use case have any relationship to reality or is it just an excuse to jump on the social networking bandwagon?

Facebook, the current leader of social networking technology, has gotten into repeated trouble for taking people’s information about themselves and their friends and using it for more than just to let people share a laugh with their friends. When Google rolled out the execrable Google Buzz, they got in to similar product because users of Gmail (myself included) thought they were getting an email service and did not expect or intend to be advertising their whole lives to the world (see my post about Buzz here).

So now they’re doing it again. You search for stuff and they show you irrelevant results from people you have had some contact with in the past, no matter how slight or even hostile that contact may have been. Does Google understand that what they’re doing this way is actually CHANGING your social circle?

Just as an experiment, I signed up for a service called Gist a while back. It aggregates stuff from the contacts you supply to it and tries to rank them for importance. One of the things I noticed was that the service gauged importance by how often some of these people posted to their blog or to Twitter, not by how often they had contact with me. So people I have only very slight contact with were shown very high in the listings merely by virtue of being busy. People I’ve had contact with but hardly ever think of were ranked as important, while those I truly care about were virtually ignored.

There may be tools to fix these rankings. I don’t know or care. I’m just trying to illustrate a point that Google social search does something similar. People I may have never had much contact with in the past, because they were never more than casual contacts, could still have their stuff show up in my search results, simply because they are active in blogging or Twitter or some other such thing. And, like anything else in search, putting them on the front page gets them more clicks, thereby increasing their importance (at least as far as Google is concerned). There is also likely to be a psychological effect that the people whose stuff you click on increase in importance in your mind (There’s an opportunity for someone to do an interesting thesis here).

Google has turned and information search into a social feedback mechanism. I’m not comfortable with this at all.

True, to some extent, all social networking does this. On Facebook I have connections to people I haven’t seen in years, or have only met a couple times. On LinkedIn I have quite a few connections to people I have never met and only know by reputation. But that’s what LinkedIn is for, so it’s okay. On Facebook, my connections are intended to carry some emotional import and I appreciate the news updates, even from people I don’t know incredibly well. It’s a chance to get to know them better and I like that.

But search? When I run a search for no sql databases, or research on trusted systems, or encryption libraries or any of a billion other topics that might grab my attention for a few minutes (All of the ones I’ve mentioned are related to courses I’ve taken, or to my job, or both), having my relationships vetted and subtly influenced at the same time is NOT what I want.

Here’s another little bit that has problematic implications: “If someone you don’t know shows up in your social search results, it’s likely that they’re connected to someone you do know.” So now Google is recommending friends. It’s annoying enough when Facebook does that. No, I don’t want to connect with the lead singer of a band that my family is all connected to (unless it’s Flogging Molly). You see, most of my family is half my age and has one millionth of my knowledge of and taste in music. Leave me alone!

Which brings us to the annoying part. There are probably many people who think that having this sort of thing integrated into search results is interesting and fun. There are probably even situations where I would find it worthwhile. I can’t think of one but it’s possible. But Google didn’t ask me if that was what I wanted. I don’t see a place where I can choose “add social search to my results.” And it didn’t show up at all in a test search I ran just a minute ago. I have no idea why not. The ways of Google are not our ways. Their thoughts are not our thoughts.

If they’re thinking at all beyond the dreaded programmer’s cry, “Hey! I just thought of a cool new feature!”

Continuing the subject of bad science (previous installment posted as Who writes this stuff anyway?), we have a study (described here) that explains that people with jobs requiring a lot of creativity often feel overworked and may find themselves sucked in outside work hours.

Sounds like an ordinary IT job to me!

Anyway, Like most science these days (or maybe it’s just science reporting, though I suspect it’s both) they seem to be unaware that correlation is not causation. What that means in this case is that it may NOT be that it’s the creativity required by the job that causes the result. It may be that people who demonstrate the capacity for creativity may get loaded up with work because, well, because that’s what it takes to get it done. Anyone who has ever supervised others knows that for a tough problem, you need someone who works hard, thinks sideways (I was going to say “outside the box” but that would be the opposite of creative, wouldn’t it?) and doesn’t let go of a problem just because the work day is over.  You want someone who will solve it for the pleasure of solving it, not just for the money or because someone who told them to.

When you find those (few) people, you treasure them. You also work them just as hard as you can get away with because there are more problems to be solved than good creative problem solvers to throw at them.

Again: Sounds like a basic (good) IT worker and an average IT job. I suppose other jobs may have similar characteristics. I just haven’t had one of those.

Interestingly, the study seemed to find that these creative people don’t mind having their work impinge on their lives outside the job. The hypothesis offered is that it feels good to them so what’s the problem? The thing to remember here is that, before starting the study, the scientists (and I use the term loosely since we’re dealing with sociology) probably didn’t know that the results would be so trivial. It would have been more dramatic if they had found a high rate of near-suicidal burnout among these creative workers but that doesn’t seem to have happened. But they couldn’t have known that before doing the study, so cut them some slack. I mean, how would social science types know that creativity is its own reward?

A more interesting question to study (take notes for future grant proposals) would relate stress to creative fatigue. If creativity is its own reward, does there still come a point where you’ve been so creative for so long (or so much in one particular area) that you just can’t think of new ideas anymore? How much stress do you feel then? How does it affect behavior both at work and outside it?

Insert joke about your favorite has-been author, producer, composer, programmer or scientist here. I’m going to take the better part of valor and not offer any jokes of my own.

The title of this post is something I once heard a newsroom editor yell (in slightly less family-friendly form) while editing the news. Being a sciency type myself, I am most likely to have that feeling when looking over the science news. The headlines reproduced below are from the last few days and I just couldn’t resist commenting on them.

New gene therapy proves effective in treating severe heart failure

You mean, there’s such a thing as mild heart failure? For the record, I don’t want that either.

Link identified between lower IQ scores and attempted suicide in men

The key word is “attempted.” The smart ones succeed.

Eyes of cattle may become new windows to detect mad cow disease

Yes. Especially when they’re red and glow. Stay away from those cows. (believe it or not, the article actually discusses looking for glowing bits in the retina, under a microscope though. Much less funny when you put it that way).

First Paper ‘Dipstick’ Test for Determining Blood Type

Wait – is this a repeat of the one about suicide? (Dipstick test. get it?)

Apologies may fuel settlement of legal disputes, study says

Well, more so than bullets, I suppose. I want to know what government agency actually spent money to find this out.

Visual system interprets sign languages

Ummm, isn’t that why it’s sign language and not , y’know, spoken? Also repeat comment above. (Okay, okay. The visual system discussed in the article isn’t the human vision system. It’s an artificial system. Couldn’t the headline have said, “device” or something?)

Study finds poker players using drugs to enhance performance

The other guy’s performance, actually. At least that’s what my father taught me. He said when in a hot game, try to drink less beer than the other folks at the table so you won’t play as drunk as they do. Also, repeat comment repeated above.

What are the most effective strategies for secondary suicide prevention?

Maybe it’s just me but I tend to think that the primary suicide would prevent any secondary. Not to mention pretty much everything else. Also, repeat …

(Note: I wanted to work in a dipstick or mad cow joke on this one but couldn’t think of one and still make it to dinner on time. Let me know if you think of a good one.)

Believe it or not, I don’t read the science headlines just to make fun of them. Sometimes, though, it’s almost mandatory. Usually the articles are not nearly as silly. Usually.

A while ago my boss and I had a discussion about how to interview candidates for a programming job. Not being a programmer himself, the boss wanted me to supply some good questions that would show depth of programming knowledge, especially Ruby on Rails, which is what our site (http://trailmeme.com – tried it yet?) is built in. I think he was a bit surprised that I had very little interest in asking those kinds of questions. Once I explained myself, he was willing to trust my judgment (which is a nice thing to have in a boss). But he isn’t the first person to find my approach to interviewing candidates to be different from the norm. This tells me that the rest of the world is doing it wrong and there is need for me to explain some basic principles for the benefit of all those less enlightened than myself (for those who are humor impaired, just ignore the completely insincere self-aggrandizement in the previous sentence and move on).

I remember being interviewed for a position once where the interviewer asked questions straight out of What Color is Your Whatever that Silly Book Was? One of the questions was, “What would you say is your greatest weakness?” I decided this would be a bad time to mention my disrespect for people who ask questions like that. Instead, I made up a line about not thinking in terms of weakness, instead playing to my strengths. He liked that answer but didn’t hire me anyway. Maybe he thought I was trying to hide a deep disrespect for authority. More likely he found someone he could get for less money.

Anyway, this is lesson one: Don’t ask questions that encourage people to be less than completely forthcoming or honest. Sometimes that may be hard to avoid. “Why is there a 2 year gap in your education?” If the answer is, “I was on trial for murdering a professor but got off because he deserved it,” it is very unlikely the interviewee will actually explain this. On the other hand, who cares about a gap in education, or even in employment? You can read a lot of programming stuff while waiting for the jury to bring back a verdict.

So what kinds of questions do I think are worth asking? I always ask programmers about the tools they use. What kind of IDE (for those readers who are not programmers, that’s a sort of programmer’s word processor), what version control. what favorite testing framework? I ask these things because a really great programmer doesn’t like to do extra work and good tools save work.

More importantly, I ask these questions because in ANY field, someone who has done it for a while and faced many different problems and types of conditions will develop opinions. A car mechanic might develop a preference for Chevys over Fords because the failure rate of some part, and the difficulty of replacing it, is unacceptable. Someone involved in developing commercial real estate might prefer working with a particular contractor over another because of the quality and timeliness of the work. The preferred contractor might prefer a particular building material over another for longevity or ease of installation or even for some completely irrational reason, like having once used that material on the same day the Yankees lost the series (it happens, trust me).

When you work with stuff, you naturally develop opinions about it. This may seem like a radical idea but I’ve found it to be very dependable. Humans are good at having opinions. In fact, people often have opinions without working with stuff. You can usually figure out who those are when you ask them why they hold an opinion. Someone who has worked with a language or an IDE or some software, will tell you horror stories about what they went through with other languages or packages. That is, they’ll describe their experience. Someone without experience will rarely be that specific, or that passionate. Worse, they might try to be fair and thoughtful.

I have a stock question that goes something like this: “Suppose I assigned you to develop a social networking type website. You own the project. You have complete control over the tools you use. Programming language. Operating system. Version control. Data structures. Everything. tell me a little about the things you’ll choose and why you choose them.”

My theory is that, even if the person makes choices I would not, if they have reasons for those choices, I can respect their opinion. So far, this theory has not been proved. I have heard some pretty bad answers to the question. The worst one is the one where they try to seem professional and thoughtful. “I would choose whatever tools were appropriate for the client’s needs.” To me, this is another way of saying, “I have no idea and I’ll be damned if I’m going to commit myself to anything that you could maybe use against me.” This person will probably never be a good programmer. Programmers make decisions every day. Choosing to use some form of internal caching over a database is a decision that has far reaching consequences. if you don’t have the courage to make the decision, you’re going to hold up development while you try to get someone else to take time away from their own job to decide for you. [Note: I'm not claiming this is a perfect example of the idea. It's just an illustration]. Consultation is fine but you should have a preference and be able to express it or what good are you?

My questions seem to be better at getting bad answers than good ones but I find the bad answers very illuminating. If I ask what kind of IDE someone prefers and the answer is, “What’s an IDE?” That tells me I’m not dealing with a professional programmer. NEXT!

Try the alternative. If I ask, “How would you solve the n+1 query problem in Ruby on Rails?” It sounds like a good, solid, technical question. The trouble is that it can be answered after spending about 2 seconds with Google. (The answer is eager loading … usually. But it’s a job interview. If you give me a 20 minute answer going over all possible special cases I may worry that you’ve given it too much thought and really need a hobby). It’s also in all the books about Rails. It doesn’t tell me much about your skill level. (Though, again, someone who answers, “What’s that?” Has just flunked computer science 101 and might not get a lot more of my attention)

On the other hand, if I ask, “What do you use for version control” and the answer is, “At my last job we used SourceSafe” that tells me that you haven’t given a lot of thought to the question of version control but at least you know what it is. If, on the other hand, the answer is, “At my last job they used SourceSafe but I hate the locking model. For my own projects I use Git because there’s no problem with locks and branching and merging are almost effortless. Plus I can use it on the road when network connections are spotty” – this tells me that I’m talking to someone who has given actual thought to productivity. (Okay, actually you can get this answer from 5 minutes with Google. But you have to want to).

I guess the bottom line is that when I ask a question of a job candidate, I don’t want answers from a textbook. I’ve been interviewed myself when this was avoided by asking for a specific example of something I did on the job. I’ve usually been able to come up with answers. It’s not always easy. In  an interview situation where people may be nervous and even exhausted (ever been the 12th person to interview somebody who has been doing serial interviews all day long? Don’t expect coherent answers) that may be asking a bit much.

It’s also possible that I just like to torture people with unexpected questions. But they’re questions a pro should be able not just to answer but argue about. Arguing for your ideas is one of the basic skills needed for good teamwork. Hey! I never thought of that angle before. Maybe I’m on to something more interesting than torturing candidates after all.

I’ve been taking a class in computer forensics and, possibly because the textbook is very dull, sometimes my mind wanders to odd implications of what I’m reading. There are some known facts about most operating systems that work in favor of forensic investigators. For example, the contents of deleted files linger on a system, sometimes for a very long very long time. The traces can be found and reconstructed by someone with the right tools and know how.

There are times when there are legitimate reasons to try to avoid this. The most widely known of these is when the defense department gets rid of old equipment. It’s important to wipe the data on a hard drive in such a way that it is close to impossible to recover, in order to protect defense secrets. And whatever porn and games the poor defense workers may have downloaded during lunch.

What about resistance members (assuming there are any) in totalitarian countries (assuming they can even get their hands on a computer)? Don’t they also have legitimate reason to hide the traces of what they’ve done? How about spies? When someone from a free country tries to gather hidden information in a totalitarian country (let’s say British spies in Iran, since the Soviet Union is gone and the CIA is not what it once was), being caught could mean torture and death. For them, having an operating system that reliably deletes evidence could literally be a life saver.

That was what got me thinking, wouldn’t it be goo dif those people had access to an operating system that automatically did things to protect their lives?

This is the same principle that is behind the advocacy in some areas of the widespread use of encryption and Internet anonymizing systems such as Tor. Less free countries may define things as crimes that those of us in more free places take for granted, things like advocating democracy, or buying and selling tools for free speech (It was said that in the old Soviet Union, a person needed a license to possess a typewriter), or practicing the wrong religion. Just as a thought experiment, I’m extending that idea to a range of other things.

Besides secure deletion, what other things would an operating system do to protect the life of its operator?

Obviously, anonymized Internet connections would be needed. That’s difficult. Network cards have built in identifiers. It would be nice to forge those without specialized hardware. However the IP address, a higher level identifier, is changeable and systems should change their IP frequently as well as forging the numbers whenever feasible (note: It’s not feasible if you need return traffic to find you. Outgoing email it may be possible but you won’t get any sort of confirmation back.

Required encryption is important. Even two different users on the same computer should have absolutely no ability to read each other’s data without the correct passwords. I remember seeing on some awful TV mini series about aliens, a character played by the wonderful Matt Frewer had rigged his computer so that it would execute something (I don’t remember what) if he failed to log in every 24 hours. On a business computer this would be highly inconvenient but for a spy or “freedom fighter” it could really be a life saver to have the system automatically and securely delete a user’s entire account including all data if they did not log in after a specified period of time.

This is harder to do than it sounds. It can only be done if the system is running. If it is turned off, no dice. That means that the hardware needs to be set up to automatically boot up – on battery power if necessary – at least every 24 hours to check to see if accounts need to be deleted.

A related area of concern is the swap file (or paging file, depending on the system). Modern operating systems can’t keep all the data needed to operate all their running programs in memory at once. Instead, they write currently unused data to the hard drive and read it back in when needed. Typically, a system may be swapping some data out of memory and some other in many times per second. The problem is that there is data in this swap file that can be recovered and used against the owner of the system. Since it’s unlikely to create a workable system that does not use some form of swap file, the only answer I can think of is encrypting it with a one time key generated at boot up. This will inevitably slow down the entire system. Hopefully, things can be optimized to minimize the performance hit. For safety sake, the swap file should still be deleted (securely) whenever the system is shut down.

Even more importantly, though, the same swap file and encryption key should not be shared between different users. That means that logging out and logging in as someone else should not allow data written to swap by the previous user to be accessed in any way by the system during the new user’s session. The best solution to this problem is probably to force a complete reboot whenever switching users. This could be inconvenient in some ways but, so far, I haven’t thought of anything better.

Camouflage is important, too. In the kind of places where people will need this sort of system for the kinds of uses envisioned here authorities might simply outlaw the operating system on the not complete insane assumption that whoever has it must be trying to hide something.So if the system is accessed without the correct passwords, it should pretend to be some more innocuous operating system, like Windows or Mac. It’s tempting here to have it pretend to be something really unusual, like Beos or Next, but that would attract too much attention. We don’t want attention.

A real danger to one of these systems would be having someone copy the hard drive and examine it forensically without ever booting it up.  There is no complete defense against this, though some possible partial defenses include setting up the BIOS to refuse to copy data from user space (unless over the Internet, which is not nearly as desirable forensically) and putting a bomb inside the case.

Hmmm. This is sounding like a tough job. Good deletion is the easiest thing I’ve come up with so far. There’s probably a reason for this. For one, operating systems are hard. For another, most of the things people do take place above the OS level. Here’s a simple example: How about a web browser that keeps no cache and only records urls in history and bookmarks that have nothing to do with places the user has visited? It may not be a great browser but it won’t give too much away about the user when the secret police examine it. You do have to take care that the list of fake urls avoids sites that serve up malware, kiddie porn, and in some cases unapproved news and opinion. Of course, the way those lists can change, keeping the list “secret police safe” may be a tall order. Better not to keep any of this data at all.

One of my basic rules of thumb is that if I’ve thought of something, probably someone else has too. I haven’t ever run across research about such a super private OS but that could just mean I don’t look at the right journals. It could also mean that the problem is so hard it’s hardly worth the effort. Of course, if some agency such as the NSA has done this kind of research – which they should – someone on the outside like I am would never know.

Note to the NSA or other such interested agency: Feel free to offer me a really big grant to follow up on this kind of thinking. I don’t actually know much about OS programming but I can definitely think about it!

It seems that everywhere I look lately there’s news about a new “weakness” found in the RSA algorithm. This has been reported with headlines screaming about the “severe” weakness and how everything in the universe that is encrypted depends on RSA. For examples of those rather overheated stories look here and here.

Let’s have a moment of sanity please. The sky is not falling. The attack described depends on manipulating the power supply of the targeted system, making tiny changes in the voltage to generate bad output from the algorithm. It’s a very interesting attack technique but the actual risk of it happening in the real world is incredibly low. Anyone who can get close enough to manipulate the power to a unit can do lots of other much more interesting things to it.

In general, no one can get close enough to perform this kind of attack.  Locking the doors on the server rooms is a standard IT practice. You see, most criminals who get close enough to attach the equipment needed to play games with the power supply are much more likely to simply unplug it and steal the computer.  We guard against that sort of thing and, incidentally, against creative attacks on the power as well.

This is just one more example (in a nearly infinite list) of why the news should never be taken at face value. Read carefully. THINK. Apply salt liberally and move on to something less ridiculous.

The big tech story of the week is the one about Google making people mad with it’s new “Buzz” service. The most interesting aspect of this story is that everyone seems to have gotten it wrong.

Here’s the short version of the story: Google has some new social media application that makes all your email contacts into “friends” in the social networking sense and a lot of people objected to that, claiming that email contacts should be kept private, not advertised to the world as a friends list. This is stupid on so many levels – Google, their users, all the “analysts” – it’s hard to know where to start. So I’ll start at the beginning as far as I knew it.

The other morning, as I do most mornings, I brought up my gmail account and glanced to see if there was anything new. There was some kind of banner or thing about something called “Buzz.” I immediately thought “Hmm. Could this be a whack at Yahoo’s boring Buzz bookmarking service?” But no. I saw that my boss had already been there and made a comment. I also saw that to reply to his comment I had to create a “profile” that would make all of my email contacts into friends who I could then get Buzzy with, or some such thing.

I decided not to create the profile because I don’t use my gmail account for general email purposes. I have a yahoo account for that. My gmail account is mostly for poetry and other writing. I use it to communicate with the members of the Science Fiction Poetry Association, a lot of editors and a few close friends and family. It’s the kind of account – intentionally – receives the kind of joke emails that people forward all the time. In other words, while it’s a public address, I tend to use it for more private purposes.

Weirdly, Buzz shows that I have 6 followers, including 4 who do not have public profiles – which I also do not have. How do you follow someone who does not have a profile to follow? And if you don’t have a profile, how is it possible to follow someone else without a profile? What the hell is going on here?

Anyway, notice the one interesting bit here: The complaint the privacy advocates have is that this new Buzz thing is advertising information people want kept private and that Google should have given them more warning of that fact. Google did give warning – enough that I decided not to sign up for the thing (but it still tells me there’s new stuff for me to look at there, which I find truly annoying). But, apparently, a lot of people failed to notice the warning and are mad AT GOOGLE FOR THEIR OWN FAILURE TO READ.

Don’t take my word for it. Here are some links to stories about privacy concerns with Gmail Buzz:

• http://www.theregister.co.uk/2010/02/11/google_buzz_privacy/
• http://www.businessinsider.com/warning-google-buzz-has-a-huge-privacy-flaw-2010-2
• http://abh-news.com/google-buzz-privacy-issues-for-gmail-users-1126.html
• http://www.washingtonpost.com/wp-dyn/content/article/2010/02/12/AR2010021201490.html

Believe it or not, this was highly predictable. At a previous job I used to take help desk calls sometimes (It wasn’t exactly my job but it had to be done). One of the things I found amazing was how often someone would call up complaining about an error message when they tried to do something and then not know what the error message was. The conversation went something like this:

Idiot User: “Hi. I’m trying to use [name application here] and it doesn’t work.”
Me: “What do you mean it doesn’t work? Does it give you an error message?”
Idiot User: “Yeah. It does.”
Me: “What does the error message say?”
Idiot User: “I don’t know. I just clicked okay.”

Of course, it’s impossible to diagnosis a problem when the only symptom is that you clicked okay but that’s not important right now. What’s important is that it is perfectly and absolutely normal for people to look for that little “okay” button and click it WITHOUT READING ANYTHING ELSE. For Google’s Gmail Buzz and any other service anyone ever wants to create the implication of this long standing and widely known user behavior is that people will almost alays accept the defaults, even if it is not in their best interests to do so.

As Facebook has shown many times and Google has proved yet again, when people accept the defaults without even looking at them and later find out there was something about those defaults they didn’t like, THEY’LL BLAME YOU, NOT THEMSELVES. Therefore, as Facebook has had shoved in their faces over and over again, forcing users to opt in instead of allowing them to opt out, saves you a lot of bad publicity and hassle down the road.

Yes, the users messed up by not reading. Google’s even bigger mistake was expecting the users to read in the first place (btw: This is an easy mistake to make and despite having articulated the lesson here, I can not claim to be too smart to be immune from this same error. Funny, huh?)

But there’s more that Google did wrong on this one and to understand that, we need to spend a few words discussing social networking theory and practice. Most of the world was introduced to social networking by websites like MySpace, Facebook and Twitter. However, the theory of social networks is not new nor is it restricted to the Internet. The social sciences have long studied the way humans for associational networks and how information and influence travels along those networks.

Also, completely independent of social networking websites, there has long been interest in the way email can be used to learn about a person’s social network. Who do you receive the most emails from? Who do you send the most emails to? A lot can be learned about relationships by studying these things.

I was first exposed to these ideas years ago when I was testing a demo of software being sold to law enforcement as an aid to complex investigations. One of the things the software did was take phone records as input and produce a visual depiction of communication patterns. The idea was that this was how police could find out who was really running the gang they were investigating (though really it would only discover who was running the operations, rather than who was calling the shots but that’s another story). The application to email is obvious.

And this is where Google really tripped up. They have wanted to get involved in the social networking arena for some time (check out orkut.com, for example) but have never found anything that caught fire. Then some genius found out about social science research into using email to examine people’s social networks and thought, “Hey! We’ve already got all their social network info! All we have to do is start using it!”

This completely overlooked an aspect of email that comes up very often when dealing with users (yes, back in my pseudo help desk days): The expectation of privacy. The upshot is that, no matter how many times you tell people that the company reserves the right to monitor their communications, and no matter how often you explain to them that nothing on the internet is truly private, people still think of their email as being private communications. They put their most personal stuff into email, things they wouldn’t want anyone else to know about.

It’s not all just forwarded jokes. It’s stuff that gets dragged into court in cases of sexual harassment, divorce, fraud, product tampering, negligence, even murder (In an unusual twist to that with immense privacy implications, see here). Everything people would ever talk about, and anyone they would ever talk to, can be discovered in their email, including their deepest and most humiliating secrets.

Even people who don’t have humiliating secrets to hide can be very touchy about their email. Even if they only use it for work, that doesn’t mean they want the boss reading it. The flip side to privacy is trust. When someone snoops into someone else’s email, or their contacts, or their desktop files, or whatever, the person whose stuff is being snooped feels distrusted. The response is generally anger.

Contrary to the popular formulation, privacy is important to nearly everyone, not just those who have something to hide. And by exposing people’s email contacts in one huge batch, Google ran head on into this deep need for privacy. They got anger in return. This is the real story. It’s not that Google failed to display their instructions in neon with all kinds of opt in notices to force people to think about what they were doing. It’s that by touching email AT ALL, Google made people worry about who they trusted and who trusted them. Consequently, Google lost trust from some of its users.

In this particular aspect, the users are not at fault. Google made the enormous mistake of thinking of email as a resource to be leveraged. Ironically, they tried to develop a social networking feature without giving enough thought to the social context.

The really funny part about this is that they needn’t have bothered. My second thought when I first saw that there was such a thing as Gmail Buzz, was, “I already have this stuff on Facebook. I don’t need yet another social network.”

UPDATE (same day): I found a link wayyyyy down at the bottom of my gmail page that said “turn off buzz.” So I did. That’s one annoyance out of the way!

UPDATE 2 (also the same day): How did I get all the way through this post without commenting that the backlash on this issue was like Google walked into a buzzsaw?