Adam Goucher

So you need to start caring about Security and Privacy

adam — Thu, 23 May 2013 01:58:39 +0000

Every company has a moment where they are forced to care, I mean, really care about Security and Privacy. Sure, you should always care in the back of your mind, but at some point it comes front and center and a key pillar of everything you do. The question is whether you are going to be ready for when it comes. Odds are, no. (Because if you were, you would already be caring.) I’ve seen this moment happen a couple times, so here is my list of things you now need to pay a whole lot of attention to.

First, Security and Privacy are related, but different. In hand-wavy terms; Security is related to keeping information in and the bad guys out, and Privacy is about individual people’s information in your trust. And it is a trust, not a right.

With that, here is my brain dump on the topics.

Security

Who is the officially designated person for security problems?
Physical security – padlocks on the cage, etc.
Can people wander the office without being questioned?
Is all input validated on the backend as well as the frontend
Disarm all input
Disarm all output
Don’t blindly use user provided strings in queries
PCI compliance?
Don’t store your password in the clear…
The first question about any change should be ‘what are the security ramifications’?
Do security bugs get put into the main bug database?
How are security things disclosed

Privacy

Who is the officially designated person for security problems?
Do you know what PII (Personally Identifiable Information) you have
Do you know what flows the PII participate in?
People need to be bucketed based upon their informational needs
Don’t clone your production database down into lower environments — without scrubbing the PII
Do not display PII to anyone other than the owner of it. (Unless they really need it.)
PII should be encrypted in the database
Thou shalt not grant access to the production database
Log the stink out of access, including read, of things that contain PII
The second question about any change should be ‘what are the privacy ramifications’?
Do privacy bugs get put into the main bug database?
How are privacy things disclosed
What legislation around PII affect you? And in what jurisdiction? Is it where the head office is? Where the app is hosted? Neither?

Writing secure code is actually not that hard. We know how to do this. At this point it is really just sloppy code. But putting the people and process around how the code that is generated is hard.

This is just a quick off-the-top-of-my-head list, but the key thing to remember is that its not paranoia if they are actually after you. And ‘they’ are.

Am I Coming To Your Conference (or ‘Consultant Math’)

adam — Thu, 11 Apr 2013 03:50:27 +0000

A pattern than I unwittingly did and have noticed in others is that you need to work out what your variation of ‘consultant math’ is for yourself. No matter how many people tell you, you need to blunder through to your own understanding of it. Often by going waaaaay overboard on conferences the first year or so after going independent.

Here is my variation of ‘consultant math’ around whether I will attend your conference or not. Not that anyone will listen to this and not thing ‘hey! I’m different!’ — no, no you are not. But anyways…

The first part of the equation is whether or not I am getting paid to be there. This can either be in term of an honorarium, or flight or hotel (or all three!). Consultant math is pretty easy when this happens … in most situations. But there is the whole problem of Opportunity Cost. Let’s say you are giving me all three of these variables to attend your conference [presumably to speak] but I have to spend a complete day flying there, and a complete day back. That is really 3 days I could be billing people to work on their stuff. If the honorarium only covers 85% of a single day’s worth of billing, well, things start to break down a bit.

Fear not though, the math still might work in your favour if there is a good chance I am going to get new business from attending an interacting with the other attendees. This actually has two parts to it. First, you need to know who your target demographic is. After 3+ years at this, I have a pretty good feel on who I am targeting and the sorts of things they need to hear in order to get them to sign on. (New consultants are unfortunately at a disadvantage and I think this is a large part of the reason why they go to so many conferences in the shotgun approach.) You, as a conference organizer need to convince me that there is a match between my target demo and your audience. (Tip for consultants: check their sponsorship packages, they often include this information in there.) If there is a match, I might eat the Opportunity Cost and turn it into a Marketing Cost.

There is another variation of Consultant Math I have been tinkering with a bit recently, and that is the Sponsorship Option. This is advanced level math and should only be employed once you know your audience since it often requires more risk [in terms of outlay of money]. Upon successful completion of this math you end up sponsoring a target conference [or event association with it]. For instance, the Consultant Math with Sponsorship Option looks quite favourable for Pycon or php[tek], but is pretty abysmal for CAST, Star*, STPCon or even, ironically, Selenium Conf.

I explained Consultant Math to a could non-consultants today [including ish a conference president] which is usually a sign I need to blog about [my variation] it.

My notes from TorontoGROWtalks 2013

adam — Mon, 25 Feb 2013 03:09:32 +0000

So I’m in a pitch contest right now, and easily the biggest outcome is to switch me mentally from ‘talking about my app’ to ‘building my app’. And part of that shift was to got to TorontoGROWtalks last week. Waaaay too much stuff to absorb. (And thank-you startupnorth for the discount code!). Notes are as follows…

Brant Cooper

took 3 minutes to mention FAKEGRIMLOCK
Innovation: Disruptive, Sustaining or Rippling?
hockey stick growth curve is an inverse mullet
disruptive organizations optimize for learning
sustaining organizations optimize for execution
pre product fit, you are pushing the product on the market. after product fit, market pulls the product from you — I’m pushing now, need to stop that, and soon.
first vp of sales needs to be able to sell without a demo — I don’t look forward to hiring sales.
figure out your marketing, by going out and sell
product creates the buzz, marketing is about amplifying your buzz
you are not ready to market until there are things in the top of your sales funnel
how do your measure a ‘passionate user’? figure out how to measure satisfaction and then passion
if you have sharing, measure sharing since its an indicator of passion

Mike Meltzner

product ownership isn’t an authority, it is a responsibility
voice decisions the rest of the group has already decided on
Startup stages: a comparison of 3 models — Marmer stages was mentioned
job of ceo is to say same 3 things to everyone outside of the company (Speak the vision. Keep the tank full. Build a team). job of product owner is to say same 3 things to everyone inside of the company (important thing this month, thing we’re ignoring, where we are going)
be in everyone’s business everyday so they know to include you in decisions
default to no — but help me to help you convince me to say yes
no — for now…
who is handling the project — starters, or finishers?
unsuccessful product ownership involves handoffs

Kate Rutter

was an updated version of
ux != ui
ui = delivery mechanism
ux
- a mindset
- experiential payoff
- inspires the right kinds of ideas
- guides decisions
if people are not going to pay for it, you don’t have a business, you have a hobby
make decisions intentionally
startup workshops
if you cant have one user being passionate about it, how do you plan on having 1000s?
‘you dont have a product box, you have a product sieve’
the market will give you permission to succeed
customers [give you money] vs end experiencers
cohort watching is useful
there is nothing better than being schooled by your product
each stage of the stack has its own ways to test it
you should be okay looking like an idiot, because this is how you learn

Laura Fitton

is there any conceivable world where someone would pay for this blob of marketing
be useful
- relevant to your user
- relevant to your product
start responding to how they make decisions
its not spammy feeling if it is properly segmented
HubSpot Academy
will customer’s thank you for your content?
hits don’t count; conversions count
‘them’ vs your ‘them’
‘smarketing’ — combination of sales and marketing

Scott Kveton

creation stories should all be like urban airship’s
‘more of a feature than a product’
its not lying, its selling forward
think hard on pricing and packaging
you really /do/ need to listen to customers
customers will direct your business far better than you can
outgrew ec2 within 3 months
torn down and re-built tech 3 times

Dan Martell

to have a hockey stick you need to have the flat part
as soon as you truly understand your core product, get rid of everything else
you need to instrument /everything/
know your baselinesactivity streams
- curious
- casual
- core <--- pay attention to these ones
use in app surveys
know your funnels
unless you have core, don’t do this…
- opn (other people’s networks)
- shareable moments

Michael Litt

solve the problem the customer has, not the problem they think they have
do not trust the data, talk to the customer
metrics only work /after/ you have product-customer fit
everything at the start should be outbound
ever developer should talk to customers

Mark organ

most important ingredient for founder or investor: conviction
‘how does cookie monster mean capital’?!?!?!
you need to have deeper knowledge about things more than anyone else
to bootstrap
- conviction
- microniche
- serviceize (charge for anything)
- high pricing (multiply by 3x – 5x your competitor)
- systematic scale
to raise money
- conviction
- beautiful story
- abp (always be pitching)
- get feedback and connections
- firm, but coachable
the best way to get a canadian investor is to get an american investor first
you do have spend time on planes to raise money

My hair-brained scheme to kicking testing conferences out of a rut

adam — Thu, 06 Dec 2012 17:02:46 +0000

The Call For Participation for CAST 2013 is out and includes

We would like to hear your experiences, stories, thoughts, observations, demonstrations around the lessons that you have learned in Software Testing, as well as how these could influence the way that we approach testing in the future.

And I expect a lot of interesting experience reports from interesting people (it is CAST after all). But how much really, really, new stuff will be added to the craft. I’ll be my normal cynical self and suggest that it is likely not as much as one would hope.

What I would love to see is a CFP that looks something like this.
We would like to hear your experiences, stories, thoughts, observations, demonstrations around the lessons that you have learned outside of Software Testing, as well as how these could influence the way that we approach testing in the future.

My talks a couple years ago all fell into this form. Kids in Armor and Testing Inspiration When You Least Expect It are both examples of what I want to hear people talk about. I want to hear Alan Page discuss how Orchestra composition helps him test better, how Ben Kelly tests better due to years of thwacking people in the head with bamboo, etc.

I also want non-testers to be brought in as the keynotes. Sure, I enjoy listening to James, Michael, Cem, Matt and company, but if you are keynote-ing, you are up against an astronaut in my ranking scheme (Keynote vs. Track). Now I am a baseball fan so opinions are skewed but I think the Garfoose or Shawn Green could also be great. Though they likely cost more since its an actual speaking gig rather than just the prime audience grabbing moment. Or Mary Robinette Kowal about puppetry. Or a Buddhist monk about meditation and breathing. Or. Or. Or.

That there is a speaking circuit where one can recycle topics is a smell I think …

… and another smell is that I essentially wrote this same post back in April including some of the same speaker suggestions; The ‘Un-Testing Conf’. That’s just silly. I should do some work…

Be Yourself!

adam — Thu, 06 Dec 2012 16:09:21 +0000

Speak Up! is the latest of Jesse Noller’s community-oriented projects. [I'm pretty sure he has cloned himself to do all this stuff.] Having sat in many ‘how to give a talk’ talks (meta!) I’m amazed how often people focus on construction of slides, the narrative and practice (feh!) and forget the most important thing.

BE YOURSELF!

I’ve been saying this for a while, but this paragraph from Cheek was my hero sums thing up well.

Later that season, Cheek would give me the most important piece of advice anyone has in this business – Be Yourself. It’s easy to lose that once you get close to the pinnacle of the profession, to try to change who you are or how you do what you do in order to impress people or to make your way up the ladder that much easier, but Tom told me squarely that there was a reason I was in that booth with them and it was because of what I’d done on the way there. That my success was based on how I went about my work both on the air and off and that any future success would be based on the same thing.

Pulling it back to speaking, people want you and your content not something and someone else when they select you. Don’t change. Even if that means blatantly ignoring cultural norms. For example:

When I did Set Course for Awesome I was warned that having cussing in the deck could be offensive to the audience. Noted. Did I change it? Not a chance.
If I am speaking at your event I’m going to wear what I am comfortable wearing that day which is likely to include some sort of snarky tshirt. Looking at some of the photos from Justin Hunter who is over in India at a conference today the audience is in jackets and ties. I don’t think I own a jacket and tying a tie is an exercise in hilarity.

Also, if you put a mic in front of me, you are going to get snark. And opinion. Because that is me. That’s what you get. Anything else would be dishonest. And being dishonest to your audience is not a good way to engage them.

Yes, I think that slides that have fewer words are better in general and that you should tell a story when behind the mic. But if that isn’t you then don’t do it. Be yourself.

COP FLUNG A GUN

adam — Mon, 12 Nov 2012 21:25:56 +0000

One of the traditions in my wacky subset of the testing world is not only building mnemonics, but adding / remixing them. And so, I introduce COP FLUNG A GUN. Which is COP FLUNG GUN plus Automation.

Automation

The primary way to test mobile apps remains to be, literally, by hand. But we’re starting to figure out how to do it though automation. (No thanks to the OS vendors…) As automation frameworks get more mature we should check whether automation hooks are provided by the developers. Unique ids or other identifiers for all interaction bits, synchronization points exposed, etc.

Also included in this is integration into your CI system, since that is the bit that controls the execution and reporting of the automation.

The Dark Tower

adam — Sun, 28 Oct 2012 16:20:21 +0000

There are certain things that I can say have been with me most of my life. The Dark Tower series is one of them. (The Wheel of Time series, Star Wars, comicbooks have as well. Likely baseball too now that I think about it.)

So last night I’m paying ish attention to twitter while watching the world series and see this flit across my stream.

Wow! Finally finished Stephen King’s Dark Tower series. Those are long books. Enjoyed it, and liked the ending.

— Corey Haines (@coreyhaines) October 28, 2012

Which got me thinking about the whole series. Yes, the whole series. This is the point where my world-builder’s-disease kicks in. See, the Dark Tower is not just the ‘core’ seven books. It is in fact pretty much all his books. If it had a wizard, it is a Dark Tower book. If it had a lost puppy sign, it is a Dark Tower book. If it happened in Derry, it is a Dark Tower book. And yes, The Stand is absolutely a Dark Tower book.

But does the Dark Tower deserve all the praise it gets? I dunno.

For pure scope and intent, absolutely. But that almost feels like it was added after the fact. Or at lease communicated afterwards. Anyone reading King long enough understood his ‘universe’ held Derry near its center, but then suddenly it was part of something else. And his books would have asterisks next to books that were part of the Dark Tower. If it was always his intention to have things part of a larger epic, how come they were not there in the books that came out in the 80s and early 90s?

As for the individual books, on the whole I didn’t really enjoy them. The Gunslinger is a decent read, the next two feel like they are just moving the plot along and the flurry of the last three after his accident feel like a person’s sudden response to facing their mortality and fearing their epic won’t get completed. His inclusion of the accident in the final book seemed heavy handed and too breaking-the-fourth-wall ish. I literally stared at the page in disbelief when that happened. It is an interesting thought experiment to see how it all would have turned out had it not been for that fateful day…

‘Wizard and Glass’ however is an outstanding book and likely my favourite King book (with The Stand (unabridged) being a close second). Its enough of a standalone story that even if you have no want to read the whole series, it is worth a read. ‘The Wind Through the Keyhole’ is much the same tone and feel as ‘Wizard and Glass’ though outside of the main series. Chronologically it takes place before ‘Wizard and Glass’ but is another standalone book.

The more I think about it, the more I feel like the Dark Tower series proper has failed yet the world it has setup has succeeded greatly. ‘Wizard and Glass’ is character backstory set in the world. ‘Keyhole’ is character backstory about the world. ‘Hearts in Atlantis’ is secondary character backstory, etc. Similar to how the Terminator and Star Wars franchises were flushed out through Dark Horse Comics in the 90s, the Dark Tower is being flushed out by Marvel Comics.

What I think I would like to see however is more ancillary stories come out set in the Dark Tower world. Maybe not even by King, kinda like the Star Wars universe. (See The Thrawn Trilogy for instance to see how other can expand a universe while still staying true to it.) But now that Roland has been eight years since Roland reached the top of the tower and ‘Keyhole’ only came out this year gives me great hope that the world will continue to produce things to consume my money.

Other random bits in my head around the Dark Tower…

‘The Stand’ is a pretty ambitious book to choose for your first highschool book study
I read ‘The Drawing of the Three’ (or started at any rate) during a school trip to Stratford to see ‘As You Like It’ and was listening to The Cure’s ‘Mixed Up’ so certain songs are forever associated with the book (such as ‘The Forest’)
After reading the last page of ‘The Dark Tower’, I immediately went to this page

Continuity

adam — Thu, 20 Sep 2012 14:51:32 +0000

I’m starting to get back into comicbooks thanks largely to the reboot of the Valiant Universe. I have almost all the original Valiant books in boxes in the basement and I think the reason I was drawn to them was its tight continuity within the universe. Events in one book directly impacted, and did not contradict events in another.

The ‘big’ universes of Marvel and DC have long had issues with continuity due to their sheer size and age. I suspect managing this problem is part of the reason why DC rebooted their universe last year (dumb, dumb move if you ask me…) and Marvel is apparently thinking about it as well. That is not to say that its not possible to have working continuity in a universe that large, and with more than one title per character (Batman has 8 titles right now). My example here is the classic Knightfall crossover (which Batman Returns is loosely based). It was across all the then Batman books and, well, was excellent. And in tone for each of the books. There wasn’t much bleed over into the other books in the universe though… but oh, did Batman bleed.

(What I need right now is a word that means something like hindsight, but for recognizing the choices that might have led the the current now.)

While reading Cem’s The Oracle Problem and the Teaching of Software Testing it dawned on me that in my brain, ‘Continuity’ and ‘Consistency’ are synonyms.

Wiktionary provides a narrative device in episodic fiction where previous and/or future events in a story series are accounted for in present stories as one definition of continuity and Freedom from contradiction for continuity.

So how do I think Valiant did originally, and should now, keep their continuity in tact? Glad you asked! Or didn’t and I’ll say it anyways.

Keep the number of titles manageable. And my manageable I mean in the area of 16. Now I have no idea the economics of the publishing world are like, but that could give a couple ‘mini worlds’ that could tightly collaborate sacrificing the whole. (Harbinger, Shadowman, X-O, A&A).
Don’t let the groups of books dictate direction. Where the universe is heading should be a Publisher level decision. The v1 universe had that direction available for anyone to see in big-scope-terms in Rai 0 (best cover of all time)

That’s it. Only two things.

Tying this back to testing/agile, I am starting to notice [now that I am looking for it...] this sort of failings in teams. Teams trying to tackle waaay too much work in a given iteration and when features hit production they don’t work as well as they should have when get into their hands. Happens. All. The. Time. I’ve also seen when agile teams or test groups wag the product dog based on their own whims and biases. Management is supposed to do that. Not you.

(Unless the name card on your desk says ‘publisher’.)

Detail is Dead Wood

adam — Sun, 09 Sep 2012 01:06:01 +0000

Beyond the Matrix is an excellent article for a number of reasons. Not only does it give a peek to how a major movie came together, but it also gives insight into the difference between writing a book and making a movie.

Here is a paragraph from page 7.

The set was rudimentary: the control room of the satellite-communication center would be completed with computer-generated imagery, imagined by the Wachowskis down to the minutest detail. The scene in the control room, for example, features an “orison,” a kind of super-smart egg-shaped phone capable of producing 3-D projections, which Mitchell had dreamed up for the futuristic chapters. The Wachowskis, however, had to avoid the cumbersome reality of having characters running around with egg-shaped objects in their pockets; it had never crossed Mitchell’s mind that that could be a problem. “Detail in the novel is dead wood. Excessive detail is your enemy,” Mitchell told me, squeezing the imaginary enemy between his thumb and index finger. “In film, if you want to show something, it has to be designed.” The Wachowskis’ solution: the orison is as flat as a wallet and acquires a third dimension only when spun. Mitchell, who had been kept in the loop throughout the process (and has a cameo in the film), was boyishly excited by the filmmakers’ “groping toward exactitude.” “I was like Augustus Gloop in the Wonka factory,” he told me. “I’ve witnessed a long sequence of decisions, which I never had to make while writing a book. Intellectually, I know it’s a replacement, but I don’t feel a loss at all.”

It is this sort of detail that appeal’s to my tester’s mind. And one which I have been noticing since hearing an interview with L. E. Modesitt, Jr. on Writing Excuses (I think it is this episode, but didn’t double check) where he discusses things like the support logistics of great fantasy battles that authors seem to forget [ignore].

And now, you will notice them too.

(You’re welcome)

Notes from Blue Jays Winter Tour (Toronto) 2012

adam — Tue, 28 Aug 2012 23:54:46 +0000

My son is presently baseball-mad and so went to training day with some Blue Jays last January as part of their Winter Tour. I have no idea how to coach baseball to kids, so I took some notes. They’ve been floating around the basement since then so I’m de-paper-ing them by putting them here.

Stretches
- butt kicks
- side strides
- long steps
- frankensteins (high kicks)
- flamingos (balance on one foot, lean forward)
- circles with arms
- fall, then sprint
- cross legs, then up without using arms then sprint
- watch the pitcher, then sprint
Batting
- Use a tee
Bunting
- line up with the front of the box so if the ball drops straight down it doesn’t hit the plate; if it does its a strike
- pinch the bat
- rotate feet (into ‘athletic position’)
Base Running / Paths
- touch at the front of the bag
- run straight!
- lean when crossing the plate to make it seem like you are further ahead than you actually are
- game: relay where half run from home and half run from second
- the high-5 at the bag is important
- when going multiple bags, start running in the shape of a banana halfway to first
- take two steps from the bag, square them, then two side steps to be in optimal position
Infield Fielding Drills
- direct grounder
- glove side
- throwing side
- random side
Outfield Fielding Drills
- routes to ball
- glove positions (above or below hips)

Logic Problems & Programming in Interviews

adam — Thu, 05 Jul 2012 02:49:00 +0000

Twitter is excellent for planting troll-bait one-offs, but the blog is better for planting long-tail ones. So this afternoon I tweeted something along the lines of this.

Unless your are interviewing someone for a job that requires [stupid] logic problems, then they don’t have any place in an interview.

And I stand by that. Though a lot of twitter seems to disagree, to one degree or another.

Context. I was at a client today and the two people I am working with came back from an interview…

Them: Do you like logic problems?
Me: Not really.
Them: Want to try the one we just gave in the interview?
Me: Sure…
Them: How long should we give him? 8 seconds? 5?
Me:
Them: Design an algorithm that solves Me: (immediately) Dunno.
Them: Give up?
Me: Nope. I didn’t come up with something immediately and don’t care enough to focus any thought on it
Them: …

This of course led to a much larger conversation about the value using these sorts of things in an interview has. My mentioned above, I don’t think they have any. (Or at least very tiny minuscule amounts to avoid the use of an absolute…).

More context. The interview is for a position that will primarily be webdriver automation using python with some manual testing thrown in there as well (and will be taking over the stuff I put in place.)

So. How does the ability to answer a logic problem help determine the candidate’s ability to do the job? Very. Little. Here is a secret; web automation is not a ‘logic’ problem. It’s not even that hard. It is way easier to be an automator than a tester.

What should you spend the time in an interview for a web automator instead of wasting everyone’s time with the logic problems? Ummmm, how about you have them automate your app. Right there. For real. Projected on the screen.

What oracles do they establish?
Which patterns do they use?
How ‘robust’ are their locators?
How is their synchronization?
etc.

These are all critical things to understand about an automator. Especially the one that will take the lead on the project. Notice a distinct lack of explanation why manholes are round or how to determine the odd billiard ball out in a minimum number of steps.

(Oh, and looking up syntax is absolutely ok. It shows they know where to look. We have this internet thing — we don’t need to memorize everything anymore. Now, reliance on an IDE to program for you is not ok.)

I see logic problems in an interview almost as a crutch that the interviewer is using to hide that they don’t really know what they are looking for. And as an interviewee, if a logic puzzle rears its head in an interview the bozo bit gets partially flipped on the whole situation. Do I want to work at an organization that values people who can solve logic problems? Or that are awesome automators and/or testers. (This isn’t to say there isn’t a possible intersection between the two groups, but it is not guaranteed.)

What are a few arguments I have received today in favour of logic puzzles; none of which sway me…

It shows a high IQ and high IQ is needed for a testing role – People play lawyers on the internet, so I’m going to play neuroscientist. So first, all being good on logic problems means is … you are good at logic problems. A lot of people spend their free time doing these things so they train their brain to spot patterns and build their own solving heuristics for them. Same for crossword-ers or word search-ers. Heck. Look at the whole ‘tester game’ thing. (Of course, the tester game phenomenon has a large degree of arrogance associated with. And no, I won’t play.) Back on topic. No, you also do not need a high IQ to be an excellent tester. Again, I could be conflating IQ with other things (only playing neuroscientist remember…) but there are more firms hiring Autistic people as testers (article).
It lets us see how they work through a problem(1) – Sure, but is it a class of problem they will experience at the job you are hiring for? Those are the problems you are hiring for.
It lets us see how they work through a problem(2) – As usual, Michael Bolton provided a Feynman reference (even if it is fictional); Round Manhole Covers, or: If Richard Feynman applied for a job at Microsoft. Its a fun read and on target in capturing what I have seen of Feynman. But he gets a job offer for marketing. Not in test. Not in web automation. His answer would tell me nothing about his ability to do those jobs, though would tell me not to engage him in an argument about manhole covers.

A slight variation on this is of course, FizzBuzz. FizzBuzz is loved online and is somehow seen as the bar that should be met to be called a programmer. I have not once had to use anything FizzBuzz like in automating apps since I started in early 2000. (And that is of course the only experience I can have.) If you are hiring someone whose responsibility includes programming, absolutely you should have them write code. But have it be relevant to the job! As mentioned above, for a web automation job, have them write web automation code! And if you must use FizzBuzz, then you should write a web app that does it and have them automate it.

Way at the beginning I mentioned that the job that originated this discussion is mainly automation and partly testing. The automation side I think I’ve beat to death. Interviewing for testing is, perhaps unsurprisingly, very simple. Have them test! Logic problems will not help you determine if they have a set of heuristics and philosophy for mobile, or payment gateways, or video, or, or, or. Unless your product is vapourware (or super crazy top secret — at which point, have them test something else), have them test your real product! Again, live! And not just silently do it pointing out [possible] bugs but out loud explaining their thoughts. (I wouldn’t have them write bug reports as an interview is a fake scenario and you’re not going to get a ‘real’ bug report.) Flashing back to Feynman, this is the useful part of the article. The thinking and rationality behind the thoughts. As an interviewer, that’s what I care about. Does it clash with mine, does it create overlaps, does it fill capability holes, etc.

But you can’t get that from the creative process solving a logic problem! As a community we claim to care about context, but interviewing outside of context seems like a waste of everyone’s time.

Two of the most dangerous terms in Agile; Done and Whole Team

adam — Mon, 07 May 2012 11:46:38 +0000

I have come to realize in the last two weeks that there are two words in the Agile lexicon that really bug me. Not because they are bad per se, but that they are so commonly misused and abused. Not to mention being insanely context sensitive.

I’m doing what might be considered an agility assessment for a company and am entering the report generation phase after a number days of observation and data gathering. There is nothing too ‘OMG! MY PANTS ARE ON FIRE!!!’ but one thing that sticks out is their working definition of ‘Done’.

I don’t think anyone who has played the role of Agile coach or consultant has not written a blog or paper at some point on ‘Done vs Done Done vs Done Done Done’.

This company has four different teams which are pretty autonomous and effective and one of the things each have done is define their own understanding of Done. While this seems like a good idea on the surface it is really quite dangerous. With each team having their own definition of Done there can be no common understanding when something is Done and that information rolled up with other non-contextual information to management.

Having the development teams generate the definition of Done also means it is limited to their scope of influence, care and interaction. The net result is a very naive scope of Done. Stories that are ‘done’ frequently will go out into production without adequate analytics hooks or means of monitoring the health of the component being created.

This nicely dovetails to the other most dangerous term; The Whole Team. I was at an automation workshop this past weekend and the people there spent almost 45 minutes talking around this subject. In retrospect no one asked what everyone meant by ‘the whole team’. This is another area I suspect Agile teams need to work better on. I would bet that most people’s working definition of the whole team includes the developers, the testers and someone in some sort of product ownership role (either directly or in proxy). But what about the analytics team, or the sales folks who use the output from the analytics team? Or the Ops team, including the DBAs that need to push the code out onto the iron? How about the customer support people who deal with the brunt of the code team’s mistakes?

The list is not endless, but it is large. One thing I like from the Lean Startup [Cult|Movement] is the the story isn’t done until it in production getting validated by real users. The user is on the team whether they realize it or not. We’ll just gloss over the lack of humans in the release process…

I think there is actually some sort of circular feedback loop at play between Done and Whole Team and you cannot address just one in isolation. Instead, really figure out who your team’s stakeholders are and what skin they have in the game. Only then do you have a chance, and only a chance, of coming up with an appropriate, inclusive, accurate definition of Done.

More adventures in magic automation numbers

adam — Tue, 01 May 2012 03:09:54 +0000

I’m most of the way through a client on-site engagement to do an ‘assessment’ of where their testing processes are and to suggest tweaks to how they do things. I have some broad recommendations that will morph into blog posts over the next month or so, but right now I am challenged by coming up with something that doesn’t sound too much like a consultant-y dodge around ‘When will our [functional] automation remove the need for our extensive manual regression testing?’.

This is of course a trap.

There is no way to determine this number with any degree of certainty.

But what if you really needed to provide an argument around the inability to predict this? My current, best argument around this is that we, as humans, we are really bad at predicting the future.

Just because you are having a certain degree of automation success now, does not mean that the rate of success will continue at all let alone along the same trajectory.
Is the competency of the developers going to change? For better? For worse?
Is the competency of the testers going to change? For better? For worse?
Will the market fundamentally change?
Will the types of stories that come into the development groups remains such that there is a high degree of [perceived] value from automation?

All of these are completely out of control of, well, everyone. Yes, even the ones about the competency of staff. Sure, you can change the way you hire and the way you train, but you can’t change the fact they are people and people are messy. There are however things that we can at least fool ourselves into thinking we have some degree of control over. Though not accuracy…

This particular client is migrating two older application stacks to a new one, so the source of some of the regression risk (the older stacks) will be removed from the equation (replaces, of course, with a new set of risks but the plan is they will be less).
With [better] exploratory testing will regressions be caught during ‘feature’ testing?
If the developers adopt a TDD approach to code generation rather than ad-hoc post-coding security blanket unit testing, will that decrease regressions?
Will the internal focus on a single browser allow for more focused testing (with breadth covered though a testing partner)?
Will proper management of environments lead to less false positives, less re-testing and/or idle testers?

Because we cannot predict the output of any let alone all of these things, which are all inputs into the equation that pops out a date for that magic point where we could be a half-step away from Continuous Deployment (no humans) then ‘tomorrow’, ‘July 23′, ‘Q3 2013′ are all correct. And all wrong.

But if we think of each of these are part of the overall system that results in regression duration that is deemed unacceptable and/or too long then we can address them each individually there is likely to be improvement in the overall system. To what degree and by which contributing measures is however anyone guess. And only a guess.

A Cool Graph, but…

adam — Mon, 09 Apr 2012 18:01:07 +0000

So this graphic is making the rounds today.

qtp,selenium Job Trends

Qtp jobs – Selenium jobs

And while it is interesting (at time of posting, Selenium has crossed QTP and the trends are going the ‘right’ way), it is important to also remember this:

Don’t confuse slamming your competitors with improving your own product.

— Mike Monteiro (@Mike_FTW) April 1, 2012

The ‘Un-Testing Conf’

adam — Sat, 07 Apr 2012 02:30:18 +0000

The Un-Conf is a common event type these days; no scheduled speakers, just a bunch of smart people and a space to facilitate conversation. The success of these is entirely dependent on who is there. And there is the classic Testing Conf where speakers are selected, assigned slots and talk about things directly related to Testing.

But what I think I want a Un-Testing Conf. This would would be the same format of a classic conference but without topics about Testing.

Well.

Obviously about testing.

Irony Alert

I am sick of going to Testing conferences and hearing the same sort of stuff over and over. Or something that I could quickly pick up from any number of industry related books or blogs. I want to learn new, mind blowing things. And do you know where those things hide out? Hint: its not in the testing world!

Tangent. Have a single track conference; 5 or 6 concurrent tracks is too much.

So what would my Un-Testing Conf look like? Well, it would likely be expensive since you can’t just comp the speakers a ticket and expect them to graciously accept the invitation. Currently the line-up would be something like…

Mike Monteiro – Design Is A Job
Hugh MacLeod – Social Objects
Austin Kleon – Steal Like an Artist
Dirk Hayhurst – Something baseball-y
Mary Robinette Kowal – Puppetry
Someone who has been in the International Space Station
Thomas Mahon – Customer Service (or something)
Sian Beilock – Choking (or how not to)
Mark Cuban – (Don’t know what I’d like to hear him talk about, but he is usually pretty fun and has interesting stuff to say)
Chris Blais – Dakar, and now
Roz Savage – Ocean Rowing

I can, without even trying, explain how each and every one of the above people have important things to say that would make testers better at their job.

That is a full two days of speakers already. Who else would you like to see? The only ‘rule’ is they cannot self-identify as a Tester.

Because that is how they got that way

adam — Tue, 03 Apr 2012 00:21:03 +0000

So last week I was in New Orleans for STPCon and one of the things I did was a 15-ish minute gabfest on Set Course For Awesome (as a ‘Career Tester’). One of my supporting ideas for it is that you need to shut up, listen, and figure out what it is you are supposed to be providing information on. (Protip – its not what you think it is; or the second thing; or even likely the third thing.)

One trick I have started to intentionally using is to understand why something is the way it is. Especially for the things that stand out as particularly bone-headed. Often a series of perfectly rational thoughts happened in a perfectly logical order only to have nasty repercussions.

There are lots of examples of this, but when I was in New Orleans I took a ‘pirate tour’ (I know, shocking) and it provided yet another example of this.

In 1788, a candle lit a bit of curtain and started a pretty nasty fire
It was Good Friday, and the Cathedral bells had burlap on them so as to not ring until Easter Monday
New Orleans burned. Almost to the ground.
But they rebuilt! In stone!
And inside each stone building was a well — not only for drinking purposes but to have ready access to water in case of another fire
Mosquitos breed in water
There is water in every home
Between 1817 and 1904 there were over 41 000 yellow fever deaths
Yellow fever is transmitted by mosquitos

All perfectly reasonable individually, but viewed through the lens of history, 41 000 people died from yellow fever because a candle’s flame caught a curtain.

The trick, especially for those of us who see lots of companies is to trace back things and where possible address the original concern rather than just the current manifestation of the concern.

Set Course For Awesome (as a ‘Career Tester’)

adam — Sat, 24 Mar 2012 03:44:19 +0000

I’ve been asked to do a 15-ish minute talk as part of Anne-Marie Charrett and Fiona Charles‘ upcoming workshop at STPCon on being a Career Tester (as compared to going into Management or Consultant — even though I have been both). As I thought about this, I think this could actually turn into a keynote worthy talk (of course, then I would actually have to practice it?!), but I have come up with what I think is basically my advice to myself of 15 years ago when I landed on this career path. Now of course, to deliver it as a double-stuffed-lightning-talk for them.

Set Course For Awesome

Testing is an awesome and rewarding career. But it is also ‘new’ even in the larger context of IT jobs which are themselves new in the grand scheme of things. This means that unlike someone in the trades, the career path for testers isn’t well defined and everyone’s will be slightly (or significantly) different. The thing that will be common to them all though is that those who are happy(est) will have taken responsibility for their career and planned it out. The plan may not work out, and will almost guaranteed to need course corrections along the way, but the act itself is useful. So set a course for awesome. (And lets not concern ourselves about the alternatives to that shall we? Great.)

Shut Up and Listen

It was during RST with James Bach that I really understood that the role of a tester isn’t to ‘break the app’ or ‘find all the bugs’ but to provide information about the application. It wasn’t until some time later than I realized that it is actually more subtle than that. Our job is to provide information that matters. And how do we do that? Easy. We Shut Up and Listen. To what? That is also easy. To the people we are providing the information to. Now that I’ve completed PSL I can safely start quoting Gerry Weinberg so here are two useful things to remember.

No matter how it looks at first, it’s always a people problem
Things are the way they are because they got that way

If you can start to understand the people in the system you are operating in, and why the [larger] system operates the way it does then you can provide them the information they need. But you can’t do that unless you stop, shut up, and listen.

Hooray

Too often people let others drive their career trajectory or they just listlessly bounce from one place to another. This handy venn diagram should help you organize your thoughts. Make the three lists. Find the Hooray point. That is your destination. Or more correctly, your current destination as it is likely to change as you experience new things. You are not the person you were five years ago, and you are not today who you will be in five years. I actually have a large copy of this that needs framing and will go on my office wall.

Steal Like an Artist

I’m going to say this right now. In March. That Steal Like an Artist is my Tester Book of the Year for 2012. And it has no direct links to testing. Which is also part of the point. Not only are the 10 points in it completely relevant to being a tester but it comes from a different field entirely. Go outside of the testing community for inspiration. One of my first public testing talks was about being a houseleague lacrosse coach. The most useful books for my testing practice recently have been about how the brain handles stress, how to really practice something and about apprenticeship patterns. None of them were written by testers, but each made me a much better tester. Take inspiration from everything and everyone. When you are constantly looking for it, it is amazing how often you can find it.

Be Yourself

The trick to Stealing Like an Artist vs blind theft is that artists make what their steal their own. You will not truly succeed just regurgitating words and ideas that you heard from people at conferences or read on their blogs. The two I see all the time is the SFDPO mnemonic and the notion of testing tours. Think of your own. I know you mean well by parroting it, but internalize it, morph it, evolve it until it is yours. It is impossible for you to be anybody but yourself. So don’t try. Take what you like and what inspires you, make it part of you, and ignore the rest.

Have a Therapist

Talk to everyone you can. But have a couple close people you can bounce ideas and questions off of. Both for ‘shop’ and life in general. You might be smart, but someone is always smarter. You can’t grow if you just talk to yourself or your teddy bear. And it works the same way too. Some of the best lunch chats I’ve had have been initiated by me but I went home with less notes than the person I was trying to talk something through with. The term mentor I think is too constrained. Remember, its always a people problem and people problems require therapists.

Be Totally Fucking Amazing

Quality this, Quality that. Blah. Blah. Blah. As a tester, you shouldn’t fall into the trap of thinking ‘Quality is Job One’. Its not. Being totally fucking amazing is job one. Your customers do not care that the code is quality, or that it follows some style guide line or has only 0.45444 bugs per 1000 lines of code. They care that it is amazing. That’s all. The same applies for your career. Whoop-dee-freakin-do that you have been at the same company for 10 years and got a commemorative clock (do they even do that any more??!?). Are you being amazing anymore? If not, why are you still there? Print out the Hooray graph, plot your course to awesome and go be totally fucking amazing.

Here is the slideshare link to share amongst yourselves.

Set Course for Awesome (as a Career Tester)

View more presentations from Adam Goucher.

The impact having an online presence makes to an author

adam — Thu, 22 Mar 2012 14:47:50 +0000

I just finished readying Mary Robinette Kowal‘s book Shades of Milk and Honey which is often described as ‘Jane Austen with Magic’ which so far as I can tell is accurate. (There is magic and I trust that is Austen-esque.) This is clearly a large step outside of my normal Epic Fantasy genre reading and I would have ignored the cover illustration completely in the bookstore (had it a copy, which is didn’t). And even if I had, the Jane Austen part would have scared me away based solely on reputation than actual opinion (and that I couldn’t get through ‘Pride and Prejudice and Zombies’). But I bought it soley on who wrote it. And how I know about her wouldn’t be possible even 5 years ago.

But first, a mini-review.

Having suffered through some ‘classic’ Victorian era books in highschool, I somewhat feared that this would be as equally difficult to get around just the phrasing, but it was a quick, and enjoyable read — I think I went through it in 3 sittings (though one was exceptionally long). The characters fit my mental model of how they would behave and act in ‘proper society’ and the magic system seemed perfectly natural (and thankfully the origins of which were not explained). The book itself (at least the hardcover) has the pages not chopped for even-ness which lends, I think, the appropriate amount of ‘era’ to the book.

Oh. And I only found one type-o; ‘colour’ in one sentence and ‘color’ in the next which is really an spell-checking dictionary problem.

Now for the real reason for the post; an author who is active in various social media platforms can dramatically affect how a reader experiences there work. And not just in a sales perspective (though certainly that does help). Here is the list of ways in this particular case.

Mary has been one of the hosts of Writing Excuses for a while now so I get a dose of what she sounds like in 15 minute chunks every Monday. What this did was ‘change’ the voice in my head while I was reading. This is not something I had noticed before but it is usually male voice, but this time it was not only female, but Mary
Also from Writing Excuses I knew that the Austen has used [almost] every work in one of her original works as Mary took the complete works of Austen, uniqued it and then used that as her spell-checking dictionary. Which. Is. Awesome. And while appreciating this fact runs slightly counter to one of the themes of the book which is to appreciate the beauty in art without dissecting in, it is still awesome.
At one point there is a puppet show in the story, and while it is explained, nothing beats actually seeing it. Which you would, if you followed her twitter and watched this interview in which she performs the show
Which she is well qualified to do since she is a Puppeteer first. That itself is hilarious when she is tweeting about making puppet.

Time to glue his toenails. I’m debating about inserting the glue under his skin or on the surface.

— Mary Robinette Kowal (@MaryRobinette) March 12, 2012
Mary blogs frequently sometimes posting period costume which helps complete the mental picture of things while reading.

We (I) completely blew the social media aspect of promoting Beautiful Testing. Mary seems to have it nailed. And I have Glamour in Glass already on order.

Its all about the Bordens

adam — Fri, 09 Mar 2012 03:18:48 +0000

In today’s tech world (and I suspect non-tech as well) there are two basic funding models for companies; bootstrap and VC. I’m bootstrapped since I opened shop with only about 40 hours notice.

I enjoy being my own boss, but as people I know de-independent themselves or I see their company gets acquired I think about ‘what if?’ situations.

What If … ‘I sold the company’

I sometimes think about what sort of company I would ‘sell’ my company to. I suppose for enough cash I would sell to almost anyone but what sort of company would be the right ‘fit’?

The obvious one might be Sauce Labs since they are Selenium in the cloud, but they don’t have a Consulting / Professional Services wing. Now, acquiring me could become one, but I don’t think that is their strategy especially with the launch of their Ambassador (Partner) program. I am a Partner btw, so contact me if you are looking at them as I have a magic partner code. And while I think their OnDemand product is outstanding I like being able to recommend different solutions when the situation warrants. I’d likely also have to stop writing BrowserMob scripts as well which would be factored into any sale price, but is something to keep in mind when selling out to a Service company.

Selling to Neustar (which bought BrowserMob last year) has all the same problems, only flipped around.

So what about a pure Consulting company? Thoughtworks is where Selenium was created and does make sense to some degree, but from all accounts, you are on the road as a Thoughtworker way too much. I also think I know more ex-Thoughtworkers than I do current ones. Leandog would be a much better fit and I have done work for them in the past. (Last I looked I was even listed on their website.) I’d be a bit scared of the travel with them too though, and I’m not sure how much of their customer base is Python or PHP which are my best languages (though I am not unfamiliar with Ruby as well).

In either acquisition scenario there would have to be a line item to get me proper work clearance for the US (which is a pain).

What If … ‘I got an investor’

I think bootstrapped companies tend to stick closer to their original entrepreneurial goals, but of course they [often] suffer from lack of capital to really expand and start to scale. Which is where I am now. I’m trying to build out a couple products and do consulting at the same times. Sometimes the two line up, sometimes they don’t.

So were I to take investment from a VC (or Angel) what would I do? In a word, hire! But in a very targeted manner. I think for the model that I have going, what I need to be successful are testers who know how to program. The distinction between developers-who-test and testers-who-develop is oft argued but I think its important. Automation experience certainly helps, but this is along the lines of ‘hire for character; you can teach skills’. I can teach them the automation how-to but you can’t teach the natural ‘think like a tester’ that is essential in awesome automation. Oh, and of course they need experience in teaching. There is a big gap between being able to do something and communicate how to someone who is still running headlong into a learning curve.

For revenue purposes, I would need a Java programmer first. As much as I dislike the language, it is used a lot and I end up turning away work at least once a week for it. The next two are interchangeable in hiring order; a Python person and a PHP one. They would be working, when not on client stuff, on Py.Saunter or SaunterPHP.

That puts the company at four people with others brought in as necessary (on contract initially), and I’m fine with remote work so likely wouldn’t need a ‘proper’ office and could continue to sublet where I am. With runway for a year I’m thinking the investment would have to be somewhere around $450 000. Which is not a small amount (at least to me) but I’m pretty sure that with a bit of a marketing push that support contracts for Saunter could become a decent revenue source. Not to mention more and more companies are looking for Selenium help as they realize that even though the software cost is zero, the cost of not doing things ‘right’ is a lot more.

So anyways. That’s what’s being going around my head the last couple days as a result of some press releases, etc. that I stumbled across. By no means am I putting out a ‘for sale’ sign, but these sorts of situations are ones that business owners need to think through. I just happen to do a lot of that thinking out loud and in public.

(Oh, and for the non-Canadians, Robert Borden, the 8th Prime Minster of Canada is on our 100 dollar bills)

How to ‘properly’ build multiple Facebook games

adam — Thu, 29 Dec 2011 05:39:20 +0000

Want to absolutely kill my productivity? Introduce me to a timeboxed game on pattern recognition (Bejeweled, Tetris) or bubble stacking (Bubble Island). Facebook is very dangerous for these. One of my favourite is Diamond Dash by Wooga — which happens to have a number of other games too.

A week or two ago I was playing Diamond Dash while babysitting a script run a little monster appeared on the screen saying something like “Get to level 5 on Monster World and earn a special bonus gem”. Ooooo! Bonus gem! So like a good little pawn I got to level 5 and unlocked my gem.

But now I started thinking. If these two games are integrated, what about the other ones? And so I leveled a couple more times in Monster World and soon a Raccoon showed up in the corner. Play 10 levels of Bubble Island and get a unlockable plant (Monster World is one of this silly ‘farming’ games). I’m more than happy to play 10 games of Bubble Island and so got my plant.

Is it integrated with the other games? Not yet, or at least not so far that I have discovered.

This got me thinking though that this is exactly the right strategy for creating multiple properties on a social platform like Facebook. I would go so far as to suggest that part of the product roadmap for an additional property is the integrations with the existing ones. Taking the Wooga games as the example I would likely do the following (just off the top of my head).

Bubble Island -> Monster Island (existing) – Play 10 games and get a plant (Bubble Bush); though the plant should be integrated into the gameplay. So far none of the ‘quests’ have required it.
Monster World -> Bubble Island – Plant a Bubble Bush and get 3 fireball special shots; available once a day
Monster World -> Diamond Dash (existing) – Get to level 5 and unlock the Plasma Gem
Monster World -> Diamond Dash (existing) – Plant a Heart Flower and get an extra heart; available once a day
Diamond Dash -> Monster World – Once you get the spellbook, collect certain amount of diamonds to add new Arabian Knights themed decorations
Diamond Dash -> Bubble Island – Once you get the spellbook, collect certain amount of diamonds to add new level in the Adventure mode appears

And that doesn’t even include the Happy Hospital and Magic Land games (which I think are the Monster World engine with Pets and Knights respectively) which I would likely do at the higher levels of Monster World since the quests there take a while to compete so while they are churning away you could be doing the lower level, quicker ones of a different game.

The point being that social networking works best when the network effect kicks into overdrive. That process can be given a boost by incentivizing your existing audience to try other things that they have already shown that they keep coming back to.

Oh. And I would make sure a list like this was published somewhere reputable that was highly SEO’ed. Discovering during gameplay is one thing, but making it know that you can get these things if you play just a bit more could have value too.

Really step up to the plate for the Open Web

adam — Sat, 24 Dec 2011 03:03:22 +0000

The internets were ablaze this week with SOPA stuff, and in near-inferno around GoDaddy supporting (but that is a different post altogether). And while it is nice that the geek community is up in arms, its a bit concerning how little mainstream media attention it is receiving. Even more so from the media up here since Canada is in ARIN and so [apparently] falls under the US government’s jurisdiction. Not to mention the the copycat law that will be introduced using the American one as ‘proof’ of its good-idea-ness

But even if (when) the beast that is SOPA gets enacted there will be ways to get around it. Heck, there already are a couple plugins for Firefox that do just that. Which leads me to an idea.

We Believe in an Open Web. And we’re dedicated to keeping it free, open and accessible to all

This is the main text on the Mozilla website. What if they just included these plugins, or the tricks that they do in the Firefox browser. The day that it gets presidential assent is when it should should go into Aurora.

But this post isn’t to call out [just] Mozilla and put them in political bind (since they are incorporated in the US). Google has been just as strong in their opposition and they control the Chromium/Chrome browser. And apparently Microsoft opposes it too, and of course they control Internet Explorer.

Of course, Google and Microsoft could give a damn about the Open Web…

Badges; carrot, not stick!

adam — Sat, 17 Dec 2011 14:35:33 +0000

There seems to be two growing schools of thought about badges forming (or at least becoming more visible.). Not surprisingly, I think the origins of these have to do with who has what in the game. For instance, the StackOverflow folks have from the beginning talked up using points and badges as a recruiting tool. Oh, and they have a for-money job board…

What is really concerning to me though is an increase in the number mentions of badges being used as a stick. Case in point is an email on the Open Badges mailing list this morning.

Obviously we can all see the benefits of having such a badge system that is supplemental to formal education. In the workplace, specific skill badges can be connected to increased salaries. Right now at my workplace we are dealing with secretaries who are untrained in newer technologies and they are unable to assist faculty and administrators in their departments. Now if we accept and link specific badges with salary raises or promotions, now we have intrinsic value for them to actually pursue more training.

No. What you have is another stick to beat people with. Horrible corporate training has always existed only now you might get a badge rather than a certificate you print at the end. Yes, some will try for the badge the same way some would put the certificate saying they completed the corporate ergonomics training on their cube wall.

Now, with the whole gamification trend, I expect corporate training management systems to grow badges, if they haven’t already, as a way to tracking who has done what. That’s not really using badges though, that’s just replacing checkboxes with graphics. Kinda like how a lot of teams transition to Agile by calling the Project Manager a Scrum Master…

Another risk that comes from using badges as a stick is invasion of corporate interests that have nothing to do with the improvement of the badge holder, but of the badge issuer. Certified Tester badge anyone? You know there will be dozens of those if the idea of badges takes off. And like the certificates they supplement you’ll be able to buy them rather than earn them rather than earn them.

Top Ed-Tech Trends of 2011: The Higher Education Bubble talks about badges supplementing formal education — in what feels kinda stick-like. It is a good article on the whole though. Yes, we (society) need to rethink credentialing and accreditation — tried to cross a border for work recently only having a highschool diploma? Saying you need to have certain badges from certain issuers for a job or raise however is just replacing one broken system with another.

Badges, to me, should be not forced upon someone but the carrot that inspires learning. Learning that would likely happen anyways actually, but just provide an indicator of having done it. Again, I fall back onto the example of Cubs. There are around 45 badges a Cub could earn during their three years, but they don’t have to. Yes, they’ll get some just by showing up and doing the activities we (the leaders) plan, but a lot more need them to show their own initiative. Some will, and some won’t. And that’s ok.

And know what? They’ll all be allowed to move up to Scouts. If badges were being used as a stick there wouldn’t be nearly as much enthusiasm around them or the programs. Actually, it would look at lot like a corporate environment.

Badges as the centre of the Web Maker solar system

adam — Wed, 07 Dec 2011 18:25:32 +0000

Shockingly, companies don’t just cede control of their roadmaps to me so I’m left to just blog about what I would do if I did wrest control of them. To continue a recent topic series, I’m tackling Mozilla.

Mozilla appears to be downplaying Firefox as the only thing it does and is emphasizing its mission of creating a Internet that is open. Firefox is currently its largest tool in order to achieve that mission, and likely will be for some time, but I suspect we will continue to see play more of a supporting role and/or means-of-financing other work.

In the place of Firefox the Web Maker meme will become more broadly spread. If I was creating schemes for 2012 and beyond I would look to badges to drive this. In fact, I would make them central. For every project that Mozilla undertakes, the question of integration into the badges idea should be asked.

What spurred this idea was the weekly Community Call yesterday. About 1/3 of it was a discussion around Drone Journalism which is the use of unmanned drones (mini helicopters, etc.) to capture footage for film or citizen journalism. This is kinda interesting for a make-your-own-drone perspective and from being able to capture/disseminate information when it is being controlled by ‘official’ sources, but I’m not seeing how it fits into the Mozilla mission. Sure, it could be a consumer of Popcorn.js, but that’s really about the limits I can see in terms of Open Web and Web Makers.

Let’s go with the notion that the linkage to Popcorn.js is the correct one. If so then there could be a badge series for Drone Journalism:

Created a Popcorn video using a commercial drone
Created a Popcorn video using a home-made drone
Report on a Community event using a drone

Or something similar. Certainly something like the MoJo (Mozilla Journalism) stuff could have a badge sequence, School Of Webcraft already has one, the HTML5 evangelists inside Mozilla also have something I think. Last week Mitchell Baker (Chief Lizard Wrangler at Mozilla) said in a talk at the Toronto offices that Mozilla is not going to be political organization, but even some of its grassroots Open Web/Privacy stuff could have a badge system; Defender of the Internets! badge anyone?

The ideas of Open Web and Web Makers are powerful, and absolutely is where I would point Mozilla towards (not as if Google or someone else is going to do it; ‘evil’ is relative it seems). But without a framework to hang initiatives around they risk being just a collection of independent projects under a large umbrella. Badges, which is also part of the Web Maker idea, could provide an ideal structure to connect all these things.

Interesting things happen when you visualize stuff

adam — Sun, 04 Dec 2011 15:12:58 +0000

David Christiansen runs TroopTrack and posted a bit of an analysis of your conversion rates; powered largely by a couple graphs. Its an interesting analysis, but what is really good from a startup perspective is the findings in the Correlation or causation? section.

Customers who have a good experience with a support ticket during their trial have a higher likelihood of purchasing a subscription
Customers who see an improvement made during their trial period have a higher likelihood of purchasing a subscription

That is an important finding, and one that in my gut I think is more true for more products than people give credit.

When I was at HP we pretty much knew who was considering buying our product because the Support group had contract with them. Right now I have a similar interaction with people who start to use Saunter — though that is because I need to update the docs.

SaaS products are easy to capture this information and interaction; an opensource product that you sell service and support around is much harder. And a problem I need to figure out at some point in the near future.

Disclaimer: David is a friend and I want TroopTrack to succeed — even if it is so he can add Canadian stuff to it.

Scouts, Badges and Mozilla

adam — Fri, 02 Dec 2011 21:01:33 +0000

And so I continue to think aloud Badges and Web Makers.

If you look at the badges at p2pu which is pilot project for the open badges stuff at Mozilla one thing stands out. They’re just text, and frankly, kinda boring. This might be enough for people who are used to static badges like those on StackOverflow but I think it completely fails to capture the attention of the next generation of Web Makers.

I have no information to back up that claim other than what I observe around ‘real’ badges at Cubs with the kids looking at them, guessing what they are, etc.

Two years ago, George Dinwiddle awarded me an ‘Agile Merit Badge’ at the Agile Conference. While cleaning up the basement from a partial flood I found it again and it clicked a few more things in place in my brain.

First, the they need to be image driven, not text based. Not the least bit because the world does not communicate in only one language. (Hurray for the arrogance of the English speaking world!) Secondly, through the use of steganography the information about issuer, what was done to earn, repudiation authority (thinking something like OCSP — OBSP?) and such could be embedded in it.

But more importantly, an image can be turned into something tangible; like a real badge. Something that could be stitched onto a physical backback or blanket.

Of course this badge thinking also lines up well with one of Mozilla’s goals around the Web Maker stuff I think. How do we get the next generation to learn about how the Internet is built so they can make it their own? Well, there is a global organization for youth that already has an established tradition of badges; Scouting.

One thing I would like to see MoFo do is work with Scouting to create a ‘World Wide Web’ badge. Here is the requirements for the closest thing that Scouts Canada has to this; the Computer Badge. It is very dated and desperately needs a refresh. Or better still, replacement by something else that is relevant to the kids in the program now and in the next decade or so.

I realize the Internet is more than just the web, but let’s face it, that is the mechanism for 90% of people’s interaction with it.

Off the top of my head, the requirements could be something like

Explain what the following HTML elements do: …
Explain the role of HTML, CSS, JS
What are the dangers around Phishing, Pharming, Spam?
Something about privacy
Something about online bullying
Create a simple homepage for yourself or your pack

That would give the basics of both Web Safety and Web Makerism to kids earning the badge. (It would also be an easy badge to complete in an evening at the local library or similar with lots of computers and X-Ray Goggles).

Badges for Scouts and Venturers could require higher levels of sophistication; CSS, Canvas, Video, etc.

There is actually precedent for organizations working with Scouts (well Canada at least) with the new Scuba badge (Scouts, Venturers) that was done in conjunction with PADI. That organization for that badge makes perfect sense. I think Mozilla makes perfect sense for something about the Web. And how awesome would a Mozilla head badge be?!?!

They know how to contact me; in the meanwhile I need to figure out who to contact in Scouts Canada about this. This idea could have legs.

Tags and Badges

adam — Thu, 01 Dec 2011 23:43:51 +0000

Yesterday Scott Barber tweeted a link to The Certification Experience to which I responded (without reading it; and it is a good read)

i’m now thinking something like the lovechild or open badges and entaggle is the route forward

If you are not aware of it yes, Entaggle is site for ‘tagging’ people as a form of peer credentialing created by Elisabeth Hendrickson. In a follow-up to her I flushed out what I think the differences are between ‘tags’ and ‘badges’ which she allowed me to make public.

So here is my thinking of tags vs. badges. Tags themselves seem to be more about the relationships between the tagger and the taggee (‘someone I know in person’, ‘someone I look up to’) whereas a badge is something that is earned and/or has requirements of demonstration of skills. Badges also have a more formal issuing authority and repudiation scheme (the tag ‘someone I look up to’ was removed from taggee by tagger, how is that made visible?).

In the Mozilla context of badges, there could be an HTML badge that you get by live-editing an existing page to submit a form to some ‘secret’ place that registered that you have ‘completed’ the badge (though X-Ray Goggles or Firebug or whatever).

(I also have badges on the brain from Cubs and have been looking at Red Star type pages the for the last week plotting the second half of the year.)

What would an Agile Tester badge look like?

Would need an Issuing Authority (Entaggle?)

Likely have a list of Tags that that included both mandatory and optional ones (Exploratory Tester, Lives in the spirit of the Agile are mandatory, Performance Tester is optional)

When the requirements are ‘met’ they get the badge added to their backpack

Repudiation happens automatically when the requisite tags are removed from the person

I think too the Mozilla badge vision has the notion of levelling so could have Bronze/Silver/Gold which require a greater set of tags to earn (and keep).

Badges from a ‘game’ perspective is a pretty easy way to motivate ‘kids’ but there is nothing inherently motivating (at least for me) in trying to earn a ‘tag’. A badge, maybe. Especially if the badge gets some clout behind it. (Though yes, a tag could and should also get clout behind it as well.)

Badges could also, in the Entaggle context, become an aggregate of tags distributed by a number of people. Hmmm. More feature requests!

Web Makers and the Open Web

adam — Wed, 30 Nov 2011 14:50:30 +0000

I follow Mark Surman on Twitter and it was there that I learned about his Web Maker ideas for MoFo. He has posted a 30 minute slidecast of the plans for 2012 on his blog; go watch it. I’ll wait.

I haven’t had something ‘click’ this strongly in a while and its only just occurred to me why — if the community of Web Makers is wide and broad enough, the openness of the web is more assured than if only the ‘geeks’ of the know how to run the web.

If everyone understands the basics of markup (X-Ray Goggles) and can make interactive sites without the use of expensive, proprietary frameworks (Popcorn.js) then things like citizen journalism become possible and the effectiveness of [stupid, inane] things like national firewalls and clueless politicians (Mozilla vs. SOPA) is greatly reduced.
I’ve been on the side of ‘open’ for years. I’m just as the right age where Linux and Open Source became viable alternatives when I was most capable of learning them and so have very strong Open Source Anarchist tendencies (I won’t write for places that have registration or paywalls for instance and will not be taking any presenting gigs at conferences that do not make the proceedings freely available as well). Information really does want to be free — contrary to ‘well intentioned’ governments or corporate entities.

But I think the badges is where things have the greatest potential though. [As much as I dislike the word 'gamification',] I suspect badges are a powerful meme for encouraging the generation of Makers; those who are 8 – 15 years old now. Not being a Gamer I wasn’t sure on the idea until ‘badge night’ at Cubs the other week. (I’m now the Akela-equivilant for our Pack.) Badges, or more specifically receiving badges, is a huge motivator. Badges are a self-perpetuating cycle. Once one gets a particular badge then others are motivated to also get that badge. And there is often one (or two) kids that self-motivates to find new badges they can get which keeps the cycle intact.

The Badge backpack also has the potential to be a type of ‘team jacket’. At least around here, when a kid makes it onto a Rep sports team the parents are told of the registration and tournament fees. And once they pick their jaw off the floor they are also told about the other ‘optional’ fees — like team jacket and bag. Though these are ‘optional’ they are really ‘mandatory’ as every kid wants to go to school in the jacket that announces them as a Chief (baseball), Warrior (Lacrosse), Iroquois (Soccer) or Wolf (Hockey). If the backpack can be embedded into their ‘place’ on the web easily (through the skills learned with X-Ray Googles) and isn’t forgeable (though all credit to the kids who eventually figure out how to) then it can be a competition to have the most complete backpack.

The Web succeeded because it was open. If it is to remain open we need the next generation to be Web Makers just as the current generation of leadership were Infrastructure Makers.

Continuous Delivery is landing at CodeMash

adam — Thu, 15 Sep 2011 13:35:20 +0000

The speaker submission window is now open for the next CodeMash. For those under a rock, CodeMash is a conference in the middle of January, in Ohio — at a waterpark. Yes. A waterpark. No worries though, we’re in the indoor part.

Anyways, one of the tracks for this year’s conference is Continuous Delivery — and I’m the track judge for it.

That of course means that for the rest of September I’ll be periodically pestering people to submit talks. I’m hoping for a track that covers the whole spectrum of topics that is Continuous Delivery. I had originally floated the idea of something like

So you want to be a Git ninja
Pulling the strings of server with Puppet
Yes, your SQL is code too (dbdeploy)
Everyone’s favourite butler, Jenkins
Functional Automation [with Selenium] for fun and profit
Deployment and Immunization

but I have I think double the slots. So get those talks in now! The session window closes September 30.

The Business Model

adam — Wed, 03 Aug 2011 19:37:40 +0000

I’ve been ‘independent’ since the tail end of 2009 (one week before Christmas in fact) and have solely been doing work-for-hire stuff between now and then. I don’t scale to my ambitions, nor to my paranoias so I’ve been thinking about really figuring out what exactly is my business model, and I think I have figured it out.

Product – In my consulting I have seen that more often than not companies are not stuck with Selenium itself, but the framework around it. To that end I have released Saunter which is an ‘opinionated test framework’ similar to how RoR is opinionated. (There is more than one way to do things, but really only one easy way.)

But since I have some Open Source Anarchist thoughts, I’m giving it away under the Apache 2 license. Which means I’m going to now be in the Freemium game if I want to keep paying my bills. And which leads us to…
Support – Let’s face it. Small teams will just take the code and figure out what they want to make it do. But medium or larger companies enjoy the security blanket of having support. Here I think I will have two options.
1. Hourly – Companies can purchase blocks of my time on [essentially] a retainer basis. These blocks expire, but come with a certain level of service/response
2. Per Incident – Companies can purchase per incident support tickets
Training – A large part of what I have been doing is training, or likely better stated as coaching. I can help you be more successful rolling out Selenium. This is at ‘standard’ training rates.

Dreaming up the model is however the easy part. Actually implementing it. Well, thats a bit more challenging.

Two more podcasts on Page Speed

adam — Thu, 28 Jul 2011 13:38:08 +0000

These are a little better than last night’s. Still not sure that I would spend the coin to go to Velocity though. But I suspect the really good talks are also the ones that would be hard to record.

Performance Testing: Putting Cloud Customers Back in the Driver’s Seat

Imad Mouline
Podcast
Video and Slides from Velocity
Do you know where your third party content is hosted?
Domain sharding using EC2 buckets – since each bucket is its own host. And you pay the same thing.
Is sharding now a performance hindrance in today’s browser?
Raw Load Time vs. Perceived Load Time (above the fold)
Looks like the tools mentioned in the podcast have disappeared with the acquisition of Gomez by Compuware

Time is Money – The Measurable Value of Performance by Design

Tim Morrow
Podcast
Video and Slides from Velocity
People may still not use your site even once you fix your site because they think it will still be slow
There is a tension between features and performance
Don’t have a dedicated performance team; use people from around the company so the whole organization cares about it
Perception is everything
Your content is the important content
Progressive rendering
Time to first byte
Javascript no longer loads with the page
“Based on the time we spend making these changes, we made it back within two months. After that it was just gravy” — And that hits the bottom line. This is how you talk to Business about caring about performance.
Do you know where your hotspots are during peak time?

Three Podcasts on Page Speed

adam — Thu, 28 Jul 2011 03:03:50 +0000

One of the things I inevitably have to deal with at each site I visit is the idea that Selenium scripts are slow. Sure, they are slower than unit tests, but a lot of the perceived slowness is due to all the other stuff the page is trying to load. First, you don’t need that in the automation environment — so build into the app a way to turn it off. Second, if you are loading them, the speed must be considered. And at no point should the user’s experience be impacted.

Here are some podcasts I had lying around on page load speed that I hoped would be better. But I have notes on them now so a-publishing they will go.

Mitigating Advertising Impact on Page Performance

Ismail Elshareef
IT Conversations version of the talk. How I listened to it, but you likely want to watch the video to see the code.
Blog post on the Edmunds site on the stuff discussed.
‘We want to have control over our assets…’
Wrote their own JS loader, which they are to open source, but I can’t find it.
Seperate the loading of your page from the loading of the ads/3rd party stuff
Delay loading the ads a long as possible
Work with Business to explain the benefits of late loading. If the customer is engaged in the content they are not going to go away by the time the ad loads.
By delaying the loading of ads, they were able to speed up the entire page load time including the ads
Do not mess with ad internals
Solve the engagement problem, you solve the loading problem
Saw their click-through rate increase

Display Ads and Page Performance

Tony Ralph (which was at Yahoo during this talk, but has leapt to Google)
Podcast
Video
Monitor the speed of your ad networks
And their ads
Ad Load Performance Best Practices
The creatives’ primary goal is not performance
Maybe the value of this [slow ad] had value in the context of its displayed page
Do not treat ads an afterthought. Don’t build pages with the ‘this black box is where an ad will go’.
Give front-end people the tools and guidelines to work with
Publisher guardrails – quarantine ads
If an ad takes too long (of course you are monitoring them), backfill the content with a static local one

Worldwide Inventory of Last-mile Bandwidths & Network Latencies

Aladdin Nassar
Podcast
Video
CSI – Client Side Performance
Onclick handler on every event in the Hotmail site which starts a timer that stops at various events
Once you get the data, you get have to analyze it. Which can be harder than getting the data.
Also inject headers for the backend as well so they can merge the client and server side information for a whole picture of the performance.
There comes a point that adding any more bandwidth in a country does not improve speed
Look for biases in your data.

Fixing the Android Application Development Culture

adam — Mon, 25 Jul 2011 01:44:57 +0000

As a tester and sometimes agile coach-ish I have some pretty strong opinions as to what makes a ‘good’ development culture. Respect, communication, and oh, Testing. While not an organization per se, I think the Android culture is terribly broken. At least from the perspective of its answer to testing. So here is what I would do were I tasked with improving the culture of Android Application development.

First, it has to come from Google. And not just from someone at Google, but someone with the full weight and blessing of the Android division behind it. I’m sure Google has evangelists; an Android Testing one would be fantastic.

Typically, evangelists spend a lot of time on airplanes going to conferences. If you look at pretty much any Mobile/Android focused conference there is, if you are lucky, one session on testing. And if there is, their descriptions imply that are entirely black-box, manual testing focused. But what about automation? What about below-the-covers? What about, gasp, TDD? Mobile apps are complex; testing them is too. Let’s not pretend that just futzing with them in a couple emulators is anything close to be sufficient.

Speaking of TDD, the platform seems designed to make it really hard to do. And what tidbits of information that you can sniff out do not come from inside Google. I would love to see TDD become part of all the tutorials.

To Google’s credit, the component specific testing pages are decent, but it would be nice if the examples that come with the ADK actually had examples of the things that that they say you should test. And while on the specific topic of documentation, if it is suggested in the docs that you should use IntentService more often than Service it would be nice to have some code that does it. (So its not just test specific docs that have issue.)

Actually, I would take the documentation effort even further. Clearly Google wouldn’t want to, or be able to, approve books on Android app development before publication, but they could reach out to publishers and say something like ‘we are trying to address the lack of testing in our culture and would appreciate it if you could ask your authors to include it in their manuscripts as well’. I suspect a lot of the test infection in the Ruby On Rails community is largely because of the test focus of most of the books that are available.

Sites like StackOverflow will also Google where their testing docs lack. So far every question I have come up with has also been there. And usually without an answer that didn’t seem quite right.

Its a bit of a rant after doing some Android development, but I really do believe that within a year Google could turn around the culture from one focused on just what the user interacts with to a mature, TDD-based one similar to the Rails one. (Feels odd calling Rails mature…) The thing that is going to decide [this round of] the phone platform wars is the applications they support. Improving the quality of those apps would be an excellent salvo in that battle.

The TDD loop for Android Content Providers

adam — Fri, 22 Jul 2011 14:49:06 +0000

All Android applications are comprised of four major components: Activities, Content Providers, Services and Broadcast Receivers. What little testing and test automation that happens in this community is focused primarily on the Activity layer — if at all. A stroll down to your local tech bookstore all but confirms this fact.

But what if you wanted to have a suite of automated checks for something other than Activities? Or, shock!, you want to do something akin to TDD? Well then you would be into the great unknown. And is where I found myself even though I dislike Java, loathe Eclipse and am just learning Android. In this case things worked in my favour since I keep falling off the TDD wagon as I ‘know’ how to do something in Python or Ruby but with Java I’m using it as a learning crutch (stick?).

First, we have the test. Or at least what it ended up at as I understood the problem better. Kinda a Red/Refactor, Red/Refactor cycle.

public void testDatabaseCreated() {
    Cursor cursor;
    Uri url = Credentials.CredentialsColumns.CONTENT_URI;
    cursor = mResolver.query(url, null, null, null, null); 
    assertEquals(3, cursor.getColumnCount());
    cursor.close();
}

Essentially all this does is make sure that the [test] database was created and that it has 3 columns — they could be the wrong three, but at least there is three.

Of course along the way I had to make that file compile by fixing the constructor and understanding the not-really-documented-because-no-one-test-android-apps MockContentResolver. This class will make a backup of your application’s database and restore it when the run is done. Not quite Mocking, but close enough for what is necessary here.

Now to make it Green. Which wasn’t as straight forward as I had hoped. Showing my ignorance, I had thought there is ‘supposed’ to a 1:1 mapping of test classes to code classes. Not in the Android world there isn’t. In this case it is a 1:3 mapping which makes me think I’m not quite doing it ‘by the book’ but there is no book on doing TDD for Android. (Well, there is Android Application Testing Guide which is ok, but there is a definite market need here still.)

The first class I needed to make was the actual Content Provider class, which in typical Java style was a ‘right click and let the IDE to magic for you’. You end up with a class with six overridden methods. But naturally they don’t do anything.

Oh. And have I mentioned I decided to throw the additional twist of having my Content Provider interface with the build-in SQLite database?

Android is I guess what you would call an Event Driven environment where you override specific events to get your desired behaviours. So in my Content Provider I needed to override the onCreate method first.

@Override
public boolean onCreate() {
    dbHelper = new DatabaseHelper(getContext());
    return true;
}

Which creates our DAO for this particular Content Provider. DatabaseHelper is actually an inner class of the Content Provider.

public class DatabaseHelper extends SQLiteOpenHelper {
    public DatabaseHelper(Context context) {
       super(context, DATABASE_NAME, null, DATABASE_VERSION);
    }
 
    @Override
    public void onCreate(SQLiteDatabase db) {
        db.execSQL("CREATE TABLE " + CREDENTIALS_TABLE_NAME + " (" + CredentialsColumns.CREDENTIAL_ID + " INTEGER PRIMARY KEY AUTOINCREMENT,"
                                 + CredentialsColumns.USERNAME + " VARCHAR(255),"
                                 + CredentialsColumns.PASSWORD + " VARCHAR(255)" + ");");
    }
 
    @Override
    public void onUpgrade(SQLiteDatabase arg0, int arg1, int arg2) {
 
    }
 
}

As you can see, I haven’t worried about doing the onUpgrade scenario yet. (And that the new class wizard names arguments horribly.)

I had tried originally to create the DatabaseHelper class as a separate class so it cold be tested in isolation, but there is no information anywhere that I could find that describes how to do it. So I put it as an inner class as the examples I could understand all had it there.

The third class that I needed to create just so my simple test could pass is the one that represented the actual table. It is pretty boring, except for the CONTENT_URI line which is important.

public final class Credentials {
    private Credentials() {}
 
    public static final class CredentialsColumns implements BaseColumns {
        private CredentialsColumns() {}
 
        public static final Uri CONTENT_URI = Uri.parse("content://" + SauceLabsCredentialsProvider.AUTHORITY + "/credentials");
 
        public static final String CREDENTIAL_ID = "_id";
        public static final String USERNAME = "username";
        public static final String PASSWORD = "password";
    }
}

Access to/from a Content Provider is done through a URI. (If you are familiar with RoR RESTful URL’s you’ve got the concept.) In this case the CONTENT_URI is specific to a credentials table that the Content Provider has the ‘Authority’ to access. To be compliant with the spirit of Android development, you don’t do raw SQL against the database, but instead do queries against a URI.

This is enough code to create a database on the emulator/device and to allow my test to create the mocked one.

Or would have been if I had remembered to register it in the manifest and then didn’t totally screw up my environment with a package rename gone hopelessly wrong. (If you get a deployment claiming the Content Provider is already installed, check that your paths in terms of code match your manifest.)

Once I solved that I still had a null pointer on the query itself — because I hadn’t implemented it yet. (It was late and my brain was shutting off…)

@Override
public Cursor query(Uri uri, String[] projection, String selection, String[] selectionArgs, String sortOrder) {
    SQLiteQueryBuilder qb = new SQLiteQueryBuilder();
    qb.setTables(CREDENTIALS_TABLE_NAME);
 
    SQLiteDatabase db = dbHelper.getReadableDatabase();
 
    Cursor c = qb.query(db, projection, selection, selectionArgs, null, null, sortOrder);
    return c;
}

And then I was green.

Now what I have facing me is

public void testInsertCredentials() {
 
}
 
public void testQueryCredentials() {
 
}
 
public void testDeleteCredentials() {
 
}

which will force me to flesh out more of the overridden methods. Only when I have those all green will I actually wire it up to an Activity. So it is possible to do something like TDD for Android Content Providers — its just not straight forward nor documented well. Or at all.

How Not To Present At A Meetup

adam — Tue, 28 Jun 2011 12:38:29 +0000

I was at a meetup last night for a technology I am just starting to wrap my brain around. And while I didn’t learn anything about the platform I don’t think I did crystallize some thoughts around how not to present at a meetup. Something I like to think I know something about since I have been attending and/or presenting at them for at least 15 years.

A meetup is not a customer sales meeting. It may be awesome that you have lots of cool companies as customers — but we don’t care
If the tech is cool, show off the tech! The front of a room of 15 geeks is not the main stage at Demo
It is fine to refer to a crib sheet — but don’t be constantly referring to it and switching slides because your notes say you are on the wrong one
Have meetup appropriate slides
If it isn’t your code, don’t show it. Do not say ‘I had one of our techies whip this up’
If you show code, don’t show it in Microsoft Word. Oh, yes. I am serious…
You may be smart, and know some other smart people, but we don’t care
If you are give 60 or 90 minutes, then that is how long you session should be. Not 27 minutes.
Speak to your actual audience, not from your script. Saying ‘…while emailing your husband’ to 15 men seemed a bit odd. Perhaps the speaker was getting into the spirit of Pride week.

I’ll give this group a couple more shots before writing it off to see if last night was anomaly. But forewarned, if I’m in the audience, this is the criteria I’m measuring you against.

A Facebook application checklist

adam — Tue, 08 Mar 2011 21:48:43 +0000

I have only ever casually tested a Facebook app (which now appears offline), but were I to, I think use a checklist that looks something like this to guide my thoughts.

Sound, if present, can be disabled – One Mahjong game I play doesn’t have this option and is really annoying (and if it wasn’t for my wife having a score 1500 points more than me I wouldn’t play it at all).
No [obvious] algorithm problem – Another Mahjong game I have played in the past will stack the tiles sometimes in a way that it is impossible to complete the level
Spelling things correctly!
Good use of the Facebook API?
Cool use of the Facebook API? – My locale on Facebook is Pirate right now…
Non-scumbag use of the Facebook API? – Things like posting to my wall without prompting me each time count as scumbag moves.
Duration of activity – The apps I go back to most often have a timeboxed activity length. And that timebox is small; one or two minutes tops
[General] Re-playability - Is there ever a reason for me to come back?
[Daily] Re-playability – Is there something to entice me to come back tomorrow? And the next day? And the day after?
[Session] Re-playability – Can I sit here for 3 hours playing with the app? Not that I’ve lost entire days on Battle Tetris… No, certainly not.
Don’t be a Ponzi scheme – Some apps I’ve seen unlock certain behaviour only when you spam x number of friends with invites and/or they use the app. As soon as I hit that ponzi-wall, I’m out of there.
Security – Is communication to the host server encrypted? I have a plugin for Firefox that lets me post any score I want back to Bejeweled Blitz…

No doubt this grossly simplifies things, but I suspect it could generate a lot of testing ideas. Which is exactly what a good checklist does.

Jedis and Testing

adam — Tue, 01 Mar 2011 16:59:19 +0000

One of the part of James’ recent post that people have latched on to is this sentence.

Context-driven testers strive to become the Jedi knights of testing.

So I wouldn’t call myself a hard-core Star Wars geek, but I do know a significant amount about the Star Wars Universe. (I lived in Grand Cayman during kindergarten and grade one and we had a bootleg vhs of the first movie so I must have seen it a couple hundred times by the time I was 7.)

Now, I’m all for Star Wars metaphors but I think think the Jedi one is dangerous from a geek perspective.

See, the Jedi started out as a noble but then became obsessed with maintaining their own power base, fractious and increasingly irrelevant. And then they were wiped out by forces they denied existed until it was too late.

But it gets scarier if you dig a little further beyond the movies and into official (and unofficial sources).

To become a Jedi requires the deepest commitment and most serious mind. It is not a venture to be undertaken lightly. As such, Jedi instruction is rigidly structured and codified to enforce discipline and hinder transgression. – Right; discipline and seriousness to the craft is important. Don’t step out of bounds though? That’s kinda Factory School in its thinking.
While the Jedi undoubtedly saved many lives through their duties, many disenchanted citizens saw only their failures.
The way of the Jedi had become the way of wisdom and patience, backed by swift and decisive action when necessary. However, the Jedi Council sometimes showed what appeared to be a lack of decisiveness, such as during the Mandalorian Wars, preferring to work with events and patterns over the long term. Their inaction spurred Revan to fight back, and eventually led to the Jedi Civil War. To the rest of the galaxy, the line between Jedi and Sith became blurred during the conflict, and both sides were blamed for the destruction wrought on worlds such as Katarr, Telos IV, and Dantooine. – civil war and splintered factions. That never happens in the testing world.
Uncomfortable emotions such as hate, anger, and fear were thought to be destructive and lead to the dark side, so such things were banned from Jedi practice. – emotions are essential for testing.
Etcetera

However there is hope to to the use of the Jedi as a goal to be striven for. And that lies in the Gray Jedi.

The term Gray Jedi, or Gray, had two meanings. First, it was used by Jedi and Sith to describe Force-users who walked the line between the light and dark sides of the Force without surrendering to the dark side, and second, it described Jedi who distanced themselves from the Jedi High Council and operated outside the strictures of the Jedi Code. However, those who were considered to be true Gray Jedi met both qualifications and did not belong to any particular Force tradition.

This, to me, is exactly what one should be aiming to be — sticking with the metaphor that is. One can use the technique that is appropriate at the time for the task, relies on their own moral and ethical compass. Here is another blurb.

“The so-called gray Jedi have been with us since the beginning. Although they do not break with the Jedi orthodoxy concerning the dark side, they bristle when asked to take orders from the Council. Gray Jedi make compromises, cut corners, and hide their actions from scrutiny, all under the assumption that their experience makes them authorities on policy. They are mavericks who are difficult to control, but can be valued members of the Order after they have been persuaded to follow the established hierarchy.”

Put in the context of the authoritarian Jedi High Council this is often how the Context folks are seen. 92 page IEE829 compliant test plan? Not needed; let me test instead.

So context driven testers striving to be a Jedi knight is a worthy goal — as long as you provide context around which kind of Jedi knight you are aiming to be.

Choke

adam — Tue, 15 Feb 2011 01:13:12 +0000

I find I can’t read books on testing or automation anymore; at best I skim them only to really pay attention for a page or two before going back to skimming. (I know I’m paying attention when I pause the music for those who are curious.) This actually doesn’t bother me as I have believed for a long time that the next leap is not going to come from inside the field but outside. One area of interest is brain science.

I heard about Choke during the initial publication promotion blitz and immediately asked for a review copy (pro tip: always ask before buying). I was drawn to the book initially since I think the only reason (well, aside from regulatory requirement) to create a Test Plan well in advance of the actual testing is to get your thoughts organized before the [inevitable] crunch when you are under stress. Some people thrive under stress, but too often they experience The Choke where you perform less-than-optimally. And of course, in the classic development model this is a bad thing as the testers are the last gate before production. (Agile’s disdain for the ‘crunch’ is I think one of the reasons for its impact on potential quality.)

Sian Beilock is a psychology professor at The University of Chicago which to me means she is qualified to write about the brain. Choke has been written in such a way to be able to be consumed by non-neuroscientists which is a good thing; I once read a book on fiberoptics that should have required a post-graduate degree in math to just open up. That doesn’t mean its a quick or easy read, just that it is possible to read it. The first half of the book explains the different types of chokes and what is triggering it in the brain. And of course provides some mechanisms for combatting chokes — it is easier to combat when you know is happening to you. The second half brings it back to the business and sports realm.

A sampling of things I underlined in my copy

Choking is sub optimal performance
Experienced people benefit from hearing the thoughts of less-experienced people
Taking a step back before tackling a stressful task helps not only in the task but afterwards as well. The body/brain needs time to recover from a stressful event.
Finding meaningful ways to group separate information into bundles can take the burden off working memory and help you remember more — Mnemonics anyone?
Athletes’ tendency to overthink their performance is one big predictor of whether they will choke in important games or matches
Functional Fixedness
Adults learn new languages better when they are distracted and not concentrating too-hard on learning. This is because the distraction makes their prefrontal cortex look like a child’s
Stereotype threat
Changing how a math problem is presented on the page changes how the brain will attempt to solve it
Fight-or-flight stuff would be interesting
If you can manage to interpret your body’s response to a situation as a positive, as a call to action, you are likely to thrive
A trick to preventing the choke is to write down what is making your nervous about a situation changes how the brain reacts to it — incidentally, I no longer coach making a Test Plan but a Mindmap instead.
Meditation and Mindful Breathing are two tricks to help both combat and recover from a stressful event. Breathing is another thing to learn about this year. If you ever see me on a plane or just before I start a presentation I’m very consciously controlling how I breathe
The meditation bit is about being able to clear thoughts easier — specifically the negative ones.
When athletes think about themselves screwing up, they are more likely to
Worrying takes up working-memory — which you need!
When doing something you have done thousands of times before, concentrating on doing it correct is exactly what causes you to fail. ‘Just do it’ indeed.
Beginners should think about what they are doing. But only until they don’t need to anymore.
Practice only makes perfect if it is in conditions that reflect what would experience when using that skill.
Focus on what to do rather than how to do it
Learned Helplessness
Schemas for information are important
Mirror neurons is not ‘new’ to me, but still is fascinating

Is Choke a must have book for the average tester? Likely not, though if you find yourself getting stressed out during release crunches this might help you perform better during them. Now, as someone who has to talk a lot either in sales calls, training classes or conferences and who has choked in the past there are things that I certainly will be doing and help coalesce some of my own observations. If you thinking of moving onto the speaking circuit or going independent there are certainly worse ways to spend your book budget than on Choke.

And now a bit of semi-related commentary…

I’m not really excited about this year’s iterations of conferences I have been excited about in the past. It could be burnout (certainly it is partly) but it is also a reflection of who is there. There really is a ‘speaking circuit’ with the same people (us consultants make up a lot of it) sometimes speaking on the same set of ideas. People like Ms. Beilock are who we should be inviting to speak at these things. I suspect we understand most of the testing techniques by now — the next trick is how to apply them with the full knowledge of what is going on in our heads.

Startup Pricing

adam — Thu, 20 Jan 2011 15:29:01 +0000

While at CodeMash last week, I wandered by an OpenSpace which I will broadly call ‘So I have questions about running a Startup — with chocolate’ since I can’t remember the official name and there was chocolate. The discussion ranged from hiring, to marketing, to twitter sniping. One thing that got mentioned was the area of pricing, which kinda surprised me. To me, this is actually the area I have the least questions about for my not-really-started-but-kinda-started startup. In fact, it is just straight math.

Context:

My product is to be SAAS a service
I don’t want to be the next Google – If I ever get over 25 employees, someone point out this post to me. In fact, I realistically see it only being me for a decent length of time.
It is my intention to have only one obvious product – Some companies have a portfolio of products with the intention of driving traffic to their main one. I don’t want to be that. Do one thing, really, really well.
Bootstrapped – I understand why some people turn to VC, etc. but since I don’t want to be the next Google and don’t need to build my own private cloud, etc. I’m thinking I’m ok here. (I’m also have lots of opinions on startups and don’t want to cede control of things.)

Ok, that is a pretty stringent set of assumptions, here is my math.

$desired_income + $fixed_costs + $variable_costs = $total_needed
x * y = $total_needed

Let’s break this down. I actually think that the first equation is the more important one. You need to figure out what you want out of things in order to be able to do anything else. So let’s pretend I need $5000 take-home a month, which for arguments sake is $7000 before taxes (I have no idea if that math works; I’m currently in the ‘so screwed’ category for my taxes…). Thats only $84000 a year which doesn’t seem that great from a living in the lap of luxury perspective, but we’re just playing with numbers.

The fixed costs and variable costs are nothing that surprising: rent, servers, internets, phone, etc. For the sake of easy math, let’s call that $1500 a month.

This means that my startup only needs to bring in $8500 a month. That’s it.

On to the next part, which is actually surprisingly un-complicated in comparison now that we have a number that the math must meet. x is the number of users of the system, and y is the monthly subscription revenue. (Subscription is the model for the future btw.). Which means the all these are possible pricing structures…

# Users	Subscription Amount
4	2125
8	1062.50
16	531.25
250	350
500	17
1000	8.50
2000	4.25

One other ‘context’-y point is my gut feeling is the ‘right’ price point is $5 / user, but I have nothing to base this on except that is what FogBugz was per user and that seemed reasonable. I could keep busting numbers now, but it looks like I would need to hit around 1700 users to achieve my goals. Which isn’t really that large a number. And anything after that is gravy.

Oh, and yes, I know that as the number of users increases, so too does the costs so the number is likely higher than 1700, but not that much higher to completely invalidate the number exercise.

The trick in all this is to do the first half of the exercise and figure out what you want out of your startup? Insane amounts of wealth and a drunken trip to the Porsche dealership on IPO day (I know people who did this during the first bubble — shockingly, you can’t buy a car with champagne oozing out your pores) or an app that fills a niche and provides you with a sustainable income. I’ve done the exercise and I’m choosing the second.

Now to find the time to write some code…

Using skill in Bridge to guide tests

adam — Sun, 16 Jan 2011 01:37:53 +0000

How a bridge champ beats the market discusses how a 10 time national champion Bridge player uses his skills in that area to run a successful (if small) hedge fund. One paragraph stuck out when reading it.

The connection with bridge, says Moss, is the ability to decide what data matter, and then to have the judgment to act on it. “In bridge and investing, you are constantly being bombarded with an enormous amount of information,” he says. “The key is seeing all the possibilities.” There’s more. In bridge, an opponent’s tempo of play will tell an expert more than an amateur. Similarly, in investing, knowing which market indicators to monitor, and when, is more critical than watching every piece of information.

The last sentence is actually the most important one to testing. We can’t possibly test every permutation of inputs into the system or monitor every single response from it. We can however understand our [and our business'] context, what our mission is for testing, what things might also lead to new, interesting information.

And knowing how, when and why do that is what separates the professional testers from the amateurs.

Talent Is Overrated

adam — Sun, 28 Nov 2010 16:28:19 +0000

Talent Is Overrated is one of those books that you wouldn’t know about unless you are bored in an airport and wandering the bookstore. But wander I did and am glad for it. Talent is Overrated is basically another take on the whole notion of Deliberate Practice which gained some publicity in Outliers and the whole ’10 000 hours’ thing. Colvin doesn’t use that figure but offers 10 years instead. But rather than just touching on it for a chapter or two, what we have here is a full book.

The first three chapters are the setup by illustrating the fallacy of the existence of prodigies and how they all, whether intentionally or not, did deliberate practice on their field of excellence for at least ten years before achieving their peaks.

Chapter five is here things get interesting with the elements of deliberate practice.

Designed specifically to improve performance
Can be repeated a lot
Feedback on results is immediately available
Highly demanding
Isn’t much fun

These characteristics are discussed a bit in an article in Fortune by the author (which is also an editor there). Different people have different takes on these characteristics, naturally. This one lists eight for instance. (That site also links to How Entrepreneurs Acquire the Capacity to Excel: Insights from Research on Expert Performance which is a less-mass-market paper on the subject.)

Chapters six and seven discuss techniques for actually doing deliberate practice including some models for it. I think the model that works for testing is the sports model. This model’s learning is broken into two categories; general conditioning and specific training. In sports, conditioning is things like cardio and stamina but in testing it is more systems thinking and application and domain knowledge. Specific training is interesting because the application of the training will always be different. Just as no two pitches or golf lies are the same not two testing contexts are exactly the same.

Since this is primarily a business book, you can likely skip the how-to-apply-it-to-the-office unless you are the one making culture decisions at which point it is imperative to read. The last chapter is about passion (a truly extraordinary drive) and its motivation and had some good things for structuring both intrinsic and extrinsic motivators.

I have no idea how to apply the concepts of deliberate practice to testing. I suspect things like Weekend Testers come close, but don’t really cut it as they are only once a week. Testing Dojos don’t either also due to their infrequency. The Miagi-Do School of Software Testing sounds like it might also come close but I can find nothing to link to that really describes it. (Matt — fix that would you?)

Off the top of my head, I would guess that only James and Michael really do what could be considered deliberate practice by constantly proposing new games, puzzles and challenges to each other. But that arrangement works (I suspect) largely on their work with RST and might not for others.

Anyways, back to the book review. Should you care about deliberate practice? Absolutely. Should you buy the book? For me, I don’t regret the $20-ish I spent on it but if you just want the deliberate practice nudge and none of the background / proofs then you can likely skip it. Instead, go do some practice.

Software G Forces

adam — Sun, 14 Nov 2010 20:25:09 +0000

In what could have been the the most important keynote at a Quality Conference this year, Kent Beck delivered his Software G Forces talk at STP Con last month. To me, the most impactful idea of the year is Continuous Delivery and this talk is about not just the technical problems you encounter, but the organizational and business ones that companies have to tackle in order to do it. I’ve since suggested to more than a few people that they should print out the deck, tape it on the wall and use it as a checklist.

Software G Forces

View more presentations from KentBeck.

A lot of my notes were taken straight from the slides so are wonderfully redundant since the deck is embeddable. But there are a couple things that I don’t think are there.

Inspiration is preparation plus panic
The trick is to identify the visionary crazy people from the crazy crazy people
Automated acceptance tests – when you have 3 months to deploy you don’t have 2 months to test
Subscriptions – when you release all the time, customers don’t want/expect to be billed each release and it also decouples the billing cycle from the development one
Getting rid of the QA department does not mean getting rid of the testers — just the organizational split
Even with complete automation as a goal, things that need human judgement still require humans
Rollbacks are triggered by business metrics not technical metrics
If death is the alternative, then radical change is appropriate. Otherwise, test the water

Star West 2010 – Summary

adam — Tue, 19 Oct 2010 12:41:36 +0000

As part of my ongoing experiment of going to conferences to increase my visibility and promote my company I attended my first Star West conference this year. So far the experiment isn’t going so well and I lose money on every conference which is apparently not uncommon as the payoff from the speaker circuit takes a couple years to start. Now they tell me…

I had missed the submission deadline for Star West and didn’t want to pay full price for admission so I volunteered to host the Test Automation track for SQE. Insider Secret – Conferences are always looking for people to do this and you will often get comp’ed the admission by doing it. This meant that I didn’t get to choose which sessions to attend and also meant I had to stay in them for the entire duration.

Not being able to leave meant I learned a valuable lesson – don’t judge a session by the first 2 minutes. I tend to give up on sessions too early opting instead to sit in the hallways and socialize with people I only see at conferences.

So here are write-ups on every session from the Star West 2010 Test Automation track. It would have been nice to include slide decks in each from something like slideshare but those cost money to access and certainly are not embeddable.

The thing that most impressed me about my Star experience is the program book they give you. For each session is a page that has the topic blurb at the top and then ruled paper for the rest. This means notes are nice and tidy and in one spot. CAST and other similarly sized conferences should take note of this though I don’t see it scaling to Agile size without massive deforestation.

My experience was however blighted by some of the HP booth staff who I think owe me an apology. And I suspect that as a result I’ll be persona non grata at SQE events for awhile. Which is fine with me for the moment as I don’t think I’ll be doing the conference thing for at least 2011.

The HP Booth Staff (Goons) Owe Me an Apology

adam — Tue, 19 Oct 2010 12:38:18 +0000

So a curious thing happened to me while in the Dialogues session at Star West this year. As mentioned in the write-up, I was in the Getting Started group and by nature of being seated next to the easel, I took the marker to become the Secretary for the group.

I think there were about a dozen people in this particular group and at the risk of sounding overly pompous I was the only person who had succeeded with automation. Which makes sense if you think about it as groups were self organizing — those who didn’t need to talk about getting started went to more advanced / specific topics. I only went to this group as Star West was part of a year-long experiment on going to conferences as marketing. (I’m actually writing this on the plane to STPCON.)

The latter half of the conversation focuses around tools and tooling. And as I remarked on Twitter later, Selenium doesn’t have to do much marketing to keep growing as there is a lot of dissatisfaction in the market around QTP if this sample is any indication.

With full disclosure given that I work on and with Selenium, I gave general consultant-y sounding answers around tools. Some of which made it to The Relationship of Testing Tools to Economics & Freedom. To me, using QTP to test web applications is a darn near unethical waste of company resources (cash). Yes, there are times when you might need to use it. For instance, when I was in the WinRunner world there were plugins for driving terminal emulators and powerbuilder applications and I wouldn’t be shocked if QTP wasn’t the best thing in those categories.

My argument against QTP aside from the cost goes something like this…

It’s closed source (so you can’t build your own lightsaber)
It doesn’t have a ‘real’ scripting language. Where real means cross-platform and where skills learned in it an be transposed to others as well. VBScript might as well be Vendor Script in automation.
It is Windows only — and there are increasingly more and more non-windows consumers of web applications every day. If you have a public web application that doesn’t care about the increasing Mac (or even Linux) market, let me know and I’ll create a clone to target it.

All three of these were brought up during the session and even though I repeatedly said “I could take this over quite easily, but don’t want to” it became somewhat Q & A.

Unbeknownst to me though there was a ‘mole’ [used tounge-in-cheek] in the group who did not enjoy my providing an alternate view to QTP being ‘all that’. This person did two things after the session that significantly changed my Star West experience.

First they wrote a long, and scathing comment on the back of the feedback form about my ‘bashing’ of QTP. Feedback forms are very important to the Star conferences and are used as input for whether speakers are invited back and to use it to comment on another session participant would be an un-necessary blight against the actual session coordinators which is unfair. And frankly cowardly. Whomever you are, you’ve lost my respect.

I’m extrapolating on that the second event is related to the same person, but during the break I was still in the room talking to a few remaining people when two people from HP’s booth approached and introduced themselves. Someone had come up to their booth and was told that I was ‘bashing’ (same word as on the feedback form) on QTP and informed that HP, as a sponsor ‘was not allowed to bash Selenium so I should stop bashing QTP’

Let’s pause for a second for context.

I was not at Star West as a sponsor.

I was not at Star West as a speaker.

I was there as a volunteer.

My badge said ‘Delegate’.

Now, I did get comp’ed entrance into the conference by volunteering as a track host. This meant I didn’t get to pick which sessions I went to as I had to be in all the ones I was hosting — so though no money was exchanged a price was paid. The job of the track host is to help the speaker get their laptop working on the screen, mic’ed up, introduce them and then thank them at the end. And be a gopher if needed. But other than that you are just another session participant. It is not a paid position and the person does not assume any representation of SQE (the organizer of the Star events).

So I, a delegate, was told to be quiet by a vendor/sponsor. Ummm, ya, HP — your money didn’t buy you that. Or shouldn’t have at any rate.

Now, if I was either of those things then the rules would be different. Star prides itself on not having marketing sessions and playing fair-ish on the trade show could be seen as just good etiquette. Remember though, that I was a delegate.

I’m pretty pissed that those two from the HP booth had the nerve to try to censor me. The proper thing they should have done is to ask to talk to me and see if they could try to address my concerns about their product. They’re not going to win me as a convert, but at least they would know what I was saying and the reasons for it. But instead they sent down the goon squad.

And for that, I think I’m owed apology — from HP.

Three final points.

First, when recounting this to Rob Sabourin (whose session I missed as I was doing my volunteer duties) later he suggested that listening to a recording of what I actually said and how I said it might be useful. There are a couple people in the testing community that when get onto passionate subjects can come across stronger than intended. I concede that this might be such a situation. Alas, no such recording exists (to my knowledge).

Volunteers at Agile all wear shirts emblazoned with ‘VOLUNTEER’ on them so noone can accidentally assume they are anything but during their shifts. All conferences, including Star should pickup on this model if not with shirts then buttons or ribbons or something else. I fear that because I was seen in all the automation sessions more was assumed about my status that it really was. (Though ironically my session hosting duties for this particular one were limited to cleaning the room in-between and after the session.)

And finally, a sincere apology to Dot and Mieke on any possible repercussions from the feedback you might have received. And to Lee Copeland (program chair of Star West) who had his morning ruined by having HP track him down so as to have to track me down to have a chat (though I think the right answer should have been to tell HP I was neither a speaker nor sponsor so HP can’t complain).

Handling Failures in Automated Acceptance Tests

adam — Tue, 19 Oct 2010 12:28:27 +0000

Alex Imrie’s talk has the distinction of being the one I took the most notes in. In a nutshell, it was about when and how to handle when things blow up in your scripts. And they will. In her intro she said that she “shouldn’t be here” since she comes from a linguistics background. I would argue that makes her more qualified than most to be there; we need an even more diverse set of backgrounds thinking about stuff in test.

We want our automation results to be constant
Some definitions
- An ‘event’ is a deviation from plan
- An ‘exception’ can be handled in the script
- An ‘error’ can’t be dealt with in the automation
I think I have a problem with the chosen terminology since they already mean one thing in a programming context and now we’re overloading them with something else. The distinction and need for a name though is valuable
Events can happen from the development process, the test environment or the test platform. Ideally we want them to be in the platform as that is where the interesting things are from a testing perspective.
The big solution to these is really better communication (we knew that) but also one can try to build in some manual tester intelligence
React to the real issue, not the inherited problem
Use state validation before and after an action — which can clutter the script, but is valuable especially things like Selenium which will happily click on a hidden field.
But of course that means state hooks were added for you.
If you can’t eliminate dependencies, at least know them
Setup and teardown are often the most complex parts of automation
‘Local’ event resolution has the workaround in the test itself
‘Global’ event resolution is more like a high level exception handler that logs a message and then bails out of the script
Can your teardown work in both happy and sad test situations?
Start with a Global handler and add Local ones as needed
Continuous Integration + AUtomated Testing = Merciless Transparency

Futility-based Test Automation

adam — Tue, 19 Oct 2010 12:26:21 +0000

Clint Sprauve did a pretty slick talk at Star West of the format ‘do these things and you will fail’ style that is popular these days. You know you are attending a talk in the US when it starts with a football metaphor; kinda worked, but likely could have gone better with a different audience. Ah well.

Unit tests is like only spellchecking for a novel — spell correct but is want to read it?
The techniques…
- Record and Playback only
- Dependencies between Tests
- Elaborate Test Frameworks
- Substantial Duplication of Tests
- Happy Path only
- No (or not enough) Unit Tests
- No (or not enough) GUI Tests
- Poor readability of test cases
- Go Agile and force BAs and SMEs into automation
- Keyword Driven Testing
Manual testing didn’t die because we created automated testing
A Pragmatic Approach
- Don’t let tools’ drive your quality efforts
- Use DSLs (either Cucumber-ish or Keyword)
- and a couple other things I didn’t catch

Testing Dialogues: Automation Issues

adam — Tue, 19 Oct 2010 12:24:40 +0000

Thursday morning at Star West found me in a workshop held by Dorothy Graham and Mieke Gevers. It was actually two workshops split by the morning break. The attendees self-organized into 7 or 8 groups to talk around a particular topic.

For the first part I went to the ‘Getting Started’ group as I certainly have no small number of ideas on how to do it. There was lots of angst about having automation thrust on people and in particular QTP (which then inspired The Relationship of Testing Tools to Economics & Freedom). It also resulted in another unfortunate incident which I’ll document later.

The second part was I think much better than the first. In the second, there was only two groups; one which Mieke participated in directly on building a framework and one on ROI which Dot facilitated. This format lent to greater depth of dialogue and deeper learning. In the future, I think they should do it in this format again.

Each group took notes on chart paper art-show style which were photographed. Once I get a copy of them I’ll update the post.

State-driven Testing: An Innovation in UI Test Automation

adam — Tue, 19 Oct 2010 12:20:57 +0000

Dietmar Strasser talked at Star West 2010 about incorporating State into Keyword-driven automation UI tools to limit the amount of keywords presented and reducing use of keywords at the wrong time. It seemed to drift from product demo to cool idea show-and-tell but ended up not really being either as it is just an internal tool they are playing with. My notes are really tool specific and not really relevant to the general audience. A cool experiment, but there are way too many layers I think and can see creating / maintaining within this system being a huge task. His internal usage says otherwise, but then again, he wrote the tool. How many times have I fallen into that trap?

Keyword growth is a problem.
State-driven Testing = Keyword-driven Testing + (UI) State Management
The tool has an XML based DSL to model the state
That XML is parsed to control the state stack
Who does what?
- DSL – tester, domain specialist
- Test Design – tester, domain specialist, product owner
- Implementation (Java only right now) – developer, automation engineer

Automating Test Design in Agile Development Environments

adam — Tue, 19 Oct 2010 12:17:34 +0000

Antti Huima loves algorithms and his talk at Star West 2010 reflected this. And that is actually a good thing.

If V-Model is constructing a skyscraper, Agile is working on an oil painting (since oil never really dries and is constantly reworked). This is the best metaphor I think I’ve seen on the old vs. new way of doing things.
He mentioned crunch time though a couple times when describing agile. If you have a crunch time, you’re not agile.
Automated Test Design (ATD) in this context means producing test cases by running some sort of algorithm or program. Such as Pairwise, Finite State Machines or System Model Driven.
Its automated since a machine is doing it
BUT humans are involved in every step
When choosing your ATD strategy, think about
- How quickly can you update your model and generate new tests?
- How small an increment can it handle
You can use agile for implementation and v-model for test design
Rules of thumb around requirements
- Small changes in requirements must lead to small changes in models
- Must be able to start with a small model rather than the whole thing immediately
- No significant delays caused by tooling
- Support for generating directly executable tests
Only model what you need right now
Helps TDD since the tests are created for you. I don’t think I believe this one. TDD’s big win is not the suite of regression tests (which are nice) but the evolving design and testability that result of building the tests by hand. Having a machine do them misses the point.
You can’t as Deep Blue why it did a [chess] move. The same applied for your generated tests. Which is the big weakness. Go ahead and ask your algorithm ‘why’
Models don’t have to be monolithic

Building a Successful Test Automation Strategy

adam — Tue, 19 Oct 2010 12:15:21 +0000

Karen Rosengren’s Star West 2010 session was about how to build a test strategy. The big thing for me is that I need to learn about Value Stream Mapping.

Identify the problem — capture metrics that shoe you have a problem and the value of solving it. A big thing here is that you are making sure to understand not only that automation can solve a problem but that it should actually be solved.
Symptoms are not useful in identification
Too often the automation team’s job is to solve your company’s equivalent of world hunger
Start from where you are. So where are you?
Value Stream Mapping – sit the team down and find out what you are doing now
True value is adding when I’m running the tests. The setup etc. is necessary time but not value add.
Big wins to be had automating to build deployment and environment configuration. She uses Tivoli (since she works for IBM), but nicely avoiding turning it into a Tivoli sales presentation — which it could have quite easily.
The Star audience is largely non-technical so she pointed out you are going to need programmers.
Split metrics into two buckets
- Business Impact – illustrate the business value and help sell projects
- Operating – how well the implementation is going
Automation is not just test execution
Rank scripts by time and effort when deciding what to automation

Where I start to experiment with Cucumber

adam — Sat, 02 Oct 2010 15:37:50 +0000

Earlier today I made a mini-plea on Twitter for TDD references (specific to Rails 3) since I’m starting to build an app and should do it the ‘proper’ way. Ends up what I really wanted was BDD/ATDD references; specifically Cucumber + Rails 3.

Following the concept of Minimum Viable Product as a model I sketched out the first two Cucumber features while waiting for the boy at karate this morning. And already I had one of those ‘oh, wait!’ moments by realizing I hadn’t done the ‘show the landing page’ stuff before the ‘signup’ parts.

So here are my features. Again, the eventual signup form will eventually have a tonne more information, but all I need right now is username and password.

Feature: Landing
  In order to use test maps
  As a user
  I want to see the landing page
 
  Scenario: Display landing page
    Given user is not cookied
    When user goes to site
    Then landing page is displayed

Feature: Signup
  In order to create test maps
  As a new user
  I want to register
 
  Scenario: Display registration form
    Given user is unregistered
    When user clicks they want to signup
    Then signup page is displayed
 
  Scenario: Happy path signup
    Given username is available
    And password is strong
    And password is verified
    When user clicks signup
    Then user is created
 
  Scenario: Username available
    Given username is already in use
    And password is strong
    And password is verified
    When user clicks signup
    Then message 'Username is not available' is displayed
 
  Scenario: Password length
    Given a password is too short
    And password is verified
    And username is available
    When user clicks signup
    Then message 'Password needs to be 5 or more characters' is displayed
 
  Scenario: Password verification
    Given a password is not verified
    And password is strong
    And username is available
    When user clicks signup
    Then message 'Password fields do not match' is displayed

Anyone have any stylistic suggestions?

Now all I need to do is implement the actual steps. Heh; all…