It’s been so long since I wrote anything here, I fear I’ve forgotten how. All I’ve written in the last few months is academic analysis of concussion data for scientific journals, so here’s hoping this doesn’t come out that way. But here goes…

The failure to regulate “super shoes” hurts running. It undermines one of the sport’s most valuable qualities, namely that the outcome, the title, the victory, goes to the athlete whose physiology is optimized through training and genetics, then enabled by tactics, to cross the finish line first.

We celebrate human victories born of VO2max, running economy, thresholds, neuromuscular efficiency and effectiveness. We hail records and PBs as breakthroughs made by the athlete, and celebrate champions as superior to their peers because of these human qualities. In the vacuum created by absent leadership and regulation, shoes have changed that. It’s why, when we recorded a podcast analysing the Nike Vaporfly shoes, we said that they “broke running“.

Last week, or indeed, over the last four years, the authorities have had the chance to ‘fix’ this. They chose not to, and instead, through a combination of passive governance, inaction and weak action (motivated by who-knows-what), they effectively codified shoe tech into a policy that will continue to distort running. It will continue to change the relationship between input and output, separate the sport into haves and have nots, and ultimately force it to be recalibrated and understood in a very different way from what it is now.

This is an article explaining why I think this.

[ribbon toplink=true]From first principles: the fundamental premise of sport – meaning in results[/ribbon]

First, let’s try to disassemble the whole thing into its smallest principle. And if you don’t agree with these principles, and land on a different outcome, at least we’ll know why. You can also save time reading the rest of this article!

As I see it, sport has value because the results have a meaning that we create and accept using sometimes arbitrary lines or classifications. This is why, at the very highest ‘strategic’ level, men and women compete in separate categories – it ensures that the outcome of any sport is determined by the things that society, generally, has determined are meaningful. Mixing men and women destroys the meaning of women’s results because they’d be determined by factors other than those we deem to be important.

For the same reason, age classifications exist – there is no meaning in sporting results when a 25 year old beats a 16 year old in just about all sports (with a few exceptions). We have weight classifications to ensure that combat sports retain their meaning and integrity for a 63kg fighter who does not have to go up against a 91kg fighter. The Paralympics tries their best to ensure ‘meaning’ of results for athletes with different degrees of the same disability by creating numerous classifications. This should be obvious to anyone.

In principle, then, one of the fundamental roles of any sports governing body is to defend these lines, however arbitrary they may appear to be. Why have a line at 63kg separating featherweights from lightweights in Olympic boxing? Why a cut-off of 18 years between junior and senior? Indeed, even sex boundaries between male and female are being challenged as arbitrary (wrongly, in my opinion, but there you go). But you need these lines, or the meaning of sport is undermined. Salary caps, by the way, try to achieve the same thing.

The same is true for doping vs anti-doping, by the way. Anti-doping creates lines that are meant to assure us that results are not determined by pharmacologists and doctors, or by which athlete is the best responder to EPO, or who is the athlete most willing to take risks, have the most money etc.

But, why a line at some pain-kilers, but not others? Altitude tents, but not EPO? Cortisone, allowed out of competition, but not in, despite evidence that it’s effective no matter when it’s used.

Point is, for the integrity of sport, these lines must not only be drawn, they must also exist in the right place, and then be enforced, to ensure that meaning exists in the results of the sport.

That meaning, in turn, is decided upon within each sport. So the result of a Formula 1 race, or season, has a different underlying meaning to that of a tennis tournament, or a marathon. This should, again, be accepted by all, I’d think. We celebrate a unit of driver-car (or is it car-driver?) or motorbike-rider in some sports, knowing that the equipment makes a bigger difference to the outcome than the pure ability of the driver. But because they work as a unit in an understood way, it’s accepted. Those who follow sports understand this.

[ribbon toplink=true]The principle of equipment-determined results and the first problem for running[/ribbon]

Now let’s apply this “sports results must have meaning” philosophy to equipment. Different sports decide on different ‘lines’ that create boundaries on what equipment is allowed and disallowed. But, and I can’t make this point strongly enough, they all regulate their equipment. I cannot think of a single sport that does not impose some regulation on the equipment used.

Boxing? Glove weights are regulated depending on fighter weight.

Tennis? Balls and racket size and hitting area are regulated. Hockey? Length and width limits are specified.

High jump? Yep, the thickness of the sole is regulated, this after a Russian jumper used a shoe with a thickness of 40mm, so effective that it allowed him to “emerge from mediocrity like a chorus girl hopping out of a paper cake” (as quoted in Sean Ingle’s good article on the shoe wars). In response, back in a time when authorities clearly had some courage and a moral compass, a limit of 13mm was introduced, problem solved. High jump results are thus determined by something other than a shoe.

Swimming? Who can forget what happened between 2008 and 2009 when the absence of regulations led to a clearance sale on world records, a near clean sweep of Beijing medals for one company, and the subsequent absurdity of it. In response, rules were created limiting the type of material used and the length.

In all these instances, the regulation imposes limits that are meant to ensure that the results have a desired meaning. That is, they want the outcome, their champions, to be recognized for displaying attributes that they believe are meaningful. If we did not believe that biological sex should be regulated, we’d have a “human race” rather than a men’s race and a women’s race. If we did not believe that skill execution should be “constrained” by tennis rackets, we’d have no limits on head size.

Running? No regulations (until now). Despite the awareness that it was possible to change the sport using the shoe, no policy existed. A vacuum. Into which Nike stepped, with development beginning on a shoe in 2013, first producing the Vaporfly that powered Kipchoge to within sight of a 2-hour marathon, then helping to break just about every city marathon record, dozens of world records at distances ranging from 10km to marathon, and possibly hundreds of other course and national records.

I would argue that this disrupted the meaning of running. It broke the principle. The premise is that running, ‘natural’ as it is, should not be decided by who wears the best shoe, but by who has the optimal combination of physiology, psychology and tactics. We read famous scientific papers describing the physiology of elite marathon runners, the VO2max, the economy, the lactate threshold. We hold those attributes up as essential, and then celebrate the best runners as manifestations of them.

Even the mid- or back-of-the-pack runner celebrates a PB as their breakthrough – they’ve changed themselves. And even though most won’t necessarily know that they’ve improved their VO2max by say 4 ml/kg/min, or that their COT has dropped from 215 ml/kg/km to 205 ml/kg/km, it’s their breakthrough. Human.

It’s much the same for many sports. I don’t want a five-set final at the Australian Open thinking “Shit, I wonder if this result would be different if Thiem and Djokovic could swap tennis rackets? If only my guy had a different sponsor”. Nor do I wish to watch swimming wondering whether the guy in Lane 4 might actually be a better swimmer than the guy in Lane 5, but with a swimsuit that is 3% worse.

If this happens, the meaning of that sport is undermined. But this is the situation that has developed in running, now facilitated by the regulatory equivalent of the Emperor’s clothes, is that we have to watch running events not just wondering, but KNOWING, that if the runners could swap shoes, the outcome would certainly change.

The point is this, and I’ll return to this again – when the difference made by technology is larger than the normal difference between athletes, then the integrity of the result is changed. If that tech is unevenly distributed, with differences in access to it, then it becomes unfair. In the spirit of the piss-taking that shoes and running have become, I’ll call this “Ross’ rule”.

But if this rule is met, the meaning of running changes, and the outcome is no longer about the physiology but about the shoe.

That’s what began in 2016, accelerated in 2018 and 2019, and is now set to continue. And this not simply about Nike getting a jump on its competitors, it’s about the fundamental fabric of running, and even if every shoe company is able to match the Vaporfly effect, the sport will not return to a position of integrity, because the ‘shoe wars’ will change the meaning. I want to try to explain briefly why.

[ribbon toplink=true]Responders, non-responders and the problem with “equality”[/ribbon]

The main problem with the regulation is that it cannot possibly achieve parity between athletes while still allowing the sport to keep its meaning, as described above. Why? Because the tech, as powerful as it is, will create a gap between responders and non-responders that is larger than the normal difference between athletes. As a result, the “best case scenario” you can think of will still see shoe tech decide the outcome.

When the first batch of lab tests on the Vaporfly came out, there were some striking observations. The first was the average effect: 3% to 4% less oxygen used at the same speed, which translates into an estimated 2% to 2.5% performance benefit for the elite marathon runner. That’s the source of the titles, the records, the initial wave of controversy.

But second, and now more important than the first, nobody found “negative responders” among the semi-elite runners tested. The four studies I saw all showed the change in running economy for every participant tested. I recall one athlete who had zero response, everyone else saw a benefit (a reduction in O2). A study has since come out showing negative responders, which makes the points below even stronger.

The range of these responders was striking. The first paper had a spread between 2% and 6%, with an average of 4%. The second was similar, though they found a 0%, and then up to 6% (see the figure below, from Hunter et al). The outcome you see here is not perfectly understood. The consistent theme of thee studies is that rear-foot strikers get larger benefits than forefoot strikers, and this study also showed that those with shorter ground contact times get a larger positive benefit (not surprising).

But, that finding has not been replicated in other studies, and some studies even disagree on the global kinematic changes that occur when wearing the shoe, even though they all agree on the size of the benefit. So the jury is out as to who responds and who does not, but it’s clear that the spread is large, and it’ll get larger as the average benefit increases.

This creates a major problem for result integrity. In this study, for instance, there are runners, five in total, who get more than 4% benefit, and there are 7 who got less than 2%. A similar spread was subsequently confirmed in a third study, and if responses are normally distributed, it means that you’re going to see a pretty large gap between the best and worst responders.

The same shoe, then, spreads runners out by about 6%. And that was the first version of the shoe. It should be relatively obvious to you that if the later iterations are better, then the upper limit will increase, possibly to 8%, maybe some even get 10%. Even if the bottom remains at 0%, the range of benefit is thus stretching out, with performance implications that shouldn’t be ignored.

What this means, on this the 6th February 2020, given the flaccid shoe regulations announced last week, is that in a group of 100 runners who are similar in every measurable physiological sense, and for the sake of example, have performances that differ by 2% when wearing the “old shoes”, the result when they all wear the very same shoe is that you’ll see a spreading out based on who is lucky enough to respond well to the tech and who is not. The worst of them, 2% behind the best, could suddenly be 4% ahead of that best runner, if they happen to get the right combination of responses.

Therefore, even if you want to be eternally optimistic, and assume that the likes of Saucony, Brooks, New Balance, asics and adidas are able to bridge the gap to the Vaporflys, you still won’t be able to say that running rewards the best runner, because you now have to factor in another factor that makes one runner “better” than another, namely their response to the external factor of a shoe.

This is not unlike the situation that developed in cycling, peaking during the time when EPO use was rampant and unregulated. In that sport, one of the key determinants of success became the cyclist’s response to the drug. Some got more, some less. It changes talent ID, because the addition of this external factor overrides the normal difference made by the things that cycling deemed meaningful to its result.

Now we’ll see the same with shoes. Eventually, if you follow this to its logical conclusion, it means that the runners who emerge from the pack at a young age will be those who are responsive to the shoe. Those who do not simply won’t win. The benefit of being a “6 percenter” is so large compared to being a “2 percenter” that the poor 2 percenter would need to be 3 to 4 minutes faster just to be level.

Now think about what this means for the pathway that produces runners. Trace it back to high schools, into colleges, via scholarships and contracts, and financial viability. The most accessible sport in the world changes.

It means, ultimately, that the primary determinant of success, the key predictor, is whether a runner responds or not to an object they can lace onto their feet within seconds of starting the race. That, to me, is not the meaning of running, and so I return to the above principle and conclude that shoes will determine the result even if parity is achieved in terms of average benefit between Company A and B.

The bottom line is that the range of responses to the equipment are larger than the difference between athletes, and hence, the integrity of the result is diminished (as per Ross’ rule). And, I can’t stress this enough, so I’m typing it again, this is true EVEN IF every shoe company gets to the same point of average overall benefit.

Your response to this may well be “so what”? The result is determined by who responds anyway. Who responds to training, mostly? A genetic lottery of sorts, that means that to be elite you need the right genes that optimize your hardware, combined with the genes that optimize that hardware in response to the stress of training. Yes, sure, I appreciate that, but now you can add an object, purchased at the store for $250, laced on the feet, that skews the things that have always given running its meaning.

Some may get that. Some may not. Some will be lucky enough to land up in the shoe that gives them 6%, meaning they finish a marathon 3 min faster than the poor sucker who gets the shoe that gives them 1% to 2%, but is actually a better runner (as we understand it in the time BVF (before Vaporfly). A new element in the mix exists, but it’s not innate, or earned, but rather bought, and through luck and access, will sort the running populations.

[ribbon toplink=true]Problem 2: Can the other companies respond?[/ribbon]

More briefly, a few other issues with the Emperor’s clothes shoe regulation. First, the notion that other companies will respond remains to be proven. If you’re optimistic, I reckon you’d say that within a year, maybe two, shoe companies will achieve a degree of parity.

I certainly hope so. At least it sorts out the issue of “haves” vs “have nots” that have tarnished running for the last three years. I’d still argue that the issue persists, but it will be smaller in scale.

The question is whether they will? Patents, cost and access to materials that will allow this stand in the way. I know that some companies have been innovating in ways that would allow them to steer clear of Nike patents on the curve of the plate. They have developed other ways to engineer more ‘spring’ and propulsion. Those ways are now illegal, because the policy dictates that it is so. I’d argue this is ultimately a good thing, but it does formalize an existing advantage behind patent barriers.

Effectively, this squeezes the range of innovation in a way that means that a plate must do the job, but now you’ll hit up against patents. I know that one company is using two plates, but they’re on the same plane rather than staggered (and so thus legal). Another brand has a plate spanning three quarters of the shoe, rather than its full length. Patent side-step.

The problem is this – will these ‘side-steps’ allow the same effect, or was there a “best way” to achieve performance enhancement, now inaccessible? For all I know, there is an even better way, and Brooks or Saucony or adidas will find it, and then this conversation can begin anew, but with a different front runner, and a marathon record of 1:56.

On the other hand, if the full length plate with that curve is the best way, combined perhaps with the air pods now present in the latest Alphafly Next%*, then the sport will continue to split into the privileged vs the unfortunate, which ultimately undermines the commercial model that sustains it.

* An outrageously ludicrous shoe, by the way, and very, very fast, from what I’ve heard from a few sources. It is a slight modification on the joke of a shoe that I suspect could have powered Kipchoge to a sub-1:58 in Vienna last year, so “easy” was that 1:59:40 performance. But, appropriately, it was released less than a week after the Regulations were announced, because it makes such a mockery of them. It has a 40mm stack height, a remarkable co-incidence given the regulation limit, and really just so lucky for Nike that they just happened to hit the mark only a few days after we knew what would be allowed. Add some air pods (see pic above) near the front to concentrate force to the plate to enhance energy return and propulsion more, and they seem to have thickened the forefoot to aid rule compliance. It makes me suspect, not that I didn’t already, that the regulations are compliant with their shoes, rather than the other way around)

In any event, one of the intriguing questions that has been created by the Regulation is whether others can match the Nike. Best case is yes, and quickly, which means the only issues that would remain are:

a) cost of the shoe – some exceptional “candidate athletes” may never even reach the shop window because they can’t afford to optimise their performance. This is something that is true of say, sailing or horse-riding, much less so in running;

b) the afore-mentioned issue of response vs non-response, even in 100 runners all wearing the same equipment. But then, if you accept this “new normal”, just include it in your mix of factors required for running success. Maybe some of those who’ve been gaslighting this conversation from the start can team up to write a paper or magazine article on the characteristics of world-class runners and establish this as Talent ID priority number 1. Here, I’ll even write your main finding for you free of charge:

“The key attribute of world class runners is that their Oxygen Cost of Transport (COT) is reduced by 6% or more when they lace up a pair of high stack height shoes with carbon plates. This response, shown by studies to exist in 10% of elite runners, negates the need for a VO2max or running economy in the 95th percentile, and is a characteristic of every Olympic medal winner tested in the last two decades”.

Worst-case is that companies never bridge the gap. Patents and cost of innovation are beyond them, since they may not have the financial clout to compete in the lab and in the courtroom. That would create for running a situation like that which exists for motorsports. The wealthy succeed, the middle class fall away, the bottom class never even starts.

I suspect the answer will be somewhere in the middle. I think companies will make inroads, perhaps not complete, but enough to make it appear like the brand of shoe is not deciding the result. That would achieve a degree of stability.

When this happens, I don’t know. Not in 2020. Too soon. Perhaps 2021, more likely 2022. You may as well accept that the medals in Tokyo are going to be determined more by shoes than by physiology, and that what you’re watching would indeed be different in they swapped equipment. Beyond that, who knows?

[ribbon toplink=true]Problem 3: Implementation[/ribbon]

The next issue is implementation. Once you draw the lines, even if you assume they’re in the correct place, you need to enforce them. I don’t know how this is done. You need more than an x-ray because carbon is radiolucent, and so it’s MRI or physical inspection.

But given what the policy will allow vs disallow, things like overlapping or stacked plate arrangments, you’d need to cut the shoe up, or run it through an MRI and is that going to happen.

Even stack height, whose limit has been set at 40mm, is not as simple to measure as you may think. One of a handful of really stupid things in those regulations is the failure to set stack height limits based on the size of the shoe. The Regulation allows minor variations in height due to size, but not clearly stated, and so shoe companies will just mislabel the shoe as larger than it is, which will then necessitate that the size of the shoe and its height be assessed either at the start or finish.

But if it’s the start, it has to be right there, because you’d need to ensure that the athlete doesn’t change after measurement, so you need “watchers”. That deals with the outside, the inside remains hidden until you get the MRI or its simpler cousin, a hacksaw and an honest pair of eyes.

Even then, given how the IAAF failed totally to enforce its regulation on prototypes, and the various conflicts and bumbling implementation we’ve seen on doping enforcement, hands up if you truly trust a shoe inspection to be trustworthy? Not for me, no.

So I have grave concerns over the integrity of the process by which the rules are enforced. Given what we know about the lengths to which companies and teams go to cheat, it’s inevitable that someone will try to bypass the rules, adding 1 to 3mm here and there, finding another 0.5% to 1%. We also allow modification of the shoes on the grounds of medical need. So basically, we need TUEs for shoes. Hilarious. And what happens when that system is manipulated because a medical modification also achieves a performance boost? (this is inevitable)

And to think, it could have been so much simpler – just set that stack height limit so low that the space in which an engineer could work is massively constrained, and you’d to worry about one variable only. Plus, it might have been actually been effective.

[ribbon toplink=true]Problem 4: The prototype issue[/ribbon]

Next, the issue of prototypes. First, it’s funny that so many people are complimentary about their ban on prototypes. How easily we forget that the only regulation that existed for the last few years was that the shoe had to be available to all. OK, it was incredibly vague, specified nothing, and was not fit for purpose, so when they announced that it would now have a four-month exclusion period and had to be sold online or in stores, it seems like progress.

But the funny thing is that in announcing that, all they really did was dress an old rule up in some new clothes and then sell it as new. And many fell for it. In effect, they were reminding themselves with their own non-compliance with the one rule that existed. A cynic may wonder why this time around will be different?

The issue that remains, of course, is what it means to be sold? If 100 pairs are made available between now and Tokyo, but sold out in three days, and not replaced for months, does that constitute “sold”. If they sell 50 pairs direct to consumer, and no more, is that reasonable?

Here too, an example of how they might have done it differently – just limit the stack height to say, 20mm, and then let people innovate. Who cares if they innovate with prototypes into a space that is too small to achieve meaningful performance gains? Remember, the whole point about stack height is that it provides the “scaffold” for the addition of plates and other gadgets, the subsequent curvature of the plate to create propulsion and energy advantages (briefly, curving the plate means no loss of energy at the MTP, but also no increase in energy cost at the ankle, as happens if the plate is straight), and also provides a thickness of cushion that aids propulsion.

Had they made the limit 20mm, or the equivalent of lightweight racers for the last three decades, these issues would be greatly diminished, to the point, I’d suggest, where the difference made by tech is smaller than the difference between runners. Ergo, as per Ross’ rule, there’d be no problem.

[ribbon toplink=true]Problem 5: The process, and a better way[/ribbon]

Next, the process. Let me offer the way I would have done it.

I’d have formed a committee of internal World Athletics people and consultants, because after all, they must be accountable for the outcome. It would consist of technical staff, legal staff, athletes (not linked in any way at all to Nike), and physiologists/biomechanists. Not World Athletics employees, per se, but people whose names they want associated with an important policy or regulation.

Then, given the huge potential for conflicts, knowing how pervasive Nike’s funding of the sport has been for decades, the disclosure of who these people are and what links they have, is a prerequisite for the process to have credibility. That’s not to say that commercial links automatically kill the process, but it needs to be stated. Conflicts of interest can be insidious, indirect, distant even, but just put them out there, and accept that some will dismiss anything, but at least you’ve been transparent.

Next, and here’s the way to transparency, make the process fully accessible to everyone. Not before it happens, because otherwise you run the risk of poisoning the well before you begin. But I’ll tell you what I would have done:

I’d have identified five or six domains that would impact on the viability and validity of any rule. They would be:

• Physiology – how much advantage do the shoes provide, and what does it mean for performance?
• Biomechanics – mechanisms, engineering principles, future iterations, predictions etc
• Legal – patent and commercial law aspects
• Commercial – implications for the business model of sport, including sponsorships of athletes and events, and contract issues related to commercial agreements
• Ethics – what are the ethics of accepting tech, including elements such as fairness, cost barriers, access etc?
• Athletes – the roleplayer who is ultimately affected

Then, what I’d do is invite global input. Form the committee or policy group and then invite the whole world to submit their proposals and insights in those domains. Ask the experts around the globe to submit brief (500 words, maybe) proposals describing the key issues and implications within their domain. Collect and store them all.

Based on these submissions, identify three or four experts in each domain, making sure that you always have a pro vs con, a person arguing for and against the rule scenarios that you know exist. Invite these people to a two or three day meeting at which you hear them present, in detail, their arguments. Have discussions after each presentation, so that your legal experts can comment on the biomechanics, and the biomechanists can add their insights to the commercial issues or physiology considerations. Record everything. Heck, livestream the thing – what have you got to hide, right? It’s not a super sensitivity issue, and even if things come up like some company’s declining sales, you can let those presenters request a “blackout” for that session.

These people could even be industry insiders, like Nike and adidas and asics, who get to lobby briefly for their position, and athletes from both sides. Others must be outsiders, no skin in the game. Objective people who call it as it is, not as they need it to be. The key is that you get people representing both sides of the issue, three to four per domain, and then hear their full arguments.

Once completed, every single submission is made public for the whole world to see. If a physiologist has explained key concerns and principles of performance, and the ethics experts have backed that up with their ethical POV, then let the world see this, and evaluate it themselves. Make this available, either as videos of the presentations and discussions, or written submissions, so that there is maximal transparency.

Then make your decision. Transparent, open, consultative, evidence-based, and considering all factors. Earth-shattering stuff, no?

But no, this being sports governance, it’s happened in secret, with a summary given in the regulation, the usual vacuous nonsense written by a PR person and put into the mouth of the president. Plus, it reached a totally wrong position, but maybe that’s just me.

[ribbon toplink=true]The better solution[/ribbon]

I alluded to this above. In my opinion, the stack height limit at 40mm is way too high. Twice as high, perhaps. I would have gathered evidence on the stack heights of all the racing flats used to win marathons from about 1995 to the present day, and set the stack height limit at the median or mean of that group of shoes. This would probably land close to 20mm.

I believe this would be a significant first step in removing the advantage, because the 20mm ‘space’ afforded is not large enough to allow the ‘play’ we are seeing in shoes now. That space, as mentioned, is the scaffold in which innovation can occur, and it’s so large that clever engineers can do too much to enhance performance without the capacity to regulate. They can curve the plate, they can build up the foam cushioning, they can add the air-pods and other devices, some unknown. Cut that to 20mm, and the problem does not go away entirely, but it is constrained to the point where it becomes a lot more difficult to engineer an advantage that is greater than the difference between athletes.

Plus, it’s easier to regulate in the short term. Perhaps future innovation would overcome this limit, achieving the same gains seen in a 40mm space, because of new materials. But again, the scale of the improvement would be much more difficult to attain. And it could be addressed at the time, perhaps by assessing the components in that space somehow.

But 40mm? That’s what affords the companies the chance to go crazy like we’ll see with this new Alphafly Next%, and also creates the patent issues I mentioned above, the responder vs non-responder separation, and generally does not resolve the problem, but compounds it.

[ribbon toplink=true]In conclusion[/ribbon]

In conclusion, weak, vacuous leadership allowed a vacuum to develop. Nike made an amazing shoe to fill it, sneaking it into the Olympic Games in 2016 (and the US trials before that). At that point, authorities should have intervened, because for all the gaslighting going on in the media explaining away how they couldn’t act, that Nike just played by the rules, did what others had done, at that point, they could well have done something in response to a recognized issue.

By the time the research started to support the obvious, they could have acted again. But no, nothing. Not even the implementation of their own policy, despite being aware of the concerns of athletes like the courageous Kara Goucher.

So we saw Next%, then phase 3, and the sub-2 hour effect, an engineering venture disguised to look like a sports science breakthrough. Now we have “action”. But, in the words of Sebastian Coe, “we don’t believe we can rule out shoes that have been generally available for a considerable period of time”. And so they didn’t.

What Coe conveniently omits is that the only reason those shoes were “available for a considerable period of time” is because you didn’t act when you could have, and should have. You allowed a problem to develop under your noses, and then, pressed into action, did nothing, but pretend that you did, by solving a future problem while leaving the existing one alone. You played a role, possibly more active than will ever be admitted, in the creation of the problem, and now want to play it as though you’ve only just arrived to courageously fix the problem in a diplomatic way, respecting the ‘complexity of the situation and the market’. You don’t get to play that card when you’re responsible for the market in the first place, through inaction borne of who-knows-what or a screwed up moral compass. It’s absurd.

Not only that, but they added a mass of uncertainty to the already non-existent integrity of the outcome, because the policy doesn’t do anything like enough to ensure that tech doesn’t have a bigger effect than physiology on the outcome. So now there can be neither confidence in the process, nor the result. It’s a step backwards, plus it was done in a way that looks more like the regs fit the market, and not the market to fit the regulations.

All in all, it’s sad for running. Among the great appeals of this sport were its simplicity, its accessibility, and the opportunity it gave to celebrate its history, records, breakthroughs. Even PBs outside the spotlight, to the slower mortals among us had meaning because they meant that “I have improved”, not “I am the same runner with a 2% faster time thanks to my purchase last week“.

And while on this subject of breakthroughs and records, please let’s not say stupid things like “We should just run barefoot”, or the common gaslighting method used by many that innovation has always been part of the sport, and the 2016 shoes were way better than Jim Peters and Roger Bannister’s shoes in the 1950s. Yes, but if you’re daft enough to actually entertain the debate about whether Kipchoge is a better runner than Jim Peters, great, but find someone equally daft who’ll have that one. I’d need ten beers for it to even seem worthwhile. Nobody cares. We get it, we expect innovation over many generations.

But what we don’t want is the sudden change of performance at a scale that renders ‘within race’ and ‘between immediate generations’ comparisons utterly invalid. I think it’s a sad loss for the sport, for example, that we cannot truly evaluate Kipchoge in the context of his own peers, many of whom did not have access to the shoes he will. That 2:01:39 of his is impossible to assess relative to the generation immediately before him, and even to some with him. We know he’s the greatest because of the wins, going undefeated since his debut. But even there, might some wins have been losses without the advantage?

Then Bekele got with 2 seconds of him, but he had the Next%, Kipchoge only the 4%. That too is a comparison we can’t make. What a pity, the loss of ability to actually interpret performance because the gains were too large, too fast, too unevenly distributed, thanks to non-existent regulation.

Running was a sport where a limit was set at physiology, not revolutionary equipment. Incremental gains in performance happened thanks to equipment, certainly, but it was spread equally and small enough that it was not decisive. It’s different now, and the the distortion of “input vs output”, the resultant loss of integrity, and the necessary recalibration hurts the sport massively, in my opinion.

The end. Until the marathon goes under 2:01 and then 2:00, which is going to happen really soon. Not to mention track records. So on we go. A bad taste with every great propelled performance.

Ross

Disclaimer/conflict of interest statement: Since 2009, adidas SA has sponsored a local road race called Two Oceans, and I wrote training programmes for the public for that event on their behalf. I also provided running training and physiology advice to clubs and at workshops sponsored by adidas SA. I had no involvement with the global parent company on shoe development or sales, and at my request, a contract clause that stipulated that I did not comment on or promote their shoes. If that leads you to dismiss the above arguments, that’s your prerogative, though I’d argue that the criticism here is not of Nike, who in fact are acknowledged for making an amazing shoe, but of the policy that allowed it. Your call on interpretation.

Last week, I posted a lengthy but detailed article explaining how the current initiatives to sanction high tackles in rugby arose. It was an article that described a process, start to finish, that led experts to advise that head injury risk for both players might be reduced if high tackle sanctions were more frequently and consistently applied. In it, I said I’d use the recent U20 World Champs as a case study to further explain why some actions and behaviours are more risky or likely to cause head injuries than others.

This is that piece! So if you’re interested, do read on and hopefully it clears up the issue around risk or propensity, and why simply counting and “believing your eyes” might not be the best way to understand risk!

[ribbon toplink=true]Background – the rationale[/ribbon]

First, the rationale, which is really important and thus worth re-iterating:

The strategy to harshly sanction dangerous high tackles is based on evidence gathered by analyzing when head injuries occur, and then recognizing that there are certain factors or behaviours that increase the risk.  Once those were identified, World Rugby consulted with expert groups that included players and coaches, and they proposed and refined the initiatives that are now in place, including the Sanction Framework, whose intention it is to:

• Improve consistency in decision making
• Act as a disincentive to what is recognized as the more high-risk tackle situation, namely one where the tackler is upright, and is making higher contact with the ball carrier, such that his head is in close proximity to the ball carrier’s head or shoulder.

What World Rugby is NOT trying to do is dictate what height the tackle should be (if they wished to do this, they would have lowered the height of a legal tackle to the armpit, sternum or even the waist.  This is currently being explored in two trials – one in Stellenbosch for armpit height, another in a few competitions for waist height, after a French Rugby proposal to trial this. Once those trials are completed, then based on evidence, this situation can be revisited).

Instead, what World Rugby ARE trying to do is to get players and coaches to take ownership of tackle technique and execution to reduce the risk.  Effectively, the mandate is to make adjustments in technique in order to reduce the risk.

The specific adjustments that should be made are up to the coaches and players, for they know best, but the ultimate intention is to reduce head contacts to both the tackler and the ball carrier in the sport.  To appreciate this, we can begin, conceptually, by asking when the dangerous head contacts happen?  And we know that the answer is:

1. For the ball carrier, head injuries can only really happen when their head is struck by the opponent. This happens more often in illegal high tackles
2. For the tackler, there is always a risk of head contact, but it is HIGHER when the tackler is upright, and when the tackler’s head is in close proximity to the ball carrier’s head or shoulder. You can read all about the detail behind these points in the article I wrote last week

What the sanction framework is doing is disincentivizing the highest risk outcome of the tackle – head contact to either player, and in particular higher risk contact to the tackler’s head.  How the coach achieves this is very much up to them, but sanctions are intended to draw attention to the risk, and to compel those in the sport to act for the protection of both players – tackler and ball carrier.

With that in mind, below is a brief analysis of the U20 championship head injury set.  This uses identical methods to the analysis that was undertaken on three years of global rugby to arrive at the initiatives that were proposed by the various expert working groups.

[ribbon toplink=true]The process at U20s[/ribbon]

In order to understand risk of head injuries, you need to know two things:

• How many injury events occurred?
• How many total events occurred?

The latter is basically asking “How many could there have been?”. This allows you to work out the relative likelihood of a given event to cause an injury.  In effect, you’re able to to identify “Per 1000 events of each type, how many injuries happened?”

So, we begin with the head injury cases.  World Rugby has access to the full medical record of the tournament in CSx, and can thus determine the precise number of HIA 1 entries – these are players removed after a head impact, and then they are either cleared of concusison, or diagnosed as concussed as part of the entire HIA process. The table below presents all the tournament’s HIA 1s, in random order:

In total, there were 24 HIA 1 cases documented in the CSx system.  Of these, 20 occurred in tackles, two in open play (players diving onto a loose ball), one at a ruck, and one occurred where no video footage exists of the injury.

Note that these proportions – 20 out of 24 HIAs in tackles (83%) – are consistent with our global study, where 75% of head injuries happened in tackles.

Next, we can analyze the 20 cases that happened during tackles, and determine various numbers of events with known risk factors.  In the table above, we have identified:

• Which player was injured in the tackle?
• What was the tackler’s body position in the tackle?
• What was the ball carrier’s body position in the tackle?
• Where was the head contact with the opponent?
• Was the tackle deemed foul play?

The summary of these variables is shown in the following tables:

First, we can look at an overall summary, which then sub-divides HIAs into those occurring to the tackler and those to the ball carrier.

Here we see that of the 20 tackle HIAs:

• 15 were to the tackler (75%)
• 5 were to the ball carrier (25%)

Note that this too is the same as the finding from the global study, where the split was 72% to tacklers and 28% to ball carriers.

When we look at HIAs to the tackler, we see:

• Of the 15, 4 occurred when the tackler was upright – 27% of the total
• 10 happened to a bent at the waist tackler – 67% of the total
• 1 happened when a tackler was diving – 7% of the total

When we look at HIAs to the ball carrier, we see:

• 2 of the 5 happened when the tackler was upright – 40%
• 3 of the 5 happened when the tackler was bent at the waist – 60%

In order to further explore the risk of HIAs when tacklers are either upright or bent, we can combine the above analysis and produce the tables below:

Here we see that six HIAs happened when tacklers were upright (4 to the tackler and 2 to the ball carrier), and that 13 HIAs happened when tacklers were bent at the waist.

[ribbon toplink=true]The risk issue – counting and believing your eyes is not enough[/ribbon]

At this point, one might look at the following statistics:

• Six HIAs happen when tacklers are upright
• 13 HIAs happen when tacklers are bent at the waist

and conclude that bent at the waist tacklers are more dangerous, because more HIAs happen in this scenario.

However, this would be incorrect, because it would not be factoring in the important concept that you can only know the risk of something if you know how many total events of that thing occurred.

For instance, we know that many more people die in car accidents than motorbike accidents.  But this does not mean that cars are more dangerous than motorbikes, because we also know that many, many more people drive cars than bikes.  And so when we correct for the total number of cars, then we would realize that cars are actually far less likely to kill than bikes, and so if we were asked to recommend what people use for safer transport, we would suggest cars.

So what we should be doing is understanding that in risk management, if you cannot totally remove the risk event (tackling), then you have to ask one simple question:

Of two options, which is LEAST DANGEROUS?  In the case of tackles, we are asking:  Is it more dangerous to tackle upright/higher, or is it more dangerous to tackle bent/lower?

In order to answer this, we must know:

• How many times is a tackler bent at the waist?
• How many times is a tackler upright?

World Rugby does have these numbers, because its analysts have been coding them ever since the height of a tackle and body position were recognized to be risk factors.

In the U20 tournament, we know that in the 30 matches, there were:

• A total of 5820 tackles (average 194 completed tackle events per match)
• A total of 1185 UPRIGHT TACKLES (average 39.5 per match)
• A total of 4074 BENT AT THE WAIST tackles (70% of all tackles)

Now, we can use these numbers to actually work out how likely a head injury is when a tackler is upright or bent:

So, we can see from the table above that a tackle where the tackler is upright is actually more likely to result in a head injury than a tackle where the ball carrier is bent at the waist.

In the U20 Championship, an upright tackle caused a head injury every 197.5 situations.  A bent at the waist tackle caused a head injury every 313 events.

The conclusion is that the relative risk of being upright is 1.59 times higher than being bent at the waist.

Note that in the global 3-year analysis from 2013 to 2015, we found a very similar number – it was 1.44 times more dangerous for the tackler to be upright than bent at the waist.  The U20 championships consolidates that finding, though I can’t stress enough that it’s a small cohort, but I use it to illustrate the point.

Put differently, what we are seeing here is the following:

• There ARE more head injuries when a tackler is bent at the waist than when the tackler is upright (just like there are more car-related deaths than motorbike deaths).  In fact, we can quantify this for the case tournament – 13 Bent HIAs and 6 Upright HIAs, a ratio of 2.2 to 1
• However, there are MUCH more tackle events where the tackler is bent at the waist.  This ratio is 3.4:1, because we had 4074 bent tacklers and 1185 upright tacklers.
• Basically, upright tacklers are like motor bikes, and bent tacklers are like cars
• As a result of this combination, we can work out that one of the players involved in a tackle is 59% more likely to experience a head injury when the tackler is upright than when they’re bent at the waist

Note that this does not distinguish between whether the head injury would be to the tackler or the ball carrier when the tackler is upright.  This can be done (if you’re interested, see Appendix A at the end of this article). 

However, for the global purposes of risk management, it is the total risk of injury that is important, and it is clear from the above that risk is reduced for a bent tackler compared to an upright tackler.

Therefore, returning to this important question:

Of two options, which is LEAST DANGEROUS?  In the case of tackles, we are asking:  Is it more dangerous to tackle upright/higher, or is it more dangerous to tackle bent/lower?

The answer is that it is relatively safer to tackle bent at the waist, even though you will see more cases from this action.  Bent at the waist is the car. Upright is the motorbike. And so given that you are going to swap one behaviour for another, the number of incidents does not matter, only the risk.

Note again, and I emphasize this in closing, the main priority here is not to force the player to go lower by making it obligatory. 

There remains a choice, and it is clear that some low tackles are more dangerous than they may have been if the tackler remained upright and went higher.  The specific situation will influence the risk, but when one steps back from the detail, we see that it is upright tackles that currently pose a higher risk to players.  Therefore, you can reasonably conclude that if technique is to blame, it is the technique of upright tackles that is inferior than bent at the waist tackles.  Or you could suggest that many upright tackles will be safer if they are lowered in height.

The message is for coaches to prioritize safe technique that avoids contact to the head of either player, irrespective of whether they’re in upright tackles or not, because as the very first table suggests, when the tackler is upright, the danger exists from a head to head or head to shoulder impact.

Thus, the priority in terms of player welfare is to identify and then execute the technique that is LEAST LIKELY to result in head contact for either player.

If this means a lower tackle, then that is a choice that the coach and player may make.  If it means a higher tackle, then the same is true. The point is avoidance of head, through better technique, regardless of method. But if method changes that risk, then it must be factored in.

And high tackle sanctions are simply intended to draw attention to the intention.

Ross

[ribbon toplink=true]Appendix – more detail on interaction of behaviour and risk[/ribbon]

For those interested in a second-level of analysis, below is a brief summary of the relative risk to the players involved in the tackle, and how their body positions combine to create risk.

What this shows is the following:

• When the tackler is upright, the risk to the tackler is twice as great as to the ball carrier.  This is in agreement with our previous global study result
• When the tackler is bent at the waist, the risk to the tackler is 2.45 times greater than to the ball carrier
• However, when we correct for the total number of cases, we see that:
  • The greatest risk exists for an upright tackler, to himself
  • Next largest risk is for a tackler who is bent at the waist, to himself
  • Third is for an upright tackler, but the BC is injured
  • The lowest risk exists for a ball carrier when the tackler is bent.  This is obvious because a bent tackler has almost no chance of striking the head of a ball carrier, unless the ball carrier also bends into contact.  This does happen, as the data show, but is very rare overall – 0.74 Head injuries per 1000 such instances, a risk of 1 in 1358.
• When combined, as per the previous tables in the main analysis, the risk of a head injury when the tackler is upright is 1.59 times greater than when the tackler is bent.  

Next, let’s consider how the risk changes for different combinations of tackler and ball carrier body position.

Unfortunately, for the U20 Championships, this analysis is basically ineffective, because there are too few cases of HIAs for comparisons. This is a table showing HIA 1s for different tackler and ball carrier positions:

So you have half the combinations with zero cases, and thus no meaningful calculations. We also don’t have a control or denominator for these combinations, and as I explained above, that is non-negotiable if you’re trying to understand risk. The 6 and 13 HIA 1s for upright and bent tacklers, respectively, is what you saw in the main part of this article, and is used to show that an upright tackler has higher risk per 1000 such events than a bent tackler.

Given the thin data from this short tournament, we have to go back to the big study for the data on this. The table below shows the data. It is a little busy (apologies), but here you can see the risk per 1000 tackles for every combination of tackler and ball carrier body position.  I have highlighted the key cells with light red and green shading, and will describe them below:

So, for “Tackler Upright”:

• When the ball carrier is also upright, we had 131 HIAs, with a risk of 2.8 HIAs per 1000 such tackles.
• If the ball carrier is bent at the waist, the HIA risk drops to 2.0 HIAs per 1000 tackles.  Also note that this is quite a rare event, and happens only 9.3 times per match on average

For “Tackler bent at the waist”:

• When the ball carrier is upright, there are 133 HIAs, with an overall risk of 2.6 HIAs per 1000 such tackles
• When the ball carrier is also bent at the waist, there are 83 HIAs, with a risk of 1.1 HIAs per 1000 tackles.  Note that this is the most common situation of the four looked at here – 48 times per match.

The conclusions from this perspective of analysis is that irrespective of the tackler’s body position – upright or bent – the risk is lower when the ball carrier is bent.

The same is true in the other direction, though, and we can show that irrespective of the ball carrier’s body position, the risk is lower when the tackler is bent at the waist.  This is perhaps even more important a concept.  

Look at the comparison between the Columns labeled A – these two columns compare the risk when the ball carrier is upright.  You can see that if the ball carrier is upright, you have a risk of 2.8 if the tackler is upright (A1) and 2.6 if the tackler is bent at the waist (A2).  Thus, for an upright ball carrier, we generally would want the tackler to be bent at the waist where the relative risk is lower.

Now look at Columns labeled B – if the ball carrier is bent at the waist, the risk is either 2 HIAs per 1000 tackles (B1) for an upright tackler, or it is 1.1 HIAs per 1000 tackles for a bent at the waist tackler (B2). 

So here again, the risk is lower when the tackler is bent at the waist.

Point is, and this was in fact the key conclusion of a scientific paper on the interaction between the players body positions and risk, irrespective of the ball carrier body position, the risk is higher when the tackler is upright, and lower when the tackler is bent at the waist.

Of course, there is an interaction between them.

So we can rank the risk from highest to lowest:

1. Greatest risk – both players upright
2. Tackler bent, ball carrier upright
3. Tackler upright, ball carrier bent
4. Lowest risk – Both players bent

Why a red card? The tackle shown to the right saw New Zealand’s Scott Barrett sent off this past weekend against Australia, and it has triggered substantial discussion. The red card was the result of the application of a new World Rugby High Tackle Sanction Framework by referees, which is aimed at a) increasing sanctions for dangerous play to reduce head injury risk, and b) improving consistency in decision-making.

That framework is going to be a magnet for discussion and criticism in coming months, particularly under the bright spotlight of the Rugby World Cup. But its development wasn’t an overnight ‘guess’, and certainly wasn’t arbitrary. It was the result of years of research and discussion that then triggered focussed interventions with easily a hundred people involved in the sport in capacities ranging from playing and coaching, to competition organizing.

It began in late 2016 with a “zero tolerance directive” after a multidisciplinary group met to discuss head injury risk based on video analysis of head injuries. That was launched in January 2017. Then came numerous iterative discussions identifying the need for a framework to support those high tackle decisions, which then went back and forth within a multidisciplinary Group including Match Officials, medics, judiciary, citing etc. It was then discussed and modified at a dedicated Player Welfare safety meeting in France, and again during the following approval steps: Law Review Group; Rugby Committee, Council and EXCO.

It is a journey whose background is worth knowing about, rather than simply saying “Game’s gone soft” and accusing the referee of “hiding behind the laws”.

[ribbon toplink=true]The overall purpose[/ribbon]

The bottom line, or top line, if you wish to stop reading here, is to reduce the number of times high risk head impact occurs in matches. That means head impact to the ball carrier (the obvious) and the more risky head impact to the tackler (less obvious).

And the theory is that harsher sanctions for high tackles are meant to act as a “stick” that achieves one of two things (or both):

1. Lower the height of tacklers and the associated body position of tacklers into what is a safer, lower risk position for their heads (based on data, explained below)
2. Change the technique or technical execution of higher, upright tackles so that head contact is avoided or reduced

Both of those are means to an end. The path taken to reach that “end” is up to the coaches and players, and whether they choose to adjust to different tackle types and heights (option 1), or to focus on technique in the higher risk tackles (option 2). But the premise is that sanctions – penalties and cards – are the message to trigger those necessary adjustments.

There’s a lot behind that message, and so this is a post in which I want to try to explain the data side of the process of risk reduction, how the risk factors were identified, why certain decisions were made, who made them, and what the intention of zero tolerance on head contact actually is?

There are a few important principles that underpin the risk reduction approach, and understanding those is key. The most common rejection of the high tackle sanction approach involves something like the following: “Given that 70% of head injuries happen to the tackler, this approach of giving out cards is not going to help anyway, it’s focused only on the ball carrier, and the tackler is being neglected. Asking the tackler go lower will increase their risk. You’ll hear coaches and pundits say this, maybe even some in the media.

In this article, I want to try to explain to why this is untrue. So bear with me, and I will try to describe the method behind what you may see as madness. I apologize if this gets lengthy, but please, if you are interested, read it and share it widely. Point by point, here we go.

[ribbon toplink=true]The background: What is risky?[/ribbon]

1. Concussion is the most common injury in professional rugby. It probably has been for a while, but since about 2011, the combination of increased awareness (some of which came across the Atlantic from the NFL), better tools and a lower diagnostic threshold for detecting and diagnosing concussions have led to more of them being identified. As you read this, the global concussion incidence is about 16 to 18 per 1000 hours of rugby. That works out to about 2 concussions every 3 matches. Given the uncertainty over the future prognosis and health of people with multiple concussions, this is a good reason to ask not only whether they’re detected and managed, but “how can concussion be prevented?

2. You can’t prevent something if you don’t know how it happens. So step 1 in prevention is to develop a good understanding of risk. This means knowing:

“Who is likely to have a head injury?”
“In which event or phase of the game is the injury likely to occur?”
“What factors or circumstances around that phase or event increase the risk?”

3. Between 2014 and 2016, 611 head injury assessment events in global professional rugby were analyzed to try to answer those questions. That research study found that the majority of them – 464 or 76% – occurred in the tackle. No surprise there. Next highest was rucks, 73 out of 611 (12%). Also no surprise. The tackle, by virtue of being the most frequent match event, with the highest risk( that is, number of injuries per 100 tackles), became the focus of the next phase of analysis.

4. Slightly more surprising, of head injuries that happen in the tackle, 72% occur to the TACKLER, and 28% to the ball carrier. So the tackler has a risk that is 2.6 times higher than the ball carrier. You can read more about this in a thread I wrote towards the end of last year. This finding was somewhat surprising – most injuries show a 50/50 split. It is also the source of “misinformation” and misunderstanding, so let’s look at this a little more closely.

5. If you were told that the tackler is almost three times more likely to be injured than the ball carrier, what would you do? How do you protect a player from themselves? How do you reduce the risk when the risk is created by the player who initiates the risk event? I hope that you can see the paradox here, the challenge that this creates for a law-maker, because typically, the law protects the recipient of an action (“Don’t shoot someone, they’ll get hurt”) rather than the initiator of the action!

6. With that in mind, I’ll try to explain how the various groups went about addressing this paradox. The first requirement is that you have to understand why and when the tackler has an increased risk – it comes down to mechanism of injury. There’s no way around this. You’ve got to ask “What actions or behaviours expose the tackler to risk?”. This allows you to create a spectrum of risk, like the one shown below, and then work with specific experts to figure out how to move behaviour from right, in the red zone, to left, in the green zone.

7. So the key question is this: “What attributes or characteristic of a tackle tend to lie in the red zone, on the right hand side of that spectrum, and what lies in the green zone, on the left side?”.

8. The reason this matters is because when you’re trying to reduce risk, you have to know whether you’re going to eliminate the risky stuff (the red), or are you going to substitute high risk for low risk? There’s a crucial difference between the two approaches. If you’re going to eliminate risk, it’s easy in theory – you simply ban the stuff that lies in the red zone, and it’s done with. For instance, you could make it illegal to drive any car after 8pm, and you would guarantee a huge reduction in the number of drink-driving cases. It would also be wildly infeasible, but in a theoretical world, such a ban could work. However, if you can’t eliminate something completely, then you’re probably going to have to substitute it for something else (for instance, take the bus or an Uber after a night out).

To bring this analogy to rugby, if there’s a certain type of tackle that has high risk, can you ban it completely? Or are you more likely to try to swap it for a tackle that is lower in risk? To do this, you have to first know what’s likely to be swapped with what.

That is, what is your A (high risk tackle/driving after drinking) and what is your B (lower risk tackle/taking the bus)? Once you know that, you can ask how you might shift players from A to B? From red to green. This is where your “levers” come in, and that’s where this process was ultimately headed from day one. The data collection began knowing that eventually, we’d need to ask coaches, players and referees how to swap high risk for low risk behaviours.

9. So, below is what that risk spectrum looks like for 464 tackle head injury events (that needed HIAs). In a follow up post to this, I will use the U20 Championships to explain exactly how risk is worked out, and walk step by step through some examples, so check in later for that, but basically, this is a spectrum that was generated by analyzing the 464 HIAs from tackles, and another 3000 tackles that did NOT cause a head injury. By comparing injuries to non-injuries, you can work out what circumstances are more likely and less likely to cause a head injury. This is called propensity, and it is calculated as number of injuries per 1000 of that kind of event.

10. Ok, so we’re making some progress. We see some obvious things in the red zone – higher speed increases risk, accelerating is more dangerous than not accelerating, front on tackles have more risk than side on or tackles from behind. Most people will have anticipated these differences.

Less obvious or intuitive is that “higher tackles” are more likely to injure players than lower tackles, and that “Upright players” create more risk than “bent at the waist players”. This applies to both the tackler and the ball carrier, take note, so both the tackler and ball carrier have lower risk if they’re bent or upright. These are crucial points, so let’s look at them more closely.

11. A “higher tackle” is a legal tackle, but one where the head of the tackler is at or above the shoulder of the ball carrier. In other words, contact is at the ball carrier’s sternum or higher, and the tackler and ball carrier’s heads are sharing ‘air space’.

A lower tackle is one where the tackler is making contact on the lower trunk or torso, or the legs of the ball carrier. In this scenario, the tackler’s head is in proximity with the lower trunk, hips, upper leg, knee or feet of the ball carrier. Imagine a line across the chest at the level of the sternum, or under the armpit – that’s the line separating high from low tackles in this analysis.

12. Linked to this is “upright players”. This has a clear definition for the tackler – it is a tackler who presents the front of their chest to the ball carrier prior to contact, as opposed to the top of their shoulder. Or, if unclear, the tackler’s feet are directly beneath their shoulders and their spine is vertical at impact. It stands to reason that typically, an upright tackler’s head will be near or close to the head or shoulder of the ball carrier (unless the ball carrier is bent – see later). And so “upright tacklers” and “higher tackles” (see point 11) are related, to some degree. One tends to cause the other.

What the research found was that upright tacklers were 50% more likely to be injured than bent at the waist tacklers, and that this was true irrespective of the ball carrier’s position. It is rugby’s prisoner’s dilemma – it doesn’t matter whether the ball carrier is upright or bent, the greater risk comes when the tackler is upright.

13. This is key – a tackler who is upright is MORE LIKELY to have a head injury than one who is bent, and at a global level, ball carrier position doesn’t affect this relative risk. Why? Because it is more likely that the upright tackler will have a head to head or head to shoulder contact, which is a “higher tackle” contact and more dangerous. Below are the specific numbers relating these risks to one another for tacklers.

You’re looking here at the PROPENSITY spectrum for each situation to cause a head injury to the tackler. Propensity is risk – it is the number of head injuries per 1000 such tackle situations. Alternatively, you can think of it as “how many such tackles happen before a head injury to the tackler?”, which I have also shown in this figure (the 1 in 89, for example).

14. So, what we have is head to head contact causing a tackler head injury 11.3 times per 1000 such tackles. Or, once every 89 situations where heads are in proximity. Look at head to hip contact – that injures the tackler 1.7 times per 1000 such tackles, or 1 in 581. In other words, a head to head impact is more dangerous (6.5 times, to be precise) than a head to hip impact. (Just a note on the actual numbers – these data come from a time where the HIA number was an underestimate, due to lack of adherence with the ‘new’ process. The numbers in the current game will be higher, across the board, because the HIA number in 2018 is about 30% than it was from 2013 to 2015)

15. When we group tackles by those where the contact is above the blue line (higher tackles) and those below the blue line (lower tackles), we get an overall picture that says the following: “Higher contact tackles, where the tackler’s head is above the sternum, near the shoulder or head of the ball carrier, are 4.25 times more likely to cause injury than lower tackles, where the tackler’s head is below that line”. The risk is 3.8 HIAs per 1000 high contact tackles, and 0.9 HIAs per 1000 low contact tackles.

This is why “Higher tackles” are in the red zone of the spectrum, and lower tackles are in the green zone of the risk spectrum. Note also that Upright tackles are 1.5 times more likely to cause a head injury than bent at the waist tackles.

16. Yes, head to knee impact has high risk. A head injury occurs once every 323 tackles at this very low tackle height (3.1 HIAs per 1000 situations). That’s almost twice the average risk, so of course there’s reason to worry about these – you wouldn’t want players diving in recklessly, eyes down, head first, because a) the head injury risk is quite high, and b) it increases risk to the ball carrier’s knees.

But note that the head injury risk for the tackler is still lower here than head to head risk. So given a choice, where it’s only about the tackler’s head, which would you advocate? (a reductionist question, I know, but to emphasize a point). Head to hip is even lower still. Where you really want the tackler’s contact, especially for those high risk active shoulder front-on tackles, is head to upper body or trunk (as future evidence would confirm).

17. But when you watch rugby, you will see loads of head to hip head concussions. They stand out, right? More than any other kind (the data support you on this). Perhaps you’ll conclude that head to hip, or head to knee is most dangerous because that’s what you see more of, as an absolute number.

But this common approach to the issue overlooks the key principle of reducing risk, namely that it’s not about the number, but the likelihood of injury. Why? Because your approach here is not to eliminate risk, but rather to swap A for B, and that’s a risk problem, not a numbers one.

Here’s an analogy: Imagine you counted up all the people who die in car accidents every year, as well as those who died in motor bike accidents (sorry for the morbid analogy). You’d count, for argument’s sake, 1000 car deaths, and 40 motor bike deaths. Would you then conclude that cars are 25 times more dangerous than bikes, and thus suggest that everyone should rather ride motor bikes as part of your overall road safety initative?

Of course not! Why? Because you haven’t accounted for EXPOSURE to risk. And you can’t make any kind of “risk swap” decision unless you know how many people were driving cars (and how far they drive) compared to motor bikes. You have to know this total in order to work out the likelihood of the event, and only then can you suggest that one risk be swapped for another.

Imagine for instance that there were ten million kilometers of car journeys, and 100,000 kilometers of motor bike journeys that year (people are 100 times more likely to travel by car in my analogy). Suddenly, the numbers take on a new meaning. The risk from cars would be 100 deaths per million car kilometers, and 400 deaths per million motor bike kilometers. Motor bikes are four times more dangerous than cars. And so your strategy to reduce OVERALL deaths would be to advise that people replace motorbike journeys with car journey – if they’re going to travel, choose the one less likely to cause the negative outcome per exposure, or per kilometer (traveling on buses or trains is likely safest of all, by the way).

18. My point is this – when people say that they see more concussions from head to hip or head to knee, they may be mistaking volume for risk. Unless one knows how many COULD have happened, one can’t properly assess risk. This is why you have to look at exposure to get the denominator, and that’s what produces the risk spectrum.

19. And so we come back to the absolutely crucial point about risk reduction and the “swapping” of behaviour that I explained in Point 8 above. Remember, we have to identify what has relatively high risk (A) and what has lower risk (B), because we’re going to swap A for B.

So the question that matters is this:

Of two options, which is LESS DANGEROUS to the tackler?  Is it to have the head in close proximity to the ball carrier’s shoulder and head, or is it to have the head in proximity to the hip, upper leg and knee of the ball carrier? The spectrum and propensity answers this:

20. Now, let’s return to the question I asked YOU in point 5 above: How will you protect the tackler? What will you change in order to reduce the risk of a head injury to the player who initiates the event?

I hope that you’re thinking about it and recognizing that you’ve got to move from right to left, from red to green on the risk spectrum. And given what you’ve seen about what is risky, I hope that you’ve realized that if you can reduce the number of head-to-head impacts in the sport, then you’re going to reduce overall risk. If you can get the players heads out of one another’s airspace, then concussions should fall, compared to when heads are near hips, and even knees.

And it doesn’t matter if you replace head-to-head with head-to-knee or head-to-hip impacts, overall risk should fall. Your best option is to swap those bad head-to-head contacts for head-to-upper body and upper leg contacts (though there’s an interaction effect of different risk factors in player here, but I won’t go into that now)

21. Linked to this, if you can get the tackler into a bent position more often, where those higher head contacts are less likely, then provided you do so at the right time, the risk is going to be lower. What does “at the right time mean”? It means reacting to the dynamic situation created by the ball carrier’s movement because there are some situations where the tackler can avoid head contact by staying higher or upright.

So this is not a black and white, hard and fast “rule” saying that lower is always better (though some frame it that way). Rather, this is about priority 1 – avoid head contact for both players. Generally, that’s less likely to happen for a bent tackler, but not always. There is a recognized complex interplay, but the big picture is quite clear. It’s not about saying “You must go lower”, it’s about saying “go where you have less chance of dangerous head contact”.

22. Taken together, what we have is a picture that says the following:

Priority 1 – avoid head impacts for both players, especially from higher contact. In particular, avoid a tackler’s head striking a ball carrier’s head, or being above the ‘blue line’ for higher contact. Second priority – avoid head to knee impact for the tackler. But if it’s one or the other, lower is better, higher is worse. Not from illegally high tackles per se, but from head to head and head to shoulder collisions, where there is “higher contact”. Of course, any head contact is worse than none, but this is clearly the highest risk one.

And how do you avoid head impacts? Two ways – one is to improve technique when tackling higher, the other is to tackle lower, rather than higher. All that matters is that you create swap the relatively high risk head impacts to the tackler with the relatively low risk impacts.

We have now identified the desired “destination”.

23. That destination is to avoid head impacts for both players. It should be obvious that the risk to the ball carrier will be much lower if this destination is reached – the only way a ball carrier can be concussed from a low tackle is if their head hits the ground or if they suffer whiplash. These do happen, but very rarely (under 9% of total). So if you can get the tackler away from the ball carrier’s head, then it is obvious that head injury risk to the ball carrier is going to drop dramatically.

But, we’re interested in the tackler, who is injured 2.6 times more often. And really, this is the only question that needs to be asked:

Of two options, which is LESS DANGEROUS for the tackler?   Is it more dangerous to tackle upright/higher, or is it more dangerous to tackle bent/lower?

The spectrum above made it clear that in this study, globally, the LESS DANGEROUS option is to tackle lower, bent at the waist, or with technique that avoids the head of the ball carrier. And that’s the suggestion that was made by an expert working group consisting of coaches, referees, players and officials in late 2016.

[ribbon toplink=true]The data ends, and the interventions begin – zero tolerance to head contact[/ribbon]

24. That expert working group was the point at which the science steps aside, and the practitioners take over. Science can describe the picture, but it can’t and shouldn’t suggest how to change it. So a group of expert players and coaches including Eddie Jones, Gus Pichot, Paul O’ Connell, Alain Rolland, Rachel Burford, and John Jeffries was convened, and spent two days in Dublin discussing all these risks. Their first conclusion and suggested action step was that risk could be reduced if behaviour was changed away from the “red zone” of high risk towards the “green zone” of lower risk.

25. How do you achieve that behaviour change? The recommendation made by that expert group was to more harshly punish illegal high tackles. They didn’t suggest only that – the issues of speed, acceleration, tackle direction and tackle type were all discussed, but those experts felt that the “low hanging fruit”, the approach that would be most readily accepted with the smallest possible ‘harm’ was to focus on the height and position of tacklers, as you can see in the summary below. The speed and acceleration issues were deferred for future discussions (and would result in, among other ideas, the 50/22 law trial, but that’s for another time)

Also notice that “Technique” sits atop everything – all the risk can be reduced with improved technique. They knew that, and spoke about that, but the question is “How do you compel people to address technique more?” Their suggestion was to more harshly sanction high risk tackles, using existing law to push behaviour from right to left on the spectrum.

26. And here’s where the paradox is hopefully resolved – illegal high tackles are not directly the cause of the head injury to tacklers, but they tend to happen when tacklers are executing actions that are higher in risk, namely upright tackles that involve higher contacts. These are the kind of tackle circumstances you want to avoid (the red zone of the spectrum), and so sanctioning the ‘extreme’ (illegal tackles) is an indirect way to get at the risky behaviours (legal, but high risk tackles).

27. So, given that there was already a law in place to sanction dangerous contact to the head of the ball carrier, the ‘lever’ suggested was to apply that law more strictly. The rationale from the expert group was that this would put responsibility on tacklers to avoid contact with the ball carrier’s head by targeting lower on the ball carrier’s body, thereby putting their own heads in a relatively safer position. This would reduce the risk of head injury to BOTH PLAYERS.

It was designed to compel coaches and players to say “OK, the consequences if I aim to tackle higher and get it wrong are going to be more severe, so I’d better reduce the risk of a yellow or red card by targeting lower on the ball carrier, perhaps at the upper trunk” (which is in our green zone, you may recall).

Alternatively, the player/coach could say “I’m still aiming higher, but I have to make sure I don’t strike the head of the ball carrier. It’s risk and reward, but let me work on my higher upright tackle technique until I can do it safely and effectively”.

Either scenario is positive because it avoids head contact for both players and that’s priority one.

28. And the crucial point is that in avoiding the head of the ball carrier, the tackler is also reducing their own risk of head injury. BOTH players benefit:

• Ball carriers will benefit directly because their heads are totally out of danger of direct impact
• Tacklers will benefit indirectly because their heads will be in the relatively safer position of being near the ball carrier’s torso/trunk, rather than the head and shoulder of a ball carrier. This is summarized below.

29. Based on that advice from the Expert Working Group, World Rugby announced a “zero tolerance” directive in January 2017, which identified a category for “Reckless tackles” and “accidental high tackles”.

[ribbon toplink=true]The global penalty and card spike, and a new set of challenges[/ribbon]

30. As you might expect, this saw an increase in the number of penalties and cards in the global game. For the next 12 months, compared to the 24 before, there were 64% more penalties for high tackles, and 41% more cards.

It used to be that you’d see a high tackle penalty every 125 minutes, it became one high tackle penalty every 76 minutes. That equated to 6.4 high tackles every 10 matches before, and 10.5 high tackle penalties every 10 matches after. One a game, basically.

As for cards, that went from 1 high tackle yellow card every 13 matches, to 1 high tackle yellow every 9 matches. And red cards used to be given once every 270 matches, and after the zero tolerance directive, it become one red card for a high tackle every 56 matches.

31. So, as expected, right? Except there were two concerns. The first was inconsistency. The change was not uniform across all the world’s competitions – some competitions saw high tackle penalties go up by 90%, but their yellow and red cards actually went DOWN! You were half as likely to be sent off for a high tackle than before. Other competitions saw a bigger increase in cards than penalties, making it more likely you’d be severely punished for high tackles. So it was clear that the subjective judgement of officials around the world was creating large disparities in sanction for basically the same action. That would undermine any ‘zero tolerance’ approach, for obvious reasons.

Second, the issue was whether these changes were enough to cause the desired change in tackler behaviour? Look at those rates – one high tackle penalty per match, on average. That’s one every two matches for your specific team. One yellow card every nine matches on average, so your team would play 18 matches before getting one. There were more frequent yellow cards for deliberate knock-ons in some competitions, and other offences like ruck entry or offsides would be penalized 4 to 5 times per team per match. A high tackle was carded once every nine high tackle penalties. The ‘stick’ just wasn’t large enough, or used often enough.

The feeling, then, was that if the initiative was to work, then the sanction needed to be applied more often, more severely, and more consistently.

[ribbon toplink=true]The High Tackle Sanction Framework[/ribbon]

32. And that led to the creation of the High Tackle Sanction Framework beginning in late 2018, completed in May this year. There is a straight line between the zero tolerance directive suggested by that first expert working group and this Framework, because the sport is still trying to achieve the same outcome, just with greater consistency.

You can read all about the frameworks’ aims, components, implementation, and then watch a video tutorial on it here. The decision-tree that constitutes the framework is shown below.

33. The idea here is that just as pilots use these kinds of processes to fly planes, and doctors use decision trees to diagnose disease (the concussion diagnosis process informed aspects of this framework, for instance), so too could a referee benefit from a logical, step-by-step process that identifies important aspects, links them to clear definitions, provides signs or indications of risk/danger, and then comes up with a final decision.

34. To use it, the referee first establishes whether they are dealing with possible foul play. If they are, then the initial decision is whether they are assessing a high tackle or a shoulder charge? Those are differentiated using the definition provided for a shoulder charge.

Once that’s established, they ask the same three questions, in the following order:

1. Is there head or neck contact?
2. What is the degree of danger? (and the signs or indicators of high danger are provided, based on what is known to increase risk of injury. The list is indicative, leaves the answer up to the referee’s expertise, but is not arbitrary – these are the ‘inputs’ that create the output of risk)
3. Are there mitigating factors? Here again, a list of factors to consider is provided. And again, whether they are applied or not is up to the referee, who retains autonomy and authority in the process.

Questions 1 and 2 produce an initial decision, and then mitigation may be applied if it is clear and obvious. That produces a final decision, one that can be communicated with three answers to three questions.

35. The hope for the framework is that it will provide clear, logical decisions that can be understood by all. Note that I say “understood”, and not “agreed with”, because perfect agreement is impossible. Everyone knows this. Or they should. There will always be differences and subjective aspects to the decisions – some may say that mitigation was present and should have been applied, others may disagree. Some may say that a tackle has “low danger”, others may deem a tackle to be “high in danger”.

And this is fine – reasonable people can disagree on these junctions and factors. These same subjective aspects were there before the Framework, but few people verbalized them logically. So one consequence of putting the framework out there is that it will highlight any inconsistencies or omissions, creating the appearance of MORE errors (kind of like the cars vs motorbike problem, actually). But inconsistency has always been an issue, and the framework is meant to reduce it, creating more agreement between observers.

So the idea is that this process should improve reliability and increase agreement. It will definitely increase transparency (you’ll now know exactly why you disagree with the referee, instead of basing it on ‘feels’ and which anthem you happen to sing before a match), and ultimately, it should result in more consistent sanctions for the behaviour that the experts consulted throughout the process have identified as being undesirable.

36. Remember the premise here – the application of sanctions through penalties and cards is meant to make players adjust their technique to avoid the head of the opponent, with a side effect of exposing themselves to less risk. This may mean lowering the height by 10cm (see images above), or it may mean footwork, it may mean decision-making.

37. In the midst of inevitable over-reaction to the card decisions using the Framework, the most important thing to know about the Sanction Framework is that it is only in play when there is a dangerous tackle, and that cards will only be given if there is a shoulder charge, or head contact for the ball carrier.

In other words, the path taken through the framework is dictated by two things:
1. Whether the tackle is a shoulder charge, rather than one using the arms (Pathway 1 and 2 vs Pathways 3 to 5, see decision tree above)
2. Whether or not there is head contact from shoulder or arm? (Pathway 5 vs the rest)

If there is no head contact, the worst possible outcome is a penalty (Seatbelt tackle, see Pathway 5 in the Framework) because contact is still high. Or, it may be deemed a legal tackle, no foul play at all, depending on the ball carrier’s body position and height. The referee won’t even consider using the Framework then. This happens frequently in those pick-and-go situations where the ball carrier runs very low to the ground, head down, and actually initiates the head contact, which is often with the chest or upper body of the tackler. These happen 40 to 60 times a match, and people freak out about them for no reason.

The reason I emphasize this is that there can still be high contact on the ball carrier’s body, including to their head (ball carrier head into tackler torso, for example), but only if the tackler is assured of not striking the head with their shoulder or a forceful arm. The picture strip below shows three such examples, all from within a few seconds of the weekend’s match. One is dangerous foul play – the Barrett red card incident, with the Sanction Framework process spelled out beneath it – Q1, Q2, Q3, decision. Two are not. They never have been.

The point is, there is no obligation to tackle lower, just to tackle safer. And “safer” means avoiding the head, which sometimes means staying upright, or not putting the shoulder into the ball carrier’s head, even if it is clumsy rather than malicious. A big part of this is risk vs reward, and making players accountable for taking those risks. Ultimately, “safer” is the list of actions under the green zone of the spectrum, and may be achieved by a focus on technique improvement, irrespective of tackle type or height.

[ribbon toplink=true]Subsequent confirmation research[/ribbon]

38. Perhaps at this point you’re thinking that this is a lot to base on one study. I had the same concerns. But since we did this, a few other studies have supported the theory. First, Matt Cross took a similar data set, and using a completely different analysis method, found that the high risk factors for head injury were speed, acceleration, and head contact. In other words, it confirmed the finding using different approaches to overlapping data.

39. Then Greg Tierney did a series of studies looking at injury and head biomechanics. One found that for front-on tackles, the highest risk direction, the propensity was lowest when tacklers targeted below the upper trunk for upper body tackles, and suggested tackling the lower trunk for lower body tackles and avoiding the upper legs. That echoes the spectrum we produced, where the safest zone is the torso of the ball carrier. It’s like a “Goldilocks zone” – not too high, not too low, just in the middle (with credit to rugby expert Ken Quarrie of New Zealand!)

40. Even more interestingly, Tierney also studied the mechanics of the head during tackling, and found that the ball carrier’s head linear acceleration, angular acceleration and change in angular velocity values were between 50% and 150% higher greater for upper trunk tackles than for mid/lower trunk tackles, which “support[s] the proposition of lowering the current tackle height laws to below the chest.” And granted, this study looked at the ball carrier’s head, not the tackler’s, but this emphasizes that you can greatly reduce the risk to one half of a tackle situation with a lower height, and as I’ve explained above, the tackler also has less risk when lower.

[ribbon toplink=true]Conclusion[/ribbon]

Will it work? Everyone hopes so. Of course there are concerns. There were also other options – lowering the height of the tackle, for instance, to the armpit is one such option. That was explored unsuccessfully in England, where no reduction in concussion risk was found. So far, the same trial has been a bit more successful in Stellenbosch, but it’s early days yet. A trial proposed by France Rugby will soon explore lowering the height all the way down to the waist, which is basically the equivalent of giving the maximum possible dose of a drug and seeing what side effects it causes!

That’s why it’s interesting to trial – it may be the first step, it may be the last one, but you’ll never know until the step is taken. Based on the above, I think that too low is bad, too high is definitely bad and something in the middle is best. Also, choice matters to the tackler, and taking it away is not ideal, because it may force the tackler to go low when it’s actually better to stay high. That’s why the key priority is to do what is necessary to avoid the head. While in general, this involves being bent or aiming lower, that’s not always the case. But there’s enough rationale to ask this question, so the ida of a trial is intriguing.

There are also trials to reduce line speed (the 50/22 law), because pretty much every coach who was consulted in the above mentioned process said that the focus on speed came at the expense of tackle “integrity” and technique, so if you can reduce speed, then the technique may improve and fewer head contacts will occur. Not to mention that taking speed out of contact makes the tackle inherently safer. So that’s an avenue running in parallel with the technique and height interventions.

So those are on the horizon, but what I’ve explained above is the attempts by everyone to “nudge” behaviour in a direction that makes the sport safer without being radical about it. Of course there are concerns – the tackle is a complex, dynamic event where risk can be transferred, created, swapped, and so it’s possible that one problem is solved, another is created. The plan is to replicate the same study in the future. And in a follow up post, I’ll walk you through the U20 World Championship to describe whether that confirms or refutes the picture I’ve described here.

But hopefully you appreciate that none of this is guesswork, and there’s data and logic behind the desired changes. And a lot of hope that they might turn down the risk of concussion. Early signs are positive, globally, it’s just a matter of monitoring, refining and confirming.

Ross

This is a guest post. Something I’d love to host more of in the future. So if you’re reading this and thinking “I’d sure like to have a say on Topic XYZ”, then feel free to email me at ross.tucker@mweb.co.za, and if the pitch sounds right, and it’s a topic I think fits the overall vision of that site, I’ll publish it completely unedited. Your voice and opinion for everyone to read and discuss. Even if I don’t necessarily agree!

In fact, if anyone from WADA is reading this, and if you feel the need to respond, then send me a mail and I’ll publish that too. On one condition – it cannot sound like a press release. Candid, direct, honest comms, then I’m all in, and I know the readers of this site will appreciate that too.

So please, reach out, whoever, whatever, and wherever you are!

[ribbon toplink=true]Today’s Guest: Prof Erik Boye on anti-doping[/ribbon]

Today, it’s Prof Erik Boye. Erik and I have never met, not in a human way, anyway. But we’ve worked together since April last year, when he was one of two other academics who contributed to the scientific rebuttal of the IAAF’s DSD Research and Regulations. This after Prof Roger Pielke Jr asked if we’d be interested. you may recall we eventually got hold of a portion of the IAAF’s data and found that it contained duplicates, omissions and ‘phantom times’, affected up to 30% of the results, and thus could not possibly be deemed trustworthy enough to base any kind of policy on.

Erik is based in Oslo, Norway, and his research interests span far more than doping, and include cell regulation and cell signalling. You can read a brief summary of his academic career here

On the anti-doping side, Prof Boye has written numerous papers raising concerns about the integrity of the anti-doping process. Among these is the Steven Colvert case, where he was one of four authors who challenged the validity of the EPO test that saw the Irish sprinter banned. More recently, Boye and Pielke Jr published this article, “Scientific Integrity and anti-doping regulation”, which provides an idea of Boye’s approach to the issue.

Below, then, is a piece that he kindly wrote for me, unedited, it its original form, and for which I’m obviously extremely grateful. I make no editorial comments, other than the following introduction to his article, and why I think it matters:

[ribbon toplink=true]Positives, negatives and trust in anti-doping[/ribbon]

Speaking broadly on the principle, one of the things we’ve seen over and over and over in anti-doping, is that confidence in the system is at an all time low. Some would say the lack of trust has been hard-earned, but among skeptics (or realists, if you wish), the reality is that a positive test doesn’t necessarily indicate a doper, and a negative test definitely doesn’t indicate a clean athlete.

Supplements, inadvertent doping, legal challenges, dog studies, bargaining power, low sensitivity, timing considerations, physiological complexity and cost, all add up to undermine the in-hindsight simple view that testing a blood or urine sample would catch a cheat and exonerate a clean athlete.

In that context, the only way to climb up from a position of low trust is to demonstrate, at every possible step, the absolute scientific rigor and evidence-based fairness of what is being done. We need to get back to a place where a test result, positive or negative, means what it actually claims to mean. So that’s why these views are so important, and I think it’s why Prof Boye is so passionate about them. Let him explain the rest…

[ribbon toplink=true]Anti-doping and the bias in power: Prof Erik Boye[/ribbon]

The World Anti-Doping Agency (WADA) is declared the international, independent organization to promote, coordinate and monitor the fight against all forms of doping in sport. There is every reason to question whether WADA is independent, but here I shall deal with how WADA exercises its power over individual athletes. I shall conclude that there is an unfair balance inherent in the system that favours the views and positions of WADA and weakens the arguments of athletes. This systematic bias endangers the right of law for individual athletes.

[ribbon toplink=true]Access to science experts[/ribbon]

The WADA-accredited laboratories are collecting and analyzing biological samples on behalf of WADA. They report Adverse Analytical Findings to the antidoping system, which decides whether to press charges against athletes with positive samples. In hearings, the lab scientists appear as experts for WADA. The laboratories are in pole position to prepare, select and promote their own findings. Therefore, they face a conflict of interest: they strengthen the antidoping mission (to catch as many dopers as possible) when promoting their own positive findings and, at the same time, reduce the impact of the athlete’s experts.

Furthermore, the laboratories are financially dependent upon WADA and have an incentive to please them. The lawyers residing in the panels of CAS hearings frequently meet with WADA-accredited scientists and develop a confidence in their expertise that may run counter to trusting opposing and unfamiliar expertise. This lack of power balance is not fair to the athlete. An ideal system would collect evidence from independent scientists who have no vested interest in the outcome.

WADA-accredited laboratories routinely employ one another to present a “second opinion”. However, another WADA-accredited laboratory is not independent; the laboratories are all bound by rules saying that they cannot dispute the conclusions of another WADA-accredited lab (discussed in a recent publication from our group). A fair and meaningful second-opinion process would include truly independent scientists.

The nature of science is hypotheses, discussion, testing and retesting. It is unscientific when one party demands to always be right. Errors do occur in any scientific system and a lot of false positives are bound to occur in the testing in the WADA-accredited laboratories. Where are the examples of the antidoping system admitting that they are wrong? Too few to be trustworthy. Simple statistical considerations say that there must be many, many innocent athletes who have been sanctioned for doping.

[ribbon toplink=true]Access to data experts[/ribbon]

Following positive results in a WADA-accredited lab, the athlete is confronted with electronically manipulated data and is not allowed to see the original analytic data. The laboratory rarely makes available more data than what supports their own conclusion, and they sometimes neglect to point out uncertainties and questionable findings. This restriction of access reduces the possibility for the athlete’s experts to counter the charges.

[ribbon toplink=true]Access to legal support[/ribbon]

Lawyers are expensive. Many athletes do not have the resources to fight a doping allegation, thus creating an economic imbalance: the rich can contest WADA decisions, but not the poor. “To put it bluntly, the more money you have available to put together a comprehensive defense, the more likely you are to be exonerated”, stated Andy Brown.

To mitigate such an economic imbalance, WADA has been granted immunity from civil jurisdiction (in Canada). The idea seems to be that wealthy athletes can tap the antidoping system for large resources, with their good access to legal expertise. This protection reduces WADA’s financial risks emanating from their business of sanctioning dopers. Such an immunity must necessarily be accompanied by defined and automatic control instruments to make sure that WADA is not breaking its own regulations or inhibits the rights of athletes.  Little attention is devoted to the opposite problem: athletes who have no funds or resources to challenge doping charges from an advantaged, well-positioned organization. Some athletes even have to pay to see the data, which may certainly represent a hurdle. The financial imbalance calls for an obligatory insurance instrument enabling poor athletes to challenge an unfair decision, either within the sports system or in a private court.

[ribbon toplink=true]Abuse and threats[/ribbon]

In some cases, it has been demonstrated that the antidoping movement’s wish to sanction as many athletes as possible has resulted in undue pressure, negligence and threats against scientific journals, unwillingness to discuss and to make data available, as documented here. To my knowledge this information has never been challenged. As stated by Johan Olav Koss, head of FairSport, at an antidoping meeting in Oslo in June, 2019: “Athletes have a true fear of retribution” and “Researchers are being threatened when challenging WADA”. This state of affairs is not worthy of an organization performing a public function. Can we accept that WADA is outside of public control? The demonstrable asymmetry in means and power puts due process in peril.

[ribbon toplink=true]Level playing field[/ribbon]

The expressed raison d’être of WADA is to protect the ‘clean athlete’ and to provide a ‘level playing field’. However, when an athlete falls prey to an error in a WADA laboratory the athlete is considered guilty and has lost all forms of protection, it seems, and no level playing field is in sight. The game is tilted and is no longer fair. It appears acceptable to presume guilt and to deny obvious rights of defense. There is no level playing field in the game that accused athletes must play.

[ribbon toplink=true]Transparency[/ribbon]

The entire antidoping system should be subject to transparency, which means that athletes are allowed to see, in detail, what they are charged with and what is occurring in their case. This will allow the athlete to engage external, independent expertise. Without an independent assessment of the analytic data the few rights that still exist for athletes are eroded. Transparency will restrain an otherwise superior part, promoting respect and objective evaluations and neutral behaviour. Furthermore, a dependable, robust and transparent court system should provide the checks and balances that ensure that controversial decisions can be revised. In an internal report, WADA is preoccupied with increasing the efficiency of taking dopers with a telling silence about false positives or innocent, sanctioned athletes.

[ribbon toplink=true]Conclusions[/ribbon]

WADA demands of all sports organizations and athletes to be compliant with the WADA rules, as spelled out in the WADC, “the Code”. It would be a good idea if WADA could make clear their rules and ambitions for their own procedures, such as transparency and acceptance of the possibility of errors on their side. Furthermore, they should pledge and practice to treat the athletes with respect (in particular those charged with doping), and make relevant information available as far as possible.

Prof Erik Boye, Professor Emeritus at the Institute for Molecular Biosciences, University of Oslo

OK, I’ll try to be brief, but I need to cover important context, so may fail…

But I have to share some thoughts on this controversy that at first glance looks to be between swimmers – Sun Yang the common denominator, Mack Horton the first protagonist, Duncan Scott the latest. However, if you go a little deeper, the two incidents ask some very important questions about anti-doping, bans, trust and confidence, and the integrity of the whole system. So that’s what I want to focus on, but first, you need to know the backstory and context.

The latest incident is below. It happened yesterday, when Sun, gold medalist, and Scott, bronze medalist in the 200m Freestyle at the World Championships, had an exchange on the podium during the medal ceremony. Well, strictly speaking, Sun screamed and Scott stayed silent, but refused any interaction, including the obligatory and customary handshake.

Here you see that prior to the medals being given out, Sun leans over to one of the bronze medalist for a handshake (they gave two bronze – there was a dead heat), and then instead of even trying to shake Scott’s hand, just screams at him. There must have been some kind of altercation prior to coming out for this, because Sun Yang knew to not even reach over and try:

Then after the medals are awarded and the anthem played, Scott refuses to mount the top step of the podium for the photograph, and when they walk off and Sun has another go at Scott, then gets into his face at one point saying something like “You are a loser, I am winning”, and off they go.

Quite remarkable scenes, not typical in your sanitized & hyper-managed world of sport’s medal ceremonies, and unusual only because of the non-conformity. Both men received warnings from FINA, I suppose Scott’s is for showing disrespect to the official process and ceremony, and Sun Yang for his outburst before and after the medals were given out.

This was, as mentioned, the second such incident of the World Champs – only a few days earlier, Mack Horton of Australia, had refused to stand up on the podium at all after winning silver behind Sun Yang in the 400m freestyle. Their history goes back a fair way, which brings us to the ‘why’ of all this?

[ribbon toplink=true]The hammer and the blood sample[/ribbon]

The root of this, of course, is doping. Or rather, to be precise (and precision matters, as I wish to explain), the perception of doping. By Sun Yang, again. The reason it’s hot right now is because of a leaked FINA report earlier this year that revealed that last September, Sun Yang, already a controversial figure in doping, had destroyed his own blood samples during a doping collection in China. So out came a hammer, and that was that – no samples to test.

Now, anyone with even a cursory knowledge of anti-doping will appreciate the gravity of this offence. An athlete can’t refuse a test, unless there are exceptional circumstances. They certainly can’t or shouldn’t destroy their samples without mighty good reason.

On face value, these were not exceptional circumstances (though as we’ll see in the detail, there’s some grey area). I say that because the doping officials showed up at the designated spot at the time Sun Yang at given them – every elite athlete has to declare a time and place that they’ll be available for testing for at least one hour every day. Sun Yang’s window was between 10pm and 11pm on 4 September.

The officials showed up at around 11pm, but Sun Yang was not there. He arrived a while later, and then went to a place to provide the blood samples. This is where it gets murkier. I don’t want to get massively bogged down in the detail of the case, though it’s actually really important because it illustrates some really crucial concepts to how we evaluate what has happened in South Korea this week.

But basically, the summary is this: Sun Yang doesn’t accept the accreditation of two of the three officials. He contests that the samples are being improperly collected. He challenges the conduct of the Doping Control Assistant as highly improper. They deny this, and proceed to get a blood sample, but not a urine sample (there’s nobody Sun Yang trusts to chaperone him and see it being provided). Over the course of about four hours, various people get called and consulted – doctors, lawyers, heads of Federations, the head of the IDTM, the organization that oversees the anti-doping.

None of this is enough to provide reassurances, and eventually, by about 3am the next morning, they reach a point of no return where Sun Yang’s entourage apparently calls for a hammer because they do not want the blood samples provided to leave the premises. The DCO and head of IDTM are both apparently trying to make it very clear what this would mean, and the consequences should Sun Yang not comply.

At 3.15am, hammer meets glass, samples are destroyed, and we head to a Doping Panel run by FINA. Again, I don’t want to get bogged down in the weeds of the legal decision that was made by the FINA Doping Panel in the case, you can read the 59 pages if you wish, other than to say that it’s a mess of technicalities and confusion around what constitutes proper accreditation, and FINA’s belief that the Doping Officers had it, Sun Yang didn’t.

Ultimately, the Doping Panel finds that the collection process was “not properly commenced”, “the blood collected was not properly authorized and thus was not properly a sample”, and that the conduct of the DCA was highly inappropriate.

Somewhat bizarrely, the Panel also concludes that the repeated warnings by the DCO to Sun Yang that destroying or not providing the samples would lead to consequences “did not get through”. The Doping Panel sides with the athlete who says that he was never told what consequences would apply for non-compliance. That strikes me as amazing, because this is not some uninformed newbie who might not understand how anti-doping works, and it seems like in the course of 4 hours, someone would have laid out the scenarios if the blood samples were not released or were destroyed. However, the DCO apparently never provides an official Refusal Form that states, in black and white, what the athlete is inviting with their refusal. This is another big mistake on the part of the collection team – they didn’t ever achieve ‘crystal clear’ communication , or what the Panel calls “a bang”, about what Sun Yang was actually inviting or doing through his refusal to release the samples to them, let alone take a hammer to them.

In any event, all of this adds up to sufficient technicalities that the final decision is that Sun Yang is not guilty of a doping violation, though they do call him “foolish” for risking his career on his belief that the accreditation was not proper. They warn him (sternly) that it’s better to provide the sample under protest than to destroy it (or refuse it), and dismiss many of his other claims. But it’s not enough to issue a sanction, so he is declared not guilty, and off we go, to South Korea.

And that brings us to 2019 World Championships. Because rather than suspend Sun Yang pending the outcome of a WADA appeal process after FINA offered a stern rebuke, Sun is swimming, and winning golds, in the pool in South Korea. And that triggers athlete’s anger, for understandable reasons.

[ribbon toplink=true]Staring at the Sun, and blinded to all else[/ribbon]

Now, there’s a lot of detail above, and trust me, I’ve given you a very brief summary of a clearly chaotic evening. Other swimmers are now faced with a perception issue here – there is clearly no trust in Sun Yang. Or FINA. After all, he served a three month ‘secretive’ ban a few years back for what he argued was necessary heart medication, in similarly murky circumstances. That got him a reduced ban, which means this is a second perceived ‘leniency’ for the same athlete, and he is, in the eyes of many swimmers, now a two-time drug cheat (and the appeal process may yet confirm that).

For many of them, HE is the problem. And so when he is on the podium, with a medal that means they’re getting a medal one degree less valuable than they think their performance deserved, it is understandable that they feel disgruntled. So they express their right to protest. I have no problem with this. I know that Richard Ings, former head of the Australia Anti-Doping Agency, has been a critic of the athletes, calling for “innocent until proven guilty” attitudes and punishment for those protests. And that’s certainly true in terms of official sanction of the athlete, but I don’t begrudge other athletes the right to express their unhappiness in protest.

However, what I think should be applauded is a protest against a system that has failed the athletes. Not against Sun himself. Yeah, I get it – you need a bad guy, and who better than the guy who’s done some time, who displays what is a shitty attitude, seems aggressive and surly? Perfect villain, you couldn’t script a better one. He screams at people who remain calm, he taunts people from the winner’s position. What a douche.

So then we can even call his anger “roid rage”, a leap of faith for sure, but further confirmation of how dirty he is (and how clean we are, by extension), and lump testosterone use in there among his transgression (as many have done, including some high profile people on twitter). After all, he must surely also be on the ‘roids, right? Guilty as hell.

This is not right. And no, I’m not defending the guy to the hilt here – he should not be in the pool in Korea. But those protests should not be directed at HIM. They should be directed at the system that couldn’t get the anti-doping accreditation right. That couldn’t create that “bang” moment that made him realize what it would mean to destroy the samples, but still do it. That couldn’t turn his bad behaviour into a rule violation, because of self-inflicted technicalities.

Their protest might be at a system that many onlookers have assessed and realized is wanting, and failing to deliver clean swimming, for reasons ranging from lack of testing to lack of forceful sanction. Even more broadly, athletes might choose to protest a general failure of anti-doping in sport.

So if a swimmer wants to protest, go for it, and well done for finally standing up for clean sport. I applaud that, and I wish it happened more often, not only when the “different” bloke is standing there winning the medals.

But if you’re standing up specifically to Sun Yang, the ONLY guy who in your mind violates your desire for clean sport, then I think you need to hit the pause button and understand that just because you know about a hammer and a blood vial at 03.15am on September 5, doesn’t mean you’re making things better when you direct your wrath at that one person. Stare too long at the Sun, and you’ll see little else you should be seeing.

[ribbon toplink=true]”Us vs Them”, naive trust, and the danger of rationalization[/ribbon]

What this has created is huge, typical polarization. Even after posting those videos yesterday, in which I gave no opinion on who is right or wrong on the matter, my Twitter timeline turned into a kind of ‘new Cold war nastiness’ tennis match between pro-Sun and pro-Scott voices. This is, apparently, a drop in the ocean compared to what Mack Horton and Duncan Scott are no doubt getting on social media.

Point is, this is ugly. It’s harsh. But it’s also inevitable. Anti-doping has reached a point where there is so little trust in the process that even a considered legal process that gives two parties the right to state their case won’t reassure people. Sun is guilty, in their minds, of multiple doping offences. There may also be other reasons for this certainty – I’ve long felt that the athletes know who is doping and who is not, and Sun is emblematic of the mistrust. That first ban, and now this, probably just pour petrol on a fire that to many already existed.

But here’s the problem I have with this, and forgive the hypothetical I have to make to lead me to a point. If Sun Yang was an American swimmer, would the Australians and British stay off that podium? And vice-versa? If he was not so easy to villainize as a product of a system that is so easy to mistrust (remember turtle blood?), a clandestine system that we know has state run properties like the other “evil doers Russia”, would it be as easy to leap to accusations of “drug cheat”, to say that he should be banned for life?

If he was a media darling, a “Sir” who does nifty victory celebrations and victory laps with kids, who Instagrams happy family pictures, appears on TV reality shows, who does charity events, who sells us our breakfast cereals and endorses our cars (in English!), who appears on our TVs in talkshows and cameo appearances, would we evaluate him by the same standards, even if there’s as much (if different) reason to doubt?

I suspect not. And I don’t mean this to either defend Sun Yang, or to play whataboutery. On the first issue, destroying your samples with a hammer after you’ve been around the scene for two Olympic cycles is tremendously stupid, and worse, reveals a mindset and disrespect about doping. In fact, if there’s a louder way to scream “My blood is dodgy”, I don’t know what it might be.

But the problem is, you can’t know that for sure. He may genuinely feel that people are out to get him, a level of paranoia that would lead to destroying possibly clean samples. But his failure to comply with the normal process for refusing a sample should at least mean he’s not swimming in the pool in Korea, in my opinion.

On the second issue, the fact that other swimmers are likely doping just as much, if not more, but haven’t been caught, should be obvious to anyone except those who still believe in the purity of anti-doping. As I’ve discovered, it’s difficult to argue with people who believe in a simple world where a sample that is negative places you in the group of “clean athletes”, while a positive sample marks you out as a dirty one.

“Was he found guilty?”, they’ll argue when you bring up high profile cases of doping violations that were eventually set aside or reduced for whatever reason. And if the answer is “no, not officially”, they will say “Aha, so how dare you conflate Athlete X with that lying, cheating bastard Sun Yang”. Well, my friends, if you read the FINA doping report, you will see, unequivocally, that Sun Yang was also found not guilty. How does it feel to be in a prison of your own parochial making?

And this is the problem. It happened in 2016 when the Russians were in the dock. They are evil, we are good. It happens about Kenyans and Ethiopians – “They have no testing, they hide away and dope to win, but never get tested”. But not “our runners” who travel there to train. That’s just for altitude and solitude. Doping must be genetic, only.

And then it played out very amusingly in the Tour last week too. A French guy unexpectedly makes it to the top of a tough climb after beating your darling in a time-trial, both surprising results, and so he must be dirty – “extraordinary transformation for a one-day racer, impossible, doper!”. Even though those two performances and his transformation and improvement are not even in the top 10 of extra-ordinary performances and transformations that you’ve been cheering wholeheartedly since 2011″.

How do the same facts, or actually, an even more extreme set of the same facts, lead to totally different conclusions, if not for reasons of wilful blindness? It’s quite extraordinary. And I worry that the focus on Sun is doing the same.

To think of anti-doping in term of “innocence” and “guilt” just because the system declares it so is charmingly naive, to the point of funny and ridiculous. And dangerous. Sun Yang is officially “innocent” of a violation, even though he destroyed his samples. Does that mean he’s trustworthy? Heck no.

If we see Anti-Doping Violations as legit only if proven and confirmed by official process, then we cannot condemn Athlete A and celebrate Athlete B. And similarly, if two athletes are banned for shorter periods because the system decides both were “inadvertent dopers” but not deliberate cheats, then we are massive hypocrites if we celebrate one’s future success while we condemn the other to a life ban based on…what exactly? The flag on their shirt? The anthem they sing?

I wish I still had that sense of confidence in the system. Truth is, people are too keen to believe in black and white when they live in a world full of grey. And again, this may sound like anathema to those who want the evil villainous Yang to take the fall for all of sports-kind, but I think there are more indicting and suspicious things than that FINA report about a hammer. I am more suspicious of sudden late-emergers and transformative performances late in careers. Of associations with known dodgy coaches. Of TUEs being abused. Of missing three out of competition tests because of forgetfulness. Or even one because of a broken doorbell. Of clear lies around the possible doping act, the people, the leaks. These are “hammers”, too, they just make less noise and splash less blood on the floor.

But I don’t expect to ever see an athlete protest another for any of those things, because they’re “uncertain”. And that’s fine – I’m not writing this hoping that people stay off podiums just because they don’t like being beaten and think someone’s doping. Well, folks, newsflash – it’s all uncertain. Just read the 59 pages of that FINA Doping Panel. Sun Yang was clearly guilty of something that night, but it wasn’t as black and white as your desire needs it to be. We rarely see it as black and white was that, in fact. That alone makes it unusual (and thus worthy of condemnation).

But the grey is everywhere, and while we are all in the spirit of zero tolerance for previously failed tests, for unexplained anti-doping rule violations, for inadvertent doping with reduced bans, and applauding athletes for making stands against these things, let’s hold that line to the point where we do it for people other than the evil Chinese or Russians, and show a little less hypocrisy and bit more introspection.

WADA have appealed the FINA Doping panel decision and CAS should hear that in September. Perhaps more details emerge and we’ll be closer to finding out what transpired. If we do, we’ll be 0.001% wiser about the murkiness that is elite sport. And it’s great that the exposure of this darkness is protested, I take heart from it. I wish I saw it more, but directed at the system, not the athlete-pinata who is too easy to miss when you swing hard.

Ross

So the Court of Arbitration for Sport have released their full decision in the Caster Semenya-IAAF case. You can read all 632 points of it here, and then evaluate the arguments offered by both sides in your own time!

It is, as we all know, an incredibly complex case, made all the more so by how polarizing it is. It cuts to what you believe about elite sport’s place in society, attitudes towards fairness (no matter which direction you argue this from!), and ideas about governance and policy and how it should be achieved.

I’ve said and written a ton on this issue, just in the last few months, so I don’t want to repeat all the same themes, but with the full decision now available, i can share a little more insight into my own experiences at CAS and explain how I think the verdict was reached. So this is a relatively short post based on some tweets I shared yesterday on the verdict, but if you want more detail, perhaps consider the following:

1. Lengthy point by point article on my reaction to the verdict
2. A background podcast discussion with Mike Finch on The Real Science of Sport podcast, discussing sex, gender and the physiological issues
3. The Caster Semenya Decision explained: A podcast discussing the case in the week of the verdict

[ribbon toplink=true]The CAS Decision: “Home ground advantage” and a difference in framing”[/ribbon]

I was an expert witness in the case. I came to be an expert witness in the case because together with Prof Erik Boye of Norway, and Prof Roger Pielke I had written an academic criticism of the IAAF’s research in about July/August last year. When I first saw the IAAF research, it was my “watershed” moment. I remember thinking “is that it? This is all you got? This policy is now much weaker than the one before it”. And then when we looked more closely at it, we saw so many problems that it was difficult to avoid the realization that this evidence was fatally flawed. Our criticisms of the erroneous data were profiled by the New York Times in July 2018, and then I got a call from the legal team for Athletics South Africa a few weeks later.

I say that upfront as a disclosure, because it sets up my own paradigm and prioritization in this debate, in the sense that my direct involvement with this case was a function of my appraisal of the evidence of this case. And as I shall explain below, I believe the case comes down to evidence vs principle/theory.

So, as you read the CAS Verdict, you may start to notice a clear separation in arguments. Specifically, the issue is framed differently by the opposing parties.

On one side, you’ll see the IAAF arguing forcefully for biological principles of testosterone & performance. To make this argument, they frame it as an issue of men vs women. In this argument, when you strip away the complexity of social factors, it leaves the “simple” conclusion that males should not compete in women’s sport because of androgenization (by Testosterone). This allows them to motivate their position as one that protects women’s sport.

And to be clear, it’s difficult to reject this principle. Indeed, I’d say it’s impossible – they are solid arguments. These same arguments made for a Transgender female (that is, male who reidentifies and wants to compete in women’s sport) are critical, and I would be 100% behind them.

So if you start this debate by asking “Why does women’s sport exist?”, then you head off in a direction that few people can rebut. And so, quite unsurprisingly, the IAAF’s platform is male vs female, and they work very hard to frame this as a PRINCIPLE based argument, using THEORIES of testosterone’s role in determining male vs female performance differences.

But you also see the other side (Semenya/ASA) arguing that the issue should be framed as “Women with DSD vs women without DSDs”. This in turn compels questions to be asked of the research, the magnitude of advantage. They ask: “Where is the evidence? Can the advantage be quantified? How large is it?”.

This was the mandate set by CAS in 2015, after all – go back and look at the CAS decision in the Dutee Chand case. Look at these words:

In effect, what you see there is the “template” for how Semenya would try to win the case – keep brining it back to evidence, because “the numbers matter”. “Degree of advantage is significant”, as per CAS in 2015. This is where Semenya’s case would be strongest. Show that the DSD Regulation lacks evidence, and thus can’t be viewed as necessary given the risks or harm it may cause.

So just as it is predictable and expected for the IAAF to frame their argument as male vs female, so too it was predictable that Semenya’s side would want to frame it as “women vs women with differences”.

Indeed, way back in January, when all the documents and arguments were submitted ahead of the hearing, and all parties had stated their case, I remember thinking it would come down to which of these – principle or evidence – mattered more to CAS? Would biological theory be given more weight than the lack of evidence & poor evidence? Or would CAS, as in 2015, prioritise data, the numbers, the degree of advantage?

I even wrote an email to some people in which I equated this issue as one of “home ground advantage”. In sport, whoever gets to play at home has an advantage, and in my mind, this case was going to come down to whose framing allowed them to “play at home”? Would CAS give credence to the Male vs Female comparison, even though the evidence in DSDs did NOT support it? Or would CAS give priority to the lack of evidence, despite the numerous real challenges in providing that evidence? The former would massively favour the IAAF, whereas if we “played” in the realm of evidence, then Semenya was clearly stronger (because the IAAF’s evidence was that bad).

In the end, the final decision revealed the answer to this question. The IAAF “played at home”, and managed to make a compelling principled argument, despite an appalling scientific/magnitude-based one.

[ribbon toplink=true]Key arguments from CAS[/ribbon]

In this regard, paragraphs 551 & 552 of the Semenya decision stand out. CAS saw, recognised & accepted that good regulatory policy needs good science, & that this was “laudable”, but then say it’s not their place to assess it. I’m not sure how you can divorce the outcome (the Regulations, specific as they are) from the process (the research).

I think that if the Regulations are meant to be an evidence-based document, which they absolutely are – that’s why only the 400m to 1500m are covered by them – and they were produced DIRECTLY as a result of the Bermon study, then it is absolutely crucial that the process by which the evidence is gathered, analysed and applied used is prioritized.

So for CAS to declare “It’s not our problem to evaluate the process by which the rule was created” is quite extraordinary. They instead want to confine their role to deciding whether the Regulations are necessary, reasonable and proportionate. Can you see that they have thus CHOSEN to limit their decision to PRINCIPLE AND THEORY only? The evidence doesn’t matter – it’s important, yes, but it’s not our problem.

[ribbon toplink=true]Tension between theory and evidence, and the inevitable conflation of DSDs and Trans MTF[/ribbon]

This was always the ‘tension’ for me. The IAAF’s theoretical case (Points 285 to 292 in particular, for those who want to understand it) was strongly made, well argued. I’m a supporter of the principle. But the evidence & process supporting the regulations was, and I’m being kind, empty & utterly inadequate to justify the harm it introduces to people.

This is why the IAAF had to downplay the evidence & amplify the theory. It is noticeable in the verdict how little that Bermon study is cited in support of the Regulation. Instead, the IAAF relies heavily on a theoretical review paper by another expert witness, David Handelsman, to make their case. Bermon is relegated to the periphery, out of sight and scrutiny, because that’s where the IAAF were weakest.

On the other hand, Semenya/ASA played up the evidence & avoided the theory. Rather talk more Bermon, less Handelsman! So the verdict ends up looking like two monologues, rather than a debate where A & B directly address one another. It was, actually, a frustrating process. Even in the group debate sessions (the so-called Hot Tubs), we often didn’t reach consensus because two separate agendas were being debated.

[ribbon toplink=true]Transgender concerns and DSDs in the crossfire[/ribbon]

Another major factor that I believe drove the verdict (compared to 2015 Chand) is best summed up in these points, highlighting how the growing concern around transgender females (MTF, so this is men who re-identify and then compete in women’s sport) may have impacted the decision.

What you’re reading above is really crucial, in my opinion, to understand how the “wind direction changed” between 2015’s Chand decision and now. This witness, Lambelet Coleman, was asked whether a victory for Semenya in this case, the DSD one, would have implications for the transgender regulations. She said, without hestitation, “yes”. So too did a number of other witnesses in Lausanne, and I remember sitting in that room thinking “this is not a good sign”.

Why? Because the conflation of DSDs with Transgender MTF is a sign that the thinking has moved across to Men vs Women, where the IAAF “plays at home”, rather than women with differences. The reason the two Regulations overlap, by the way, is that both DSD Regulation and Transgender MTF Regulation require that the athlete reduce their testosterone levels in order to be eligible to compete. So when the question kept getting asked, and when the IAAF kept framing it as an issue of “males competing in women’s sport”, it was becoming clear which way the wind was blowing – you will not find any such language in the Dutee Chand case of 2015.

Now, this “shift” is in part because the IAAF were so much better at making the case this time around. But I think it was undoubtedly driven by a growing unease that the transgender male to female athlete is going to be a really big issue for sport in the coming years. And so there was a natural, albeit false conflation of the two concepts. It happened outside the courtroom even more, in the leadup to the case. Sometimes out of ignorance, sometimes malice, people were quite happy to say DSDs and Trans females in the same sentence, creating this perception of a ‘tsunami’ of men coming to destroy women’s sport.

But in the courtroom, what this did was to allow the IAAF to really go hard on their framing, as I described above, that this was an issue of males vs females. This, unfortunately, is where the DSD case also gets especially polarizing, because the next point you reach is one where you label Semenya as a male who identifies as female.

And if you read the verdict, you see this often. For example, here are 3 paragraphs where DSD is juxtaposed with “biologically male” or “transgender male-to-female” athletes in the same sentence – the messaging is clear. It’s about “men vs women”, which, as I said above, is how the IAAF needed to frame it.

It’s not surprising then, that we’ve ended up is a situation where the IAAF has a policy for DSDs that is basically identical to its policy for transgender MTF athletes. It now a DSD woman the same way as it treats someone who is born, raised and lives as a man, then makes a choice to change and compete as a woman (despite protestations otherwise). It is no longer about the Testosterone, but the XY chromosomes. It is now sex testing, not testosterone regulation.

As mentioned, the argument they made in this regard was expected. I’d have done it too. Why? Because any arguments they tried with evidence to actually quantify the advantage would collapse instantly. In fact, their evidence was so ridiculous that they actually show that in 17 out of 22 events, testosterone makes no difference to performance. They prove their own model wrong – I called this “the testosterone paradox” at the trial. Testosterone does not discriminate – in practice, or in theory. In their own model. Yet the policy does, to certain events only, which creates this weird internal inconsistency that should have undermined this policy, if not the theory.

But, the IAAF realized the evidence weakness, so they sidestepped it, and instead drove the theoretical point, framing DSDs as males (Handelsman vs Bermon, in other words). And I must say they made this argument very well, much better than in 2015. In particular, their written arguments were daunting and solid.

What this ultimately enabled them to do was argue that #semenya & other DSD athletes are males. Just ones who “are not as good as the best males”. Those are not my words – here’s Paragraph 289:

This, in turn, shifts the debate towards male vs female physiology, and it’s really hard to dispute that, right? There is a 10-12% difference, undisputed. So ultimately, the IAAF “played at home”, and the requirement to quantify the magnitude of advantage that CAS had prioritised in the Chand case of 2015 no longer applied. Instead, theory & principle won, and all those things like the poor evidence, the medical harm etc fell away.

The quality of evidence could thus be acknowledged as poor (which CAS did do. They just didn’t weight it as that crucial). Risk of harm could be recognised. The efficacy of the drugs could be questioned. But stacked up against a (convincing) argument for the protection of women’s sport from DSDs (& trans MTF, by conflation), it fell short.

[ribbon toplink=true]Changes from Chand and the great unknowns where evidence should have mattered[/ribbon]

And that’s that. It’s 632 points, so I can’t possibly do it justice, but I just wanted to share with you my thoughts on how the decision was reached. The IAAF made a tremendously strong theoretical case – the circumstances around Semenya allowed it, compared to Dutee Chand, and obviously they’d had a lot of practice and a few years to fix up the gaps.

They did in 2019 what they may have tried in 2015, but then again, the policy in 2019 was different – it’s no longer about testosterone, it’s about XY, and then testosterone is, to borrow from Roger Pielke’s argument, the method to “fix people” with the Y chromosome.

It all leaves me feeling really uneasy. A DSD is not a Trans MTF athlete. By the very nature of having a DSD, they do not develop typically as males. So take the case of 5-ARD, which the CAS verdict makes clear is a big issue here. The IAAF argue that they are “biologically indistinguishable…in all relevant aspects”.

What they mean by this is that with the exception of the external genitalia, which may be ambiguous or female, leading to the identification of a person as female at birth, a person with 5-ARD is biologically male and should thus compete as a male for the purposes of sport.

That’s the theory. But my position is that if you’re going to impose on someone a new biological reality at the age of say 18, because they happen to be a fast female runner, then you had better be really, really sure that it’s necessary. And so until you can establish that having 5-ARD does not affect any other physiology that may impact on sport, I can’t see how you can move so swiftly towards labeling someone male (even if you do say you accept their gender identity), and then prescribing possibly harmful drugs to them? That’s why more research into 5-ARD is required – does the absence of DHT affect any physiology that may have sporting implications? If the answer is “we don’t know” or “maybe”, then you absolutely have to hit pause and assess what those implications are before you continue.

The same is true, even more emphatically, for PAIS (Partial Androgen Insensitivity Syndrome). This occurs is a person who cannot, by definition, use testosterone. So on what basis is that person being treated as having performance advantages that are the same as biological male who does not have PAIS? The fact that the DSD Regulations names and treats five conditions without discerning that they may not all have performance implications is a problem to me.

This is why the “numbers matter” – the policy is not benign. It’s potentially catastrophic and damaging. The most powerful testimony of the whole case came from Drs Payoshni Mitra and Lih-Mei Liao, who spoke of the challenges in dealing with DSD cases in a clinical setting, let alone the sporting one. DSD women are different from trans MTF in many respects, past, present and future, and I’m not comfortable with a Regulation that treats them equally, using medical protocols that are not known to be safe, and not chosen by the athlete.

Anyway, that’s that for now. The most complex issue in sport isn’t going away, so perhaps the appeal process will bring it back into view.

Ross

About 6 weeks ago, we launched The Science of Sport Podcast, and today’s short thought is just a push towards that and a brief overview of what we are about.

“We” is Mike Finch and me. Mike is a journalist, editor of the SA Runners World and Bicycling magazines for as long as I’ve known him, and a guy who just loves the world of sport, especially endurance sport. For the last two years, every time we’ve been out, we’ve said “We really need to do a podcast”, and so here we are, 2019, and finally making it happen.

The beauty of a podcast is obviously the dynamic options that conversation affords. I’ve wanted to do one for ages because I think it might be a more effective way of translating sports science for the public, and making the sports news more relevant and engaging. And Mike is the perfect foil, because he knows what the issues are and which questions really matter.

So we’ve had great fun with the four episodes out so far, and I just wanted to give those a ‘bump’ with this post today, and talk about what else is coming soon.

[ribbon toplink=true]The Podcast so far – episodes[/ribbon]

1. The Pilot – Introduction to the podcast, and ourselves, what kind of topics we’d be tackling in the coming months
2. Caster Semenya: Explaining Sex and Gender in Sport – this was a podcast we did a week BEFORE the CAS decision came out, because we wanted to cover the basics of the issue so that the decision would make more sense. So it’s a lesson in DSDs, testosterone and the performance differences between men and women, and what was at stake in the CAS verdict
3. The Caster Semenya Decision Explained – once the decision came out, we spent 90 minutes discussing it. There’s some repetition, of course, but that’s the important stuff, so hopefully you’ll forgive that. We dissect the key issues around the decision and anticipate what might happen next.
4. Running Shoe Technology: Good Science or Good Marketing? – for a change of theme, we addressed the science (and frequent lack thereof) in the running shoe world. We talk pronation, cushioning, injury prevention and the ideal way to choose the best shoe

Our fifth episode is imminent – tomorrow – and is a really candid interview with Dominque Scott, South Africa’s Olympic 10,000m runner. She kindly gave us her time back in April, and we spoke about training theories, physiology, altitude the US College system, sacrifice, mindset, guilt, ambitions, dreams, and a whole lot of other stuff I’d like to think you don’t hear often from elite athletes. So look out for that next.

All links above are iTunes, where you can find us as “The Real Science of Sport”, and we are also on Spotify, and Overcast

As for the future, we are recording one on doping this week, and then will tackle some really fascinating topics related to Talent ID and sporting performance development in coming weeks. That leads up to an interview with David Epstein, whose new book, “Range”, has just come out, so we’ll talk to him about that and more.

I wish we could do one a week, but we are both stretched with our other work – I just got back from two weeks in Dublin and London, for instance, and he’s traveling to all kinds of events too, so the plan is fortnightly, and we hope you enjoy it and will share them widely!

Ross (and Mike!)

This is a guest post, sort of. Well, it’s a post that was co-authored by myself, along with two other academics – Roger Pielke Jr from the USA and Erik Boye of Norway. You may recall that last year, about this time actually, the three of us tried to look at the data that was being used by the IAAF as a key element to select the events covered by its DSD Regulations (the now accepted-by-CAS DSD Regulations).

We got hold of some of the performance data from the Bermon & Garnier paper, and when we analysed it, we found all kinds of data errors, including duplications and the presence of what we called ‘phantom times’, performances that had been analyzed even though they didn’t actually exist in the competitions looked at.

That led us to first try to call for the retraction of the paper (based on the Journal’s policy that flawed or erroneous data are grounds for retraction), and then eventually, in a separate journal, a paper that described the problems with the IAAF’s data.

The CAS decision is now in the books, and the DSD Regulations will stand despite the evidence (my assessment) rather than because of it. As I wrote here, and said a few times last week, my read of it is that the decision was the result CAS weighting the concepts and theory more heavily than the evidence that was presented.

The very best case you can make for the IAAF’s evidence, including that part of it which they have not published for medical confidentiality reasons, it that it establishes a hypothesis through observation of a convenience sample in which the collection methods are in contradiction with the IAAF’s own DSD Regulation, and where no potential confounding factors are controlled for. Their evidence might be considered a passable first step, not the last necessary one, in creating any kind of regulation.

Given that the quality of evidence still matters, and even more, the scientific process that informs governance (even in sport!), organizations have to be pulled up on misleading statements. When a policy has the ramifications of this one, both socially and for the health of those it may affect, it is all the more important that the very best be expected of the science that underpins it.

And so this guest post, written by the three of us, aims to respond to one particular point raised by the IAAF in the aftermath of the CAS decision. That point, shown below, comes from the IAAF’s Briefing Notes on its Female Eligibility Regulations:

We believe that scientific process matters. Given that some of the statements made above are misleading, some are evasive, and some are outright false, Roger, Erik and I have the following response:

[ribbon toplink=true]Response to claims of robust, peer-reviewed research and “misplaced criticisms”[/ribbon]

We would like to respond to several incorrect and misleading claims made by the IAAF in its Briefing Notes released on 7 May 2019. These claims relate primarily to a 2017 research paper that provided the evidence used for the selection of specific track and field events covered by the IAAF’s DSD Regulations (Bermon and Garnier BJSM, hereafter BG17).

Last year, at our request, the IAAF shared with us 25% of the data used in that study. As far as we know, we are the only researchers in the world outside of IAAF who have had a chance to reproduce a segment of their dataset and replicate parts of their analysis. Further, the IAAF analysis of this data remains the only performance data that IAAF cites as the basis for the selection of its restricted events under its DSD Regulations. The IAAF continues to assert the validity of this data even though we have shown conclusively that the data suffered from systematic errors rendering any analysis and conclusions unreliable

We are concerned because the IAAF representations of BG17 made in their recent statement are factually inaccurate and misleading. This matters not just because the flawed IAAF data and research using that data are a key element upholding the new IAAF DSD Regulations, but more broadly, this issue is a matter of fundamental scientific integrity.

In its new Briefing Note, IAAF writes:

[T]he 2017 Bermon & Garnier BJSM paper was criticised for its statistical approach. A new set of statistics were provided on a modified database (taking into account some of the criticisms raised).

This is incomplete at best and highly misleading at worst. It is true that some scholars criticized the statistical approach of IAAF (see Sonkesen et al; Menier; Franklin et al).  However, the concerns about BG17 go well beyond those statistical concerns.  They also include methodological considerations that no reanalysis can overcome, and perhaps most concerningly, the possible persistence of erroneous data in the database.  Our reanalysis of BG17 using data provided by the IAAF, and published in ISLJ, found that in the four restricted events covered by the DSD Regulations, between 17% and 33% of the data were erroneous, including duplicate data and ‘phantom’ data that did not exist during the competitions analyszed.  This was highly concerning and led us to call for the publisher to retract the paper in accordance with the journal’s own retraction policy.

The presence of erroneous data is not even in question – the IAAF subsequently acknowledged that more than 20% of the data in BG17 was flawed, and had been dropped from its subsequent analysis. This is what IAAF means above when they use the phrase “modified database.”  It is noteworthy that this modified database has never been reviewed or evaluated in the same way that we were enabled to do for a portion of the original database.  We thus have no assurances of whether the same number or types of errors may be present in this modified database.

Further, when the IAAF states that “other criticisms of this paper are misplaced”, they sidestep the most damaging criticism of all, that documented in our paper in ISLJ, which represents the only available external and independent analysis of their data. It has been peer reviewed, is available to anyone and is certainly known to IAAF. If the IAAF scientists disagree with our findings, why do they not enter a scientific discussion about the issue? It is scientifically dishonest to act as if our reanalysis, criticisms and concerns of the research do not exist.

The IAAF further states:

All published papers have been peer-reviewed.

This too is untrue. In response to numerous criticisms of its original study, including the highlighting of significant data errors, the IAAF re-evaluated the data and submitted a paper as a “Discussion” (in effect, a short letter) that was published in the BJSM in 2018, seeking to re-do the flawed BG17 study.  It is explicitly noted in that paper that the IAAF Discussion was not sent out for peer review, but instead was reviewed solely by the BJSM editor. Internal, editorial review is not what anyone in the scientific community would characterize as “peer-reviewed.”

In our critique of the IAAF’s original study (BG17), we also looked at the analysis in the follow-up Discussion in BJSM (which we call BHKE18). We found that analysis also to be unreliable. Here is what we concluded in our paper:

Clearly and unambiguously, the results reported in BG17 change quantitatively in BHKE18 upon removal of 220 data points and introduction of new methods. The results of BG17 are clearly unreliable, and those of BHKE18 are of unknown validity. Further, without access to the medical data and all linked performances used in BG17, it is impossible to know how or why certain athletes/results were removed and others not. What is unequivocal is that BG17 used unreliable data, and thus, its results are also unreliable. Different data and methods were used in BHKE18, leading to significantly different results, based on the almost certain use of flawed data, leading consequently to unreliable results. The bottom line is that the use of flawed data makes it impossible to know what, if any, relationship exists between the variables of BG17 and BHKE18 or to verify the reported results.

The fact that IAAF themselves performed the research and analysed the data on which it has based its controversial regulations is non-transparent and problematic. They have failed to respond to our criticism, to explain what data errors have been detected and possibly corrected or to release their data for independent verification. These facts highlight the nature of the deep conflict of interest that the IAAF researchers have in this case.  Here is what we stated in our paper:

The IAAF set itself up for problems by conducting research on performance effects associated with testosterone using in-house researchers. This creates at a minimum a perception of a conflict of interest that could have been mitigated to some degree by allowing independent researchers access to data and evidence, in order to replicate findings. In this case, such access was not allowed, except for the small amount of data shared with us, which was subsequently found to contain numerous errors. The unwillingness of the IAAF to correct or acknowledge errors highlights its conflict of interest.

An alternative to the approach to science and evidence employed by the IAAF would have been to provide research funding to an independent body which could request proposals from researchers unaffiliated with the IAAF to address the scientific questions at issue.11 We would not find it appropriate for cigarette companies to provide the scientific basis for the regulation of smoking or oil companies to provide the scientific basis for regulation of fossil fuels. Sport regulation should be held to the same high standards that we expect of researchers in other settings where science informs regulation and policy.

We believe that a comparison of the statements made by IAAF with our analysis can only conclude that IAAF is failing to uphold basic standards of scientific integrity that should be expected in such an important matter that affects global sport and individuals’ lives.  We should all expect better.

Prof Roger Pielke Jr

Ross Tucker

Prof Erik Boye

Prepare for a long read. Do it in instalments if it helps! I’ve written it as a list, I don’t know if that helps or hinders. But these are my thoughts, start to finish, on the DSD in sport issue, as pertains to Caster Semenya and others, and the CAS decision that ruled in favour of the IAAF’s regulations.

The decision was a disappointment to me, because of the scientific matters and in particular the scientific process that was followed in supporting those regulations. It is not a disappointment because of the concept of testosterone and its effects on performance. I remain sympathetic to the IAAF’s challenge in this regard, and while I think they have not applied a good scientific process to the creation of the policy, I want to be measured and say that on many elements of this issue, I think they have strong theoretical arguments.

But here is, top to bottom, a summary of what I think are the key issues here. If you want byte-sized, try someone else. This is a big meal, I hope you enjoy and digest it:

[ribbon toplink=true]1. The evidence issue – a historical context to how we got here and a watershed moment[/ribbon]

1. In my reading of the situation, there is insufficient evidence to support the regulations. There is concept, theory, and biological rationale, but not evidence. And the regulations are, for all intents and purposes, an evidence-based document.  Their foundation is evidence, because the regulations in their current form evolved out of a mandate received from to return with evidence supporting them. 
2. You’ll recall that the Chand decision mandated the IAAF to go away and come back with evidence of the MAGNITUDE of the advantage, and they said on numerous occasions that the numbers matter.  There is thus a DIRECT line between the instruction “Provide evidence” and what is now in place as a regulation
3. Therefore, that Chand decision fundamentally reframed the debate – it moved it from being conceptual in nature, to being conceptual PLUS evidence.  
4. At Chand, the arguments were theoretical: “Here is a model for testosterone and what it does to performance” as the argument by Group A.  Group B responded with “It doesn’t matter, because this is a natural advantage and should not be regulated”, along with human rights arguments.  The former (Group A), is in my opinion a stronger position that Group B, but the point is that neither side had evidence.
5. Post-Chand, the arguments must be theoretical supported by scientific research: “Here is the evidence, it means that our model is correct”.  The Chand decision is thus the first “watershed” moment, for me personally, and I believe for the case.
6. And so on the matter of where I stand now, it is the evidence that absolutely has to be appraised.  We have to focus on it.
7. First we have to identify which evidence to look at. The only evidence worth considering here is direct. That is, there are a few studies where T is given to women and they improve, or reduced and people get worse. This is ‘surrogate’ evidence, indirectly showing how T improves performance (which is really obvious). I don’t discount that, but the issue here is not women who raise their T, it’s women who have elevated levels of T normally and chronically.
8. In that regard, there is not a single well controlled study on 46 XY DSDs that looks at performance benefits. There is one study that looks at how the performances of THREE athletes declines when they lower T either with medicines or through surgical removal of testes. That suggests 5%. This is interesting, and compelling, aside from the fact that there are only three athletes in the group, and it’s limited to observation without any level of control over crucial confounding factors. More numbers, and better control over confounding factors would shift the needle a lot in favour of the concept of performance advantage in DSDs.
9. The other study is the infamous original Bermon & Garnier paper where the IAAF took its female athletes and related their performances to their T levels. This is the one we called for to be retracted, because we felt it was that bad, as I discuss below.
10. This study was on T levels and performance in all women, including DSDs. It is NOT a study on DSDs per se, but nevertheless, is crucial here because this is the study that determined that the 400m, 400m H, 800m, 1500m and Mile events are on the list of events covered by the Regulation. So, with the nuance and limitations, we have to assess what this means for the DSD issue.
11. In this regard, my personal opinion is that this evidence is substantially lacking, to the point of being fatally flawed, for a few reasons:
    1. It finds evidence of a small advantage due to higher T levels, but only in a very narrow range of events, This is absolutely vital, because it creates a very important “internal contradiction” or paradox in the IAAF’s argument – it sets their concept up AGAINST their evidence, and I think this alone should be enough to reject it.  More on that later.
    2. When they found that performance was related to T levels in only 5 events, rather than concluding that they’d shown evidence for a performance benefit of T, they could just as easily have concluded that there is no evidence for T in 17 out of 22 events in women’s athletics. Position determines perspective, and all that…
    3. The methods used to gather the data are not high enough in quality to be supported as scientifically robust.  The sampling time, the control for other factors like the long flight times, menstrual cycle and use of oral contraceptives etc negate any possible interpretation of the set of studies. The performance side of the association was similarly uncontrolled, and some athletes with massive under-performances are including in the cohort, confounding it significantly.  That same study, repeated again, could show a totally different thing, because the confounders to T levels and the performances were not even acknowledged, let alone controlled for or evaluated
    4. The statistics used are highly, highly dubious.  Indeed, it’s not even solely that they used bad stats – their data is bad, and simply cannot be trusted.  This is why we (Pielke, Boye and myself) called for its retraction last year from May onwards.  When we analysed a portion of the data, we found errors, to the point of non-existent “phantom” data points.  Even after catching and apparently correcting those, there has never been full transparency on what now exists in literature. Calling for a retraction is a massive step in academia, but we believe it was warranted, because the scientific study that ultimately informs the selection of events covered by the Regulations was so bad that it should never have been published.  It is fatally flawed
    5. On this evidence, I must re-emphasise that the study by Bermon et al mentioned above did NOT look at 46 XY DSDs vs 46 XX athletes.  It looked at high T vs low T in a group of women, which included some 46 XY DSDs. So one can argue that the study was not intended to answer the DSD question.  That’s fine, but it is being used to guide the regulation, because it is THAT study that identified the 400m, 800m and 400m hurdles events as the ones where the advantage exists (the 1500m/mile were added later, despite no evidence).  So again, there’s a direct line between the study and what made it into the regulations that were challenged (with one or two changes – they added the 1500m and removed hammer and pole vault)
    6. I think though, that when you have a regulation based on a study that actually assessed something subtly different, that’s a problem in itself.  It requires that you extend the study findings to a new group, data errors and all, and assume that what is “true” in one group will be true in another.  Some will argue this is not all that large a stretch, and there’s some merit to that, given the way it’s being extended, but it still lacks evidence.
12. So now given that CAS has mandated the IAAF to provide evidence of magnitude (the numbers matter, recall) and that the evidence provided by the IAAF falls way, way short of the standard, robustness, quality and magnitude required, I think the case should have been over.  Perhaps another mandate, one to keep looking, but on the science, I think a solid foundation for the Regulations was non-existent
13. The reason this matters is because the size of any advantage (and the quality of the research that provides it) is crucial to determining what should be allowed in terms of ‘intervention’.  If there is any risk of harm (and there is), then you need a really compelling case to justify it.  A compelling case means evidence of a large advantage.  
14. So my position is that the regulation as it stands should not have been defended by CAS.  

[ribbon toplink=true]2. Into the grey – the concept of biological advantage that is unfair and created by testosterone, and the line between men and women[/ribbon]

That said, there is a distinction here, in my opinion, between the EVIDENCE and the CONCEPT.  This is something I’ve long argued, and I want to address the CONCEPTS briefly:

1. Men’s and women’s sport exists separately for a reason, and nobody reasonable should dispute that the separation the way it is exists is important.  The separation cannot be “at all costs” – it’s just sport, after all, and so at the point where this separation starts to harm people, in a real way,  I think everyone needs to step back and take a look at their values.  It’s not about “all costs”, but it is reasonable for sport to manage the separation fairly, provided it’s objective and guided by some reasonable process and rationale.
2. Sports authorities have every right to defend a line that is drawn, and so in the same way that sports should measure the mass of boxers before they fight, or the degree of function in athletes with Cerebral Palsy before they let them compete, sports have a certain obligation to “protect” the categories they create, however arbitrary they may seem (and sex separation is not arbitrary!).  The issue here is whether that protection is objective, non-harmful and based on accurate enough variables so as to not create a net negative outcome
3. I do NOT believe this is primarily a human rights issue. I’ve never been hugely fond of this argument, partly because it is too emotional, partly because it is unnecessary, and partly because I don’t totally agree with the way it is made by most people. 
4. That said, when I read the media interviews with the likes of Chand and Semenya, I can see that they have had their dignity harmed, their rights to privacy taken away and that is arguably a human rights issue.  In my opinion, these consequences relate more to the way the regulation is implemented than to the regulation itself.  There is little upside for them, on a human level, when things go so wrong and confidentiality fails.  However, I am coming at this from a sporting perspective, and so while I would not discount that, I weight it less when I am trying to evaluate the Regulation’s content (not its enactment).  And my feeling is that no person has a right to be an ELITE athlete and to make money and win titles in sport.  We all have a right to a level of physical activity and sports participation, but not necessarily to enter the elite sports world.  
5. Back to the root issue, the reason for the separation between men and women is the effect of androgens on the physiological systems that are germane to performance.  One can debate for hours how predictive high levels of T are for performance, and there is without doubt a lot of poor thinking in this space – when people argue that high T does not guarantee performance and thus T is not the difference between men and women, it makes me want to bang my head repeatedly on a desk.  It is a poor argument, appallingly lazy and disingenuous.  
6. Fundamentally, the difference between the men’s world champion and the women’s world champion in any given event is ‘androgenisation’ or virilization that drives a set of secondary sex characteristics including higher muscle mass, lower fat mass, greater strength, increased cardiovascular capacity and function, more haemoglobin, different skeleton.  There are of course other changes, but they are not germane to sport
7. When discussing T and performance, it is important, in my opinion, that the issue is framed appropriately.  By this, I mean that one should NOT be asking whether women with high T outperform women with low T, or that men with high T are better athletically than men with low T.  This approach creates numerous “loopholes” and erroneous thinking, the main one being that you can explore this relationship, NOT find it, and then conclude that T is not important.  This is what many academics have done – they will look for associations between T and performance in a group of athletes, all men or all women, and then find no association, thus concluding that T has no effect.  
8. The problem with this argument is that when you look at a relatively ‘homogenous’ group, like women, or men, who exist in those groups because of the effects of that variable (T, in this case), then the effect of that variable disappears, because they’re all matched for it already!  For instance, in a group of elite marathon runners, VO2max will have poor predictive value for performance.  But that’s because they all have high VO2maxes to begin with!  If you were “stupid” about how you frame your search for VO2max and performance, you’d conclude that it has no impact on performance.  This is what people are doing with testosterone in men vs women.
9. So, what should be asked is whether VO2max, in the example above, differs in the elite group compared to the general population, and then it becomes obvious that it is crucial.
10. So too, the point about testosterone is not that it guarantees better athletic performance, but rather that it creates the potential, in the right person (Athlete A), for a performance advantage that is so large that a person who is identical in every respect with the exception of testosterone (Athlete B) will be 10% to 12% slower than Athlete A.
11. In this regard, Testosterone is the ticket that gets you through the door, into the party.  It doesn’t determine what happens once inside, because once in, you’re part of a select group (50% of the world, give or take, in this case), who have the POTENTIAL or the opportunity to achieve XYZ% advantages.  But it’s totally false to say it doesn’t matter, when in fact it’s the thing that gets you started!
12. So the use of testosterone as a differentiator between men and women is reasonable, in my opinion, but imperfect.  Anyone who disagrees on the “imperfect” nature probably doesn’t understand the concept of how testosterone is responsible for the difference between two otherwise matched human beings.  
13. One of the big problems with using T is that the concentration of T alone doesn’t tell you how much effect it has on the body – the same level in two human beings can have different effects or outcomes.  And the same outcomes could occur despite different levels of T.  Ultimately, one might one day be able to quantify an overall “virilization sensitivity index” and combine the amount and the sensitive to understand exactly how much influence T has. This might one day look like the methods used to assess insulin sensitivity, but we are not there yet.
14. Another problematic argument in this area of concept is the notion that having a high T value is a natural advantage, and should thus be ignored, because after all, we don’t regulate things like height in basketball or foot size and arm length in swimming. I’ve seen academics make this argument, and I don’t agree with it. I want to talk briefly about why this is problematic, and ultimately, possibly damaging to the sport, if these people were to win their case:
    1. First, we don’t compete in categories of short people in basketball or people with small feet and short arms in swimming.  This is important, because our decision NOT to “protect” the people with these “disadvantages” means that we don’t need to concern ourselves with things like feet size or arm length. It’s irrelevant. If we decided to police foot size in swimmers, maybe it would become relevant.
    2. But, and here’s the thing, we HAVE decided to protect the competitive integrity of women’s sport. We do this because we understand that in two athletes who are equal in every respect except for biological sex, the effects of testosterone in the biological male create such a large disadvantage for the athlete without those effects (biological female), that they would vanish from sport altogether. Thus, having drawn a line, we must defend the line. 
    3. Perhaps one can make a reasonable argument that we SHOULD defend a line of height in basketball – short people, below 1.81m.  But then, someone who is 1.815m tall can’t play down.  That would be easy to do because height is easy to measure.  Sex unfortunately is not, but the point remains, we decide somewhat arbitrarily to create categories.
    4. I also reject this comparison because I guarantee you that having small feet or short arms is not as much of a disadvantage as having no benefit from testosterone.  In other words, these “disadvantages” are not equal in size to that of being female in sport.  If all things are equal except for Variable A, then the person with smaller feet is not going to be 4000^th in the world rankings for humans.  But when all things are normal except for Testosterone, that person would be. 
    5. Qualitatively and quantitively, then, testosterone differs from these commonly used arguments. I don’t know where I’d put things like muscle fiber type in this conversation – it may have the same effect size as “androgenization” due to high T, but I don’t know.  Regardless, it’s moot because again, we don’t have an Olympic champion for “slow-twich muscles under 60%” and another for “fast-twitch muscles over 60%”.  If we created that category for those slow-pokes, then we’d have to concern ourselves with this argument
    6. Finally, this argument that says “Don’t regulate natural advantages” ends in a disaster for women’s sport.  Why?  Because being “male” is a natural genetic advantage! It may be the greatest of them all! If you have the Y chromosome, and the testes, and the testosterone, and you can use the testosterone, then you have a huge advantage compared to a woman who has all the same other attributes, but not those.  So if you really want to get rid of regulation and separation based on “natural genetic advantages”, then you should just as well throw all humans into one race, and crown the “World’s fastest human”, and see how women get on.
15. So, to wrap up the concept – sport has a right or even obligation to manage the male vs female issue.  Doing so on the basis of testosterone is reasonable IN THEORY, though it does demand certain “nuance’ that I don’t know is always recognised by both sides of this issue.

[ribbon toplink=true]3. DSDs and the male-female spectrum – why you shouldn’t just treat a DSD as a biological male[/ribbon]

1. Note that this is not the same thing as the DSD individual vs female issue.  A 46XY DSD is not necessarily biologically male – this, indeed, is a point of debate in this issue.  They DO lie on a spectrum, and since I’m interested in performance, you can pin that performance advantage of having a condition causing a 46 XY DSD on a spectrum of 0% to say, 12% (the full male advantage).  But at this point, knowing where to drop that pin is the problem. 
2. The very nature of a DSD is that they are not able to use male hormones in the “typical way” – this is why they exist as DSDs, rather than as men or women, clean and simple.  So when we apply everything I’ve written above, we must just be mindful of the distinction, and the reality that if someone is born “female enough” (I hate that word, but I can’t think of a simpler qualifier right now) to be identified as a baby girl, raised as a girl, becoming a woman, there is already some “atypical” or different physiology that may have performance implications
3. A final point on concept – the way one words the testosterone advantage is quite important.  I need to explain this
   1. If you say, with confidence, “athletes with high testosterone levels have performance advantages”, you are being more prescriptive than biological complexity allows.  What you should say if you are being precise is that “Athletes with high testosterone levels have a potential biological advantage that MAY create an insurmountable advantage when comparing two athletes who are matched in every respect other than having high T levels”.  This may sound semantic, but I think it’s quite important.  
   2. The reason this matters is because of how it sets up a discussion around transgender male to female athletes, by the way.  The moment you commit to a “athletes DO have an advantage when their testosterone levels are high”, then you create what is an easily falsifiable theory – all one has to do is to find a case of a group of people who have high T and are not good athletes, compared to a group who has low T and ARE good athletes, and you can say “This group of people disproves that T is significant for performance”.  This is what a lot of academics have done (my head on desk banging days testify to this!).  It’s much better, and much more accurate, to say “Testosterone MAY drive the biological factors that confer advantages that no other characteristics can match, such that two individuals who differ only with respects to T may be 10% to 12% apart in performance”

OK so that’s the concept.

[ribbon toplink=true]4. Appreciating that DSDs are not created equal, and the implications for performance advantages[/ribbon]

What you have now is a concept, which I think is theoretically sound, and then you have the requirement for EVIDENCE.  And this then brings the discussion back to where I began above – the evidence.  So:

1. IN THIS GROUP OF ATHLETES with DSDs, who are not men who transition to women as occurs for trans athletes, but are people who were “female enough” to be identified as girls at birth, are raised as girls, start competing as girls and then enter women’s sport, is there evidence of this “insurmountable advantage”?
2. One of the other conceptual problems, in my opinion, is that you have a policy that covers a collection of conditions, but there is no theoretical basis to think that all those conditions would confer an advantage.  So even BEFORE evidence is considered, the theory stumbles a little bit.  That is, DSDs in a term describing a collection of conditions, and in the IAAF regulation there are five such conditions identified. Only some of those are documented in sport, so we can consider them in turn to understand why there is a scientific problem here that the concepts cannot deal with:
   1. Some of the conditions causing DSDs almost certainly do NOT give an advantage, because the person who has high T levels is completely insensitive to it.  So in this condition, you have someone who is XY, produces testes that produce and release testosterone, but that testosterone is like a key that doesn’t fit the lock, because the body’s receptors are insensitive to it. All the keys in the world won’t open the door.  That’s why they develop as female even though they have internal testes and high T levels.
   2. But they’re COMPLETELY insensitive.  Now you can nudge the needle one degree over, and you land on a condition that is called PARTIAL Androgen Insensitivity.  Here, the person can use the T, but only partly.  I would say that by definition, this person cannot be managed by the same regulation as our previous case who is completely insensitive to androgens.  But you have a real problem here, because you have to ask “How partial does it need to be before it becomes unfair?”.  If you’re only partly able to use the T, then are you getting the advantage?  Arguably not, but where do you put the point on the spectrum?  From a scientific perspective, there’s a dilemma here. 
   3. I can’t see how these two conditions (CAIS and PAIS) an be treated the same as one another.  Then you go one step further, to Alpha-5-reductase deficiency, which is a condition where a person can fully use testosterone, but cannot convert it to another hormone called DHT.  In the absence of DHT, the external genitalia do not develop in the typical way, and so you have ambiguous or female appearance at birth, but potential for testosterone’s benefits later, particularly post adolescence.  This is a lot trickier to handle, but the scientific issue here is whether DHT has any effects on any physiology that may affect performance?  For instance, might it be that DHT influences fat mass, muscle mass and muscle strength?  If so, then a person who lacks it has disadvantages, and now you have to weight that disadvantage against the theoretical benefit of using Testosterone.  Which itself may be used more in some people than others.   
3. The point here is that the collective regulation of a number of different conditions creates conceptual problems, because the conditions that are grouped together as DSDs don’t all present the same.  The potential performance enhancement from each ranges from 0% to the full 12%.
4. If you start from the position that a 46 XY DSD is “fully biologically male”, then you’re saying that the potential performance advantage is going to be 12%. But you’re wrong, because it’s clear that some are much less, and so your argument absolutely must be more nuanced than this. And that means the same question “Is there an advantage?” cannot be answered for them at once
5. Ultimately, I think that if a regulation is to exist, it should actually be narrower, and cover specific conditions, not a catch-all for what eventually appears as a symptom (having high T levels

[ribbon toplink=true]5. Where the concept meets the legal system[/ribbon]

1. Back to the question – is there evidence for a biological-male like advantage in performance in a DSD woman?  Possibly.  In fact, and let me be clear about this – I think that in many cases, especially the PAIS and the Alpha-5-Reductase cases, there IS likely a biological difference due to T, and it almost certainly is worth a performance benefit.  However, in the absence of knowing its size – where on that spectrum should we drop the pin? I find it very difficult to assess what to do with the advantage.
2. Also, in reality, there is no evidence to support the notion (even if the notion is very sound, theoretically).  It hasn’t been studied – even the Bermon study was looking at high T vs low T within women, not 46XY DSDs vs 46XX women.  So the use of that study itself poses some flaws, but it’s so crucial to the Regulations that it must be evaluated.  How do we know it is crucial to the evaluation?  Well, because the events that are covered by the Regulation are the 400m, 400m H, 800m and 1500m, and those are the events found in the Bermon study to have the advantage when T is higher.  The 1500m was added later, which is another matter, but the point is, the Regulations are an evidence led document.
3. So we proceed with the earlier evaluation of that research.  The advantage demonstrated by the IAAF’s research is trivial in only some events, similar in nature to what you’d find if you compared the groups by muscle fiber type, or height probably, or enzyme content etc.  That magnitude of advantage needed to be higher for the foundations to be solid enough.  I remember when the study came out, and I read it, thinking “Is that all they got?  This study design, this result, in this handful of events?  Is that it?”
4. The fact that the performance advantage, admittedly small, existed in only a few events is absolutely crucial, because of that paradox it creates.  Think about it:
   1. The concept being put forward is that testosterone is responsible for biological differences that drive performance advantages between men and women.  
   2. This MUST be true across the full spectrum of events – testosterone does not discriminate.  You see this in the records.  The gap is between 10% and 12% for every running event from 100m to marathon, and for jump events.  Whether the physical effort lasts 2 seconds or 2 hours, the male-female gap exists from short to long on that spectrum.  Testosterone does not discriminate.
   3. Similarly, when you look at the doping practices engaged by eastern Bloc countries and the USA in the 1980s, you see performance improvements everywhere. Not just in the 400m, 800m and 1500m. Everywhere. From 100m to marathon, from swimming to running. Testosterone does not discriminate
   4. So when you then find a difference from high T in some events only, there’s an internal contradiction, because the evidence (or lack thereof) creates a tension and actually disproves your own model!
   5. The way this paradox plays out is particularly bizarre, and highlights the issue. As it stands, an Athlete would be ELIGIBLE on a Friday and INELIGIBLE on a Saturday simply because they’re running different events on those two days! The same person. They might be in the 200m on Friday, and the 400m on Saturday. Or the 3000m on Friday and the 1500m on Saturday.
   6. This situation is, to be blunt, whacky. The same athlete, who is deemed ineligible because they have an advantage by virtue of “male-like sporting biology” is eligible in another similar event because…um…the male-like advantages stopped being large enough? Not according to the model.
   7. Point is, the model and its evidence create an internal tension that either negates the model or the evidence!
5. In defence, and one thing I have realised in this debate is that there is ALWAYS a counterpoint, the IAAF might here argue that the reason they haven’t found the effect in other events is because there haven’t been enough athletes with the condition in those events, whereas in the 800m, for instance, they have some.   OK, fine, but then there is a question – why would the DSD group not be found across the spectrum too?  If this argument is true (and we don’t know if it is the case, other than that the Bermon study allowed one to identify where the half a dozen athletes were with high T levels), then DSDs should be present everywhere too.  So this undermines the argument.  Rather than supporting the regulation, the Bermon evidence actually suggests there’s a problem with the model or concept.
6. I don’t believe there is a problem when it comes to biological male vs biological female, however. I think this “paradox” is evident when one compares DSD to female.

[ribbon]6. First, do no harm – the medical ethics of the Regulation[/ribbon]

Finally, the issue of harm.  This was my second “watershed” in terms of my feeling on the policy.  The first was when I saw the Bermon evidence, and how weak it was, and then when we tried to analyse it and basically ran up against a wall of ‘opaqueness’ and non-transparency.  

The second is since I got involved, discussing with various medical professionals about the issues related to the medical treatment necessary to compete.

1. Recently, the World Medical Association declared in a statement that any doctor who complied with the policy would be in violation of medical ethics.  What they are in effect saying is that doctors should be very careful about complying with the regulation. That seems to me a fairly big deal, making the Regulations un-implementable.
2. Indeed, in the 10 min after writing this, I read that the WMA will urge doctors NOT to comply with the Regulation. They have said: “The World Medical Association has reiterated its advice to physicians around the world to take no part in implementing new eligibility regulations for classifying female athletes.” Whether or not this carries the weight to actually stop implementation, I do not know.
3. The basis for the WMA opinion is that the regulation would require a person previously healthy to start using medication, possibly in super high dosages, for a purpose it is not intended.  That off-label use of a drug, without clear signs of safety or management, was thought that by WMA group to be untenable.
4. The doctors I have engaged with seem to agree, and they find the concept of turning a healthy athlete into a patient difficult to understand.  An oath to protect the health and beneficence of the patient is challenged by the Regulation that compels off label use of medications in unproven dosages and without studies for efficacy and safety. The athlete has little choice here, so there is an element of co-ercion, but the net result is the possibility of harm (thrombolytic events being the main one, especially in athletes for whom air travel is a necessary risk factor).  This is where proportionality comes in.
5. In this entire debate, as I said above, it seems to me that every single point has a counter point, such that both sides often claim the same thing as evidence supporting them!  It really is remarkable that way.  There is one exception, though, and this is it – there is no counter argument to the harm argument, and by harm, I mean physical harm, as well as the psychological harm that I touched on previously.  Some of the most compelling testimony in the Chand decision came from those involved on the social side.  I think that medicine and science has an obligation to pay attention to those things – they are medical and ethical rights, if not human rights.  So I know I said up top that to me it’s not a human rights issue, as in a “right to compete”.  I do think, though, there are arguments for medical and ethical rights that are very much in play.
6. The combination between the weak evidence for the advantage, and the possibility of harm, in my opinion, is why this policy should not have survived Semenya’s challenge

[ribbon toplink=true]7. The bottom line: Concept and theory vs Evidence and Factual data. The winner is…[/ribbon]

1. The decision of CAS basically comes down to whether the CONCEPTS described above outweigh the EVIDENCE problems described above.  It was always going to be that way.  In the Chand decision, CAS recognized that there was insufficient evidence, and they took the sensible option of asking for it.  Now, in 2019, we’re in a situation where one party is saying “Right, here’s the evidence we needed”, and the other is saying “that’s nowhere near enough”.  It is apparent from the outcome that the CAS have sided with the former party, and have decided the evidence is sufficient.
2. It would seem to me that had this been based on the quality of evidence, the integrity of the scientific process, and procedural integrity, rather than the biological concepts for testosterone and performance advantage when comparing biological males to females, the outcome arrived at would not be possible.
3. The key points of the decision summarized in the press statement and the summary, have an element of schizophrenia about them.  It’s interesting to me that when the Chand decision was granted, it felt like they were saying “Your concept is sound, we support it, but we see no evidence, so that’s what is missing”.  And here, it seems a mirror image, as if they’re saying “The evidence is there (I dispute this, of course), but there are some issues about the concept and its implementation”.
4. So it seems different, and I’m not sure why. I do wonder whether the growing concerns about the Transgender Regulations, and the fear about men identifying as and competing as women, has infiltrated the thinking. Well known scientist, doctor, sex and gender expert, and lawyer, Paula Radcliffe, has certainly been trying hard to conflate the issues when she can. I wonder if the prevailing wind direction shifted between 2015 and now, such that fear of a “trans-invasion” has coloured the way we view biological male, biological female, and ambiguous cases?
5. In the CAS decision, it seems that there is a lot there that they are concerned about – the inclusion of 1500m despite no evidence of a performance difference is the obvious one where CAS has suggested the IAAF to reconsider.  I doubt they will – having sat down and consciously added the 1500m to the list of events covered by the Regulations, I can’t see the IAAF saying “Alright then, since you asked nicely”.  And the other area of interest is the notion that if new evidence emerges regarding harm, then the IAAF might decide change the situation based on the observed risks and harm to the athletes.
6. It is important to note that this and any other decision on this Regulation is theirs and theirs alone to make. CAS has no further role in the Regulations, and the IAAF now control everything that hpapens. The “living document’ of the decision allows them to add, remove or change things as they see fit, without oversight and presumably without option for challenge by the athletes affected. This is a major win for the IAAF.
7. Bottom line – CAS heard and recognized all the things I have described as concerns – the weak evidence, the Bermon study’s failings, the risk of medical harm.  They simply decided that they didn’t matter as much as the theory and concept.  That seems substantively different compared to 2015/16, when they ruled on pretty much the same set of things (minus the evidence), which means that the evidence was enough to sway them.
8. That really disappoints me, on a personal and procedural level.  If a study that has multiple data errors is still good enough to provide a reasonable support to a discriminatory policy (their words, not mine), then we really failed to make the case.  Or, the quality of the evidence just didn’t matter much to them, and we failed to convince them of it.
   
   It comes down to what you weight more – the concept of the advantage, or the lack of evidence for it.  They went for the former.  The end.

Ross