Inside School Research

Key Federal Officials Blogging on Innovation in Education

2009-11-20T13:31:01Z

Over at the National Journal's education blog, two top federal education officials have started a lively conversation on the role of innovation in education. The questions being posed by John Easton, the director of the department's Institute of Education Sciences, and Jim Shelton, the chief of the office of innovation and improvement, are:

What are the essential components of an effective innovation, research, development, and dissemination infrastructure in education? How can we tap into the collective expertise of practitioners when designing and refining new school programs? Finally, what are the capabilities that need to exist at the local, state, and national levels and how should organizations that provide them fit together into a coherent whole?

The 17 responses so far run the gamut. Sandy Kress, President Bush's former education adviser, argues for keeping rigorous research at the center of federal innovation efforts. Diane Ravitch, co-author of the Bridging Differences blog here at edweek.org, contends that too many education reforms are being imposed on schools by non-educators. Chad Wick, the CEO of KnowledgeWorks, blames the lack of a shared vision for education for the failure of the reforms tried so far. You'll also find a pitch for charter schools, a call for more market-based incentives in education, and a critique of the focus on student test scores as a sole measure of education success in many new initiatives.

This is no idle conversation, though. Shelton's office is overseeing a new grant program, funded with $650 million in economic-stimulus dollars, to promote and scale up innovative education strategies. Likewise, Easton's office will soon be fixing new priorities for the research that gets funded by his institute and setting a new direction for the 10 education laboratories around the country that work with states and districts. The federal Education Sciences Reform Act, the law that gave birth to the institute, is also overdue to be reauthorized. Some of the ideas from this conversation may find their way into federal policy.

Penn GSE, Milken Competition to Focus on Entrepreneurship in Education

2009-11-19T13:45:03Z

Graduate business schools often hold competitions to nurture the development of innovative business plans. But when was the last time you heard about a business-plan competition geared entirely to educational entrepreneurship? Probably never—at least not until now.

A blog post on Eduwonk alerted us yesterday to this upcoming competition being sponsored by the University of Pennsylvania's Graduate School of Education and the Milken Family Foundation. The first annual Milken-PennGSE education business-plan competition will award one prize of $25,0000 and another of $15,000 for the best business plans aimed at addressing educational problems that crop up at any point from cradle to grave. The winning plans will be required to outline the problem, its proposed solution, and possibilities for scaling up their innovations.

A description of the competition posted on the graduate school's Web site yesterday notes that:

"Despite the size and import of learning, we have immense challenges; many hold out the hope of entrepreneurship to help solve some of these challenges. That said, until now there hasn't been a single education business plan competition in the world and while entrepreneurship in bio tech, software, engineering and medicine is quite robust with clusters of start ups surrounding some of the world's great universities, nothing similar exists in education.

Doug Lynch, the school's vice dean, said the competition grew out of an invitation-only meeting the university held over the summer to brainstorm for ways to promote more innovation in education. The 13 judges, who include a wide range of corporate CEOs and foundation officers, include many of the participants from that event. (Eduwonk is one, too.) Submissions can describe either nonprofit or for-profit ventures and competitors have until Jan. 15 to signal their intent to participate. Winners will be announced in May.

New Software Program Uses Brain Science to Motivate Students

2009-11-17T21:11:52Z

More than three decades of research has shown that, when it comes to academic achievement, children who focus on effort tend to be more successful than those who focus on innate ability. The problem, though, is that many kids decide early in life that more effort isn't, well, worth the effort. They believe people are either born smart or dumb and that no amount of work is going to change that situation. In the face of such persistent beliefs, how do you motivate kids to try harder?

Stanford University psychologist Carol S. Dweck and her colleague, Lisa Sorich Blackwell, have hit on one possible solution. They developed a software program, called "Brainology," that uses brain science to persuade middle school students that intelligence is a malleable, rather than a fixed trait.

Studies have shown that it seems to help. Compared with peers in traditional classrooms, middle school students who used the program improved their motivation to work harder in school and, over time, their achievement as well. (You may have read about some of that related research in this space before. A good, readable summary of all the research that led to the development of this program can also be found in this 2007 article from Scientific American.)

What's new is that, starting today, the program will be available on the commercial market. Designed to be "like an owner's manual for your brain," the software teaches middle schoolers that when people practice and learn new skills, the areas of the brain responsible for those skills become larger and denser with neural tissue, and that new areas of the brain become active when performing related tasks. They're taught that the brain continues to grow nerve cells, or neurons, daily, and that this process speeds up when a lot of active learning is occurring.

Marketed by a for-profit company called Mindset Works of San Carlos, Calif., the program may be a bit pricey from some: It costs $99 for one child for families, and less for siblings, and $20 a head for educators who order 20 or more programs. Could a skilled, well-informed teacher get the same results for free? Possibly.

Vanderbilt Scholar Tapped for New Bush Think Tank

2009-11-17T17:51:40Z

Here's a tidbit that you may have missed: Vanderbilt University scholar James Guthrie has been tapped to direct education policy studies at former President George W. Bush's new think tank.

Based at Southern Methodist University in Dallas, the George W. Bush Institute will be part of the Bush Presidential Center, which also includes Bush's presidential library and museum. Besides education policy, the center will focus on global health, human freedom, and economic growth. Read all about it in the Houston Chronicle and in these press releases from Vanderbilt and SMU.

Guthrie currently directs the Center for Education Policy at Vanderbilt, where he specializes in studying improving school leadership. At the Bush institute, he'll be teaming up with Sandy Kress, who was Bush's campaign adviser on education, to continue that line of inquiry.

Construction on the center isn't expected to begin until fall, but Guthrie and Kress are already planning a national conference on education leadership, policy, and school reform at SMU for this March.

Texas Districts, Scholars Form Research Consortium

2009-11-16T16:20:42Z

Researchers and practiioners from Dallas, El Paso, Houston, and 16 other Texas school districts got together last month to formally kick off the formation of the Texas Consortium on School Research.

Modeled after the 19-year-old Consortium on Chicago School Research, the new group enlists university academics, local educators, and administrators to work together on research projects that address the common, real-world problems that school districts have. Over time, they hope to create a "community of practice" for sharing findings and strategies. The Texas consortium's research partners are the University of Texas at Dallas Education Research Center, the Regional Education Laboratory-Southwest, which is one of the 10 regional education laboratories funded by the federal government, and the Chicago consortium.

The Texas group is the latest of several district-university research partnerships to get off the ground: Recall that similar efforts have been launched in Baltimore, Newark, N.J., and New York City. And the Chicago group hosted a conference over the summer to help nurture some of those burgeoning efforts across the country, which you can read more about here.

Do I spot a trend?

Charter School Research: The Beat(ing) Goes On

2009-11-13T13:45:44Z

The back-and-forth on charter school research never seems to end.

The latest analysis takes to task the much-publicized study of New York City's charter schools that was conducted by Stanford University economist Caroline M. Hoxby and colleagues. (You can read more about the original study in this EdWeek article. Read the full text of the Hoxby study here.)

In the new critique, Sean F. Reardon, a colleague of Hoxby's at Stanford, points up what he sees as flaws in the New York City study. For one, he says, in measuring the effects of charter schooling on students in grades 4-12, the study relies on statistical models that include test scores from the previous year, which are measured after the admissions lotteries take place. Because of that timing, he says, the scores could be affected by whether students attend a charter school, which "destroys the benefits of the randomization" that is a strength of Hoxby's study.

Reardon, a research methodologist, also faults the Hoxby study for "inappropriately" extrapolating data on students' achievement gains between kindergarten and 8th grade to calculate learning benefits over their entire school careers and raises some other, equally technical, issues as well.

Reardon's bottom line: While it makes an important research contribution, the report "likely overstates the effects of New York City charter schools on students' cumulative achievement, though it is not possible—given the information missing from the report—to precisely quantify the extent of overestimation."

You can find the full paper at the Web site for the Think Tank Review Project, which posted it on Thursday. The project, which specializes in reviewing studies in the news on hot-button education issues, is a joint project of the University Colorado at Boulder Education and the Public Interest Center and the Arizona State University Education Policy Research Unit.

And here's a side note: If you think nobody pays attention to all these studies on charter schools, think again. The final guidance issued by the federal Education Department this week for its $4 billion Race to the Top program indirectly references some of that research in relaxing language, in the draft guidelines, that many had interpreted to be an endorsement of charter schools as the chief remedy for failing schools. The new language reads:

"Notwithstanding research showing that charter schools on average perform similarly to traditional public schools, a growing body of evidence suggests that high-quality charter schools can be powerful forces for increasing student achievement, closing achievement gaps, and spurring educational innovation.

That first clause sounds to me like a reference to a national study released over the summer by the Center for Research on Education Outcomes, or CREDO, which is also based a Stanford. The wording also partly explains, in a nutshell, why so much of the research on charter schools seems to conflict. Some charter schools are very good and some, not.

Is Head Start a Good Investment? A Prominent Economist Weighs In

2009-11-11T12:58:35Z

A lot of policymakers and educators have been drawing hope from recent studies of the High/Scope Perry Preschool Program. If you haven't heard of it, the Perry Preschool project was an early-intervention program for children from disadvantaged families that was run out of Perry Elementary School in Ypsilanti, Mich., in the 1960s. A forerunner of the federal Head Start program, the project has long been considered a success.

Recent re-analyses of program data show that, from a public-investment perspective, it provided a pretty good bang for the buck, too. Those studies estimate the rates of return to be about 16 percent to 17 percent. You'd be hard-pressed to find a checking account or a certificate of deposit that pays that well. The rate is high in part because, compared with nonparticipants, fewer of the program grads required special education services or ended up in prison later on in life.

In a new working paper, however, Nobel Prize-winning economist James J. Heckman and colleagues suggest those earlier estimates might have been a little too generous. In their paper, which was posted online this month at the Web site for the National Bureau of Economic Research, the economists argue that the previous analyses were flawed because of problems in the randomization process for the original experiment, missing data and costs, and a failure to calculate standard errors.

When they account statistically for all those issues, they come up with a rate of return that falls somewhere between 7 and 10 percent. That's still statistically better than zero, they maintain. But it's not nearly as exciting as 16 or 17 percent. Look for this study to figure heavily in debates over President Obama's plans to expand the federal Head Start program.

Research Advisory Board Wants a Higher Bar for Innovation Grants

2009-11-10T15:36:00Z

The national board that advises the U.S. Department of Education on its research operations voted to weigh in yesterday on the proposed rules for the new Investing in Innovation or "i3" program—and just in the nick of time.

The window for commenting on draft guidelines for the $650 million grant program was scheduled to close at midnight last night, but the six members of the National Board for Education Sciences, at their meeting earlier in the day, approved a resolution that calls for strengthening and expanding the criteria the feds hope to use to decide how to distribute and evaluate grants through the program.

Part of the $100 billion in economic-stimulus funds for education, the i3 program is designed to spur the "next generation" of improvements in education and promote the spread of proven innovations. The biggest awards, or "scale up" grants, would be worth up to $50 million each for programs that are supported by "strong" evidence of research. A second category of "validation" grants of up to $30 million each would go to programs with a "moderate" research track record. The draft rules also call for awarding "development" grants of up to $5 million each for programs with "reasonable research-based findings or theories." (For further details, see my colleague Michele McNeil's story inEdWeek.)

The problem with the draft rules, the board maintains, is that their definition of "strong evidence" and their evaluation requirements for validation grants give equal weight to random-assignment experiments and quasi-experimental studies. Only the former, which involve randomly assigning subjects to either experimental or status quo groups, are considered the "gold standard" for determining whether an intervention works. Said board vice chairman Jon Baron:

"Our thought is that awarding scale-up grants solely on the basis of quasi-experimental studies would likely lead to implementation of some programs that are not effective."

What the department ought to do instead, the board says, is express a clear preference for randomized experiments. The board is not the only group to stake out that position: The Knowledge Alliance, a Washington-based group that represents research groups, also submitted a comment calling for giving more weight to randomized research in the definition for strong evidence. That group does not, however, favor setting out the same sort of methodological hierarchy as the criteria for evaluating validation grants, according to Jim Kohlmoos, the association's president.

The board's resolution also says the program should require evaluations to include estimates of program costs, and not just benefits, and that the new fund guidelines ought to encourage ways to design evaluations so that research costs and the burdens on schools and districts are kept as low as possible.

But, as Kohlmoos points out, there's also a risk in setting the methodological criteria too high for this program: Will there be enough education programs with a research track record strong enough to qualify for the new grants? Time will tell.

The Last Annual 'Bracey Report' Is Published

2009-11-09T21:09:12Z

The last "Bracey Report on the Condition of Public Education" is now available online. Readers of this blog will recall that Gerald Bracey was working on this report the night before his death last month at the age of 69.

In this, his 18th edition of the report, he offers a critical analysis of the research behind three popular education reform pushes: the call for high-quality schools, mayoral takeovers of school districts, and the drive for higher academic standards. The report is being published jointly by the Education in the Public Interest Center at the University of Colorado and the Education Policy Research Unit at Arizona State University.

Should School Close When the Flu Hits? New Evidence for Educators

2009-11-06T16:30:54Z

A study described today in the "Science Daily" blog offers some timely advice for educators on the optimal time for shutting down schools when a flu outbreak strikes.

"You'd want to get a school closed before an epidemic peaks, to prevent transmission of the virus, but you also don't want to close a school unnecessarily," explains John Brownstein, an epidemiologist and study co-author. However, he says, most schools base those decisions on fear, expediency, or politics.

To develop some evidence-based guidance for schools, Brownstein and colleagues from the Children's Hospital Boston Informatics Program and the University of Nigata in Japan analyzed data from 54 Japanese elementary schools over four consecutive flu seasons. They tested dozens of school-closing scenarios and finally decided that the ideal early-warning trigger should be when a school has an absentee rate of 4 percent or more for two days in a row.

And, no, I don't know if this rule of thumb would hold for swine flu, which, from what I read, seems to be more contagious than the garden-variety flu virus.The Centers for Disease Control is advising against closing schools, but that's a switch from advice it gave last spring.

At any rate, you can read the full study for yourself. It appears in the November issue of a journal called Emerging Infectious Disease.

Study Shows How Discrimination Creeps Into Grading Practices

2009-11-05T15:27:35Z

A study released this month by Harvard University's Kennedy School of Government uses an innovative experimental design method to get an intricate picture of how cultural discrimination plays into the grading decisions that teachers make.

For their study, researchers Rema Hanna and Leigh Linden traveled to India, a country known for its deeply entrenched caste system. The researchers then recruited elementary and middle school-level students from all levels of society to take part in a contest. Students took a battery of tests in mathematics, language, and art and were told that the highest-scoring student in each age group would win 2,500 rupees, or $58, which is about half of their parents' average monthly income. I'd say that's pretty high stakes.

Likewise, 120 teachers from both government and private schools were recruited to grade the test and were paid about $5.80 for their efforts. The tests, however, were randomly assigned different student characteristics. One student, for instance, would be listed on a cover sheet as a member of the Brahmin caste, the highest of India's four social groups, while another might be described as being in the lowest caste, the Shudra. The tests were also graded separately by a research staff member who had no knowledge of any of the students' characteristics.

As might be expected, the results showed that teachers, on average, assigned scores to students from low castes that were 3 percent to 9 percent lower than those of students who were described as being from a high-caste group. What was particularly interesting, though, was that teachers from low-caste groups were driving most of that discrimination; no evidence of bias could be found for teachers from high-caste groups. And they tended to direct it most often toward the lowest-performing students in the low-caste group—the students who presumably best fit the stereotype.

On the other hand, the researchers did not find any evidence that teachers were grading boys' or girls' tests any differently—except in the cases of the highest-performing girls, who were graded slightly more harshly than their high-performing male counterparts. (Incidentally, check out the Web-chat that I moderated on the gender gap at the top in mathematics and science. Could this be another possible explanation?)

Another new wrinkle in this study: Teachers' caste biases seemed to lessen as they got closer to the bottom of the test pile. The researchers hypothesize:
"It appears that when grading students early in the process, when the overall distribution of scores in unknown, teachers may use the caste of a student not as a signal of performance, but rather as a signal of where the child will eventually land in the overall distribution of tests." Hanna and Linden suggest that schools can counteract some of that bias by taking time to familiarize teachers with new tests.

This is all important, of course, because a long line of studies have documented what they call the "Pygmalion effect" in education. That is, children tend to fulfil their teachers' expectations for their performance. That's one strike that disadvantaged children don't need—in any society.

Texas Merit-Pay Pilot Failed to Boost Student Scores, Study Says

2009-11-04T14:37:02Z

Back in August, I told you about some early results from a study of a performance-pay program that was being tested in the Lone Star State.

Piloted between the 2005-06 and the 2008-09 school years, the now-defunct Governor's Educator Excellence Grants, or GEEG, program distributed more than $10 million a year in federal grants to 99 Texas schools that managed to turn in high scores on state tests despite enrolling large numbers of students from low-income families. The program differed from some other merit-pay schemes, though, because it required schools to involve teachers in designing the performance-incentive plans for their own schools.

In the earlier study, which was conducted by researchers from the National Center on Performance Incentives, at Vanderbilt University, we learned that, when given a say, teachers tend to be remarkably egalitarian. They favor relatively modest awards and spread them widely.

In the new study, released just this month by the same group of researchers, we learn whether the pay incentives for teachers translated to any improvements in their students' test scores. The answer, in a word, is no. The third-year findings indicate that, overall, the program had a "weakly positive, negative, or negligible effect on student test-score gains."

The relative size of the bonuses did not seem to matter much, either. In this case, the range was somewhat limited because most of the plans called for individual bonuses that were less than the $3,000 minimum recommended by the state. "Perhaps because the incentive structures were so weak," write authors Lori L. Taylor and Matthew G. Springer, "we find no evidence that variations in plan design led to variations in student performance gains."

The experiment wasn't a complete bust, though. The study did turn up evidence that the incentives had an impact on teacher turnover. Teachers who received no bonuses were more likely to leave their schools, while teachers who got an award were more likely to stay. The report concludes:

"If we assume that award recipients were more effective in the classroom than non-recipients—which might be a relatively strong assumption—then the evidence suggests that even weak incentives achieved the objectives of employers."

The less-than-stellar results don't mean that merit-pay programs will disappear from Texas schools anytime soon. According to the Associated Press, the state legislature, at Gov. Rick Perry's urging, has rolled several of the state's performance-incentive programs into one. The newly consolidated District Awards for Teacher Excellence, or DATE, is being funded to the tune of $200 million a year.

Cornell Authors to Discuss Gender Imbalances in STEM Fields

2009-11-03T14:07:37Z

Cornell University researchers Stephen J. Ceci and Wendy M. Williams open their new book, The Mathematics of Sex, with a description of the infamously difficult Honors Math 55 course at Harvard University. They write:

"It is legendary among high school math prodigies, who hear terrifying stories about it at their computer camps and Math Olympiads. Some go to Harvard just to have the opportunity to enroll in honors Math 55 ... Each year as many as 50 students sign up but at least half drop out within a few weeks ...The final class roster, according to The Crimson, is 45 percent Jewish, 18 percent Asian, 100 percent male."

It's a compelling anecdote for a book that explores the reasons why males are overrepresented in mathematics and mathematically intensive scientific professions such as physics, computer science, chemistry, operations research, mathematics, and engineering—even after all the strides women and girls have made in those fields in recent years. They note, for instance, that on math aptitude tests taken in high school, boys outnumber girls in the top 1 percent by a factor of 2 to 1. The ranks of females continue to thin, the book points up, the farther they go up the professional ladder. Anywhere from 64 percent to 93 percent of the professors on the tenure track in math-intensive fields, these authors say, are men and the number of women who intend to have a research career in those fields declines by 30 percent over the course of their doctoral training.

To figure out why, Ceci and Williams spent three years reading and synthesizing more than 400 studies on gender differences from seven different fields of study. They explored whether the lopsided distribution of math and science talent was due to innate or biological differences in skills such as spatial ability. They considered whether social and cultural biases erected barriers that kept women from reaching the top in those fields and debated whether women were simply less interested in math-intensive careers. In the end, they conclude that all three explanations contribute to the gender asymmetry in these fields. But a particularly important reason may be the choices that women themselves make about the kinds of lives they want to lead.

Interested in hearing more? You're in luck. Ceci and Williams are the featured guests today for a Web-chat that I'll be moderating on gender differences at the top in mathematics and the mathematically oriented sciences. The chat is scheduled to take place from 3-4 p.m. Eastern time. Tune in and read what they have to say.

The Turnaround Dilemma: Convert or Close Down?

2009-11-02T16:18:12Z

The report out last week on the results of a study looking at Chicago's efforts to close down failing schools got me thinking. In its study, the Consortium on Chicago School Research examined the impact on students of shutting down 18 chronically low-performing elementary schools in the Windy City. (Check out my colleague Dakarai Aarons' article on the study, if you haven't already.) The bottom line, according to this study, was that the students who were displaced by the closings just ended up at other low-performing schools in the district. Their achievement, as measured by test scores, did not improve all that much, compared to that of students who continued to attend similarly low-performing schools.

The findings are important because Secretary of Education Arne Duncan, who presided over the turnaround efforts in Chicago, is pushing a similar effort at the national level. Districts won't be required to close down schools to qualify for the new federal turnaround grants, but it's clear from the draft guidelines issued so far that the federal government really likes that approach.

That means school superintendents faced with failing schools have a difficult choice to make: Transform the schools or shut them down and start from scratch? Over at the Institute for Research and Reform in Education, founder Jim Connell has developed a simulation tool to help school superintendents chip away at that question.

First off, you should know that the nonprofit institute has a little skin in this game. It developed the First Things First program for improving high schools—a strategy that has historically tended to focus on converting existing high schools.

For argument's sake, though, let's play along. Imagine that you're a superintendent of a 14,300-student district and you want to boost the on-time graduation rates in your five lowest-performing high schools by 20 percent within five years. Would it be better to replace the schools or convert them? You could get some idea by plugging data about your schools into the spreadsheet that Connell developed based on his experience with First Things First schools. If the replacement schools enrolled 200 students a year, the calculator would tell you, it would take 11 new schools to match the success rate that you could get from transforming the existing schools instead.

That figure is based on lots of assumptions, of course, some of which may not apply in every district. Connell explains the thinking behind his analytic tool in a paper soon to be posted on the IRRE Web site. Contact him directly to try the calculator out for free.

UPDATE: I now have a link for Connell's paper, if you're looking for it. You can find it here. Also, just to clarify, the example above is hypothetical. It wasn't actually computed with the calculator but it gives you some idea of the kind of information the new tool might yield.

Study: Once a Cheater, Always a Cheater

2009-10-30T13:34:48Z

The next time you find yourself thinking, "kids will be kids," think again. Survey results released yesterday by the Josephson Institute on Ethics suggest that kids who cheat in high school are far more likely than non-cheaters to lie to their spouses, bosses, and employees when they grow up.

The results are based on a 2008 survey of nearly 7,000 people in five age groups: 17 and under, 18-24, 25-40, 41-50, and over 50. The study found, first of all, that younger generations are far more likely to behave dishonestly than older generations—or at least to admit it to researchers. Among high schoolers, 64% said they had cheated on an exam in 2008, 42% admitted lying to save money, and 30% copped to having stolen something from a store.

The study also found that, regardless of how old they are now, people who cheated in high school were three times more likely to lie to a customer (20% vs. 6%) or inflate an insurance claim (6% vs. 2%) and more than twice as likely to inflate an expense claim (10% vs. 4%) than people who never cheated in high school. The high school cheaters were also twice as likely to lie to or deceive their boss (20% vs. 10%) or lie about their address to get a child into a better school (29% vs. 15%) and one-and-a-half times more likely to lie to spouse or significant other (35% vs. 22%) or cheat on taxes (18% vs. 13%).

If that's not reason enough to boost character education efforts in schools, here's another: A growing number of studies are beginning to suggest that some character education programs can improve kids' academic achievement, too.