Planet DCS@UofT

Video on climate change without using climate models

2013-05-11T23:00:38+00:00

This video refers to research that was done long before the expression “climate change” was coined.

If Anything, They Overestimate My Javascript

2013-05-08T20:50:05+00:00

From my open source report card:

More seriously, there’s a whole lotta unhealthy spin goin’ on when “open source” means “what you did on GitHub”.

From spreadsheets to dataflow diagrams

2013-05-07T04:59:27+00:00

Via It will never work in practice, I discover the work (and of Felienne Hermans, Martin Pinzger and Arie van Deursen on “Supporting Professional Spreadsheet Users by Generating Leveled Dataﬂow Diagrams” (to avoid the paywall, download this free technical report). As they say in the paper abstract: “In this paper, we ﬁrst study the problems

The post From spreadsheets to dataflow diagrams appeared first on Modeling Languages.

Merging is the Real Revolution

2013-05-01T13:06:17+00:00

Many people at Mozilla think that Javascript and HTML5 are the future of the web. Respectfully, I think they’re both red herrings: I think what makes Mozilla and other successful open source projects work is older, less exciting, and still only kind of works. It’s called “merge”, and if we really want to help people collaborate on a global scale, we ought to put a lot more effort into making it easy to use.

What’s merge? It’s what you do after a “diff”. What’s diff? It’s something that shows you the differences between two files in a human-readable way. More specifically, suppose that you and I are both working on a program. We’re sitting in front of different machines, trying to fix different bugs or add different features, and it just so happens that we both need to change graphics.java. After we’ve both made our changes, the world looks like this:

At this point, we need to combine our changes. We could scroll through two copies of the file side by side, copying edits from one to the other, but we’d almost certainly miss something or make a mistake. What we should do is use a program like diff to highlight the changes for us. Or better still, we should use a tool like merge to show your version of the file on the left, mine on the right, and the merge in between:

When we’re done merging, what we have is the best of both worlds—the best of your ideas combined with the best of mine. The biological term for this is recombination, and it’s at least as important to evolution as its more famous cousin, mutation, because it lets good genes (or ideas) cooperate.

Diff and merge make open source possible. They let dozens, hundreds, or thousands of people remix their work—not just take what others have done and build on it, but give back their own changes and ideas to be stirred back into the original for further remixing:

When remixing is hard, open collaboration doesn’t take root [1]. Education is a prime example: at some point in their career, every teacher has picked up someone else’s PowerPoint slides and used it as a starting point for their own lecture on the subject, but hardly anyone ever gives their changes back to the author of the slides they started from. It’s easy to say that’s because remixing isn’t part of educational culture, but there’s a reason it isn’t: PowerPoint decks can’t be diffed and merged [2]. If it takes me an hour to scroll through my slides, comparing them one by one with yours and copying changes back by hand, I’m not going to use what you send me, so you’re not going to send it in the first place [3]. Going back to our biological metaphor, people who can’t merge are stuck in a universe that has mutation but not recombination, and that’s a really inefficient way to improve fitness.

I’m thinking about all of this now because of the IPython Notebook and Mozilla Thimble. They’re both really exciting tools, but neither makes collaboration easy [4]. If I want to merge your changes to a project into my copy, I can’t view them side by side in the browser and pick the pieces I want from each. Instead, I have to merge two JSON files if I’m using the Notebook and—well, I’m not sure what I’d do with Thimble. I could view the differences in the text of the HTML and CSS, but anyone who can do that can build web pages without Thimble in the first place.

More to the point, people shouldn’t have to drop down a cognitive level or two in order to collaborate this way. Lots of graphic design tools can highlight and merge the differences between two photographs; DiffEngineX does it for Excel spreadsheets (though you need a pretty wide screen to use it effectively), and so on. There’s no technical reason we can’t diff and merge all our files; it’s just that programmers mostly work with text, so they haven’t built merging tools for other formats. (And increasingly, I believe they work with text because it’s what they can diff and merge in version control…)

We’re smarter when we work together. It’s more fun, too, so I think tools ought to make collaboration as easy as adding a caption to a picture of a cat:

We were collaborating on a global scale before HTML5 and Javascript came along, and I’m confident that we’ll still be doing so ten years from now when they’re both regarded as legacy technologies. If we want kids to hack web pages the way we hack code, we need to make merging as easy as reading email or uploading files to Dropbox. And if we want their teachers to remix each other’s lessons, we need to show a little humility and make our methods work with their files. If we do that for them, they will learn to work the way we do and raise up a generation that thinks open collaboration is normal.

And that, my friends, would be a revolution.

The exception is systems like Wikipedia that have just one copy of the document which everyone edits simultaneously, but like Google Docs and Etherpad, that clearly doesn’t work for programming, slide decks, or other situations in which people want to try different things at the same time.
PowerPoint “merging” tools like these two just concatenate multiple presentations into one, or generate a specialized deck from a template by filling in blanks with names and dates (rather like spam generators).
At this point programmers often say, “Then write your slides Markdown or LaTeX or HTML5 or some other text-based format so that merging is easy,” but that’s like saying, “If you take all the pictures out of your book, it’ll compress much better.” PowerPoint, LibreOffice, Keynote, and other WYSIWYG presentation tools have survived and thrived because they make it easy for people to mix graphics and text however they want, just as they would on a whiteboard. As this blog post shows, it’s a lot harder to do this with text-based tools: I had to switch from my editor to a drawing package to create the diagrams included above, then upload them, and if you ask your browser to search for “Original Version”, it still won’t find that label in either of the diagrams. Given the choice between whiteboarding (which they take for granted) and merging (which they’ve never done before, and whose value they don’t yet understand), almost everyone will choose the former.
More precisely, neither makes asnchronous collaboration easy. TowTruck lets people share dynamic browser sessions in real time, which is really cool, but as noted in [1], that’s a very different model than forking and merging.

MoDELS 2013 – List of Tutorials and Workshops available

2013-04-24T10:40:19+00:00

You can already check the list of accepted workshops and tutorials for the MoDELS 2013 conference (in Miami next October). Among the workshops, I’d strongly recommend you to take a closer look at the “OCL, Model Constraint and Query Languages” workshop, for sure the most important venue to discuss with experts on OCL and textual

The post MoDELS 2013 – List of Tutorials and Workshops available appeared first on Modeling Languages.

Handling Unsolicited Commercial Email

2013-04-23T16:56:00+00:00

My email address is all over the web: at the time of writing this, a search on google for my email address produces about 15,800 results. So anyone who wants to find my email address can do so easily. Many people or companies who want to sell me something send me email out of the blue. I get a great deal of such unsolicited commercial email, too much to read or pay adequate attention to. I simply delete them. Unfortunately, many sources of such email persist. So for some time now, I've elicited the help of technology. I process my incoming email using procmail, a powerful piece of software that lets me script what happens to my email. When I receive unsolicited commercial email, if it is from a vendor or organization I don't have a relationship with, I will often add a procmail rule to discard, unseen, all future email messages from that vendor. I've got about 400 organizations (mostly vendors) in my discard list so far, and the list slowly grows. Am I still getting unsolicited commercial email from these sources? I am, but I am not seeing it. It's the same effect, really, as manual deletion (i.e. the message is deleted, unread), but it's easier for me, because I am not interrupted. But of course I think it would be better still if the email were not sent at all.

If you are a vendor with whom I do not have a pre-existing relationship, and you want to send me email introducing your products, please don't. I do not accept cold salescalls either. Instead, advertise effectively on the web, so that if I am looking for a product like yours, I can find you. If you must contact me directly, send me something by postal mail, where, unlike email, the communication does not have an interruptive aspect.

Cameron Neylon Speaking in Toronto on May 1, 2013

2013-04-19T17:23:20+00:00

Network Enabled Scholarship — Reconfiguring Research for the Web
Dr. Cameron Neylon
Director of Open Access Advocacy, Public Library of Science
4:00 p.m., Wednesday, May 1, 2013
Room 205, Bissell Building, University of Toronto, 140 St George St

The web, like all network technologies before it from the mobile phone to writing itself, has the potential to enable a qualitative change in our capacity as people, organizations and societies. We are starting to see the first glimmerings of how our research capacity might change with projects like Galaxy Zoo and Polymath but these remain isolated examples. What will it take to exploit the network capacity that the web brings us to enable a step change in the efficiency and effectiveness of our research?

This seminar will be the first in a series highlighting new opportunities to network knowledge through application of knowledge media design values and methodologies.

UMLtoSQL, UMLtoSymfony/PHP and UMLtoDjango/Python are now open source

2013-04-18T13:58:16+00:00

Long-time readers know that some time ago the portal included a set of online code generation services: UMLtoSQL: automatically generated a SQL DDL script to create the database corresponding to the input UML class diagram UMLtoPHP: automatically generated full-fledged PHP applications from UML class diagrams. The service generated Doctrine models that thanks to the PHP/Symfony

The post UMLtoSQL, UMLtoSymfony/PHP and UMLtoDjango/Python are now open source appeared first on Modeling Languages.

EU Innovation & Competitiveness 2011

2013-04-16T18:42:00+00:00

This is one of the most comprehensive reports regarding EU's innovation and competitiveness (dated, June 2011). It's a 765 page report spanning several interesting topics. The report is divided into three main parts:

Part I: Investment and performance in R&D - Investing in the future
Part II: A European Research Area open to the world
Part III: Towards an innovative Europe - contributing to the Innovation Union

Here is the full report (~44MB).

Overall Picture

Since this is a huge report, my suggestion would be to first have a look to the overall picture. The following report shows how Europe is positioned in research and innovation in the world and provides insights about the new geography of knowledge.

Overall picture: Europe’s competitive position in research and innovation - acting in the new geography of knowledge

Country Profiles

If you are interested for a fact-sheet detailing a specific country's profile, then follow the research and innovation performance by country page.

Greece

Here is the one about Greece's research and innovation performance.

Enjoy!

More than 25 universities using our MDE book

2013-04-15T00:40:16+00:00

You don’t write a technical book to make money so the best satisfaction for an author is to see his book being read. This is the reason why I’m very happy to announce that our model-driven introductory book is now being used to teach MDE in more than 25 universities all around the world (sure

The post More than 25 universities using our MDE book appeared first on Modeling Languages.

The Internet: Saving Civilization or Trashing the Planet?

2013-04-10T22:44:48+00:00

Next year, I’ll be teaching a new undergraduate course, as part of an initiative by the Faculty of Arts and Science known as Big Ideas courses. The idea is to offer trans-disciplinary courses, team taught by professors from across the physical sciences, social sciences, and humanities, that will probe important ideas about the world from different disciplinary perspectives. For the coming year, U of T is launching three Big Ideas courses:

BIG100: “The end of the world as we know it”;
BIG101: “Energy: From Fire to the Future”;
BIG102: “The Internet: Saving Civilization or Trashing the Planet?”

I’m delighted to be teaming up with Prof Miriam Diamond from Earth Sciences and Prof Pamela Klassen from Study of Religion to teach BIG102. Our aim is to give students some understanding of how the technologies that drive the internet work, and then to explore how the internet has reshaped the way we use information, our knowledge and beliefs about the world, and the impact that creating (and disposing of) internet technologies has on the environment, on the economy, and on the dynamics of innovation. A key goal is to foster critical thinking and information literacy skills, and especially to be able to think about and analyze a complex system-of-systems from different perspectives.

For the first term, we’re planning to cover a broad set of provocative questions, to get students thinking about the internet from different perspectives:

What is a big idea? (A course introduction, and a primer on trans-disciplinary thinking)
Who invented the internet? (Myths about the internet, and why they stick)
How does the internet work? (An introduction to some of the key technologies)
How new is the internet? (A short history of communications technologies, to put the internet in its historical context)
Has the internet changed us? (We’ll explore in particular, how the internet is transforming universities and learning)
What is the environmental footprint of the internet? (An initial assessment of energy consumption, resource extraction, and waste disposal)
Does the internet make us smarter? (An exploration of how internet search works, and how it affects our approaches to problem-solving)
Is the internet a time-saver or time-waster? (How the internet offers endless distractions, blurs distinctions between work and leisure, and its overall effect on productivity)
Can you be anonymous on the internet? (The idea of your information footprint – who’s keeping track of data about you, how they do it, and why)
Is the Internet a Cheater’s Paradise? (From plagiarism to adultery – how the internet facilitates cheating, new ways of discovering it, and virtual vigilante justice)
Who’s Not Online? (The idea of the digital divide, and the demographic and socio-economic factors that limit people’s access)
Gadgets as Gifts? (Just in time for the Christmas break, we’ll explore the environmental impact of our love of new gadgets, and whether there are sustainable alternatives)

In the second term, we plan to pick three themes to explore in more detail, so that we can explore inter-connections between some of these questions, and get the students engaged in independent research projects that synthesize what they’re learning:

The Internet and the Innovation Imperative.
- Is the Internet Innovative? How Moore’s law has driven innovation; the dotcom boom and bust; and the current hype around new technologies such as 3D printing, sensor networks, and the semantic web.
- What are the Resource Implications of the Internet? We’ll use material flow analysis to explore extraction and disposal and likely shortages of strategic minerals, and the geo-political implications of attempting to feed an exponential growth in demand.
- The Environmental and Human Health Burden of the Internet. Building on the discussion of resource implications, we’ll look at the health implications of mineral extraction and e-waste disposal, and the burden this places on people and ecosystems, especially in poorer countries.
- What is the Opportunity Cost of the Internet? Does investment in internet innovation mean we’re underinvesting in other things (eg clean energy, transport, social innovation). Have we developed an over-optimistic belief that IT technologies can solve all problems?
The Internet, Democracy, and Security.
- Censorship & Internet Governance. How much power do governments have to control what happens on the internet? Does the internet enhance or undermine democracy?
- The Underbelly of the Internet: Hackers, Espionage, and Trolls. How internet systems can be exploited by different groups, for example by crime syndicates who break into secure systems, by political groups who use a web presence to spread misinformation, and by internet trolls who violate social norms to disrupt and intimidate online discussions.
- Does the Internet make us a more open society? The open source movement and its successors (open government, creative commons, etc) are based on the idea that if everyone has access to the inner workings of systems, this removes barriers to participation, fosters creativity, and makes those systems better for everyone. But does it work?
- Transnational Jurisdiction: Legal boundaries and the Internet. We’ll wrap up this theme with a question about who should police the internet.
The Internet, Communities, and Interpersonal Relationships
- Does your Google-Brain make you forget? How has instant access to vast amounts of information changed our memories and our perceptions of ourselves? For example, does GPS route-finding mean we lose our ability to navigate and our sense of place? And what are the implications of the kind of personal digital archives that technologies such as Google Glass might allow us to create?
- Can you find love on the Internet? An exploration of how the internet changes personal relationships, from the role of dating sites and virtual social networks, to the way that online porn affects our perceptions of gender roles and body image.
- Can you find God on the Internet? How the internet affects religious communities, tolerance of different worldviews, and the very nature of faith.

Of course, this outline is still a draft – we’ll refine it over the next few months as we prepare for the first group of students in September.

We’re still exploring which textbooks to use, and even whether ‘books’ makes sense for a course like this – we’re hoping to make this a constructivist learning experience by using a variety of different internet-based media and information access tools throughout the course. However, we’re currently evaluating these books:

TechNO-fix, by Michael and Joyce Huesemann (a fascinating counter-point to the vast literature on how great technology is!);
The Shallows: What the internet is doing to our brains: by Nicholas Carr
Networked: The New Social Operating System, by Lee Rainie & Barry Wellman

Feel free to suggest other books and material!

Unsustainable

2013-04-09T15:17:11+00:00

We’re taking the kids to see their favourite band: Muse are playing in Toronto tonight. I’m hoping they play my favourite track:

I find this song fascinating, partly because of the weird mix of progressive rock and dubstep. But more for the lyrics:

All natural and technological processes proceed in such a way that the availability of the remaining energy decreases. In all energy exchanges, if no energy enters or leaves an isolated system, the entropy of that system increases. Energy continuously flows from being concentrated to becoming dispersed, spread out, wasted and useless. New energy cannot be created and high grade energy is destroyed. An economy based on endless growth is unsustainable. The fundamental laws of thermodynamics will place fixed limits on technological innovation and human advancement. In an isolated system, the entropy can only increase. A species set on endless growth is unsustainable.

This summarizes, perhaps a little too succinctly, the core of the critique of our current economy, first articulated clearly in 1972 by the Club of Rome in the Limits to Growth Study. Unfortunately, that study was widely dismissed by economists and policymakers. As Jorgen Randers points out in a 2012 paper, the criticism of the Limits to Growth study was largely based on misunderstandings, and the key lessons are absolutely crucial to understanding the state of the global economy today, and the trends that are likely over the next few decades. In a nutshell, humans exceeded the carrying capacity of the planet sometime in the latter part of the 20th century. We’re now in the overshoot portion, where it’s only possible to feed the world and provide energy for economic growth by consuming irreplaceable resources and using up environmental capital. This cannot be sustained.

In general systems terms, there are three conditions for sustainability (I believe it was Herman Daly who first set them out in this way):

We cannot use renewable resources faster than they can be replenished.
We cannot generate wastes faster than they can be absorbed by the environment.
We cannot use up any non-renewable resource.

We can and do violate all of these conditions all the time. Indeed, modern economic growth is based on systematically violating all three of them, but especially #3, as we rely on cheap fossil fuel energy. But any system that violates these rules cannot be sustained indefinitely, unless it is also able to import resources and export wastes to other (external) systems. The key problem for the 21st century is that we’re now violating all three conditions on a global scale, and there are no longer other systems that we can rely on to provide a cushion – the planet as a whole is an isolated system. There are really only two paths forward: either we figure out how to re-structure the global economy to meet Daly’s three conditions, or we face a global collapse (for an understanding of the latter, see GrahamTurner’s 2012 paper).

A species set on endless growth is unsustainable.

Congratulations to the Machine March Madness Winner

2013-04-09T13:13:09+00:00

Well, after another exciting March Madness tournament, Louisville emerged as the winner of March Madness, and Ryan Boesch emerged as the winner of Machine March Madness, with his algorithm beating out the field of 22 other machine competitors and all the human baselines. Congratulations, Ryan!

I asked him a few questions, which he answers below:

1. What inspired you to compete in the Machine March Madness competition?

Last year I finished a class on Convex Optimization during the winter quarter and was planning to take a Machine Learning class in the spring quarter. I was looking for a project to apply what I had learned. I saw this competition and submitted a last minute bracket.

2. What do you attribute your win to? What is your model best at?

The win was of course very lucky. Basketball games are random in nature so to find which model is actually the best it would require many years of tournaments. One tournament is not statistically significant.

There is nothing particularly special about my model. I used Danny's model, only I fit the parameters using convex optimization instead of batched gradient decent.

3. What do you think the most promising direction(s) towards improving your model would be?

Most Promising: My current model simply matches teams and sees which has the higher predicted score. It doesn't account for difficult of previously played games in the tournament. For example, say team 1 has a 51% chance to win the first round and also 51% chance to win the second round against team 2. If team 2 has a 95% chance of winning the first round then they are more likely to make it to round 3 even though they only have a 49% chance to beat team 1 in the second round. This is taken into account in Nate Silver's picks for example.

Second Most Promising: When in a pool with other competitors the goal is no longer to maximize your expected score, but instead to maximize your expected chance of winning. These two optimizations do not always result in the same picks. I may consider taking this into account in future years. I found this paper on Nate Silver's blog which analyzes this idea.

4. What advice would you give to future competitors?

Be wary of over fitting your model.

5. What would you change about the competition in future years?

We should try to get out and advertise for the competition earlier and to a broader audience to maximize participation.

Movisa: A DSL tool for Human Machine Interfaces (HMI) in Industrial Automation

2013-04-08T06:03:44+00:00

Guest post by Stefan Hennig on model based user interface development in the Industrial Automation domain. Enjoy! The operative states of industrial automation facilities are monitored and operated by qualified personnel using pertinent Human Machine Interfaces (HMI). These software systems map the technical process and the automation devices, respectively, to graphical symbols and thus, they provide

The post Movisa: A DSL tool for Human Machine Interfaces (HMI) in Industrial Automation appeared first on Modeling Languages.

Machine March Madness Final Four Outlook

2013-04-05T14:30:16+00:00

Once again, we have a guest post by Scott Turner, our local Machine March Madness competitor and analyst, who also runs http://netprophetblog.blogspot.com/. Thanks for another great post, Scott!

The Machine March Madness Contest finds itself in peculiar waters this year -- no doubt because the Tournament is in strange waters itself. There is only a single team left alive from the twelve #1, #2, and #3 seeds. Going into the Final Four, only four brackets have their champion prediction left alive -- and all four have Louisville. In fact, only one bracket has anyone left alive other than Louisville -- and that's "Tim J's Nets for Nets", who has Syracuse to the final game (but losing to Ohio State). But despite the craziness in the Tournament this year, the top four predictors in the Machine March Madness Contest are in the top 5% of all brackets.

If Louisville manages to win out, the champion will be "Ryan's Rank 1 Approximation" with 121 points. He will beat out all the human competitors as well.

If Louisville loses in the final game, the champion will be "Predict the Madness" (tied with "Danny's Dangerous Picks"), with "Ryan's Rank 1 Approximation" and my own Prediction Machine both a single point behind.

If Louisville somehow loses to Wichita State and Syracuse beats Michigan, then "Tim J's Nets for Nets" will vault all the way from fifteenth into a tie for first with "Predict the Madness". If Syracuse loses to Michigan, then "Predict the Madness" will win outright.

It has certainly been a crazy year for the Tournament!

Modelling allergy – a common hypersensitivity disorder

2013-04-04T10:28:38+00:00

Spring is here and with it many seasonal allergies. Today, however, I´d like to mention a yearlong one, the modelling allergy a clear case of hypersensitivity disorder that provokes a strong allergic reaction when the person is faced with software models (and related model-driven artefacts). There’s no clear treatment for this allergy. As specific anti-histamines

The post Modelling allergy – a common hypersensitivity disorder appeared first on Modeling Languages.

Special issue of GMD: Community software to support the delivery of CMIP5

2013-04-02T22:03:56+00:00

We now have a fourth paper added to our special issue of the journal Geoscientific Model Development, on Community software to support the delivery of CMIP5. All papers are open access:

M. Stockhause, H. Höck, F. Toussaint, and M. Lautenschlager, Quality assessment concept of the World Data Center for Climate and its application to CMIP5 data, Geosci. Model Dev., 5, 1023-1032, 2012.
Describes the distributed quality control concept that was developed for handling the terabytes of data generated from CMIP5, and the challenges in ensuring data integrity (also includes a useful glossary in an appendix).
B. N. Lawrence, V. Balaji, P. Bentley, S. Callaghan, C. DeLuca, S. Denvil, G. Devine, M. Elkington, R. W. Ford, E. Guilyardi, M. Lautenschlager, M. Morgan, M.-P. Moine, S. Murphy, C. Pascoe, H. Ramthun, P. Slavin, L. Steenman-Clark, F. Toussaint, A. Treshansky, and S. Valcke, Describing Earth system simulations with the Metafor CIM, Geosci. Model Dev., 5, 1493-1500, 2012.
Explains the Common Information Model, which was developed to describe climate model experiments in a uniform way, including the model used, the experimental setup and the resulting simulation.
S. Valcke, V. Balaji, A. Craig, C. DeLuca, R. Dunlap, R. W. Ford, R. Jacob, J. Larson, R. O’Kuinghttons, G. D. Riley, and M. Vertenstein, Coupling technologies for Earth System Modelling, Geosci. Model Dev., 5, 1589-1596, 2012.
An overview paper that compares different approaches to model coupling used by different earth system models in the CMIP5 ensemble.
S. Valcke, The OASIS3 coupler: a European climate modelling community software, Geosci. Model Dev., 6, 373-388, 2013 (See also the Supplement)
A detailed description of the OASIS3 coupler, which is used in all the European models contributing to CMIP5. The OASIS User Guide is included as a supplement to this paper.

(Note: technically speaking, the call for papers for this issue is still open – if there are more software aspects of CMIP5 that you want to write about, feel free to submit them!)

That’s My Secret, Captain…

2013-03-28T14:36:37+00:00

…I’m always proud:

User interface modeling with the new IFML standard

2013-03-27T11:22:49+00:00

UML is good to model many things but clearly misses the point when it comes to design complex user interfaces and interactions. Attempts to use UML for this usually end up with complex profiles that are almost impossible to apply. Therefore, I was happy to see that the OMG launched a RFP for an Interaction Flow Modeling

The post User interface modeling with the new IFML standard appeared first on Modeling Languages.

js-sequence-diagrams: A text to UML Sequence diagrams with JS

2013-03-25T15:32:12+00:00

JS-Sequence-Diagrams is a javascript library to turn text into vector UML sequence diagrams. It uses Jison to parse the text, and Raphaël to draw the image. It includes two styles to render the diagram, the “normal” and the “hand-drawn” (this seems to become a common feature, yUML also had what they called the scruffy mode) as you can see in

The post js-sequence-diagrams: A text to UML Sequence diagrams with JS appeared first on Modeling Languages.

Upset Analysis by Scott Turner

2013-03-25T16:17:28+00:00

This is a guest post by Scott Turner, who is a perennial Machine March Madness competitor, and who runs http://netprophetblog.blogspot.com/.

table.tableizer-table { border: 1px solid #CCC; font-family: Arial, Helvetica, sans-serif; font-size: 12px; } .tableizer-table td { padding: 4px; margin: 3px; border: 1px solid #ccc; } .tableizer-table th { background-color: #104E8B; color: #FFF; font-weight: bold; } There are 22 entries in the Machine Madness contest this year, so analyzing them is a much bigger task than in past years. Nonetheless I dug through all the brackets and looked at all the first round upset predictions to see how well the machines did.

Correct Upset Predictions

Interestingly enough, every first round upset was picked by at least two of the predictors except for Harvard -- which no one picked -- and Florida Gulf Coast, which only "Larry's Upsetting Picks" predicted. The only consensus upset pick was Minnesota over UCLA, which was predicted by exactly half of the predictors. Iowa State also got broad support (40%) but none of the rest of the picks had more than 4 predictors in support. Here's the full table of the upsets that occurred and who predicted them (stretch your window!):

Entry	Sum	Minnesota	Iowa St	Oregon	Wichita St.	Mississippi	Temple	California	La Salle	Fla GC	Harvard
*Danny's Dangerous Picks	2	1	1
Andy's Astounding Bracket	2		1					1
Ask me about my T-Rex	3	1	1		1
Curtis Lehmann's Crazy Bracket	0
Dan Tran's Dazzling Bracket	3		1	1				1
Guess O'Bot 3000	0
K. V. Southwood's Fine Bracket	2	1	1
Larry's upsetting picks	4		1			1			1	1
LA's Machine Mad Pick	1						1
Leon's Super Legendary Bracket	1		1
Marginal Madness	1					1
Mark's LR bracket	4	1			1		1	1
MatrixFactorizer	3				1	1	1
natebrix's Neat Bracket	2	1							1
noodlebot	1	1
Predict the Madness	1	1
Ryan's Rank 1 Approximation	1	1
Scott Turner's Prediction Mach	2	1	1
ScottyJ's Grand Bracket	0
The Rosenthal Fit	2			1		1
TheSentinel	2	1		1
Tim J's Nets for Nets	4	1	1	1	1
Ave Correct:	1.9	50%	41%	18%	18%	18%	14%	14%	9%	5%	0%

My conclusion here is that UCLA-Minnesota and Notre Dame-Iowa State were probably mis-seeded.

UCLA-Minnesota is an interesting case in human psychology. Minnesota lost 11 of its last 16 games, finished 8th in its conference and lost in the first game of the conference tournament, while UCLA won 11 of its last 16, won the Pac-12 regular season conference title, and lost in the title game of the conference tournament. It's no wonder UCLA got a 6 seed and Minnesota an 11. But in fact, Minnesota was playing against much better competition through the conference games, and most of its losses came to ranked opponents and/or on the road. Machines understand the concept of a "good loss" much better than people.

The Notre Dame-Iowa State mis-seeding wasn't so egregious. This probably should have been an 8-9 matchup instead of a 7-10, in which case a win by Iowa State would have hardly been surprising.

All of the rest of the games were probably true upsets.

Incorrect Upset Predictions

Most of the predictors also made a number of incorrect upset predictions. Most predictors had one or two missed upsets, although six of the predictors made no missed upset predictions (primarily because they made mostly chalk predictions). Here's the full table:

Entry	SUM	Colo	St.Mary	Cincy	Bucknell	Okla	Missou	Belmont	SDak	Davidson	Valpo	Iona	Villa	Akron	Montana
*Danny's Dangerous Picks	2		1				1
Andy's Astounding Bracket	0
Ask me about my T-Rex	2		1	1
Curtis Lehmann's Crazy Bracket	1	1
Dan Tran's Dazzling Bracket	2	1	1
Guess O'Bot 3000	3	1			1	1
K. V. Southwood's Fine Bracket	4	1			1	1		1
Larry's upsetting picks	7	1						1	1	1	1	1			1
LA's Machine Mad Pick	0
Leon's Super Legendary Bracket	8	1	1	1	1			1		1			1	1
Marginal Madness	0
Mark's LR bracket	4	1	1			1	1
MatrixFactorizer	2	1		1
natebrix's Neat Bracket	2	1			1
noodlebot	0
Predict the Madness	1		1
Ryan's Rank 1 Approximation	1			1
Scott Turner's Prediction Mach	2		1			1
ScottyJ's Grand Bracket	0
The Rosenthal Fit	0
TheSentinel	2		1				1
Tim J's Nets for Nets	4	1		1	1				1
Average Missed:	2.1	45%	36%	23%	23%	18%	14%	14%	9%	9%	5%	5%	5%	5%	5%

As a general rule, the predictors that made the most correct upset picks also made the most incorrect upset picks. Notably, "Larry's Upsetting Picks" made the incredible call of the FGCU upset (and also called the second round upset) but also made seven incorrect upset picks.

There was almost a consensus (45%) on Colorado over Illinois. That's an interesting contrast with the Minnesota pick -- Illinois should have benefited in most of the predictors from a tough B1G conference schedule, but many of the predictors thought Illinois was still vulnerable. Illinois had a 16 point halftime lead in this game, but let it slip away and need some late game heroics to win, so this was certainly a reasonable prediction.

St. Mary's over Memphis was another popular pick. Memphis won by 2 when a last-second shot by St. Mary's missed, so this also seemed like a reasonable upset pick.

Upset Profits

An important question is whether any of the predictors profited from their upset predictions -- that is, whether the points they gained from correct upset predictions were more than the points they lost from missed upsets. In general, this is complex to calculate because we have to look at how the predictions affect the later rounds of the tournament. But it's easy enough to look at just the first round scoring. Here's the table:

Entry	Upsets	Missed	Overall
Andy's Astounding Bracket	2	0	2
The Rosenthal Fit	2	0	2
Ask me about my T-Rex	3	2	1
Dan Tran's Dazzling Bracket	3	2	1
LA's Machine Mad Pick	1	0	1
Marginal Madness	1	0	1
MatrixFactorizer	3	2	1
noodlebot	1	0	1
*Danny's Dangerous Picks	2	2	0
Mark's LR bracket	4	4	0
natebrix's Neat Bracket	2	2	0
Predict the Madness	1	1	0
Ryan's Rank 1 Approximation	1	1	0
Scott Turner's Prediction Mach	2	2	0
ScottyJ's Grand Bracket	0	0	0
TheSentinel	2	2	0
Tim J's Nets for Nets	4	4	0
Curtis Lehmann's Crazy Bracket	0	1	-1
K. V. Southwood's Fine Bracket	2	4	-2
Guess O'Bot 3000	0	3	-3
Larry's upsetting picks	4	7	-3
Leon's Super Legendary Bracket	1	8	-7

We see that a couple of the predictors ("The Rosenthal Fit" and "Andy's Astounding Bracket") came out two points positive, fifteen of the predictors gained one or zero points, and five of the predictors lost points. Interestingly, both "The Rosenthal Fit" and "Andy's Astounding Bracket" made only two upset predictions and got both of them right -- and there was no overlap in their predictions. Furthermore, neither of them predicted the "easiest" upset of Minnesota over UCLA.

Conclusions

None of the predictors performed very well at picking upsets, and there wasn't wide agreement on the upset picks. The consensus would have selected only the Minnesota-UCLA upset and been +1 in scoring, but no individual predictor did that. Most of the predictors did not hurt themselves with their upset picks (at least looking at only the first round), but none really saw significant benefit. Given the potentially large downside of missing upset predictions, in future contests it wouldn't be an unreasonable strategy to force your predictor to make all chalk selections in the first round.

Some Final Four Analysis...

2013-03-24T09:23:12+00:00

... over at Scott Turner's blog: http://netprophetblog.blogspot.co.uk/2013/03/machine-madness-some-final-four-analysis.html

The 2013 Machine March Madness Field

2013-03-22T02:33:56+00:00

Thanks everybody who entered this year's Machine March Madness competition. Based on the descriptions of the approaches, it's clear that a lot of hard work and ingenuity has gone into the contest. I'm excited to see how all the different approaches do.

Below, you can see the competitors's descriptions of their approaches. We'll also have some longer posts diving into more details coming up in the near future. If there are any in particular that you're itching to hear more about, leave a note in the comments.

If you have entered but not sent me a description of your approach yet, please do. I'll update this post as more descriptions come in.

Without further ado, here is your 2013 Machine March Madness field!

-----------------------------------------------------------------------------

Marginal Madness
Kevin Swersky
http://tournament.fantasysports.yahoo.com/t1/2909174

I'm using variational Bayesian matrix factorization with normal priors on the latent factors, and Gaussian-inverse Wishart hyperpriors on the hyperparameters of the priors. Inference is performed using mean-field (no direct optimization of any model parameters is done). The entries of the matrix are R(i,j) = P(team i beats team j) using the empirical counts over the 2012-2013 season. I found that the brackets produced using this were much more stable with respect to the number of factors than any other representation. I used 20 factors, the number of which was chosen based on squared error on 25% randomly held-out entries of R. For my predictions, I just took the mean vectors and ignored any uncertainty learned by the model. Ideally, I should have selected the number of factors, or assessed the stability of the model by using the variational lower bound, but I was lazy. To predict the final score, I used gradient-boosted regression trees from scikit-learn on the feature vectors produced by the factorization.

-----------------------------------------------------------------------------

Larry's Upsetting Picks
Laurent
http://tournament.fantasysports.yahoo.com/t1/1398519

I'm using a PMF-based model and I'm also modelling several other aspects such as teams' strength over time (both over a season and across seasons) as well as conferences' strength. These different aspects are combined linearly together to form a prediction.

I also tried using a team's winning percentage (both over the season and over the last few games) but that didn't lead to an improvement.

On a technical note, I also noticed that in PMF instead of using the raw score, using the difference in scores gives slightly increased (winner determination) accuracy.

-----------------------------------------------------------------------------

K. V. Southwood's Fine Bracket
K.V. Southwood
http://tournament.fantasysports.yahoo.com/t1/3003299

I created an ensemble model based on 3 individual models:

1) multiple linear regression model based on predicting the points margin

2) multiple linear regression model based on predicting offensive points scored

3) logistic regression model based on predicting win vs. loss

-----------------------------------------------------------------------------
Ryan's Rank 1 Approximation
Ryan B.
http://tournament.fantasysports.yahoo.com/t1/1636526

Brief description of approach (same as last year): For each season (e.g. 2006-2007) I have enumerated the teams and compiled the scores of the games into a matrix S. For example, if team 1 beat team 2 with a score of 82-72 then S12=82 and S21=72. Ideally, each team would play every other team at least once, but this is obviously not the case so the matrix S is sparse. Using the method proposed by George Dahl, I define vectors o and d which correspond to each teams offensive and defensive ability. The approximation to the matrix S is then just the outer product od' (for example (od')_12=o1d2=S12est). This is a simple rank one approximation for the matrix. If each team played each other at least once then the matrix S would be dense and the vectors o and d could be found by finding the SVD of S (see http://www.stanford.edu/~boyd/ee263/notes/low_rank_approx.pdf). Because this is not the case, we instead define a matrix P that represents which teams played that season. For example, P12=P21=1 if teams 1 and 2 played a game. Now the problem stated by George can be expressed compactedly as, "minimize ||P.*(o*d')-S||_F". Here, '.*' represents the Hadamard product and ||.||_F is the Frobenius norm. In this from, it is easy to see that, for constant vector o and variable vector d, this is a convex problem. Also, for constant vector d and variable vector o this is a convex problem. Therefore, by solving a series of convex problems, alternating the vector variable between o and d, the problem converges rapidly in about 5 to 10 steps (see "Nonnegative Matrix Factorizations" code here http://cvxr.com/cvx/examples/). From this point the problem is easily expanded to handle higher rank approximations.

-----------------------------------------------------------------------------

Scott Turner's Prediction Machine
Scott Turner
http://tournament.fantasysports.yahoo.com/t1/1760363

Linear regression on a number of statistics, including strength ratings to predict MOV (Margin of Victory). The basic model is used to predict game outcomes throughout the year, but there are some modifications for the Tournament. Additions this year include a new metric for analyzing possible upsets, an algorithm for forcing upset selections based upon the (predicted) score required to win the pool, and some modifications for neutral-court and tournament games. More details at http://netprophetblog.blogspot.com/.

-----------------------------------------------------------------------------
noodlebot
Joe
http://tournament.fantasysports.yahoo.com/t1/2298853

See my blog post and project page.
http://joenoodles.com/2013/02/ncaa-d1-basketball-db/
https://github.com/jnu/ncaa

-----------------------------------------------------------------------------
Danny's Dad (Human Baseline)
Danny's Dad.
http://tournament.fantasysports.yahoo.com/t1/2664431

Literally, Danny's Dad's picks.

-----------------------------------------------------------------------------
Obama's Bracket (Human Baseline)
Barack Obama
http://tournament.fantasysports.yahoo.com/t1/1673628

The President's picks.

-----------------------------------------------------------------------------
MatrixFactorizer
Jasper Snoek
http://tournament.fantasysports.yahoo.com/t1/1597161

Probabilistic matrix factorization augmented with Gaussian Processes and Bayesian optimization. More details will be forthcoming in a longer blog post (Update: here).

-----------------------------------------------------------------------------

LA's Machine Mad Pick
LeAnthony M.
http://tournament.fantasysports.yahoo.com/t1/1647581

I used 2011 final four stats data rather than last years. Including RPI, Off eff, turnovers, & def eff. A fitness function of the final standings NCAA tournament standings feed into an evolving genetic program giving me a final equation. I feed in this equations, this years team of 64 to compute the final standing of the 2013 tournament.

-----------------------------------------------------------------------------

Predict the Madness
Monte McNair
http://tournament.fantasysports.yahoo.com/t1/2002207

???

-----------------------------------------------------------------------------

TheSentinel
Chuck
http://tournament.fantasysports.yahoo.com/t1/2997354

Similar strategy as last year. Used Ken Pomeroy's Pythag ratings with the log5 calculation to determine probability of winning the game.

Used a Monte Carlo simulation at 65 iterations which provided a few interesting upsets, Oregon over Oklahoma St. (I believe they were miss seeded myself!).

-----------------------------------------------------------------------------

Danny's Dangerous Picks
Danny
http://tournament.fantasysports.yahoo.com/t1/1421921

Developed a variant on probabilistic matrix factorization, where the scores of a game are modeled as the output of a neural network that takes as input a learned latent vector for each team as well as the elementwise product of the latent vectors for the two teams. Latent vectors for each team are learned for each team for each season jointly with the neural net parameters, which are shared across all seasons from 2006-2007 through the present. I used 5D latent vectors and a one hidden layer neural net with 50 hidden units.

-----------------------------------------------------------------------------

Human Bracket
Lee
http://tournament.fantasysports.yahoo.com/t1/3297751

The Commissioner's human bracket.

-----------------------------------------------------------------------------

The Rosenthal Fit
Jeffrey Rosenthal
http://tournament.fantasysports.yahoo.com/t1/1666195

Details here: http://www.tsn.ca/story/?id=418503

-----------------------------------------------------------------------------

Last Year's Winner (Baseline)
Jasper Snoek
http://tournament.fantasysports.yahoo.com/t1/1644140

(The winning algorithm from last year, run on this year's data but otherwise unmodified. Entered as a baseline.) I modified Danny's starter code in two ways: First, I added an asymmetric component to the loss function, so the model is rewarded for getting the prediction correct even if the absolute predicted scores are wrong. Second, I changed the regularization so that latent vectors are penalized for deviating from the global average over latent vectors, rather than being penalized for being far from 0. This can be interpreted as imposing a basic hierarchical prior.

I then ran a search over model parameters (e.g., latent dimension, regularization strength, parameter that trades off the two parts of the loss function) to find the setting that did best on number of correct predictions made in the past 5 years's tournaments.

-----------------------------------------------------------------------------

Leon's Super Legendary Bracket
Leon
http://tournament.fantasysports.yahoo.com/t1/1712730

Defensive efficiency vs Offensive efficiency; tie-breakers favored defense over offense. Chose final score using season averages in wins/losses.

-----------------------------------------------------------------------------

Tim J's Nets for Nets
Tim J.
http://tournament.fantasysports.yahoo.com/t1/1546944

Based on full season statics for each team run a discriminant analysis for correlation with wins including seasons 2000-present.

Then I trained a neural network only on neutral location games, measuring both performance in mean squared error and actual past year bracket scores from 2007-2012, and predicting the bracket for this year.

-----------------------------------------------------------------------------

natebrix's Neat Bracket
Nate
http://tournament.fantasysports.yahoo.com/t1/1931619

The method is a variation on Boyd Nation's Iterative Strength Rating that incorporates margin of victory and weights late-season games more strongly. This link has more:
https://nathanbrixius.wordpress.com/2013/03/20/ncaa-tournament-prediction-model-2013/

-----------------------------------------------------------------------------

-----------------------------------------------------------------------------

Mark's LR bracket
Mark???
http://tournament.fantasysports.yahoo.com/t1/2504134

Logistic Regression???

-----------------------------------------------------------------------------

Ask me about my T-Rex
Zach Mayer
http://tournament.fantasysports.yahoo.com/t1/1827557

???

-----------------------------------------------------------------------------

ScottyJ's Grand Bracket
???
http://tournament.fantasysports.yahoo.com/t1/1876867

???

-----------------------------------------------------------------------------

Guess O'Bot 3000
???
http://tournament.fantasysports.yahoo.com/t1/1646914

???

-----------------------------------------------------------------------------

Andy's Astounding Bracket

???
http://tournament.fantasysports.yahoo.com/t1/1645698

???

-----------------------------------------------------------------------------

Dan Tran's Dazzling Bracket
???
http://tournament.fantasysports.yahoo.com/t1/1668480

???

Predicting March Madness by Jasper Snoek

2013-03-21T18:45:05+00:00

Now that March Madness is officially underway, and the deadline to submit new bracket predictions has passed, I'm ready to divulge the details of my super secret, possibly excessively advanced, march madness prediction model. For a few years now, there has been a special "elite" pool to predict march madness. The twist is that all the predictions have to be made by a computer algorithm - no humans allowed. This means we can't use seed information, predictions from experts or the POTUS's executive insight. Instead, we predict based only on data (my model uses only scores). This is the second year that I am entering an algorithm. My entry from last year, which won the pool and beat the vast majority of humans in the Yahoo challenge, is being used as a baseline. This means I have to submit something more sophisticated this year to stay on top.

The model:

The Simple Version:
A few years ago, the world of machine learning (a subfield of artificial intelligence that combines statistics, math and computer science to get computers to learn and infer from data) was rocked by the Netflix challenge. Netflix offered a prize of a million dollars to anyone who could beat their movie recommendation system by 10%. One of the most powerful and surprisingly simple algorithms to come out of that challenge was Probabilistic Matrix Factorization (PMF). The idea was that a movie rating was a simple product of a set of hidden or 'latent' factors pertaining to the movie and the user. Although the factors are not pre-defined, you could imagine that the model may learn one factor for a movie that corresponds to the amount of action and then a user would have a factor encoding how much they like action (and similarly for e.g. romance). We learn the model by adjusting these factors to maximize the probability that the user would give the ratings that we can see. To predict someone's rating for a given movie they haven't seen yet, you just multiply their factors by the movie factors.

Similarly to movie ratings we can create factors for basketball teams to predict game scores. Here the factors (again learned by the model) could correspond to offensive skill and defensive capabilities. This was the basis of my model for last year. There was a small twist in that I altered the way that the model was learned - to focus only on scores for which it predicted the wrong winner.

This year my model is significantly more complex but builds on the same principles. It has two levels of latent or hidden factors. The first encodes factors for each team - such as offensive skill, defensive skill, etc. The second layer combines team factors just like in standard PMF, but instead of mapping directly to the scores they map to a hidden representation that encodes the game. My reasoning is that the resulting score of a game is much more complex than a product of simple factors pertaining to each team. The idea is that the game representation now encodes things like: will the game be close or will it be a blowout - will it be high scoring or a defensive brawl? From the game representation I have a mapping to the difference between the home team score and the away team score. Now this is where things get a little complicated. Since there are relatively only a small number of games in this season (just over 5000) and this model is already fairly complex, rather than directly try to learn a function mapping from the game factors to the scores, I model a distribution over all possible mappings. The idea: given all (infinite) reasonable mappings from factors representing the game to scores, what is the most probable outcome? To do this I use a statistical model called a Gaussian process.

The factors encoding teams.

Now to learn the model:

I take all of the game scores from the past season. For each game, I tell the model which team is the home team, which is the away team, and then adjust the team factors and game factors in order to maximize the probability of the real score. In order to choose the number of factors at each step, I use a new automatic parameter tuning algorithm I personally helped develop called Bayesian optimization.

What do the factors look like?
Just to the right I have an example of the factors that are learned if I train the model using just two factors for the teams (for those of you in machine learning, these are the weights of the neural network) and I have plotted where each of the teams are in this factor space (along with their seeds). You can see that the model is putting the better teams in the lower left and the worse teams near the top right. It doesn't seem to fancy the odds of South Dakota... I'll explain later why I call the model "Turducken".

Below this I have a picture zoomed in on just the bottom left. You can see that the powerhouses are all encoded in this region. You can click on these images to zoom in. Now you can see that two factors already encode quite a bit about which teams are better. My model uses two hundred factors - so it is encoding something that is quite significantly more complex.

Zoomed in on the bottom left.

Below there is a picture of the factors learned to encode games. There is a dot for each game which is colored by relative score. So a 1 means that the home team wins by a lot and a two means that the home team loses by a lot ("a lot" here actually means about 50 points). So the model takes the team factors on the right and multiplies them to get to the game factors below. Then from the game factors it predicts by how much the home team will win or lose.

What is this Bayesian optimization?
One really exciting area of machine learning that has advanced a lot over the past year is related to how to build systems that work more automatically. To really eke out the best performance, you usually need an expert to sit and tweak a bunch of knobs, see what happens, and repeat many times. It's really time-consuming and nearly
impossible for a non-expert (and even difficult for experts). But there is work on automating this process, building a system to automatically tune the knobs and decipher the results. I am using Bayesian optimization that I left running overnight to automatically determine how many factors to use for teams and for games based on how well the model can predict the scores of 500 games that I pulled out of the set of data that the model learns from. The procedure decided to use 200 factors per team and just two per game.

In Machine Leaning Speak:
The devil is of course in the details. The model I am using is a buzz-word powerhouse. I call it a deep semi-parametric Bayesian probabilistic matrix factorization that is optimized using Bayesian optimization. My fellow machine learning PhD friend, George Dahl, calls it a "statistical Turducken". It uses a neural network trained with 'dropout' to perform a nonlinear probabilistic matrix factorization into a latent space that encodes games. A Gaussian process mapping is then used to map from games to the score difference. The input to the neural network is a binary encoding of which team is the home team (so the number of dimensions equals the

An example of the factors learned by the model to encode 'games'.

number of teams) and then similarly a binary encoding of which team is the away team. So the input to the model is a numTeams x 2 dimensional binary encoding with two bits on. This may seem wasteful, but note that now the weights to be learned by the neural network correspond exactly to latent factors pertaining to each team. The teams get different factors depending if they are home or away (as I personally have no college basketball expertise, I have no idea if this is a wise design choice). The neural network maps these factors into a hidden unit representation and then to a latent space. From the latent space I map using a Gaussian process with a squared exponential kernel to score difference.

The model is trained using backpropagation - from the marginal likelihood of the Gaussian process I backpropagate error through the kernel of the GP to the weights of the neural network. I use stochastic gradient descent on randomly chosen minibatches of 250 games at a time and a 50% dropout rate on the hidden units of the neural network. I used Bayesian optimization on a validation set of 500 games to determine the number of hidden units in the neural network (i.e. the number of factors in the PMF), the latent dimension of the input to the GP and the number of epochs to train the model for.

What did it predict?
You can check out the bracket that it predicted here: http://tournament.fantasysports.yahoo.com/t1/1597161
As of this writing, the model is 4/4 including a minor upset of Wichita over Pittsburgh.

You can take a look at our pool here: http://tournament.fantasysports.yahoo.com/t1/group/162045
Interestingly, even though it doesn't know anything about the seeds, it predicted the four number one seeds in the final four. According to the turducken, Indiana is going all the way. This is pretty remarkable - the algorithm is in close agreement with some of the top human basketball experts. That is already a validation that it is doing something reasonable. There are not too many
controversial predictions here, though it is predicting some upsets (e.g. Notre Dame over Ohio St.). It will be really exciting to see how it does as the next days play out!

And we're off!

2013-03-21T10:46:38+00:00

The Machine March Madness competition has officially begun. It looks like we ended up with 22(!) algorithmic submissions, and we have 3 human baselines. You can see all the picks here:
http://tournament.fantasysports.yahoo.com/t1/group/162045

I'll put together a post of all the methods soon. Note: if you are a competitor and have not emailed me a description of your method yet, please do so ASAP.

Model-driven learning design

2013-03-21T08:32:28+00:00

In a collaboration with Juan Manuel Dodero and his colleagues from the University of Cadiz, we have explored the application of MDE to a new (for me) scenario: the generation of learning environments from a learning design model conforming to a learning design DSL. It’s an easy joke but let me say that I learnt

The post Model-driven learning design appeared first on Modeling Languages.

Final Stretch for Machine March Madness Predictions

2013-03-20T08:55:02+00:00

We're in the home stretch for this 2013 Machine March Madness competition, and it looks like this is going to be an exciting year. So far, 17 entrants have joined the Yahoo system, which makes it easily the most popular year yet, and I'm expecting at least a few more before the deadline tomorrow. It also sounds like there are some pretty sophisticated approaches in the works, so I'm excited to see how it all plays out. We should also have some fun baseline predictors to compare against, although if you have ideas about other non-algorithm brackets you'd like to see added, leave a note in the comments.

Once the brackets are locked in, I'll put up a post describing each of the competitors methodologies, and then we can ask some of the competitors to go into more detail.

Finally, if you are planning on competing, please make sure to read the instructions on the Google group to see how to enter, and remember to send me a brief description of your method. Good luck every person, and every machine.

March Madness Team Embeddings

2013-03-19T17:42:40+00:00

I went with a new approach to Machine March Madness predictions this year. I won't go into the details right now, but here's a neat visualization that comes out of the algorithm. What you need to know is that I'm sticking with the basic original idea of using latent real-valued descriptors for each team, but I'm abandoning the requirement that there are segregated offensive and defensive descriptors for each team. Instead, the model this year represents each team with a set of numbers that can be used to explain both offensive and defensive performance.

So I'll skip all of the details and jump straight to showing you what the model has learned from this year's regular season. Below is a visualization of what happens when I ask the model to use two numbers to describe each team, then I plot the learned numbers as x and y coordinates on a standard plot.

These results lose the easy interpretability as offensive and defensive strengths, but the model is such that teams in similar locations on the plot will typically be predicted to perform similarly. To help with eyeballing the results, I've color coded 1 through 4 seeds: #1 seeds are blue, #2's are green, #3's are red, and #4's are magenta.

I won't try too hard to explain what's going on, but it does seem to group the stronger teams in the lower and left parts of the plot, and the weaker teams in the upper and right parts. Anybody notice any other interesting patterns?

MDE to the people – Upcoming talk

2013-03-18T07:55:43+00:00

For those of you around Barcelona next Friday, you’re more than welcome to join in the talk I’ll be giving at the UPC (the official excuse for the trip to Barcelona is Elena Planas PhD defense on Lightweight Verification of Executable Models , more on this next week). Abstract of the talk: Research on Model-driven

The post MDE to the people – Upcoming talk appeared first on Modeling Languages.

Worried about your RSS subscription? – Follow the modeling news by email

2013-03-16T23:53:26+00:00

Copyblogger makes a really good point in his post “A Real Simple Solution to the Death of Google Reader“, if you feel abandoned by Google and wonder what to do to keep all your RSS feeds (including our own one, at least until Google decides to kill feedburner as well), there’s an easy solution get

The post Worried about your RSS subscription? – Follow the modeling news by email appeared first on Modeling Languages.

Structured Flowcharts outperform pseudocode

2013-03-15T10:16:01+00:00

Greg pointed me to this old (1989) IEEE Software article: Structured flowcharts outperform pseudocode: an experimental comparison by David A. Scanlan. In the paper the author empirically shows that significantly less time is required to comprehend algorithms represented as flowcharts. He ended up saying “I am not suggesting that we should all retrieve our flowchart

The post Structured Flowcharts outperform pseudocode appeared first on Modeling Languages.

neil

2013-03-12T16:47:55+00:00

Software development is rife with references to business value, particularly in agile approaches: the Agile Manifesto declares that “Our highest priority is to satisfy the customer
through early and continuous delivery of valuable software.”

The trouble is that it isn’t clear what ‘valuable’ means. I’m sure that the point of this phrase, as with most of the Manifesto, is to start a discussion rather than to behave as a prescriptive methodology. I believe “value” is inherently context-dependent, so in that sense it is reasonable to leave it vague.

On the other hand, many people refer to business value as the Holy Grail of software development: this is what you are supposedly optimizing in Scrum. Other methodologies help focus on “impact”. Lean approaches have one remove ‘waste’ from the value stream. And yet no one has ever pinned value down, as Racheva et al. have shown [1] (in the software domain, anyway – many attempts have been made in economics).

Business value does have the nice property of communicability, though. It gives developers that one number to sell the project to the business, and allows for a conversation about scope, cost of delay, and prioritization that is difficult to do with purely qualitative methods. And for the mathematically inclined, it lends itself to algorithms like linear programming for optimization.

One paper does try to break business value into more reasonable components, which I quite liked. It is by Heidenberg et al. [2]. They break business value into five dimensions, each of which is ranked on an ordinal scale with four possible categories:

Monetary Value – this is the number calculated by, say, a business analyst.
Market Enabler – does delivering this feature create new market opportunity?
Technical Enabler – does this feature help prepare the company for other features?
Competence Growth – measures how much the work will improve the team’s skill.
Employee Satisfaction – do the developers like working on this feature?
Customer Satisfaction – how much will customers appreciate this work?

One of the big problems with agile planning is that making categories 2, 3, 4, 5 visible is often hard. It is comparatively easy to sell a customer a feature that ranks highly in monetary value or customer satisfaction – these are the slick and cool UI widgets, the mission-critical Word reporting function, and so on. But making architectural work (category 3) visible is very challenging. If we had this six-factor model, prioritizing important architectural work would be easier.

[1] Z. Racheva, M. Daneva, and K. Sikkel, “Value Creation by Agile Projects: Methodology or Mystery?,” presented at Product-Focused Software Process Improvement, 2009, vol. 32, no. 12, pp. 141–155.

[2] J. Heidenberg, M. Weijola, K. Mikkonen, and I. Porres, “A Model for Business Value in Large-Scale Agile and Lean Software Development,” presented at EUROSpi: Systems, Software and Services Process Improvement, 2012, pp. 49–60.

Final 2012 Full-Bracket Results

2013-03-12T10:49:23+00:00

Hopefully everyone had a chance to watch the exciting game between Kentucky and Kansas this past Monday. This post only covers the results of the full tournament bracket and not the second chance Sweet Sixteen bracket.

Here are the full standings, including ESPN analysts (E) and my own picks.

TheMatrixFactorizer	127
Jay Bilas (E)	126
Lee's picks	124
The Pain Machine	122
Baseline	120
Danny's Dangerous Picks	117
By The Numbers	104
Dick Vitale (E)	102
Obama	102
Predict the Madness	99
Ryan Boesch	98
TheSentinel	86
AJsMadness	73
machine_learning_first_try	45

Great contest this year and congratulations to this year's winner, TheMatrixFactorizer! It not only won the full-bracket contest, it also squeezed past ESPN analyst Jay Bilas by a point. Once again, machines triumph over humans in our contest. I, for one, welcome our new March Madness predicting robot overlords.

Machine March Madness 2013. Want to participate?

2013-03-12T10:48:57+00:00

Lee and I have been a bit busy this March Madness season and haven't organized the Machine March Madness competition as well as we usually do.

Now it's March, and the tournament is beginning in a little over a week.

This is a post to see if people would be interested in competing this year despite the short notice (update: there appears to be plenty of interest. It's on). Scott Turner has graciously volunteered to help out with data needs (update: data posted at the Google group), and there is still starter code and the Google group available as resources.

If you'd be interested in competing, please drop me an email or leave a note in the comments. Also, if you'd like to compete, but something is holding you back, please leave a note in the comments, and we'll see what we can do.

MDE gangnam style, anybody?

2013-03-11T11:00:39+00:00

Since MDE is not cool per se, we need to find original ways to convey our message. We already mentioned the creative marketing strategy of LucidChart creating youtube videos showing popular songs modeled as flowcharts but I like even more the initiative of creating a cover of a popular song like this NoSQL Style (Gangnam

The post MDE gangnam style, anybody? appeared first on Modeling Languages.

Automatic grading of UML models

2013-03-08T01:58:58+00:00

Dear lazyweb, I was wondering if any of you knows/uses any tool to automatically grade (UML) models created by students. I’m fully aware this far from easy (which probably explains why I don’t know of any such tool that is widely used, at least not among the people I know that teaches MDE). Clearly, a

The post Automatic grading of UML models appeared first on Modeling Languages.

Lightweight mechanism to reason on String values

2013-03-03T23:03:57+00:00

Our paper Lightweight String Reasoning in Model Finding by Fabian Büttner and myself has been accepted for publication in the Journal of Systems and Software Modeling (SoSyM). The full paper (preprint version) is freely available here. The abstract is the following: Models play a key role in assuring software quality in the model-driven approach. Precise

The post Lightweight mechanism to reason on String values appeared first on Modeling Languages.

neil

2013-03-02T21:58:44+00:00

In case this helps other people:

PennDOT rules on what paperwork is needed can be found here. In addition, keep in mind the following:

We needed a letter from my employer, I94+passports, old licences, 2 proofs of residence (lease+bills), and a rejection letter from Social Security for the SSN (for my wife) and a letter with the number on it for me (haven’t got the physical card yet).
If you have a H1-B, and your spouse has an H4, you will need to go with your spouse if s/he is getting a licence as well – you can’t go separately.
PennDOT does not take cheques drawn on foreign banks, only US banks. Fortunately you can get money orders easily at grocery stores. There is a Giant Eagle that does this near the Penn Hills licence centre.
Staff are pleasant but extremely over-worked, so be patient. Downtown Pittsburgh was less busy during the week than Penn Hills on the weekend.

Australis Curvy Tabs: More Progress!

2013-03-01T19:07:18+00:00

I wrote a while back about how Matt, Avi Halachmi and I have been ironing out performance problems with the Australis curvy tabs.

Well, it looks like that work is finally paying off.

Our SVG usage seemed to be the big slow-poke, and switching to PNGs gave us the boost that we needed.

But enough squawking, let’s see some charts.

Before Optimizations

Let’s compare – here’s a chart showing the difference between pre-curves and post-curves, before our optimizations:

Here’s the before shot

Note: it’s been a while since I’ve done data visualization work. I think the last time I did this was in grad school. So there might be way better ways of visualizing this data, but I just chose the easiest chart I could manage with Google Docs. Just go with it.

Let me describe what you’re seeing here – we take samples every time a tab opens, and every time a tab closes*. What we’re measuring is the interval time (how long it takes before we start drawing the next frame), and the paint time (how long it takes to actually draw a frame).

The blue bars represent the performance measurements we took on a build using the default theme. The red bars represent the performance measurements we took using the Australis curvy tabs.

This is where my graph could probably be clearer – in each group of four bars, the left two represent interval times, and the right two represent paint times.

So, hand-wavey interpretation – we regressed in terms of performance in both painting, and frame intervals, for tab opening and closing.

So that’s what we started with. And then we did our optimizations. So where did we get to?

After Optimizations

Here’s the after shot!

The red bars shrunk, meaning that we got faster for both interval and paint times. In fact, for tab close, we beat the old theme! And we’re really super-close for tab open.

Pretty good!

Curvy tabs for all

Last night, Matt landed our optimization patches, as well as preliminary curvy tab work for OSX* and Linux GTK on our UX branch. So, if you’re on the UX branch (and why aren’t you?), you should be receiving a build soon with some curvy tabs. They’re not perfect, not by a long shot, but we’re getting into the polish stage now, which is good.

* Some notes on our measuring methodology. All tests were performed on a low-powered Acer Aspire One netbook. Intel Atom n450 processor (1.66Ghz), 1GB of RAM, running Windows 7. The device has no graphics acceleration support. We also switched to the classic theme to avoid glass. Avi wrote a patch that opened and closed a tab 15 times, and averaging the frame intervals and paint times for each frame. Those were averaged over the 15 openings and closings. We then ran that test 4 times, giving the machine time to “relax” in between, and averaged our results.

* We don’t have hi-dpi support yet, so if you’re on a Mac with a Retina display, your curves might be fuzzy. We’re working on it.

A Software Carpentry Boot Camp for Women in Science and Engineering

2013-02-28T16:57:59+00:00

Software Carpentry is pleased to announced a two-day software skills boot camp for women in science and engineering, to be held in Boston this June. We’re currently trying to raise the $6000 needed to give 120 grad students (and others) a chance to improve their research computing skills while networking with peers; donations would be very welcome.

Why a boot camp specifically aimed at women? Because a large body of research has shown that without initiatives like this, the cycle of low participation today leading to low participation tomorrow will continue unchecked. For example, WiT reports:

In the Bayer Facts of Science Education XIV survey, women and minorities raised a number of barriers in their path to STEM careers, including:

Lack of mentors (50%)

Lack of role models (49%)

Stereotypes adversely affecting women and minorities (39%)

Lack of communication from STEM industry (39%)

Self doubt (35%)

Cost of education (31%)

“Sense of isolation” (29%)

A lack of solid math and science education in poorer schools (24%)

Issues like the lack of role models, lack of mentors, stereotypes, and a sense of isolation are effectively addressed by getting a bunch of women together in one room. We’re not just presenting the Software Carpentry material, we are also creating a community of women who will support each other in tangible and intangible ways. If you would like to learn more, one of the most thorough and most readable pieces of research in this area remains Margolis and Fisher’s Unlocking the Clubhouse, which reports their work in the late 1990s and early 2000s at Carnegie-Mellon.

Introduction to Domain-Specific Languages (slides)

2013-02-27T14:20:58+00:00

Javier Cánovas, a postdoctoral fellow in AtlanMod, recently gave on tutorial on DSLs (Domain-specific Languages). I think the set of slides he prepared are really good and could be interesting for many of the readers of the portal. So, if you want to know what a DSL is, how to build them, … take a

The post Introduction to Domain-Specific Languages (slides) appeared first on Modeling Languages.

Beautiful models and modeling languages in Pinterest

2013-02-26T15:40:50+00:00

I’ve now created a board on pinterest to feature “beautiful” models and modeling languages (you can see it here). The idea is to feature examples of different types of models and DSLs so that people can get astonished by our huge creativity. More seriously, the idea is to try to highlight both the variety of

The post Beautiful models and modeling languages in Pinterest appeared first on Modeling Languages.

Two AtlanMod news: official INRIA team and open postdoc position

2013-02-22T11:20:19+00:00

This week the flu is getting the best of me but I still I wanted to get the word out about two important AtlanMod news: After two years and half in the making, AtlanMod is now a full INRIA team (until now we were an INRIA team already but with an “under-evaluation” tag; so from

The post Two AtlanMod news: official INRIA team and open postdoc position appeared first on Modeling Languages.

Congratulations to Christian Muise

2013-02-15T23:11:54+00:00

U of Toronto PhD student Christian Muise created an application that was selected as a winner of Google’s Places API Challenge. The competition brought together 87 developers from 27 countries, challenging them to build apps that address some of the most pressing needs in our communities. The top three applications were declared winners, and will be showcased at Google I/O, the company’s annual developer conference.

Muise’s award-winning app, TTC Pass, is “a website that allows for collaborative editing of the locations for purchasing various transit fares” in Toronto. See a video of the app, or the Google Places API Developer Challenge webpage.

How Big is the Climate Change Deficit?

2013-02-15T21:23:02+00:00

Last week, Damon Matthews from Concordia visited, and gave a guest CGCS lecture, “Cumulative Carbon and the Climate Mitigation Challenge”. The key idea he addressed in his talk is the question of “committed warming” – i.e. how much warming are we “owed” because of carbon emissions in the past (irrespective of what we do with emissions in the future). But before I get into the content of Damon’s talk, here’s a little background.

The question of ‘owed’ or ‘committed’ warming arises because we know it takes some time for the planet to warm up in response to an increase in greenhouse gases in the atmosphere. You can calculate a first approximation of how much it will warm up from a simple energy balance model (like the ones I posted about last month). However, to calculate how long it takes to warm up you need to account for the thermal mass of the oceans, which absorb most of the extra energy and hence slow the rate of warming of surface temperatures. For this you need more than a simple energy balance model.

You can do a very simple experiment with a Global Circulation Model, by setting CO2 concentrations at double their pre-industrial levels, and then leave them constant at this level, to see how long the earth takes to reach a new equilibrium temperature. Typically, this takes several decades, although the models differ on exactly how long. Here’s what it looks like if you try this with EdGCM (I ran it with doubled CO2 concentrations starting in 1958):

Of course, the concentrations would never instantaneously double like that, so a more common model experiment is to increase CO2 levels gradually, say by 1% per year (that’s a little faster than how they have risen in the last few decades) until they reach double the pre-industrial concentrations (which takes approx 70 years), and then leave them constant at that level. This particular experiment is a standard way of estimating the Transient Climate Response - the expected warming at the moment we first reach a doubling of CO2 - and is included in the CMIP5 experiments. In these model experiments, it typically takes a few decades more of warming until a new equilibrium point is reached, and the models indicate that the transient response is expected to be a little over half of the eventual equilibrium warming.

This leads to a (very rough) heuristic that as the planet warms, we’re always ‘owed’ almost as much warming again as we’ve already seen at any point, irrespective of future emissions, and it will take a few decades for all that ‘owed’ warming to materialize. But, as Damon argued in his talk, there are two problems with this heuristic. First, it confuses the issue when discussing the need for an immediate reduction in carbon emissions, because it suggests that no matter how fast we reduce them, the ‘owed’ warming means such reductions will make little difference to the expected warming in the next two decades. Second, and more importantly, the heuristic is wrong! How so? Read on!

For an initial analysis, we can view the climate problem just in terms of carbon dioxide, as the most important greenhouse gas. Increasing CO2 emissions leads to increasing CO2 concentrations in the atmosphere, which leads to temperature increases, which lead to climate impacts. And of course, there’s a feedback in the sense that our perceptions of the impacts (whether now or in the future) lead to changed climate policies that constrain CO2 emissions.

So, what happens if we were to stop all CO2 emissions instantly? The naive view is that temperatures would continue to rise, because of the ‘climate commitment’ - the ‘owed’ warming that I described above. However, most models show that the temperature stabilizes almost immediately. To understand why, we need to realize there are different ways of defining ‘climate commitment’:

Zero emissions commitment – How much warming do we get if we set CO2 emissions from human activities to be zero?
Constant composition commitment – How much warming do we get if we hold atmospheric concentrations constant? (in this case, we can still have some future CO2 emissions, as long as they balance the natural processes that remove CO2 from the atmosphere).

The difference between these two definition is shown here. Note that in the zero emissions case, concentrations drop from an initial peak, and then settle down at a lower level:

The model experiments most people are familiar with are the constant composition experiments, in which there is continued warming. But in the zero emissions scenarios, there is almost no further warming. Why is this?

The relationship between carbon emissions and temperature change (the “Carbon Climate Response”) is complicated, because it depends two factors, each of which is complicated by (different types of) inertia in the system:

Climate Sensitivity – how much temperature changes in response to difference levels of CO2 in the atmosphere. The temperature response is slowed down by the thermal inertia of the oceans, which means it takes several decades for the earth’s surface temperatures to respond fully to a change in CO2 concentrations.
Carbon sensitivity – how much concentrations of CO2 in the atmosphere change in response to different levels of carbon emissions. A significant fraction (roughly half) of our CO2 emissions are absorbed by the oceans, but this also takes time. We can think of this as “carbon cycle inertia” – the delay in uptake of the extra CO2, which also takes several decades. [Note: there is a second kind of carbon system inertia, by which it takes tens of thousands of years for the rest of the CO2 to be removed, via very slow geological processes such as rock weathering.]

It turns out that the two forms of inertia roughly balance out. The thermal inertia of the oceans slows the rate of warming, while the carbon cycle inertia accelerates it. Our naive view of the “owed” warming is based on an understanding of only one of these, the thermal inertia of the ocean, because much of the literature talks only about climate sensitivity, and ignores the question of carbon sensitivity.

The fact that these two forms of inertia tend to balance leads to another interesting observation. The models all show an approximately linear response to cumulative emissions. For example, here are the CMIP3 models, used in the IPCC AR4 report (the average of the models, indicated by the arrow, is around 1.6C of warming per 1,000 gigatonnes of carbon):

The same relationship seems to hold for the CMIP5 models, many of which now include a dynamic carbon cycle:

This linear relationship isn’t determined by any physical properties of the climate system, and probably won’t hold in much warmer or cooler climates, nor when other feedback processes kick in. So we could say it’s a coincidental property of our current climate. However, it’s rather fortuitous for policy discussions.

Historically, we have emitted around 550 billion tonnes since the beginning of the industrial era, which gives us an expected temperature response of around 0.9°C. If we want to hold temperature rises to be no more than 2°C of warming, total future emissions should not exceed a further 700 billion tonnes of Carbon. In effect, this gives us a total worldwide carbon budget for the future. The hard policy question, of course, is then how to allocate this budget among the nations (or people) of the world in an equitable way.

[A few years ago, I blogged about a similar analysis, which says that cumulative carbon emissions should not exceed 1 trillion tonnes in total, ever. That calculation gives us a smaller future budget of less then 500 billion tonnes. That result came from analysis using the Hadley model, which has one of the higher slopes on the graphs above. Which number we use for a global target then might depend on which model we believe gives the most accurate projections, and perhaps how we also factor in the uncertainties. If the uncertainty range across models is accurate, then picking the average would give us a 50:50 chance of staying within the temperature threshold of 2°C. We might want better odds than this, and hence a smaller budget.]

In the National Academies report in 2011, the cumulative carbon budgets for each temperature threshold were given as follows (note the size of the uncertainty whiskers on each bar):

[For a more detailed analysis see: Matthews, H. D., Solomon, S., & Pierrehumbert, R. (2012). Cumulative carbon as a policy framework for achieving climate stabilization. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 370(1974), 4365–79. doi:10.1098/rsta.2012.0064]

So, this allows us to clear up some popular misconceptions:

The idea that there is some additional warming owed, no matter what emissions pathway we follow is incorrect. Zero future emissions means little to no future warming, so future warming depends entirely on future emissions. And while the idea of zero future emissions isn’t policy-relevant (because zero emissions is impossible, at least in the near future), it does have implications for how we discuss policy choices. In particular, it means the idea that CO2 emissions cuts will not have an effect on temperature change for several decades is also incorrect. Every tonne of CO2 emissions avoided has an immediate effect on reducing the temperature response.

Another source of confusion is the emissions scenarios used in the IPCC report. They don’t diverge significantly for the first few decades, largely because we’re unlikely (and to some extent unable) to make massive emissions reductions in the next 1-2 decades, because society is very slow to respond to the threat of climate change, and even when we do respond, the amount of existing energy infrastructure that has to be rebuilt is huge. In this sense, there is some inevitable future warming, but it comes from future emissions that we cannot or will not avoid. In other words, political, socio-economic and technological inertia are the primary causes of future climate warming, rather than any properties of the physical climate system.

Anybody using both MDA platform-independent AND platform-specific models?

2013-02-13T23:42:55+00:00

Model-driven Architecture (MDA) is the OMG´s view of model-driven development (see clarifying concepts: MBE vs MDE vs MDD vs MDA). According to the OMG, “a complete MDA application consists of a definitive PIM (platform-independent model), plus one or more PSMs (platform-specific models) and complete implementations, one on each platform that the application developer decides to

The post Anybody using both MDA platform-independent AND platform-specific models? appeared first on Modeling Languages.

Why Don’t I Just Go Ahead And Frame That Debate In A Way That Guarantees I’ll Win?

2013-02-10T04:20:03+00:00

Clay Shirky’s latest piece, which points out that offline colleges are broken, has been attracting a fair bit of attention. What isn’t is the way he and others are trying to frame the debate, which goes something like, “Yeah, MOOCs aren’t perfect, but have you actually looked at what most non-elite colleges actually offer?” This boils down to, “Our bad is slightly less bad than their bad,” but blithely ignores the fact that something much better than either exists. As Anu Partanen pointed out over a year ago, though, that “something better” would demand some uncomfortable soul-searching…

The Larch Environment

2013-02-09T20:22:16+00:00

G.W. French, J.R. Kennaway, and A.M. Day: “Programs as visual, interactive documents.” Software – Practice and Experience (2013), DOI: 10.1002/spe.2182.

We present a novel approach to combined textual and visual programming by allowing visual, interactive objects to be embedded within textual source code and segments of source code to be further embedded within those objects. We retain the strengths of text-based source code, while enabling visual programming where it is beneficial. Additionally, embedded objects and code provide a simple object-oriented approach to adding a visual form of LISP-style macros to a language. The ability to freely combine source code and visual, interactive objects with one another allows for the construction of interactive programming tools and experimentation with novel programming language extensions. Our visual programming system is supported by a type coercion-based presentation protocol that displays normal Java and Python objects in a visual, interactive form. We have implemented our system within a prototype interactive programming environment called ‘The Larch Environment’.

This is cool: see their site for more information.

Planning for new faculty positions in computer science

2013-02-08T23:11:52+00:00

Like most universities, U of T had a hiring freeze for new faculty for the last few years, as we struggled with budget cuts. Now, we’re starting to look at hiring again, to replace faculty we lost over that time, and to meet the needs of rapidly growing student enrolments. Our department (Computer Science) is just beginning the process of deciding what new faculty positions we wish to argue for, for next year. This means we get to engage in a fascinating process of exploring what we expect to be the future of our field, and where there are opportunities to build exciting new research and education programs. To get a new faculty position, our department has to make a compelling case to the Dean, and the Dean has to balance our request with those from 28 other departments and 46 interdisciplinary groups. So the pitch has to be good.

So here’s my draft pitch:

(1) Create a joint faculty position between the Department of Computer Science and the new School of Environment.

Last summer U of T’s Centre for Environment was relaunched as a School of Environment, housed wholly within the Faculty of Arts and Science. As a school, it can now make up to 49% faculty appointments. [The idea is that to do interdisciplinary research, you need a base in a home department/discipline, where your tenure and promotion will be evaluated, but would spend half your time engaged in inter-disciplinary research and teaching at the School. Hence, a joint position for us would be 51% CS and 49% in the School of Environment.]

A strong relationship between Computer Science and the School of Environment makes sense for a number of reasons. Most environmental science research makes extensive use of computational modelling as a core research tool, and the environmental sciences are one of the greatest producers of big data. As an example, the Earth System Grid currently stores more than 3 petabytes of data from climate models, and this is expected to grow to the point where by the end of the decade a single experiment with a climate model would generate an exabyte of data. This creates a number of exciting opportunities for application of CS tools and algorithms, in a domain that will challenge our capabilities. At the same time, this research is increasingly important to society, as we seek to find ways to feed 9 billion people, protect vital ecosystems, and develop strategies to combat climate change.

There are a number of directions we could go with such a collaboration. My suggestion is to pick one of:

Climate informatics. A small but growing community is applying machine learning and data mining techniques to climate datasets. Two international workshops have been held in the last two years, and the field has had a number of successes in knowledge discovery that have established its importance to climate science. For a taste of what the field covers, see the agenda of the last CI Workshop.
Computational Sustainability. Focuses on the decision-support needed for resource allocation to develop sustainable solutions in large-scale complex adaptive systems. This could be viewed as a field of applied artificial intelligence, but to do it properly requires strong interdisciplinary links with ecologists, economists, statisticians, and policy makers. This growing community has run run an annual conference, CompSust, since 2009, as well as tracks at major AI conferences for the last few years.
Green Computing. Focuses on the large environmental footprint of computing technology, and how to reduce it. Energy efficient computing is a central concern, although I believe an even more interesting approach is when we take a systems approach to understand how and why we consume energy (whether in IT equipment directly, or in devices that IT can monitor and optimize). Again, a series of workshops in the last few years has brought together an active research community (see for example, Greens’2013),

(2) Hire more software engineering professors!

Our software engineering group is now half the size it was a decade ago, as several of our colleagues retired. Here’s where we used to be, but that list of topics and faculty is now hopelessly out of date. A decade ago we had five faculty and plans to grow this to eight by now. Instead, because of the hiring freeze and the retirements, we’re down to three. There were a number of reasons we expected to grow the group, not least because for many years, software engineering was our most popular undergraduate specialist program and we had difficulty covering all the teaching, and also because the SE group had proved to be very successful in bringing in research funding, research prizes, and supervising large numbers of grad students.

Where do we go from here? Deans generally ignore arguments that we should just hire more faculty to replace losses, largely because when faculty retire or leave, that’s the only point at which a university can re-think its priorities. Furthermore, some of our arguments for a bigger software engineering group at U of T went away. Our department withdrew the specialist degree in software engineering, and reduced the number of SE undergrad courses, largely because we didn’t have the faculty to teach them, and finding qualified sessional instructors was always a struggle. In effect, our department has gradually walked away from having a strong software engineering group, due to resource constraints.

I believe very firmly that our department *does* need a strong software engineering group, for a number of reasons. First, it’s an important part of an undergrad CS education. The majority of our students go on to work in the software industry, and for this, it is vital that they have a thorough understanding of the engineering principles of software construction. Many of our competitors in N America run majors and/or specialist programs in software engineering, to feed the enormous demand from the software industry for more graduates. One could argue that this should be left to the engineering schools, but these schools tend to lack sufficient expertise in discrete math and computing theory. I believe that software engineering is rooted intellectually in computer science and that a strong software engineering program needs the participation (and probably the leadership) of a strong computer science department. This argument suggests we should be re-building the strength in software engineering that we used to have in our undergrad program, rather than quietly letting it whither.

Secondly, the complexity of modern software systems makes software engineering research ever more relevant to society. Our ability to invent new software technology continues to outpace our ability to understand the principles by which that software can be made safe and reliable. Software companies regularly come to us seeking to partner with us in joint research and to engage with our grad students. Currently, we have to walk away from most of these opportunities. That means research funding we’re missing out on.

It’s time to teach history of programming languages

2013-02-05T07:29:21+00:00

One of the first concepts I show when teaching Model-driven engineering is the MDE equation (Models + Transformations = Software ) which obviously revisits the well-known Niklaus Wirth‘s equation: Algorithms + Data Structures = Programs. I thought that by linking the two, it would be easier for the students to grasp the main aspects of

The post It’s time to teach history of programming languages appeared first on Modeling Languages.

DSL Engineering – book announcement

2013-01-31T11:00:21+00:00

Markus Völter talks about his recent (and highly anticipated ) new book on domain-specific languages. Enter Markus. Domain-Specific Languages are modeling or programming languages that are tailored to a particular application domain. By incorporating knowledge about that domain, DSLs can lead to more concise and more analyzable programs, improved code quality, tighter stakeholder integration and

The post DSL Engineering – book announcement appeared first on Modeling Languages.

Releng 2013

2013-01-30T13:08:56+00:00

Releng 2013 is a one day workshop (co-located with ICSE) to bring together release engineers and researchers to discuss the challenges in release engineering and develop areas for further research. Areas of discussion include research and practice of all activities in between regular development and actual usage of a software product by the end user, i.e., integration, build, test execution, packaging and delivery of software. The conference is May 20 in San Francisco and deadline for submission of talks in February 7th. For more information see http://releng.polymtl.ca.

Half a Hundred

2013-01-30T02:51:06+00:00

And then Maddie said, “Daddy, guess what? You’re half a hundred years old!”

Is Model-driven Engineering (dis)covering the Social Web?

2013-01-29T23:16:13+00:00

In today’s guest post, Marco Brambilla presents his work on bridging the gap between model-driven engineering and the social web. IMHO, this is a key aspect that has been largely ignored by MDE tools so far. Most applications integrate nowadays social components and we must be able to model (and “generate”) the relationship of our

The post Is Model-driven Engineering (dis)covering the Social Web? appeared first on Modeling Languages.

Top 10 lies about MDE

2013-01-28T13:54:55+00:00

Richard Paige and Louis Rose just published in the JOT journal the article/post Lies, Damned Lies and UML2Java where they summarize the 10 most dangerous misconceptions about model-driven engineering. For those with no time, this is the list headlines (you´ll need to go to the journal page to read the full details, no paywall!): MDE

The post Top 10 lies about MDE appeared first on Modeling Languages.

Simple Climate Models to play with in the classroom

2013-01-25T20:09:07+00:00

I’ve been collecting examples of different types of climate model that students can use in the classroom to explore different aspects of climate science and climate policy. In the long run, I’d like to use these to make the teaching of climate literacy much more hands-on and discovery-based. My goal is to foster more critical thinking, by having students analyze the kinds of questions people ask about climate, figure out how to put together good answers using a combination of existing data, data analysis tools, simple computational models, and more sophisticated simulations. And of course, learn how to critique the answers based on the uncertainties in the lines of evidence they have used.

Anyway, as a start, here’s a collection of runnable and not-so-runnable models, some of which I’ve used in the classroom:

Simple Energy Balance Models (for exploring the basic physics)

Zero dimensional Energy Balance from Wolfram. Allows you to adjust one parameter, the greenhouse effect, and explore the resulting equilibrium global temperature. Also serves to show off Wolfram’s Computable Document Format (CDF) which might be a neat way to share simple models with students.
A simple spreadsheet zero-dimension energy balance model from Climateprediction.net. I like the idea of getting the students to do this in spreadsheets, because most of them already understand spreadsheets. This one has a parameter for heat capacity, so you can see how long it takes to reach a new equilibrium temperature.
Energy Balance Model with ability to add atmosphere layers from Phet. This simulation shows off how greenhouse gas molecules interact with photons, trapping them or allowing them through. Pretty neat for giving students an intuition for how the greenhouse effect works.
One-dimensional Energy Balance model from Shodor. Calculates the equilibrium temperature for each latitude zone on the planet, allowing you to specify cloud and ice albedo, solar constant, longwave radiation loss, and starting temperatures for each zone. Not very usable, but good illustration of what a 1-dimensional model might do. (Note: Shodor also have a great ecosystem sim with rabbits and wolves, and a disease transmission sim).
A one-layer energy-balance model developed by Michael Mann at Penn state, for use in his course on global warming. Allows you to alter different feedback factors (albedo, clouds, ice, water vapour), to test their effect on temperature and climate sensitivity.
A Java applet Global Energy Balance Model, from Rob MacKay at Clark College (part of a whole collection of Java Physlets from Davidson University)
The Global Equilibrium Energy Balance Interactive Tinker Toy (Geebitt) from Chris Petersen at NASA GISS, a more sophisticated spreadsheet model.
An energy flow simulator written in NetLogo from Northwestern U, showing how heat transfer works between the sun, atmosphere and ground, and allowing you to adjust CO2 and cloudiness dynamically.

General Circulation Models (for studying earth system interactions)

EdGCM – an educational version of the NASA GISS general circulation model (well, an older version of it). EdGCM provides a simplified user interface for setting up model runs, but allows for some fairly sophisticated experiments. You typically need to let the model run overnight for a century-long simulation.
Portable University Model of the Atmosphere (PUMA) – a planet Simulator designed by folks at the University of Hamburg for use in the classroom to help train students interested in becoming climate scientists.

Integrated Assessment Models (for policy analysis)

C-Learn, a simple policy analysis tool from Climate Interactive. Allows you to specify emissions trajectories for three groups of nations, and explore the impact on global temperature. This is a simplified version of the C-ROADS model, which is used to analyze proposals during international climate treaty negotiations.
Java Climate Model (JVM) – a detailed desktop assessment model that offers detailed controls over different emissions scenarios and regional responses.

Systems Dynamics Models (to foster systems thinking)

Bathtub Dynamics and Climate Change from John Sterman at MIT. This simulation is intended to get students thinking about the relationship between emissions and concentrations, using the bathtub metaphor. It’s based on Sterman’s work on mental models of climate change.
The Climate Challenge: Our Choices, also from Sterman’s team at MIT. This one looks fancier, but gives you less control over the simulation – you can just pick one of three emissions paths: increasing, stabilized or reducing. On the other hand, it’s very effective at demonstrating the point about emissions vs. concentrations.
Carbon Cycle Model from Shodor, originally developed using Stella by folks at Cornell.
And while we’re on systems dynamics, I ought to mention toolkits for building your own systems dynamics models, such as Stella from ISEE Systems (here’s an example of it used to teach the global carbon cycle).

Other Related Models

A Kaya Identity Calculator, from David Archer at U Chicago. The Kaya identity is a way of expressing the interaction between the key drivers of carbon emissions: population growth, economic growth, energy efficiency, and the carbon intensity of our energy supply. Archer’s model allows you to play with these numbers.
An Orbital Forcing Calculator, also from David Archer. This allows you to calculate what the effect changes in the earth’s orbit and the wobble on its axis have on the solar energy that the earth receives, in any year in the past of future.

Useful readings on the hierarchy of climate models

Isaac Held on how simple models are like fruit flies in biological research – we can use them to give us a first test of whether an idea makes sense.
John Baez has been collecting basic introductions to the different types of climate model, and has an entire grad course that walks through the mathematics of climate modelling.

neil

2013-01-25T19:52:32+00:00

Material

The course covers software architecture, with a focus on quality attributes, security, and formal methods. I liked the range of material, even though my expertise is limited with formal methods. It is difficult to teach architecture to students in a 3 month time frame, so we expanded using the AOSA textbooks. Students did a presentation for five minutes as a way of exposing them to various different architectures.

The other large component of the class is a course project. In this semester they had to build a location-aware, social application. There were great projects including my personal fave, a zombie fighting location-based game.

My favorite part of this course, like the third-year course, is seeing how the students approach the project. Some are truly excellent coders and put an enormous amount of effort into the project.

I introduced a few new lecture topics in addition to the ones pre-existing. I added a topic on Service Orientation and SOA, important trend in particular in enterprise architecture; a new topic on REST, which was well connected to the project; and a topic on agility and architecture, based in part on the book by Dean Leffingwell, Agile Requirements. I thought all of these were useful, although they tend to be less easily tested than e.g. model checking, so perhaps students are just forgetting them. I even mentioned CMMI!

Overall

I feel that these types of courses in SE should be more about reflective apprenticeship than the lecture and project model. In other words, there should be more focus on feedback about the way in which students do design, more experiential learning, and less memorization of specific techniques such as formal methods (which should really be in a separate course, in my opinion). Mark Guzdial called this reflective apprenticeship and points to the work of Donald Schön.

A parallel might be drawn with the way law schools operate. If you want to learn about how one practices law, argues cases, etc., you hire experienced practitioners as adjunct faculty. There seems to me to be a real difference between the skills of an academic SE faculty member and a person who has spent years building high-availability, mission critical software. There are a few of the latter at UBC, such as Philippe Kruchten, but in general it is exceedingly difficult for them to be hired without academic credentials. Not to mention the increasingly large salary gap between industrial SE and academic SE.

The other thing I disliked is that it seemed to me that a few students hid out during the project, latching on remora-like to their more capable teammates to secure a good mark with no work. It irritates me to pass students who are not able to write code (not good code, or even mediocre code – just not write code! See Fizzbuzz). It is very difficult to (defensibly) identify these people, however. One technique which I should have used is to ask questions of the indifividual students during their final demos. This would help to identify who actually knows what the heck is going on.

Academic dishonesty and software engineering

On the one hand we cannot prohibit collaboration and code re-use: these are fundamental practices in software engineering. On the other hand, we need to assess the student’s actual contribution. I had a few interesting cases that suggest our pollicies in this area need more attention.
1. One group used a sample Ruby on Rails project to bootstrap their application. It came with most of the controllers they needed. They then customized the UI and logic to implement the functionality (poorly).
2. Another team hired a third-party designer to do custom artwork for the project (which looked fantastic).
3. A different team had a friend with web design expertise work on the CSS for the project.
4. Several teams used Twitter’s Bootstrap UI library or JqueryUI to simplify their efforts on the design end.
5. Many groups used third-party libraries to simplify their life, like JQuery, Rails, image libraries, etc.

Obviously, most of these are exactly what would happen in industry. On the other hand, it definitely gains one an advantage. The Twitter Bootstrap apps all looked an order of magnitude better than the custom apps.

My principle was the remixing and reuse was fine, as long as it was properly acknowledged. In the design case, we could try to discount that aspect of the UI in the marking. But it is almost certainly the case that some groups did NOT acknowledge their use of other people’s IP, and yet benefited from it. I don’t have a good solution to the problem of detecting code reuse. And furthermore, the burden of proof is pretty high to call something cheating, and requires more than a gut feeling or a commit to Github that touched hundreds of files at once (i.e. a bulk commit of 3rd party code).

IT role

One of the things which I think will only become more prevalent is the use of third-party services to manage the course. In the past, students would use CS department machines and servers to do their assignments, a CS database server, and store code on the department subversion or IBM RTC servers.

This semester I don’t think we used a single department resource, save for email (and that only because I was forced to for privacy reasons) and the course webpage. Class discussions took place on Piazza.com; code and issues were managed with Github, and students nearly always have their own laptops and Android devices (there was not a single group that chose iOS, incidentally, although nearly half the class has Macbooks. I think the 99$ fee is a real stumbling block – that and Objective-C).

I did not get any support or materiel from the department, apart from the classroom and photocopier. I could just as easily have run this course from my home. So what should the IT section do? They could manage Github for me (they were extremely reluctant to do this, and very hesitant about even installing Bugzilla, apparently). They could provide more AV services to record classes. They could manage virtual machines for me, so that each student could install the same setup — things like Puppet and Vagrant will be key in the coming years.

Finally, the UBC wireless infrastructure is truly terrible. You get better wifi at the Starbucks. Latency between two machines in my office was 200ms! The connection is constantly dropping or extremely slow, such that even demos are affected by the web performance.

Student perceptions

In an unscientific survey, I asked the following questions:

1. How could the TAs and myself improve your experience?

Students were either positive (but they had names attached to their responses, so that isn’t unexpected) or asked for more help. One of the big challenges they face is sorting out silly configuration problems. They would like more advice on design choices as well. I think this is a real opportunity to make the project more like an apprenticeship model, a la Software Craftsmanship: take some senior developers, get them to do an hour of code review, an hour of design feedback, etc. And there seem to be many companies eager to help out (and recruit) for whom this might be doable.The other issue was that due to 4) below, TAs and myself often did not know much about the technology (e.g., Microsoft’s C#/Azure platforms). However, this is definitely a learning objective in the course. Admittedly in industry one would often be able to ask senior devs these questions. However, the ability to track these answers down is invaluable, I feel.

2. Were the AOSA readings useful?

Most students responded that they appreciated the opportunity to present to a large audience. But the overall lessons of the architecture in these systems was lost on them, because it did not have a lot of relevance to the project, which consumed the majority of the time. Asking exam questions was difficult, as there was a lot of material that would have to be studied. I think I would keep this module but be more strict about the time limit (5mins) and give some introductory examples/prep before hand.

3. Did Github work for you?

Students loved Github. Egit was less good (and personally I find it less usable than the command line). A major improvement over RTC.

4. Was the freedom to choose language good or bad?

Most students loved this aspect as well. In the past the project, worth 40% of the course, has been in e.g. Java+Tomcat for everyone. Feedback here indicated that main problems were finding team members with similar interests (in, e.g. RoR), getting help from TAs, and a possible penalty on the final, where the code snippets are in Java.

5. What annoyed you about the project?

Unsurprisingly, most complaints were about the time it took – one student spent 80 hours over two weekends on what was, however, a really cool UI – and the vagaries of group work with fellow team members, some of whom get sick, abandon their teammates, or simply are not good at programming. Students would appreciate more help on scoping the project, and getting the thing started earlier. We tried to address this by insisting on an early ‘project idea’ review in the first 3 weeks, and by doing a 30 minute design review midway. However, some people have to learn the hard way, and ultimately, we are constrained by how many teams there are – 27 in this case. Multiply that by 30mins and you can see the magnitude of the challenge. I had 3 TAs to help, but that is still a ton of work. And I think students got frustrated, since they see a 1-1 interaction, not 27-1 that I see.

RR: Should we resurrect Software Engineering? by Jean Bézivin

2013-01-23T11:49:59+00:00

Always very interesting to listen Jean talking about software engineering and role of MDE in it (among many other things). You can read the abstract of this CHOOSE forum 2012 talk in Jean´s blog and check the presentation in slideshare (and below) Bern.jb from Jean Bézivin If you liked this post, you should subscribe to

The post RR: Should we resurrect Software Engineering? by Jean Bézivin appeared first on Modeling Languages.

Planet DCS@UofT

Video on climate change without using climate models

Top 5 software modeling posts – January/March 2013

If Anything, They Overestimate My Javascript

From spreadsheets to dataflow diagrams

Merging is the Real Revolution

MoDELS 2013 – List of Tutorials and Workshops available

Handling Unsolicited Commercial Email

Cameron Neylon Speaking in Toronto on May 1, 2013

UMLtoSQL, UMLtoSymfony/PHP and UMLtoDjango/Python are now open source

EU Innovation & Competitiveness 2011

More than 25 universities using our MDE book

The Internet: Saving Civilization or Trashing the Planet?

Unsustainable

Congratulations to the Machine March Madness Winner

Movisa: A DSL tool for Human Machine Interfaces (HMI) in Industrial Automation

Machine March Madness Final Four Outlook

Modelling allergy – a common hypersensitivity disorder

Special issue of GMD: Community software to support the delivery of CMIP5

That’s My Secret, Captain…

User interface modeling with the new IFML standard

js-sequence-diagrams: A text to UML Sequence diagrams with JS

Upset Analysis by Scott Turner

Correct Upset Predictions

Incorrect Upset Predictions

Upset Profits

Conclusions

Some Final Four Analysis...

The 2013 Machine March Madness Field

Predicting March Madness by Jasper Snoek

And we're off!

Model-driven learning design

Final Stretch for Machine March Madness Predictions

March Madness Team Embeddings

MDE to the people – Upcoming talk

Worried about your RSS subscription? – Follow the modeling news by email

Structured Flowcharts outperform pseudocode

neil

Final 2012 Full-Bracket Results

Machine March Madness 2013. Want to participate?

MDE gangnam style, anybody?

Automatic grading of UML models

Lightweight mechanism to reason on String values

neil

Australis Curvy Tabs: More Progress!

Before Optimizations

After Optimizations

Curvy tabs for all

A Software Carpentry Boot Camp for Women in Science and Engineering

Introduction to Domain-Specific Languages (slides)

Beautiful models and modeling languages in Pinterest

Two AtlanMod news: official INRIA team and open postdoc position

Congratulations to Christian Muise

How Big is the Climate Change Deficit?

Anybody using both MDA platform-independent AND platform-specific models?

Why Don’t I Just Go Ahead And Frame That Debate In A Way That Guarantees I’ll Win?

The Larch Environment

Planning for new faculty positions in computer science

Top 5 posts on software modeling and MDE of the year (2012)

It’s time to teach history of programming languages

DSL Engineering – book announcement

Releng 2013

Half a Hundred

Is Model-driven Engineering (dis)covering the Social Web?

Top 10 lies about MDE

Simple Climate Models to play with in the classroom

Simple Energy Balance Models (for exploring the basic physics)

General Circulation Models (for studying earth system interactions)

Integrated Assessment Models (for policy analysis)

Systems Dynamics Models (to foster systems thinking)

Other Related Models

Useful readings on the hierarchy of climate models

neil

Material

Overall

Academic dishonesty and software engineering

IT role

Student perceptions

RR: Should we resurrect Software Engineering? by Jean Bézivin