For those who don’t know, Warren was the author of PyMol, the best pure viewer I have ever used, and a revolution in it’s day. An open source viewer better than anything you paid for. It was certainly a key reason why companies like Accelrys ended up releasing free molecular viewers. Seeing slide after slide at conferences showing figures generated in PyMol was a clear sign that it was possible for open source software to be best in breed.

Warren Delano, entrepreneur, open source advocate, scientist, and maker of one of the best software packages ever, Thank You!!! You will be missed. I hope the community can pick up PyMol and continue developing it

Chris Anderson once infamously wrote The data deluge makes the scientific method obsolete, an opinion that I do not share. Eric Drexler on the other hand comes at this new age of data driven science with the right mindset. In a post on data explosion and the scientific method, Eric writes

Tradition demands that science always be hypothesis-driven: First, try to guess the truth, and only afterward collect experimental data to test whether the guess predicts the results. Indeed, this has been termed “The Scientific Method”. The new data-driven approach suggests that we collect data first, then see what it tells us. This becomes practical when experimental methods can amass enormous amounts of data, enough data to test more hypotheses than any mortal scientist could conceivably imagine.

The thing that is important here is that testing hypotheses doesn’t go away. In fact, as Eric points out, in some ways this is no different from what we’ve done in some sciences for a long time; Eric mentions astronomers and microsocopists. Darwin’s work also falls into this category. In other words, science has long been based on observations, with those observations leading to hypotheses. The difference now is the sheer speed and volume at which data is collected, overwhelming our old pattern recognition tools, our eyes and mind. To help us make sense of those observations we need machines, but to do good science, we still need to develop models and theorems and then test them. As Eric writes in talking about data driven biology

For these methods to work, we must know enough about patterns (repetition, correlation, difference, functional correspondence…) that we can recognize some of them and separate the real patterns from the statistical illusions. This too is a hypothesis, but there is no pretense of vast insight.

We should not forget that

Related articles by Zemanta

• First Anniversary (and the scientific method revisited) (metamodern.com)
• Decisions With Incomplete Knowledge (dilbert.com)
• Science magazine and JoVE announce scientific-video partnership (biosingularity.wordpress.com)

I’m starting a manifesto. There are no technical, political or funding reasons why an open data platform for science couldn’t excel

He then followed that up with five tweets (Matt’s Twitter stream). I don’t know if that’s the entire manifesto, but I reproduce those tweets below, a series entitled Towards a science data platform

• Easy, flexible retrieval and reuse above all else
• A laser sharp focus on scientific productivity and progress
• Scalability and speed are not mutually exclusive
• Well designed, high quality programming interfaces are a prerequisite
• Be effortless to do the right thing: provenance capture, reproducibility, portability

This is definitely the developer view, and the one I can relate to. Just earlier today, I was thinking about the lack of innovation in scientific software, not necessarily on the algorithm side, but on the pure software and platform side, with a limited number of platforms out there. This happened during a search for the number of cool tools to launch and manage clusters of EC2 instances (and other computational resources), and speaks to Matt’s fourth point (well, designed interfaces). The part I agree with most, there are no reasons anymore not to have a scientific data platform, just excuses.

I hope Matt starts a website that will add to this train of thought. Would love to participate.

Software is really becoming the broader language of science. Even broader than mathematics, but we don’t really know how to fund it.

Those are words from Ed Seidel, Director of the NSF’s Office of Cyberinfrastructure. They are included in an article at Inside HPC, where Ed tries to clear up the air on some controversial positions on the status of supercomputing software.

Whether you agree with Ed Seidel on whether Fortran is antiquated or not, it’s pretty clear, and not just in HPC that science does not know how to fund sustainable software development, and if we do want to get a generation of programmers into scientific computing, we have to embrace more “modern” programming languages, or develop our own, because whether you like it or not, getting good Fortran programmers is going to get harder and harder. Some of his statements are very encouraging. He adds that reproducibility, bringing software engineering disciplines out of business applications and into scientific software are key issues. He goes on to say that we need to do more work on abstraction layers, documentation and teaching researchers to contribute to existing codes.

Will the NSF take the lead in making all of that happen? I sure hope so, but we need to do more as a community as well and teach those who are coming into scientific computing about these principles.

Related articles by Zemanta

• Chronicle article paints supercomputing codes as “rickety,” misses the mark (insidehpc.com)
• Biologists, programming and house cleaning (mndoci.com)

Scientists spend years collecting and generating increasing amounts data. The data ranges from raw instrument data, “finished” data (e.g. a genome sequence which is constructed after aligning all the short reads from a next-gen sequencer), and annotated data, which has been marked up to add additional information. We have repositories where a lot of this data goes, RCSB, NCBI, etc. In many cases there is clarity in these destinations and for the better part, resources like RCSB and NCBI are well funded and long lived (although I am always nervous about RCSB). However, many data repositories are dependent on funding, with no guarantees that the funding will be renewed. Given the size of some of these data resources, shouldn’t we be thinking of a more sustainable model for funding? This is a general problem for infrastructure resources, given the cost and the fact that you shouldn’t be looking at these from a 3-5 year perspective. This especially baffles me when libraries come into play. Shouldn’t the timescale there be in the 10’s of years?

I’d love to hear from those in the thick of funding infrastructure resources, especially data repositories and what their concerns are in this space?

Related articles by Zemanta

• Scientific data sharing (dbms2.com)