[mechanical_fish]

I'm a former biophysics postdoc myself. Now I work for an open-source software company.

This post strikes me as charmingly naive. You have to love this guy. And yet any essay that discusses the incentive structure of science but doesn't use the word "grant" until the last sentence is beating around the bush. Follow the money, my friends.

The publications are a side issue. To the extent that your count of top-tier publications matters when trying to get an academic job, it's because it's correlated with your ability to bring in money. (Money comes from peer review too, and what your peers want to read about is also what they want to fund.) What the hiring committees really want is grants. Grant money pays for labs and salaries. It pays for grad students and postdocs. And grant money literally buys prestige: Big projects come from big grants, and big grants require strong track records and a bunch of preliminary data, which in turn comes from smaller grants, or from the shared equipment that your neighbor bought with her grants.

The fact that there aren't that many top-tier peer-reviewed journals is a side effect of the limited number of top scientists, and the number of scientists is limited by available resources, not by lack of knowledge or connections or education. I could literally pick up the phone and reach a dozen Ivy-educated postdocs who would be full-time scientists if they could afford it.

Why can you find so much great software on Github? There are lots of reasons, but a fundamental one is: Moore's Law. Computer hardware has become so dirt cheap that you can be a programmer in your spare time. You can literally be a twelve-year-old kid with a $200 cast-off computer and yet do top-notch software work. If computers cost millions of dollars each, like they did in 1963, we wouldn't have Github. We'd have the drawer of a desk on the ninth floor of Tech Square. (After all, in the old days half the AI researchers in the world lived within a few miles of that drawer, and the others were just a phone call away.) That's how most advanced science works today: There's no need for more publishing infrastructure for scientific technique, because the available methods of getting the word out -- top journals, second-tier journals, email, the phone, bumping into people in the hallway at conferences -- scale well enough to meet the limited demand. Because just having the recipe for your very own scanning multiphoton microscope doesn't do you much good: You need a $150,000 laser, and a $200,000 microscope, and tens of thousands of dollars in lenses and filters and dyes, and a couple of trained optics experts to maintain the thing, and that's before you even have something to photograph.

I wish there were a magical way to turn everyone's suburban basement into a cancer research lab, the way Github has turned everyone's couch into a potential CS research lab, but there's no magic bullet. A few technologies, like DNA sequencing, are sufficiently generic, useful, and automatable to be amenable to Moore's-Law-based solutions, so we probably will soon be able to (e.g.) drop leaves into the hopper of a $1000 box and get a readout of the tree's genetics. But something like cancer research is never going to be cheap. To study cancer you must first have a creature that has cancer. Mice are as cheap as those get, and mice are not cheap, especially if you know what the word mycoplasma means.

[ramanujan]

I used to think this as well. But if you think about things which are ridiculously expensive -- like cars or airplanes or Google's datacenters -- all of them got their start in someone's garage.

After spending some time with the Open Wetware/DIYBio guys, I realized that no one (other than the DIYBio guys) has really spent time reducing the costs of the fundamental toolkit of the molecular biologist.

Now they are starting to do so:

http://openwetware.org/wiki/DIYbio:Notebook/Open_Gel_Box_2.0...

That gel box is going to be about $200-$300 all in. You might not get a multiphoton microscope right off the bat, anymore than you'd get access to all of Goog's servers when you were just starting out with your laptop...but reducing the price of entry for the hobbyist biologist/biochemist to say $5k or less is going to be really important.

That's because the role of the DIYBio community is set to wax in a big way. With the NIH budget cuts and the coming fiscal collapse of the US Government, the sun is about to set on a ~70 year period (1945-2015) of centralized US government research. The period before 1945 was more innovative by many measures (e.g. http://www.nytimes.com/2011/01/30/business/30view.html?_r=1), and potentially the period afterwards will be as well.

[mechanical_fish]

I agree that the DIYbio folks et al are incredibly exciting. But saying that those projects have to do with "building a Github of science" is putting the cart before the horse. To extend my metaphor for a moment, it's as if the engineers of 1963 had decided to popularize open source software by inventing and promoting Github, instead of inventing the personal computer and Ethernet first.

(As an aside, there are fascinating historical examples of people who did invent the web before inventing the personal computer, like this guy:

http://en.wikipedia.org/wiki/Paul_Otlet

Of course, he ended up as many of these slightly-too-early visionaries did: His work lost all its funding and he was kind of sad.)

Build the gel box and then worry about the social network. Or better yet, don't worry about the social network at all: These gel box users will network themselves, no problem. It would be a challenge to stop them from finding each other online.

[ippisl]

I don't know much about expensive lab equipment, but if you follow the DIY world, or the open source world , or even indian/chinese innovators, you see people building physical stuff at fractions of the price of commercial stuff.

A few example to illustrate my point:

1. Open farm tech - tractor and brick compressor and 1/3 to 1/10 of the commercial prices. they plan a whole set of manufacturing equipment at those reduced prices.

2. students took pictures of earth from space using £90 in equipment.

3. a diy electron microscope, a guy made himself [1], probably costs a lot less then commercial ones.

There many more extremely cheap stuff like this. Maybe the first place of open sourcing science is in support of open source tool development.

[1] http://blog.makezine.com/archive/2011/03/diy-scanning-electr...

[mechanical_fish]

I now see a terrible flaw in my essay of last night: I took the cheap and easy rhetorical route of emphasizing the costs of the fanciest equipment. Whereas the thing that kills you in science budgeting is actually the mundane stuff that you need in vast quantities.

Perhaps I should have talked about something boring, like absolutely clean, absolutely sterile containers. There is nothing sexy about containers and pipettes, and they are individually cheap. But they add up. The most reliable technique is to just buy quantities of disposable ones. You can try to save money by using recyclable containers instead, by washing dishes a lot, but washing dishes by hand is expensive even if you don't do it very patiently and carefully. And you need to be really careful, because if you move your cell culture from dish to dish to dish ten times, and even one of those dishes is contaminated, guess what? Your experiment might now be contaminated. And for every Alexander Fleming (a lucky guy whose contaminated experiment turned out to be a Nobel-Prize-winning medical breakthrough) there are a thousand experiments that have to be discarded because the data isn't reliable.

But even that argument is misleading, because the most expensive ingredient in science is not even a material good. It's time. Science is about patience and consistency. Doing an experiment once is not science. Doing it one thousand times and getting absolutely consistent results -- that is science. The work of being a scientist is about carefully building and debugging a reliable sequence of steps ("grow, filter, sort, lyse, plate, stain, image"): A sequence that can be repeated over and over to obtain thousands of data points that are extremely self-consistent.

The reason why professional scientists use such expensive equipment is that the equipment is actually cheap compared to the cost of spending two years taking data that turn out to be full of errors because your tools weren't reliable. Too much random error and you won't see your data amid the noise; too much systematic error and you might eventually have to throw out 100% of your work and start over. Trust me: If you want to experience soul-crushing misery [1], work sixty hours a week taking data for two years, then set the data on fire because it's unsalvageable. I have seen this happen many times. It has happened to me. It happens all the time in science, but you can't afford to have it happen too often.

So, yeah, you can take pictures of earth from space for $200, but can you take the same picture one thousand times, under consistent lighting and from consistent altitude and position? Yeah, you can build an electron microscope in your basement, but will it keep working every day for five years while you take your data? How much maintenance will it require? How much time will you waste waiting for it to pump down every time you change samples, or tweaking the knobs for hours every time to get a usable picture? Will it spread thin layers of carbon on your electronics, or go through a six-month phase where it can't focus well enough to see your samples?

---

[1] Fortunately, the misery is temporary. If it isn't, you're just not cut out to be an experimental scientist. Science requires reserves of forward-looking optimism! So what if yesterday was hell? It was a learning experience that will make tomorrow better! ;)

[ippisl]

Thanks for a very detailed and interesting description of the scientific process.

From the way you describe it, it seems that doing science in your area, is not the science that would attract small hacker groups , because it's long and boring.

hackers tend to contribute to the interesting parts of open source , i.e. languages like python and ruby, and organizations tend to do the big, boring heavy lifting, things like android , linux , etc.

By the way, what about technologies like combinatorial testing, automation , and miniturization ? what would be their affect on the way biology is done ?

[Goladus]

I wouldn't say it's boring. Not everyone has the patience science requires but for those that do, there's plenty of incentive to do disciplined data collection.

It's most likely just that the resources required to do it are still out of reach, unlike the python and ruby communities where your biggest expenses are a computer with a text editor and access to an internet connection.

Even some computing resources are out of reach for individuals, even if just barely. Amazon hosts public datasets, which is nice, but but the 500,000 instance-hours of HPC compute time needed to do analysis is not going to be in the budget of a hacker-in-the-basement (ie someone with no funding, whether it be from grants, revenue, or investment).

[neuroelectronic]

Also, as rapid prototyping picks up steam, this should also (eventually) help with sourcing equipment more cheaply. Lab on a chip (aka microfluidic tech) is a great example of Moore's law making it into labs.

[wgrover]

Your "desk drawer in Tech Square" analogy is apt -- right now our publications are locked in a handful of those drawers, and folks can't even open the drawers unless they've bought keys from Springer/Elsevier/ACS/AIP/ACM/IEEE/etc. And how do we find out what's in the drawers? Searching the databases owned by ISI/Thompson-Reuters and friends, again for a fee.

Sure, a "Github of science" wouldn't turn anyone's basement into a cancer research lab, but it would mean that a lot of researchers at less-affluent universities would finally have full access to the literature of science.

[mechanical_fish]

I loathe the for-profit journals as much as anyone, but I'm deeply suspicious of any hypothesis of the form "the reason why journals still cost money is that we haven't yet invented the right electronic social network for sharing scientific information".

We've had the technology to publish science online for decades. We have tinkered with it dozens of times. The web was originally invented for exactly this purpose. Far older things, like TeX, were invented for this purpose. Nowadays we have everything from PLoS to arXiv to Google Scholar to custom in-house blogs to PDFs sent through email.

The continued existence of for-profit journals is an economic, political, and anthropological problem, not a technological one. PLoS and the like are slowly changing things, but I still suspect that the only way to free our journals within less than a generation or two is to lobby (e.g.) the NIH to require that their funded projects be published in free journals. When a grant agency talks, people listen. When postdocs talk, alas, it makes a very subtle sound. ;)

[enjalot]

I agree with your statements, but perhaps the appeal of a github for science isn't the technology, it's the culture that comes with it. You can run your own git servers and submit patches over email, but when you can fork with the click of a button and do a pull request with another click, it encourages that much more sharing.

I've had this discussion with some of my professors, mostly just about open sourcing research code (I'm in Scientific Computing) and some of them wont do it because they want to squeeze a few publications out of one code and don't want anyone 'stealing' their publication. I find it disturbing, but it's ingrained in the culture. Changing it is important to me, but I don't see how yet.

[stevenbedrick]

The NIH is already actually doing that, at least sort of- any article reporting NIH-funded has to be submitted to PubMed Central, from where anybody can view and download the full text, figures, etc. I believe that there's an embargo exemption that allows journals to hold off on submitting to PMC for a few months or maybe up to a year, but after that it's public.

Right now, it's just NIH, but it's only a matter of time before AHRQ and the other biomedical funding agencies get in on the action, and there's no reason (in principle, anyway) why NSF or DOE couldn't also join in or do something similar on their own.

[chancho]

Where do Google Scholar and CiteSeer fit into this analogy?

Can't speak for other fields, but in CS any outlet worth submitting to will let you post a preprint on your own website, and from there Google Scholar will pick it up and place the link to your free copy right next to the paywall link. It's not a perfect system---it's like an extreme form price discrimination, like when Microsoft turns a blind eye to piracy in emerging markets---but it does give access to those who can't pay, and its not as dystopian as you make it out to be.

[wgrover]

I think that CS is a lot more open than other fields, even putting preprints online and soliciting feedback before publication (that's unheard of in chemistry and biology). You're right that some authors put PDFs of their papers on their websites and Google Scholar finds them, but I frequently need papers that I can only get through interlibrary loan (and I work at MIT; even a relatively wealthy university can't afford everything).

[marciovm123]

Thanks for the love =).

I agree about funding being an ulterior motive for a hiring committee to look at publication history. In my defense, I brought it about 2/3 of the way through with:

"We need trusted ways to quantify just how useful that API and associated code are to the scientific community, which can be listed on a scientist's profile and utilized by committees making hiring and funding decisions."

Science is constrained by resources, like everything else, but that doesn't mean scientists shouldn't try to optimize what can be done with the resources available.

Also, you imply that good science requires your own experimental data - and bearing associated costs - but that's only true when experimentalists aren't incentivized to share their data.

[neuroelectronic]

Shoulders of giants and all that.

[ChristianMarks]

If I have seen further than others, it is only by standing on the shoulders of grants.

[dougabug]

Do you need exclusive access to your very own scanning multiphoton microscope? Seems like laboratory virtualization would go hand in hand with open source science.

[mechanical_fish]

You can share them. Many labs do. Indeed, most of these things are probably shared in one way or another.

And you're correct: The trick to encouraging open source science is not to focus on the social networking tech -- that will be ready when you need it -- but to first attack the problem of doing quality lab work on the cheap. That's where the bottleneck is.

The biggest problem with shared facilities is the tragedy of the commons. In engineering -- or machining or woodworking or cooking, for that matter -- you quickly learn the importance of having your own tools. It only takes seconds to ruin a good tool. It only takes seconds to contaminate your cell culture, or your neighbor's cell culture, or an entire room full of your department's laboratory mice.

And mailing your samples off to a distant "virtual" lab is fine if you're studying disposable samples, or inorganic samples, or samples that have been permanently fixed and preserved on a glass slide. But living cells ship poorly even when you're allowed to ship them at all, and animals ship even more poorly than that. So often you've got to live next door to the equipment you're trying to share, and that's still expensive.

[dougabug]

The cost of maintaining your own tools is quite high, as is the expertise required to maintain them in working order. It seems to be that the inexperience or lack of skill which leads to shared equipment getting ruined is due to the fact that extremely sensitive equipment is being handled by students and junior scientists with little engineering background. The same argument could be made for the value of having your own servers. The scientific equivalent of a datacenter, would necessarily entail staffing of highly qualified, experience personnel. Such expertise would undoubtedly be costly, but it would be amortized over a large amount of equipment.

[ylem]

Staffed by who? There are actually some user facilities for materials synthesis--the problem is that if someone is making someone else's material instead of working on something that they're interested in, then I don't think that you're going to get the same level of productivity out. Also, materials synthesis takes time and running through lots of dead ends. When I was a grad. student, a postdoc from a collaboration would drive in to our lab with powders she had made--try to grow a crystal for a week (sleeping for maybe an hour or two a night on a floor in our office) and then go back to her home institution and come back in a few weeks. That just doesn't work--you don't have the responsiveness to be able to figure out what dead ends you're wandering down...Imagine if you were writing code and could only run it once every few weeks. Now, imagine that it was thousands of lines of nontrivial code and you're trying to debug it that way...

[dougabug]

Staffed by scientists and engineers, I imagine. You can get some pretty sophisticated parts built by foundries. At one point, making a microchip was prohibitively expensive. Nowadays, when you create chips basically by coding them in high level synthesis languages. Spinning a chip does take weeks. Obviously when a process takes longer to carry out, with high iteration cost, careful methodology is called for. Simulation and error checking software becomes valuable. Heck, once upon a time computers themselves were massively expensive, among the most expensive machines built. Computing wasn't born cheap. Mass production and decades of technological advances made them so. Machines for combinatorial science may someday be more effective than graduate students.

[archgoon]

My lab has a SEM. It costs $50/hour to operate. I have been led to understand that most of that cost is to cover if something goes wrong (which means we can't lower the per hour cost by having more people use it).

Think how much more expensive coding would be if it cost $50/ per compile (or hell $10/per compile).

[dougabug]

$50/hr seems quite reasonable. I doubt I'd need more than 100 hours a year of use, and even if I required several hundred hours, that's still readily affordable.

[mbreese]

You may not need exclusive access to one, which is why a lot of institutions setup "centers" for this type of work (center for visualization, for example). Then multiple people can share the equipment and cost - if you can recruit people with a similar research focus. So, you may not need one exclusively, but you do need one locally available.

[dougabug]

Seems like many samples could be prepared locally and shipped to the analysis center ala Netflix. Another possibility would be making samples to order at a location near where the necessary equipment is located. For instance, I need a brain tissue sample from xxx type of mouse, infected with pathogen yyyy.

[ignifero]

That is the kind of thinking that is holding life sciences back: lack of openness, obsession with centuries old journals etc. Starting in comp. Neuroscience, and coming from a physics background I was shocked by how little info neuroscientists share with each other. I cant even access most of the papers I need from home. There is no repository of data or even computational models (well there is modeldb) . There have been several attempts to make the life sciences more open on the web, but it's not working because there is no culture of openness. For cryin out loud, most of the journals don't even have an online comments section. Big, ambitious projects are missing (think LHC-scale), and the pioneers in these are private institutions like the Allen brain institute. Maybe it's not possible to make a github for science, but it's certainly possible for a high profile university to require its staff to publish in open access journals only. That would be a start