OpenWetWare:Strategy

• Jason R. Kelly 15:41, 15 March 2008 (CDT): Goal here is to do our best to loosely specify the research process and highlights points were information is generated or where collaboration occurs. Our hypothesis is that science would benefit from the information being shared and the collaboration happening widely and in the open.

1. Define each of the stages in the open research pipeline. How important are they to both the existing OWW community and new communities for enabling research or helping it flow more smoothly?
2. Determine the tools, both on OWW and in other places, that support each of the stages in the open research pipeline. Are these the right tools? How do they help the researcher move from one stage to the next, or between stages?
3. Where are the bottlenecks in the open research pipeline?
4. Where are the opportunities to improve the process?

Ideas

• Current widely accepted practice: Ideas are normally not widely shared at this point in the pipeline. Occasionally they will be shared among close colleagues, collaborators, or within labs. Sometimes contained in discussion section of peer-reviewed publications.
• Best examples of sharing/openness: In science: IGEM:Idea exchange. I've yet to find an example of no-strings sharing in fields other than secondary education. Some stuff of interest nonetheless: Iowa Electronic Markets and the Hollywood Stock Exchange are examples of online idea futures markets, tools that can harness the collective intelligence of a community. Modeled on the traditional stock markets, participants put up real (or virtual) money to buy stock in an idea, and the process of trading allows the idea to be assigned a value. The higher the value, the more the market users believe in the idea. kluster.com is an example of reward-driven idea exchange. An individual or an organization can run projects on kluster with the option of offering a cash reward for those who help solve their problems. If a participant helps in any way with an idea that finally gets selected that person will get a piece of the reward based upon the amount of work they did. One of the most recent web-based collaboration tools I've seen is Twiddla, which offers online whiteboarding.
• Pain level: Nonexistant to minimal

Strategic opportunities

See Refined Ideas.

Background Research

• Current widely accepted practice: This is an information aggregation process. Background research is most commonly disseminated in the form of closed-access, peer-reviewed literature reviews. In general, this is something that scientists are willing to share easily.
• Best examples of sharing/openness: Aggregator sites such as Postgenomic, Scintilla, OpenWetWare:Reviews, Cite-u-like, and Connotea. Do Scopus and PubMed (My NCBI) fall into this category? In non-scientific fields there are many examples of tools to aid this process: delicious, Technorati
• Pain level: Minimal (unless you don't have access to paper- or web-based background material)

Strategic opportunities

Not clear that there are any strategic opportunities for OWW in helping to conduct background research, though there may be some possibilities in helping researchers compile and maintain lists of what they've consulted. Tools and services for maintaining reference lists may fall under the umbrella of experimental work, however.

• Could easily imagine a collaboration with a journal to provide some scientific credit to curating a WikiReview on OWW (a la OpenWetWare:Reviews). Nature was considering doing a wiki review as a pilot at one point, but I think you really need to put a bunch of them in the same place for it to take off. Could imagine OWW providing tech support and an active, wiki-savvy community of scientists and a journal providing scientific currency by formalizing the selection of a curator(s). I bet we could get a journal to bite at this, and sri/others have had preliminary talks with journals that showed interest.

Refined Ideas

• Current widely accepted practice: These are actionable ideas based on experts parsing previous work, and seeing a gap in knowledge that can be addressed experimentally. These are almost never shared outside of labs.
• Best examples of sharing/openness: None?
• Pain level: Minimal

Strategic opportunities

Not clear whether there are a lot a short- or long-term strategic opportunities for OWW in the idea sharing realm. For both raw and refined idea sharing we might look at running a scientific futures market on OWW. Here's an interesting editorial about the idea. It's also not clear that researchers are comfortable with or perceive that there's value in sharing either raw or refined ideas with the world. It would be worth composing a list of questions (Which ideas do think are valuable to share and which are not? What would you hope to gain from sharing your ideas? etc.) and asking some sample users for input.

Experimental Plan

• Current widely accepted practice: The experimental plan for conducting research. It is normally shared within labs and among collaborators. Individual researchers will often seek advice from trusted scientists in the field as well (e.g. thesis committees).
• Best examples of sharing/openness: Thesis proposals? I posted mine ;)
• There might be incentive to share something like this if there was a market of free-lance scientists that were interested in taking on side projects / plan proposer would need to feel that they were protected from theft of their ides.
• Pain level: Low to medium (much of the experimental plan is made in conjunction with advisors and collaborators.)

Experimental Work

• Current widely accepted practice: Current practice is to keep your notebook on paper on your shelf. It is almost never shared or digitized. Protocols are typically available from copyrighted books, or closed-access journals.
• Best examples of sharing/openness: Jean-Claude Bradley & Open Notebook Science crowd, OpenWetWare Lab notebooks, IGEM lab notebooks. protocols online, OWW protocols
• Pain level: High

Strategic opportunities

• There are many opportunities here, because this is a real pain-point for scientists. Being open can save a lot of time, because the right piece of feedback could save weeks or months of effort by an experimentalist. Protocols and processes make a good "introduction to openness" because they're easy to share, they feel "safe," and it's easier to see the return on investment (you put up a protocol, get good feedback or lots of hits, you feel good, and are willing to try it again...).

Protocols

• Protocols seem like a perfect "lily-pad" as they are generally considered non-competitive and are often traded among scientists independent of the published literature. The success of protocol sharing on OWW is an early demonstration of this. The existing peer-reviewed protocol journals might be looking for a partner to keep "wiki-fied" versions of their content?, they would have the "golden-copy" and community sites like OWW could annotate/edit/etc on a shorter time scale than they can peer-review and copy-edit. So we'd fill the gap between editions of a book like molecular cloning, for instance.
  
  Our goal might be to build a complete protocols pipeline that would have the technical infrastructure to connect networks of researchers and communities.
  • better ranking of protocols, protocols w/ DOIs -- these may be more roadmap things then strategic level.
  • "Approved-by" protocols - as a way to give credit to protocol contributors and to also have protocols up a notch in the publishing pipeline. This is done now sort-of via the Name: system, where a lab is informally sanctioning a protocol. Would a formal process encourage more contribution or stifle it due to too many hoops (e.g. a PI-only approval process might be too big a road block since PI's don't have time / don't care)? The sanctioning process might be made more palatable by adding a marketing angle. All new PIs need to build the reputation of their labs. By building a process whereby tested protocols can be uploaded to a separate "lab-approved" tank that could offer DOI tagging and search, we may be able to market it as both a time-saving and attention-getting resource. More discussion needed.
  • Protocol publishing: Can we assemble sets or collections of vetted protocols based on what scientists need? How about "The First-Year Grad Student's Guide to E. coli Protocols? Or things like that.
  • Open version of Current Protocols? Where does CP fall short and how can OWW protocols fill that gap?

Supplementary materials

• Do all journals offer supplementary materials online? If not, or if these materials are not available without a subscription, OWW could offer a supplementary materials service to fill this gap. Questions: Does a general supplements site already exist, or are all web-based supplement services keyed to particular journals? How do supplementary materials intersect with protocols? What tools/data structures can OWW offer to add value (or entice scientists to use the service)?

Lab Notebooks

• This is a very hard sell for the typical scientist. However there is a lot of pain associated with a paper lab notebook, it's hard to search, hard to backup, hard to add data into, etc. So a solution for electronifying notebooks might be a good sell. Strategically, scientists spend a lot of their time in their notebook, so if you can get notebooks it might be a great bridge to other things.
• Video notebooks could be another opportunity, but this is really fringe among typical scientists.
• To be successful, an OWW lab notebook service would need to have an easy-to-use interface, and offer flexible search and reference options; for example, cross referencing between entries by date and by project. Though it's not in keeping with the open nature of OWW, if we do decide to offer a lab notebook service, we may also need to offer a privacy option for it to be successful. (if we did this, we would need to be clear about what successful means in this case: lots of users with private notebooks that never get opened up would not be success, time-delayed opening of private lab notebooks might be considered success).

Question/Answers

• The significant discussion that takes place on the forums at protocols-online points to a real need for Q/A among scientists that are trying to figure out what protocols to use.

Experimental Debugging

• Current widely accepted practice: Current practice is to do this in your head / in your lab notebook. Not shared.
• Best examples of sharing/openness: Nature Preceedings, Journal of negative results in biomedicine

Experimental Results (raw data / pre-analysis)

• Current widely accepted practice: Varies by field. Fields like genomics release data immediately. Most fields don't release data until it's been analyzed.
• Best examples of sharing/openness: what's the agreement that led to opening the genomics data?

Analyzed Results

• Current widely accepted practice: Analyzed results are often published in peer-reviewed journals, presented at conferences, shown on posters.
• Best examples of sharing/openness: Open access journals pretty much have this one licked, though peer-review and copy-editing still imposes a significant time delay.

Conclusions

• Current widely accepted practice: Scientific conclusions based on analyzed results. These are often published in peer-reviewed journals, presented at conferences, shown on posters.
• Best examples of sharing/openness: Open access journals share this openly with a delay largely due to peer-review and copy-editing.

Strategic Opportunities that don't fit pipe exactly

• Jason R. Kelly 14:02, 21 March 2008 (CDT):This is just a catch-all for good ideas that might not obviously fit in above.

Education opportunities

• There is a lot of overlap between education and research, lots of classes are making good use of OWW at the moment.

Collaboration

• Lab-level
  • There is a major pain point at lab organization. e.g. information about Equipment, Protocols, Materials, etc, are not stored in a persistent, search-able way in most labs. When someone leaves the lab an information vacuum is left in their place, labs who don't "get" open science still get this, so it's a pretty nice carrot. It's also probably the main source of quality information on OWW.
  • New PIs is an interesting customer set. They need tools to coordinate the organization of their lab. they also are still in a position where they need to publicize themselves / make a name for themselves. Good tools for lab organization might be a nice carrot for getting them to open up. There pain points are: training their new staff and advertising of themselves.
  • Also, training of new lab techs/grad students is a major pain point, having a centralized resource where people could develop/re-use materials and not have to start from scratch might be a huge boon.
    • Sean suggested a "new PI startup package on OWW" - might be an opportunity there.

• Institute-level
  • Knowing what the scientist down the hall is working on would be great. this problem is currently addressed by out of date lab websites, OWW as lab front page with user profiles is a nice solution.
• Scientific-field level
  • Conferences could be supplemented with better electronic collaboration opportunities, don't know if there is a specific idea there.
  • Groups of distributed collaborators could be interesting. Whatever happened with the Sorger wiki thing? is that still on the private site?
  • Keeping tabs on what other people in your field are doing. There is a bigger incentive to not step on toes than there is to scoop someone, by being open people are more likely to route around your work (in most fields). Sean commented on how this happens informally, maybe a strategic opportunity there.