OpenWetWare:ProjectDevelopment/OWW peer review process

Description

A process by which authors on OWW can invite community peer review of their content.

Challenge or opportunity the development project answers

Giving the community a more quantitative means of knowing which content is well-tested, what could be changed or improved, and why.

• Reshma 18:23, 30 June 2008 (UTC): I would argue that this proposal actually enables more qualitative peer review that quantitative. Instead of merely getting a rank or rating, you get to see the actual individuals who have approved or vetted a particular protocol. I think such a system would allow OWW to more explicitly plug into the soft credit system in research.

Examples of what could be done or how it would work

This system could be used for any piece of content on OWW, but I'll use a protocol as an example.

A button called "prime time" appears at the bottom of every protocol. When the author of the protocol feels that it is ready for wider distribution or review, he or she clicks the button. The protocol is time- and date-stamped. It can be uploaded to a repository with a link back to the working version.

• Reshma 18:19, 30 June 2008 (UTC): Does a protocol need to be marked as ready for prime-time. Couldn't we just enable the review system on every page by default? Would there be some backlash against this?

When committed to Primetime, the protocol acquires a dashboard of buttons in its sidebar. One of them might be "I use this". Another might be "add review comments" etc. Yet another might be "Users". If you like the protocol and use it, you click on the "I use this" button, and your name is added to the list of users. If you want to make comments, you click the comments button and a text box appears. The button could also take you to protocol's talk page.

Now, let's say you do a search for "Beta glactosidase assay." Up pops the list of available protocols, ranked by the number of users it's collected. If you roll your cursor over the user button in protocol's dashboard it shows a list of OWW member names who have "certified" it. Roll over the "review comments" button and read the comments they've left.

Why the project should happen now

Community peer review could be an important step toward formal peer review by external publications, and it can help potential users find the best protocols more quickly.

• Reshma 18:19, 30 June 2008 (UTC): Peer review is a core part of the research process from grant funding to publication, yet we've been struggling with mechanisms for introducing peer review into OWW. It's not clear to me that there is enough traffic on individual pages on OWW to warrant a digg or slashdot type ranking system. (Does any page beyond Sean's beta-galactosidase assay and the E. coli genotypes page get 100s-1000s of views that make a digg style system worthwhile?) But it would be pretty interesting if contributors and readers alike could see who is using their protocol. It gives contributors additional incentive to make their protocol high-quality ... some one might see it and mark it. It provides additional valuable information to readers ... hey this protocol worked for a lot of people. And it brings us more in line with the norms of the research community. This mechanism could also easily feed into a system for promoting protocols and publishing them beyond OWW.

Who are the immediate customers?

All OWW members.

• Reshma 18:19, 30 June 2008 (UTC): OWW contributors can get more tangible feedback on their contributions ... beyond just page views. OWW viewers can get a better idea of which pages are widely used by folks ... beyond google rank.