CAGEN: Critical Assessment of Genetically Engineered Networks

The Critical Assessment for Genetically Engineered Networks (CAGEN) competition is intended to drive new approaches to designing robust, synthetic biological circuits. The competition is targeted at teams of established researchers designing circuits that implement a given function and the assessment of their circuit's performance across a set of multiple operating environments.

The CAGEN competition is sponsored by the Keck Foundation, as part of the National Acadamies Keck Futures Initiative (NAKFI).

Overview of the Competition

The Critical Assessment for Genetically Engineered Networks (CAGEN, pronounced "cajun") is a new competition designed to improve the robustness and performance of human-designed biological circuits and devices operating in cells. The competition is intended to bring together leading research groups in biological circuit design to compete to demonstrate their abilities at designing circuits that perform in a prescribed manner in a variety of cellular contexts. Each year, a steering committee will propose a challenge problem that involves the design of an increasingly complex set of biological functions in a range of environments. Teams must submit their sequences, plasmid DNA implementing their circuit and data characterizing the performance of their system against a specified test suite. The 3-5 best performing designs will be selected as finalists and results will be reviewed and verified by the CAGEN steering committee, who will then select the overall winner based on a set of quantitative metrics.

As part of a recent NAKFI proposal funded by the Keck Foundation, we are implementing the first iteration of the competition, including selecting the challenge problem, implementing a set of reference test protocols, announcing and publicizing the competition, implementing the selection process and choosing a winner. If successful, we believe that the competition can be proposed for continued funding from other sources and that over the medium term (5--10 years) CAGEN could lead toward a more robust set of biological design methods that allow human-designed circuits and devices to perform at levels closer to their biological counterparts.

CAGEN Challenge Guidelines

The following guidelines have been developed to guide the creation and selection of CAGEN challenges:

1. Challenges should specify tasks that, if achieved, would imply that significant improvements in the state of the art have been made. The descriptions should as much as possible be agnostic as to the approach: circuit design, organism used, etc. should not be specified. On the other hand, it should be clear from the stated metrics that the task is not doable without a robust design.
2. Challenge descriptions will be posted on the CAGEN website along with discussion threads for at least one month. The descriptions can be be revised during this time.
3. The author/moderator can request that the challenge be approved by the CAGEN board, after which time it is posted as an official challenge.
4. Changes to the challenge description can be made yearly, after the awards (if any) for that year are made.
5. Contestants submit publishable work to CAGEN along with a description of how the work addresses the challenge. There will be a yearly due-date.
6. The CAGEN board assigns reviewers for each challenge. The reviewers use the metrics defined in the challenge description to evaluate each proposal. The following outcomes are possible:
   1. No award: none of the submissions significantly improve the state of the art.
   2. A single award: one of the submissions meets the metrics in the challenge and substantially improves the state of the art.
   3. A tie: multiple submissions offer a similar level of improvement on the state of the art.

For the first year of the competition, the CAGEN challenges will be developed and selected by the CAGEN steering committee.

Timeline for CAGEN competition

• July 2010: first steering committee meeting; selection of draft competition specifications
• October 2010: internal distribution of draft proposals to steering committee
• January April 2011: candidate proposals posted for comment
• April May 2011: selection of 2011 CAGEN challenged announced
• May 2012: submission deadline
• June 2012: selection of finalists

CAGEN steering committee

Adam Arkin (UC Berkeley) Barry Canton (Ginkgo BioWorks) Peter Carr (MIT Media Lab) James Collins (Boston University) Drew Endy (Stanford)

Eric Klavins (U. Washington) Richard Murray (Caltech; chair) Georg Seelig (U. Washington) Pamela Silver (Harvard) David Sprinzak (Tel-Aviv University)

2012 CAGEN Challenge

The 2012 CAGEN challenge will run from May 2011 to May 2012. Comments are currently being solicited for the draft challenge problems.

Challenge Problems

The following challenge problems are currently under consideration by the CAGEN steering committee:

• CAGEN: Robust Gene Expression Challenge
• CAGEN: Robust Synthetic Development Challenge
• CAGEN: Universal Logic Gates Challenge
• CAGEN: Competitive Gene Expression Challenge

How to Participate

To provide comments on a draft CAGEN challenge:

1. Read through the list of CAGEN challenges and complete the online survey indicating your interest in participating
2. Send suggestions and comments on improving the challenge problem to the contact listed on the challenge problem
3. Subscribe to the cagen-announce mailing list to receive information about changes to the challenge problem and announcement of the final selection

To participate in a posted CAGEN challenge:

1. (optional) Send e-mail to the chair of the CAGEN steering committee, indicating that you plan on participating in one or more of the challenge problems
2. Subscribe to the cagen-announce mailing list to receive information about the CAGEN challenge
3. Submit a technical paper describing the results of your design by the deadline for the competition (roughly May 2012)

To suggest a new CAGEN challenge

• New CAGEN challenges are not currently being accepted

External Resources

More information

• NAKFI proposal for CAGEN

Other critical assessment experiments and competitions

• CASP - Critical Assessment of Techniques for Protein Structure Prediction
• CAPRI - Critical Assessment of Prediction of Interactions