CAGEN: Critical Assessment of Genetically Engineered Networks

The Critical Assessment for Genetically Engineered Networks (CAGEN) competition is intended to drive new approaches to designing robust, synthetic biological circuits. The competition is targeted at teams of established researchers designing circuits that implement a given function and the assessment of their circuit's performance across a set of multiple operating environments.

The CAGEN competition is sponsored by the Keck Foundation, as part of the National Acadamies Keck Futures Initiative (NAKFI).

Overview of the Competition

The Critical Assessment for Genetically Engineered Networks (CAGEN, pronounced "cajun") is a prototype competition designed explore this mechanism as a means to improve the robustness and performance of human-designed biological circuits and devices operating in cells. The competition is intended to bring together leading research groups in biological circuit design to compete to demonstrate their abilities at designing circuits that perform in a prescribed manner in a variety of cellular contexts. Each year, a steering committee will propose a challenge problem that involves the design of an increasingly complex set of biological functions in a range of environments. Teams must submit their sequences, plasmid DNA implementing their circuit and data characterizing the performance of their system against a specified test suite. The 3-5 best performing designs will be selected as finalists and results will be reviewed and verified by the CAGEN steering committee, who will then select the overall winner based on a set of quantitative metrics.

As part of a recent NAKFI proposal funded by the Keck Foundation, we are implementing the first iteration of the competition, including selecting the challenge problem, implementing a set of reference test protocols, announcing and publicizing the competition, implementing the selection process and choosing a winner. If successful, we believe that the competition can be proposed for continued funding from other sources and that over the medium term (5--10 years) CAGEN could lead toward a more robust set of biological design methods that allow human-designed circuits and devices to perform at levels closer to their biological counterparts.

2012 CAGEN Challenge: Robust Gene Response

The 2012 CAGEN challenge is on robust gene response. The goal of this challenge is to design a circuit that can express a fluorescent protein at a controlled level upon the introduction of a chemical inducer, with minimal variation in expression between cells and in multiple contexts. At conditions yielding maximum expression, the circuit should quickly bring the volume-normalized fluorescence from 1X to 10X in response to the addition of an inducer of the designer's choice. The circuit must work at multiple temperatures, with minimal variation in the fluorescence over time, operating temperature and cell choice.

The 2012 CAGEN challenge was announced 23 May 2011. The deadline for submissions is 15 June 2012. More information on the specifications and submission format are given on the challenge page. Participants are welcome to send comments to the steering committee via Richard Murray (murray-at-caltech-dot-edu).

How to Participate

To participate in a posted CAGEN challenge:

1. (optional) Send e-mail to the chair of the CAGEN steering committee, indicating that you plan on participating in one or more of the challenge problems
2. Subscribe to the cagen-announce mailing list to receive information about the CAGEN challenge
3. Submit a technical paper describing the results of your design by the deadline for the competition (June 2012)

2012 CAGEN Timeline

• 23 May 2011: CAGEN challenge for 2012 announced
• 15-17 June 2011: CAGEN challenge announced at SB 5.0
• Summer 2011: baseline designs and data to be posted, along with scripts for computing score
• October 2011: final deadline for submissions to be announced
• 23 May 2012: slight updates to competition rules (remove some TBDs and add details)
• 15 June 2012: design submissions due

CAGEN Organization

CAGEN Challenge Guidelines

The following guidelines have been developed to guide the creation and selection of CAGEN challenges:

1. Challenges should specify tasks that, if achieved, would imply that significant improvements in the state of the art have been made. The descriptions should as much as possible be agnostic as to the approach: circuit design, organism used, etc. should not be specified. On the other hand, it should be clear from the stated metrics that the task is not doable without a robust design.
2. Challenge descriptions will be posted on the CAGEN website along with discussion threads for at least one month. The descriptions can be be revised during this time.
3. The author/moderator can request that the challenge be approved by the CAGEN board, after which time it is posted as an official challenge.
4. Changes to the challenge description can be made yearly, after the awards (if any) for that year are made.
5. Contestants submit publishable work to CAGEN along with a description of how the work addresses the challenge. There will be a yearly due-date.
6. The CAGEN board assigns reviewers for each challenge. The reviewers use the metrics defined in the challenge description to evaluate each proposal. The following outcomes are possible:
   1. No award: none of the submissions significantly improve the state of the art.
   2. A single award: one of the submissions meets the metrics in the challenge and substantially improves the state of the art.
   3. A tie: multiple submissions offer a similar level of improvement on the state of the art.

For the first year of the competition, the CAGEN challenges will be developed and selected by the CAGEN steering committee.

Timeline for CAGEN competition

• July 2010: first steering committee meeting; selection of draft competition specifications
• October 2010: internal distribution of draft proposals to steering committee
• January April 2011: candidate proposals posted for comment
• April May 2011: selection of 2011 CAGEN challenged announced
• May June 2012: submission deadline
• June July 2012: selection of finalists

CAGEN steering committee

Adam Arkin (UC Berkeley) Barry Canton (Ginkgo BioWorks) Peter Carr (MIT Media Lab) James Collins (Boston University) Drew Endy (Stanford)

Eric Klavins (U. Washington) Richard Murray (Caltech; chair) Georg Seelig (U. Washington) Pamela Silver (Harvard) David Sprinzak (Tel-Aviv University)

Future Challenge Problems

In addition to the 2012 Challenge Problem, additional challenge problems have been formulate and may be considered for future competitions:

• CAGEN: Robust Synthetic Development Challenge
• CAGEN: Universal Logic Gates Challenge
• CAGEN: Competitive Gene Expression Challenge

External Resources

More information

• NAKFI proposal for CAGEN

Other critical assessment experiments and competitions

• CASP - Critical Assessment of Techniques for Protein Structure Prediction
• CAPRI - Critical Assessment of Prediction of Interactions