Parameter Sensitivity and Estimation

General Discussion on Parameter Estimation and Sensitivity

Parameter Estimation from Time-Variant Output

Commonly used parameter estimation algorithms fall into two main categories: local minimization algorithms and genetic algorithms. Local minimization algorithms basically work by minimizing the objective function (aka cost function of the model fit to the data) in parameter space around a user supplied initial parameter guess. Starting at different initial parameter guesses can lead to different minima, so in order to gain confidence that a global minimum has been reached, the algorithm must be run repeatedly starting from different initial guesses that span the allowed parameter space. Such algorithms include the Levenberg-Marquardt gradient descent and downhill simplex algorithms. Genetic algorithms are stochasitc rather than deterministic. Multiple sets of parameter values are maintained at all times during the optimization. These sets undergo modification by so-called 'genetic operators' such as random mutation and crossover. The resultant parameter sets are then evaluated based on the corresponding value of the objective function, and the top sets are carried through to the next iteration of mutation and crossover. Genetic algorithms are not guaranteed to cover the entire parameter space. Hybrid algorithms also exist whereby sets of parameters are iteratively modified by the genetic operators and suject to local minimization. These algorithms have the advantage of genetic algorithms in that they automatically move throughout parameter space. The local minimization aspect of the hybrid algorithms acts to speed up the convergence to optimal solutions. In practice there is no difference between estimation of parameters from systems with static and time-variant inputs.

Parameter Estimation in Practice

There are several pre-packages tools which allow people to perform parameter estimation. For example, Matlab has implementations of several local minimization algorithms in the optimization toolbox [1][2], and a genetic algorithms toolbox which allows for various implementations of genetic algorithms [3][4]. There are also numerous other software packages that are designed specifically for parameter estimation, including Jacobian.

Parameter Sensitivity to Guide Experimentation

The behavior of a model will depend critically on some parameters, and will have a weak dependence on other parameters. This dependence can be quantified by the normalized parameter sensitivity. This is defined as the differential change in the quantitative behavior of the model (measured by some metric X) for a given fractional change in each parameter of interest. That it, the normalized parameter sensitivity is delta(X)/(delta(k_i)/k_i) = dX / d(log(k_i)). In general, parameter estimation will yeild more information about more sensitive parameters. If we are able to design experiments that increases the dependence of the behavior of the model on a given parameter (ie design experiments that increase the parameter sensitivity), then we can potentially gain more information about that parameter from the experimental results. Higher parameter sensitivity does not necessarily lead to better parameter estimates as parameter sensitivity does not take into account the correlation between parameters which can confound correct parameter estimation.

Parameter Constraint From Estimates

Sensitivity analysis can be used to place error bounds on the parameter estimates. Essentially, each parameter value can be independently varied in order to determine the parameter range in which the model's fit to the data remains within specified bounds. This can be done for parameters with high sensitivity (which we would expect to have narrow error bounds), and for parameters with very low sensitivity (which we would expect to have very large bounds).

Reaction Timescales Can Limit Estimation Ability

The parameters governing rate limiting steps in a model will have high parameter sensitivity. It is thus these parameters that we have the best ability to measure.

Specific Discussion on Rate Constant Estimation Using Time-Varying Input/Output

Rates as Specific Instances of Parameters

Parameters are all of the values in the model that are fixed throughout a given simulation. In models based on mass action kinetics that include synthesis and degradation reactions, the only parameters are rate constants. In my models, protein abundances (as determined by Kirsten) are used to constrain the relationship between the synthesis and degradation rates of proteins, and thus protein abundances can be thought of as parameters as well. But the manner in which abundances appear in the rate equations is really no different from how rate constants appear, so abundances are equivalent to being rate constants in this case.

Parameter Estimation of Rate Constants in Mass Action Kinetics

If the model representation is not constrained to mass action kinetics, then we can't make any assumptions about the form of the equations, and thus rate constants are no different from bounded parameters (since we can place both upper and lower bounds on rate constants) in any arbitrary model. If we restrict ourselves to mass action kinetics, then we do gain some things. For instance, we know that our rate equations are all continuous and twice differentiable. These properties should make it easier to estimate the parameters. In fact, some estimation algorithms require these properties. Other than this, I don't know of any algorithms that take into account the very specific structure of a mass action kinetics model, or whether there would be any significant estimation ability gained from restricting the model structure. The more high-powered tools, such as Jacobian essentially create a symbolic (analytic) representation of the model, so they presumably gain whatever there is to be gained from precisely knowing the model structure rather than treating it as a black box with the parameters as input.

Parameter Estiamtion Using Time Varying Input/Output

As far as I can tell, and I'm by no means an expert on parameter estimation, parameter estimation software essentially knows nothing about what the input of a model is. Thus, a time varying input is essentially a property of the model itself during the estimation routine. The parameter estimation algorithm is completely unaffected by the nature of the input that is driving the model's behavior. However, what time-varying input can give us is different model behavior than a simple step input. Another, more accurate way to think of this is that a simple step input is just one of an infinite set of possible time-varying inputs. Different inputs will results in different behavior which may or may not improve our ability to estimate different parameters. So time-varying input is no different from other model perturbations (like modeling mutants/deletions). Of course, mutliple output timepoints (which defines time-varying output) helps improve parameter estimation as one would expect. Parameter sensitivity, as outlined above, can help us chose inputs that can lead to better estimates of different parameters.

Using Parameter Sensitivity to Design Better Experiments

The idea of doing modeling work with the current state of knowledge in order to design informative experiments is obviously not my idea. For parameter estimation, optimal experimental design seems to have been introduced to biology in 1999 (varying only the input signal, not doing other perturbations - see PMID 10563462). This work, and others since then, have concentrated on maximizing the Fisher information through minimization of either the D- or E-criteria. The D-criterion is a measure of the global parameter estimation uncertainty (ie how poorly/well we can estimate all parameters). The E-criterion is a measure of the largest parameter estimation uncertainty (ie how poorly/well we can estimate the most difficult parameter to estimate). I have yet to see any evidence that anyone else has thought to design a set of experiments in which each was meant to increase the estimateability of different parameters, or decrease/change the correlation between parameters. That is not to say that no-one has done this before, but I have yet to come across it.

Roger's specifications

What I am hoping for from you, perhaps with Xiuxia Du, is a crisp 3-5 paragraph discussion of the means you can see us using to extract reaction rate constants from time variant output in response to time variant input. Then, back off that goal. If we cannot extract rates, can we constrain rates? If the output we measure is the result of a number of different upstream reactions, can anything in the time variant output give any information about any of the reactions that contributed to it, for example, the slowest one? Or can such analysis tell us anything about which reaction rates we guessed wrong, or suggest to us which reaction rates might be most sensitive to bad guesses? And in each case, what steps would we follow to divine this information from the output.

If they rates are not special not, why are they not? In any case, given that we have numbers of molecules and we have rates, does the fact that we can sometimes measure numbers but not so easily measure rates give us anything? Why or why not?

Roger's Comments about the first draft that need to be addressed

First, it is about parameters in general, which is not what I thought I was asking. Maybe rates are special, maybe they are not.

Second, it is not about time variant input and time variant output, which is what is explicitly on the table.

Third, and consequentially, it does not place for me your trial of different input pulses to see which produced quite different outputs if initial dissociation rate was off into any larger intellectual context. What are the roots of what you did? Why did you do that? What prior examples exist?