User:Timothee Flutre/Notebook/Postdoc/2011/11/10
From OpenWetWare
(→Bayesian model of univariate linear regression for QTL detection: fix typo sign lambda*) 
(→Bayesian model of univariate linear regression for QTL detection: start section on binary phenotype) 

(5 intermediate revisions not shown.)  
Line 9:  Line 9:  
  ''  +  ''This page aims at helping people like me, interested in quantitative genetics, to get a better understanding of some Bayesian models, most importantly the impact of the modeling assumptions as well as the underlying maths. It starts with a simple model, and gradually increases the scope to relax assumptions. See references to scientific articles at the end.'' 
Line 23:  Line 23:  
* '''Likelihood''': we start by writing the usual linear regression for one individual  * '''Likelihood''': we start by writing the usual linear regression for one individual  
  <math>\forall i \in \{1,\ldots,N\}, \; y_i = \mu + \beta_1 g_i + \beta_2 \mathbf{1}_{g_i=1} + \epsilon_i \text{ with } \epsilon_i \overset{i.i.d}{\sim} \mathcal{N}(0,\tau^{1})</math>  +  <math>\forall i \in \{1,\ldots,N\}, \; y_i = \mu + \beta_1 g_i + \beta_2 \mathbf{1}_{g_i=1} + \epsilon_i \; \text{ with } \; \epsilon_i \; \overset{i.i.d}{\sim} \; \mathcal{N}(0,\tau^{1})</math> 
where <math>\beta_1</math> is in fact the additive effect of the SNP, noted <math>a</math> from now on, and <math>\beta_2</math> is the dominance effect of the SNP, <math>d = a k</math>.  where <math>\beta_1</math> is in fact the additive effect of the SNP, noted <math>a</math> from now on, and <math>\beta_2</math> is the dominance effect of the SNP, <math>d = a k</math>.  
Line 35:  Line 35:  
<math>Y  X, \tau, B \sim \mathcal{N}(XB, \tau^{1} I_N)</math>  <math>Y  X, \tau, B \sim \mathcal{N}(XB, \tau^{1} I_N)</math>  
  Even though we can write the likelihood as a multivariate Normal, I still keep the term "univariate" in the title because the  +  Even though we can write the likelihood as a multivariate Normal, I still keep the term "univariate" in the title because the regression has a single response, <math>Y</math>. 
+  It is usual to keep the term "multivariate" for the case where there is a matrix of responses (i.e. multiple phenotypes).  
The likelihood of the parameters given the data is therefore:  The likelihood of the parameters given the data is therefore:  
Line 164:  Line 165:  
  * '''Bayes Factor''': to do  +  * '''Bayes Factor''': one way to answer our goal above ("is there an effect of the genotype on the phenotype?") is to do [http://en.wikipedia.org/wiki/Hypothesis_testing hypothesis testing]. 
+  We want to test the following [http://en.wikipedia.org/wiki/Null_hypothesis null hypothesis]:  
+  <math>H_0: \; a = d = 0</math>  
  +  In Bayesian modeling, hypothesis testing is performed with a [http://en.wikipedia.org/wiki/Bayes_factor Bayes factor], which in our case can be written as:  
  +  <math>\mathrm{BF} = \frac{\mathsf{P}(Y  X, a \neq 0, d \neq 0)}{\mathsf{P}(Y  X, a = 0, d = 0)}</math>  
  +  We can shorten this into:  
+  <math>\mathrm{BF} = \frac{\mathsf{P}(Y  X)}{\mathsf{P}_0(Y)}</math>  
  * '''R  +  Note that, compare to frequentist hypothesis testing which focuses on the null, the Bayes factor requires to explicitly model the data under the alternative. 
+  This makes a big difference when interpreting the results (see below).  
+  
+  Let's start with the numerator:  
+  
+  <math>\mathsf{P}(Y  X) = \int \mathsf{P}(\tau) \mathsf{P}(Y  X, \tau) \mathsf{d}\tau</math>  
+  
+  First, let's calculate what is inside the integral:  
+  
+  <math>\mathsf{P}(Y  X, \tau) = \frac{\mathsf{P}(B  \tau) \mathsf{P}(Y  X, \tau, B)}{\mathsf{P}(B  Y, X, \tau)}</math>  
+  
+  Using the formula obtained previously and doing some algebra gives:  
+  
+  <math>\mathsf{P}(Y  X, \tau) = \left( \frac{\tau}{2 \pi} \right)^{\frac{N}{2}} \left( \frac{\Omega}{\Sigma_B} \right)^{\frac{1}{2}} exp\left( \frac{\tau}{2} (Y^TY  Y^TX\Omega X^TY) \right)</math>  
+  
+  Now we can integrate out <math>\tau</math> (note the small typo in equation 9 of supplementary text S1 of Servin & Stephens):  
+  
+  <math>\mathsf{P}(Y  X) = (2\pi)^{\frac{N}{2}} \left( \frac{\Omega}{\Sigma_B} \right)^{\frac{1}{2}} \frac{\frac{\lambda}{2}^{\frac{\kappa}{2}}}{\Gamma(\frac{\kappa}{2})} \int \tau^{\frac{N+\kappa}{2}1} exp \left( \frac{\tau}{2} (Y^TY  Y^TX\Omega X^TY + \lambda) \right)</math>  
+  
+  Inside the integral, we recognize the almostcomplete pdf of a Gamma distribution.  
+  As it has to integrate to one, we get:  
+  
+  <math>\mathsf{P}(Y  X) = (2\pi)^{\frac{N}{2}} \left( \frac{\Omega}{\Sigma_B} \right)^{\frac{1}{2}} \left( \frac{\lambda}{2} \right)^{\frac{\kappa}{2}} \frac{\Gamma(\frac{N+\kappa}{2})}{\Gamma(\frac{\kappa}{2})} \left( \frac{Y^TY  Y^TX\Omega X^TY + \lambda}{2} \right)^{\frac{N+\kappa}{2}}</math>  
+  
+  We can use this expression also under the null.  
+  In this case, as we need neither <math>a</math> nor <math>d</math>, <math>B</math> is simply <math>\mu</math>, <math>\Sigma_B</math> is <math>\sigma_{\mu}^2</math> and <math>X</math> is a vector of 1's.  
+  We can also defines <math>\Omega_0 = ((\sigma_{\mu}^2)^{1} + N)^{1}</math>.  
+  In the end, this gives:  
+  
+  <math>\mathsf{P}_0(Y) = (2\pi)^{\frac{N}{2}} \frac{\Omega_0^{\frac{1}{2}}}{\sigma_{\mu}} \left( \frac{\lambda}{2} \right)^{\frac{\kappa}{2}} \frac{\Gamma(\frac{N+\kappa}{2})}{\Gamma(\frac{\kappa}{2})} \left( \frac{Y^TY  \Omega_0 N^2 \bar{Y}^2 + \lambda}{2} \right)^{\frac{N+\kappa}{2}}</math>  
+  
+  We can therefore write the Bayes factor:  
+  
+  <math>\mathrm{BF} = \left( \frac{\Omega}{\Omega_0} \right)^{\frac{1}{2}} \frac{1}{\sigma_a \sigma_d} \left( \frac{Y^TY  Y^TX\Omega X^TY + \lambda}{Y^TY  \Omega_0 N^2 \bar{Y}^2 + \lambda} \right)^{\frac{N+\kappa}{2}}</math>  
+  
+  When the Bayes factor is large, we say that there is enough evidence in the data to ''support the alternative''.  
+  Indeed, the Bayesian testing procedure corresponds to measuring support for the specific alternative hypothesis compared to the null hypothesis.  
+  Importantly, note that, for a frequentist testing procedure, we would say that there is enough evidence in the data to ''reject the null''.  
+  However we wouldn't say anything about the alternative as we don't model it.  
+  
+  The threshold to say that a Bayes factor is large depends on the field. It is possible to use the Bayes factor as a test statistic when doing permutation testing, and then control the false discovery rate. This can give an idea of a reasonable threshold.  
+  
+  
+  * '''Hyperparameters''': the model has 5 hyperparameters, <math>\{\kappa, \, \lambda, \, \sigma_{\mu}, \, \sigma_a, \, \sigma_d\}</math>. How should we choose them?  
+  Such a question is never easy to answer. But note that all hyperparameters are not that important, especially in typical quantitative genetics applications. For instance, we are mostly interested in those that determine the magnitude of the effects, <math>\sigma_a</math> and <math>\sigma_d</math>, so let's deal with the others first.  
+  
+  As explained in Servin & Stephens, the posteriors for <math>\tau</math> and <math>B</math> change appropriately with shifts (<math>y+c</math>) and scaling (<math>y \times c</math>) in the phenotype when taking their limits.  
+  This also gives us a new Bayes factor, the one used in practice (see Guan & Stephens, 2008):  
+  
+  <math>\mathrm{lim}_{\sigma_{\mu} \rightarrow \infty \; ; \; \lambda \rightarrow 0 \; ; \; \kappa \rightarrow 0 } \; \mathrm{BF} = \left( \frac{N}{\Sigma_B^{1} + X^TX} \right)^{\frac{1}{2}} \frac{1}{\sigma_a \sigma_d} \left( \frac{Y^TY  Y^TX (\Sigma_B^{1} + X^TX)^{1} X^TY}{Y^TY  N \bar{Y}^2} \right)^{\frac{N}{2}}</math>  
+  
+  Now, for the important hyperparameters, <math>\sigma_a</math> and <math>\sigma_d</math>, it is usual to specify a grid of values, i.e. <math>M</math> pairs <math>(\sigma_a, \sigma_d)</math>. For instance, Guan & Stephens used the following grid:  
+  
+  <math>M=4 \; ; \; \sigma_a \in \{0.05, 0.1, 0.2, 0.4\} \; ; \; \sigma_d = \frac{\sigma_a}{4}</math>  
+  
+  Then, we can average the Bayes factors obtained over the grid using, as a first approximation, equal weights:  
+  
+  <math>\mathrm{BF} = \sum_{m \, \in \, \text{grid}} \frac{1}{M} \, \mathrm{BF}(\sigma_a^{(m)}, \sigma_d^{(m)})</math>  
+  
+  
+  * '''Implementation''': the following R function is adapted from Servin & Stephens supplementary text 1.  
+  
+  <nowiki>  
+  BF < function(G=NULL, Y=NULL, sigma.a=NULL, sigma.d=NULL, get.log10=TRUE){  
+  stopifnot(! is.null(G), ! is.null(Y), ! is.null(sigma.a), ! is.null(sigma.d))  
+  subset < complete.cases(Y) & complete.cases(G)  
+  Y < Y[subset]  
+  G < G[subset]  
+  stopifnot(length(Y) == length(G))  
+  N < length(G)  
+  X < cbind(rep(1,N), G, G == 1)  
+  inv.Sigma.B < diag(c(0, 1/sigma.a^2, 1/sigma.d^2))  
+  inv.Omega < inv.Sigma.B + t(X) %*% X  
+  inv.Omega0 < N  
+  tY.Y < t(Y) %*% Y  
+  log10.BF < as.numeric(0.5 * log10(inv.Omega0)   
+  0.5 * log10(det(inv.Omega))   
+  log10(sigma.a)  log10(sigma.d)   
+  (N/2) * (log10(tY.Y  t(Y) %*% X %*% solve(inv.Omega)  
+  %*% t(X) %*% cbind(Y))   
+  log10(tY.Y  N*mean(Y)^2)))  
+  if(get.log10)  
+  return(log10.BF)  
+  else  
+  return(10^log10.BF)  
+  }  
+  </nowiki>  
+  
+  In the same vein as what is explained [http://openwetware.org/wiki/User:Timothee_Flutre/Notebook/Postdoc/2011/06/28 here], we can simulate data under different scenarios and check the BFs:  
+  
+  <nowiki>  
+  N < 300 # play with it  
+  PVE < 0.1 # play with it  
+  grid < c(0.05, 0.1, 0.2, 0.4, 0.8, 1.6, 3.2)  
+  MAF < 0.3  
+  G < rbinom(n=N, size=2, prob=MAF)  
+  tau < 1  
+  a < sqrt((2/5) * (PVE / (tau * MAF * (1MAF) * (1PVE))))  
+  d < a / 2  
+  mu < rnorm(n=1, mean=0, sd=10)  
+  Y < mu + a * G + d * (G == 1) + rnorm(n=N, mean=0, sd=tau)  
+  for(m in 1:length(grid))  
+  print(BF(G, Y, grid[m], grid[m]/4))  
+  </nowiki>  
+  
+  
+  * '''Binary phenotype''': using a similar notation, we model casecontrol studies with a [http://en.wikipedia.org/wiki/Logistic_regression logistic regression] where the probability to be a case is <math>\mathsf{P}(y_i = 1) = p_i</math>.  
+  
+  There are many equivalent ways to write the likelihood, the usual one being:  
+  
+  <math>y_i \; \overset{i.i.d}{\sim} \; Bernoulli(p_i) \; \text{ with } \; \mathrm{ln} \frac{p_i}{1  p_i} = \mu + a \, g_i + d \, \mathbf{1}_{g_i=1}</math>  
+  
+  Using <math>X_i</math> to denote the <math>i</math>th row of the design matrix <math>X</math> and keeping the same definition as above for <math>B</math>, we have:  
+  
+  <math>p_i = \frac{e^{X_i^TB}}{1 + e^{X_i^TB}}</math>  
+  
+  As the <math>y_i</math>'s can only take <math>0</math> and <math>1</math> as values, the likelihood can be written as:  
+  
+  <math>\mathcal{L}(B) = \prod_{i=1}^N p_i^{y_i} (1p_i)^{1y_i}</math>  
+  
+  We still use the same priors as above for <math>B</math> and the Bayes factor now is:  
+  
+  <math>\mathrm{BF} = \frac{\int \mathsf{P}(B) \mathsf{P}(Y  X, B) \mathrm{d}B}{\int \mathsf{P}(\mu) \mathsf{P}(Y  X, \mu) \mathrm{d}\mu}</math>  
+  
+  The interesting point here is that there is no way to calculate these integrals analytically. Therefore, we will use [http://en.wikipedia.org/wiki/Laplace_approximation Laplace's method] to approximate them, as in Guan & Stephens (2008).  
+  
+  
+  * '''Link between Bayes factor and Pvalue''': see Wakeley (2008)  
+  
+  to do  
+  
+  
+  * '''Hierarchical model''': pooling genes, learn weights for grid and genomic annotations, see Veyrieras ''et al'' (PLoS Genetics, 2010)  
+  
+  to do  
+  
+  
+  * '''Multiple SNPs with LD''': joint analysis of multiple SNPs, handle correlation between them, see Guan & Stephens (Annals of Applied Statistics, 2011) for MCMC, see Carbonetto & Stephens (Bayesian Analysis, 2012) for Variational Bayes  
+  
+  to do  
+  
+  
+  * '''Confounding factors in phenotype''': factor analysis, see Stegle ''et al'' (PLoS Computational Biology, 2010)  
+  
+  to do  
+  
+  
+  * '''Genetic relatedness''': linear mixed model  
+  
+  to do  
+  
+  
+  * '''Discrete phenotype''': count data as from RNAseq, Poissonlike likelihood  
+  
+  to do  
+  
+  
+  * '''Multiple phenotypes''': matrixvariate distributions, tensors  
+  
+  to do  
+  
+  
+  * '''Nonindependent genes''': enrichment in known pathways, learn "modules"  
+  
+  to do  
+  
+  
+  * '''References''':  
+  ** Servin & Stephens (PLoS Genetics, 2007)  
+  ** Guan & Stephens (PLoS Genetics, 2008)  
+  ** Stephens & Balding (Nature Reviews Genetics, 2009)  
<! ##### DO NOT edit below this line unless you know what you are doing. ##### >  <! ##### DO NOT edit below this line unless you know what you are doing. ##### > 
Revision as of 19:01, 23 November 2012
Project name  Main project page Previous entry Next entry 
Bayesian model of univariate linear regression for QTL detectionThis page aims at helping people like me, interested in quantitative genetics, to get a better understanding of some Bayesian models, most importantly the impact of the modeling assumptions as well as the underlying maths. It starts with a simple model, and gradually increases the scope to relax assumptions. See references to scientific articles at the end.
where β_{1} is in fact the additive effect of the SNP, noted a from now on, and β_{2} is the dominance effect of the SNP, d = ak. Let's now write the model in matrix notation:
This gives the following multivariate Normal distribution for the phenotypes:
Even though we can write the likelihood as a multivariate Normal, I still keep the term "univariate" in the title because the regression has a single response, Y. It is usual to keep the term "multivariate" for the case where there is a matrix of responses (i.e. multiple phenotypes). The likelihood of the parameters given the data is therefore:
A Gamma distribution for τ:
which means:
And a multivariate Normal distribution for B:
which means:
Let's neglect the normalization constant for now:
Similarly, let's keep only the terms in B for the moment:
We expand:
We factorize some terms:
Importantly, let's define:
We can see that Ω^{T} = Ω, which means that Ω is a symmetric matrix. This is particularly useful here because we can use the following equality: Ω^{ − 1}Ω^{T} = I.
This now becomes easy to factorizes totally:
We recognize the kernel of a Normal distribution, allowing us to write the conditional posterior as:
Similarly to the equations above:
But now, to handle the second term, we need to integrate over B, thus effectively taking into account the uncertainty in B:
Again, we use the priors and likelihoods specified above (but everything inside the integral is kept inside it, even if it doesn't depend on B!):
As we used a conjugate prior for τ, we know that we expect a Gamma distribution for the posterior. Therefore, we can take τ^{N / 2} out of the integral and start guessing what looks like a Gamma distribution. We also factorize inside the exponential:
We recognize the conditional posterior of B. This allows us to use the fact that the pdf of the Normal distribution integrates to one:
We finally recognize a Gamma distribution, allowing us to write the posterior as:
where
Here we recognize the formula to integrate the Gamma function:
And we now recognize a multivariate Student's tdistribution:
We hence can write:
We want to test the following null hypothesis:
In Bayesian modeling, hypothesis testing is performed with a Bayes factor, which in our case can be written as:
We can shorten this into:
Note that, compare to frequentist hypothesis testing which focuses on the null, the Bayes factor requires to explicitly model the data under the alternative. This makes a big difference when interpreting the results (see below). Let's start with the numerator:
First, let's calculate what is inside the integral:
Using the formula obtained previously and doing some algebra gives:
Now we can integrate out τ (note the small typo in equation 9 of supplementary text S1 of Servin & Stephens):
Inside the integral, we recognize the almostcomplete pdf of a Gamma distribution. As it has to integrate to one, we get:
We can use this expression also under the null. In this case, as we need neither a nor d, B is simply μ, Σ_{B} is and X is a vector of 1's. We can also defines . In the end, this gives:
We can therefore write the Bayes factor:
When the Bayes factor is large, we say that there is enough evidence in the data to support the alternative. Indeed, the Bayesian testing procedure corresponds to measuring support for the specific alternative hypothesis compared to the null hypothesis. Importantly, note that, for a frequentist testing procedure, we would say that there is enough evidence in the data to reject the null. However we wouldn't say anything about the alternative as we don't model it. The threshold to say that a Bayes factor is large depends on the field. It is possible to use the Bayes factor as a test statistic when doing permutation testing, and then control the false discovery rate. This can give an idea of a reasonable threshold.
Such a question is never easy to answer. But note that all hyperparameters are not that important, especially in typical quantitative genetics applications. For instance, we are mostly interested in those that determine the magnitude of the effects, σ_{a} and σ_{d}, so let's deal with the others first. As explained in Servin & Stephens, the posteriors for τ and B change appropriately with shifts (y + c) and scaling () in the phenotype when taking their limits. This also gives us a new Bayes factor, the one used in practice (see Guan & Stephens, 2008):
Now, for the important hyperparameters, σ_{a} and σ_{d}, it is usual to specify a grid of values, i.e. M pairs (σ_{a},σ_{d}). For instance, Guan & Stephens used the following grid:
Then, we can average the Bayes factors obtained over the grid using, as a first approximation, equal weights:
BF < function(G=NULL, Y=NULL, sigma.a=NULL, sigma.d=NULL, get.log10=TRUE){ stopifnot(! is.null(G), ! is.null(Y), ! is.null(sigma.a), ! is.null(sigma.d)) subset < complete.cases(Y) & complete.cases(G) Y < Y[subset] G < G[subset] stopifnot(length(Y) == length(G)) N < length(G) X < cbind(rep(1,N), G, G == 1) inv.Sigma.B < diag(c(0, 1/sigma.a^2, 1/sigma.d^2)) inv.Omega < inv.Sigma.B + t(X) %*% X inv.Omega0 < N tY.Y < t(Y) %*% Y log10.BF < as.numeric(0.5 * log10(inv.Omega0)  0.5 * log10(det(inv.Omega))  log10(sigma.a)  log10(sigma.d)  (N/2) * (log10(tY.Y  t(Y) %*% X %*% solve(inv.Omega) %*% t(X) %*% cbind(Y))  log10(tY.Y  N*mean(Y)^2))) if(get.log10) return(log10.BF) else return(10^log10.BF) } In the same vein as what is explained here, we can simulate data under different scenarios and check the BFs: N < 300 # play with it PVE < 0.1 # play with it grid < c(0.05, 0.1, 0.2, 0.4, 0.8, 1.6, 3.2) MAF < 0.3 G < rbinom(n=N, size=2, prob=MAF) tau < 1 a < sqrt((2/5) * (PVE / (tau * MAF * (1MAF) * (1PVE)))) d < a / 2 mu < rnorm(n=1, mean=0, sd=10) Y < mu + a * G + d * (G == 1) + rnorm(n=N, mean=0, sd=tau) for(m in 1:length(grid)) print(BF(G, Y, grid[m], grid[m]/4))
There are many equivalent ways to write the likelihood, the usual one being:
Using X_{i} to denote the ith row of the design matrix X and keeping the same definition as above for B, we have:
As the y_{i}'s can only take 0 and 1 as values, the likelihood can be written as:
We still use the same priors as above for B and the Bayes factor now is:
The interesting point here is that there is no way to calculate these integrals analytically. Therefore, we will use Laplace's method to approximate them, as in Guan & Stephens (2008).
to do
to do
to do
to do
to do
to do
to do
to do
