User:Matthew Whiteside/Notebook/Bayesian Networks

From OpenWetWare

Jump to: navigation, search

Search this Project

Customize your entry pages

Contents

Project Description

Bayesian networks (BN) are an area I want to develop a working knowledge in. The aim in this project is to identify particular research directions that employ BNs or extend BN work.


Project Goals:

  • Review BN literature
  • Identify state-of-art algorithms and tools
  • Discover applications of BNs in life sciences

Literature Review

Review of Bayesian Basics

Copied from Weng-Keen Wong, 2005.

Probability Primer

  • Random variable
refers to event and some degree of uncertainty about outcome of event
  • Probability
The relative frequency that outcomes occurs if repeated large number of times under similar conditions
"Bayesian" definition: probability is degree of belief in an outcome
  • Conditional probabilities Ρ(A = true | B = true)
Out of all outcomes in which B is true, how many also have A equal to true. Read: "probability of A conditioned on B" or "probability of A given B"
E.g.
H = "have a headache"
F = "coming down with flu"
P(H = true) = 1/10, ..... P(F = true) = 1/40, ..... P(H = true | F = true) = 1/2
"Headaches are rare, flu is rarer, but if your coming down with flu, there's a 50-50 chance you'll have a headache"
  • Joint probability distribution P(A = true, B = true)
The probability of A=true and B=true.
P(H=true|F=true) = P(H=true,F=true)/P(F=true), or the probability they both occur divided by the probability the conditioned variable occurs"
Can be any number of random variables e.g. P(A=true, B=true, C=true)
For every combination of variables, need to know how probable that combination is
A B C P(A,B,C)
F F F 0.1
F F T 0.2
. . . .
The probabilities of these combos need to sum to 1.Once you have the joint probability distribution, you can calculate any probability involving A,B and C.
E.g. P(A=true) = sum of P(A,B,C) in rows with A=true
P(A=true, B=true | C=true) = P(A=true, B=true, C=true) / P(C=true)
for k boolean random variables you need table of size 2k
Indepedence reduces the number of table entries.
  • Independence
for n coin flips, the joint distribution P(C1,...Cn), and if coin flips are not independent, you need 2n table entries
If independent, then P(C1,...Cn) = Πni=1 P(Ci)
Each P(Ci) has two table entries, for a total of 2n values
  • Conditional independence
A and B are conditionally independent given C, if any of the following:
  1. P(A,B|C) = P(A|C)P(B|C)
  2. P(A|B,C) = P(A|C)
  3. P(B|A,C) = P(B|C)
Knowing C tells me everything about B. I don't gain anything from knowing A. Two possibilities: A doesn't influence B, or C provides all information that A would provide.


Bayesian Networks

A Bayesian network is made up of

  1. A directed acyclic graph
  2. conditional probability distribution tables for each node.

These tables contain the conditional probability distribution P(Xi | Parents(Xi) for node Xi in graph. This only includes the immediate parents, and not higher ancestors. If you have k Parents, this table has 2k+1 probabilities (but because probabilities sum to 1, only 2k need to be stored.


Properties:

  1. Encodes conditional independence between variables in graph structure
  2. Compact representation of join probability distribution over variables

Conditional independence (or Markov condition): given its parents, node (X) is conditionally independent of non-descendents. Using this markov condition we can compute joint probability distribution over all variables in BN using:

P(X1=x1,...Xn=xn) = Πni=1P(Xi=xi|Parents(Xi))


Inference: Using a BN to compute probabilities is called inference. Usual form is P( X | E), X = query variables, E = evidence variables. Exact inference is possible in med-small networks. Must use approximate techniques for large networks. Can also have many unobserved values.


Design: Either you can get an field expert to design BN structure, or you can try and learn it from data.

Personal tools