The index of relatedness or the coefficient of relationship expresses the chance of a gene being shared between two individuals A and B. To calculate the relatedness find the common ancestor(s). Having located the common ancestor(s), count the generation distance between the common ancestor(s) and respectively A and B. Add up the generation distance of A and the generation distance of B to find the total generation distance of A and B.

r = a * (1/2)^g

where

r = relatedness

a = common ancestor(s)

g = generation distance

E.g.

First cousins have 2 common ancestors, thus a=2, with a generation distance of 2+2=4, thus g=4.

relatedness between first cousins = 2 * (1/2)^4 = 2 * (1/2 * 1/2 * 1/2 * 1/2) = 1/8

For relationships as distant as third cousin (2 * (1/2)^8=1/128) the relatedness is close to the baseline probability that a gene will be shared by any random individual taken from the population.

– Joe Chang, Professor of Statistics, Yale University (yale.edu)