The index of relatedness or the coefficient of relationship expresses the chance of a gene being shared between two individuals A and B. To calculate the relatedness find the common ancestor(s). Having located the common ancestor(s), count the generation distance between the common ancestor(s) and respectively A and B. Add up the generation distance of A and the generation distance of B to find the total generation distance of A and B.
r = a * (1/2)^g
where
r = relatedness
a = common ancestor(s)
g = generation distance
E.g.
First cousins have 2 common ancestors, thus a=2, with a generation distance of 2+2=4, thus g=4.
relatedness between first cousins = 2 * (1/2)^4 = 2 * (1/2 * 1/2 * 1/2 * 1/2) = 1/8
For relationships as distant as third cousin (2 * (1/2)^8=1/128) the relatedness is close to the baseline probability that a gene will be shared by any random individual taken from the population.
– Joe Chang, Professor of Statistics, Yale University (yale.edu)