An Improved F(st) Estimator.

The fixation index F(st) plays a central role in ecological and evolutionary genetic studies. The estimators of Wright ([Formula: see text]), Weir and Cockerham ([Formula: see text]), and Hudson et al. ([Formula: see text]) are widely used to measure genetic differences among different populations,...

Full description

Saved in:
Bibliographic Details
Main Authors: Guanjie Chen, Ao Yuan, Daniel Shriner, Fasil Tekola-Ayele, Jie Zhou, Amy R Bentley, Yanxun Zhou, Chuntao Wang, Melanie J Newport, Adebowale Adeyemo, Charles N Rotimi
Format: Article
Language:English
Published: Public Library of Science (PLoS) 2015-01-01
Series:PLoS ONE
Online Access:https://doi.org/10.1371/journal.pone.0135368
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1849726482608291840
author Guanjie Chen
Ao Yuan
Daniel Shriner
Fasil Tekola-Ayele
Jie Zhou
Amy R Bentley
Yanxun Zhou
Chuntao Wang
Melanie J Newport
Adebowale Adeyemo
Charles N Rotimi
author_facet Guanjie Chen
Ao Yuan
Daniel Shriner
Fasil Tekola-Ayele
Jie Zhou
Amy R Bentley
Yanxun Zhou
Chuntao Wang
Melanie J Newport
Adebowale Adeyemo
Charles N Rotimi
author_sort Guanjie Chen
collection DOAJ
description The fixation index F(st) plays a central role in ecological and evolutionary genetic studies. The estimators of Wright ([Formula: see text]), Weir and Cockerham ([Formula: see text]), and Hudson et al. ([Formula: see text]) are widely used to measure genetic differences among different populations, but all have limitations. We propose a minimum variance estimator [Formula: see text] using [Formula: see text] and [Formula: see text]. We tested [Formula: see text] in simulations and applied it to 120 unrelated East African individuals from Ethiopia and 11 subpopulations in HapMap 3 with 464,642 SNPs. Our simulation study showed that [Formula: see text] has smaller bias than [Formula: see text] for small sample sizes and smaller bias than [Formula: see text] for large sample sizes. Also, [Formula: see text] has smaller variance than [Formula: see text] for small Fst values and smaller variance than [Formula: see text] for large F(st) values. We demonstrated that approximately 30 subpopulations and 30 individuals per subpopulation are required in order to accurately estimate F(st).
format Article
id doaj-art-ad412614e35f4e579d721537af036fc6
institution DOAJ
issn 1932-6203
language English
publishDate 2015-01-01
publisher Public Library of Science (PLoS)
record_format Article
series PLoS ONE
spelling doaj-art-ad412614e35f4e579d721537af036fc62025-08-20T03:10:09ZengPublic Library of Science (PLoS)PLoS ONE1932-62032015-01-01108e013536810.1371/journal.pone.0135368An Improved F(st) Estimator.Guanjie ChenAo YuanDaniel ShrinerFasil Tekola-AyeleJie ZhouAmy R BentleyYanxun ZhouChuntao WangMelanie J NewportAdebowale AdeyemoCharles N RotimiThe fixation index F(st) plays a central role in ecological and evolutionary genetic studies. The estimators of Wright ([Formula: see text]), Weir and Cockerham ([Formula: see text]), and Hudson et al. ([Formula: see text]) are widely used to measure genetic differences among different populations, but all have limitations. We propose a minimum variance estimator [Formula: see text] using [Formula: see text] and [Formula: see text]. We tested [Formula: see text] in simulations and applied it to 120 unrelated East African individuals from Ethiopia and 11 subpopulations in HapMap 3 with 464,642 SNPs. Our simulation study showed that [Formula: see text] has smaller bias than [Formula: see text] for small sample sizes and smaller bias than [Formula: see text] for large sample sizes. Also, [Formula: see text] has smaller variance than [Formula: see text] for small Fst values and smaller variance than [Formula: see text] for large F(st) values. We demonstrated that approximately 30 subpopulations and 30 individuals per subpopulation are required in order to accurately estimate F(st).https://doi.org/10.1371/journal.pone.0135368
spellingShingle Guanjie Chen
Ao Yuan
Daniel Shriner
Fasil Tekola-Ayele
Jie Zhou
Amy R Bentley
Yanxun Zhou
Chuntao Wang
Melanie J Newport
Adebowale Adeyemo
Charles N Rotimi
An Improved F(st) Estimator.
PLoS ONE
title An Improved F(st) Estimator.
title_full An Improved F(st) Estimator.
title_fullStr An Improved F(st) Estimator.
title_full_unstemmed An Improved F(st) Estimator.
title_short An Improved F(st) Estimator.
title_sort improved f st estimator
url https://doi.org/10.1371/journal.pone.0135368
work_keys_str_mv AT guanjiechen animprovedfstestimator
AT aoyuan animprovedfstestimator
AT danielshriner animprovedfstestimator
AT fasiltekolaayele animprovedfstestimator
AT jiezhou animprovedfstestimator
AT amyrbentley animprovedfstestimator
AT yanxunzhou animprovedfstestimator
AT chuntaowang animprovedfstestimator
AT melaniejnewport animprovedfstestimator
AT adebowaleadeyemo animprovedfstestimator
AT charlesnrotimi animprovedfstestimator
AT guanjiechen improvedfstestimator
AT aoyuan improvedfstestimator
AT danielshriner improvedfstestimator
AT fasiltekolaayele improvedfstestimator
AT jiezhou improvedfstestimator
AT amyrbentley improvedfstestimator
AT yanxunzhou improvedfstestimator
AT chuntaowang improvedfstestimator
AT melaniejnewport improvedfstestimator
AT adebowaleadeyemo improvedfstestimator
AT charlesnrotimi improvedfstestimator