Merge pull request #29 from poseidon-framework/fix_fst

FST fix and user input robustness
poseidon-framework · Mar 13, 2024 · ba8e025 · ba8e025
2 parents e3bafad + 39525c7
commit ba8e025
Show file tree

Hide file tree

Showing 15 changed files with 812 additions and 139 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,4 @@
+- 1.0.1.0: fixed a bug in the FST estimation, added more graceful errors in case of illegal input for fstats
 - 1.0.0.2: Switched to poseidon-hs v1.4.0.3 and a new compiler (GHC 9.4.7) and stackage snapshot version (LTS-21.17)
 - 1.0.0.1: Switched to poseidon-hs v1.4.0.2
 - 1.0.0.0: Some key updates:

diff --git a/CHANGELOGRELEASE.md b/CHANGELOGRELEASE.md
@@ -1,3 +1,15 @@
+# V1.0.1.0: FST fix and better user input checks
+
+This release fixes a bug in the estimation of FST. We have now exactly implemented the estimator presented in Bhatia et al. 2013
+(see whitepaper), and confirmed numerical equivalence with estimation by admixtools. 
+
+We have also now checked for degenerate statistics (for example F2(A, A) or F3(A, B, A)) and in those cases now output exactly zero.
+Previously, due to bias corrections, results could become negative.
+
+Finally, we have implemented a check for illegal sample sizes. In particular for F3(A, B, C), where C is a haploid outgroup, we now 
+throw a helpful error that one should rather use F3vanilla, since F3 in that case is undefined due to its use of the heterozygosity of
+C in the bias-correction term (see whitepaper).
+
 # V1.0.0.2: performance-fix due to update of poseidon-hs dependency
 
 This release simply updates the dependency on poseidon-hs, which in turn [fixed a performance leak](https://github.com/poseidon-framework/poseidon-hs/releases/tag/v1.4.0.3) with our entity selection framework.

diff --git a/docs/main.tex b/docs/main.tex
diff --git a/docs/references.bib b/docs/references.bib
@@ -1,4 +1,18 @@
 
+@ARTICLE{Bhatia2013-qj,
+  title   = "Estimating and interpreting {FST}: The impact of rare variants",
+  author  = "Bhatia, G and Patterson, N and Sankararaman, S and Price, A L",
+  journal = "Genome research",
+  volume  =  23,
+  number  =  9,
+  pages   = "1514--1521",
+  month   =  "1~" # sep,
+  year    =  2013,
+  url     = "http://genome.cshlp.org/cgi/doi/10.1101/gr.154831.113",
+  file    = "All Papers/B/Bhatia et al. 2013 - Bhatia et al. 2013 - Supplemental_Material.pdf;All Papers/B/Bhatia et al. 2013 - Estimating and interpreting FST - The impact of rare variants.pdf;All Papers/B/Bhatia et al. 2013 - Supplemental_Material.docx",
+  issn    = "1088-9051"
+}
+
 @ARTICLE{Patterson2012-yq,
   title    = "Ancient admixture in human history",
   author   = "Patterson, Nick and Moorjani, Priya and Luo, Yontao and Mallick,

diff --git a/docs/xerxes_whitepaper.pdf b/docs/xerxes_whitepaper.pdf
diff --git a/manual_tests/fstats_tests/runFstats.sh b/manual_tests/fstats_tests/runFstats.sh
@@ -2,15 +2,15 @@
 
 stack run xerxes -- fstats --statFile fstats.txt -d ../../../community-archive/2012_PattersonGenetics -f testTable.txt --blockTableFile testBlockTable.txt
 
-# .-----------.-----------.---------------.--------.-------.---------.----------------.--------------------.------------------.
-# | Statistic |     a     |       b       |   c    |   d   | NrSites | Estimate_Total | Estimate_Jackknife | StdErr_Jackknife |
-# :===========:===========:===============:========:=======:=========:================:====================:==================:
-# | F3        | French    | Italian_North | Mbuti  |       | 593124  | 5.9698e-2      | 5.9698e-2          | 5.1423e-4        |
-# | F3        | French    | Han           | Mbuti  |       | 593124  | 5.0233e-2      | 5.0233e-2          | 5.0324e-4        |
-# | F3        | Sardinian | Pima          | French |       | 593124  | -1.2483e-3     | -1.2483e-3         | 9.2510e-5        |
-# | F4        | French    | Russian       | Han    | Mbuti | 593124  | -1.6778e-3     | -1.6778e-3         | 9.1419e-5        |
-# | F4        | Sardinian | French        | Pima   | Mbuti | 593124  | -1.4384e-3     | -1.4384e-3         | 1.1525e-4        |
-# '-----------'-----------'---------------'--------'-------'---------'----------------'--------------------'------------------'
+# .-----------.-----------.---------------.--------.-------.---------.----------------.--------------------.------------------.---------------------.
+# | Statistic |     a     |       b       |   c    |   d   | NrSites | Estimate_Total | Estimate_Jackknife | StdErr_Jackknife |  Z_score_Jackknife  |
+# :===========:===========:===============:========:=======:=========:================:====================:==================:=====================:
+# | F3        | French    | Italian_North | Mbuti  |       | 593124  | 5.9698e-2      | 5.9698e-2          | 5.1423e-4        | 116.0908951980249   |
+# | F3        | French    | Han           | Mbuti  |       | 593124  | 5.0233e-2      | 5.0233e-2          | 5.0324e-4        | 99.81843057232513   |
+# | F3        | Sardinian | Pima          | French |       | 593124  | -1.2483e-3     | -1.2483e-3         | 9.2510e-5        | -13.493505348221081 |
+# | F4        | French    | Russian       | Han    | Mbuti | 593124  | -1.6778e-3     | -1.6778e-3         | 9.1419e-5        | -18.35262346091248  |
+# | F4        | Sardinian | French        | Pima   | Mbuti | 593124  | -1.4384e-3     | -1.4384e-3         | 1.1525e-4        | -12.481084899924868 |
+# '-----------'-----------'---------------'--------'-------'---------'----------------'--------------------'------------------'---------------------'
 
 stack run xerxes -- fstats --statConfig config.yaml -d ../../../community-archive --maxSnps 100000 -f testTable2.txt --blockTableFile testBlockTable2.txt