Reproducible Research

Codes and data to reproduce the results of the following papers:

• 2024_SimRef "Validation of ML-UQ calibration statistics using simulated reference values: a sensitivity analysis" by P. Pernot (2024) arXiv

• 2024_RCE "Negative impact of heavy-tailed uncertainty and error distributions on the reliability of calibration statistics for machine learning regression tasks" by P. Pernot (2024) arXiv

• 2023_BVS "Can bin-wise scaling improve consistency and adaptivity of prediction uncertainty for machine learning regression ?" by P. Pernot (2023) arXiv

• 2023_Adaptivity "Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity" by P. Pernot (2023) APL Mach. Learn. 1:046121; arXiv

• 2023_isotonic "Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statistics" by P. Pernot (2023) arXiv

• 2023_ENCE "Properties of the ENCE and other MAD-based calibration metrics" by P. Pernot (2023) arXiv

• 2023_Primer "Validation of uncertainty quantification metrics: a primer based on the consistency and adaptivity concepts" by P. Pernot (2023) arXiv

• 2022_SampleMean "Comparison of recent estimators of uncertainty on the mean for small measurement samples with normal and non-normal error distributions" by P. Pernot and J.-P. Berthet (2022) arXiv

• 2022_Confidence "Confidence curves for UQ validation: probabilistic reference vs. oracle" by P. Pernot (2022) arXiv

• 2022_Tightness "Prediction uncertainty validation for computational chemists" by P. Pernot (2022) J. Chem. Phys. 157:144103; arXiv

• PU2022 "The long road to calibrated prediction uncertainty in computational chemistry" by P. Pernot (2022) J. Chem. Phys. 156:114109; arXiv

• Gini "Using the Gini coefficient to characterize the shape of computational chemistry error distributions", by P. Pernot and A. Savin (2021) Theor. Chem. Acc. 140:24; arXiv

• ML2020 "Impact of non-normal error distributions on the benchmarking and ranking of Quantum Machine Learning models", by P. Pernot, B. Huang and A. Savin (2020) Machine Learning: Science and Technology 1:035011; arXiv

• SIP "Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory", by P. Pernot and A. Savin (2020). J. Chem. Phys. 152:164108; arXiv,
  and
  "Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. II. Applications", by P. Pernot and A. Savin (2020). J. Chem. Phys. 152:164109; arXiv.

• ECDFT "Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors", by P. Pernot and A. Savin (2018) J. Chem. Phys. 148:241707

• PUIF "The parameters uncertainty inflation fallacy", by P. Pernot (2017) J. Chem. Phys. 147:104102; arXiv

• CalPred "A critical review of statistical calibration/prediction models handling data inconsistency and model inadequacy", by P. Pernot and F. Cailliez (2017) AIChE J. 63:4642; arXiv

• PU_DFA "Prediction Uncertainty of Density Functional Approximations for Properties of Crystals with Cubic Symmetry" by P. Pernot, B. Civalleri, D. Presti and A. Savin (2015) J. Phys. Chem. A 119:5288-5304

• NegVar "Model's output variance can increase when input variance decreases: a sensitivity analysis paradox?" by P. Pernot, M. Désenfant and F. Hennebelle (2015) 17th International Congress of Metrology, B. Larquier, Ed., EDP Sciences