Claret Bruno Model

[gowerc]

Closes #338

Most details are in the linked issue. Otherwise not too much special here just based most off the code off of the existing SF model with no real fundamental changes.

Only concern I have is that the ESS seems to be quite low at ~10%. Though this model has a double exponential and some terms that are hard to pick a part so I don't think its a validity concern, its not like the old non-central b parameter that only had single figure ess even with thousands of samples.... but yer not 100% sure that this is ok to ignore but my gut feeling says it is.

EDIT - Talking to Francois he was saying that the SB model tends to have slightly worse performance metrics than the GSF which means we are roughly in line with this expectation so I think this is safe to ignore.

[github-actions]

Unit Tests Summary

    1 files    150 suites   6m 46s ⏱️
  135 tests   132 ✅ 3 💤 0 ❌
1 013 runs  1 009 ✅ 4 💤 0 ❌

Results for commit e667dcd.

♻️ This comment has been updated with latest results.

[github-actions]

Unit Test Performance Difference

Test Suite	$Status$	Time on main	$±Time$	$±Tests$	$±Skipped$	$±Failures$	$±Errors$
LongitudinalClaretBruno	👶		$+0.00$	$+1$	$+1$	$0$	$0$
SimLongitudinalClaretBruno	👶		$+0.00$	$+1$	$0$	$0$	$0$

Additional test case details

Test Suite	$Status$	Time on main	$±Time$	Test Case
Grid	💚	$23.17$	$-3.59$	Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser
GridPrediction	💔	$130.31$	$+3.63$	GridPrediction_works_as_expected_for_Survival_models
LongitudinalClaretBruno	👶		$+0.00$	Can_recover_known_distributional_parameters_from_a_SF_joint_model
LongitudinalClaretBruno	👶		$+0.01$	Centralised_parameterisation_compiles_without_issues
LongitudinalClaretBruno	👶		$+3.13$	LongitudinalClaretBruno_works_as_expected_with_default_arguments
LongitudinalClaretBruno	👶		$+0.01$	Non_Centralised_parameterisation_compiles_without_issues
LongitudinalClaretBruno	👶		$+0.00$	Print_method_for_LongitudinalClaretBruno_works_as_expected
LongitudinalRandomSlope	💔	$12.22$	$+2.72$	Print_method_for_LongitudinalRandomSlope_works_as_expected
SimLongitudinalClaretBruno	👶		$+0.14$	SimLongitudinalClaretBruno_works_as_expected
SimLongitudinalClaretBruno	👶		$+0.00$	print_methods_work_as_expected
SurvivalLoglogistic	💚	$12.07$	$-2.26$	sim_os_loglogistic_is_consistant_with_flexsurv
SurvivalQuantities	💔	$20.77$	$+2.59$	SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected
brierScore	💚	$7.35$	$-2.32$	brierScore_SurvivalQuantities_returns_same_results_as_survreg
brierScore	👶		$+0.00$	match_order_works_as_expected

Results for commit dc4538b

♻️ This comment has been updated with latest results.

[gowerc]

Hmm @danielinteractive , would you mind focusing on the dsld code. For some reason the joint model is working perfectly with no links but as soon as I add the DSLD link it and the lambda_0 parameter go very wrong.

For example with link_dsld = -0.1, link_ttg=0.2 and lambda_0=1 I am getting the following:

# A tibble: 3 × 7
  variable         mean     q01     q99  rhat ess_bulk ess_tail
  <chr>           <dbl>   <dbl>   <dbl> <dbl>    <dbl>    <dbl>
1 link_dsld     -0.0418 -0.0454 -0.0384  1.00    2128.    1651.
2 link_ttg       0.216  -0.917   1.35    1.00    3398.    1370.
3 sm_exp_lambda  4.75    3.27    6.51    1.00    2905.    1726.

I've been staring at the code for ages now and I can't see any obvious place where I have gone wrong :(

EDIT - NVM just spotted a couple of mistakes / missing brackets in my formula. Am re-testing again now...
EDIT2 - ok still getting incorrect results after applying the fix :(

  variable         mean     q01     q99  rhat ess_bulk ess_tail
  <chr>           <dbl>   <dbl>   <dbl> <dbl>    <dbl>    <dbl>
1 link_dsld     -0.0358 -0.0394 -0.0324 1.00     2181.    1508.
2 link_ttg       0.215  -0.930   1.37   1.00     3030.    1359.
3 sm_exp_lambda  8.36    5.86   11.5    0.999    3197.    1493.

[github-actions]

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          33       4  87.88%   51, 395, 415, 470
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       125       8  93.60%   146-148, 198, 202, 244, 290, 296
R/JointModelSamples.R                 63       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           83       6  92.77%   156-160, 175
R/LongitudinalGSF.R                   83       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           42       5  88.10%   98-102
R/LongitudinalSteinFojo.R             76       6  92.11%   117, 149-153
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            236       8  96.61%   481, 577, 589-607
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        52       0  100.00%
R/SimLongitudinalGSF.R                53       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        143       1  99.30%   13
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               2961     120  95.95%

Diff against main

Filename                          Stmts    Miss  Cover
------------------------------  -------  ------  --------
R/brier_score.R                      +1       0  +100.00%
R/LongitudinalClaretBruno.R         +83      +6  +92.77%
R/SimLongitudinalClaretBruno.R      +52       0  +100.00%
TOTAL                              +136      +6  -0.02%

Results for commit: e667dcd

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

[danielinteractive]

Thanks @gowerc , I think I found something

[gowerc]

@danielinteractive - Unfortunately still not working :(

With real values of
link_dsld = -0.1
link_ttg = 0.2
lambda = 1

I'm getting samples of:

  variable         mean     q01     q99  rhat ess_bulk ess_tail
  <chr>           <dbl>   <dbl>   <dbl> <dbl>    <dbl>    <dbl>
1 link_dsld     -0.0390 -0.0433 -0.0345 1.00     3397.    1734.
2 link_ttg       0.189  -0.889   1.28   0.999    4259.    1487.
3 sm_exp_lambda  5.62    3.92    7.68   1.00     3781.    1384

[gowerc]

@danielinteractive - @gravesti managed to work out that there was no issue with the implementation, the problem was that my simulation parameters were too extreme and the majority of people were dying instantly. Using more sensible parameters the model works as expected and is able to recover all the known parameters 🎉

[danielinteractive]

Great news!! Thanks both!!

[gowerc]

@danielinteractive are you able to still do the review for this or should I transfer to @gravesti instead?

[danielinteractive]

Hi @gowerc , yeah sure, anything specific to check still?

[gowerc]

@danielinteractive - Not really, I would say the Stan code is probs the highest risk area though.

[danielinteractive]

thanks, otherwise looks good

[danielinteractive]

thanks Craig!