Remove global AD flag "ADBACKEND" and function suite

[sunxd3]

Addressing #2132, ref #2047 (comment)

This is likely to be quite hairy, as the ADBACKEND goes back.

Steps towards completion:

• Replace Turing.ADBackend() with AutoForwardDiff as default for adtype argument in samplers
• Remove setadbackend function, the main issue here is that tests use it quite extensively, so some refactoring is needed
• Remove ADSAFE, CHUNKSIZE, and RDCache and related functions
• Update documentation and register a breaking release

[sunxd3]

For the second item, adtype needs to be thread into tests like

Turing.jl/test/runtests.jl

Lines 68 to 96 in 878cbe1

	Turing.setrdcache(false)
	for adbackend in (:forwarddiff, :reversediff)
	@timeit TIMEROUTPUT "inference: $adbackend" begin
	Turing.setadbackend(adbackend)
	@info "Testing $(adbackend)"
	@testset "inference: $adbackend" begin
	@testset "samplers" begin
	@timeit_include("mcmc/gibbs.jl")
	@timeit_include("mcmc/gibbs_conditional.jl")
	@timeit_include("mcmc/hmc.jl")
	@timeit_include("mcmc/Inference.jl")
	@timeit_include("mcmc/sghmc.jl")
	@timeit_include("mcmc/abstractmcmc.jl")
	@timeit_include("mcmc/mh.jl")
	@timeit_include("ext/dynamichmc.jl")
	end
	end
	@testset "variational algorithms : $adbackend" begin
	@timeit_include("variational/advi.jl")
	end
	@testset "mode estimation : $adbackend" begin
	@timeit_include("optimisation/OptimInterface.jl")
	@timeit_include("ext/Optimisation.jl")
	end
	end
	end

[sunxd3]

Related documentation PR TuringLang/docs#430

[sunxd3]

@yebai @devmotion the majority of the tests are passing, while the failed ones are numerical
https://github.com/TuringLang/Turing.jl/actions/runs/7086821638/job/19289666947?pr=2134#step:6:32455
https://github.com/TuringLang/Turing.jl/actions/runs/7086821638/job/19289666947?pr=2134#step:6:24542

Maybe it's time to give the PR a look?

[torfjelde]

@sunxd3 can you undo the formatting changes? It makes it somewhat annoying to review + makes version history a less clear

[coveralls]

Pull Request Test Coverage Report for Build 7194200743

• 0 of 9 (0.0%) changed or added relevant lines in 2 files are covered.
• 2 unchanged lines in 2 files lost coverage.
• Overall coverage remained the same at 0.0%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/essential/ad.jl	0	2	0.0%
src/mcmc/hmc.jl	0	7	0.0%

Files with Coverage Reduction	New Missed Lines	%
ext/TuringOptimExt.jl	1	0.0%
src/essential/ad.jl	1	0.0%

Totals
Change from base Build 6890657606:	0.0%
Covered Lines:	0
Relevant Lines:	1390

---

💛 - Coveralls

[codecov]

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (6649f10) 0.00% compared to head (db8197c) 0.00%.

❗ Current head db8197c differs from pull request most recent head 816f1f9. Consider uploading reports for the commit 816f1f9 to get more accurate results

Files	Patch %	Lines
src/mcmc/hmc.jl	0.00%	7 Missing ⚠️
src/essential/ad.jl	0.00%	2 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #2134   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files          21      21           
  Lines        1421    1390   -31     
======================================
+ Misses       1421    1390   -31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sunxd3]

@torfjelde another look? I am not familiar with the test failure, would love to take a closer look if not obvious why.

[yebai]

Can we also add adtypes interface to optimisation-based algorithms, e.g. MLE, MAP?

[sunxd3]

If I am not mistaken, Optim.jl seems to only support ForwardDiff (https://github.com/JuliaNLSolvers/NLSolversBase.jl/blob/78af38393b14992ea996899c6486d971b5bfa612/src/NLSolversBase.jl#L5). If that's the case, it may not make much sense to add ADtypes now

[yebai]

Thanks, @sunxd3 -- it looks good overall. I left a few minor comments above.

[yebai]

The remaining failing test is reproducible on the master branch so unlikely related to this PR @sunxd3 can you update the docs so we can merge this PR?

[sunxd3]

@yebai the documentation is updated here, although the building will error with the master branch version of Turing

[sunxd3]

Also @yebai @devmotion, the current v0.30.0 is not registered yet, I did bump the minor version to 0.31.0, but maybe we can treat the PR as a part of 0.30 release?