Investigate using Pathfinder

[marcdotson]

Let's document in Writing, using the mle-optimize branch, what cmdstanr is doing when $optimize() finds the MLE. What does this mean for hierarchical models?

[marcdotson]

[marcdotson]

Instead, use the pathfinder branch to try and implement this new variational approximation method.

[marcdotson]

Currently requires using the release candidate:

cmdstanr::install_cmdstan(version = "2.33.0-rc1", cores = 4)

And possibly a workaround for it not being implemented in cmdstanr yet?

run_pathfinder = function(data,mod){
	data_file = tempfile(fileext='.json')
	output_file = tempfile(fileext='.csv')
	cmdstanr::write_stan_json(data,file=data_file)
	processx::run(
		command = mod$exe_file()
		, args = c(
			'pathfinder'
			, 'data'
			, paste0('file=',data_file)
			, 'output'
			, paste0('file=',output_file)
		)
		, echo_cmd = T
		, stdout = ""
		, stderr = "2>&1"
	)
	
	(
		paste0("grep '^[#l]' '",output_file,"'")
		%>% system(intern=T)
		%>% strsplit('\n')
		%>% unlist()
	) -> header
	header_nlines = which(stringr::str_starts(header,'lp'))
	found_samples_col_names = unlist(strsplit(header[header_nlines],','))
	(
		data.table::fread(
			cmd = paste0(
				"tail -n+"
				, header_nlines + 1
				, " '"
				, output_file
				, "' | grep -v '^[#]' --color=never"
			)
			, data.table = FALSE
			, sep = ','
			, header = F
			, col.names = found_samples_col_names
			, colClasses = list(numeric=1:length(found_samples_col_names))
		)
		%>% as_tibble()
	) ->
		out
	return(out)
}

[marcdotson]

That doesn't work. Keep an eye out for Pathfinder being implement in cmdstanpy first.