makeCorrelated(sampleSet1,sampleSet2, amount)

[OAGr]

It would be great to manually be able to specify the amount of correlation between distributions. One simple way of doing this is just to do it on pairs of distributions. This operation would break correlations with previous distributions though, unless there was a bit more data involved.

a = normal(5,2) -> toSampleSet
b = normal(30,4) -> toSampleSet
newArgs = makeCorrelated(a,b,1)
a = newArgs[0]
b = newArgs[1]

It would work roughly like,

makeCorrelated(a,b,correlation) {
  a = order(a)
  b = order(b)
  if (correlation < 0){
    reverse(b)
    correlation = -1 * correlation
  }
  newB = mapSamplesWithIndex(a, {|_, index| random(correlation) ? b[index] : randomSample(b)})
}

Note that it would be better to have a slightly more fancy system not sample the same point twice. So instead, n items are kept in place, and m are shuffled randomly.

There might well be a much better algorithm here, I'm kind of making this up. But I'm pretty sure it would work, it's just kind of hacky.

Questions

It would be great to figure out some clever system for properly correlating several different distributions to each other. I'm not sure what this would look like though. Maybe something like,

// a...z are distributions
correlatedItems = setCorrelations(distributionRecord, [["a", "b", 0.3], ["a", "c", 0.4], ["b", "c", 0.2], ["b", "d", -.3]])
correlatedItems.a // correlatedA.

We might need to save data of all the samples and their respective correlated other samples in global state.

[quinn-dougherty]

See also: #374

[daaronr]

This seems very important, maybe top-priority for me.

Most/all of the examples of Fermi Monte-Carlo estimations I've seen in EA/using Squiggle seem to use only independent random variables. In most cases there are strong reasons to think some of these are strongly negatively or positively correlated. I want to start using this approach, but without being able to explore the sensitivity to this it doesn't necessary seem empirically better than the 'old school' approaches.

From my hypothesis notes:

is there an easy compositional way to do this that I'm overlooking? Like maybe a 'shared random variable' that feeds into two distributions? But I'm not sure if that can be done in the current system, because ... can the 'draws from one distribution' be carried over as inputs into the 'draws from another distribution'?

[Hazelfire]

Yeah! There's an easy way of creating correlated random variables using sample sets. It's detailed in Squiggle's documentation here:
https://www.squiggle-language.com/docs/Guides/Gotchas (It's in a silly location, and probably should be fixed. I thought I had a PR for that but clearly not)

The TLDR is that if you use sample sets, the operations that you perform on them will be done in order.

sampleSetDist = normal(5,2) |> SampleSet.fromDist
sampleSetDistToPointSet = sampleSetDist |> PointSet.fromDist
symbolicDist = normal(5,2)
[sampleSetDist-sampleSetDist, sampleSetDistToPointSet-sampleSetDistToPointSet, symbolicDist-symbolicDist]

In this example, the first element of the array is 0 but the others are not. The sample sets are fully correlated and the subtraction is performed in each order for each sample. The second and third ones are assumed to be independent and therefore don't result in 0.

I use a very similar construction in my GiveDirectly model (https://observablehq.com/@hazelfire/givewells-givedirectly-cost-effectiveness-analysis):

value_doubling_consumption(x) = utility(x.baseline_consumption * 2) - utility(x.baseline_consumption)

In this example, baseline_consumption must be a sample set, or they will be assumed to be independent and this will have a probability mass below 0, which is very undesirable.

[OAGr]

Yep - just to clarify - if you have Sample Set distributions that come from the same early distributions, they will be correlated. However, we don't have other ways of forcing correlations - and there could be a good number of reasons for wanting this.