random DLA z can not be higher than Z_QSO

[londumas]

In v4.2, random DLA redshift should not be higher than Z_QSO.
This is not important, but it should not be hard to correct.

[londumas]

Using /project/projectdirs/desi/mocks/lya_forest/saclay/v4.4/v4.4.7/master_DLA.fits I get still the issue of random DLA with redshift higher than the host quasar, but now I also see some mock data DLA with the same issue.

[londumas]

The issue to have mock DLA with ZDLA>ZQSO is only in v4.4.7.

[londumas]

Here is the stack of the 10 cross-correlation HCDxQSO multiplied by r, we can see clearly an asymmetry between xi(rp<0) and x(rp>0), where rp is computed as a function of the relative distances of HCD and QSO with respect to us. This can be linked either to this issue, to other issues or to some bugs in picca. Of course solving this ticket is not a priority, since it has little to do with BAO from the auto and cross of Lya.

[TEtourneau]

I created a new branch correct_dla_randoms, where I tried to fix this bug.
I first tried to improve the original code dla_randoms.py, to avoid having z_dla > z_qso, but I didn't manage to both fix the bug and keep the same n(z) distribution for DLA and random DLA.

So I modified the code dla_saclay.py, in order to compute both DLA and random DLA.
So now, the random DLA are computed exactly the same way as DLA, but without looking where delta is above a given threshold (DLA can be anywhere in the forest).
Here is the n(z) distribution I get:

blue is n(z) for DLA
orange is n(z) for random DLA

and here is the z_dla - z_qso distribution:

with again blue: DLA
orange: random

So the bug is fixed, but the n(z) distribution looks quite different. The difference looks significative (there is 33 000 DLA and 215 000 randoms), I have to look into that.

[londumas]

@TEtourneau, perfect, this looks good. We need the DLA and the random DLA to be on the line of sight of a quasar, is that the case?

[TEtourneau]

Yes, it is.
The differences between the n(z) for mocks and randoms are not an issue ?

[londumas]

Yes, it is.

@TEtourneau, good.

The differences between the n(z) for mocks and randoms are not an issue ?

Yes it is an issue, could it be a random fluctuation? by eye it does not seem possible, but did you look at more than 1 box?
Could it be an effect of the velocity gradient?

[TEtourneau]

Maybe the blue distribution (mocks) can fluctuate (only 33000 DLA for this plot), but not the randoms (more than 200,000).
In principle, I'm not taking into account neither the density field, the velocity field nor the velocity gradient field.
I'll try to produce a larger catalog, to see if the difference is still there.

[londumas]

@TEtourneau, I wonder if I know what is happening.
If I understood correctly you draw the DLA according to a given
threshold. Is it on delta_gaussian, on delta = delta_gauss+delta_small_scale,
or on the transmission of Lyman-alpha? Is the threshold evolving with redshift?

[TEtourneau]

To draw the DLA, we use the delta_l field, the one for the large scale fluctuations.
So we define a threshold mu (linked to the bias we want), and flag all the cells that have delta_l > mu
And then, in each flagged cell, we draw or not a DLA (using poisson distribution and f(N) from pyigm).
Since we want a constant bias (b(z) ~ 2), the threshold is contant with redshift.

Concerning the randoms, we do the exact same procedure, except that we flag all the cell within the lya forest (we don't do the delta_l > mu step).

[TEtourneau]

I produced a new random catalog, with 65 milion random DLAs, from the version v4.4.0:
/global/projecta/projectdirs/desi/mocks/lya_forest/saclay/v4.4/v4.4.0/master_DLA_randoms_test.fits

Here are some plots, showing the n(z) distribution for the DLA:

the n(z) distribution for the host quasars:

and the distribution of z_dla - z_qso:

So the n(z) of DLA and random DLA is not the same, and it's not a statistical fluctuation. This difference may come from the difference in n(z) host qso distribution.
I don't really know if there is a bug in my code, or if there is some effect that I don't take into account. I'm digging into this.

[londumas]

@TEtourneau, thank you very much for this, it is clear that it is not a statistical fluctuation indeed.
My guess is that it comes from a z-dependant criterion on the delta threshold.
Could you plot the distribution of mock DLA is you use a very different threshold?
At the end of the day, it is quite simple to downsample the random mock DLA to have the same distribution as the mock DLA. But first let's be sure we understand what is ahppening.

[TEtourneau]

@londumas You was right. There was still a redshift dependent correction remaining in the random treatment.
Here are the new plots, with the corrected n(z) distribution:

There is still a small difference at the low z tail, because of the RSD.

I pushed the commits in the branch correct_dla_randoms and merged the branch into the master.

[londumas]

@TEtourneau, very good. THank you.