Simulate correlated DM noise in wideband TOAs #1868
Conversation
abhisrkckl
changed the title
abhisrkckl
changed the title
abhisrkckl
added
the
awaiting review
| @@ -232,6 +235,7 @@ class ScaleDmError(NoiseComponent): | |||
| category = "scale_dm_error" | |||
|
|
|||
| introduces_correlated_errors = False | |||
| introduces_dm_errors = True | |||
There was a problem hiding this comment.
What is the purpose of this?
There was a problem hiding this comment.
Currently there is no way to distinguish between achromatic noise models and noise models that represent DM noise (such as PLDMNoise) that affects the DM part of the wideband likelihood. I added this attribute to distinguish such noise components. The timing model components that introduce a DM correction can already be distinguished by checking if they are derived from the Dispersion class.
| assert t.is_wideband() | ||
|
|
||
| # There is correlated noise in DMs | ||
| assert ( |
There was a problem hiding this comment.
So this is some sort of a chi^2 test? Is there no better way to check for correlated noise?
There was a problem hiding this comment.
Yes. A better test will be something like Ljung-Box test. Unfortunately, it is not available in scipy. We will need to add statsmodels as a dependency to use that test.
There was a problem hiding this comment.
It looks like that test would not be very hard to implement, maybe include in PINT utils. Although it may not be needed just for this, it could be more broadly useful.
There was a problem hiding this comment.
OK... I have added the Ljung-Box test. For this I added statsmodels as a dependency for tests. It is not a dependency for the package itself since it is not used anywhere in the main code. #1776 may change that though.
|
Looks good. I wonder if this test should be used elsewhere for correlated noise checks. How useful is it to test for the presence of correlated noise in the data? Should we have a utility function for that? Not needed for this PR - just asking. Should I merge? |
|
Yes please merge this.
…On Fri, Jan 17, 2025, 16:17 David Kaplan ***@***.***> wrote:
Looks good. I wonder if this test should be used elsewhere for correlated
noise checks. How useful is it to test for the presence of correlated noise
in the data? Should we have a utility function for that? Not needed for
this PR - just asking.
Should I merge?
—
Reply to this email directly, view it on GitHub
<#1868 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE6CGN3D3RJJ2SGW6MAGHFL2LENJBAVCNFSM6AAAAABTB7CBHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJYGU4DSNZXGI>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
I think it is useful to check for correlations to make sure that the noise
model is correctly describing the data. Is the noise model is correct the
whitened residuals will be Gaussian and white.
…On Fri, Jan 17, 2025, 17:09 Abhimanyu S ***@***.***> wrote:
Yes please merge this.
On Fri, Jan 17, 2025, 16:17 David Kaplan ***@***.***> wrote:
> Looks good. I wonder if this test should be used elsewhere for correlated
> noise checks. How useful is it to test for the presence of correlated noise
> in the data? Should we have a utility function for that? Not needed for
> this PR - just asking.
>
> Should I merge?
>
> —
> Reply to this email directly, view it on GitHub
> <#1868 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AE6CGN3D3RJJ2SGW6MAGHFL2LENJBAVCNFSM6AAAAABTB7CBHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJYGU4DSNZXGI>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
Partially fixes #1660. Fitting a model to wideband TOAs in the presence of correlated DM noise is still not implemented.