-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Statistics for targeted properties #90
Comments
A heat map of the generated values is a very interesting idea and while there is one in PropEr TPBT tutorial I think it is possible to get one with random testing too. |
I was wondering about that as well. The heat map works great for the example on TPBT, because the search space is two dimensional over natural numbers. This of course does not generalize well. I find it hard to imagine what a possible visualization might look like, possibly because of missing examples. I wonder if principal component analysis (PCA) could be used for something like this, but the resulting visualizations are certainly not easy to interpret if one doesn't already have some understanding of how PCA works. The problem with PCA is that we cannot easily map generated terms for a variable to a corresponding axis. Some metric might have to be applied. Alas, as the bindings also allow a variable to accept completely different types, such a metric is not exactly something which is easy to find. |
Aren't that two things, which should be handled independently:
My impression is that the appropriate visualisation depends on the use case and may be used for general properties as well whereas the streaming part is something new which is not available in Proper. |
The heat map is interesting thingy. It is a visualisation of the statistics usually done by
collect()
and its friends, but which cannot be used by PropEr's TPBT implementation. Also the examples from Fred Hebert's book would be way nicer because such a statistic could replace the printout of all generated data.At best, such a statistic would be a part of PropEr. If we do it in PropCheck only, than I assume we need to start some GenServer collecting the data and generate a
ref
for each run. In contrast to regular properties, we would to explicitly send the data to server process. Perhaps we can hide this in theforall_targed
etc macros.Originally posted by @alfert in #87 (comment)
The text was updated successfully, but these errors were encountered: