-
-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory usage and tail calls #534
Comments
Hi @yorickhardy. First of all, thanks for the report! These both seem like real issues, let me find some time this week to take a closer look. |
Regarding:
Suspect this is related to the amount of memory added to a thread after major GC. Need to look into this more. Regarding the
|
Perform full scanning of function application list to ensure self-recursive calls are found. This prevents infinite loops in the beta expansion code when compiling simple recursive calls.
Thanks! I am sorry that I have not contributed any fixes, I am going to (eventually) try to put more time into understanding cyclone and help where/if I can. |
No worries, and bug reports are always appreciated! I would welcome fixes as well, however, these two issues in particular are in areas that would be difficult to track down... and I still need to investigate the first one :) |
As this runs, Consider the memory usage when using fixnums or doubles:
That said, I suspect we can do better here, especially since the interpreters can. Will need to spend time looking into this further. |
Snippet of code from
Compare with code from
Questions: Why are we doing an allocation here but not above, and can we safely speed up / optimize the latter code? |
I did not realize that it had switched over to bignum! Thanks. I have more or less isolated the memory usage problems that I was encountering (firstly, I was using many short lived threads instead of a thread pool and assumed that thread-join would (eventually) garbage collect the thread: that assumption is false as far as I can tell). Here is another example with bounded(?) but large memory use:
and similarly with constantly growing memory use:
which surprised me! This simple example grows a bit slower but in the same way:
Strangely, I am not sure whether these examples are helpful, or just examples of poorly written scheme! At this point I am not convinced that the remaining part is a valid issue, or poor programming on my part - so please close the issue if you are satisfied. |
Hello again, The memory consumption that motivated this issue is all due to the use of many threads, via srfi-18 and I was very surprised that terminated threads consume memory. Nevertheless, I don't think the issue is entirely valid as reported, since the underlying problem was threads (not GC and tail calls). Apologies if I have wasted too much of your time. (I do appreciate that you have looked into the reported examples and that you are willing to investigate improvements.) Thanks! |
Glad you got it working @yorickhardy! I appreciate your feedback and think there are genuine issues that are being raised here, though I have not dug into your latest |
@yorickhardy Do you have an example of |
Sure! I hope I have not missed anything obvious ...
|
Hello again, I am not sure if this is the whole story, but after adding a bit of debug output it seems that the memory allocated in I am not yet able to make a more meaningful contribution, but I am trying to work towards it! |
Hey @yorickhardy, appreciate the update! That's interesting.... we do I wonder, could it be that major GC is not being triggered by the example program? |
Yes, that seems to be the beginning of the issue. Am I correct in saying that the collector only starts (sometimes) when allocating scheme objects? In my debugging output, I hacked together a workaround to force the collector out of the |
Correct, the collector will only trigger a major GC when allocating objects and the runtime detects a need to start that collector. For example, a percentage of memory being used up. |
Thanks. In I still need to track down how the heap allocations are eventually freed, and then I can try to force the freeing of memory to see if that shows better memory use for the example program. |
Hello,
I am trying out
(srfi 18)
in cyclone and noticed that memory was being "consumed" very quickly by a simple monitoring thread, the following small example seems to exhibit this behaviour:Similarly, the cyclone compiler can be encouraged to use excessive memory (and compile time) by:
Both examples work as expected in
icyc
(with regards to memory use).Am I doing something unreasonable here? I am using cyclone-0.36.0 on NetBSD 10.99.10 amd64 (current-ish).
The text was updated successfully, but these errors were encountered: