Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory usage in compute_lfc_stats_multicore #53

Open
wants to merge 1 commit into
base: better-multithread
Choose a base branch
from

Conversation

aksarkar
Copy link
Member

This change reduces memory usage by roughly 10%.

Profiling via mprof using the following snippet

devtools::load_all()
data(pbmc_facs)
temp <- de_analysis(pbmc_facs$fit, pbmc_facs$counts, verbose=FALSE, shrink.method='none', control=list(nc=32, nsplit=32))

on an AWS c7i.8xlarge instance yields the following usages.

On 295d732 295d7323

On g19ec112 g19ec112:

@pcarbo
Copy link
Member

pcarbo commented Mar 28, 2024

@aksarkar I'm going to hold off on merging your PR for now. I think there is the potential to greatly improve the memory usage, but I have to play around with the implementation, and make sure that it doesn't also slow code the multithreading.

@aksarkar
Copy link
Member Author

The timing of my test case indicates that it slightly improves the wallclock time also. In any case, it does not make appear to make the running time worse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants