Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of memory after intersects(s1, s2) #206

Open
Ronneesley opened this issue May 3, 2024 · 6 comments
Open

Out of memory after intersects(s1, s2) #206

Ronneesley opened this issue May 3, 2024 · 6 comments

Comments

@Ronneesley
Copy link

Hello,

First of all, congrats for your work. It's owesome!

I'm working with your module, and need check 100 thousands of intersections.
After doing my loop, my memory is empty, it's about 32 GB. Although the shapes are small.
Here a sample of my code that the problem can be observed:

for i = 1:100000
    LibGEOS.intersects(shape2, shape1)
end

I observed that the problem happends even when I check the intersection of two small shapes many times.

I don't know why the problem happens, but I suspect of some global data. Inspecting the code at: https://github.com/JuliaGeo/LibGEOS.jl/blob/master/src/geos_functions.jl, I saw the context variable that could be linked at shape, that is a global variable at my code, because it is at a dataframe (Dataframes.jl), I read like this:

using Shapefile, DataFrames
table = Shapefile.Table(path)
df = DataFrame(table)

If it's the problem, how could I clear this context? If it isn't the problem, how can I check the interssection without lack of memory?

@Ronneesley
Copy link
Author

Note: There is a difference os memory usage of:

for i = 1:100000
    LibGEOS.intersects(shape1, shape2)
end

and

for i = 1:100000
    LibGEOS.intersects(shape2, shape1)
end

@asinghvi17
Copy link
Member

What's the average number of vertices for your shapes? Have you tried other geometry libraries like ArchGDAL.jl?

@Ronneesley
Copy link
Author

Hi @asinghvi17 ,

shape2 has 17 points and shape1 4262.

No I didn't know ArchGDAL, I'll try now.

@Ronneesley
Copy link
Author

Hello @asinghvi17 ,

I tried use ArchGDAL.jl and memory remains stable. My code was:

shape1n = ArchGDAL.createmultipolygon(GeoInterface.coordinates(shape1))
shape2n = ArchGDAL.createmultipolygon(GeoInterface.coordinates(shape2))

for i = 1:100000
    ArchGDAL.intersects(shape1n, shape2n)
end

and it runned very stable.

@asinghvi17
Copy link
Member

asinghvi17 commented May 3, 2024

Great! In this case, you should use GeoInterface.convert(ArchGDAL, shape1) - it's more efficient.

Looks like there is a memory leak in the Julia wrapper of LibGEOS (this package), since ArchGDAL depends on the underlying C library GEOS.

@evetion
Copy link
Member

evetion commented May 4, 2024

Relates to #184

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants