mapcache creates multible session for getting http sources and not a single one

[a14stoner]

I am seeding a map with a http source.
Within one our mapcache creates around 720.000 sessions (one per request).

Is it possible that mapcache requests the images within one session and not creates one session for every requested image?

[sdlime]

Hi @a14stoner, this is probably a better question for the mapserver-users mailing list. More folks will see it there and should be able to help. I'm not a mapcache expert but I think folks will need more information on:

• when you say seeding, are you referring to using mapcache_seed? With what options?
• if not, how is mapcache deployed (Apache module or FastCGI)?

I typically don't pre-seed and just let the cache build as users interact with the tilesets. I've not noticed anything strange in that case. I do occasionally do targeted re-tiling for small data changes and use mapcache_seed for that.

[a14stoner]

Hey, thx tog the questions. I use mapcache_seed with -n 16 I need to pre-seed bc i dont have internet connection at the customers site. Regards

…

On Thu 30. Mar 2023 at 16:52, Steve Lime ***@***.***> wrote: Hi @a14stoner <https://github.com/a14stoner>, this is probably a better question for the mapserver-users mailing list. More folks will see it there and should be able to help. I'm not a mapcache expert but I think folks will need more information on: - when you say seeding, are you referring to using mapcache_seed? With what options? - if not, how is mapcache deployed (Apache module or FastCGI)? I typically don't pre-seed and just let the cache build as users interact with the tilesets. I've not noticed anything strange in that case. I do occasionally do targeted re-tiling for small data changes and use mapcache_seed for that. — Reply to this email directly, view it on GitHub <#310 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOIWVIOOKCQNKLLRZ2RAGWDW6WMZLANCNFSM6AAAAAAWI6MB6Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[marisn]

This seems to be doable. I don't have time (and need) to implement it, thus just notes.

Download happens in http.c function mapcache_http_do_request and there is no session/connection/DNS reuse implemented.
https://github.com/MapServer/mapcache/blob/main/lib/http.c#L102

Libcurl supports sharing of cookies, DNS, ssl and even connections. The documentation has nice examples:
https://everything.curl.dev/libcurl/sharing

One option would be to create three new functions: http_setup, http_do_request and http_teardown. This would allow to change separate places in code without a major rewrite of each mapcache_http_do_request callee. As for the seeder process – each worker thread could setup its own libcurl handle with shared properties and thus there would be no need for making a clever mutex-based locking. I don't know if there should be a cli flag to disable/enable such functionality.

[jratike80]

@a14stoner, please tell why did you close the ticket. I hope you have found a solution for your issue.

[marisn]

IMHO this should be reopened as it is a reasonable thing to implement. It could speed up seeding process a bit (although only testing could tell).

[a14stoner]

I simply dont need it anymore. If you want I'll reopen it. :)