stl_bind bind_vector removes constness

[drexlerd]

Hi,

I am returning a const std::vector<const T*> and declared std::vector<const T*> as opague type and used bind_vector to declare the bindings. Unfortunately this adds modifiers, allowing users to overwrite internal memory, by appending elements to the vector.

After checking the source code, there is no way on how to make the vector behave like a const vector. I am thinking of adding bind_const_vector` that does not create modifiers of any kind. Is that the correct solution of am I missing something?

Thanks

[jiwaszki]

@drexlerd this is a really great question. I hope that I understand it well (or maybe to well)... but have you ever seen a "constant list" in Python? This is a long lasting question of immutable containers. I have encountered same questions in my work with pybind... so in short, it is a little bit deeper than pybind itself.

Here is some background story:

• Example of frozen/const containers in Python, frozenset: https://docs.python.org/3/library/stdtypes.html#frozenset
• Here is a reflection of frozenset in pybind: https://github.com/pybind/pybind11/blob/master/include/pybind11/pytypes.h#L2247
• Also rejected ideas like frozendict: https://peps.python.org/pep-0416/
• Ofc there are existing extensions that tries to tackle that problem, i.e. https://github.com/aio-libs/frozenlist

Afaik pybind always tries to be in-line with Python as much as possible. As Python does not expose anything like frozenlist, so do pybind. Pybind as a binding library, is not required to create "standard objects" of any type to suplement well-known standard library of Python.

I also wish that such frozen-alternatives exist but even in Python you need to write them yourself. Although it could be great to have "templated approach" that utilise the collections.abc.Mapping abilities! But again, who should be defining such interfaces? As this could be a great addition, it also raises a lot of questions (and maybe even proposals to Python itself).

Hope that gives you some insight of the complicated world of frozen-containers of Python. What is your view on this? Cheers!

[drexlerd]

@jiwaszki I highly appreciate your detailed response and the pointers. I still have to process parts of it but I have a thought on this already.

pybind always tries to be in-line with Python as much as possible

It could also be the other way around, i.e., being as much as possible in-line with C++. Ultimately, our goal is to make C++ implementations available from the Python side and not the other way around. For example, when returning a const std::vector<T>& I would expect to get a ConstVectorT on the python side with accessors but no modifiers. A corresponding VectorT would act as a mutable counterpart. Also, I do not see why this is not in-line with Python since those are Python classes with a bunch of member functions that mimic the C++ semantics.

[jiwaszki]

Oh... I totally understand your view. I missed critical part of using opaque type (bind_vector). It totally slipped my mind when writing and I put an explanation of automatic conversions happening between C++ and Python in pybind.

That would be really great to have const-like containers, I would love to see implementation of it! Sorry for the fuss:)
Tbh I would suggest to try the interface similar to tuple, as this is something that looks like immutable list-like container.