Consider replacing cookie_factory (and maybe nom)

[FreezyLemon]

nom works for our parsing needs, but cookie_factory (made by the same team, similar API to nom) is effectively abandoned. I think it is unlikely that this crate will receive further upstream support. On the other hand, matroska could carry the burden (for now?) and just deal with problems if and when they arise.

If a switch away from cookie_factory should ever be made, it might make sense to move to a combined (de-)serialization crate and ditch nom too.

[FreezyLemon]

carried over from hackmd, original comment by @Luni-4

Since we can still allow to make heavy refactors, I agree with you in exploring other alternatives. But which ones? Currently I do not have anything in my mind. I know they have created a more updated crate based on nom, such as winnow, but it seems it does not present any deserialization feature. In general, I agree with you about moving away from cookie_factory since it is no more maintained.

[lu-zero]

@Geal do you have suggestions :) ?

We can probably benchmark and decide if we should embrace bitstream-io or use and promote more our own bitreading/writing facility (we have one).

[Geal]

So, cookie-factory is not really abandoned, it's mainly that I have not seen a significant need for more features. I was looking for a better API at some point, but did not find a satisfying approach.
If you're moving away from nom and cookie-factory, it could be a good idea to revisit the original architecture, and use a library for the purely EBML part, but then making a good API from scratch for the matroska level stuff

[Luni-4]

To reduce the number of dependencies of the matroska crate we can use data crate and introduce all primitives we need there. In this way we can have serialization and deserialization features accessible from a single place.

We can structure data crate using nom (or winnow) for parsing primitives and take some cookie_factory concepts for the serialization part. Is is something feasible according to you?

[FreezyLemon]

Yes, I think that's feasible. That would additionally help with organizing the code, which is good. Maybe the parsing/serializing can be abstracted away into traits so the code outside of data isn't dependent on backend changes (e.g. switching away from nom at a later date)

[Luni-4]

I perfectly agree with this plan. @lu-zero do you agree?

[lu-zero]

The proposed code should live in bitstream, or a similar crate, not data. (and we should benchmark bitstream-io to avoid duplication if enough of the ecosystem is already using it)

[Luni-4]

Starting from @Geal new information, we can define on this repository:

• A crate to parse and produce the EBML format, so an independent library
• A crate to mux and demux matroska with a new series of API

To achieve this, we can keep on using nom, so we do not have to change everything, for parsing. We can try a different approach for serialization since cookie_factory seems unusable for our case.

We can eventually move away our pieces of code inside av-data. In this way we do not have to continuously interact with another repository and its updates, simplifying our a life a bit.

What do you think @FreezyLemon @Geal @lu-zero?

[FreezyLemon]

The code is already organized to mostly separate EBML from Matroska-specific logic, but I agree that we could move it into a separate crate.

To achieve this, we can keep on using nom, so we do not have to change everything, for parsing. We can try a different approach for serialization since cookie_factory seems unusable for our case.

I think cookie_factory is generally suited for use in matroska. I thought that the library was abandoned, which it turns out was wrong. I guess we can just go ahead with nom & cookie_factory and reevaluate this at a later point in the implementation.

[Luni-4]

I agree with you 👍

[lu-zero]

if we keep using cookie_factory we should help maintaining it (and get another release out), same for nom vs winnow discussions.