A proposal for more semantically correct handling of Mobjects

[Darylgolden]

In Manim currently, Mobjects are initialized as instances of certain classes. These classes themself follow an inheritance structure, with VMobject being the parent of most geometrical objects. While this approach seems sensible at first glance, there are a number of issues with it that lead me to believe that it is not a clean approach:

• An object's type does not necessarily match its geometry: A user can, for example, create a Square and transform it to a Circle. However, after the transformation, the transformed Mobject is still a Square; the only thing that has changed is the rendering information (geometry and style data). If they were to try to access this Mobject's radius for example, they would not be able to, since the Mobject is still secretly a square. The mismatch that can occur between a Mobject's actual geometry with its type, attributes and methods can cause confusion and reflects Manim's poor design. Furthermore, if a renderer would like to optimize the rendering of certain Mobjects, it cannot determine whether a Mobject of type Square is really a square, making the task very difficult.

• Other problems with Transformation: To transform a Mobject to another Mobject, Manim currently has to modify all attributes related to geometry and style data. These attributes are ungrouped and it can be difficult for humans to understand and keep track of all these attributes. It would be better if this information was properly encapsulated.

• Geometrical objects are tied too closely to VMobjects: other than a few exceptions, every Mobject in Manim is a VMobject. A VMobject (vectorized mobject) is essentially a collection of bezier curves. This offers a lot of flexibility - this is what allows all VMobjects to share common animations - Create, Write and transformation between arbitrary VMobjects. However, this flexibility comes at the expense of long render times. It's often the case that such flexibility is not required; the user should be able to decide when this is true and switch to a cheaper rendering method in those cases.

In short, using types (which can't be changed after instantiation) to provide information on Mobjects isn't a good idea, when a large part of the philosophy of Manim is in allowing Mobjects to freely transform.

Immutability?

In the past, I have considered ideas with Mobject immutability, i.e. Mobject data is locked unless specifically unlocked. While this resolves the renderer optimization problem (a renderer can trust that a locked object of a certain type really has geometry corresponding to that type), it does not solve the problems of mismatched type and attributes, and it also does not make sense to render a special type of VMobject in a way that ignores its VMobject-ness (being composed of bezier curves) entirely.

A proposed solution

Let's take a Circle for example. Normally, when Circle() is run, Python calls the __new__ method, instantiating a new object of the Circle type. However, I propose that instead of doing this, we instead instantiate an abstract mobject wrapper object, and pass the specific information to be encapsulated within the wrapper. This might look something like this:

class Object:
    pass


class Wrapper:
    def __init__(self, cls, *args, **kwargs):
        self.obj = cls._initialize(*args, **kwargs)
        self.rendering_data = self.obj._to_data()
        # or perhaps just a cached property that should be 
        # accessed at render time


class Mobject:
    @classmethod
    def _initialize(cls, *args, **kwargs):
        obj = Object.__new__(cls)
        obj.__init__(*args, **kwargs)
        return obj

    # etc

class Circle(Arc):
    def __new__(cls, *args, **kwargs):
        return Wrapper(Circle, *args, **kwargs)

    def __init__(self, radius, color, **kwargs):
        self.radius = radius
        self.color = color 
        super().__init__(
        # stuff,
        **kwargs
    )

This way, when a x = Circle() is run, what is actually assigned to x is a wrapper encapsulating a Circle object.

This has multiple advantages. When transforming one object to another, Manim can simply swap out the obj attribute of the wrapper (first to something like GeneralVMobject and finally to the destination object). Since the methods and attributes of the encapsulated object change correctly, this also allows certain methods (like getting the bounding box) to be specialized for certain types of Mobjects, rather than always using the expensive general VMobject method.

We could also have a material attribute with the wrapper. This determines whether the Mobject is rendered, for example, as a VMobject or primitive. Another use case is rendering a surface either as a checkerboard or a proper OpenGL mesh. The decoupling of renderer specific information from Mobjects also helps in supporting multiple renderers (although I think that in the long term, it's too complicated to support multiple renderers, unless we hire someone).

Things I haven't figured out yet

Where exactly should the methods for determining rendering data (in the case where we're rendering as a VMobject, where the bezier curves should be located) live?

We should be very clear on what style attributes we want VMobjects to have. Things like varying stroke width and custom color gradients are going to be very hard to represent without vertex information, and it's perhaps worth scrapping them from Manim entirely.

Disadvantages

Overriding the __new__ method is not exactly that clean either.

Users will have to type thing.obj.attribute rather than just thing.attribute to access what they want.

[MrDiver]

Variable attributes

We should be very clear on what style attributes we want VMobjects to have. Things like varying stroke width and custom color gradients are going to be very hard to represent without vertex information, and it’s perhaps worth scrapping them from Manim entirely.

I think it might actually not be as hard as you think. We can still keep the information as it is for the user. But the decision being made should determine which internal structure this data should take.

Quick example for opacity:

We pass in 0.4 as a value to the constructor. This will be set for the opacity attribute but also for fill_rgbas and stroke_rgbas where the little 0.4 is suddenly transformed in to an array with the length being the number of points of the mobject. This means we have two references to the same data. We can either inspect self.opacity or self.fill_rgbas[0,3] to get the opacity. But instead of this it would be a better way to ditch the rgba data completely and just save a list.

---

Suggestion saving as a list:

So a new approach would yield passing 0.4 as opacity and in the mobject [0.4] is saved as a universal value but no weird data is

This also would allow the variable values over the span of the mobject with [0.4,0.6] passed to opacity and also having the same data saved in self.opacity

Then in the renderer all the magic with aligning the data and interpolating could be done once it is actually needed.

Yes this would remove the functionality of manually messing with rgba's mid code but on the other hand ... who ... why ... what ... why would you want to do that?! If someone actually wants to do that they could just call a generate_color_data and be done with it.

Immutability

In general i am a big fan of immutability because it solves a lot of problems and also makes writing pure functions a little easier because youre forced to think about what the hell you're doing. On the other hand we are working with python and moving things around in python scope is one of the biggest bottlenecks we currently have. Making the objects immutable would mean that we have to copy even more things around which is just a mess in the current version of manim.

Decoupling the renderer.

Instead of us having all this magic interconnection between mobjects and the renderer there should be a clear distinction.

I really like the way raylib handles this

Here we can see all the math stuff is in rcore and the low level rendering happens in rlgl which is specifies a pseudo OpenGL 1.1 interface (not the best API to be honest BUT it's a clearly defined interface) therefore the different renderers can implement those functions how they want to and be switched out without the user noticing any change because it is clearly defined how the rendering should behave.

On top of that we have high level shapes defined in rshapes like circles and squares and all of that. Those are built on top of rlgl which is our communication layer to the render API. This means no matter which low level renderer you are using as long as it implements a few basic functions the objects will render the same way they did before without the need of seperate objects for every renderer.

As example a high-level function in raylib is DrawLineBezierCubic (to be clear a high-level function raylib is WAYYYY lower than what we need in manim because it is a general purpose rendering library not a math animation tool)
https://github.com/raysan5/raylib/blob/6f3b2a76614369f03721b9bac71727b6a5fcf393/src/rshapes.c#L211-L234

void DrawLineBezierCubic(Vector2 startPos, Vector2 endPos, Vector2 startControlPos, Vector2 endControlPos, float thick, Color color)
{
    const float step = 1.0f/BEZIER_LINE_DIVISIONS;


    Vector2 previous = startPos;
    Vector2 current = { 0 };
    float t = 0.0f;


    for (int i = 0; i <= BEZIER_LINE_DIVISIONS; i++)
    {
        t = step*i;
        float a = powf(1 - t, 3);
        float b = 3*powf(1 - t, 2)*t;
        float c = 3*(1-t)*powf(t, 2);
        float d = powf(t, 3);


        current.y = a*startPos.y + b*startControlPos.y + c*endControlPos.y + d*endPos.y;
        current.x = a*startPos.x + b*startControlPos.x + c*endControlPos.x + d*endPos.x;


        DrawLineEx(previous, current, thick, color);


        previous = current;
    }
}

But the point is that it is built upon a rlgl render function which interfaces with the actual render api.

Another example is a Circle
https://github.com/raysan5/raylib/blob/6f3b2a76614369f03721b9bac71727b6a5fcf393/src/rshapes.c#L441-L456

// Draw circle outline
void DrawCircleLines(int centerX, int centerY, float radius, Color color)
{
    rlCheckRenderBatchLimit(2*36);


    rlBegin(RL_LINES);
        rlColor4ub(color.r, color.g, color.b, color.a);


        // NOTE: Circle outline is drawn pixel by pixel every degree (0 to 360)
        for (int i = 0; i < 360; i += 10)
        {
            rlVertex2f(centerX + sinf(DEG2RAD*i)*radius, centerY + cosf(DEG2RAD*i)*radius);
            rlVertex2f(centerX + sinf(DEG2RAD*(i + 10))*radius, centerY + cosf(DEG2RAD*(i + 10))*radius);
        }
    rlEnd();
}

Here we can see the power of the render interface because we can define a circle on a somwhat high level.

For our usecase we would probably still mostly be bezier based and focus on optimizing those function but it would make it a lot easier to switch renderers and manage them if they would just implement a few functions.

This might look like
draw_bezier, draw_beziers, draw_fill, draw_fills, draw_mesh, draw_mesh_instanced, draw_point_cloud, draw_point_cloud_instanced

With functions like this we could just pass in the corresponding data like the points and the color as parameters and the rest will be handled in the blackbox which is maintained by the graphics magicians or something like that.

---

The wrapper

I am afraid that this will completely break any hope for autocompletion whatsoever because having a proxy object in between removes any capability of knowing what you're actually working with.
I would rather change the way animations behave because it is pretty confusing anyway which object is kept and which is not. But sadly i have no idea. It might be useful to attach an animation chain to a Mobject itself and then call play on that which returns the new mobject at the end.

circle = Circle()
square = Square()
animation = circle.animate.into(square)
new_shape = animation.play()
assert new_shape == square

But this is just a janky idea at the moment and not really thought out well. But it might spawn some other ideas on how to handle that mess ?