Background image for tracing

[ollimeier]

Playground draft PR for discussion.

[justvanrossum]

Quick feedback:

• instead of specifying all separate transformation parameters, use a fontTools.misc.transform.Transform object, which should serialize as a list of six items. We could also use DecomposedTransformation, like we do in components. I'm not 100% which would be better in this context
• Since I doubt the image will ever be part of interpolation, putting it on the StaticGlyph object may not be the best thing to do. Potentially it could be a new attr on the Layer object instead. But also here: I'm not 100% confident either way.

[ollimeier]

@justvanrossum I tried a lot, but sadly without success, yet. The main issue I have, is: how can we display an image stored locally. It's not allowed to access files locally from an http request. Do you have an idea how to solve this issue? Thanks a lot in advance.

[justvanrossum]

I didn’t realize you’d work on that today. That approach cannot work, as you’ve found out. We need to serve images as part of the font data.
I’m not yet sure what that’s concretely going to look like.

[ollimeier]

I didn’t realize you’d work on that today. That approach cannot work, as you’ve found out. We need to serve images as part of the font data. I’m not yet sure what that’s concretely going to look like.

I am not sure if this is something useful or not, but I figured out, that we can load an image via http://localhost:8000:
Please see my latest commit and the following screenshot:

The image is misplaced (it should actually be at the position of the rectangle), but it's there – yeah :)
No base64 used.

[ollimeier]

Correct position of the image:

Screen.Recording.2024-10-11.at.18.25.16.mp4

[justvanrossum]

The Image type should get an attribute named transformation of type DecomposedTransform.

Loading of the image should be done outside of the vis draw func, although loading could perhaps be triggered by draw (but only if it's cheap), then we should trigger a canvas redraw once the image has been loaded. It is vitally important that the draw func does not do any loading itself. The effect will be that if you put a glyph in edit mode, there may be a delay before the background image appears.

While it may be desirable to load images via regular http(s) instead of the websocket, this raises questions about how this data should then participate in undo and external changes. To do this correctly may add a lot of complexity that can perhaps be avoided if we allow image data to travel as base64 inside JSON messages. If we go that route, we may have to enforce a maximum image size.

I have some ideas that I'm trying to formulate, about potentially avoiding base64 encoding the data, but I'm not there yet.

Perhaps we can start with a simplistic approach, without worrying too much about efficiency.

(I'm considering to make the infrastructure general for any binary data, not limited to image data.)

ReadableFontBackend:

• getBinaryData(name: str) -> bytes

WritableFontBackend:

• putBinaryData(name: str, imageData: bytes) -> None
• deleteBinaryData(name: str) -> None

The FontHandler class will be responsible for converting to/from base64 (and can play a role in a different scheme, once we come up with one).

The Font class should then probably get a binaryData attribute, of type dict[str, bytes], but this is where things get complicated. I need to look into how FontController should deal with this data.