Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Most images don't load using 0.13.8 #3425

Open
matrixbot opened this issue Nov 2, 2024 · 7 comments
Open

Most images don't load using 0.13.8 #3425

matrixbot opened this issue Nov 2, 2024 · 7 comments

Comments

@matrixbot
Copy link
Collaborator

This issue was originally created by @makidoll at matrix-org/dendrite#3425.

Background information

  • Dendrite version or git SHA: 0.13.8+763c79f
  • SQLite3 or Postgres?: Postgres (15-alpine)
  • Running in Docker?: Yes
  • go version: sh: go: not found
  • Client used (if applicable): Element on Web, Element X, Cinny

Description

  • What is the problem: Requested images return no data, but status 200 OK
  • Who is affected: Single user
  • How is this bug manifesting: Images rarely or never load in when I'm using the above clients
  • When did this first appear: Since the 0.13.8 update

I've tried resetting all my clients too. Example below returns Content-Length: 144 with Content-Type: application/json without data. Couldn't find any relevant logs either.

Steps to reproduce

  • Setup Dendrite monolith using Docker
  • Use any of the above clients

See images below

Screenshot from 2024-09-16 17-44-30 blurred

Screenshot from 2024-09-16 17-37-00 blurred

Screenshot from 2024-09-16 17-38-27 blurred

Screenshot from 2024-09-16 17-38-35
@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @makidoll at matrix-org/dendrite#3425 (comment).

Seems like images might be coming through now on multiple devices. Maybe it's related to some sort of cache in Dendrite

@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @makidoll at matrix-org/dendrite#3425 (comment).

This is still happening. Every time I restart any client, all images are lost but over time they start to appear. Very strange

@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @makidoll at matrix-org/dendrite#3425 (comment).

matrix-org/dendrite#3424 might have helped but still not fully fixed

@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @makidoll at matrix-org/dendrite#3425 (comment).

Seems like the fix is to just refresh element a few times and the image requests slowly turn from content-type json to jpeg

@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @S7evinK at matrix-org/dendrite#3425 (comment).

https://github.com/matrix-org/dendrite/blob/6cd1285ca0276bebd407110c37031bc4622f0b79/mediaapi/routing/routing.go#L204
Is the only place where we actively set the Content-Type to application/json. I couldn't find a reason why we would return HTTP 200 but still have application/json set, yet.

@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @pat-s at matrix-org/dendrite#3425 (comment).

Since some weeks I also have this issue. Not exactly sure when it started, but reverting to 0.13.7 didn't help.

I see the following in the logs:

 time="2024-10-06T11:04:39.793349630Z" level=error msg="failed to open file" MediaID=VCJpgjatIIsIuTzgLKwjBiYx Origin=obermui.de error="os.Open: open /data/media_store/-/Q/YSg8KmsDhugLEHnnAShX7wSr2kgmuS │
│ 8BYCE7qJLWo/file: no such file or directory" req.id=1e7gVaofwHzQ req.method=OPTIONS req.path=/_matrix/media/v3/thumbnail/obermui.de/VCJpgjatIIsIuTzgLKwjBiYx

seems like dendrite is failing to query images from other servers? Did something change for synapse-based installations so that the path is now different?

For the above, the queried path doesn't seem to exist on the server:

https://obermui.de/_matrix/media/v3/thumbnail/obermui.de/VCJpgjatIIsIuTzgLKwjBiYx

EDIT: In my case it was likely a different issue and solved by matrix-org/dendrite#2954 (comment).

@matrixbot
Copy link
Collaborator Author

This comment was originally posted by @sleroq at matrix-org/dendrite#3425 (comment).

Seems like the fix is to just refresh element a few times and the image requests slowly turn from content-type json to jpeg

Seems like there is a limit on the number of thumbnails server able to generate at the same time. For me it's 5. When you refresh, browser won't download cached ones and eventually you load everything.

On the server it looks like this:

Details 

monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | time="2024-10-17T08:08:23.240293449Z" level=info msg="Signalling other goroutines waiting for this goroutine to generate the thumbnail." Height=480 MediaID=a1039ef6a83a883e5579c65a2503482b6e10b878ccca5c189260a3fb76c60fbf Origin=sleroq.link ResizeMethod=scale Width=640 req.id=hobHFli7AMuA req.method=GET req.path=/_matrix/media/v3/thumbnail/sleroq.link/a1039ef6a83a883e5579c65a2503482b6e10b878ccca5c189260a3fb76c60fbf
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | time="2024-10-17T08:08:23.255353988Z" level=info msg="Signalling other goroutines waiting for this goroutine to generate the thumbnail." Height=480 MediaID=da2498ff0cd0c84686161db92cdbba24880e8a638a157b129b1b4ba5ddf4c9de Origin=sleroq.link ResizeMethod=scale Width=640 req.id=o7taqB4UG4ve req.method=GET req.path=/_matrix/media/v3/thumbnail/sleroq.link/da2498ff0cd0c84686161db92cdbba24880e8a638a157b129b1b4ba5ddf4c9de
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)
monolith-1  | 2024/10/17 08:08:23 http: superfluous response.WriteHeader call from github.com/matrix-org/dendrite/mediaapi/routing.makeDownloadAPI.func1 (routing.go:214)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@matrixbot