How to avoid "moving" items in collections?

[gitttt54]

Hi all,

coming from here.
I use kometa via docker in a unraid system.

I now have this config for trakt-liked-lists and its works ok on small liked lists (about 10-50 items)

templates:
  template4TraktLikedLists:
    trakt_list_details: <<value>>
    sort_title: +++06_4_<<collection_name>>
    collection_order: custom

dynamic_collections:
  Trakt Liked Lists:          # This name is the mapping name
    template: template4TraktLikedLists
    type: trakt_liked_lists
    data: me

unfortunately, when a liked list is very long, say over 500-1000 items, my plex server tends to freeze and I got hundrets of those messages in my plex logs:

Waited over 10 seconds for a busy database; giving up.

or even

Saving activity history aborted with DB exception: std::exception

As soon as I kill the kometa docker, the plex server works fine again.

The issue seems to be with trillions of those entries in the kometa log, which seems to overcharge the plex server:

| Moving Gefährliche Schönheit - Die Kurtisane von Venedig (1998) after Im Zwielicht (1998)          |
| Moving Der Gejagte (1998) after Gefährliche Schönheit - Die Kurtisane von Venedig (1998)           |
| Moving Der Pferdeflüsterer (1998) after Der Gejagte (1998)                                         |
| Moving Gia - Der Preis der Schönheit (1998) after Der Pferdeflüsterer (1998)                       |
| Moving Buffalo '66 (1998) after Gia - Der Preis der Schönheit (1998)                               |
| Moving Oscar und Lucinda (1997) after Buffalo '66 (1998)                                           |
| Moving Der Boxer (1997) after Oscar und Lucinda (1997)                                             |
| Moving Kundun (1998) after Der Boxer (1997)                                                        |
| Moving Wag the Dog - Wenn der Schwanz mit dem Hund wedelt (1998) after Kundun (1998)               |
| Moving Jackie Brown (1997) after Wag the Dog - Wenn der Schwanz mit dem Hund wedelt (1998)         |
| Moving Besser geht's nicht (1997) after Jackie Brown (1997)                                        |
| Moving Apostel! (1998) after Besser geht's nicht (1997)                                            |
| Moving Harry außer sich (1998) after Apostel! (1998)                                               |
| Moving Amistad - Das Sklavenschiff (1997) after Harry außer sich (1998)                            |
| Moving Good Will Hunting: Der gute Will Hunting (1997) after Amistad - Das Sklavenschiff (1997)    |
| Moving Die unsichtbare Falle (1998) after Good Will Hunting: Der gute Will Hunting (1997)          |

So, how do I solve this issue?

I want the liked lists sorted as they come from trakt.tv, so I guess its "custom".

but why does kometa sort those lists then (whilst moving each item) ? and how can I make those process lightwighter?

Thank you very much for any ideas/hints!

[chazlarson]

They're being moved because with collection_order: custom you are telling Kometa to put them into a particular arbitrary order.

If you want the items in the collection in a particular arbitrary order, then Kometa has to move them into that order.

[gitttt54]

This would not work with the trakt-liked lists I was talking about, I guess? Also, it seems to me like a lot manual work for over 50 lists.

[gitttt54]

I just ran some tests, if it's useful.

collection_order: release ## NO moving occurs

Ok, I was not aware that some sortings options do not sort as their name suggests.

I need to test that whith those lists, where the trakt-user-sort is "important".
https://trakt.tv/users/bob-frapples/lists/remakes
https://trakt.tv/users/majeed_pk/lists/remake-and-the-original-movies

Thank you.

[chazlarson]

Ok, I was not aware that some sortings options do not sort as their name suggests.

That's not what that means. It means that the step you mention when Kometa moves things around does not occur.

collection_order: release will display the collection in order by release date, but Kometa does not have to actively move them around into that order.

[gitttt54]

I just ran some tests, if it's useful.

collection_order: release ## NO moving occurs

I need to test that whith those lists, where the trakt-user-sort is "important". https://trakt.tv/users/bob-frapples/lists/remakes https://trakt.tv/users/majeed_pk/lists/remake-and-the-original-movies

I conducted the tests, and the results were both positive and, lets say, disadvantageous.

First of all, Kometa no longer sorted/moved the lists, which was already a relief for the database. Thank you for pointing that out!

However, unfortunately, the test lists are now sorted incorrectly and do not match the Tracks User's desired order. Plex sorts the list by, well, release :-)

So, we’re getting closer to the solution. :-)

Is it possible to specify a different sorting criterion for some of the lists?

Maybe something similar to "title_override", where the user can easlily "pick" some lists an rename them?

I imagine that, via a similar option, the user could simply pick a few lists and have them sorted differently.

Thank you!

[chazlarson]

First of all, Kometa no longer sorted/moved the lists, which was already a relief for the database. Thank you for pointing that out!

This was your stated goal, avoiding the moving of items in these lists.

However, unfortunately, the test lists are now sorted incorrectly and do not match the Tracks User's desired order. Plex sorts the list by, well, release :-)

You can pick one or the other. If it is important that the list be in the same order as it is provided by the source, then Kometa has to move them into that order to ensure that's the case. There's is no way for Kometa [as currently designed] to put the things in a spacific arbitrary order without moving them into that order. If you don't want the moving to happen, you can't have the special arbitrary order.

Is it possible to specify a different sorting criterion for some of the lists?

Not with the dynamic collections you are using. If you created individual collections for each list, you could control the sort order for each one.

You could file a feature request, but this is such a niche use case that it seems unlikely it will be added.

The stated use case of many lists, each containing thousands of items resulting in trillions of item moves to order collections is insanely niche and I suspect that whatever could be changed in Kometa it will still perform badly in this edge case.

Even if it were adopted, that still doesn't avoid moving thousands of items around.

A better solution is probably to schedule these collections to get processed less frequently.