Does PMM skip scheduled tasks if a prior task takes a long time?

[gitttt54]

Hi,

PMM runs in a docker container with
PMM_TIME: 00:00,01:00,02:00,03:00,04:00,05:00,06:00,07:00,08:00,09:00,10:00,11:00,12:00,13:00,14:00,15:00,16:00,17:00,18:00,19:00,20:00,21:00,22:00,23:00

Say, PMM starts at 05:00 and the first task takes 5 hours to complete,

are the tasks for 06:00,07:00,08:00,09:00,10:00 being skipped/suspended/append to the scheduling?

To me it seems that they are just skipped and if the 5 hour task is completed, PMM continutes with the tasks 11:00,12:00,13:00...

Anybody can tell me whats the case?

[chazlarson]

Skipped. If a run takes longer than an hour there's no reason to tell PMM to run every hour.

[gitttt54]

Skipped

Thank you.

If a run takes longer than an hour there's no reason to tell PMM to run every hour.

I'm not sure, the way you see it is the only way which makes sense, but I'm glad to learn from your experience!

My PMM configuration is something like:

-run every hour and do some minor tasks, such as watchist updates for all users.

-schedule some longer taking tasks (such as 1-10 minutes, but sometimes >1-2 hours) to run in the morning hours, so the PMS will not that much penetrated.

-schedule really long taking tasks (>4-20 hours) to run every week or month, because they're not needed up2date every day.

So I configured the docker conrainer to run every hour to be free to schedule every of these tasks (and new ones too) at any time I think it will be pleasant for the PMS and the users.

Don't know, if there is a "better" way to do...

[chazlarson]

If you are scheduling it to run once an hour, but some of those runs take longer than an hour, I'm not sure what you expect to happen. If it stacked them up then PMM would probably never stop running. I can't see how that makes any sense or what problem you're trying to solve by wanting it to stack up runs.

If you are using internal schedules to control what happens at various times of the day, then probably this doesn't work even if they got queued.

For example, if you have things set internally to run hourly(6) intending them to be part of the 06:00 run, then if you queued the 06:00 run and started it at 10 after the 5-hour 0500 run completed, those things won't happen anyway since hourly(6) no longer applies.

If those "missed" runs were queued, then those "4-20 hours" runs would stack up dozens of deferred runs.

[chazlarson]

are the tasks for 06:00,07:00,08:00,09:00,10:00 being skipped/suspended/append to the scheduling?

What are these "tasks" and how are they scheduled for those times?

[gitttt54]

If you are scheduling it to run once an hour, but some of those runs take longer than an hour, I'm not sure what you expect to happen.

I'm not really expecting, I'm asking to understand how PMM works to adept that to my configs.

Actually, sometimes, when I take a look at a running log, I see "Starting 06:00 Run" but the actual time is, say 11 o'clock. Thats why I was confused and asked. There is always a good chance, I did'nt understand something quite well, so I can't tell if its me, the config, or PMM ;-D The wiki is great, but sometimes its too much or too less information.

Anyways, thank you for the explanations!
It makes sense to me and I guess, I have to re-think my scheduling...

are the tasks for 06:00,07:00,08:00,09:00,10:00 being skipped/suspended/append to the scheduling?

What are these "tasks" and how are they scheduled for those times?

At this time the short watchlist tasks should run.

libraries:
  Filme:
    library_name: Filme
    metadata_path:
    - file: config/mm-Filme-watchlist-only.yml   # takes <5 minutes
    - file: config/mm-Filme-admin.yml.           # takes 1-10 minutes, sometimes more
    - file: config/mm-Filme-flixpatrol.yml       # havent measured yet, I guess its <5 minutes each 
      schedule: hourly(4)
    - file: config/mm-Filme.yml                  # between 10 minutes and 1-2 hours
      schedule: hourly(5)
    - file: config/mm-Filme-dynamic_collections.yml # takes veeeeery looong, so I change that to weekly()
      schedule: hourly(5)

[chazlarson]

This:

    - file: config/mm-Filme-flixpatrol.yml       # havent measured yet, I guess its <5 minutes each 
      schedule: hourly(4)

Means: "If PMM happens to be running between 04:00 and 04:59 server time, process this metadata file". It has no direct relationship to this:

PMM_TIME: 04:00

and this:

    - file: config/mm-Filme-flixpatrol.yml       # havent measured yet, I guess its <5 minutes each 
      schedule: hourly(4)

Will not trigger a run at 4:00.

when I take a look at a running log, I see "Starting 06:00 Run" but the actual time is, say 11 o'clock.

"actual time" where? "Starting 06:00 Run" is referring to the time inside the container, which depends on the time on the server and the timezone of the container. If you're in Berlin and your PMM container's time zone is "America/Chicago", then those times aren't going to match what you see on your watch.

[chazlarson]

My suspicion is that these:

      schedule: hourly(4)

are evaluated when the script starts, not during the run, so if you have something like this:

    - file: config/mm-Filme-flixpatrol.yml
      schedule: hourly(6)

and the 5:00 run takes over an hour, this will never get run, since at 5:00 when the run starts, this schedule doesn't apply, and the script won't run at 6:00 when this schedule would apply.

[gitttt54]

"actual time" where?

I get the point. Thank you!

So, if the schedule: hourly(n) does not trigger a run at those hours (that was why I set PMM_TIME to every hour a day),

how does it work then?

With all the info, I crumbled out of the wiki and many posts here, it "seems" to work. (unless a task takes too long, as you explained)

I see, why hourly(n) and PMM_TIME are not in relationship and that's a very good point to keep in mind.

So what I did until now was telling PMM to awake each hour and run all those tasks which are not scheduled at a specified time,
and I was lucky, that the time in the docker container was about the same time, I wished the scheduled tasks to run.

I guess, PMM had some tolerance for the time to help out people like me....

At the moment, I have no clue, what will be a working approach for my intended scheduling, but with you kindly explanaitions, it'll much easier to re-think...

Thank you!

[chazlarson]

how does it work then?

It works exactly as I said above: "If PMM happens to be running [in this hour] server time, process this metadata file"

    - file: config/mm-Filme-flixpatrol.yml
      schedule: hourly(6)

That will get processed if PMM is running between 6:00 and 6:59 by its clock, however it gets started; manual run, PMM_TIME, whatever.

If you never run PMM, this metadata file will never get processed. If PMM only runs at 18:00, this metadata file will never get processed.

[chazlarson]

Personally, I would do this with specifically scheduled manual runs, each of which was configured to do exactly what I wanted to have happen at each time, taking care not to schedule overlapping runs.

[gitttt54]

What do you mean by "manual runs"? Start PMM like this by hand ?

# docker exec -it pmm_test python plex_meta_manager.py --run -c config/config.yml -ca -rt -is

[chazlarson]

Sure, though I typically use docker run -rm -ti ... to create and delete an ephemeral container for each run.

docker exec -it pmm_test ... assumes that the pmm_test container is up and running where docker run -rm ... creates one for the run and then tears it down afterward.

[gitttt54]

Good hint, thank you. I was not aware, that was possible. I'm new to docker (and of course PMM)...

But that means, I have to work manually to something, I don't want to waste more time as optimizing the configs and getting it to automaticly run. I'm just too old to be an admin and spend too much time at the console. I want to spend more time in fromt of the TV (or better, outside with friends'n'foes) than keeping in mind every week ..... wait a minute.... I can schedule those manual calls of docker also via cron... so what you're telling me is not to use the PMM "scheduler", but do everything via, say, cronjobs from the hosting system?

[chazlarson]

There are a variety of ways you can set up the schedules. If you want to continue the way you are, go ahead. I'm not trying to convince you not to spend time with your family. That's ridiculous. Do whatever you want.

I'm trying to give you the tools and understanding so that you can decide what you want to do with this tool.

However you do it, you are going to have to spend some time setting it up. Maybe you're happy with the slight unpredictability of the system you're currently running, maybe not. That's up to you, not me. Personally, I'd rather spend a little time setting it up to do exactly what I want and then not have to touch it again, but if that is too much "admin" work then don't do that.

Yes, I am suggesting skip the internal scheduler at least for the hourly stuff and use cron or similar to schedule specifically-configured runs at specific times so you are totally in control of what each run does and when it happens.

[gitttt54]

I'm trying to give you the tools and understanding so that you can decide what you want to do with this tool.

I surely understood your words like this...

Personally, I'd rather spend a little time setting it up to to exactly what I want and then not have to touch it again

...and thats the way I prefer it too...

and as we all know: you set it up and it runs, and when you come back something is different ;-D

Ok, enough for a day, thank you for your help! Very much appreceated!