SystemHousekeepingJob runs multiple times in direct succession when executions were missed

[tobiasge]

Deployment Type

Self-hosted

NetBox Version

v4.4.0

Python Version

3.12

Steps to Reproduce

1. Don't run the worker for a few days
2. Start worker

Expected Behavior

Job only runs once and is then re-scheduled in the normal interval

Observed Behavior

netbox-worker-1  | 13:18:07 default: handle(job=<Job: 9790c13b-319f-493a-984c-f72abb8ab29c>) (9790c13b-319f-493a-984c-f72abb8ab29c)
netbox-worker-1  | 13:18:08 Successfully completed handle(job=<Job: 9790c13b-319f-493a-984c-f72abb8ab29c>) job in 0:00:00.842352s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:08 default: Job OK (9790c13b-319f-493a-984c-f72abb8ab29c)
netbox-worker-1  | 13:18:08 Result is kept for 500 seconds
netbox-worker-1  | 13:18:08 default: handle(job=<Job: System Housekeeping>) (090122fb-5a5d-4df3-93f2-c4080632d3a7)
netbox-worker-1  | 13:18:08 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.505269s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:08 default: Job OK (090122fb-5a5d-4df3-93f2-c4080632d3a7)
netbox-worker-1  | 13:18:08 Result is kept for 500 seconds
netbox-worker-1  | 13:18:09 default: handle(job=<Job: System Housekeeping>) (6f1c996a-0e81-449a-9633-084d1ee016a7)
netbox-worker-1  | 13:18:09 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.513766s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:09 default: Job OK (6f1c996a-0e81-449a-9633-084d1ee016a7)
netbox-worker-1  | 13:18:09 Result is kept for 500 seconds
netbox-worker-1  | 13:18:10 default: handle(job=<Job: System Housekeeping>) (c40ea1a9-1c35-4b08-8a53-8a51478f45ca)
netbox-worker-1  | 13:18:10 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.502376s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:10 default: Job OK (c40ea1a9-1c35-4b08-8a53-8a51478f45ca)
netbox-worker-1  | 13:18:10 Result is kept for 500 seconds
netbox-worker-1  | 13:18:11 default: handle(job=<Job: System Housekeeping>) (b74fbbd7-8ebd-4b1e-8e70-71ab95b1966c)
netbox-worker-1  | 13:18:11 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.549482s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:11 default: Job OK (b74fbbd7-8ebd-4b1e-8e70-71ab95b1966c)
netbox-worker-1  | 13:18:11 Result is kept for 500 seconds
netbox-worker-1  | 13:18:12 default: handle(job=<Job: System Housekeeping>) (556727ed-c157-497c-aa19-0289075e16b6)
netbox-worker-1  | 13:18:12 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.497449s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:12 default: Job OK (556727ed-c157-497c-aa19-0289075e16b6)
netbox-worker-1  | 13:18:12 Result is kept for 500 seconds
netbox-worker-1  | 13:18:13 default: handle(job=<Job: System Housekeeping>) (a486417e-6261-46fa-831a-d5657697f150)
netbox-worker-1  | 13:18:13 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.532598s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:13 default: Job OK (a486417e-6261-46fa-831a-d5657697f150)
netbox-worker-1  | 13:18:13 Result is kept for 500 seconds
netbox-worker-1  | 13:18:14 default: handle(job=<Job: System Housekeeping>) (3da8476f-e39b-4314-b70c-e150b8ebd198)
netbox-worker-1  | 13:18:14 Successfully completed handle(job=<Job: System Housekeeping>) job in 0:00:00.508011s on worker e56a179060f94f61a3fa50b4e9653623
netbox-worker-1  | 13:18:14 default: Job OK (3da8476f-e39b-4314-b70c-e150b8ebd198)
netbox-worker-1  | 13:18:14 Result is kept for 500 seconds

[tobiasge]

I think the problem is, that in netbox/netbox/jobs.py the time in new_scheduled_time (line 96) is not checked if it is already in the past.