Hi,
The service stopped again, sometime around 8:33pm last night. When I execute jobs manually, it will run the job, and will calculate the next run time, but when it reaches that run time it doesn't automatically execute the job. I went digging through some log files and found some stuff, but I have no idea if it'll be helpful or not.
This is from the
log_server20181029.txt file. For some reason, it looks like the time on the server skipped a second, going from 8:33:11 to 8:33:13. It recalculated all the next run times ok, but it didn't run them at these new times.
10/29/2018 8:33:10 PM Info Task completed (Success)->'Set PROC_QTY' (8485755)
10/29/2018 8:33:11 PM Info Task started(Set Job Variable): Set CATCH_WT_QTY (8485758)
10/29/2018 8:33:13 PM Debug Current time: 10/29/2018 8:33:13 PM
10/29/2018 8:33:13 PM Debug Last time: 10/29/2018 8:33:11 PM
10/29/2018 8:33:13 PM Info Running missed Jobs between 10/29/2018 8:33:11 PM and 10/29/2018 8:33:12 PM (RunMissedInterval)
10/29/2018 8:33:13 PM Debug Triggers will be recalculated due to time change in the system. (10/29/2018 8:33:11 PM - 10/29/2018 8:33:13 PM)
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Backup Settings' is: 10/29/2018 11:00:00 PM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Delete Old Server Log Files' is: 10/29/2018 11:10:00 PM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Tasks in Stat Code 88' is: 10/29/2018 8:35:00 PM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Update Carton Types' is: 10/30/2018 12:55:00 PM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Update Skid Traveller Database' is: 10/30/2018 5:30:00 AM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Users Inactive for Over 90 Days' is: 1/1/2019 9:00:00 AM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Update E-Commerce Ship Days' is: 10/30/2018 6:00:00 AM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Table History Accumulator' is: 10/29/2018 8:35:00 PM
10/29/2018 8:33:13 PM Debug Next execution (7) for job 'Run Skid Traveller Reports' is: 11/4/2018 9:00:00 AM
This morning, I manually ran some jobs to get it to calculate the next run time of 8:10am. I then turned on extended debug logging and waited until 8:10. This is a piece of today's
log_20181030.txt file:
10/30/2018 8:10:14 AM Info JobAPI->Save->Saving Jobs (64) - in shut down: False
10/30/2018 8:10:15 AM Debug GetFinishedTriggerResultObjects->Done: 5, not done: 1, time not right: 0, finished: 5, total: 6
10/30/2018 8:10:15 AM Debug RemoveTriggerResultObjects->Removed 5, left: 1
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Start
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: If STAT_15 = Yes, continue with update IsNotification: False, minutes: 9612
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: Update Message Stat Code to 95 (error) if XML Status = FAIL IsNotification: False, minutes: 10565
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Montreal PTS Hub - 002 - Write Detail Message, Task: Set XML Detail IsNotification: False, minutes: 9856
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Montreal PTS Hub - 001 - Main, Task: Set LAST_KNOWN_LOCN IsNotification: False, minutes: 4167
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: Log File Not Found if FILE_EXISTS = False (skip to end of job) IsNotification: False, minutes: 11280
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: Update Message Stat Code to 95 (error) if XML Status = FAIL IsNotification: False, minutes: 11048
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Montreal PTS Hub - 001 - Main, Task: Set USER_DEF_VAL_1 IsNotification: False, minutes: 4167
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: If STAT_15 = Yes, continue with update IsNotification: False, minutes: 8229
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: Log File Not Found if FILE_EXISTS = False (skip to end of job) IsNotification: False, minutes: 10698
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: If STAT_15 = Yes, continue with update IsNotification: False, minutes: 7156
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: If STAT_15 = Yes, continue with update IsNotification: False, minutes: 11048
10/30/2018 8:10:15 AM Debug RemoveTaskProcesses->Long running Job: Calgary PTS Hub, Task: If STAT_15 = Yes, continue with update IsNotification: False, minutes: 10565
There are many lines that say
RemoveTaskProcesses->Long running job. The first one above is on line 3701 of the log file; the last is on line 216828, and every line in between is related to removing a task process. So it looks like it's trying to remove a long running job, but I don't understand that because there actually aren't any jobs running at all, long or otherwise.
I tried turning the server off and back on using the button in the top-left corner of the client. When I turned it back on, it did recalculate all the next run times, but again, it didn't automatically execute the jobs when they reached those times. The only way I've found to completely fix this is to reboot the machine, but that's not a viable solution if I have to do that every couple of weeks.