I thought I was going crazy! One of my jobs would disappear. I copied it back from another server. I checked a few days later and it was missing again! I didn't realize at the time it was related to someone else's job that was also going missing after I copied my job back. long story short, a copied job from one server has the same jobid when copied to another server, thus causing the job to get overwritten. I found a feature request post from 7 years ago by someone that had this same issue, asking that the jobid not get copied so the copied job would get a new one thus preventing this from happening. This has caused a production issue as the job that went missing is a critical job.

Since copying between servers could be problematic then, what is the preferred way to do this? Note the overwriting also happened when restoring from an export (as expected). I would sure hate to have to recreate jobs as some are fairly complex and the ability to copy is a huge benefit.

Am I doing something wrong? Is it best practice to check to see if the jobid exists on the target server first by checking manually somehow? If so, how?
We need a way to ensure jobIDs are unique across servers for multiple-server installations.

Thank you,
Gary

Update 1: I exported just this job thinking maybe I could edit the ID before importing into the new server but in the XML the jobID is in there in 17 places. That may break more than it fixes anyway. Just adding this info as I work through it.

Update 2: in post 15 of this thread: https://www.visualcron.c....aspx?g=posts&t=7865  Support states if you copy a job it gets a new ID. I just copied a job to 2 servers and it kept the same ID.

Update 3: To see if a jobID already exists on the target server, export the jobs on the target server, then extract the jobs.xml file from the .zip that gets created and search for the jobid.

Maybe after checking and finding out the job ID already exists on the other server, try cloning the job on the same server - that will definitely get it a new ID. Then copy THAT one to the new server.

Rebeccah

Great idea! After checking that the cloned ID doesn't already exist as well, but I would expect this to reduce the risk. I'm afraid I no longer trust moving jobs from one server to another. I have since discovered the same issue with tasks. Since the servers do not seem to enforce unique IDs per server, one can't be guaranteed a jobID created on one server does not already exist on another, no matter how it was created.

After working with support we think this can happen when a job is cloned on one server, copied to another server, then that job gets cloned or copied and moved back (or some scenario like that). At any rate, I feel how a job or task got created should not matter, the tool should not silently overwrite an existing job when moving.

I have put in a feature request to check the jobID/taskID when moving moving jobs/tasks and if it already exists, warn the user, show the job/task name and prompt to overwrite the existing job/task or copy with a new jobID/taskID.

I clone jobs on the same server a lot, as I have a lot of very similar jobs and use cloning to create a template for the new job. I have version 8.2.4 and haven't found task IDs getting overwritten - but I DO find that references to the tasks or to variables based on task IDs (such as task output) don't get their IDs updated or get left unspecified. So, when I'm cloning a job to create a new one, I have to go through each task and ensure that all of its references are correct. It still is better in most cases than creating from scratch, as I ensure that I don't forget anything.

Also, when pushing to a new server I have followed a previous suggestion by another user to use the same IDs on all of the servers. I never create new jobs, tasks, or other objects directly on my VAL or PROD server, only on the DEV server and then I copy it to the new server along with all of its dependencies, when I am ready to promote it. That is the only way to ensure that all of the dependency references stay intact from one server to the next.

I do like the idea of having VisualCron notify of an ID crash before copying.


I currently do all of my promotions by exporting settings and merging the settings files, because I never know what I'm going to get when I do it through the GUI -- but I'm not sure if I recommend that to others. It's a lot of work, because the exported settings files contain a lot of unimportant (for merging) information, as the runtime information such as last and next run times, output values, etc, are mixed together with the job, task, notification, condition, etc. object definition information. I use Beyond Compare 4 to do the diff and merging, and I have a custom rule in Beyond Compare that ignores result values, stdout and stderr values, most scheduling information, and a few other noncritical things - and calls an XSLT stylesheet transformation that I have created, which sorts the major elements by ID and sorts the most critical pieces of information to the top within the major elements. Beyond Compare 4 includes an option to use a Patience diff, which does a far better job of aligning these settings files than any other alignment strategy I have seen. Combining all of these tools together, I am able to do my diff and merge, check the settings files into my revision control, and then import the merged settings into the server I'm promoting to, overwriting everything that is there. It takes me a half a day to get source and target server settings exported and checked into revision control as snapshots (documenting any changes in my commit message), do my merge, check in the merged settings (again with commit message documenting what is being promoted), import the merged settings to the target server, and confirm that I didn't break anything in the process, so I generally wait until several jobs and their dependencies are ready for promotion and do them all at once. Hopefully, I don't find any changes in the target server that need to be ported back to the source server, or the whole day is pretty much shot.

Without Beyond Compare, it would be impossible to do things this way, and with BC4 and the Patience Diff, it takes about half as long to do the diff and merge part and is far less error prone than with BC3.

Rebeccah

Thank you for taking the time to reply and share your update process. I want to implement your promotion process here and tighten up how we do things.