I posted a topic on how to actually use Variable with the file trigger to create a process that would take files from Share1, move them to Share2, and create an equally named .TRG file in Share2
Here's the topic if you want to see my solution:
http://www.visualcron.co....aspx?g=posts&t=2727 

This seems to work, a few files at a time. However, if I am watching for .TXT files in Share1, and I copy 15 .TXT files at once into Share1, Not all of the files get moved and/or have their TRG file created on Share2. Its as if the File Trigger process missed seeing that file or files were 'created'.

I copied 15 files, one file got missed. It'll never get picked up now.

I copied the same 15 files again to Share1, 3 files got missed.

I copied the same 15 files again to Share1, all were picked up.

Its random. And by the way, i'm talking about 15 blank text files with no data in them, so it wouldn't be any sort of I/O constraint while testing.

I'm testing this on the latest build of 6.1.4 on our Q server using local storage (no network involvement)

File names are as follows (quick easy copy/paste creation of 15 files):
New Text Document - Copy.txt
New Text Document - Copy (2).txt
New Text Document - Copy (3).txt
New Text Document - Copy (4).txt
New Text Document - Copy (5).txt
New Text Document - Copy (6).txt
New Text Document - Copy (7).txt
New Text Document - Copy (8).txt
New Text Document - Copy (9).txt
New Text Document - Copy (10).txt
New Text Document - Copy (11).txt
New Text Document - Copy (12).txt
New Text Document - Copy (13).txt
New Text Document - Copy (14).txt
New Text Document - Copy (15).txt


Thoughts?

Brian

Do you have the "Put Job in queue" checked? That one is needed.

Also, let us know which Variables you are using. Maybe screenshots will be best to verify.

---

Do you have the "Put Job in queue" checked? That one is needed.

Also, let us know which Variables you are using. Maybe screenshots will be best to verify.

Put Job in Queue is definately checked.

Job: "File Trigger - Move Files and create TRG"
Task1: Move Files (Does a copy_files task using variables), deleting source when done)
Task2: Create TRG File (Does a write_file task using a REGEX replace function and variable for the name)

So, if I copy 15 files from my 'HoldingArea' folder into the 'Source Folder (lower left of my screenshot), it will move the files to the 'Destination' folder one at a time... and also create a TRG file. All of this is local, no network drives involved.

Now, to get the screenshots, I did have to copy the 15 files into 'Source' 3 times before I got it to fail, but the fact is, it still fails.

Try 1: I copied the 15 files to 'Source' and all 15 files were moved and TRG files created in 'Destination'. I then deleted all files from 'Destination so I could try again.

Try2: Same as above. Copied all 15 files to 'Source' and it did as it should, I deleted the files from 'Destination' so I could try a 3rd time.

Try3: Copied the same 15 files to 'Source'. File4.txt and File5.txt got missed and remain in 'Source'. The destination has 26 files (13 .txt and 13 .TRG), and it should be 30 total if it had worked right.

Brian

More attachments.

Before Copying and After copying (3rd try when it failed).

Bump 🙂

Hi,

please let me know if this version works better:

http://www.visualcron.co....aspx?g=posts&t=2743 

If not, please turn on extended debugging and send the log_serverDATE.txt after copying files to source folder.

---

Hi,

please let me know if this version works better:

http://www.visualcron.co....aspx?g=posts&t=2743 

If not, please turn on extended debugging and send the log_serverDATE.txt after copying files to source folder.

I will try this later today. However, based on your request, am I to assume that you cannot reproduce the problem? If not, I'll build an entirely new server and leave Antivirus, etc... off the machine and work towards our standard to see what, if anything, is causing the issue.

Environment:
VMware Virtual Machine
OS: Windows 2008 Standard R2 SP1, fully patched.
AV: Symantec Endpoint Protection v12.x
Hardware: VMware ESX 4.1 (latest patches) on HP BL685c G7 Blades (4 x 12 Core AMD Opteron 7174) with 256gb of Ram. (Shared with only 7 VM's at the time)


Brian

Hi,

please let me know if this version works better:

http://www.visualcron.co....aspx?g=posts&t=2743 

If not, please turn on extended debugging and send the log_serverDATE.txt after copying files to source folder.

I will try this later today. However, based on your request, am I to assume that you cannot reproduce the problem? If not, I'll build an entirely new server and leave Antivirus, etc... off the machine and work towards our standard to see what, if anything, is causing the issue.

Brian

We was able to reproduce it one time. We did a minor change and have not seen the issue since then. But it is a timing issue which can appear different in different environments. So, it would be interesting if you could test the new version.

---

We was able to reproduce it one time. We did a minor change and have not seen the issue since then. But it is a timing issue which can appear different in different environments. So, it would be interesting if you could test the new version.

Ok, i'll update our Q server to 6.1.5 and get back to you in a few minutes.

Brian

In my testing, I think what may be happening is that its assuming that some of the files are the same, since they technically are, and i'm copying them in again as soon as the folder is empty.

So when I copy those 15 files into 'Source' and it moves all of them, and then I immediately copy the same 15 files into 'Source' again, it is seeing it as a duplice file event, based on this piece in the debug log:

This was immediately at the start of the 2nd time I copied the files:

10/16/2012 8:27:49 AM Debug Ignoring duplicate file event from system, file: C:\Temp\Source\File4.txt

File4.txt is the file that got left in 'Source'. So honestly, I think there really isn't any issue... as who is going to be doing what i'm doing in reality. No one typically (nor should they) copies the same file name (literally exact same file, data 'n all) over 'n over again in quick succession.

I've attached the log. I stopped VC, cleared the log, started VC and then started my 2 sets of copying files, then stopped the service so the log was as short as it could be. (This is a temporary server I built, running 2008 R2 and a trial version nof 6.1.5)

Brian

Moral of this story, I need to remember to turn on the extended debugging and look at the logs. I honestly had never done that before and am quite happy with the level of granularity it provides for troubleshooting.

Brian