Hi
I need to copy a lot of folders that only has a done.dat file in it. The basic structure is

C:\Folder\1000001\done.dat
C:\Folder\1000001\1000001\image2.jpg
C:\Folder\1000001\1000001\image3.jpg
C:\Folder\1000001\1000001\image4.jpg
C:\Folder\2000001\done.dat
C:\Folder\2000001\1000001\image2.jpg
C:\Folder\2000001\1000001\image3.jpg
C:\Folder\2000001\1000001\image4.jpg
C:\Folder\3000001\1000001\image2.jpg
C:\Folder\3000001\1000001\image3.jpg
C:\Folder\3000001\1000001\image4.jpg

You'll see that C:\Folder\3000001 does NOT have a done.dat file - I don't want to copy folder C:\Folder\3000001

Currently I'm doing this by
1. listing the folders
2. Looping through the folder list from the first task
2a. Copying the folder but using a condition with the file C:\folder\{LOOP(CurrentValueX)}\done.dat so using the loop to complete the path of the done.dat file
3. Log the files and folders that were copied

With 1000s' of folders this takes a very long time (e.g. a manual windows copy of 20Gb took 55 minutes after I cancelled the job that was running for more than 4 hours on the same folders.
I've

I'm pretty sure that if I had just a normal copy job,

You'll be better off having VC call a third party file copying program that can handle the rules for copying. Check out teracopy or fastcopy or something like those. Find one with a command-line interface. See here for a list: File copy utilities 

In the meantime, you can shorten the list in your step one by listing only folders that have a done.dat in them.

Hi
Thanks for the reply, but how would I go about shortening the listing? I've tried with conditions but can't seem to get that to work.

Thanks
Neil

List full paths to done.dat files instead, then your folder paths are the part of the string before "\done.dat". So if {LOOP(CurrentValueX)} contains
"C:\Folder\2000001\done.dat" then to get the folder use {PATH(GetDirectoryName|{LOOP(CurrentValueX)})} which will return ""C:\Folder\2000001".

That's the path you would want to pass to a file copying tool though instead of having windows do it. That is, unless the tool could gather the paths instead in the first place. It would still be faster I'm sure.

robocopy would work great for this

Here is a proper reply

Add a task to your job "Process - Execute"
Command: c:\windows\system32\robocopy.exe
Arguments: C:\FolderSource C:\FolderDestination Done.dat /S /Z /W:5 /R:5 /MT:4 /FFT /V /FP /TS /TEE /log:C:\FolderDestination\Log.txt
Working Directory: c:\windows\system32\

That will pull over any folder that contains 'Done.dat' AND the 'Done.dat' files as well. The files will be in their respective subdirectories

It will make a log in the destination you choose
If you want the out put of the log, create another task to Read that file and then email it yourself or even just attach the log to the email.

There is NO Std out when running this in VC using the arguments above (even though I tried to include that in the window using /TEE)
UNLESS
you add the tag /UNICODE at the end. Then you will get Std Out in VC but you get a weird character at the top for some reason.
(the arguments now look like this:
C:\FolderSource C:\FolderDestination Done.dat /S /Z /W:5 /R:5 /MT:4 /FFT /V /FP /TS /UNICODE /TEE /log:C:\FolderDestination\Log.txt)

This method does NOT work in Server 2008 but it did in 2019 (I assume 2012 and above)
I used VC v9.1.5