I have a simular situation.
I have to download the newest file from a (s)ftp site, but I don't know what the newest file is.
What I did was:
- Download every file found in one dir not overwriting existing files;
- Have a text file containing the allready processed filenames;
- Use a script (I used perl for this) to compare the downloaded files with this text file;
- Copy the missing files to a different folder and process them;
- Last step is to add the just processed filenames in the text file, so next downloads will not use these files anymore.
This works for me for several jobs.
I has one negative aspect: you have a ftp mirror dir on your site, so I hope the files are not soo big.
What you can do is to empty the ftp mirror folder (every time/sometimes), but your download task will run much longer.
Regards
Erik
Edited by user
2009-08-20T07:54:12Z
|
Reason: Not specified
Uses Visualcron since 2006.