I need to have my job pass through a folder content validation before proceeding and I would like to not take up 5 tasks with 5 condition sets to do this. The validation needs to do the following:

The folder should have exactly 5 files contained in it, or the job should stop
The five file names have to each start with a specific string (different for each file) but will each have a date appended on the name, so have be identifiable as
name1*.xml
name2*.xml
name3*.xml
name4*.xml
name5*.xml
If any of the 5 files does not exist, the job should stop.

I thought I had a pretty good solution using one task with a condition set with 6 conditions one that tests for the file count (based on a previous file list step) and 5 more that test for the file name strings I'm looking for. Then I ran into the condition action choice "On Match Any" which broke everything.

The problem is that any available choice for "On Match Any" is a wrong choice. I really just don't want to test for "On Match Any" because I only want to determine the action if all or none of the conditions are met.

Since I can't associate multiple condition sets to a single task, and the "On Match Any" is preventing me from doing this in a single condition set, I can't think of anything to do other than create 6 tasks with 6 condition sets, which is not making me happy..

I am admittedly not a super deep user of this product, so I'm hoping that someone is seeing something to do here that I'm missing?

Thanks

I do not think Conditions is the best for this. I suggest using the List files Task with a regular expression so you match all your names. Then you can use the Count Variable from the result to decide in the Flow of that Task if you should move to next Task or not.

It is a bit complex and I cannot help you with regex for that as I am not an expert in that.

---

Done something similar recently and used File Trigger Dependencies. In your case, if you know the 5 filenames, set 5 seperate triggers and make them dependent. The job will only then run if 5 files exist.
You may need to split your existing job up, depending on what it is doing, but it does work well.

I like the dependency method posted above by osirisja, as you can timeout the dependencies as a set, etc but for the sake of information gathering here is a way using a regex to match filenames as mentioned by Henrik. Information is power!

If you would have shown the exact file names, the regex would be more accurate, but to match your example, here is the regex to match files named based on your description:
name1_20180502.xml
name2_20180502.xml
name3_20180502.xml
name4_20180502.xml
name5_20180502.xml

Regex: name[12345]_\d{8}.xml

Starts with name, followed by character class of one of the digits 1, 2, 3, 4 or 5, followed by an underscore, then 8 digits (the date) followed by the .xml extension.

Granted, it does not check for 5 files, but it matches the file set you need which can then be counted with a Set Job Variable task with this value: {STRING(RowCount|{TASK(PrevTask|StdOut)})}. Use this task's standard out or the value of the job variable to decide how to proceed. Don't forget to check the box for "Translate value to constant in Variable when running" if you set a job variable.

Thanks for the advise and input everyone!

I am already doing a file count using a file list and a count variable (very nifty). It's knowing that one and only one of each "flavor" of file exists in the folder before proceeding that is hanging me up.

Here is a typical file set:
Provider_MyHR_4272018.xml
psr_nwpag_20180423T131254.xml
psr_nwpgme_20180423T131402.xml
psr_nwphr_20180420T113431.xml
psr_onyxh_20180423T180002.xml

Here is what I want to avoid
psr_nwpag_20180422T134274.xml
psr_nwpag_20180423T131254.xml
psr_nwpgme_20180423T131402.xml
psr_nwphr_20180420T113431.xml
psr_onyxh_20180423T180002.xml

There are 5 files, so it passes my file count test, however two of the files have a prefix of psr_nwpag_* and there is not a file that has a prefix of Provider_MyHR_

I could probably test for them one at a time, either using 5 specific distinct conditions, or 5 file list tasks. In either case I could create a fail if any one of the conditions or tasks does not find the single file prefix it's looking for in the folder, but I feel like there should be a way to do this in one validation activity and I'm just missing a concept..

Thanks again everyone!

Robin

Hi osirisja and Gary_W,

I think I stumbled on to a work around.. While I still believe an enhancement would be useful

List Files Task that outputs the file list (used in the next step to count that there are 5 files)
Condition Set on List Files Tasks that has 5 conditions, each one looking for a FALSE condition for that file name.
If Match Any or Match All - go to a task that notifies stakeholders and skips over the rest of the tasks in that job (no error, but does not proceed, and notifies which is not as good but will do in a pinch)
If Match none - continue

If none of the conditions in the Condition set were false we go to the next task. That task has a condition set that first checks the file count using a variable from the List Files Task, or could use previous task variable
Value 1: Variable {TASK(c9d4042d-6b55-4387-8ae9-db48cee08e32,Result.NoFiles)}
Type: Int32 Equals (=)
Value 2: 5
If Match Any or Match All - Continue
If Match none - go to a task that notifies stakeholders and skips over the rest of the tasks in that job (no error, but does not proceed, and notifies which is not as good but will do in a pinch)

The first condition set validates that there is at least one file of each "flavor", but there may be more than one of each flavor, which is why the second condition set checks for a count of 5

This is a work around, the biggest issue is that it does not fail the job or track an error. Which is why we still need a feature enhancement.

I thought of a different way for the sake of argument. Working in my example. You may want to consider it or parts of it. Just another way to approach the problem.
Create a job variable that holds the master list you expect, less the date/time and extension (make sure its sorted alphabetically. Note it could come from a database table or a file too of course):
Variable JobFileListMaster contains:
Provider_MyHR
psr_nwpag
psr_nwpgme
psr_nwphr
psr_onyxh

Also create a job variable called JobFileList for later use to hold the results of whats actually found.

First task lists files using the regex ^(Provider|psr).*\.xml (start with Provider or psr and end with .xml) make sure to check regex and case sensitive. Output tab should specify sort by file name ascending. On the Output settings tab, select filename only.
Next set job variable task sets JobFileList to the output of the previous task (the list files output)
Next set job variable task removes the last underscore on from the JobFileList variable using a value of {REGEX(Replace|{JOB(Active|Variable|JobFileList)}|_[^_]*\.xml|)}
Now you can compare the job variables JobFileListMaster and JobFileList. If the same, then one of each file exists, without having to count.
For my test example, the last test is a popup with a flow that uses an on start event to do the comparison. If not equal, stop the job, Else it shows the contents of the job variables.

It's not too hard to set up and may give you some ideas.

Thanks Gary,

This seems like a more moving parts than my workaround, but I am seriously intrigued.. I think I'm following you.. but I cant quite bring this home..

This is what I've done so far

Variable JobFileListMaster = hard coded list of file name roots
Variable JobFileList = empty
List Files Task creates the list of file names that are currently in the staging folder
Set Variable task loads the JobFileList variable with the output from the previous task
Set variable task resets the JobFileList variable values with the initial values with the string values after the second underscore removed, but the .xml extension remaining
REGEX(Replace|{JOB(Active|Variable|JobFileList)}|_[^_]*\.xml|)}

Does this set up reflect what you intended so far?
Beyond that, you lost me.. what are you using to do the Boolean compare of the variable values? what is your on start flow invoking?

Thanks.. this is fun

Robin

Sorry I wasn't clear enough.

The second set variable task removes the last underscore to the end of each line, inclusive.
Now, JobFileList should match JobFileListMaster if all is well. Of course there is always room for tightening up the matching regex's but this is a proof of concept.
The next task for testing is a popup. The message just shows
{JOB(Active|Variable|JobFileListMaster)}
{JOB(Active|Variable|JobFileList)}

Its flow has an On Start condition that compares If other value (type string) - if {JOB(Active|Variable|JobFileListMaster)} does not equal {JOB(Active|Variable|JobFileList)} then stop job.

Once set up, test by changing file names in the folder and run the job.