ForEach File - Filename mask
Hello
I'm using the ForEach File task and masking the files I want to process as ABC??.TXT.
It all works well until I drop a file there like ABC12 Copy.TXT, which I don't expect to be process, but the loop still picks it up. Am I using the wrong wildcards?
newstar1860
December 20th, 2007 5:52am
It works for me, using ?. How are you verifying which files are processed?
Free Windows Admin Tool Kit Click here and download it now
December 20th, 2007 6:20am
In the loop, thefirst task I'm doing is to move the files and it picks up too many files. It's like the ? is interpreted as an *.
newstar1860.
December 20th, 2007 7:22am
How many times is the loop actually firing? Can you add a script task to it that displays a message box, or add a DTS.Log() method call so that they can be counted?
You may have already checked this, but is the task to move the files configured properly (not using a wild card, but just using the single file being processed in the loop?
Free Windows Admin Tool Kit Click here and download it now
December 20th, 2007 7:26am
Sorry, not knowing too well what I'm doing it took me a while to create a script to show me a message box.
OK it goes thru the loop twice for file ABC12.TXT and again for ABC12 Copy.TXT.
I've setup theforeach loop with the following expressions:
Directory @[User::Inbound]
FileSpec "ABC??.TXT"
and in the Variable mappings I'm using another user variable to store the filename to be processed.
Interestingly, if I change the FileSpec to "ABC?? .TXT" It doesn't find a file (as expected), so why does it find the 'ABC12 Copy.TXT' file with the original FileSpec?
December 21st, 2007 2:56am
I still can't replicate this. Are you on SP2 for SQL Server?
Free Windows Admin Tool Kit Click here and download it now
December 21st, 2007 4:21am
Yes we are on SP2.
Actually when I try it with my example filenames I get it to work as well. It only seems to have problems when the filenames are longer.
For the 2 files ABCDEF22.TXT and ABCDEF22 1234.TXT and a filemask of "ABCDEF??.TXT" it stops working for me, meaning it is picking up both files.
Would you mind trying this?
December 21st, 2007 8:29am
Interestingly, this appears to work incorrectly for me on only when I have a 8.3 character filemask, like ABCDEF??.txt. It's leading me to thinkit is a bug in the handling of8.3 filenames. Can you try a filemask with a different number of characters and see if it works?
Free Windows Admin Tool Kit Click here and download it now
December 21st, 2007 9:52pm
Sorry I was away for a few days, needed a break.
I've done a bit more testing now:
Files: "ABCDE22.TXT" and "ABCDE22 1234.TXT"
MASK:"ABCDE??.TXT"
Result: only file "ABCDE22.TXT" is moved as expected.
Files: "ABCDEF22.TXT" and "ABCDEF22 1234.TXT"
MASK:"ABCDE??.TXT"
Result: No file is moved as expected.
Files: "ABCDEF22.TXT" and "ABCDEF22 1234.TXT"
MASK:"ABCDEF??.TXT"
Result: Both files are moved although only file "ABCDEF22.TXT" should be moved.
Files: "ABCDEFG22.TXT" and "ABCDEFG22 1234.TXT"
MASK: "ABCDEFG??.TXT"
Result: only file "ABCDEFG22.TXT" is moved as expected
It really confirms your suggestion that there seems to be a problem with filenames of length 8.3.
What do I do now?
December 27th, 2007 5:15am
I would recommend you to file a bug report in the SQL Server Connect site . Make sure you include a description of the steps that leads to the reproduction of the issue.As a work around; you could try to get all files in the foreach collection; and then use variables with expressions and precedence constraints to decide which files are to be processed and which one to be ignored. You can find an example here:http://rafael-salas.blogspot.com/2007/02/ssis-loop-through-files-in-date-range.html
Free Windows Admin Tool Kit Click here and download it now
December 27th, 2007 6:10am
It would be good if someone from the SSIS team could confirm if this is a bug.[Microsoft follow-up]
December 27th, 2007 6:16am
Our Foreach File enumerator is internally using FindFirstFile and its related APIs to search for files that match the specified pattern. Those APIs support bothlong and shortfile names. My guess is the file name "abcdef22 1234.txt" has a short file name like "abcdef~1.txt" or something and that's probably the reason it turn up as a match.
So this can be a bug depending on where you stand. Please do help us log this as a bug.
Free Windows Admin Tool Kit Click here and download it now
January 3rd, 2008 12:47am
I've submitted a bug report on Connect: https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=321020
January 4th, 2008 6:20am
The Connect issue has been closed as By Design. The root of this issue is in the Windows file system and is by design.
The workaround is to make sure you don't have a eight character pattern with two wild cards as the last two characters, or disable short filename generation on your file system, which has other potential ramifications.
Free Windows Admin Tool Kit Click here and download it now
February 20th, 2008 5:04pm
I was recently cleaning up some old scripts; using wildcard expansion to process them in groups by filename. I came upon this result:
dir *41*.pl
05/12/2011 01:04 PM 773 run_oets_deliver_db41.pl
05/12/2011 01:03 PM 1,173 run_oets_gold_db41.pl
05/12/2011 01:03 PM 761 run_oets_prod_db41.pl
05/12/2011 01:03 PM 766 run_oets_prod2_db41.pl
05/12/2011 01:02 PM 765 run_oets_train_db41.pl
05/12/2011 01:04 PM 1,345 run_oets_deliver_db36.pl
05/12/2011 01:03 PM 757 run_oets_npf_db41.pl
05/16/2011 04:22 PM 820 run_oets_train_db41_W7.pl
8 File(s) 7,160 bytes
After much wringing of hands and gnashing of teeth, I realized that the system was matching on the short filename as well as the long file name:
dir /x *41*.pl
05/12/2011 01:04 PM 773 RUN_ND~2.PL run_oets_deliver_db41.pl
05/12/2011 01:03 PM 1,173 I9C1T00~.PL run_oets_gold_db41.pl
05/12/2011 01:03 PM 761 M9C1T00~.PL run_oets_prod_db41.pl
05/12/2011 01:03 PM 766 O9C1T00~.PL run_oets_prod2_db41.pl
05/12/2011 01:02 PM 765 R9C1T00~.PL run_oets_train_db41.pl
05/12/2011 01:04 PM 1,345 416KU00~.PL run_oets_deliver_db36.pl
05/12/2011 01:03 PM 757 1AUPN00~.PL run_oets_npf_db41.pl
05/16/2011 04:22 PM 820 048OS00~.PL run_oets_train_db41_W7.pl
8 File(s) 7,160 bytes
This is an ugly problem, made much worse by the advent of hexadecimal characters in the short filenames! We need a switch in the API and the utilities which use it to tell it to search long filenames only!
David
May 18th, 2011 2:03pm
Why did you append your question to this SSIS post?
Arthur My Blog
Free Windows Admin Tool Kit Click here and download it now
May 18th, 2011 3:19pm
This was the closest I could come to a relevant forum. Mea Culpa. But really, it is a problem for anyone using the API. It's broken. David
May 24th, 2011 8:35pm