SSIS skipping footers from flat files
Hello,
I have to import flat files with headers and footers. I found how not to import headers but still have issues with footers. Is there a quick way to do that?
March 5th, 2008 10:31pm
You can use a first dataflow to read the file (no destination component need it) and store the number of rows in a variable; then in a second dataflow, add an extra column to enumerate each row, and use a conditional split to discard the rows last x rows.
Free Windows Admin Tool Kit Click here and download it now
March 5th, 2008 11:21pm
Just to clarify a bit, if your footer has a different format than the rest of the rows, this might be a bit more complicated. The first data flow needs to import the entire row as a single string column. You'd then have to re-import them as a single column, use the conditional split to remove the footers) and save it to a new flat file. After that, you can reload them again with the appropriate column definitions.
There are other methods of handling this:
http://www.sqlservercentral.com/articles/Integration+Services/61880/
http://agilebi.com/cs/blogs/jwelch/archive/2007/05/07/handling-flat-files-with-varying-numbers-of-columns.aspx
March 6th, 2008 6:11am
Adding a row number and conditional split will still produce the warning, as the file is being read by the flat file source adapter. Might not help much. Single column read approach has less code then using a script task to parse out footers before using
the flat file source adapter.GagandeepSingh
Free Windows Admin Tool Kit Click here and download it now
May 30th, 2011 1:57pm