SSIS populate object Variable
Hi, I have an SSIS variable of type object. I created a for each loop on a share drive and for each iteration i have a script task that is checking a file date and based on the day, i want to store the file name in the object variable. So if the directory has 100 files and 40 of them match the date criteria, i want to store the 40 file names in the object variable. Is this possible in a script task using vb? thanks scott
May 24th, 2011 4:42pm

It is possible - but there may be a better solution. What are you going to do with the 40 file names? Talk to me now on
Free Windows Admin Tool Kit Click here and download it now
May 24th, 2011 5:21pm

Todd, Thanks for the reply. Once i store the 40 or so filenames in an object variable, i will then need to check each file name with another filename from a table in a database to see if it matches. If it does, then i will be converting the file to pdf. So in short, i need to iterate through a directory of about 1000 or so files, get only the files that have a modified date of today, then bounce those file names against file names that are stored in a table in a database to see if they have a match, then convert them to PDF. Sounds fun huh? I was hoping SSIS would help. I know i can use for each loop to iterate through the files in the share directory. I was thinking of then having a script task to check each file to see if date modified is today. IF so, store it in object variable. Then have another step to iterate the object variable and check it against the table in the database to see if there are matches. Then have a final script step to convert files to pdf. any help would be appreciated. thanks scott
May 24th, 2011 7:01pm

I'd be tempted to do most of this in a script task Use Match Files and an array list to get your list of 40 files: Sub Main() Dim Test_Date As date = Date.Today Dim Archive_Files() As String = System.IO.Directory.GetFiles("FolderPathHere") Dim New_Files As New ArrayList For x As Integer = 0 To Archive_Files.Length - 1 If File.GetLastWriteTime(Archive_Files(x)) < Test_Date Then New_Files.Add(Archive_Files(x)) End If 'Get Table records into recordset then into a string seperated by | 'Do this so we only have to loop through the recordset once Dim con As New OleDb.OleDbConnection Dim cmd As New OleDb.OleDbCommand Dim dr As OleDb.OleDbDataReader con.ConnectionString = dbConnection Try con.Open() Catch ex As Exception End Try cmd.Connection = con cmd.CommandText = "SQL TO GET LIST OF NAMES FROM DATABASE" dr = cmd.ExecuteReader() If dr.HasRows = True Then While dr.Read strNames = strNames & dr.Item(0).value.tostring & "|' End While Else strNames = "" End If dr.Close() Next For Each Found_File As String In New_Files If instr(Found_file,strNames) > 0 Then 'Create PDF End If Next End Sub Rgds Geoff
Free Windows Admin Tool Kit Click here and download it now
May 24th, 2011 11:48pm

OK - that's what I thought. There's no need to store this stuff in an object variable - just do it all in one loop. Use the Foreach Loop you have to iterate over the files. Use an Execute SQL Task to do your lookup. You'll have to use property expressions to craft your SQL based on the filename variable the loop is using Then use a third-party File Properties Task or a script task like this, or similar to Geoff's to read the file's time into an SSIS DateTime variable. (You'll need to specify the Filename variable in the script's ReadOnlyVariables property, and the Modified variable in the script's ReadWriteVariables property.) Attach an Execute Process Task (or whatever kind of task does your conversion) to the file properties (script or custom) task with a Precedence Constraint. Double-click on the precedence constraint to change it to an "Expression and Constraint", and set the expression to "YEAR(@Modified) == YEAR(GETDATE()) && MONTH(@Modified) == MONTH(@GETDATE()) && DAY(@Modified) == DAY(GETDATE())". Geoff - your solution would probably work. Personally, I don't like doing that all inside a Script. First, it's opaque to someone who knows SSIS but doesn't know code. (Yes, I know that's a small percentage of people... today. But it will (should) be the norm!) Second, you're using an unmanaged connection in the script - not a best practice in my book. Third - there's no logging inside scripts unless you manually add it. Fourth, you'll never benefit from version upgrades/patches to SSIS within a script. Of course, those are my opinions only... Talk to me now on
May 25th, 2011 1:00pm

Thanks to both of you for the response. i will try to see which option works best. I may use parts of each solution if need be. thanks again!
Free Windows Admin Tool Kit Click here and download it now
May 25th, 2011 2:06pm

Hi Todd - I do generally try to stay away from code where possible (although I must admit I'm from a bit of a coding background so it's always tempting) The code is actually incorrect in that there should be the lines dim dbconnection as string dbConnection = Dts.Connections( "PackageConnectionName").ConnectionString so the connection used is one set up in the package already The reason I went with script for this was efficiency - getting the names of the files already created and appending into a long string to be able to use instr to find values (hitting the database 1 time) would generally be faster and more efficient than hitting up the database multiple times - especially if the no of new files grew from 40 to something rather larger Other than that - thanks for the info - I'm by no means an SSIS expert so any hints and tips I can get from experienced users such as yourself are appreciatedRgds Geoff
May 25th, 2011 7:12pm

I do try to shy away from Foreach Loops where database operations are involved as well - using a Foreach Loop with an Execute SQL Task inside it always seems to ask the question "shouldn't you be doing this in a Data Flow?" Talk to me now on
Free Windows Admin Tool Kit Click here and download it now
May 25th, 2011 8:16pm

Todd, In your explanation above, you state the following: "Use an Execute SQL Task to do your lookup. You'll have to use property expressions to craft your SQL based on the filename variable the loop is using" I assume this will be where i would bounce the report name from the for each loop against the report name in the table in the database to see if there is a match. My question is what exactly to i do if the report is found in the table? Do i store the value in a parameter? I would want to just somehow state that if the report is found in table, then use that report name in the next step, otherwise either do nothing or store nothing. Does that make sense? How would i go about doing that?
May 26th, 2011 3:21pm

You can use parameters, or a resultset to communicate "back" from SQL to SSIS. I'll describe input parameters and output result sets. Assuming you have a variable called "FleName" that your Foreach Loop is populating with a file name, then you can use a statement like this on the General tab: SELECT COUNT(*) FROM table WHERE filename = ? You'd also say you're getting back a ResultSet type of SingleRow On the Parameters tab, you'd specify parameter zero to be filled by your FileName SSIS variable. On the Result Set tab, add a result name of 0 (zero), and associate it with an SSIS integer variable called FileExists. Back in the Control Flow, double-click on your precedence constraint from the Execute SQL Task to whatever's next to "do", and change the operation to "Expression and Constraint". Set the expression to "@FileExists == 0". Now the tasks connected to the arrowhead will only get run if the COUNT returns a zero... Talk to me now on
Free Windows Admin Tool Kit Click here and download it now
May 26th, 2011 4:44pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics