Dynamic tables using file names
Hi,
I am having 28 fixed length files. All are of different data types, lenghts and do have millions of records. I have to load this data into 28 tables, which has to be dynamically created based on the columns of the input files.
Can some one suggest me on how to achieve this ?
Thanks a lot.
Regards,
Ansul G
May 18th, 2011 7:31am
This thread should be merged with
http://social.msdn.microsoft.com/Forums/en-AU/sqlintegrationservices/thread/110a8e79-a7f0-482c-b229-e4a6573b38f1Jeff Wharton MSysDev (C.Sturt), MDbDsgnMgt (C.Sturt) MCT, MCPD, MCITP, MCDBA, MCSD, MCAD
Free Windows Admin Tool Kit Click here and download it now
May 18th, 2011 7:37am
See this video:
extracting data from excel files dynamically using SSIS. also demonstrates how to make connection managers and other properties dynamic.
http://www.youtube.com/watch?v=_B83CPqX-N4Arthur My Blog
May 18th, 2011 4:16pm
Hi Ansul,
You have to create it manually. because, Flat files with different structure need to be configure before package execution.
It is very simple to create destination table using SSIS.
Create a data flow task
add source file and configure fields and type
add ole db destination and create new table. Yes, SSIS will write a script for you.
Do this for all 28 files if all are different.Thanks Ayyappan Thangaraj UG Lead, Puducherry, http://SQLServerRider.blogspot.com
Free Windows Admin Tool Kit Click here and download it now
May 18th, 2011 5:34pm
Thanks a lot. I have 1500 million read write operations to be done using SSIS 2008. I am using a 3GB RAM, win 2003 server. Do you have a rough idea on how much time will it take to execute the entire flow. ON a high lever, here is the requirement.
Extract all the data from the 28 files and load it into a sybase table. Apply business transforamtions and load them into 64 tables. Extract data from this 64 tables and load them into 28 tables. Extract them to output files.
Regards,
Ansul Gupta
May 20th, 2011 1:28am
Thanks a lot. I have 1500 million read write operations to be done using SSIS 2008. I am using a 3GB RAM, win 2003 server. Do you have a rough idea on how much time will it take to execute the entire flow. ON a high lever, here is the requirement.
Ansul Gupta
What type of CPU?
What about harddrive space and speed?
What does the data look like i.e. is it mainly string, dates, numbers?Jeff Wharton MSysDev (C.Sturt), MDbDsgnMgt (C.Sturt) MCT, MCPD, MCITP, MCDBA, MCSD, MCAD
Free Windows Admin Tool Kit Click here and download it now
May 20th, 2011 1:31am
CPU is Intel Dual Core, 180 GHz
Hard drive - I am not sure, but will have near about 500 600 GB of disk space.
The data basically consists of text, numbers.
May 20th, 2011 3:41am