hi in my oledb destination i am getting some duplicate rows,i want to remove it before data loads into table

You can handle duplicates in many ways. The simplest way would be to filter them out using your Source query. How about writing a distinct one? If you do expect duplicate records, how do you want to handle them? are you planning to discard them or overwrite them in your destination table?Regards, Dinesh

i am merging two files and then putting in the table. before laoding in the table i need to make sure that merger rows are not duplicate

one way is load them to a staging table and use something like this(modify ur query accordingly) with t1 as (select intclaimid ,ROW_NUMBER() over (partition by intclaimid order by intclaimid) as r from f_Claims ) delete from t1 where r > 1 Also there is an option in sort transformation to remove duplicate values try that if ur data volume is not that high(can be performance hinder)

You are better off writing the data first to a staging table and then moving the data to your destination table by using a Merge Statement. The Merge should overwrite the duplicates based on the primary identifier you define.Regards, Dinesh

You are better off writing the data first to a staging table and then moving the data to your destination table by using a Merge Statement. The Merge should overwrite the duplicates based on the primary identifier you define.Regards, Dinesh