Sort is removing rows wrongly
Hi All,
Im using Sort Transformation in SSIS 2005, and, I dont know why, is removing rows (tick "remove duplicated rows" is not active, even watching the xml of the package)
Ive search in the forum, and found this threadbut without an answer.
If a modify the comparasion flags, the rows removed increase or decrease (depending of the flags actived), but not so far as the rows removed (the input has 117.000 rows and the output has 29.000, with some flags I get 32.000 rows).
The colum which Im sorting, is a char(1) (DT_STR), an the codePage is 1252.
Has anyone got this problem previously? Any solution?
Thanks in advance!
Sorry for my English, but Im from Spain
May 11th, 2009 8:43pm
can you filter this down to less rows and see if the issue happens still?if it still happens can you find examples of what is going on?if you sort by another column is every thing OKsorting 117,000 rows on a char(1) looks like you will have lots of duplicates, is this what you expect?PSyour English is much better that my Spanish.
Free Windows Admin Tool Kit Click here and download it now
May 12th, 2009 2:07pm
No, Its not. I had used this tranformation in many palces and it works perfect. Change your key column and check whether it persists.http://effulgentlogs.blogspot.com
May 12th, 2009 2:17pm
Hi, I have noy noticed any such behavior. You could check with samll number of records as suggested by AlunJ. Check if any specific record is skipped every time. Check for any pattern or specific type of records.Hope this helps !! - Sudeep
Free Windows Admin Tool Kit Click here and download it now
May 12th, 2009 7:09pm
Hi All,
Im using Sort Transformation in SSIS 2005, and, I dont know why, is removing rows (tick "remove duplicated rows" is not active, even watching the xml of the package)
Ive search in the forum, and found this threadbut without an answer.
If a modify the comparasion flags, the rows removed increase or decrease (depending of the flags actived), but not so far as the rows removed (the input has 117.000 rows and the output has 29.000, with some flags I get 32.000 rows).
The colum which Im sorting, is a char(1) (DT_STR), an the codePage is 1252.
Has anyone got this problem previously? Any solution?
Thanks in advance!
Sorry for my English, but Im from Spain
I Have stated by other here, I have not seen this problem before. Are you sure is the sort transformation causing the problem? I wonder if the issue is rather caused by other transfroamtions in the data flow (e.g. merge join transformation, conditional splits, etc)
Rafael Salas | Dont forget to mark the post(s) that answered your question http://rafael-salas.blogspot.com/
May 12th, 2009 7:40pm
Im sorry for the delay, I was assigned to other project, but Im back again.
Ok, Ive added some images here:http://img257.imageshack.us/gal.php?g=ssissortremoving.png
There, you can see that is a Sort Component problem... anyone has any idea?
I think it should be a column type problem.
As commented before, Im trying to get a pattern of the data filtered, but it seems that there is not a pattern...
Duplicated values are expected
Free Windows Admin Tool Kit Click here and download it now
June 5th, 2009 2:17pm
Can you put a splitter above this task in the dataflow and see if you can find where the problem goes away.I would recommend something like left(customername,1) < 'G' then change the letter
June 5th, 2009 3:09pm
Hi again...
Hmmm, the splitter worked fine, the rows sum of every splitters output is the same as the input.
In my case I can use the splitter for my objective, but Im still trying to know why the sort failed... anyone has an idea?
The column that I used to split is avarchar(1) (null allowed)with a collationModern_Spanish_CI_AS, and the possible values are 'A','C','D','S','V','P'.
Free Windows Admin Tool Kit Click here and download it now
June 8th, 2009 11:17am
WTF!!
Im thinking that this is not a database problem, Its a Visual Studio problem!!
Ok, I removed the sort component, and added the splitter... but a previous sort component in the flow, that previosly was working fine, is now removing rows...
June 8th, 2009 1:03pm
Well, more strange behaviour in ssis...
I delete de Data Flow Task, and created a new one.
Watch these three images in this urlhttp://img150.imageshack.us/gal.php?g=03ssisderived.png
You will see two tables in a merge join, with sort component working fine.
Another image, where the sort component is not working fine. The reason: I added a derived column component (the third image is this component editor) after the merge join component.
So, the merge worked, but when I added a derived column component, the sort is removing rows.
I wish I have explained the error, to get a solution, because this ssis script is making me crazy!
Thanks in advance!
Free Windows Admin Tool Kit Click here and download it now
June 8th, 2009 5:57pm
Do you still have a problem?if you think you have a bug can you re-create it.many thanks for the feedback.
June 16th, 2009 1:31pm
Ok, I got the solution... the problem was while writing the data in the destination output (it was inserting a null value in a column with a not null restriction).
Once the problem was resolved, the sort component didnt remove rows (in fact, It wasnt removing rows, just the Sort component stopped when the the fail ocurred in the output. This was the problem).
Free Windows Admin Tool Kit Click here and download it now
June 16th, 2009 6:08pm
can i know how did u removed that null row.
I am facing the same problem.
I am new to ssis
Please advice
June 17th, 2011 4:39am