ETL and Data Integration tools - Modern ETL software for everyone

Dear Support

I have a source with 12 millions records and its a relational source,out of which i need to filter out some flagged records based on 2 combination of column.

Example.

key, term_exp, Flag
101 11/2000 Y
101 11/2000 Y
101 11/2000 N
102 12/2000 Y
102 12/2000 Y
102 12/2000 Y

If there is a single value with Flag 'N' then I need to drop all records for those dates,
If all values for the same date or Flag is 'Y' then I need to process all records for the 'Y' values.

Thanks in advance,
Cesar.

Excellent question

You would need to sort data first so 'N' records are the first ones in the group.

Than you can compare previous record with current one to check that key and term_exp are different ones.

Mike

Mike

Can you be more specific please

Cesar

I have created a working example for you.

Extract everything into c:\support folder

Mike

That did the trick

Thank you,
Cesar

I am glad to hear that you are making the progress.

Another thing I wanted to tell you if the source is the table you can use sql to get the data you need.

Mike