ETL and Data Integration tools - Modern ETL software for everyone

* Version Number = Advanced ETL Processor Ent. 4.2.3.5

* Repository type = MS Access 2007

* Database type you are working with. = .MDB

* OS = Windows 7

After I added the deduplicator to the project I got 0 records output, before the deduplicator i got all records output. my understanding was to check all fields to be passed to transformation module, not sure how you check to dedupe city or state without wiping Entire record pass through. But I need both of those pieces of information to be passed.

I am very confused of what you are trying to achieve to be honest.

Can you provide more information please

Mike

I am combining information from an Access database to import into an Oracle database.In one example I am using The combined [Last Name] + [Address] + [Store ID] to create my unique key for the new system. there are duplicate records from multiple purchases over months and years so they become a duplicate name in a field that has to be a unique key. I wan to find these duplicate records and put them into a separate file so we can figure out a way to associate them to the parent record later.

I think using grouper would be a better option

Put Data Transformer before Grouper

Data reader =>Transformer=>Grouper>Validator=>Transformer=>Data Writer

Open transformer
Click copy inputs to outputs
Map fields
Create two additional output fields.
both of them would be [Last Name] + [Address] + [Store ID]
call them PK1 and PK2

open grouper

For PK1 set grouping type to group by
For PK2 set grouping type to count
For the rest of fields Last Value

Within validator filter those where PK2>2

Hope that helps,
Mike

Still getting duplicate records in output file

I think you have to sort the data before grouping it

Data reader =>Transformer=>Sorter=>Grouper>Validator=>Transformer=>Data Writer

Mike