HomeForum
Welcome, Guest

Using ADO as datasource
(1 viewing) (1) Guest
Data Warehousing and Data integration
  • Page:
  • 1

TOPIC: Using ADO as datasource

Using ADO as datasource 1 year, 1 month ago #2548

  • Tim
  • OFFLINE
  • Junior Boarder
  • Posts: 37
  • Karma: 0
Does any of your ETL tools support ado interface?

Time

Re: Using ADO as datasource 1 year, 1 month ago #2549

  • admin
  • OFFLINE
  • Moderator
  • Posts: 2183
  • Karma: 12
Yes. it does but we do not recommend using it

There are two problems with ADO.
1. It is incredibly slow for inserts.
(50 records per sec with ADO, 2500 per sec with Oracle OCI, depending on the table)

2. When exporting data from large tables it uses all available virtual memory.
What we wanted to do is to fetch 100 records into a buffet in the memory than reuse same buffer over and over again.
Instead it kept all the records in the memory.
If you have 20GIG table think how much memory you need.
We tried to use forward only cursors it did not help either

May be ADO is much better and faster now but so far all our test proved otherwise.

Our setup is only 10MB and it works straight out of the box.
Because we did run a lot of performance tests and removed all redundant code.
and we are still working on the improvements

Best Regards,
Mike.
  • Page:
  • 1
Time to create page: 0.15 seconds

Testimonials

Within an hour we were able  to populate fact and dimension tables in our datawarehouse using Advanced ETL Processor

P. Jonson

User Login

You only need to log in or register to use our support forum



Our customers

BP

BBC

HSBC


Databases we work with

Go to top