HomeForum
Welcome, Guest

Transfering 2M + records from csv to Oracle table
(1 viewing) (1) Guest

TOPIC: Transfering 2M + records from csv to Oracle table

Transfering 2M + records from csv to Oracle table 1 year, 5 months ago #1279

  • rudolfsun
  • OFFLINE
  • Junior Boarder
  • Posts: 32
  • Karma: 0
Hi

Im trying to load these files into Oracle from csv files and when it reaches 650K the tool fails with an error message of 'Out of memory'

I have increased the Virtual memory, the machine itself has 2G, and deleted all temp files etc.

Any ideas on how else I can get this to run?

thanks

Re: Transfering 2M + records from csv to Oracle table 1 year, 5 months ago #1281

  • admin
  • OFFLINE
  • Moderator
  • Posts: 2174
  • Karma: 12
Hello

There are several ways of loading data into oracle

OleDB
ODBC
Direct path
Conventional path

You did not say which one are you using

Also is there any additional transformations?
Sorting groupping etc

Peter

Re: Transfering 2M + records from csv to Oracle table 1 year, 5 months ago #1283

  • rudolfsun
  • OFFLINE
  • Junior Boarder
  • Posts: 32
  • Karma: 0
Hi Sorry its an ODBC link

Im not doing any additional transformations, just a direct load

Re: Transfering 2M + records from csv to Oracle table 1 year, 5 months ago #1284

  • admin
  • OFFLINE
  • Moderator
  • Posts: 2174
  • Karma: 12
Have you tried conventional path?

ODBC is not the fastest way of loading the data

Version of oracle and version of odbc driver?

Peter

Re: Transfering 2M + records from csv to Oracle table 1 year, 5 months ago #1285

  • rudolfsun
  • OFFLINE
  • Junior Boarder
  • Posts: 32
  • Karma: 0
Hi

Sorr I think Im getting a bit confused, the connection I am using in the ETL tool is an Oracle connection not ODBC. But that uses an ODBC connection at the back end, I think

sorry Im not being much help

Re: Transfering 2M + records from csv to Oracle table 1 year, 5 months ago #1286

  • admin
  • OFFLINE
  • Moderator
  • Posts: 2174
  • Karma: 12
In this case can you post here a screen shot of the data writer connection so we can see all the options.

Also what is your version of Oracle?

Mike
Time to create page: 0.18 seconds

Testimonials

Within an hour we were able  to populate fact and dimension tables in our datawarehouse using Advanced ETL Processor

P. Jonson

User Login

You only need to log in or register to use our support forum



Our customers

BP

BBC

HSBC


Databases we work with

Go to top