ETL and Data Integration tools - Modern ETL software for everyone

Home Products Advanced ETL Processor ProNews

Advanced ETL Pocessor News

Generate Reports using ETL

A lot of ETL vendors concentrate just on Data transformation tasks but we believe that most companies nowadays would like to automate entire business processes . One of the unique advantages of using Advanced ETL Processor is the ability to generate reports in various formats including PDF, CSV, RTF and Excel. It is also possible to automatically print reports.
You can do it in four easy steps

Use report wizard to generate report quickly.
Use report designer to adjust advanced settings.
Preview Report and check design
Add report into ETL Package

Learm more about Advanced ETL Processor

Exporting all tables into text files

This article describes how you can export all tables from any database into text files

First step is to download and install Advanced ETL Processor Professional from here

Than we need to create an ETL package. Let call it Export Package.

Here is a flash tutorial

Double click on Export Data Object and select appropriate connection type

Next step is to specify tables to export. It also possible to use mask as a filter

If necessary user can restrict number of records tables to export

And last step is to select output file format. User can export all data into one file or into separate files.

Lets run the Package

Information   10/05/2010 18:30:33   Preparing to execute: Mass Export
Information   10/05/2010 18:30:33   Package Started
Information   10/05/2010 18:30:34   Executing Export: Export all Tables
Information   10/05/2010 18:30:34   Executing:Export all Tables
Information   10/05/2010 18:30:34   Found: 10 Tables(s) to read
Information   10/05/2010 18:30:34   Mask: *
Information   10/05/2010 18:30:34   Reading all tables
.......
Information   10/05/2010 18:30:35   Source Table: Total_Sales
Information   10/05/2010 18:30:35   Data Source has: 3276 Records
Information   10/05/2010 18:30:35   Export File : c:\support\testTotal_Sales.csv
Information   10/05/2010 18:30:36   Source Table: Orders_Analysis
Information   10/05/2010 18:30:36   Data Source has: 17 Records
Information   10/05/2010 18:30:36   Export File : c:\support\testOrders_Analysis.csv
Information   10/05/2010 18:30:36   Source Table: price_list
Information   10/05/2010 18:30:36   Data Source has: 69 Records
Information   10/05/2010 18:30:36   Export File : c:\support\testprice_list.csv
Information   10/05/2010 18:30:36   Processed: 8574 Record(s)
Information 10/05/2010 18:30:36   Records per second: 98552
Information   10/05/2010 18:30:36   Time Taken: 00:00:00
Information   10/05/2010 18:30:36   Done!
Information   10/05/2010 18:30:36   Package Finished

The Blob challenge

Common task - loading files in to the database not so simple as you might think

When we thought about introducing this functionality we decided to use file system as data source or target.
So for example File system as data source provides following fields

[Directory]
[File Name]
[Size]
[Creation Date]
[Read Only]
[Hidden]
[Archive]
[File Data]

File system as data target uses just two fields

[File Name]
[File Data]

Next task actually loading files data into blob fields. We had to perform incredible amout of testing.
For example data can be loaded into Oracle Database in 4 ways

Via ODBC,
Via OLE DB,
Via Direct path loading
And Conventional path loading

Plus there are different versions of ODBS drivers and OLE DB Providers
And of course different versions of Oracle database.
And Various data types Raw, Long Raw, CLOB, NCLOB, Blob
and it has to be repeated for every database type our ETL software supports

Once file is loaded into blob field we export it back into different directory and compare files using MD5

Here is the package we used for testing