Skip to Main Content

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Hugo.

Asked: November 27, 2017 - 5:16 pm UTC

Last updated: August 06, 2018 - 2:25 pm UTC

Version: sql developer 4

Viewed 10K+ times! This question is

You Asked

Hi TOM :)

Resourse:
1) i have a table that was milions of records of the clients
2) and i have a CSV with only 1,200 clients
3) i don't have permitions to create a table.


Problem: how can i read from the CSV to join with the principal query and retrive only the data from the selected clients:

example:

-------------------------------------
with clients_selected
(
   select * from FILE.CSV
)

select process_id
  from t_process tp
  join clients_selected cs
  on cs.client_id=tp.client_id


-------------------------------------------------

the CSV files as only this:

NIF
504594583
503640719
508327466
503361135
507685903
510204775
503142263
502309440
512004048
508597927
502303433
505306182


Thanks
Hugo Bartolo

and Chris said...

If you want to read CSV files using SQL, you need an external table. So if you can't create one, you're out of luck. You'll have to read the data into an existing table with UTL_file first.

...unless you're on 12.2!

If there's an existing external table you can query, you can point it to a new location with the external modify clause. e.g.:

select * from ext_tab external modify (
  default directory new_dir
  location ('your_csv')
);


Though you can't override fields of the external table. So this will only work if the signature of your CSV matches that of the one the external table was built for.

You can read more about this at:

https://oracle-base.com/articles/12c/override-external-table-parameters-from-query-12cr2

If you're on 18c, you can create an inline external table. One that only exists for your query. For example:

select * from external (
  ( c1 int, c2 int, ... )
  default directory new_dir
  location ('your_csv')
);


https://oracle-base.com/articles/18c/inline-external-tables-18c

Rating

  (4 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

New info

Roelof, July 11, 2018 - 11:26 am UTC

Chris Saxon
July 11, 2018 - 4:43 pm UTC

Yup, handy if you can't create tables!

Question about external enhance

A reader, July 11, 2018 - 6:21 pm UTC

Is there any new feature makes that one can use external tables whose file location is outside the db server?
Connor McDonald
July 12, 2018 - 9:46 am UTC

Not directly. The file must be accessible from the db server. That does not mean it must *reside* on the server - you could access it via NFS or similar network share.

riding beggars

Racer I., July 13, 2018 - 7:46 am UTC

Hi,

Around here the DBAs don't give CREATE DIRECTORY rights or any kind of access to the database server to users or applications. They cite mainly concerns about backup-scenarios in ExaData-RACs.
Unfortunately external tables cannot apparently be setup as READ ONLY. They always want at least some logging and it seems you can't completely disable bad/reject-handling either even if the files remain empty.
So I've been thinking it would be very cool if client tools (like sqlplus) could tunnel a client-local directory to the database server where it would appear like a DIRECTORY object (without CREATE DIRECTORY rights). From there you could use anything external tables have to offer including the way cool preprocessor feature. I can't offhand think of any security concerns compared to what sqlldr or sqlplus can already do.
Do you think Oracle has any plans to add such a feature in the future?

regards,
Chris Saxon
July 13, 2018 - 8:58 am UTC

Create directory privileges should be tightly controlled. Being able to read, write, and execute scripts on the DB server opens the possibility of all sorts of exploits.

I can't see us implementing functionality to allow you to create directories accessing client machines. Unlike SQL*Plus etc, you're giving the database access to your machine.

possible solution on client side

Harry, August 02, 2018 - 10:29 pm UTC

You could use a little shell scripting on client side to generate some sql codes.
Here is how.

$ cat file.csv
NIF
504594583
503640719
508327466
503361135
507685903
510204775
503142263
502309440
512004048
508597927
502303433
505306182

$ cat generate_sql.sh
awk 'BEGIN {printf "with client_selected as (\n";} { \
if (NR==1) {skip;} else \
if (NR==2) {printf "select %s client_id from dual\n", $1;} else \
{printf "union select %s client_id from dual\n", $1;}} \
END {printf ")\n";}'
cat <<EOF
select
tp.process_id
from
t_process tp
, client_selected cs
where
tp.client_id = cs.client_id
;
EOF

$ ./generate_sql.sh < file.csv > generated.sql

$ cat generated.sql
with client_selected as (
select 504594583 client_id from dual
union select 503640719 client_id from dual
union select 508327466 client_id from dual
union select 503361135 client_id from dual
union select 507685903 client_id from dual
union select 510204775 client_id from dual
union select 503142263 client_id from dual
union select 502309440 client_id from dual
union select 512004048 client_id from dual
union select 508597927 client_id from dual
union select 502303433 client_id from dual
union select 505306182 client_id from dual
)
select
tp.process_id
from
t_process tp
, client_selected cs
where
tp.client_id = cs.client_id
;

$

More to Explore

Design

New to good database design? Check out Chris Saxon's full fundamentals class.