Skip to Main Content
  • Questions
  • running OS commands from DB - Some work others do not

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Steven.

Asked: April 30, 2002 - 12:16 pm UTC

Last updated: April 16, 2009 - 4:09 pm UTC

Version: 8.1.7

Viewed 10K+ times! This question is

You Asked

Hi. I followed the examples from the recent thread on the issue and have found limited success. I can get the java to run commands such as mkdir, rmdir and ls, but when trying to execute a shell script or create a file via "echo hello >> steven.out" it fails.

When I call the java class from the OS, the shell script runs fine.

I have assigned read, write and execute permissions on the shell script and the /usr/bin directory. I have assigned writeFilePermissions as well. Please could you have a quick look over what I have done and possibly suggest what I have forgotten or done wrong?

Java Class:

import java.io.*;

public class ExecCommand
{
public ExecCommand()
{
}

static public int runCommand(String cmd) throws IOException
{
// start command running
Process proc = Runtime.getRuntime().exec(cmd);

// wait for command to terminate
try
{
proc.waitFor();
}
catch (InterruptedException e)
{
System.err.println("process was interrupted");
return -1;
}

// check its exit value
if (proc.exitValue() != 0)
{
System.err.println("exit value was non-zero");
return -2;
}

// return success
return 0;
}

public static void main(String args[]) throws IOException
{
String cmdString = args[0];
System.out.println("command string is: " + cmdString);

try
{
//run a command
int retVal = runCommand(cmdString);

System.out.println("return value is: " + retVal);
}
catch (IOException e)
{
System.err.println(e);
}
}
}

** Note: when the call fails, it is always a -2 returned.

Procedure in the Database:

CREATE OR REPLACE FUNCTION Exec_Command(Command IN STRING)
RETURN NUMBER IS
LANGUAGE JAVA
NAME 'ExecCommand.runCommand(java.lang.String) return int';
/

The code:

DECLARE
RetVal NUMBER;

BEGIN
RetVal := SOSSIX.EXEC_COMMAND ( '/usr/bin/mkdir /home/stevenh/extracts/mydir' );
END;

-- creates a directory OK.

The code:

DECLARE
RetVal NUMBER;

BEGIN
RetVal := SOSSIX.EXEC_COMMAND ( '/usr/bin/sh /home/stevenh/extracts/post_files.sh c' );
END;

-- does not produce anything. A log file should be created when the shell script is run. The value c is a required parameter.

The following grants have been issued:

exec dbms_java.grant_permission('SOSSIX','java.io.FilePermission','/home/stevenh/extracts/-','read');
exec dbms_java.grant_permission('SOSSIX','java.io.FilePermission','/home/stevenh/extracts/-','execute');
exec dbms_java.grant_permission('SOSSIX','java.io.FilePermission','/home/stevenh/extracts/-','write');
exec dbms_java.grant_permission('SOSSIX','java.io.FilePermission','/usr/bin/*','execute');
exec dbms_java.grant_permission('SOSSIX','java.io.FilePermission','/tmp/sh*','read,execute');
exec dbms_java.grant_permission('SOSSIX','java.lang.RuntimePermission','*','writeFileDescriptor');
exec dbms_java.grant_permission('SOSSIX','java.lang.RuntimePermission','*','readFileDescriptor');

GRANT JAVADEBUGPRIV TO SOSSIX;
GRANT JAVAIDPRIV TO SOSSIX;
GRANT JAVASYSPRIV TO SOSSIX;
GRANT JAVAUSERPRIV TO SOSSIX;
GRANT JAVA_ADMIN TO SOSSIX;
GRANT JAVA_DEPLOY TO SOSSIX;


If I am on the AIX system, I can run:

java ExecCommand "sh post_files.sh c"

The shell script runs and the log file is created. Return code is 0.

Many thanks for any help given.

Steven.

and Tom said...

Well, first of all a 'command' like:

echo 'hello world' >> /tmp/foo

will *never work* since they are not a command but rather a string that the shell parses and interprets. This thing runs commands -- it does NOT work as a shell!

echo 'hello world'

would work but >> -- that is a shell thing, period.


99.99% of the time what goes wrong with shell scripts is:

o the ORACLE software owner does not have rx on the directories
o the ORACLE software owner does not have rx on the script (remember, it runs as oracle)

o the shell script tries to write or read a file ORACLE is not allowed to.


I WOULD STRONGLY encourage you to NOT grant permission on /usr//bin/* -- that lets you run the sh and that lets you do ANYTHING (warning lights should be going off)




Rating

  (18 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, May 01, 2002 - 4:32 am UTC

Thanks, I granted execute on /usr/bin/sh as I had to include the sh in the command string when running the class standalone otherwise it wouldn't work.

"o the ORACLE software owner does not have rx on the directories
o the ORACLE software owner does not have rx on the script (remember, it runs as oracle)

o the shell script tries to write or read a file ORACLE is not allowed to. "

I used chmod 777 on the directory the shell script is in and the same for the shell script itself.

I know the access rights are lax at the moment but I am just trying to get the damn thing working through the database at the moment.

Tom Kyte
May 01, 2002 - 7:21 am UTC

and so did it work? you don't say.

Apologies

Steven Hunter, May 01, 2002 - 9:18 am UTC

Sorry, my previous response should have said that I had ALREADY granted the full access rights to the directory and shell script before submitting my question to you. Therefore I am still in the same posititon.

Tom Kyte
May 01, 2002 - 9:45 am UTC

you say:

/home/stevenh/extracts/post_files.sh



should create a file.

Where does it attempt to CREATE this file and does the OS user that is running the database have the ability to CREATE that file in that place. (su to oracle, run that script. does that work?)



Ran script as oracle user

Steven Hunter, May 01, 2002 - 11:17 am UTC

su-ed to oracle, changed to the /home/stevenh/extracts directory and ran post_files.sh - it failed.

Tried ./post_files.sh - it worked and created the log file.

The log file is created in the current directory.

Steven.

Tom Kyte
May 01, 2002 - 12:39 pm UTC

use a fixed directory, like /tmp/ and try it again.

tell me -- what directory is the "current" directory when the java runs? sort of DANGEROUS to write just to the current directly, no idea what you might clobber.



You can use pipes in system commands run from database

Eric Kamradt, May 01, 2002 - 5:38 pm UTC

You can execute a pipe command in the database. You just need to run it through a shell. The following command worked inside the database on my laptop, run from PL/SQL.

bash -c "echo My name is TwoPigs > t.txt"

One could modify the java code to also return the results of standard out and err. see (</code> http://www.ckamco.com/Execute_Command.zip <code>


Tom Kyte
May 01, 2002 - 9:25 pm UTC

Yes, I think I said that you would have to run a shell -- that commands like:

"echo hello >> steven.out"

are not COMMANDS but things the shell does. I also strongly suggest (encourage, demand) that you do not use the SHELL directly as that leaves a security hole just big enough to drive a truck through in your database!


</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:952229840241 <code>

is the code I provide to get the output back as well.

STILL having trouble executing a shell script...

Adrian Billington, June 07, 2002 - 6:49 am UTC

Tom

I know this is not the place but being on the other side of the Atlantic, I can never get in to ask a question through the correct channel...

Anyway, I've read and followed this thread to the letter. I'm using your code line for line and have setup two tests. One is with execute granted to /usr/bin/ls and the other to my test shell script. This is all working as expected and I can exceute both.

Ultimately I want to be able to run a couple of ETL programs that our third-party supplies from within the database (one converts data to ASCII and the other loads it. Both programs write logs out. To emulate these I've used the ostest.ksh script to run sqlldr and generate a log and bad file in the process.

Now the SQL*Load part works fine but I'm getting no log or badfile. I've used /tmp as everyone has write access, but to no avail. Any ideas?

I'm using 8.1.6.3 on Solaris 8. I have the following permissions:-

KIND TYPE_ TYPE_NAME NAME ACTION ENABLED
-------- ----- ------------------------------ ---------------------------------------- -------------------- --------
GRANT SYS java.io.FilePermission /export/home/abill2/work/OS/ostest.ksh execute ENABLED
GRANT SYS java.io.FilePermission /usr/bin/ls execute ENABLED
GRANT SYS java.lang.RuntimePermission * writeFileDescriptor ENABLED

Regards

Adrian

Tom Kyte
June 07, 2002 - 7:13 am UTC

lets see ostest.ksh

I goofed

A reader, June 07, 2002 - 9:36 am UTC

Sorry Tom, I goofed - it works and I get my log and badfile. I'll try not to waste your time next time ;)

Regards

Adrian

Exit codes

Adrian Billington, June 11, 2002 - 7:04 am UTC

Tom, despite my last "goof" post, I am in fact having some strange results from executing a shell script. I am consistently getting a return code of 255 from the ls command it contains.

I am now using your JAVA source and RUN_CMD function EXACTLY. I have used your p_host_cmd procedure and my own anon/stored PL/SQL to execute the OS commands all to the same effect. I have wrapped your function in a utility package I have and still get the same results.

I have execute permissions on ls and ostest.ksh.

I am executing '/usr/bin/ls -l /tmp' and '/export/home/abill2/work/OS/ostest.ksh' separately.

The ls call works fine with a 0 return.

ostest.ksh contains one line only (/usr/bin/ls -l /tmp), has 777 on file and directory and returns 255 EVERY time. Because I am not getting -1, this tells me it is not raising an exception within the java itself.

As an extra, when I was testing sqlldr using ostest.ksh and deliberately generating a bad record, I was getting a return of 2, which is correct and what you would expect from sqlldr.

I am very perplexed.

Regards

Adrian

Tom Kyte
June 11, 2002 - 1:31 pm UTC

post the entire ostest.ksh script -- does it have a first line? (#!/bin/ksh or something) as well (otherwise it is a sh script, maybe not entirely relevant but want all of the details).


I just did a test where I ran:

rt_test@ORA817DEV.US.ORACLE.COM> create or replace procedure RC(p_cmd in varchar2)
2 as
3 x number;
4 begin
5 x := run_cmd(p_cmd);
6 dbms_output.put_line( 'return code = ' || x || ' from ' || p_cmd );
7 end;
8 /

Procedure created.

rt_test@ORA817DEV.US.ORACLE.COM> show errors
No errors.
rt_test@ORA817DEV.US.ORACLE.COM> /

Procedure created.

rt_test@ORA817DEV.US.ORACLE.COM>
rt_test@ORA817DEV.US.ORACLE.COM>
rt_test@ORA817DEV.US.ORACLE.COM> set serveroutput on size 1000000
rt_test@ORA817DEV.US.ORACLE.COM> exec dbms_java.set_output(1000000)

PL/SQL procedure successfully completed.

rt_test@ORA817DEV.US.ORACLE.COM> /

Procedure created.

rt_test@ORA817DEV.US.ORACLE.COM> exec rc('/export/home/tkyte/work/OS/ostest.ksh' )
total 115872
-rw-r--r-- 1 tkyte other 343 Nov 19 2001 10514.ksh
.....
-rw-r--r-- 1 root other 418 Jul 11 2001 zz
return code = 0 from /export/home/tkyte/work/OS/ostest.ksh

PL/SQL procedure successfully completed.



then I went out to the OS and did:

$ chmod a=r,u=rwx work

$ ls -ldg work
drwxr--r-- 3 tkyte other 512 Jun 11 13:26 work

and then:

rt_test@ORA817DEV.US.ORACLE.COM> exec rc('/export/home/tkyte/work/OS/ostest.ksh' )
return code = 255 from /export/home/tkyte/work/OS/ostest.ksh

PL/SQL procedure successfully completed.



So, I believe it is a file permission problem or something SIMILAR at the OS level once again. Make sure the ENTIRE DIRECTORY PATH is readable and executable


Spot on...

Adrian Billington, June 11, 2002 - 5:13 pm UTC

Tom

ostest.ksh
----------
/usr/bin/ls -l /tmp

Result: 255 and ls -l not executed.

You were spot on when you mentioned #!/bin/ksh. I'm kicking myself here because I only left it out due to my eagerness to get into the JAVA :). When I added the hash-bang to ostest.ksh, I got a 0 and the ls listing. I'm surprised the sh can't run the ls but nonetheless, I always use Korn anyway (well almost always ;) ). A very sloppy mistake on my part.

Incidentally, our DBA noticed that my tests were leaving quite a few "objects" lying around attached to the processes (SQL*Plus) I was running on. He suggested I p.destroy() them at the end of the try{ } section as good practice (obviously they would clear out when I exited SQL*Plus).

Thanks again.

Adrian

Tom Kyte
June 11, 2002 - 7:02 pm UTC

Kudos to your DBA! Well done. Not many would have been able to trace that back and determine the cause (they might just kill -9 them ;)

where to find errors returned?

George, June 14, 2002 - 3:17 pm UTC

I have reviewed a number of the postings about running an OS command through JAVA, and think I have followed them
correctly..  but I am now getting the following message
when I try to execute:

SQL> exec OS_EXECCMD( 'ps -ef' )

java.lang.ArrayIndexOutOfBoundsException
at OSUTIL.ExecCmd(OSUTIL:14)

I have copied an example exactly, then changed the name..
here are the first few lines in case the 14 is a line number

CREATE OR REPLACE and COMPILE JAVA SOURCE
NAMED "OSUTIL"
AS
import java.io.*;
import java.lang.*;

public class OSUTIL extends Object
{

  public static int ExecCmd(String[] args)
  {
    Runtime rt = Runtime.getRuntime();
    int        rc = -1;

    try
    {
       Process p = rt.exec(args[0]);

       int bufSize = 4096;
       BufferedInputStream bis =
        new BufferedInputStream(p.getInputStream(),bufSize);
       int len;
       byte buffer[] = new byte[bufSize];

....

I have also granted execute on everything "*" - (1: I'm just
testing the solution, 2: that access is only for 1 restricted user.)  

anyway, can you suggest where I might go to debug this?

thanks, George 

Tom Kyte
June 14, 2002 - 3:20 pm UTC

see
</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:952229840241 <code>
and look for ArrayIndexOutOfBoundsException





running connect and disconnect in plsql

A reader, December 20, 2002 - 10:25 am UTC

Hi

How can we run connect and disconnect commands (sqlplus) inside Pl/SQL?

Tom Kyte
December 20, 2002 - 10:36 am UTC

you cannot - think about it. where does PLSQL run? (it runs in the SERVER). If you actually disconnected from PLSQL where would you be? (nowhere). That would be sort of like an out of body experience.

If you want a plsql routine to access many databases -- we use database links for that.

SY, March 28, 2003 - 6:25 am UTC

Wow, Tom, thanks for your 'RUN_CMD' java code , that's great.

And now, I was asked to export/import daily data from DB server X to DB server Y, all procedures must run on DB Y (security reason and easy to manage).

I can do it through DB link. But some DBAs said the export will impact the performance of DB X which work as a main OLTP server, they asked me to export the data from user m of DB X to user n of DB X, and then use DB link to transfer the data from DB X to DB Y.

I think the procedures at DB Y using DB link to export data from user m to user n (m & n are all in DB X) will cause very low performance, so I plan to use your 'RUN_CMD' java code to do the job, e.g. DB Y>exec rc('SQLPLUS user/pws@DB_X @C:\1.sql');

But it seems there are some better ways I don't know...
Any comments? Thx in adv. :)


Tom Kyte
March 28, 2003 - 7:19 am UTC

oh my -- your DBA's want to double the work in order to reduce the work? I believe they have the concept backwards.

They are saying "reading the data from user M will affect performance so instead of reading the data from user M ONCE we want you to

o read the data from user M and write it to user N -- so you read the data anyway AND you add the additional overhead of writing, creating undo, redo etc in this important database

o read the data AGAIN from user N -- effectively doubling the IO it takes to achieve your goal.


So instead of:

o read user M


they want you to

o read user M
o write it all to user N
o read user N



Is that what you are saying???? If so, I'm really not understanding their "thought process" at all.

SY, March 29, 2003 - 2:13 am UTC

Thanks,Tom! Very Clear suggestion!

1. You catch my mean. The DBAs have their opinion: As user m and user n belong to different tablespace on different disks, so the their IO are isolate, and read/write in the same DB is faster than dblink on different DB, so impact will be least.

And your words tell me , that different users in the same DB have the same IO/LOG/Process, right? NEW knowledge ! :)

2. Tom , do you think that the best way to complete the job is still using DBLINK to copy directly?



Tom Kyte
March 29, 2003 - 10:00 am UTC

1) opinions are meaningless in situations like this.

Ask them "would it be ok if I setup a process to go into an infinite loop -- I mean, I won't be hitting your disk so doing all of that work should be OK with you right?"

Reallly --

In the user to user to dblink they have to

read user m's data (X ios)
write it to user n's place (Y ios to another disk, plus Z ios to log, plus M cpu cycles to perform operations, plus N ios to rollback)
read user n's data over dblink (X ios all over again plus the probable block cleanouts which will generate redo as well)
clear out user n's data

or then cay

read user m's data (X ios)


I cannot fathom the though process whereby they are thinking #1 is "optimal".


2) umm, yes.

trace file permission changing in udump

Mike, August 05, 2003 - 10:04 am UTC

Tom,
I have an interesting issue with the permission changing on the trace file in the UDUMP.

-rw-r--r-- 1 oracle dba 627 Jul 19 10:33 s0p2dbd2_ora_10929.trc
-rw------- 1 oracle dba 625 Jul 19 15:52 s0p2dbd2_ora_1510.trc
-rw------- 1 oracle dba 625 Jul 19 16:29 s0p2dbd2_ora_1559.trc
-rw-r--r-- 1 oracle dba 625 Jul 21 10:33 s0p2dbd2_ora_6358.trc

You may notice that the permissions not consistent over the time. The contains are similar. Does the permission have to do with the “Unix process pid”.

$ more oraclebase_test/product/8.1.7/admin/S0P2DBD2/udump/s0p2dbd2_ora_10929.trc
Dump file /oraclebase_test/product/8.1.7/admin/S0P2DBD2/udump/s0p2dbd2_ora_10929.trc
Oracle8i Enterprise Edition Release 8.1.7.4.0 - Production
With the Partitioning option
JServer Release 8.1.7.4.0 - Production
ORACLE_HOME = /oraclebase_test/product/8.1.7
System name: SunOS
Node name: cpc56
Release: 5.8
Version: Generic_108528-19
Machine: sun4u
Instance name: S0P2DBD2
Redo thread mounted by this instance: 1
Oracle process number: 11
Unix process pid: 10929, image: oracle@cpc56 (TNS V1-V3)

*** 2003-07-19 10:33:42.844
*** SESSION ID:(10.1) 2003-07-19 10:33:42.797
Registered presentation </code> http://admin
on database startup


$ more /oraclebase_test/product/8.1.7/admin/S0P2DBD2/udump/s0p2dbd2_ora_1510.trc 
Dump file /oraclebase_test/product/8.1.7/admin/S0P2DBD2/udump/s0p2dbd2_ora_1510.trc
Oracle8i Enterprise Edition Release 8.1.7.4.0 - Production
With the Partitioning option
JServer Release 8.1.7.4.0 - Production
ORACLE_HOME = /oraclebase_test/product/8.1.7
System name:    SunOS
Node name:      cpc56
Release:        5.8
Version:        Generic_108528-22
Machine:        sun4u
Instance name: S0P2DBD2
Redo thread mounted by this instance: 1
Oracle process number: 11
Unix process pid: 1510, image: oracle@cpc56 (TNS V1-V3)

*** 2003-07-19 15:52:21.146
*** SESSION ID:(10.1) 2003-07-19 15:52:21.105
Registered presentation 
http://admin
on database startup


$ more /oraclebase_test/product/8.1.7/admin/S0P2DBD2/udump/s0p2dbd2_ora_6358.trc    
Dump file /oraclebase_test/product/8.1.7/admin/S0P2DBD2/udump/s0p2dbd2_ora_6358.trc
Oracle8i Enterprise Edition Release 8.1.7.4.0 - Production
With the Partitioning option
JServer Release 8.1.7.4.0 - Production
ORACLE_HOME = /oraclebase_test/product/8.1.7
System name:    SunOS
Node name:      cpc56
Release:        5.8
Version:        Generic_108528-22
Machine:        sun4u
Instance name: S0P2DBD2
Redo thread mounted by this instance: 1
Oracle process number: 11
Unix process pid: 6358, image: oracle@cpc56 (TNS V1-V3)

*** 2003-07-21 10:33:57.703
*** SESSION ID:(10.1) 2003-07-21 10:33:57.642
Registered presentation 
http://admin <code>on database startup
$


Tom Kyte
August 05, 2003 - 1:09 pm UTC

they are quite old, simple chmod by someone would do that.

Mike, August 05, 2003 - 11:53 am UTC

I have checked the parameter:
_trace_files_public = true

Don't see the problem, see the solution

Edgar, June 20, 2005 - 5:09 am UTC

Tom,

Below is a more or less generic Java-programmer's way to execute OS commands.
Have You the solution, how to adopt this sample code to pass stderr & stdout streams to DBMS_OUTPUT package?

import java.util.*;
import java.io.*;

class StreamGobbler extends Thread
{
InputStream is;
String type;

StreamGobbler(InputStream is, String type)
{
this.is = is;
this.type = type;
}

public void run()
{
try
{
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);

String line=null;
while ( (line = br.readLine()) != null)
System.out.println(type + "> " + line);
} catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}

public class OScmd
{

public static int exec(String cmdToExec)
{

try
{
String osName = System.getProperty("os.name" );
String[] cmd = new String[3];

if( osName.equals( "Windows NT" ) )
{
cmd[0] = "cmd.exe" ;
cmd[1] = "/C" ;
cmd[2] = cmdToExec;
}
else if( osName.equals( "Windows 95" ) )
{
cmd[0] = "command.com" ;
cmd[1] = "/C" ;
cmd[2] = cmdToExec;
}
else
{
cmd[0] = "/usr/bin/sh" ;
cmd[1] = "-c" ;
cmd[2] = cmdToExec;
}

Runtime rt = Runtime.getRuntime();
System.out.println("Executing external command: " + cmdToExec);
Process proc = rt.exec(cmd);

// any error message?
StreamGobbler errorGobbler = new
StreamGobbler(proc.getErrorStream(), "stderr");

// any output?
StreamGobbler outputGobbler = new
StreamGobbler(proc.getInputStream(), "stdout");

// kick them off
errorGobbler.start();
outputGobbler.start();

// any error???
int exitVal = proc.waitFor();
System.out.println("External command exit value: " + exitVal);
return exitVal;

} catch (Throwable t)
{
t.printStackTrace();
return -1;
}
}
}



Tom Kyte
June 20, 2005 - 10:22 am UTC

anything you write to system.out can be redirected using dbms_java.set_output


I would rethink that "standard way". you don't want to run cmd.exe or any of the command.com's.

You want to run individual commands and use dbms_java to grant the priv to do so on EACH individually.


else, you can run *anything* as Oracle on that machine, and that would be "not a good thing"

regarding previous post,

Edgar, June 20, 2005 - 5:16 am UTC

.. or, may be, it is worth idea not to DBMS_OUTPUT, but use temporary table, for example, to grab OS command stderr & stdout and pass messages to user session (PL/SQL) this way..


Ough, sorry, i am realy to fussy man, have to improve myself

Edgar, June 20, 2005 - 5:47 am UTC

I find the answer on this site:
</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:439619916584 <code>

==>dbms_java

execute directory

Jose Guajardo, April 16, 2009 - 3:29 pm UTC

Hi Tom and I followed the instructions to the directory permissions

dbms_java.grant_permission (
SCHEMA,
'SYS: java.io.FilePermission',
'D: / oracle/ora92/database/installfop/ejecutar.bat'
'execute'
);

running the bat, but the instruction executed in the D: / oracle/ora92/database /

how can I change the directory where the command is executed in
"D: / oracle/ora92/database/installfop"

thanks
Tom Kyte
April 16, 2009 - 4:09 pm UTC

you would fully qualify the path - that is good for security and the right approach, that way you KNOW what script you are running.


More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library