Karthick, September 08, 2009 - 7:06 am UTC
Oracle has lots of products developed with Java as client. So how it does the SQL calls? Are the SQL wittern in client itself and called or they store them in procedures and call the procedure.
My main objective is to keep the SQL in one place so that for example, if i create a new index on a table i would like to find out how will it affect the application. There could be lots of DML written over that table.
So i would like to collect all the DML and do a execution plan and see what what plan CBO has got for me. And i just felt it would be easy if the SQL was available in a table. Thats why i thought of this.
September 08, 2009 - 7:22 am UTC
... Are the SQL wittern in client itself and called or they store them
in procedures and call the procedure.
...
yes, you will find BOTH are used. If I had my way - only the latter would be uesd.
... So i would like to collect all the DML and do a execution plan and see what
what plan CBO has got for me. And i just felt it would be easy if the SQL was
available in a table. ....
and when you see this plan - what then? Can you look at a plan and with 100% degree of certainty know "it is the right plan" (no, you cannot)
and when you use explain plan - and it does not bind peek - how do you know the plan you are looking at will even be used? (you do not, explain plan 'lies')
and when you use explain plan - and it assumes all binds are strings - how do you know the plan you are looking at will even be used? (you do not, explain plan lies)
and so on - no, this is not something you want to do. Especially for this - not for this reason, abandon this approach right now for that reason.
If you put the sql into stored procedures, you'll have a dependency set up so that if you are thinking of adding an index - you can immediately find ALL of the SQL that might be affected (it is in the procedures that are dependent on that table, we store that for you)
and you can use automated tools like the sql performance analyzer
http://search.oracle.com/search/search?default=true&keyword=sql+performance+analyzer&start=1&nodeid=&fid=&showSimilarDoc=true&group=Oracle+Technology+Network but please, do not do this in the manner you are suggesting, you would be adding a layer of obfuscation here that will make it impossible to maintain or understand what the program is doing, you would lose the dependency mechanism, you would negatively impact performance in a HUGE way (you would be parsing constantly - over and over and over), you would be making development tedious at best (insert a row in a table, hope that it was the right sql syntax, then make sure to read the sqlid back out, remember it, put it into the application and hope no one touches your sql later - the debug test process is hugely impacted in a negative fashion)
In a scale of 1 to 10 for "bad idea" - 1 being the least bad idea and 10 being the worst bad idea, this ranks near 10, if not 10 itself.
Nonsense
Analyst, September 08, 2009 - 8:06 am UTC
karthick,
The content of your post leads me to ask if you have any experience developing software in any capacity? I shall respond below to each of the points that you have raised in your second post.
"Oracle has lots of products developed with Java as client."
So, what is your point? There are many applications written in language X that access an Oracle database.
"So how it does the SQL calls?"
The same way that SQL has been called for over a decade: stored procedures. Have you ever read a line of Oracle documentation? Have you not heard of a stored procedure?
"Are the SQL wittern in client itself and called or they store them in procedures and call the procedure."
That is perhaps the most incoherent piece of rubbish I have ever read. I believe you are trying to ask a question about where SQL code is stored. Again, have you never read a line of database documentation? SQL is stored in the database where it belongs. Yes, you might be surprised to learn that SQL code is actually stored in a database.
"My main objective is to keep the SQL in one place so that for example"
And that is why the only place to ever store SQL is in the database!!!!! This is not negotiable. SQL is database code. It belongs in the database!! If you store SQL code in the database (which is the only place it should be deployed) then it will all be stored in one place!!! I cannot believe that in the year 2009 there are people who cannot comprehend the fact that SQL code should be deployed only ever in the database!
"And i just felt it would be easy if the SQL was available in a table"
Why?!?!?! What total nonsense! What could be easier than:
procA
There is no easier way to reliably and sensibly invoke SQL from an application than to call a stored procedure!
Karthick, September 08, 2009 - 8:47 am UTC
To the analyst from melbourne
<quotes>
"Oracle has lots of products developed with Java as client."
So, what is your point? There are many applications written in language X that access an Oracle
database.
</quotes>
Different software work in different way. for instance with APEX i learned that you don't have to write the SQL it generates for you. so i wanted to be more specific on which software i am using.
<quotes>
"So how it does the SQL calls?"
The same way that SQL has been called for over a decade: stored procedures. Have you ever read a
line of Oracle documentation? Have you not heard of a stored procedure?
</quotes>
I beg to differed here. I have seen applications calling SQL from the presentation layer itself. And even tom did confirm that by saying this
<tom's Quotes>
yes, you will find BOTH are used. If I had my way - only the latter would be used.
</tom's Quotes>
<quotes>
I believe you are trying to ask a question about where SQL code is stored.
</quotes>
NO, Where my SQL Code written for the application must be stored.
<quotes>
Again, have you never read a line of database documentation? SQL is stored in the database where it belongs. Yes, you might be surprised to learn that SQL code is actually stored in a database.
</quotes>
Oracle does not store SQL Code in its data dictionary. Only the procedure code and View code is stored in database. Native standalone SQL are just cached temporarily. They are never persistent.
<quotes>
"My main objective is to keep the SQL in one place so that for example"
And that is why the only place to ever store SQL is in the database!!!!! This is not negotiable.
SQL is database code. It belongs in the database!! If you store SQL code in the database (which is
the only place it should be deployed) then it will all be stored in one place!!! I cannot believe
that in the year 2009 there are people who cannot comprehend the fact that SQL code should be
deployed only ever in the database!
</quotes>
Again, When you say Database please be more specific. My approach also stores the SQL in the Database. Let me guess it as PL/SQL when you say Database.
September 08, 2009 - 9:35 am UTC
Again, When you say Database please be more specific. My approach also stores
the SQL in the Database. Let me guess it as PL/SQL when you say Database.
i'll be very specific:
a) drop your table, drop your procedure, drop that idea
b) encapsulate the sql in a stored procedures, a transactional API would be best (not a procedure for each insert/update/delete - but rather a set of procedures that perform a transaction)
and you'll not be sorry you did.
Look at it from a java point of view
Galen Boyer, September 08, 2009 - 11:38 am UTC
Since maybe you are not getting just how bad your idea is,
try to think about it from a java point of view.
Suppose I'm a database developer asked to do some java coding. Here
is my design because I don't want to have to learn how to code java.
I have designed a generic java class that knows how to iterate a hash
table finding a row in that table. Within that hashed row is a
"string". That string contains executable java. Once my generic java
class finds this string, it does the following.
1) Compiles the code
2) Deploys the compiled code to the app server
3) Instantiates an object of that class
4) Asks the object to do its operation
5) Sets the object to null".
That's my java design. Do you like it? Is it sound architecture? All
of my java code is in one place, one simple hash table. I would think
that would be highly maintainable, and I shouldn't lose any
performance as well, correct?
That analogy is quite akin to what you are planning to do with your
database design.
perfect counter-example!
Duke Ganote, September 08, 2009 - 1:21 pm UTC
@Galen Boyen: Excellent; I'd been mulling over the same counter-example! We'll just put all the java code into a database table...
sweet!
chuck, September 08, 2009 - 9:04 pm UTC
How about this?
Give the developer(s) a schema and table(s) to keep the sql in.
Not used for code, consider it documentation.
You could even create parent child relations with lists of where it was used, binds, notes, whatever.
You could get sql from this schema for testing any time you wanted.
A lot of trouble to maintain that.
But not as much trouble as the corner you are fixin' to paint yourself into.
hmmm...
Marvin, September 09, 2009 - 4:07 am UTC
Having monitored ask tom for number of years I am amazed at the number of times that some people forget that the real point of the forum is to educate/help people in their understanding of how best to work with Oracle and not use it as a vehicle to promote their own ego (Mr Melbourne). This thread just demonstrates The worst and best approach to helping a fellow developer thankfully most people adopt the best approach and for that I'm very grateful.
Response to the point about educating people
Analyst, September 09, 2009 - 5:11 am UTC
I have received in the past help from many people from several database websites, and am very grateful for their contribution.
However, and this is important, before seeking the advice of other people, I always ensured that I had read and sought to understand some of the documentation. Moreover, I conducted extensive research from many sources - books, internet websites, forums - to be in a position to submit a more informed opinion.
I do not believe the original poster had made a sufficient effort in these respects. Of course, we are always willing to help others. However, there are some questions that, if read by an inexperienced developer, could lead to very significant development mistakes. It was my intention, in my replies to this thread, to stress the severity of the consequences that could follow if somebody were to ever consider the approach mentioned by the original poster.
Dear Analyst from Melbourne
Mette, September 09, 2009 - 6:23 am UTC
Take a deep breath and count to 10.213.3242.2342 or a similar number :-)
I don't consider myself or my organization stupid, incompetent etc - but still we have lots of VB and Java programs with SQL imbedded, because this was the way it was done 10-15 years ago (coming from SQL Server and Informix).
I do not like Kartlicks solution either, but making him look like a total idiot and newbee, does not educate him or others - it just pisses people off !
Yes, I would prefer SQL to be in the database - BUT, it is not always the DBA that has the final word on that. At my company the projects choose the tools and the way they want to implement it (and since we have lots of java people and not many pl/SQl people you might guess the result). And sometimes we employees have to live with those decisions (good or bad) and make the most of it on these terms.
And also with ie SAS code (we have lots of that) - SAS can call a stored procedure, but can't get data from it (no ref cursors). So here we have to use SQL from within SAS ...
So please .... take that deep breath once again
Mette
Mette from Denmark
Analyst, September 09, 2009 - 7:47 am UTC
"Yes, I would prefer SQL to be in the database - BUT, it is not always the DBA that has the final
word on that."
Whether or not a DBA is available a a resource to a project is irrelevant. SQL must only be stored in the database. If the project team decides otherwise, then the team is completely incompetent in developing database systems. Do you remember the saying: ignorance is not an excuse! Consider an analogy of building a bridge. If one team does not have the capabilities to build the bridge to the required standard, then the only solution is to hire a team that can. Your comment above suggests that you would simply build the bridge not to the specification because you do not have the necessary skills. Total nonsense!
"At my company the projects choose the tools and the way they want to implement it
(and since we have lots of java people and not many pl/SQl people you might guess the result)"
Then your company's management should consider getting some courage and making decisions that are in the best interests of the company. And when developing information systems, the correct decision is to hire people who have skills and experience in database development.
"And
sometimes we employees have to live with those decisions (good or bad) and make the most of it on
these terms."
That is perhaps the most ridiculous statement I have ever read. An employee, or anyone for that matter, should never live with a decision over which they have some degree of influence. One can only hope that such people never work in an industry in which a mistake can result in fatalities.
"And also with ie SAS code (we have lots of that) - SAS can call a stored procedure, but can't get
data from it (no ref cursors). So here we have to use SQL from within SAS ..."
If what you say above is correct, then you are suggesting that SAS is not able to offer a satisfactory level of support for the capabilities of the Oracle database.
Karthick, September 09, 2009 - 7:58 am UTC
This is an idea just popped into my mind. So i just gave it a shot. This is not a design that i am proposing to any one. When some idea or thought comes in, its better to share and get it evaluated. I saw this a good place to do.
Being a participant in public forums for a wile i know people at times get brutal in there comments. I don't mind it, untill its productive for me.
Ok its good that we have a unanimous decision that this design is "VERY BAD".
I guess following are the reason for that
1. SQL in a table is not reliable. When its in a procedure its parsed and its syntax and semantic check are already done. So it ensures the code is proper. But when i have it in a table i cant ensure that untill i actually run it.
2. Version management becomes complicated. Again which leads to the reliability of the code being compromised.
3. Development activity gets complicated, because developer will have a hard time in passing the proper bind values and processing the SQL.
I think there should be more reasons in the perspective of Salability and Performance. I would like to know why this approach will not be scalable and what performance bottleneck will i be facing.
Please feel free to point me to some articles or books and say "first read this stupid" ;)
Thanks,
Karthick.
September 14, 2009 - 9:59 am UTC
This is an idea just popped into my mind. So i just gave it a shot. This is not
a design that i am proposing to any one. When some idea or thought comes in,
its better to share and get it evaluated. I saw this a good place to do.
Karthick - I agree totally - however, if you re-read your original question - it wasn't phrased as "I'm thinking of doing X for reason Y". It was rather "how do I do Z".
That is why I always push back and say "this is a bad idea, what are you trying to do - what is X and Y, don't give me Z"
... I think there should be more reasons in the perspective of Salability and
Performance. I would like to know why this approach will not be scalable and
what performance bottleneck will i be facing. ...
You will tend to over parse, your generic solution will not be able to cache cursors effectively like normal PLSQL would (put your sql into plsql, turn on tracing, run the procedure 500 times - tell me how many times the SQL in your plsql was *parsed*, it'll be about once, not 500 times). Of course you could add more and more and more procedural code to do this, but then you would be.....
reimplementing Oracle, in Oracle, using an interpreted language.
How about this, no books necessary:
Static is more performant than generic, generic sounds cool but is less maintainable, performs (in general) poorly, locks you into specific versions/ways of doing things (you work in the confines of the mini-api you built).
Generic code is something you do when forced, not as a default. And you never really have to be forced :)
Happy to offer advice
Analyst, September 09, 2009 - 8:10 am UTC
Reviewing a few of my comments above, I realise that they might have given the wrong impression. And for this, I apologise. However, I cannot apologise for my intention in seeking to point out the fact that the design that was proposed was "very bad".
karthick,
The fact that your procedure was non-trivial in terms of its technical implementation led me to conclude that you had experience in development and that, on this occasion, you were seeking advice on an approach that would ignore several very significant capabilities of the database. As such, I felt compelled to adopt perhaps a more combative tone in my reply. Remember: your code was non-trivial, and so you certainly had an idea about what you were doing.
Finally, I should say that I had not sought to make the poster look like an idiot. Indeed, far from it. Leaving aside the tone of a few of my posts, I do hope that the content has been helpful.
A reader, September 09, 2009 - 8:19 am UTC
...you were seeking advice on an approach that would ignore several very significant capabilities of the database
would love to hear more on this..
September 14, 2009 - 10:00 am UTC
you already listed many of them right above??
Mr Holiness from Melbourne
Mette Stephansen, September 09, 2009 - 8:59 am UTC
"That is perhaps the most ridiculous statement I have ever read. An employee, or anyone for that
matter, should never live with a decision over which they have some degree of influence".
Have you ever heard of lost battles?? I dont fight lost battles anymore .... and what do yopu mean by "some degree of influence" ? Do you honestly think that everyone has an influence on weird decisions??
And it is a very lost battle to tell a major finacial institution not to use SAS because it cant use ref cursors.
But anyway: You might get along being rude telling other people how studig you think we are - good for you - I prefer to be stupid (in you opinion) and polite if the choice is between the two.
Have a nice day
Mette
PS There are lots of decisions that I disagree with: political, relationshipwise, IT, management, database etc. I use my influence where I can, but I dont end up beeing a pain the ass, never knowing when to stop arguing - some causes are lost battles. I make my statement (voting, debating etc) - and then I either live with it or shuts the fuck up !
Carsten, September 09, 2009 - 10:58 am UTC
Don't be too frustrated Karthick, the idea was not really bad, but doing it yourself was the wrong way and Oracle was years ahead of you in this idea.
Try to see it this way:
Storing SQL in tables is a good idea. Actually I am pretty sure Oracle utilizes (internal) Tables to store your procedures.
But don't reinvent the wheel, it is all done: you have a interface to store and update the stored SQL even with syntax checks ("create or replace procedure..."), there is an finished interface to use the SQL with binds,return values and even rights management for your java developers ("call xxx") and as Tom pointed out there are even finished views to find SQL that accesses a given table (select * from all_dependencies).
On top of that you get a free tool from Oracle to manage the "stored SQL" called SQL developer, tools to transfer the SQL from development to production (export, import).
They even added advanced storing methods called packages to maintain lots of lots of SQL.
PL/SQL is the finished implementation of your idea. Ok they added a few extra features, but it works perfect to fit your needs.
September 14, 2009 - 10:11 am UTC
well said :)
Of course you can store SQL in a table!
David Aldridge, September 09, 2009 - 12:47 pm UTC
There being an exception to every rule, let me just mention that Oracle has a very useful feature that does leverage the flexibility of being able to store SQL in a table for later evaluation. By coincidence I've been playing around with it today, and very useful it seems to.
To paraphrase the documentation:
"This simplifies SQL queries; allows expressions to be inserted, updated, and deleted without changing the application; and enables reuse of conditional expressions by separating them from the application and storing them in the database."
I'm just evaluating it for use in a data warehouse transformation process that has to categorise a data set, with each item of data potentially falling into multiple categories. Of course, nobody knows what the categories are, or how to identify them ... "We'll know when we see the data".
Anyway, I speak of course of Expression Filters.
http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14288/exprn_expconcepts.htm#BABHFCIC
brain explosion
A reader, September 10, 2009 - 2:23 pm UTC
Holly Toledo! :o
EXPRESSIONS seems to be the equivalent of putting a bunch of different "WHERE" clauses in a table and then mass evaluate them in a single SQL statement!
The major thing to observe here is that the column containing said WHERE clauses can be indexed.
The 'CREATE INDEX ... INDEXTYPE ...' tells me that Oracle is using Oracle Cartridge technology to get this done.
Still, with this complexity, I can only see this being useful if it is part of a procedure so that oracle can do something with the results immediately afterward.
David Aldridge, September 11, 2009 - 4:50 am UTC
Not just as part of a procedure, though that's probably how it would be used in the context of the full rules manager implementation.
You can create a table of expressions, with data such as ...
"expression_name","expression"
"CUSTOMER_IS_ALIVE","DATE_OF_DEATH IS NULL OR DATE_OF_DEATH > SYSDATE"
"CUSTOMER_IS_MARKED_FOR_DEATH","DATE_OF_DEATH > SYSDATE"
"CUSTOMER_IS_DEAD","DATE_OF_DEATH < SYSDATE"
"CUSTOMER_IS_NOT_YET_CONCEIVED","DATE_OF_BIRTH>ADD_MONTHS(sysdate,9)"
... and join your customer table to the expression table and apply the expression filter to see which rules match which customers ...
select customer_name, expression_name
from customers c, expressions e
where Evaluate(e.expressions,
customer_type.get_varchar(c.date_of_birth,
c.date_of_death))
/
So multiple expressions match multiple customers etc..
With the expression filter code in place, the expression column in the expression table is protected by triggers to ensure that only valid expressions can be entered.
Fun stuff. Not neccessarily high performance (certainly not as good as implementing the expressions as SQL), but the flexibility is excellent. I can think of a lot of applications for this in areas such as marketing where requirements can change five times in the time that it takes to get a single code change through the bureaucracy.
A reader, September 11, 2009 - 11:31 am UTC
...
Fun stuff. Not neccessarily high performance (certainly not as good as implementing the expressions
as SQL), but the flexibility is excellent.
...
Out of curiosity, do you happen to have any numbers for this?
I can see each individual SQL statement running faster then the single EXPRESSION based SQL statement. I just don't see how running a single SQL statement with EXPRESSION will be slower then running 100 SQL statements without it.
Not to mention, how do you correlate 100 result sets?
...
I can think of a lot of applications for this in areas
such as marketing where requirements can change five times in the time that it takes to get a
single code change through the bureaucracy.
...
I've seen some requirements change 5 times while you're changing code....
I agree, the flexibility is a major strength.
Example: when you realize that a CUSTOMER_NOT_YET_CONCEIVED is also a CUSTOMER_IS_ALIVE, you can fix the expression rather quickly by appending 'and DATE_OF_BIRTH < SYSDATE' to the CUSTOMER_IS_ALIVE expression.
hmm...one SHOULD be able to limit which 'rules' are applied by starting with:
alter table expression_table add (
is_valid number(1) default 1,
valid_from date, valid_to date );
then add this to the original SQL:
and e.is_valid=1 and
e.valid_from > sysdate and e.valid_to < sysdate
But, I think we're diverging from the original thread.
From my experience, the only time I have seen an SQL statement being saved to a table in the database was from a (loosely called) data warehouse front-end application that saved the SQL statement for the end-user for future re-use (and sharing).
David Aldridge, September 14, 2009 - 3:27 am UTC
>> Out of curiosity, do you happen to have any numbers for this?
Nothing quantitative yet, but of course the beauty of "the system" is that the nuts and bolts are pretty fully exposed and it's rather easy to infer from them that that performance is relatively degraded.
Here's some internals ... the object type created in support of the filter contains much of the code, and the "getvarchar" method I mention above formats the arguments into a single string of name-value pairs (eg. "CUSTOMER_NAME => 'John Boy', CUSTOMER_ADDRESS => 'WALTON MOUNTAIN', NUMBER_OF_SPORT_CARS => TO_NUMBER('0')" etc), which then gets passed to a cursor that is used to check the predicates in the expression against the values.
"Row-by-row equals ..."
I think that if someone cobbled that up as part of a data transformation (for example) then you'd call them crazy, but of course it's a matter of trading off performance against flexibility. The important point is to understand the method, the alternatives, and their relative advantages and disadvantages. Right now I need the flexibility and relative robustness for making logical changes for some data profiling.
One issue of interest is that the methods do not explicitly invoke parallel_enable or deterministic. Could be that they don't need them, but I'll have to test to be sure.
Commenting Generally
Girish Singhal, September 14, 2009 - 9:38 am UTC
I think people have not heard about the story...
"when a french queen asked her official as to why people were protesting? .... the official replied that they don't have bread to eat .... to which the queen replied that well why don't they eat cake?"
What I mean to say is that the world is not perfect.
(And I really try hard not to be a perfectionist)
And there are lot of parameters (internal and external, apparent and hidden, known and unknown, knowledge and lack of it, education and bad education etc.) that affect the decisions.
So guys take a chill pill and have fun and do whatever you can do best and keep learning. Smiles....
And sorry for utilising this space for this kind of stuff...
Regards,
Girish
September 14, 2009 - 1:54 pm UTC
... (And I really try hard not to be a perfectionist) ...
why not?
David Aldridge, September 14, 2009 - 9:50 am UTC
Actually, those internals on Expression Filters are not the complete story. Back to the docs, cos there's also an index-based method that I haven't got to yet.
A good article about "Expression Filter" by Jonathan Gennick in the May/June '05 issue of Oracle mag
Logan Palanisamy, September 14, 2009 - 4:24 pm UTC
Mike Kutz, September 16, 2009 - 1:02 pm UTC
(note to self - always identify yourself)
Giris (and david) - I don't think we were 'fighting', just passionately discussing why we think which way is faster and (more importantly) giving our reasons behind it. Both of us still need to show hard numbers. (my test case is below) Either way, it seems that David and i agree that the flexibility far outweighs any slowdown.
interesting finds:
there seems to be a bug in the getVarchar() function that causes an ORA-38435.
with many rows (117) in BEER, i get two FULL TABLE SCANS.
with one entry in BEER, i get a "full table scan" and a "index (domain) by rowid".
i'm missing the dbms_stats lines, so that might fix the 117-row test.
Note for others that wish to test this, make sure you run the SQL statements MANY times. (1st run of an SQL statement does >1k consistent gets while 2nd run does <100)
spool beer.output.txt
set autotrace off;
REM object that describes a beer
create type beer_t as object (
beer_name varchar2(60),
company varchar2(60),
country varchar2(60),
alcohol number,
calories number,
carb number,
rating number
);
/
REM table for classifing beer based on its properties
create table beer_classification (
beer_class varchar2(60) primary key,
class_definition varchar2(1024)
);
REM setup EXPRESSIONS
begin
dbms_expfil.create_attribute_set( attr_set => 'BEER_T', from_type => 'YES');
end;
/
begin
dbms_expfil.assign_attribute_set(
attr_set => 'BEER_T',
expr_tab => 'BEER_CLASSIFICATION',
expr_col => 'CLASS_DEFINITION');
end;
/
desc beer_t;
PROMPT x-x-x-x-x-x-x-x-x-x-x-x
PROMPT BEER_T should have varchar2() function
PROMPT auto-added to its definition
pause
REM create a table to hold beer data
create table beer of beer_t;
alter table beer
add constraint beer_pk primary key (beer_name);
REM add some beer classifications
INSERT into beer_classification values ( 'light', 'CARB < 60' );
INSERT into beer_classification values ( 'non-alcoholic', 'ALCOHOL < 2' );
INSERT into beer_classification values ( 'domestic', 'COUNTRY = ''USA''' );
INSERT into beer_classification values ( 'extreme', 'ALCOHOL >= 9 and RATING >= 4');
REM add a beer or two
INSERT into beer values ( 'Sam Adams triple bock', 'Samual Adams', 'USA',
12, 100, 100, 4 );
commit;
REM I have an ETL tool that came with a sample beer.txt file
prompt LOAD Beer DATA HERE
pause
-- fix for bug.
-- ETL load 'fixes' only the first single-qoute
delete from beer where beer_name = 'O''''Doul''s';
commit;
-- dbms_stats goes here
col beer_name format a20;
col beer_class format a20;
set autotrace on;
select B.beer_name, BC.beer_class
from beer B, beer_classification BC
where
evaluate(BC.class_definition, B.getVarchar() ) = 1;
select BC.beer_class
from beer_classification BC
where
evaluate(BC.class_definition,
beer_t.getVarchar( 'MGD', 'Miller', 'USA', 3, 20, 100,3) ) =1;
-- begin
-- dbms_expfil.index_parameters( expr_tab => 'BEER_CLASSIFICATION',
-- expr_col => 'CLASS_DEFINITION',
-- attr_list => null,
-- operation => 'DEFAULT' );
-- end;
-- /
create index bc_domain_ix on beer_classification(class_definition)
INDEXTYPE IS EXFSYS.EXPFILTER;
select B.beer_name, BC.beer_class
from beer B, beer_classification BC
where
evaluate(BC.class_definition, B.getVarchar() ) = 1;
select BC.beer_class
from beer_classification BC
where
evaluate(BC.class_definition,
beer_t.getVarchar( 'MGD', 'Miller', 'USA', 3, 20, 100,3) ) =1;
REM BUG is here
select BC.beer_class
from beer_classification BC
where
evaluate(BC.class_definition,
beer_t.getVarchar( 'Pete''s Wicked Ale', 'Pete', 'USA', 3, 20, 100,3) ) =1;
spool off
sql
A reader, September 29, 2009 - 10:22 pm UTC
Tom:
I can see a benefit of running SQL from a table if i was locked out from production database.
Let us say i only have access to production using MOD_PLSQL or web application and i need to fix a bug and copy the procedure from TEST into PROD or add a new column to a table--and i do not have direct acces.
Can i just copy the procedure from TEST and upload it into a production table and then have another existing procedure that runs the SQL in that table..
Of course if you were on the CCB board you would not approved the table or procedure that runs it. Most people wil have no idea what is going on with the code.
October 07, 2009 - 7:41 am UTC
and why would you need a table to
... Can i just copy the procedure from TEST and upload it into a production table
and then have another existing procedure that runs the SQL in that table..
.....
think about it - if you have the ability to
a) insert into table T
b) run a procedure P and pass inputs to it (to identify a row in T)
c) have said procedure P owned by a user with CREATE PROCEDURE granted DIRECTLY to them (not via a role)
then you have the ability to
a) run a procedure P and pass inputs to it (the statement to execute)
don't you - why would you need, want, desire the table?
And by the way, many people do have a clue, if you tried to drop this into a system managed by the people that manage the systems I work around (here at work), they would laugh at you (you would not be able to create procedure P in the first place, you would not have create procedure directly granted to you) and then note your name in a book so as to make sure to really review everything you ever try to do with a microscope from now on. You are dangerous.
(sam - aka SMK - i have to say, you do really scare me sometimes - truly)
To "A reader" above
Nicosa, October 02, 2009 - 5:30 am UTC
Wouldn't that be a big security issue ?
If you can add/remove/update the production table that holds sql to be run from a website, it's almost as direct access to production don't you think ?
It would just need a bad SQL to make sure your database will be functionaly corrupted...
If SQL has to be stored in a table, I would make sure that table is not modifiable from outside.
October 07, 2009 - 3:21 pm UTC
the poster seems to think that no one cares or looks at their code and can put into production anything they want to get around any policy in place.
procedure
A reader, October 07, 2009 - 9:24 am UTC
Tom:
You make it sound like i really want to do this. It is logical question related to application security and how competent the people who manage the system to catch it and block it. If they did not laugh then they are "dangerous" as they let such a thing slip by!
Intriguing debate
Ioannis, September 16, 2010 - 2:52 am UTC
It's been a year since the last post, but i can't help myself from jotting in a few words of my own.
I'm new to programming so you'll not hear any guru suggestions or comments.
I'm designing a small web application to store Customer Data and some Financial Records. Through the path that i undertook, i ended up designing a small Content Management System for the presentation part of the app. When i was using those table fields to store navigation and presentation data, I wondered, "what if i use the same fields to store sql code in it". And so i ended up in this discussion.
The main idea was to avoid embedding sql queries in php, and use php to draw the queries out of the table fields. Now at first look, it didn't seem a bad idea, and so i started looking for it. Actually, i thought i reinvented the wheel!
I've been using blogs, and discussion, and mailing lists for quite some time now, and never had the chance to read through such a powerful debate. I couldn't help needing to read comment after comment and so on (maybe being new to programming made it even more intriguing). If we set aside "names being called", and some "colorful comments" on ones mentality, I'd have to admit, it was one the most interesting discussions I ever had the chance to read through. Hopefully I'd get the chance of having such debate around a question of mine, if things lead me to it.
Regards
sql in a table
A reader, February 03, 2011 - 8:45 pm UTC
trim statement in table
A reader, February 03, 2011 - 10:12 pm UTC
Tom:
someone suggested to add a bunch of SQL statements that trim spaces into a table as data. I do not like that solution and suggested to enter into pl/sql routine and run nightly.
statements are like
update T1 set last_name = TRIM(last_name) where trim(last_name) <> last_name;
updat T1....
since there are free entry form fields do you think it is incorrect to trim the fields and use the TRIM(LAST_NAME) in the WHERE clause of the SQL statement when we perform the search query?
DO we need to create an index for every field we trim?
February 04, 2011 - 9:08 am UTC
I can't answer this - only you can answer this.
Are the spaces relevant? Do they belong there? If so, leave them.
If not, do the mass update to correct the data integrity issue you have and then ensure that the insertion of new data only inserts data that conforms to your data rule that says "no trailing whitespace" so you never have to do it again.
Last_Names in general do not have trailing white space - this is an edit that should be enforced on the data as it enters the database one would think. Fix the existing data and enforce the rule in the future so you don't have to fix the data again.
how to call all tables in excel from one database
sanjay, July 28, 2011 - 2:31 am UTC
I wonder how can we call all table of database in MS excel 2007 by running once script or any other way
There is cretia that in cell if we chance a name database then it will bring table corsspond to that database
Also we only need Top 50 rows from each table distibuted along excel sheet along with that
How is that possible do we have to create a VB box or MAcro i am open to consider any option
Thank you
Oracle APEX?
/dev/null, September 09, 2011 - 1:55 am UTC
I'd be interested to hear then how Oracle APEX is different from this? APEX stores all user written queries in database tables, now we usually try to keep as much as possible in views (and hence on the database) however surely it's a *similar* situation here, just APEX provides a nice front end to do it in and checks query syntax (optionally) before committing your changes.
September 09, 2011 - 7:07 am UTC
APEX doesn't have to store anything, all of my queries for asktom are in STORED PROCEDURES in packages.
Select statement in a table
Brian, April 12, 2012 - 2:16 pm UTC
We store (PL/)SQL in a table as a general practice to call stored procedures, views and selects.
Our primary use of a stored select is in a job to write data transfer files. We have a generic set of Java classes to write a result set to a field length delimited or XML file. To configure a new file, no Java code needs to be written, we simply put a new query in a config table along with other parameters such as file name and folder destination.
File content is formatted in SQL with RPAD/LPAD for CSV or field length delimited files and XMLELEMENT etc. for XML files. We could of course require the select to be put in a stored procedure but that puts an opaque layer on what it's doing that makes production problem diagnosis too difficult.
Another way we use SQL in a table is to make a decision dynamically, e.g. we don't know the row in table A to which a row in table B must join until run time. We store a query in table B that figures it out and execute immediate from within a stored procedure. It's a level of customization required by a broad customer base all with their own special nuances.
While I understand that in an ideal world business logic belongs in procedures and packages, like most things there are gray areas where compromise is necessary and beneficial.
April 12, 2012 - 3:01 pm UTC
We store (PL/)SQL in a table as a general practice to call stored procedures, views and selects.
be careful with that, I see sql injection possibility out the 'wahzoo' with that.
Our primary use of a stored select is in a job to write data transfer files. We have a generic set of Java classes to write a result set to a field length delimited or XML file. To configure a new file, no Java code needs to be written, we simply put a new query in a config table along with other parameters such as file name and folder destination.
I could see a better possible way - store a view in a schema designated to be "the schema with the views that should be dumped" and a table with the view names and other parameters. You can add and remove views just as you add and remove rows now.
on what it's doing that makes production problem diagnosis too difficult.
that is an unsupported statement I would disagree with. If the code is compiled into the database in the form of a view or a procedure, it would be easier to diagnose issues as you would hit 99.9999% of the issue during the creation of the object. Hiding code in strings in tables makes problem diagnosis incredibly hard in my experience.
e.g. we don't know the row in table A to which a row in table B must join until run time. We store a query in table B that figures it out and execute immediate from within a stored procedure. It's a level of customization required by a broad customer base all with their own special nuances.
give a 'for example', I don't see how you wouldn't know a join condition until run time.
While I understand that in an ideal world business logic belongs in procedures and packages, like most things there are gray areas where compromise is necessary and beneficial.
sure - but - doing it this way shouldn't be the first choice or even the 100th choice, it is much further down the list and is in general a really bad idea.
Execute SQL from table
Mark Fink, June 27, 2012 - 1:45 pm UTC
/* My way of executing SQL from a table */
/* Mark Fink */
--- Create a table to store the SQL commands in
Create Table ATableName (Run_id int identity,
Database_SQL nvarchar(Max))
--- Create a table to place results of execute in
Create Table Newtable (New_id int Identity,
NewDate smalldatetime,
Execute_Result nvarchar(50))
--- Need a ' to be place in the command
Declare @q char(1)
Set @q = char(39)
---Set up 3 commands
insert into ATableName (Database_SQL)
(Select 'Insert NewTable(NewDate,Execute_Result)' +
' Select getdate(),' + @q + ' 1 The server is'
+ @q + ' + ' + ' @@Servername')
insert into ATableName (Database_SQL)
(Select 'Insert NewTable(NewDate,Execute_Result)' +
' Select getdate(),' + @q + ' 2 The server is'
+ @q + ' + ' + ' @@Servername')
insert into ATableName (Database_SQL)
(Select 'Insert NewTable(NewDate,Execute_Result)' +
' Select getdate(),' + @q + ' 3 The server is'
+ @q + ' + ' + ' @@Servername')
---look at the commands
Select * from ATableName
---Set the maximum records in the table
Declare @Max int
Set @Max = (Select max(Run_id) from ATableName)
Declare @Thisjob int
Set @Thisjob = 1
Declare @ExecCmd nvarchar(Max)
While @Thisjob <= @Max
Begin
Set @ExecCmd = (Select Rtrim(Ltrim(Database_SQL)) from ATableName where Run_id = @Thisjob)
-- This line is to look at the command or for testing
-- Select @ExecCmd
EXECUTE (@ExecCmd)
Set @Thisjob = @Thisjob + 1
End
Select * From Newtable
Drop Table NewTable
Drop Table ATableName
June 27, 2012 - 4:33 pm UTC
great, always glad to see microsoft sqlserver code?
huh?
no clue what your purpose is..