Re: Oracle Session Management in Web Applications
Ed, November 04, 2002 - 10:15 am UTC
Tom,
I take your point. I'm afraid I don't have much knowledge of the ins and outs of PHP! I guess the two approaches are either:
a) Using Oracle, the PL/SQL web toolkit, mod_plsql, Apache
OR
b) Using PHP, the Oracle8 Call Interface (OCI8, standard part of PHP, apparently - </code>
http://www.php.net/manual/en/ref.oci8.php <code> and Apache.
I agree that there does appear to be something wrong with the PHP driver, but I guess that's what you get for using unsupported code :-(
Sorry I did make a typo - I meant "ORA-01012: not logged on in", of course!
You say "it is infinitely faster to use a connection pool". What do you really mean by this - infinity is quite a big number!
November 04, 2002 - 10:26 am UTC
thats right -- infinity is a big number. Infinitely faster is generally taken to mean "lots faster". The slowest thing in a web page is generally connect/disconnect. If you remove that every page is faster AND the system as a whole is faster and more scalable since it is doing lots less work.
evidence
Ryan, November 04, 2002 - 4:47 pm UTC
"The slowest thing in a web page is generally connect/disconnect"
What evidence do you have?
November 05, 2002 - 8:53 am UTC
7 years of experience doing it (on the web). 15 years working with databases.
Time it. Tell me what takes long -- setting up the physical circut from the client to the server or running some tuned queries?
Why do you think "connection pools" and their ilk were even invented.
The most expensive thing you can do on an OS is process(/thread) creation and process(/thread) destruction. Connect/Disconnect generally equates to just that. If you say "well, we will just use MTS (aka shared server)" you will find the code path to connect via that is many times longer then dedicated hence you will not achieve anything going down that path either.
This is one of those commonly accepted things that is actually true.
A reader, November 04, 2002 - 6:44 pm UTC
Tom,
Do you'll use Dead Connection Detection..If yes can you elaborate a little more on that..
Thanks.
November 05, 2002 - 9:00 am UTC
don't understand where DCD (dead client detection) came into play here?
You would use that if you expect your app server to crash totally, leaving orphaned connections to your database. DCD was invented for windows basically -- cause it would crash, leaving orphaned connections. DCD allows the database to ping the client from time to time to see if it is still there. In a mature environment you do not need it. I do not use it.
Re : evidence
Ryan, November 05, 2002 - 1:44 pm UTC
You know it's kinda funny reading this when I figured one of your mantras was to always benchmark to prove a point.
You write "This is one of those commonly accepted things that is actually true", so I guess that gives the green light for not benchmarking if you don't feel like it when you think something's "commonly accepted", whatever that means :-)
November 05, 2002 - 3:10 pm UTC
I've benchmarked this dozens, no wait, probably hundreds of times.
My comment that
"This is one of those commonly accepted things that is actually true"
was made in light of that actually -- this is actually true AND it happens to be commonly accepted which is typically an OXYMORON. I was pointing out in this case
o commonly accepted
o and true
happen to happily coincide.
Can I benchmark it here? No, Have I benchmarked it -- yes, over and over and over and over and over (i invented the reUseConnect wdbsvr configuration setting that later became part of mod_plsql for this sole purpose)
Re: evidence
Ryan, November 06, 2002 - 4:21 pm UTC
Well I'm sorry that I didn't equate "This is one of those commonly accepted things that is actually true" with "I've benchmarked this dozens, no wait, probably hundreds of times" - I guess this is because I have a problem you seem to get - MY crystal ball is in the shop right now!
I can't benchmark it, as I don't have the resources at the moment, but believe me, I would if I could. My gut feeling based on 4plus years of web development is that the connection/disconnection overhead is maybe one quarter second, maybe one half.
Now whether thats important to the users that their pages are that much slower is entirely subjective, but if a query is taking 12 seconds to come back, I don't think an extra half second is gonna piss the users too much.
November 06, 2002 - 5:01 pm UTC
you are kidding right -- the problem isn't the .5 seconds -- it is the massive hit your system takes whilst 10 people at the same time all try to fork and exec a dedicated server or create a thread and the network connections get set up and the system context switches all over the place to do these things.
Riddle me this batman:
o why did "fastcgi" where by cgi bins (old stuff from 95 when the web was "newish") were cached and not forked and exec become really necessary?
o why does apache use mod's instead of cgi bin?
o why do we need connection pools?
o why do we have app servers and not just web servers with cgi's like we used to?
because -- process creation and process destruction is the most expensive thing you can do. It makes your system *not scalable*. Try to log in in single user mode -- no problem. Now, have 10 people log in AT THE SAME INSTANT. Now, tell me, what happens?
Do what you will. Sorry my comment "this is one of the commonly accepted things *that is actually true*" was so obscure.
Me -- I would never touch a system (as an end user) where the web pages took 12 seconds to generate. Wouldn't touch it. But you are right -- if your pages are taking SECONDS to generate -- you do have much bigger fish to fry, other things to be worried about. But -- if you do want to make it run slower, by all means -- don't reuse connections, use cgi-bin!
Re: evidence
Ryan, November 07, 2002 - 8:20 am UTC
I don't disagree with anything you write.
Hell, it isn't even an issue for me! I use 9iAs for everything I can - session management isn't even an issue, everything gets done for you! If your in a situation where your just dealing with Oracle, I wouldn't use anything else.
I guess I didn't express myself very well - I just want to say to the OP that if you've already gone down the road where your stuck with bad session management, it's maybe not worth your time trashing everything and starting again. But maybe I'm wrong.
As for all you "why" questions, I dont know. Now why does Microsoft release Office 97 / 2000 / XP / whatever, when MS Word 2 was 99.9% adequate for everything?
Oh, your comment "this is one of the commonly accepted things *that is actually true*" wasnt so obscure, just sometimes I like to see some EVIDENCE to back up claims like this - dont you?
Just want to confirm and elaborate on the part about pool of connections
Pat, November 07, 2002 - 9:11 am UTC
Since I have queried v$session many times and seen that there would always be just 1 session for 1 web page, can I say this: the pool of connections is managed by the HTTP server(program apache.exe), which actually connects to Oracle and therefore is listed in v$session, and it handles the creation/killing of connections, as wells as passes input/output between the browser and database server. So 1 web page with any 1 user request would show up as 1 session in v$session, no matter how many users are actually visiting the page. But once you get out of that page, assuming you are the only user, the session would still be there till the configured connection time in HTTP server runs out?
Is there any pointer for us to read more about the configuration of pool of connections, e.g. how many depending on load, how long each would be, or these are totally transparent so they are not configurable?
November 07, 2002 - 10:33 am UTC
depends on the tool you are using -- servlets, jdbc connection pools -- you control it. mod_plsql, just "happens", you either set it on or set it off pretty much.
All of the configuration parameters for all of the mod's are documented.
connection pooling
mo, January 01, 2003 - 6:04 pm UTC
Tom:
<In mod_plsql -- it is normal that a pool a connections is maintained and reused>.
1. I assume you mean the connection is the physical NET8 connection between app server and dbms and not browser/app server?
2. When you have a connection like sql*plus connection you create it using userid/password. What userid/password the app server use to create all those connections in the pool?
3. Is one connection (from pool) defined with whatever is defined in the DAD?
4. If we have blank DAD then we can't use connection pooling?
January 01, 2003 - 6:21 pm UTC
1) correct
2) depends -- did you store the username/password with the DAD? if so, that is where. if not, the user gives it to us.
3) as needed they are created
4) don't know what a "blank" DAD is -- assume you mean "created with out username/password" -- if so, you can. There are "switch user" operations we can perform. As well, we try to hook you back up with your connection if possible.
connection pool
mo, January 01, 2003 - 9:32 pm UTC
Tom:
OK I think I understand how it works now.
1. App server creates all connections in the pool based on how many DADs are created and assigns the appropriate connections to users based on their login. Correct?
2. If system was configured with blank DAD, i.e. let user enter userid/password first time he logs in, the app server will already have a pool of generic connections but then will switch userid/password info in connection to whatever he enters? Correct.
Thank you
Updating session data and redo generation
David Piazza, December 19, 2003 - 4:27 pm UTC
Tom,
We are using Oracle 9.2, a web server using cold fusion, and have a web application that stores session data in the database, "session data" meaning state information for currently logged-in users. The session data includes various information such as the user's ID and security level, and occasional other chunks of information that have to follow the user around as she/he is using the application. The storage of session information in the
database is pretty common in web-based applications.(ColdFusion calls this feature "client variables" and WebSphere calls it "persistent sessions.") It's not universal - sessions can be stored in RAM when it's guaranteed that the userwill connect to only one application server during a given work session. But when there are multiple servers behind a "non-sticky" load balancer, the servers need a way to share users' states, and the database is the mechanism for doing this.
If the user state is being updated frequently, this can translate to a lot of database writes - often one write per webpage-hit. The impact on performance can be significant. During testing of the application, using load runner, the amount of redo generated was reduced by recoding parts of the application so that the session state was being changed much less often. But, not all applications will have the luxury of stable session state.
Now, in our application, the session state has no long-term meaning. If the database were to fail and then be restored, users reconnecting to the application would be expected to establish new sessions, that is, to log in
again (whether manually or via V-GO or true SSO, it would still be a new login and a new session), so there would be no need to restore the session data up to the point of failure.
Since logging, cannot be turned off for single inserts, do you have any suggestions on limiting redo from the frequent session updates?
December 20, 2003 - 9:36 am UTC
this is exactly what html db does as well - -in the database ALWAYS.
I have not measured any significant impact on my performance (and my site is fairly busy).
So..... give me the numbers, show me the science, prove to us that redo generation is a significant bottleneck/constraint in your system.
Because -- remember -- databases were born to join and born to write. Every page on asktom not only does this state thingy, but I have the nerve to write a big audit record upon entry to the page (every page) and then update that entry at the end. Along with other audits as well. So, every page view on this site is a transaction with lots of writes. I'm not worried about it and log file sync has never been a major "wait" for me.
So, show us why you believe redo generation to be "your very big problem". Need to see the math behind this thought.
jdbc and database sessions
A reader, February 14, 2004 - 1:33 pm UTC
Hi
Let's say I have a repot generation tool programmed using JSP, Servlets, Thin JDBC, connection pool etc. If I run a report which takes 10 minutes and my PC hangs before the repot finishes what happens to the database session? Does the report keep on running? Is there a way to kill/contol database session if this situation happens? Imagine my uses arent patient and keep on running reports thinking their previous executions hanged my database will be flooded!
February 14, 2004 - 5:22 pm UTC
who cares about your PC here, the code is running on the server -- the application server.
Imagine if you will a DIFFERENT architecture whereby you do not run things from a browser that take 10 minutes. I (being a patient person normally) would not have the tolerance for anything taking more than a couple of SECONDS in a web page.
Sounds like you need to
a) submit a request for a report to run
b) email them a link to the report when it is done or
c) have a page they can goto to see the reports they've submitted and that are ready for review.
In this fashion, you can
1) control the number of concurrently executing reports easiy
2) facilitate the sharing (caching) of reports -- if two people request the same report you may well already have it
3) use off the shelf offerings that already do this sort of stuff (eg: Oracle Reports is just such a tool)
So, your PC has nothing to do with the equation here.
The application server is running the report, it doesn't "hang".
Asking people to wait 10 minutes for a page will never fly in the real world (many browsers would time out for example, very fragile).
You want to look for an alternative solution that makes it so the user doesn't have to wait.
what if application server fails?
A reader, February 14, 2004 - 7:27 pm UTC
What happens if the application server fails? Do the database sessions stop as well if there are long queries running?
February 14, 2004 - 7:51 pm UTC
depends, did your DBA set up DCD (dead connection detection) or not....
if not, the database might not realize the client is gone for a while....
DCD
gayatri, January 05, 2006 - 6:24 am UTC
Discussion is quite useful. I like to know how a Dead Connection Detection is done in Apache.
I am using Apache , mod_plsql. I want to detect inactive clients.
Pl elaborate.
Thanks.
January 05, 2006 - 10:50 am UTC
apache is the client here.
dead client detection loves inactive clients, it hates DEAD clients.
dead client detection simply has the server "ping" the client (apache) and if the client says "still here" it is happy. If client does not respond - then dead client detection will remove the session.
DCD Followup
gayatri, January 06, 2006 - 1:40 am UTC
Thank you. but, client in the sense a web browser for me. I like to know if there is any mechanism to find out idle web browsers.
Can Apache figure out such clients and close the connections and give a "Page Expired" message to the web browser when an attempt is made by the client (on web browser) after the expiry time?
January 06, 2006 - 1:48 pm UTC
client, in any sense, is not the web browser from the database perspective.
mod_plsql is.
You would have to have some sort of concept of a "session id", do you? you only asy "using mod_plsq", not what tool you are using with it to create the application.
DCD with Oracle App Server
Scott, September 12, 2006 - 4:48 pm UTC
Hi Tom,
Reading this article was very insightful, but I have a question for you. One of my clients is having a problem using Forms Oracle Application Server 10G. It seems that sometimes, a user will be working on the form, and either IE will crash (big surprise) or their internet connection will die. This will result in a session with locks, causing user errors. It takes between 15 and 20 minutes for this session to die. Would setting DCD up resolve this issue or do I have to investigate more about OAS?
Thanks,
Scott
September 13, 2006 - 7:01 am UTC
the client isn't dead though, the client is on the middle tier.
session
A reader, August 18, 2008 - 3:34 pm UTC
Tom:
Is this correct?
I have a web app that uses mod_plsql with one userid/password defined in it. Users log in with using application table accounts.
The only way for me to tell what user A did is to track the session id in V$session and tie that to his userid. I can log that in a table and then query this table and V$session to track/audit what quaeries this user A did.
2. If i had one database account per user then I will not need to do this. correct? oracle will automatcially track the activity for each database account.
August 20, 2008 - 10:12 am UTC
unless the application identifies who is really using it, you cannot figure out in the database who is using it.
does your application somehow push into the database (using dbms_session.set_identifer for example) the true identity - if not, you will have no clue who is using it at the database level.
2) correct, search for ntier proxy authentication on the documentation at otn.oracle.com
session
A reader, August 23, 2008 - 9:51 am UTC
Tom:
I don't think the app pushes the true identity. You mean if john/oranges user logged in using one common master DB account, I have to push "john" with every SQL statement to the DB? What does dbms_session.set_identifier really do.
I do not umderstand why you can't get the info from V$session. Every user who logs in has a unique session id. All i need to to insert into a table (john/his_session_id) and query v$session to see the SQL that he ran. Am i wrong here.
August 26, 2008 - 8:01 pm UTC
reader - tell me, what do you query to figure out "john"
if you can do that query, tell us what it is. If you cannot - then we are right back where we started....
that is, if the APPLICATION does not bother to tell US who is really using the APPLICATION, there is nothing magic in the database that can figure out who the heck is running the application for real - you are using a common username and password.
dbms_session is one way - just one way - for the application to tell the database "this is john"
client
A reader, August 26, 2008 - 10:41 pm UTC
I query "app_users_table" which has one record with userid "john" and password "oranges".
the query is
select count(*) from app_users_table where userid='john' and password='oranges'
What you say makes sense is that the application only provides the app userid/password once on login. but as far as db it is one master account connection.
Where do you implement the "dbms_session"? in every powerbuilder application code statement or it is defined for the whole session. If i set this up how would i go to oracle to see who ran what. is it in V$sql.
August 27, 2008 - 8:23 am UTC
you would want to
a) read about dbms_session and dbms_application_info
b) then implement it
you would typically call them right after "authenticating" - you know what, your system is so insecure it doesn't even matter. If you really store the passwords in clear text like that - coupled with the fact it would take most people 30 seconds to get your master user/password - we'd have every identity and every password.
Clarification
Andrei, April 19, 2009 - 5:53 pm UTC
Tom, in your followup from November 5, 2002 - 8am US/Eastern of this thread your wrote "The most expensive thing you can do on an OS is process(/thread) creation and process(/thread)
destruction. Connect/Disconnect generally equates to just that. If you say "well, we will just
use MTS (aka shared server)" you will find the code path to connect via that is many times longer
then dedicated hence you will not achieve anything going down that path either."
Would you like to explain why MTS Connect/Disconnect is not suitable for ordinary web applications like PHP/Apache/Oracle (one connect per web page)? Even any longer then dedicated you wrote. I thought in MTS Oracle creates n (10 for example) server processes which would not be destroyed and then created again so it would help to avoid this type of overhead. Where am I wrong an what is the REAL thing for it? Thanks, Andrei.
April 21, 2009 - 12:32 pm UTC
shared server is generally not useful for most web applications because they already implement a connection pool. We did implement database resident connection pooling (drcp) in 11g for things like PHP that don't currently do connection pooling 'correctly'.
It (drcp) has the advantages of dedicated server (direct connect to a dedicated server) without the process startup normally involved in a 'connect' request.
connections
A reader, May 24, 2009 - 7:40 pm UTC
What about using DRCP with a client like forms 6i
juan carlos reyes, May 12, 2010 - 3:39 pm UTC
Hi Tom, Do you have an observation against this practice?
Because if in some way an open form having a session open, is like a persistent connection, or I'm wrong.
Thank :)
Do many Inactive sessions
Brijesh, January 09, 2012 - 3:24 am UTC
Tom,
I have RAC 2 Node database runing the ATG application. We have 30 instance of ATG talking to database at the same time. Each instance open 5 connection and go upto 20 connection. Whenever we take look at gv$session we find 90% of the session to be inactive.
Please suggest an session management techinque to effictivley manage session
Thanks,
Brijesh
January 10, 2012 - 10:20 pm UTC
I don't know what ATG is or does.
configure your connection pools to be smaller
why do you have 30 instances of whatever ATG is running? why do you need so many?