Skip to Main Content
  • Questions
  • How lob columns are transferred by Oracle Net Services.

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, lh.

Asked: April 10, 2019 - 6:51 am UTC

Last updated: April 16, 2019 - 11:48 pm UTC

Version: 12.1

Viewed 10K+ times! This question is

You Asked

Hi

Oracle manual states:
'Starting with Oracle Database 11g, Oracle Net Services optimized bulk data transfer for components, such as Oracle SecureFiles LOBs and Oracle Data Guard redo transport services. The SDU size limit, as specified in the network parameter files, does not apply to these bulk data transfers.'


We are transferring XML and JSON data with XMLTYPE and CLOB columns. These columns are sometimes big.
How this data is transferred and which parameters could have positive effects for these transfers ?

Is there any difference in transfers between
- construction e.g. json document in select statement.
- fetching column from clob in permanent table
- fetching column from clob in global temporary table




lh

and Connor said...

They need to be securefile. Here's an example

SQL> create table t1 ( x clob)
  2  lob ( x) store as basicfile;

Table created.

SQL>
SQL> create table t2 ( x clob)
  2  lob ( x) store as securefile;

Table created.

SQL>
SQL> declare
  2    c clob;
  3  begin
  4    c := 'qwlekj lkjqweoijqweo iqjw eoqiwje oqiwje alskdlqkwjeqwe';
  5    for i in 1 .. 20 loop c := c || c; end loop;
  6    insert into t1 values (c);
  7   insert into t2 values (c);
  8   commit;
  9   end;
 10   /

PL/SQL procedure successfully completed.

SQL>
SQL> select dbms_lob.getlength(x) from t1;

DBMS_LOB.GETLENGTH(X)
---------------------
             57671680


So both tables have a single 55meg clob in them. Now I connect from another machine, query them, and check the SQLNet stats. (mystat in the following examples just queries v$mystat for stats containing the phrase)

C:\oracle>sqlplus xxx/xxx@//vm18/pdb1

SQL*Plus: Release 18.0.0.0.0 - Production on Thu Apr 11 14:59:16 2019
Version 18.5.0.0.0

Copyright (c) 1982, 2018, Oracle.  All rights reserved.

Last Successful login time: Thu Apr 11 2019 14:55:23 +08:00

Connected to:
Oracle Database 18c Enterprise Edition Release 18.0.0.0.0 - Production
Version 18.3.0.0.0


SQL> @mystat
Enter value for statname: SQL

NAME                                                              VALUE
------------------------------------------------------------ ----------
bytes sent via SQL*Net to client                                   4546
bytes received via SQL*Net from client                             5105
SQL*Net roundtrips to/from client                                    14
bytes sent via SQL*Net to dblink                                      0
bytes received via SQL*Net from dblink                                0
SQL*Net roundtrips to/from dblink                                     0
bytes via SQL*Net vector to client                                    0
bytes via SQL*Net vector from client                                  0
bytes via SQL*Net vector to dblink                                    0
bytes via SQL*Net vector from dblink                                  0

SQL> set feedback only
SQL> set long 100000000
SQL> set longchunksize 100000000
SQL> select * from t1;

1 row selected.

SQL> set feedback on
SQL> @mystat
Enter value for statname: SQL

NAME                                                              VALUE
------------------------------------------------------------ ----------
bytes sent via SQL*Net to client                              115439432
bytes received via SQL*Net from client                             6474
SQL*Net roundtrips to/from client                                    21
bytes sent via SQL*Net to dblink                                      0
bytes received via SQL*Net from dblink                                0
SQL*Net roundtrips to/from dblink                                     0
bytes via SQL*Net vector to client                                    0
bytes via SQL*Net vector from client                                  0
bytes via SQL*Net vector to dblink                                    0
bytes via SQL*Net vector from dblink                                  0

SQL> set feedback only
SQL> set long 100000000
SQL> set longchunksize 100000000
SQL>
SQL> select * from t2;

1 row selected.

SQL>
SQL> set feedback on
SQL> @mystat
Enter value for statname: SQL

NAME                                                              VALUE
------------------------------------------------------------ ----------
bytes sent via SQL*Net to client                              230817389
bytes received via SQL*Net from client                             7843
SQL*Net roundtrips to/from client                                    27
bytes sent via SQL*Net to dblink                                      0
bytes received via SQL*Net from dblink                                0
SQL*Net roundtrips to/from dblink                                     0
bytes via SQL*Net vector to client                            115343360
bytes via SQL*Net vector from client                                  0
bytes via SQL*Net vector to dblink                                    0
bytes via SQL*Net vector from dblink                                  0

SQL>


You can see that both transferred around 100meg in total, but the key statistic is the "bytes via SQL*Net vector to client". That is the optimization to ignore SDU and transfer as large as possible. It only occurred with the SECUREFILE column.

Rating

  (8 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

lh, April 11, 2019 - 10:53 am UTC

Hi

Thank You for Your prompt answer which explained a lot and raised some new points.


Did I understood correctly
- if we create in sql-statement lob column containing json/xml document it won't be using 'vector' transfer.
Is there any way how we could make it happen ?
- if we store json/xml document to global temporary table and fetch it from there, it won't be using 'vector' transfer. Or is there way to define lob in global temporary table as secure file lob?



lh
Connor McDonald
April 12, 2019 - 2:24 am UTC

The clobs will obey the database-wide rules for sercurefiles

SQL> create table t ( x xmltype );

Table created.

SQL>
SQL> select column_name, securefile
  2  from   user_lobs
  3  where  table_name = 'T';

COLUMN_NAME                    SEC
------------------------------ ---
SYS_NC00002$                   YES

SQL> sho parameter secure

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------
db_securefile                        string      PREFERRED


If its already a basicfile you could convert it as per





I am accusing You

lh, April 11, 2019 - 11:31 am UTC

Hi

You give such a great service that first one asks from You and only after that tries himself.

create global temporary table t2 ( x clob)
lob ( x) store as securefile;
ORA-43853: SECUREFILE lobs cannot be used in non-ASSM tablespace ""


lh

virtual lobs

Racer I., April 12, 2019 - 7:02 am UTC

Hi,

I think lh means what happens with LOBs that are created on-the-fly in a statement with XMLTABLE or JSONTable or similar. Can this be speeded up? Would it be necessary to store it in an intermediate table in a securefile field and select from that to get the benefit? Could this be a GTT or must it be a regular table?

regards,

Connor McDonald
April 15, 2019 - 4:36 am UTC

I don't read it that way :-)

But with any LOB, whether explicit or implicit created, you'll only get the SQLNet transfer benefit if it was created as securefile.


other way 'round

Racer I., April 12, 2019 - 7:23 am UTC

Hi,

Actually the opposite of XMLTable. Whatever returns LOBs. XMLQuery? CAST AS JSON?

regards,

Global temporary table as securefile

lh, April 15, 2019 - 10:29 am UTC

Hi


I interpret error message received when trying to create clob column as securefile to global temporary table, that is not possible.

Can one create lob column created in select statement as securefile lob ?


If securefile option is not possible to enhance network traffic what other possibilities there are ? DEFAULT_SDU_SIZE ?


lh

speedy gonzales is a pedestrian

Racer I., April 16, 2019 - 6:45 am UTC

Hi,

@lh : There is no "SELECT ... CAST(XMLQuery() AS LOB (STORE AS SECUREFILE)) ..." (I guess) and, as you discovered, GTTs can't have securefile lob fields. So how about a real table? You need to isolate parallel sessions manually (say via sequence/pk) when using this table and clean it out yourself or have some housekeeping on it.

regards,

back to basic(file)s

Racer I., April 16, 2019 - 6:52 am UTC

Hi,

@lh : another point : how do such LOBs get so big as to benefit from this? Normally you send small JSON/XML messages around. Instead of sending massive amounts of data as one LOB maybe stick to regular row sets? The potential boost for securefiles would be offset by the tags and string encoding.

regards,
Connor McDonald
April 16, 2019 - 11:48 pm UTC

Agreed. I don't think the network improvements would justify the cost of storing a lob in a permanent table, unless the network was really bad....at which point it would seem better to tackle that root cause.

lh, May 13, 2019 - 10:48 am UTC

Hi


Application and database are communication with each other by using json messages.


Messages can be quite large, because message is 'hierarcal'. e.g. if one would look at some specific order, this message would contain order row, order line rows, delivery instruction rows etc. The whole concept.

I am interested are there any methods, parameters etc, which would make accessing these lob columns faster. Even a little bit. This is big system and cumulative effects might be consirable for even small changes.



More to Explore

Administration

Need more information on Administration? Check out the Administrators guide for the Oracle Database