Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Monish.

Asked: June 15, 2003 - 4:25 pm UTC

Last updated: May 12, 2008 - 12:11 pm UTC

Version: 9ias

Viewed 10K+ times! This question is

You Asked

Hi Tom,

In mod_plsql: I need to pass '&' as a part of data via a url to a stored procedure.

Stored Procedure Name: show
Parameters: category1 (varchar2(120)), category2 (varchar2(120))


I am not able to send the following url:
</code> http://xyz/pls/mydad/show?category1=toys%20&%20games&category2=abc <code>

I have tried using: htf.escape_url and htf.escape_sc, but somehow not able to fix it yet.

I shall appreciate if you could suggest a good solution.

Thanks,
Monish


and Tom said...

well, you already sort of knew the answer ;)

You knew how to url encode the blank as %20 -- that is 20 hex, which is 32 ascii which is the ascii code for "space".

You could select to_char( ascii('&'),'xx') from dual; to get the hexcode for &, but there are other characters to worry about as well. So, I use this function:



ops$tkyte@ORA920> create or replace function urlencode( p_str in varchar2 ) return varchar2
2 as
3 l_tmp varchar2(12000);
4 l_len number default length(p_str);
5 l_bad varchar2(100) default ' >%}\~];?@&<#{|^[`/:=$+''"' || chr(10);
6 l_char char(1);
7 begin
8 if ( p_str is NULL ) then
9 return NULL;
10 end if;
11
12 for i in 1 .. l_len loop
13 l_char := substr(p_str,i,1);
14 if ( instr( l_bad, l_char ) > 0 )
15 then
16 l_tmp := l_tmp || '%' || to_char(ascii(l_char), 'fm0X');
17 else
18 l_tmp := l_tmp || l_char;
19 end if;
20 end loop;
21
22 return l_tmp;
23 end;
24 /

Function created.


which lets me code:

variable := '</code> http://xyz/pls/mydata/show?category1='
||
              urlencode( 'toys & games' ) || 
             '&category2=' || urlencode('abc');

which gives me the string:


'
HTTP://XYZ/PLS/MYDATA/SHOW?CATEGORY1='||URLENCODE('TOYS&GAMES'
||'&CATEGORY2='||URLENCODE('ABC')
-----------------------------------------------------------------------------------------------------------------------------------
http://xyz/pls/mydata/show?category1=toys%20%26%20games&category2=abc <code>




Rating

  (13 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

I also noticed that ' ' is converted into a '+' by html forms.

Monish, June 15, 2003 - 5:22 pm UTC

Thanks Tom. The solution was very helpful.

Also I noticed that if we submit the data via html form (action method=get), it coverts blank spaces into '+' instead of '%20'.


Tom Kyte
June 15, 2003 - 5:30 pm UTC

yes, that is valid in a URL encoded string (and why you have to URL encode a plus that should be a plus!)

can you use utl_url.escape function?

Yong, June 16, 2003 - 12:59 pm UTC

I am use utl_url.escape(string) for any escape.

Tom Kyte
June 17, 2003 - 6:38 am UTC

that should work as well.

always always escape_sc ?

robert, March 01, 2004 - 2:19 pm UTC

Tom, would you (always) be willing to take the extra processing and escape_sc the curl parameter like below ? thanks

htp.p(webutil.format_cell(l_activedue,'999,999,999,999',
curl => htf.escape_sc('wict.curstat?pType=2&pTypesum=' || l_activeduesum || '&pTypemin=' || l_activeduemin)));


Tom Kyte
March 01, 2004 - 3:18 pm UTC

that would be wrong, it would be:


'wict.curstat?pType=2&pTypesum=' || htf.escape_sc( l_activeduesum ) ||
'&pTypemin=' || htf.escape_sc( l_activeduemin ) ))


and yes, I would if activedumsum/min were STRINGS (seems that they might be numbers, numbers are generally safe -- it is things with embedded spaces, &, <, > and so on that'll kill you every time)

Just what I was looking for

Anu Sinha, March 03, 2004 - 12:36 pm UTC

Thank you very much.

htf.escape_sc

A reader, April 10, 2005 - 2:59 am UTC

In HTML DB environment.

I used the following line :

htp.p(x.content) --can display Chinese Characters stored in database to the browser.

htp.p(htf.escape_sc(x.content) ) --will output the strange characters like below to the browser:
&#25105;

I understand using htf.escape_sc is important, but how can I use it to display non-Engllish characters?

Thanks in advance for your help!


Tom Kyte
April 11, 2005 - 12:54 pm UTC

Those strange characters are the character converted to an entity reference. But I doubt htf.escape_sc is doing this, as its implementation is really quite simple.

Are you doing this in SQL Workshop in HTML DB? What's your database character set?

I have no problem using htf.escape_sc on Chinese characters in HTML DB.

I am doing it in Page region

A reader, April 11, 2005 - 3:21 pm UTC

I did it on Page Region:

Source code:

for x in (
   select content,COMMENTID from comment where ACCESS_ID=:P15_ap_id order by insert_date) loop    
   htp.p('<pre>');
   --the following line printed Chinese characters correctly
   htp.p(x.content );
   --the following line cannot print Chinese characters correctly
   htp.p(htf.escape_sc(x.content));
    htp.br;
    htp.p('<hr size=5 noshade>');

end loop;

I used the following query to retrieve the database character set:


  1  select * from v$NLS_PARAMETERS
  2* where parameter like '%CHARACTERSET%'
SQL> /

PARAMETER                      VALUE
------------------------------ ------------------------------
NLS_CHARACTERSET               UTF8
NLS_NCHAR_CHARACTERSET         UTF8


I view the page source from the Mozilla Firefox.

if I don't use htf.escape_sc, then the page source will be:
 <pre>
   &#25105;
</pre>

if I use htf.escape_sc, then the page source will be:
 <pre>
   &amp;#25105;
</pre>

So now do you think  htf.escape_sc is causing the problem? If so, any work-around?

Thanks in advance! 

Tom Kyte
April 11, 2005 - 4:27 pm UTC

htf.escape_sc is definitely causing "the problem".

The implementation of escape_sc is:

function escape_sc(ctext in varchar2 character set any_cs)
return varchar2 character set ctext%charset is
begin return(replace(
replace(
replace(
replace(ctext, '&', '&' || 'amp;'),
'"', '&' || 'quot;'),
'<', '&' || 'lt;'),
'>', '&' || 'gt;'));
end;


So your entity reference of &#25105; has the ampersand being correctly replaced with '&amp;'. Thus, when it's rendered on the page, you're not seeing the browser render this entity reference into a character, you're seeing the entity reference itself.

Can I ask, are you physically storing the entity reference '&#25105;' in column comment.content? Or are you storing the character and somewhere on the way out, it's being converted to an entity reference?


I noticed you're using this in a PL/SQL region. But if this is an HTML DB report, you can change the type of the report column (by editing the report column attributes) and change the "Display As" attribute of the column to "Display as Text (escape special characters, does not save state)". This will correctly escape the data without altering your entity reference.

Joel

A reader, April 11, 2005 - 4:54 pm UTC

Thank you Joel for looking at this!

Yes, we physically storing the entity reference '&#25105;' in column comment.content.


We're using the above source code in PL/SQL Procedure (Anonymous block) in a Page region. But we're NOT using an HTML DB report. We put together information and present to a page region. As such we printed out the content via htp.p.

I am not sure if HTML DB report can do this:

Table C2 has a foreign key referring to C1
Table C1 has a foreign key referring to P1

get a row r_p1_1 from parent table P1
for each row in table C1 that refers to r_p1_1 loop
print C1's row (rowxyz) information.
list all the rows in C2 that refer to C1 (rowxyz)
end loop;


Please share with us how we can work around htf.escape_sc trouble.

Tom Kyte
April 11, 2005 - 5:37 pm UTC

Well, if it were me, I'd never be storing entity references in the data in the first place, I'd just store the data. I realize there are situations where only an entity reference is applicable, but for storing Asian characters, I'd just let the database store it as is and you wouldn't be faced with this challenge.

So in your case, if you're interested in escaping the data (presumably for cross-site scripting protection) yet you don't want your Unicode character entity reference itself converted, you might have to write your own escape_sc function which will ignore them, like:

create or replace function escape_sc(ctext in varchar2) return varchar2
IS
AMP CONSTANT varchar2(10) := '&' || 'amp;';
QUOT CONSTANT varchar2(10) := '&' || 'quot;';
LT CONSTANT varchar2(10) := '&' || 'lt;';
GT CONSTANT varchar2(10) := '&' || 'gt;';
begin
return(
replace(
replace(
replace(
replace(
replace(
replace(ctext,'&#','HasH_h0ld3r'),
'&', AMP),
'"', QUOT),
'<', LT),
'>', GT),
'HasH_h0ld3r','&#'));
end;
/

This just converts your &# reference to some unique string (assumes that it would never be in your data) and then converts all other occurrences of & to an entity reference.

Joel

Just to make sure

A reader, April 11, 2005 - 6:24 pm UTC

<quote>
Can I ask, are you physically storing the entity reference '&#25105;' in column comment.content?
</quote>

In order to answer this question, all I did was to log into Oracle via sqlplus and select from that column, I saw &#25105;


<quote>
Well, if it were me, I'd never be storing entity references in the data in the
first place, I'd just store the data.
</quote>

I did store whatever the data entered via HTML DB. The data in the database was inserted via a page item defined as "Textarea w/Character Counter & Spellcheck".


Would you please advise how I could make sure we never store entity references in the database. How can I check whether if I am really storing entity references in the database or not?



Tom Kyte
April 11, 2005 - 10:57 pm UTC

FYI, you may wish to take this thread over to the more appropriate HTML DB OTN forum at: </code> http://forums.oracle.com/forums/forum.jsp?forum=137

Are you sure your browser isn't first converting these to entity references somehow?  When you query this table from SQL*Plus, you see the Unicode entity references in your data?  You may wish to look in your Apache access log to see actually what is being posted.  If in there you see Unicode entity references, then that's what you're somehow submitting to HTML DB.

I just created an application on htmldb.oracle.com without issue (and without saving these characters as Unicode entity references):

http://htmldb.oracle.com/pls/otn/f?p=27775:1 <code>

(for future readers of this post, keep in mind this application may not be there or will have been changed).

Joel


Unicode entity references

A reader, April 12, 2005 - 1:27 am UTC

First of all, your escape_sc did work well in my environment! Thank you very much for that!

Second, I did what you said "to look in your Apache access log to see actually what is being posted."
On the server side, I did tail -f access.log on the Apache/Apache/log, then in the browser, I inserted queryTest<foreign characters> via item of HTML DB --> Oracle.
Unfortunately I could not capture queryTest in the entire access.log

Third, I don't know how to tell whether my browser isn't converting my inputs to entity references somehow or not.

I don't know how you implemented
</code> http://htmldb.oracle.com/pls/otn/f?p=27775:1 <code>
did you store it into database and then use htf.escape_sc to print out?

Many thanks to Tom and Joel, I just try to understand more of "Unicode entity references". Specifically

How to tell if I store entity references in the data in Oracle?
How to tell if my browser is converting users' input to entity references ? If so how to rid it out and just store the data into Oracle

Tom Kyte
April 12, 2005 - 7:57 am UTC

The application is really quite simple. I did not use htf.escape_sc at all.

I created a table with a VARCHAR2(4000) column. I created a report on that column (select <mytext_column> from <mytable>. I edited the report column attributes and in the Display As attribute I changed that to "Display as Text (escape special characters, does not save state)". Then, I added an item called P1_DATA which is type Textarea w/ Character Counter & Spellcheck. Added a couple buttons and then added a process to insert :P1_DATA into the table. That's it. No htf.escape_sc required at all.

If you wish to determine if you're storing entity references in your data in Oracle, simply query the table.column from SQL Command Process in HTML DB or SQL*Plus. If it appears like you're storing these Unicode entity references in your data, then you need to figure out how/why they're getting there. The first place I would start is to examine the access logs of what is being posted to your page.

Decode special characters of report parameters - Very Useful

xxx, April 19, 2005 - 2:24 pm UTC

I had this problem and was looking over in metalink for a solution and didn't find any. I created a TAR and they asked me to surrond my parameters with double quotes and told me work arounds.

I had to fix this problem since we had it in many places and i looked here and found the solution. Just what I was looking for.

Thanks Tom.

rd

A reader, June 17, 2005 - 10:32 am UTC

TOM,

I am using the following in a web application.

htp.p('<td ><a href="javascript:showNotes('''||url_encode(v_notes)||''');" > click here </a></td>');

function showNotes(note)
{
var url="'||c_url||'.fls_popups.proc_display_note?pNotes=" +note;
var win=window.open(url, "", "");
}

I am facing problems in the following cases.

1)When v_notes has a single quote in the text clicking on " click here " is resulting an error.
2) When v_notes has a double quote then popup window appears but the text is not displayed in the text area of popup window.

Kindly help with work around .

TIA



Tom Kyte
June 17, 2005 - 3:47 pm UTC

so, in javascript -- how do you put a quote in a string?

perhaps it is this:



How\'s this for a string?


(guessing). If so,

htf.escape_sc( replace(v_notes,'''', '\''' ) )

(not sure you want to url encode, you want to fix <>& and such...)



Inserting text that contains &

V, January 11, 2006 - 4:19 pm UTC

Tom,
How would you insert a string that contains a &

insert int a values ('this & that');

Tom Kyte
January 12, 2006 - 10:48 am UTC

in sqlplus - & is a "special character", so this only applies to sqlplus:


sql> SET DEFINE OFF


then & will not be special anymore.

function

Sam, May 10, 2008 - 12:05 am UTC

Tom:

would this function protect any XSS vulnerabilty? is there still need to filter input data or the escape of special characters would be enough when retrieving data from db.

create or replace function escape_sc(ctext in varchar2) return varchar2
IS
AMP CONSTANT varchar2(10) := '&' || 'amp;';
QUOT CONSTANT varchar2(10) := '&' || 'quot;';
LT CONSTANT varchar2(10) := '&' || 'lt;';
GT CONSTANT varchar2(10) := '&' || 'gt;';
begin
return(
replace(
replace(
replace(
replace(
replace(
replace(ctext,'&#','&#'),
'&', AMP),
'"', QUOT),
'<', LT),
'>', GT),
'&#','&#'));
end;

Tom Kyte
May 12, 2008 - 12:11 pm UTC

just use htf.escape_sc

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library