looks like good to go with enterprise edition
John Hurley, July 22, 2005 - 1:21 pm UTC
Yes I did follow the link you cited on licensing. But it doesn't appear to mention the Transparent encryption specifically.
It looks like if you run EE on 10g Release 2 you have it already otherwise if running SE you have to license advanced security if I understand correctly.
July 22, 2005 - 1:54 pm UTC
The sentence pointed says:
...
Oracle Advanced Security provides transparent data encryption of data stored in the database....
without ASO, transparent data encryption support is not there.
ASO is an option, that is how I read it.
Amarjyoti Dewri, August 06, 2005 - 3:16 pm UTC
Hi Tom,
Regarding TDE, I wanted to know how to setup multiple databases (Oracle10g Rel2) using different locations of the DIRECTORY path in WALLET_LCOATION parameter of SQLNET.ORA file.
As I could understand, the default location is $ORACLE_BASE/admin/$ORACLE_SID/wallet. This works fine as long as all the databases are using the default location. But say for security reasons I want the wallet to be placed in different locations for different databases. How do we do that?
Thanks and Regards
August 06, 2005 - 5:13 pm UTC
I haven't tested this, but -- the default location of the sqlnet.ora which may contain the path to the wallet may be overridden via the TNS_ADMIN environment variable.
But, why "for security reasons", what would you be gaining by putting it elsewhere? Given that any sysdba for database 1 would be able to figure out where it is for database 2 immediately any how?
What do you think you would be buying by doing this?
Thanks for he response
Amarjyoti Dewri, August 07, 2005 - 12:15 am UTC
Hi Tom,
Thanks for replying. Actually I was thinking about the example if disks are stolen. The chances of getting the disks containing the datafiles as well as the wallet together might get reduced. And if we have different wallets in different locations for different databases, the chanches are reduced further.
Location of sqlnet.ora can be controled by TNS_ADMIN parameter. But the parameters WALLET_LOCATION/ENCRYPTION_WALLET_LOCATION in sqlnet.ora is one per ORACLE_HOME. Can we have multiple directories mentioned in parameter ENCRYPTION_WALLET_LOCATION/WALLET_LOCATION?
This is what I understood till now. By default database searches for ewallet.p12 file in path mentioned by ENCRYPTION_WALLET_LOCATION, if it's not mentioned then in WALLET_LOCATION, else picks it from the default database location which is $ORACLE_BASE/admin/$ORACLE_SID/wallet.
If we mention the directory path in ENCRYPTION_WALLET_LOCATION, then all databases will be accessing the same ewallet.p12 file which can have one master key common for all. This is what I want to avoid, I do not want all databases to have same master key, or the same ewallet.12 file (as suggested by the manual, have different wallets for different databases).
One work around that I could think of and did a test while writing this review, was by creating a soft link. Create ewallet.p12 files for databases in their respective default database locations with different master keys for each database. Then move them to a common location or different location and rename them to ewallet$ORACLE_SID.p12. Then create soft links for these files with name ewallet.p12 in the default database location. This is working...
Thanks and Regards
Amar
August 07, 2005 - 9:35 am UTC
all you need is
a) wallets over here
b) disks over there
you need "two" not "N"
why do you say that about the sqlnet.ora? If you have tns_admin pointing to /d01 for instance 1 and /d02 for instance 2, that was my point, if each instance has their own sqlnet.ora.....
but I don't see the point, even if they steal your datafiles and your wallet, how are they going to open it?
And how does putting wallets all over the place help? you need two locations in your concept.
About Stealing the Wallet
Arup Nanda, August 08, 2005 - 1:12 am UTC
I apologize for the uninvited intrusion to Tom's forum; but I thought I may have some suggestions for the reader who asked this question about the datafiles and/or wallets being stolen; and perhaps try to put his mind at ease.
First, here is how the TDE mechanism functions. The wallet is not opened by default when the database instance starts. The DBA opens it by issuing "alter system set encryption wallet open authenticated by <password>". It requires a password to be opened, otherwise the masterkey is pretty much useless.
Second, the masterkey is stored inside the wallet in an encrypted manner; whihc means even if the attacker steals it, he will not be able to use it much. So it does not hurt to have the same masterkey for all databases. In other words the risk is the same, not different.
Now, let's discus scenarios. Scenario# 1 - The attacker steals the disks, or the BCV copy of the disks or the backup tape files. He then creates an instance on a test machine, starts up the instance, does an incomplete recovry from the tapes (or disks). Can he read the encrypted columns? Nope. After opening the instance and before accessing the columns, he has to open the wallet, which requires a password. Yes, he could use brute force to guess the password; but that makes any user security inside Oracle vulnerable, not just this one; you could even brute force SYS password, at least in thoery.
One suggestion I would have for TDE team is to enable profiles like they are for users, which enforce better passwords and lock accounts after a while.
Scenario#2: The database is running and the the attacker has gained access to the database. This scenario has one difference - the wallet is open. Sure the attacker can read the .p12 file, but for what? He can't read the encrypted value and he has no idea how the value has been encrypted.
Remember, TDE is not for selective en/decryption of data, i.e. decryption is made available to some users only. All users who have access to the table (or the row, if under VPD) will be given access to the decrypted value. The attacker does not need to know the masterkey to decrypt them. If you must protect the column from a type of users, e.g. encrypted SALARY should be protected from all non-HR users, merely create a column-sensitive VPD policy on the table which will prevent showing the columns based on userid. You could go creative in defining the policy in any way you want - based on application contexts, based on IP addresses, the list goes on.
I happen to think that TDE is a very, very good feature in Oracle 10gR2, and a long-overdue one. It does have its shortcomings such as inbality to define function based indexes, being part of partition keys, etc.; but it can be used to acheive some quick compliance with the myriads of regulations you may have to deal with now.
Hope this helps.
August 08, 2005 - 7:45 am UTC
A much more detailed way of saying:
...
but I don't see the point, even if they steal your datafiles and your wallet,
how are they going to open it?
.....
thanks for the input Arup, appreciate it.
Amarjyoti Dewri, August 08, 2005 - 1:31 am UTC
Hi Tom,
Thanks for the reply.
>> why do you say that about the sqlnet.ora? If you have tns_admin
>> pointing to /d01
>> for instance 1 and /d02 for instance 2, that was my point, if each
>> instance has
>> their own sqlnet.ora.....
Oh, you were saying about RAC environment, where every instance will have its own ORACLE_HOME and sqlnet.ora. I think I got overexcited by the feature and did not think in that aspect. I was talking about a single server with single Oracle10g Rel2 ORACLE_HOME hosting multiple databases.
>>but I don't see the point, even if they steal your datafiles and your
>>wallet,
>>how are they going to open it?
Yes, you are right; I was under the impression that doing a decrypt after generating new master key will open the encrypted column. But it does not work that way. It throws error; ORA-28362: master key not found, confirming that you will still need the original master key (generating new wallet does not work unless generated using same master key) to decrypt.
>>And how does putting wallets all over the place help? You need two
>>locations in your concept
Here also I was talking about single ORACLE_HOME for multiple databases. But its clear now, unless master key is available data cannot be accessed.
One more thing I was doubtful about was that, when we open the wallet its open for all users, I could not find any thing saying open only for one particular user. So till the wallet is open anyone having the access to the table is able to see the data from the encrypted column. Only closing of wallet disables access to the encrypted column. So itÂ’s something like a bank manager has the master key to open the wallet, and without his permission no one access the encrypted data. But when he opens the wallet, others can see the data as well...
To prevent this wouldn't we need the same FGA or view where data was restricted user wise or role wise.
So I was wondering where actually is TDE useful?
Thanks and Regards
Amar
August 08, 2005 - 7:47 am UTC
NO, i was not talking about rac.
I was talking about:
export TNS_ADMIN = /d01
export ORACLE_SID = foo
sqlplus "/ as sysdba" <<EOF
startup
EOF
export TNS_ADMIN = /d02
export ORACLE_SID = bar
sqlplus "/ as sysdba" <<EOF
startup
EOF
foo and bar have different "?/network/admin" directories now because they have different tns_admin's -- but this would be confusing at best and not achieve anything "extra security wise"
I think Arup's comments above answers the last question.
Amarjyoti Dewri, August 08, 2005 - 1:41 am UTC
Hi Again,
Sorry I missed Arup's answer while I was typing. What I understand is that you need to combine both VPD and TDE together to provide a better and robust security at both application level and datalevel. Am I correct?
So you can encrypt the data with Transparent Data Encryption and prevent access to confidential data while wallet is open using Vitual Private Database where only specific users can access data, and that too will not be available when you close the wallet.
Thanks and Regards
August 08, 2005 - 7:49 am UTC
TDE is to protect data at rest, someone steals your backup - so what, they cannot read the sensitive information.
VPD (FGAC) is to protect data from being accessed. From end users, from people who are supposed to see some of the data but perhaps not all of the data.
For Arup Nanda
Amarjyoti Dewri, August 08, 2005 - 2:43 am UTC
<Quote>First, here is how the TDE mechanism functions. The wallet is not opened by
default when the database instance starts. The DBA opens it by issuing "alter
system set encryption wallet open authenticated by <password>". It requires a
password to be opened, otherwise the masterkey is pretty much useless.
</Quote>
But Arup, there is an option available for keeping the wallet open all the time to be used for Single Sign On Applications (only scenario where it should have autologin as suggested by Oracle Manuals). Turing on autologin in wallet manager enables the wallet to be open all the time. It create a cwallet.sso file for this. I did some testing on this which can be accessed here at the bottom of this page </code>
http://adewri.modblog.com/?show=custom&page=2658 <code>So you do not have to reopen the wallet when the instance is restarted. What precautions should be taken in such a scenario...
Thanks and Regards
August 08, 2005 - 7:59 am UTC
do not use that would be the simple response.
where does it say in order to use SSO with TDE you must do this for the DATABASE? I believe that (setting auto-login) is in regards to the user wallet, with the user credentials.
Thanks a lot
Amarjyoti Dewri, August 08, 2005 - 8:33 am UTC
Hi Tom,
[quote]I believe that (setting auto-login) is in regards to the user wallet, with the user credentials. [/quote]
Yes, you are right , it is in regards to user credentials (you miss important points when you don't read carefully. I will be careful next time).
I would like to thank you and Arup Nanda for clarifying all my doubts.
Thanks and Regards
Its all nice and beautiful but....
Dominic Dougherty, September 19, 2005 - 8:53 pm UTC
Great website, Very helpful, Oracle Rocks!!
One thing regarding TDE in 10gr2, I know it encyrpt data and it is transparent but does it do access control. It does separate dba from Security Officers But will it be able to restrict decryption to specific users.
September 19, 2005 - 11:45 pm UTC
it is *not* access control -- it is protection from THEFT of data.
vpd (dbms_rls) is for access control
grant is for access control
abotu storing keys in memory
Antonio, September 20, 2005 - 5:17 am UTC
Tom, do you know if the database engine is storing the decrypting-keys in memory locked? My reasoning is that if they are not forcefully locked (mlock() syscall on unix), they could be written out to swap space and then recovered from there.
September 20, 2005 - 10:10 am UTC
I don't know that low level detail - you'd have to ask support to track that down via development.
Operational Procedures
VA, September 20, 2005 - 3:10 pm UTC
<quote>
but I don't see the point, even if they steal your datafiles and your wallet, how are they going to open it?
</quote>
If the disks/tapes containing both the datafiles and the wallet are stolen, you still need to "alter system set encryption wallet open authenticated by <password>" to access the data.
But, in many environments, the database open is a part of the server startup procedure and this "command" above would be in some home-grown "dbstart" script or some GUI/OEM equivalent.
So, if the "attackers" gets hold of all the disks involved, they could (theoretically) access the data, couldnt they?
Thanks
September 21, 2005 - 1:27 pm UTC
well, that sounds like a "in many environments" issue doesn't it?
Sure, if you put everything there -- well, what can we do?
Wallet could be on another system even as well.
But yes, you hard code passwords and - well, what can we do?
A reader, September 21, 2005 - 12:37 am UTC
To VA from New Jersey, USA
>>So, if the "attackers" gets hold of all the disks involved, >>they could (theoretically) access the data, couldnt they?
Then you got bigger problems then security :-)
Some TDE notes
Adam Musch, October 26, 2005 - 3:45 pm UTC
One thing missing from the documentation (or perhaps implied that I missed) was that after setting ENCRYPTION_WALLET_LOCATION or ENCRYPTION_LOCATION in sqlnet.ora, one must shutdown/startup the database in order to create a wallet for TDE.
As a developer/DBA in the banking world, TDE is a godsend compared to hardware/software encryption. Compared to our systems running Oracle 9.2 with a java front-end, using a hardware/software solution with views and INSTEAD-OF triggers imposes about a 50% performance penalty, requiring twice as much CPU to comply with industry requirements.
The same data and front end running with TDE (so obviously 10gr2) not only had no penalty, but had approximately 4 times better throughput than the original 9.2 system without encryption!
I think it'll be a pretty easy sell to say:
Well, we can double our hardware and database licensure cost, or:
We can license the Advanced Security option for 10k/cpu list.
One thing about TDE though: It doesn't support CLOB/BLOB columns. However, digging through the 10gr2 documentation (specifically the DBMS_CRYPTO spec) shows that one can use DBMS_CRYPTO to encrypt LOB columns with AES128/192/256 and 3DES, all of which are supported natively with TDE.
Why didn't they make LOB columns natively supported through TDE? It looks like all the pieces are there, but they just didn't stitch them all together. I've no desire to manually attempt to perform key management and use DBMS_CRYPTO to encrypt those columns.
Do we think/know/suspect that support for LOB columns with TDE will be forthcoming, in either a major or a minor release?
Thanks!
October 27, 2005 - 4:46 am UTC
I'll ask and see if I can find out. Problem with lobs is you have piecewise access to them - you can dbms_lob.write bytes 100 - 200. They are like files - not like a "variable". There are complexities involved here you haven't thought about that make doing it to lobs really really hard.
TDE and Relational Integrity
Adam Musch, October 31, 2005 - 12:17 pm UTC
Any word on whether TDE will support RI in a future release?
Currently, each table appears to have its own initialization vector for encryption (salt), meaning a value encrypted in one table differs from a value encrypted in another table. Salt can also be added at the column level, so that repeating values in the column have different ciphertexts. However, if specifying the IDENTIFIED BY clause of the ENCRYPT command as well as the NO SALT clause, the underlying ciphertext is identical both across tables and across columns. (Thanks to markmal at dizwell for pointing this out.)
Example:
drop table t_owners;
drop table t_tables;
create table t_owners as select * from all_users;
create table t_tables as select * from all_tables;
alter table t_owners
add constraint pk_t_owners primary key (username)
using index;
alter table t_tables
add constraint pk_t_tables primary key (owner, table_name)
using index;
alter table t_owners
modify (username encrypt identified by "encryption_key" no salt);
alter table t_tables
modify (owner encrypt identified by "encryption_key" no salt);
select dbms_rowid.rowid_row_number(rowid) as rn,
DBMS_ROWID.ROWID_BLOCK_NUMBER(rowid) as bn
from t_owners where username = 'WMSYS' AND ROWNUM = 1;
SELECT dbms_rowid.rowid_row_number(rowid) as rn,
DBMS_ROWID.ROWID_BLOCK_NUMBER(rowid) as bn
FROM T_TABLES WHERE OWNER = 'WMSYS' AND ROWNUM = 1;
After dumping the two blocks, here was the pertinent data:
Block dump of t_owners:
tab 0, row 25, @0x15e0
tl: 51 fb: --H-FL-- lb: 0x2 cc: 3
col 0: [36]
54 7d d1 5b e2 12 16 83 86 6c c5 bb 1a 81 52 f7 12 cf f2 26 a7 6b cd aa 53
53 57 54 24 6f a6 04 a5 a1 76 fb
col 1: [ 2] c1 1a
col 2: [ 7] 78 69 0a 18 08 22 06
Block dump of t_tables
tab 0, row 8, @0x16c5
tl: 245 fb: --H-FL-- lb: 0x2 cc: 49
col 0: [36]
54 7d d1 5b e2 12 16 83 86 6c c5 bb 1a 81 52 f7 12 cf f2 26 a7 6b cd aa 53
53 57 54 24 6f a6 04 a5 a1 76 fb
col 1: [16] 57 4d 24 4e 45 58 54 56 45 52 5f 54 41 42 4c 45
col 2: [ 6] 53 59 53 41 55 58
col 3: *NULL*
...
Note that the VARCHAR2(30) column is now a 36 byte column (which is expected for encryption to round up to a multiple of 8), and that the ciphertext for the two is identical.
So it certainly appears that TDE could support RI between columns encrypted with the same value of the IDENTIFIED BY clause.
Any idea why it wasn't implemented? Or (oh, please, oh, please) it'll be resolved with a patch?
November 01, 2005 - 5:09 am UTC
I don't know "why", I can comment that if the primary key needs to be encrypted, I would think 500 times before using that as my primary key.
Doesn't seem like it would be the right column to have exposed as the primary key at that point.
TDE and Legacy Systems
Adam Musch, November 01, 2005 - 8:58 am UTC
Tom:
The issue is not with a new design; it's with adapting TDE with existing systems where new industry requirements (such as HIPAA and Visa/Mastercard's PCI) require database encryption. The natural key (let's say, an account number within a banking system) now must be encrypted. The requirement didn't exist when the systems were originally designed, and lots of internal developers and vendors are scrambling to find a way to comply.
I do agree with you that now, it would be obvious to use a surrogate/synthetic key in this case. However, right now that would involve reimplementing the key structures for lots and lots of tables, which means lots and lots of code rewrite.
I'll probably end up using triggers to enforce RI -- it's not good, but it is better than not having the RI at all. The tables in question are fairly static; there's not lots of insert/delete activity on the tables in question, and the keys are already immutable.
I will watch out for when TDE does support RI where the column "public" keys are identical and no salt is added at the row level.
Thanks for all of your assistance, both on this issue and all the other issues you help the community with.
November 01, 2005 - 11:02 am UTC
becareful - you show me your code for RI in triggers and I'll tell you why it is buggy :) (high probability - give it a go, I'll look at them if you post them and tell you why they won't work - or verify that you got it - it is very very tricky and involves typically the use of "lock table", like we do with unindexed foreign keys).
Best place to get your ideas into the product however is metalink - you should file an enhancement requestion.
Encrypting all columns
Partha, November 16, 2005 - 6:53 am UTC
We are developing a HTMLDB application that can be accessed through the internet. All the data is very confidential. The application / data itself is quite small with around 30 tables and 100 columns in all and around 10,000 records. One of the suggestions was to encrypt all the user visible columns in all the tables (ofcourse not the clobs / foriegn keys etc). But currently we are using the standard edition one. Is upgrading to EE and buying a ASO for this a viable option? That would also mean making all the columns 'encypt'. Is there any other option ?
November 16, 2005 - 5:09 pm UTC
who are you trying to protect the data from?
The transparent in transparent data encryption means something - the data is transparently encrypted for viewing by anyone that is allowed to see it.
TDE is useful to protect against data theft (someone stealing your database).
TDE is *not access control*
You sound more like "access control", not encryption.
So, who are you trying to protect the data from?
Followup to review "Some TDE notes October 26, 2005"
A reader, December 22, 2005 - 10:48 am UTC
In your follow up to the review titled "Some TDE notes October 26, 2005" you mentioned that you will ask about when Oracle 10g will support encryption of LOBs. I was wondering if you got an answer.
If not, is there a work around, while still using TDE, to encrypt LOBs?
Thanks
December 22, 2005 - 11:33 am UTC
there not plans to do so currently - no. you can use the procedural API's to do so, but obviously that is not "transparent"
Startup scripts?
Ben, March 13, 2006 - 3:34 pm UTC
Tom,
As I understand it, once the wallet is created, it must be explicitly opened every time the database starts up to grant users access to the data in the encrypted column.
How do we deal with that in the context of automated RMAN scripts, run in cron jobs, that shut down the database, mount it for a full backup, and open the database? If we add a command in there to open the wallet, including the password, doesn't that defeat the purpose of encrypting the master key?
March 14, 2006 - 9:55 am UTC
You can make the wallet "releasable to the database automatically" using a director (LDAP repository).
Yes, if you hard code a password in a script, that defeats the purpose.
Unattended startup and shutdowns are something I've written about many hundreds of times in the past - I never do them. They are fraught with issues. If you have a serious database, with serious data (so serious you would consider encrypting it), using enterprise edition (which you must be) you would never be shutting down for a backup. You would be in archive log mode, you would be using hot backups.
Applying TDE to big table
Mihajlo, April 06, 2006 - 2:41 pm UTC
Hi Tom,
I was testing encryption (TDE) of certain columns in a table which contains 27 million records.
Adding TDE encryption to VARCHAR2 column took 1 hour and 30 minutes. Adding TDE encryption to two NUMBER columns at the same time took less than 5 hours.
Can you explain me why it takes so much time for encrypting the data? I understand the data needs to be converted and than saved in the table, but is there anything which can help me to reduce this time.
April 07, 2006 - 8:20 pm UTC
you need to basically rewrite the entire table when you are doing this (logged operation as well). Also - encryption is reputed to be somewhat CPU intensive.
Once added, it is unlikely you will notice the difference in most cases - sure, there are extreme cases that you could (on the decrypt) - but it is unlikely.
RE: Applying TDE to big table
Mihajlo, April 10, 2006 - 3:00 pm UTC
What do you mean with "rewrite the entire table"?
Do you think all columns data need to be modified, or only for the columns which are encrypted with TDE?
Thanks
April 11, 2006 - 11:41 am UTC
the raw encrypted data is bigger than the original data. The entire table is pretty much rewritten, just like an
update t set x = x || 'x';
would - even if T has 100 columns - you are modifying every row in every block, table is rewritten in effect.
dbms_crypto
A reader, September 17, 2006 - 4:06 pm UTC
Is dbms_crypto available to ORacle 10gR2 Standard Edition?
September 18, 2006 - 1:03 am UTC
yes
DataGuard and TDE
Tino, March 05, 2007 - 10:16 am UTC
Will TDE work with DataGuard due to having the master key in a wallet outside of the database? How do we go about keeping the master keys in-synch (primary and standby sites)?
March 05, 2007 - 2:12 pm UTC
only using a physical Standby Database, the wallet must be copied manually.
Also , you need to specify the ENCRYPTION_WALLET_LOCATION in the sqlnet.ora file on the standby database.
DataGuard and TDE
Tino, March 05, 2007 - 5:14 pm UTC
So based on your response, I cannot implement TDE if I am configuring a Logical Standby database? Is that correct?
March 05, 2007 - 8:48 pm UTC
correct - the generation of the logical change record would sort of break the entire thing.
Good Memory or Good Search Engine
Tino, March 06, 2007 - 2:29 am UTC
You are pretty good, I overlooked it in the documentation under "Unsupported Data Types in a Logical Standby Database".
CTAS with TDE
Skip, March 15, 2007 - 10:36 am UTC
Is there a way around the CTAS issue with TDE? I understand that TDE is for data theft, or if someone steals the database. But if I have a production table that has say account number and it is encrypted, and my user does a CTAS as select * from the production table, the resulting table is not encrypted and therefore I now have unencrypted data in the database ready to be stolen. I am not able to prevent users from creating their own tables as it is integral to their daily work. How can I ensure that any tables they create are encrypted, DDL trigger?
March 15, 2007 - 12:32 pm UTC
seems to me you have a serious conflict between your stated needs (encryption to prevent theft) and your application (users can willy nilly copy the data all over the place at will)
you could create a DDL trigger that looks at the table being created and the table it is being created from and errors out of any of the columns are encrypted...
archive logs?
dulu, June 01, 2007 - 10:43 am UTC
Hi Tom
Please can you confirm that if you have all your data encrypted using TDE, then also the backups and archive logs are also encrypted?
Thanks
June 01, 2007 - 1:16 pm UTC
you would not have all of your data encrypted with TDE, you would selectively apply this only to columns that MUST be protected.
and those columns would be encrypted in the backups and everywhere.
a question on TDE?
Sona, July 12, 2007 - 7:44 pm UTC
Hi Tom,
1.How do we decrypt the columns encrypted with TDE, when the wallet password is forgotten or unknown since it was created by someone else?
2.How do we reset the wallet password?
3.I understand that expdp is the option to export table encrypted with TDE and we need the master key to decrypt the encrypted data. How do we import the table in another database in different host, which has a different wallet in use/open?
Thanks,
Sona
July 13, 2007 - 10:11 am UTC
Re: Questions on TDE
sona, July 13, 2007 - 7:32 pm UTC
Hi Tom,
Thanks much for the answers.
1. Regarding 'RESET' -- I just wanted to know how do we change the password of the wallet if it has been compromised in any ways?
For eg: We create the wallet using the following command.
alter system set encryption key authenticated by "passwordwal";
How do I change password to 'newpassword'?
2. Regarding Backup/Recovery:
I understand that we need to backup wallet along with the database backup for ensuring restore and recovery of the table encrypted with TDE.
Suppose we want to recover the table by using RMAN restore/recover tablespace, in another database which already has a wallet in use.
How do we use the wallet of the Database1 restored from backup to open the table restored/recovered in Database2?
3. How does TDE work with standby databases? Do we need to have wallets in standby hosts too?
Thanks,
Sona
July 13, 2007 - 7:55 pm UTC
1) check out that chapter i pointed to, all covered there.
2) see number 1
3) same (yes, you would in order to open the database)
TDE Performance Increases?
Tom, September 29, 2007 - 1:59 am UTC
Hi Tom - Great site, great name, and looking forward to OOW. In trying to gauge the impact of installing TDE, we've run some tests, and find that btree indexes seem considerably faster (200 million row test table, 4 times faster, with less CPU microseconds). Is that expected due to the increased uniqueness of the indexed value? For example, less ids look like 1234%? Or am I getting odd results from a flawed test run?
Hope this isn't a repeat question (or should have been flagged as a new question), but I've not been able to track down much regarding TDE impacts. We're seeing slower FTS and partition scans (more blocks, but that's expected), and faster index range scans with less CPU ... completely unexpected!
Thanks for this site, and all your expertise!
Tom
October 03, 2007 - 1:40 pm UTC
TDE cannot do index range scans - what did your test actually test?
More TDE Performance Info
Tom, October 04, 2007 - 11:21 am UTC
Well, not an index range scan in the sense of "ID like '12345%', but index lookups (ie, id='123451234'). I always see the explain plan call that lookup an index range scan, so guess I picked up some poor terminology.
When we look at our encrypted index, we see that it's roughly twice as large after encryption, (clustering factor is about the same on both, data was not sorted) and expected to see some degradation due to the increased IO. However, even with flushing buffer_cache, and shared pool, running our tests with new sets of ids, running encrypted query test first or running them after it's equivilant unencrypted counterpart, we still see less logical reads, less physical reads, less CPU and less elapsed time.
Here's some stats from our runs, do you have any tests that show index lookups are actually faster? Our indexed column is varchar2, and the values are not random.
Encrypted Test:
------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Pstart| Pstop |
------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1938K| 123M| 3285 (6)| | |
| 1 | HASH GROUP BY | | 1938K| 123M| 3285 (6)| | |
| 2 | PARTITION RANGE ALL | | 1938K| 123M| 3110 (1)| 1 | 5 |
| 3 | TABLE ACCESS BY LOCAL INDEX ROWID| TEST_DETAIL2 | 1938K| 123M| 3110 (1)| 1 | 5 |
|* 4 | INDEX RANGE SCAN | TEST_DETAIL2_NDX7 | 775K| | 3106 (1)| 1 | 5 |
------------------------------------------------------------------------------------------------------------------
select ENC_TEST_ID, sum(AMOUNT) from test_detail2 where ENC_TEST_ID = '1234567890123456789' group by ENC_TEST_ID;
Elapsed: 00:00:00.39
Statistics
----------------------------------------------------------
1064 recursive calls
0 db block gets
285 consistent gets
62 physical reads
0 redo size
773 bytes sent via SQL*Net to client
488 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
13 sorts (memory)
0 sorts (disk)
1 rows processed
From v$sqlstats:
CPU_TIME: 40325
ELAPSED_TIME: 376289
IO_WAIT_TM: 340019
Unencrypted Test:
select TEST_ID, sum(AMOUNT) from test_detail where TEST_ID = '1234567890123456789' group by TEST_ID;
Elapsed: 00:00:02.90
Execution Plan
----------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Pstart| Pstop |
-----------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 66 | 8 (13)| | |
| 1 | HASH GROUP BY | | 2 | 66 | 8 (13)| | |
| 2 | PARTITION RANGE ALL | | 2 | 66 | 7 (0)| 1 | 5 |
| 3 | TABLE ACCESS BY LOCAL INDEX ROWID| TEST_DETAIL | 2 | 66 | 7 (0)| 1 | 5 |
|* 4 | INDEX RANGE SCAN | TEST_DETAIL_NDX2 | 2 | | 6 (0)| 1 | 5 |
-----------------------------------------------------------------------------------------------------------------
Statistics
----------------------------------------------------------
1034 recursive calls
0 db block gets
2974 consistent gets
191 physical reads
0 redo size
779 bytes sent via SQL*Net to client
488 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
12 sorts (memory)
0 sorts (disk)
1 rows processed
From v$sqlstats:
CPU_TIME: 450156
ELAPSED_TIME: 2827453
IO_WAIT_TM: 2360136
TDE Unexpected Performance Explained?
Tom, October 05, 2007 - 10:56 am UTC
Hi Tom - We found the section of sql that was causing the discrepancy in our timings in our 10046 trace file. For an unencrypted query, Oracle ran the following, accounting for the majority of our overhead. On the unencrypted side, this took zero seconds ...
SELECT /* OPT_DYN_SAMP */ /*+ ALL_ROWS IGNORE_WHERE_CLAUSE
NO_PARALLEL(SAMPLESUB) opt_param('parallel_execution_enabled', 'false')
NO_PARALLEL_INDEX(SAMPLESUB) NO_SQL_TUNE */ NVL(SUM(C1),:"SYS_B_0"),
NVL(SUM(C2),:"SYS_B_1")
FROM
(SELECT /*+ IGNORE_WHERE_CLAUSE NO_PARALLEL("TEST_DETAIL")
FULL("TEST_DETAIL") NO_PARALLEL_INDEX("TEST_DETAIL") */
:"SYS_B_2" AS C1, CASE WHEN "TEST_DETAIL"."TEST_ID"=
:"SYS_B_3" THEN :"SYS_B_4" ELSE :"SYS_B_5" END AS C2 FROM
"TEST_DETAIL" SAMPLE BLOCK (:"SYS_B_6" , :"SYS_B_7") SEED (:"SYS_B_8")
"TEST_DETAIL") SAMPLESUB
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.01 0.00 0 0 0 0
Fetch 1 0.46 2.13 148 2833 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.47 2.14 148 2833 0 1
Our DBAs are still looking into the difference, but at least now we can safely say that TDE is not boosting our performance...
Thanks!
TDE column with Replication is not supported
A reader, January 18, 2008 - 1:08 pm UTC
Tom,
We replicate a table that we turned TDE on. We use Shareplex replication to replicate this table from Production to Reporting. Shareplex does not support TDE and in my research -I have not come across any replication -even Streams support for TDE. can you please comment..
Regards
Praveen
Here's that URL again
Stew Ashton, January 20, 2008 - 3:16 am UTC
January 20, 2008 - 7:59 am UTC
I deem it not a bug, I need the # b and # B things to function, so be it.
Re : you deem it not a bug
Stew Ashton, January 20, 2008 - 5:03 pm UTC
Sorry about using the "b" word, I had "b"s on the brain :)
Granted, you need # b to function, but do you need it to function inside a URL?
Wouldn't it be nice if all the URLs functioned too?
The code could be changed to either ignore tag formatting within a URL, or change "# b" to "## bb" within a URL, which is what I did by hand. Might not that keep you happy and help out the folks who are going to click on these links?
My two ## worth :)
TDE with PKI key pairs certificate
A reader, June 24, 2008 - 4:48 pm UTC
Hi Tom,
We want to implement TDE with PKI key pairs currently being used in the company. But when I generated the user certificate, it has
KeyUsage=Digital Signature, Key Encipherment
As Oracle suggests that the KeyUsage field should be marked for encryption or left blank.
Per Oracle doc:
===========================================
To use transparent data encryption with PKI key pairs, the issuing certificate authority must be able to issue X.509v3 certificates with the key usage field marked for encryption.
==============
Remark: for a certificate to work this way, the certificate authority must not use any Key Usage specification to sign the certificate. If you have problems seeing the certificate in V$WALLET, create a new certificate without any Key Usage specifications.
To verify the certificate, export it from the wallet using OWM and give the file an extension .crt.
For example: mycertificate.crt,
then double-click the certificate, or right-click the certificate,
then click Open , the Certificate dialog box appears,
go to [Details] and Show: [Extensions Only],
this must only include: [ Basic Constraints, Netscape Comment, Subject Key Identifier, Authority Key Identifier ]
and not [Key Usage] .
I am a little confused as it indicates KeyUsage "marked for encryption" in first paragraph, while the second paragraph says "certificate without any Key Usage specifications".
I talked to our security staff, they do not know how to have the certificate specified by Oracle generated. The certificates our website generated are with KeyUsage specified. Do you have any insights on this?
Thanks,
Issue with Oracle 8i talking to Oracle 10g with Encryption
Anne, February 06, 2009 - 2:47 pm UTC
Hi Tom,
I hope my question pertains to this topic.
We have an Oracle 8i database that talks to a remote Oracle 10g database using a database link.
When querying the remotetable from our 8i database, we now get this error :
SQL> select version from v$instance;
VERSION
-----------------
8.1.5.0.0
SQL> desc remote_table@remotedblink;
ERROR:
ORA-12650: No common encryption or data integrity algorithm
This has been working fine until early February 2009, when the remote site turned on Enforced Encryption.
The tech person at the remote site tells me that Oracle 8i database in not capable of talking to an Oracle 10g database with encryption turned on.
I googled around and searched Metalink, but did not find any documentation for this.
Please advise. Would appreciate any help.
Thanks,
Anne
February 06, 2009 - 4:17 pm UTC
do you have ASO (advanced security) licensed and available on your (very much way out of support) 8.1.5 database?
Issue with Oracle 8i talking to Oracle 10g with Encryption
Anne, February 06, 2009 - 4:28 pm UTC
Hi Tom,
I like your "very much way out of support" comment - I'm surprised that's all you said!!! :-)
No, we do not have ASO (advanced security) licensed and available on the 8.1.5 database.
Thanks,
Anne
February 09, 2009 - 5:43 pm UTC
you would need it - they are using ASO, you would need the client support that allows you to talk to an ASO 'protected' database - they are demanding encrypted protocols, you would need to satisfy that.
TDE and Index Range Scan
A reader, July 20, 2011 - 5:59 pm UTC
Is it true that in 11g, TDE allows index range scan if the encryption is done at the tablespace level as opposed to the column level encryption?
Also does TDE work with BLOBs and CLOBs?
AUTOLOGIN versus System Trigger wallet open
Paul, July 31, 2012 - 12:51 pm UTC
In a comment way back, you mention that you would not use the autologin wallet feature for databases using encryption, but only use a DBA issued command to open the wallet, at least that's the way I interpreted it.
We are having a discussion of the best methodology to use in our prod systems. For various security reasons, the databases have maintennace done to them on a monthly to quarterly basis. They are restarted as part of this process. DBAs are not involved in this security related efforts (unless something goes wrong, of course :-) )
So the decision was made to use a system trigger at startup time to execute the open. The code is wrapped.
I'm having a hard time trying to understand how someone might think this was more secure than an autologin wallet.
I'd appreciate it if you could take a few moments to consider this and offer your opinion.
Thanks in advance!!!
July 31, 2012 - 1:06 pm UTC
no, that wasn't what I meant - using the autologin wallet is valid, especially since you can now make it so that the wallet only works on the machine it was created on (complicates distributed stuff though).
So the decision was made to use a system trigger at startup time to execute the
open. The code is wrapped.
oh, that is bad. all i have to do is steal your database and wallet and I'm in!!!! who cares if the code is wrapped, the code will still execute when I restore your stolen database backup. And further, I'll unwrap your code in about 5 seconds or less.
I'm having a hard time trying to understand how someone might think this was
more secure than an autologin wallet.
ummm, me too - since it utterly defeats all of the work put into this. You might as well not encrypt!!!!!
Thanks for those points
Paul, August 01, 2012 - 3:52 pm UTC
Thats what I was thinking, but I tend to be in the minority, theres a lot of "thats the way we always done it" going on around me.
I'd like to investigate the wallet and host item you mention, I'm thinking that might be a new 11g feature. I did a quick check but dont see it, or at least it didnt seem to show up in the master index. Is there a name for that feature or functionality?
Thanks again very much!!!
You can unwrap code?
Matthew McPeak, August 01, 2012 - 6:00 pm UTC
>> And further, I'll unwrap your code in about 5 seconds or less.
I thought this was supposed to be impossible?
Interesting!
Matthew McPeak, August 02, 2012 - 11:49 am UTC
Very interesting, thank you! It makes one wonder what the point of wrapping is.