Thanks for the question, Salman.
Asked: September 29, 2023 - 8:45 am UTC
Last updated: October 16, 2023 - 1:46 am UTC
Version: Oracle RAC Database 19.3.0.0
Viewed 10K+ times! This question is
You Asked
1. We have 2 node Oracle RAC 19c (19.3.0.0) database installed over Oracle Linux 7.6 x86-64 (UEK). We gracefully shutdown cluster on each node and restart it turn by turn. The secondary node started normally after server restart and joined cluster services. However, primary node after restart is not joining cluster and is giving following errors when starting CRS:
PROCL-26: Error while accessing the physical storage Operating System error [No such file or directory] [2]
CRS-4000: Command Start failed or completed with errors.
2. Upon running ocrcheck on this node, following output is observed:
PROT-602: Failed to retrieve data from the cluster registery
PROC-26: Error while accessing the physical storage Storage layer error [Insufficient quorum to open OCR devices] [0]
3. When i checked secondary node (working node), it had all the requisite files in the location /u01/app/oracle/crsdata/mydb11/. However, when i checked the same location on primary node, there were only two directories available i.e., "crf" and "cvu".
4. Upon checking the path /etc/oracle/ on both nodes, ocr.loc and olr.loc files are present and pointing to their respective directions.
5. Cluster logs of the faulty node (primary node) are showing following errors:
OCROSD: utopen:6m': failed in stat OCR file/disk /u01/app/oracle/crsdata/mydb10/olr/mydb10.olr, errno=2, os err strink=No such file or directory
OCROSD: utopen:7: failed to open any OCR file/disk, errno=2, os err string=No such file or directory
OCRRAW: proprinit: Coult not open raw device
6. Since RAC is running perfectly on secondary node and all the required files are also present on that node, is it possible that if I copy "ocr" and "olr" directories from this node to the primary node and then start the cluster? Will this work?
and Connor said...
I spoke to RAC PM Anil Nair. A potential cause is a timing issue, namely that the required file systems are not available when the daemons start, but for anything like this, you really need to be logging an SR so that all of the required log files etc can be assessed.
Please keep us updated with any resolution that Support provides.
Rating
(1 rating)
Is this answer out of date? If it is, please let us know via a Comment