Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'...

Page 1 of 10

Oracle

Blogs Home Products & Services Downloads Support Partners Communities About Login

Oracle Blog
Giri Mandalika's Repository Something to Share

Yet Another Siebel... | Main | PeopleSoft on Solari...

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error

By Giri Mandalika on Nov 21, 2008 (Crossposting the 2+ year old blog entry from my other blog hosted on blogger. Source URL: http://technopark02.blogspot.com/2006/09/solaris-10oracle-fixing-ora-27102-out.html) Symptom: As part of a database tuning effort you increase the SGA/PGA sizes; and Oracle greets with an ORA-27102: out of memory error message. The system had enough free memory to serve the needs of Oracle.
SQL> startup ORA-27102: out of memory SVR4 Error: 22: Invalid argument

Diagnosis
$ oerr ORA 27102 27102, 00000, "out of memory" // \*Cause: Out of memory // \*Action: Consult the trace file for details

Not so helpful. Let's look the alert log for some clues.
% tail -2 alert.log WARNING: EINVAL creating segment of size 0x000000028a006000 fix shm parameters in /etc/system or equivalent

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 2 of 10

Oracle is trying to create a 10G shared memory segment (depends on SGA/PGA sizes), but operating system (Solaris in this example) responded with an invalid argument (EINVAL) error message. There is a little hint about setting shm parameters in /etc/system. Prior to Solaris 10, shmsys:shminfo_shmmax parameter has to be set in /etc/system with maximum memory segment value that can be created. 8M is the default value on Solaris 9 and prior versions; where as 1/4th of the physical memory is the default on Solaris 10 and later. On a Solaris 10 (or later) system, it can be verified as shown below:
% prtconf | grep Mem Memory size: 32760 Megabytes % id -p uid=59008(oracle) gid=10001(dba) projid=3(default) % prctl -n project.max-shm-memory -i project 3 project: 3: default NAME PRIVILEGE VALUE FLAG ACTION project.max-shm-memory privileged 7.84GB deny system 16.0EB max deny

RECIPIENT -

Now it is clear that the system is using the default value of 8G in this scenario, where as the application (Oracle) is trying to create a memory segment (10G) larger than 8G. Hence the failure. So, the solution is to configure the system with a value large enough for the shared segment being created, so Oracle succeeds in starting up the database instance. On Solaris 9 and prior releases, it can be done by adding the following line to /etc/system, followed by a reboot for the system to pick up the new value.
set shminfo_shmmax = 0x000000028a006000

However shminfo_shmmax parameter was obsoleted with the release of Solaris 10; and Sun doesn't recommend setting this parameter in /etc/system even though it works as expected. On Solaris 10 and later, this value can be changed dynamically on a per project basis with the help of resource control facilities . This is how we do it on Solaris 10 and later:
% prctl -n project.max-shm-memory -r -v 10G -i project 3 % prctl -n project.max-shm-memory -i project 3 project: 3: default NAME PRIVILEGE VALUE FLAG ACTION project.max-shm-memory privileged 10.0GB deny system 16.0EB max deny

RECIPIENT -

Note that changes made with the prctl command on a running system are temporary, and will be lost when the system is rebooted. To make the changes permanent, create a project with projadd command and associate it with the user account as shown below:
% projadd -p 3 -c 'eBS benchmark' -U oracle -G dba % usermod -K project=OASB oracle -K 'project.max-shm-memory=

Finally make sure the project is created with projects -l or cat /etc/project commands.

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 3 of 10

% projects -l ... ... OASB projid : comment: users : groups : attribs:

3 "eBS benchmark" oracle dba project.max-shm-memory=(privileged,10737418240,deny)

% cat /etc/project ... ... OASB:3:eBS benchmark:oracle:dba:project.max-shm-memory=(privileged,10737418240,deny)

With these changes, Oracle would start the database up normally.

SQL> startup ORACLE instance started. Total System Global Area 1.0905E+10 bytes Fixed Size 1316080 bytes Variable Size 4429966096 bytes Database Buffers 6442450944 bytes Redo Buffers 31457280 bytes Database mounted. Database opened.

Related information: 1. 2. 3. 4. 5. What's New in Solaris System Tuning in the Solaris 10 Release? Resource Controls (overview) System Setup Recommendations for Solaris 8 and Solaris 9 Man page of prctl(1) Man page of projadd

Addendum : Oracle RAC settings Anonymous Bob suggested the following settings for Oracle RAC in the form of a comment for the benefit of others who run into similar issue(s) when running Oracle RAC. I'm pasting the comment as is (Disclaimer: I have not verified these settings): Thanks for a great explanation, I would like to add one comment that will help those with an Oracle RAC installation. Modifying the default project covers oracle processes great and is all that is needed for a single instance DB. In RAC however, the CRS process starts the DB and it is a root owned process and root does not use the default project. To fix ORA-27102 issue for RAC I added the following lines to an init script that runs before the init.crs script fires.
# Recommended Oracle RAC system params ndd -set /dev/udp udp_xmit_hiwat 65536 ndd -set /dev/udp udp_recv_hiwat 65536

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 4 of 10

# For root processes like crsd prctl -n project.max-shm-memory -r -v 8G -i project system prctl -n project.max-shm-ids -r -v 512 -i project system # For oracle processes like sqlplus prctl -n project.max-shm-memory -r -v 8G -i project default prctl -n project.max-shm-ids -r -v 512 -i project default

So simple yet it took me a week working with Oracle and SUN to come up with that answer...Hope that helps someone out. Bob # posted by Blogger Bob : 6:48 AM, April 25, 2008 Category: Solaris Tags: opensolaris ora-27102 oracle solaris sun troubleshooting Permanent link to this entry Yet Another Siebel... | Main | PeopleSoft on Solari... Comments: Good.This doc helped me. I had one more question. I have a 3 node cluster , which is already configured, When I do a ps -ef |grep oracle withy Project details , I see it running with user.oracle instead of the default system project. How can I setup it up that way? Regards, vijaykumar Kammar Posted by vijay on December 22, 2008 at 11:00 PM PST # Hi, Thanks a lot for this blog...helped me solve a critical issue!! ~Kaushik Posted by Kaushik Sethna on March 06, 2009 at 02:49 AM PST # Thanks a lot for this blog...helped me solve a critical issue!! Posted by seo on March 09, 2009 at 04:35 AM PDT # tnx Posted by Evgeny on March 18, 2009 at 10:41 PM PDT #

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 5 of 10

We were struggling with the same problem for two days. This article is written in very precise manner and we could solve our problem. Thanks for such a nice article. Posted by Tanweer Khan on April 07, 2009 at 04:33 PM PDT # Thank you for this great blog. I was trying to install an SAP - Oracle central instance and it wasn't going well. Very informative and easy to follow. Thanks much! Posted by Rick on April 21, 2009 at 04:11 AM PDT # Thanks for such a good and concise article Regards Lanto Posted by Lanto RAZAKALALAO on April 27, 2009 at 09:49 PM PDT # good doc Posted by Babu on June 07, 2009 at 02:42 PM PDT # Thanks this blog. but I found there have some issue when set default project: bash-3.00# prctl -n project.max-shm-memory -r -v 8G -i project default prctl: default: No controllable process found in task, project, or zone. but I edited the file /etc/project as following, it worked. zone1 oracle $ cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3::::project.max-shm-memory=(privileged,2147483648,deny) Posted by Jason Wang on July 20, 2009 at 01:10 PM PDT # So Sun, in its infinite wisdom, took a process where all one had to do was edit one file and replaced it with this debacle? No wonder the company died. How does a change like this even get approved? Lets take a simple, one step process and replace it with projects, multiple settings and even more commands with untold flags that one has to know. Brilliant. Thanks. Posted by Ian on August 12, 2009 at 04:13 AM PDT # So Sun, in its infinite wisdom, took a process where all one had to do was edit one file and replaced it with this debacle? No wonder the company died.

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 6 of 10

http://www.gifico.com http://www.lv-cheap.com Posted by louis vuitton on November 18, 2009 at 03:11 PM PST # I'm running into trouble using the projmod commands to set max-shm-memory for a group above 16GB. My system memory is 64GB. Is there a hard-cap on project memory of which I should be aware? Posted by Aaron Long on May 20, 2010 at 08:14 AM PDT # I was very impressed with this article. First go off the bat and the DBA could start the DB`s Regards, Avin... Posted by Avin on November 17, 2010 at 11:42 AM PST # Hi, This is really a nice post and helped me "help" my sys-admins to increase the parameter value. But, after many days after that activity, today when I checked the output of --- > prctl -n project.max-shm-memory -i project 100 project: 100: user.oracle NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-shm-memory privileged 20.0GB - deny privileged 60.0GB - deny system 16.0EB max deny Now it's not allowing me to increase oracle sga size greater than 20G. Will using -x resolve this ?? Or any better idea how can I resolve this ?? Posted by Jitu Keshwani on January 23, 2011 at 07:03 AM PST # impresive!!!!! it helps for our problem, and went successfuly... Thank you! Posted by esmael on January 24, 2011 at 08:46 PM PST # One of the best article that ever been written on DBA stuff. Posted by raj on February 09, 2011 at 04:01 AM PST # This post helped to solve critical issue, great blog. Thanks Mohan Posted by guest on July 12, 2011 at 10:30 AM PDT # hi,i met the same problem as u. My OS is solaris 10, and my memory size is 16GB. While i try to install the Oracle 10g, i got a out of memory error. And i have set the max-shm-memory to 12GB. Still get the same problem

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 7 of 10

Posted by guest on August 29, 2011 at 02:03 PM PDT # Its directly related to the memory size allocated on the configuration panel. It defaults to a very very large size (27GB in my case). Rather than change my system config using this article, I lowered the size allocation on the dbca config panel to a more reasonable size. Either way works. Posted by guest on September 29, 2011 at 05:08 PM PDT # Post a Comment: Name: guest E-Mail: URL: Notify me by email of new comments Remember Information?

Your Comment: HTML Syntax: NOT allowed Please answer this simple math question 1 + 23 =
About Preview Post

Benchmark announcements, HOW-TOs, Tips, Troubleshooting.. The views expressed in this blog are my own and do not necessarily reflect the views of Oracle.
Search

Enter search term: Search only this blog

Recent Posts

Oracle RDBMS & Solaris : Few Random Tips (Feb 2012) Solaris Tip: How-To Identify Memory Mapped Files Unwanted Software Installers Oracle Application Testing Suite (OATS): Few Tips & Tricks Solaris Tip: Resolving "statd: cannot talk to statd at <target_host>, RPC: Timed out(5)" Siebel Troubleshooting : An ODBC error occurred; SBL-GEN-03006: Error calling function: DICFindTable m_pReqTbl

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 8 of 10

Oracle Database on NFS : Resolving "ORA-27086: unable to lock file - already in use" Error Siebel Connection Broker Load Balancing Algorithm Oracle RDBMS : Generic Large Object (LOB) Performance Guidelines Oracle 11g: Travel back in time with the Database Flashback
Top Tags

benchmark best bi business c c++ certification cmt compiler connector connector/c++ connector/j consolidation containers crm data database e-businesssuite enterprise erp f5100 fix flash hardware how-to index intelligence java m4000 mysql mysqlnd opensolaris oracle payroll peoplesoft performance php practices rdbms server siebel solaris sparc sun t5440 troubleshooting

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandalika'... Page 9 of 10

tuning ultrasparc ultrasparct2plus virtualization

Categories

Benchmarks Best Practices CMT Enterprise Generic How-To Java MySQL Oracle Performance Solaris Sun Tips Troubleshooting Workaround

Archives

March 2012 Sun Mon Tue Wed Thu Fri Sat 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Today
Bookmarks

Menu

ISV Engineering Java home My External Blog MySQL DevZone OpenSolaris Sun Developer Network

Blogs Home Weblog Login

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012

Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error (Giri Mandali...

Page 10 of 10

Feeds

RSS Atom All /Benchmarks /Best Practices /CMT /Enterprise /Generic /How-To /Java /MySQL /Oracle /Performance /Solaris /Sun /Tips /Troubleshooting /Workaround Comments All /Benchmarks /Best Practices /CMT /Enterprise /Generic /How-To /Java /MySQL /Oracle /Performance /Solaris /Sun /Tips /Troubleshooting /Workaround Comments

The views expressed on this blog are those of the author and do not necessarily reflect the views of Oracle. Terms of Use | Your Privacy Rights

https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing

03-03-2012