CRS-4228: Value of attribute ')' is missing

After changing disk group of database which is configured on cluster ready service , start and stop dependency attributes configured for old disk group must be changed in order to provide proper configuration.

$ crsctl status res ora.test.db -p

NAME=ora.test.db

TYPE=ora.database.type

ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--

...

START_DEPENDENCIES=hard(ora.TESTRECO.dg,ora.TESTDATADG.dg) weak(type:ora.listener.type,global:type:ora.scan_listener.type,uniform:ora.ons,global:ora.gns) pullup(ora.TESTRECO.dg,ora.TESTDATADG.dg)

STOP_DEPENDENCIES=hard(intermediate:ora.asm,shutdown:ora.TESTRECO.dg,shutdown:ora.TESTDATADG.dg)

...

Disk group TESTDATADG is not using anymore and all data files have been moved another disk group named TESTDATADG2. 

Change the attributes related with old disk to indicate new disk group name.

$ crsctl modify res ora.test.db -attr START_DEPENDENCIES="hard(ora.TESTRECO.dg.ora, ora.TESTDATADG2.dg).........."

gives CRS-4228 error.

The comma is used as a delimiter between attributes , so string after comma "ora.TESTDATADG2.dg)"  is viewed as resource which does not exist actually. In order to prevent this error , single quotes is used between atrribute value like this,

$ crsctl modify res ora.test.db -attr START_DEPENDENCIES="'hard(ora.TESTRECO.dg.ora, ora.TESTDATADG2.dg)..........'"

SOA Gateway soagenerate.sh gives error server.java.io.FileNotFoundException

After clone of Live E-Business System existing web services on SOA Gateway should be regenerated and redeployed to use them for test purposes. In order to do this , a file is prepared which contains all web service interface names line by line.

vi /tmp/services.lst
XX_TEST_PKG
XX_TEST2_PKG

soagenerate.sh under $FND_TOP/bin  is executed to generate WSDLs

$FND_TOP/bin/soagenerate.sh

Enter the single or multiple generate as S/M :
M
Enter the filename of the interface to be generated :
/tmp/services.lst

If you encounter following error ,

 Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.DBConnection.getJDBCConnection : Obtained new Jdbc Connection via WebAppsContext
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.DBConnection.getJDBCConnection : Obtained new Jdbc Connection via WebAppsContext
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.IRepAccess.getClassId : Class Id for irep_name XX_TEST_PKG = 4283
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.ui.ProviderAccess.generateWSDL : Generating Service : 4283
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : Sending Message : Generate
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : For Class Id : 4283
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : Encoded ClassId : 4283
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : SOA_SERVER_URL : http://ebstest.localdomain.com.tr:8010
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : Host : http://ebstest.localdomain.com.tr:8010
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : Port : 8010
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : ticket is null : false
  Wed Feb 21 07:54:56 EET 2018 : oracle.apps.fnd.soa.provider.util.ServerAccess.sendMessageToServer : Error while sending message to server.java.io.FileNotFoundException: http://ebstest.localdomain.com.tr:8010/webservice
  s/SOAProvider/EbizAuth?Generate=4283&soa_ticket=EgsTINbmlZ7ofmxqvGD-gw..


you must reset ASADMIN password which explained before in this blog post
http://erdemcer.blogspot.com.tr/2014/04/oracle-ebs-soa-gateway-testquery-does.html

After resetting ASADMIN password , reexecute soagenerate.sh .

As last step execute re-deploy interfaces

$FND_TOP/bin/soadeploy.sh

Enter the single or multiple deploy as S/M :
M
Enter the filename of the interface to be deployed :
/tmp/services.lst

ORA-27154: post/wait create failed / ORA-27301: OS failure message: No space left on device

After SAN migration of database  everything gone fine and system worked for 1 days successfully . But in an instant because of multipath configuration db has crashed due to dbwr failure.After fixing storage unit failure whenever would like to open database below error occured. Actually  there was no configuration change on database but immediately following error occured.

ORA-27154: post/wait create failed

ORA-27300: OS system dependent operation:semget failed with status: 28

ORA-27301: OS failure message: No space left on device

ORA-27302: failure occurred at: sskgpcreates

Mon Mar 05 12:32:54 2018

According to notes inadequate setting of semaphore can cause not to accommodate the request. Setting semaphore  to higher value resolved issue

In /etc/sysctl.conf

Before change 

    kernel.sem = 250 32000 100 128

After change

    kernel.sem = 250 32000 100 256

After applying settings as root execute "sysctl -p" command