USCDB Overload

abrahim
abrahim  Diamond  (1)
7 years 10 months ago  View: 2268  Reply: 4
1F


Fault Identification

This issue occurs when ALM-12012 Flow Control Started is generated on the alarm console.

Troubleshooting Method

Stop some low-priority tasks or service provisioning to reduce the message load on the Unified Subscriber Center Database (USCDB).

Troubleshooting Procedure

http://localhost:7904/pages/CEC1230A/01/CEC1230A/01/resources/public_sys-resources/icon-note.gif NOTE:
  • The troubleshooting procedure takes about 10 to 30 minutes.
  • Each step in this procedure can be performed to reduce the system load. If ALM-12012 is cleared after a certain step is performed, ignore the subsequent steps.

Set the CPU usage thresholds and message delay thresholds to the default values.

  1. Open the MML Command - USCDB window.
  2. Run LST FLOWCTRL to check whether the CPU usage thresholds and DRU, DSU, DSG, and LDAP message delay thresholds are set to the default values. The following is an example command:

    LST FLOWCTRL:;

    %%LST FLOWCTRL:;%% RETCODE = 0  Operation succeeded  The result is as follows ------------------------            System configuration parameter  =  System parameter Flow control configuration parameter name  =  Flow control parameter                       Flow control switch  =  ON          CPU flow control start threshold  =  95            CPU flow control end threshold  =  85         DRU time span start threshold(ms)  =  500         DSU time span start threshold(ms)  =  500         DSG time span start threshold(ms)  =  500           DRU time span end threshold(ms)  =  100           DSU time span end threshold(ms)  =  100           DSG time span end threshold(ms)  =  100        LDAP time span start threshold(ms)  =  80          LDAP time span end threshold(ms)  =  50 (Number of results = 1)  ---    END

    The preceding command output lists the default CPU usage thresholds and DRU, DSG, and LDAP message delay thresholds.

    • Yes: Go to 4.
    • No: Go to 3.
  3. Run MOD FLOWCTRL to change the CPU usage thresholds and DRU, DSU, DSG, and LDAP message delay thresholds to the default values. The following is an example command:
    MOD FLOWCTRL: FLWCTRFLG=ON, CPUFLWSTR=95, CPUFLWEND=85, DRUTSSTR=500, DSUTSSTR=500, DSGTSSTR=500, DRUTSEND=100, DSUTSEND=100, DSGTSEND=100, LDAPTSSTR=80, LDAPTSEND=50; 

Restart the abnormal nodes of the USCDB.

  1. Open the MML Command - USCDB window.
  2. Run LST NODE to query the ID of the cluster using the module number for which ALM-12012 is generated. In this step, Module number is set to the example value 103. The following is an example command:

    LST NODE: MID=103;

  3. Run DSP NODE to query the status of all nodes using the cluster ID obtained in 5. In this step, Node ID is set to the example value 11. The following is an example command:

    DSP NODE: CID=11;

  4. Check whether the status of a certain node is Abnormal.
    • Yes: Go to 8.
    • No: Go to 9.
  5. Run RST MODULE to restart the node whose status is Abnormal. In this step, Module number of an abnormal node is set to the example value 103. The following is an example command:

    RST MODULE: MID=103;


Disable the in-memory database consistency check and the data synchronization between the in-memory database and the physical database.

Alaul
Alaul  Diamond 
7 years 10 months ago
2F
Very useful documents for O&M.
abrahim
abrahim  Diamond 
7 years 10 months ago
3F
Thanks
Armetta
Armetta  Diamond 
7 years 10 months ago
4F
Excellent resolution of the problem and Useful commands

user_2837311
user_2837311  Diamond 
3 years 11 months ago
5F
useful document, thanks