Support Center > Search Results > SecureKnowledge Details
Various traffic issues on cluster due to FWD daemon taking all slots on cluster subscriber list
Symptoms
  • Various issues with traffic through a cluster:

    • Random traffic outages
    • Dynamic / Static routing stops working
    • Users are not recognized by Identity Awareness
    • All Identity Awareness commands on cluster members show: "daemon did not respond or not running!"
  • /var/log/messages file on cluster members repeatedly shows:
    FW-1: cxl_status_update_register: registering Name_of_Process not successful because too many processes are registered.

    Examples:

    • FW-1: cxl_status_update_register: registering pdpd not successful because too many processes are registered.
    • FW-1: cxl_status_update_register: registering pepd not successful because too many processes are registered.
    • FW-1: cxl_status_update_register: registering routed not successful because too many processes are registered.
    • FW-1: cxl_status_update_register: registering cprd PID 22279 not successful because too many processes are
    • FW-1: cxl_status_update_register: registering cvpnd not successful because too many processes are registered.
    • FW-1: cxl_status_update_register: registering ted not successful because too many processes are registered.
    • FW-1: cxl_status_update_register: registering gated not successful because too many processes are registered.
    • registered.
  • After enabling generation of core dump files per sk92764 / sk53363, core dump files for various processes might be generated in the /var/log/dump/usermode/ directory:

    • fwd
    • routed
    • pdpd
    • pepd
    • cvpnd
    • gated
  • Output of "cphaconf debug_data" command (under cluster debug 'fw ctl debug -m cluster + conf') on cluster members shows that cluster subscriber list is filled only with instances of FWD daemon (refer to sk93306 - Chapter 'ClusterXL Monitoring and Troubleshooting' - section "'cphaconf' command" - "cphaconf debug_data").

    Example:

    ;---- Status subscribers ---- 
    ; 
    ;Subscriber: 0 pid 5823 sig 18 desc fwd PID 5823; 
    ;Subscriber: 1 pid 760 sig 1 desc fibmgrd; 
    ;Subscriber: 2 pid 7268 sig 18 desc fwd PID 7268; 
    ;Subscriber: 3 pid 5831 sig 18 desc fwd PID 5831; 
    ;Subscriber: 4 pid 15189 sig 18 desc fwd PID 15189; 
    ;Subscriber: 5 pid 10097 sig 18 desc fwd PID 10097; 
    ;Subscriber: 6 pid 15839 sig 18 desc fwd PID 15839; 
    ;Subscriber: 7 pid 23038 sig 18 desc fwd PID 23038; 
    ;Subscriber: 8 pid 4106 sig 18 desc fwd PID 4106; 
    ;Subscriber: 9 pid 29513 sig 18 desc fwd PID 29513; 
    ;Subscriber: 10 pid 17118 sig 18 desc fwd PID 17118; 
    ;Subscriber: 11 pid 27227 sig 18 desc fwd PID 27227; 
    ;Subscriber: 12 pid 3092 sig 18 desc fwd PID 3092; 
    ;Subscriber: 13 pid 23341 sig 18 desc fwd PID 23341; 
    ;Subscriber: 14 pid 15326 sig 18 desc fwd PID 15326; 
    ;Subscriber: 15 pid 13008 sig 18 desc fwd PID 13008; 
    
Cause

FWD daemon registers itself too many times to cluster status updates. As a result, it takes all the available 16 registration slots, preventing other daemon from registering.


Solution
Note: To view this solution you need to Sign In .