Support Center > Search Results > SecureKnowledge Details
'WA' (disk I/O) consumes 99% CPU on Standby cluster member after RIP was configured on ClusterXL running Gaia OS Technical Level
Symptoms
  • 'WA' (disk I/O) consumes 99% CPU on Standby cluster member after RIP was configured on ClusterXL running Gaia OS.
  • Output of 'top' command and 'vmstat' command on Standby cluster member show 99% 'WA'.

  • /var/log/messages file on Standby cluster member repeatedly shows:
    routed[PID]: RipClusterProcessMsg: cxl ifname [lo]
    routed[PID]: RipClusterProcessMsg: iflp ifname [eth0]
    
  • Issue started after RIP was configured on cluster members.

  • Disk IOwait ('WA') on Standby cluster member is significantly decreased when changing logging facility in the /etc/syslog.conf file from /var/log/messages to /dev/null.
    Note: This change should not be used permanently, because the system messages will not be printed anymore to /var/log/messages file.

  • If Primary member is Active and Secondary member in Standby:

    1. On Primary member (Active):
      • Everything works correctly
      • Output of 'top' command on Standby cluster member shows 0%-1% 'WA'
      • /var/log/messages file on Active cluster member repeatedly shows:
        routed[PID]: RipClusterSendRoute: recv interface [eth0]


    2. On Secondary member (Standby):
      • Output of 'top' command on Standby cluster member shows 99%-100% 'WA'
      • Clish 'show routes rip' command does not display any routes
      • Expert 'netstat -rn' command on Standby cluster member shows that some routes are missing
      • /var/log/messages file on Standby cluster member repeatedly shows:
        routed[PID]: RipClusterProcessMsg: cxl ifname [lo]
        routed[PID]: RipClusterProcessMsg: iflp ifname [eth0]


  • If Primary member is Active and Secondary member in Down ('cpstop'):

    1. On Primary member (Active):
      • Output of 'top' command on Standby cluster member shows 99%-100% 'WA'
      • Clish 'show routes rip' command does not display any routes
      • Expert 'netstat -rn' command on Standby cluster member shows that some routes are missing
      • /var/log/messages file on Active cluster member repeatedly shows:
        routed[PID]: RipClusterSendRoute: recv interface [eth0]
        routed[PID]: cpcl_should_send() returns -3


    2. On Secondary member (Standby):
      • Output of 'top' command on Standby cluster member shows 0%-1% 'WA'
      • Clish 'show routes rip' command does not display any routes
      • Expert 'netstat -rn' command on Standby cluster member shows that some routes are missing
      • /var/log/messages file on Standby cluster member repeatedly shows:
        routed[PID]: cpcl_cxl_runtime_status: HA mode not started
Cause

RIP messages are printed with syslog priority 'LOG_NOTICE'. Due to large number of RIP messages, the I/O activity (writing these messages to /var/log/messages) increases significantly.


Solution
Note: To view this solution you need to Sign In .