S1720&S2700&S3700&S5700&S6700 Series Switches Product Troubleshooting - Preliminary-7 High CPU Usage

Created: Sep 26, 2017 11:32:34Latest reply: Sep 26, 2017 11:32:47 1456 1 0 0

7  High CPU Usage

a462f39e22304633878a596768e1e6c2 NOTE:

Devices of different models may support different functions, features, and commands. For details about the commands used in this document, see the command reference.

7.1  CPU Tasks and CPU Usage

This topic describes CPU tasks, functions, and usage after the device starts.

After the device starts, the CPU keeps running the following tasks:
  • System tasks of maintaining and managing the device status
  • Protocol tasks required in current network environments
  • Tasks of processing the packets received and sent from the forwarding plane
When Huawei switches are operating, the following functions need to use CPU resources:
  • Device component management: This function manages components in the device and checks the running status of components, such as cards, power modules, and fan modules.
  • Stack management: This function manages and maintains the status of member switches in a stack.
  • External access management: This function processes the network management traffic sent to the CPU, such as Telnet, SSH, HTTP, and SNMP traffic.
  • Network control protocol management: This function sends and receives protocol packets, performs protocol computing, and updates forwarding tables (such as MSTP, MAC, and FIB tables). Network control protocols include STP, LLDP, LNP, LACP, VCMP, DLDP, EFM, GVRP, VRRP,and routing protocols.
  • MAC address learning: This function helps synchronize MAC addresses between stack member switches.
  • Packet software forwarding: For example, L2PT forwards Layer 2 protocol packets through software.
  • ARP entry learning and aging
  • Processing of other packets sent to the CPU

Many active tasks may run on the CPU anytime. For example, there are more than 200 tasks on the S5700LI. The number of tasks running in the system varies according to the device model. Generally, if the device supports a large number of features, more tasks run in the system.

Because the system is always operating, CPU usage cannot be 0% even though no service configuration and network traffic exists on the device. In a stack, the stack member status needs to be periodically maintained, and most services are running on the master switch. A switch has a higher CPU usage when it functions as the master switch in a stack. When the number of stack member switches increases, CPU usage of the master switch increases accordingly.

In the following scenarios, the CPU runs with a heavy load and cannot schedule other tasks in a timely manner. As a result, services may become abnormal.
  • Packets are sent from the forwarding plane to the CPU at a high rate. For example, owning to a network loop, the CPU receives a large number of packets within a short period.
  • A task occupies the CPU for a long period.

You can run the display cpu-usage command on the device to check the current CPU usage, including the average CPU usages within the last 5 seconds, last 1 minute, and last 5 minutes, historical highest CPU usage and time when the highest CPU usage occurs, and CPU usages of current tasks within the last 5 seconds in descending order.

a462f39e22304633878a596768e1e6c2 NOTE:

In most cases, common data packets are forwarded by switch hardware without involving the CPU. Therefore, a high CPU usage does not affect data forwarding.

7.2  CPU Packet Processing Principles

This topic describes the packets that are processed by the CPU and packet processing principles.

Huawei switches forward common data packets through hardware without involving the CPU. The following types of packets will be sent to the CPU for further processing:
  • Protocol packets to be terminated by switches

    All the packets with a local destination address need to be sent to the CPU.

    • Protocol control packets, such as STP, LLDP, LNP, LACP, VCMP, DLDP, EFM, GVRP, and VRRP
    • Route update packets, such as RIP, OSPF, BGP, and IS-IS packets
    • SNMP, Telnet, and SSH packets
    • ARP and ND Response packets
  • Packets requiring special processing

    • ICMP packets with the Option field
    • IPv6 packets with the hop-by-hop option
    • IPv4/IPv6 packets with a TTL value smaller than or equal to 1
    • Packets with a local destination address
    • ARP, ND, and FIB Miss packets
  • Packets on which ACL-based packet classification is performed

    • Packets discarded by the ACL deny action after the logging function is enabled
    • Packets redirected to the CPU through traffic policies
  • Multicast packets

    • PIM, IGMP, MLD, and MSDP packets
    • Unknown IP multicast packets
  • Other packets

    • DHCP packets
    • ARP and ND broadcast request packets as well as ARP packets sent when dynamic ARP inspection is configured on a Layer 2 switch
    • L2PT software forwarded Layer 2 protocol packets (Devices on two ends of a tunnel forward Layer 2 protocol packets through software, and intermediate devices forward these packets through hardware.)
    • First packet in N:1 VLAN mapping (Subsequent packets are forwarded through hardware.)
Switches use the QoS mechanism to process the packets sent to the CPU and ensure that important packets are processed first. Switches classify eight queues of different priorities according to different types of packets sent to the CPU. Different switch models may support different types of packets sent to the CPU. The following uses S5700LI as an example. Table 7-1 and Figure 7-1 describe queue classification on the packets sent to the CPU. A larger queue ID indicates a higher queue priority.
Table 7-1  CPU queues for packets

Queue ID

Packet Type

Description

7

IPC, RPC, LACP

Internal management packets

6

VP

Internal software forwarded protocol packets

5

Telnet, SSH, LNP, DHCP

Management plane protocol packets

4

ARP Request

Important control plane protocol packets

3

STP, SMLK, EOAM, VCMP

Important control plane protocol packets

2

LBDT, LLDP, DLDP, IGMP, ICMP, NTP, 802.1x, GVRP, L2PT, ARP Miss, FTP, SNMP

Control plane protocol packets

1

Other

-

0

Other

-

Figure 7-1  Placing different types of packets into CPU queues 
a4f11b3955a245e7ab2680e30fa63382

7.3  Determining Whether a High CPU Usage Is a Fault

In some situations, a high CPU usage does not result in network problems. For example, a high CPU usage caused by known network events or administrator operations is normal and should not be treated as a fault. It is a fault only when it makes a device unable to process services normally.

7.3.1  A High CPU Usage Is a Normal Situation

In some network applications, a high CPU usage is a normal situation. Generally, a large network requires more CPU resources to process network traffic. When more member switches need to be managed in a stack, more CPU resources are required to maintain and manage the stack.

The device status is considered as normal in the following situations:
  • CPU usage does not exceed 80% when a device runs for a long period.
  • CPU usage does not exceed 95% when the device runs for a short period.

The following is list of scenarios where CPU usage commonly becomes high, but should not be considered a fault.

  • Spanning tree

    In MSTP, CPU usage is directly proportional to the number of instances and active ports. In VBST, each VLAN runs an independent instance. Therefore, VBST uses more CPU resources than MSTP when VBST and MSTP have the same number of VLANs and ports.

  • Routing table update

    When a Layer 3 switch receives a route update message, the switch uses CPU resources to update routing information to the forwarding plane. In a stack, the switch also needs to synchronize routing information to other member switches. During a routing table update, the following factors affect CPU usage:
    • Number of routing entries
    • Update frequency
    • Number of routing protocol processes that receive the update message
    • Number of member switches in a stack
  • Command execution

    CPU usage temporarily becomes high when commands that have a long execution time are used, for example:
    • The copy flash:/ command is executed in the user view.
    • Some debugging commands have a large amount of display information, especially when debugging information is displayed through the serial port.
  • Other scenarios

    • A port fast learns MAC addresses after having the sticky MAC function enabled.
    • A large number of ports are added to a large number of VLANs. For example, port groups are used to add a large number of ports to a large number of VLANs and change the link type of these ports.
    • Frequent or a large number of IGMP requests
    • Frequent network management operations
    • A large number of concurrent DHCP requests (For example, when a switch functions as a DHCP server, it restores connections with a large number of users.)
    • ARP broadcast storm
    • Ethernet broadcast storm
    • A large number of concurrent protocol packets are forwarded through software. For example, L2PT transparently transmits a large number of BPDUs within a short time or DHCP relay/snooping-enabled switch forwards DHCP packets through software.
    • A large number of data packets that cannot be forwarded through hardware are sent to the CPU, such as ARP Miss packets.
    • A port frequently alternates between Up and Down states.

7.3.2  Impact of a High CPU Usage on the System

A high CPU usage adversely affects the system processing capability and may result in the following network faults:
  • The STP topology changes, potentially causing network loops.

    A switch periodically receives BPDUs through the CPU to maintain its root or alternate port role. If an upstream device cannot send BPDUs on time because its CPU is busy or the local CPU is too busy to process received BPDUs on time, the switch considers the original path to the root bridge faulty. It then selects a new root port, causing network reconvergence. If the switch also has an alternate port, the switch uses the alternate port as the new root port. In this situation, a loop may occur on the network.

  • The routing topology changes.

    Keepalive packets of dynamic routing protocols are processed by the CPU. If the CPU is too busy to receive and send Hello packets on time, route flapping occurs. Route flapping includes OSPF flapping, BGP flapping, and VRRP flapping.

  • Reliability detection protocols flap.

    Keepalive packets of detection protocols such as 802.3ah, 802.1ag, DLDP, BFD, and MPLS OAMare periodically processed by the CPU. S5700HI detects the timeout of 802.1ag, BFD, and MPLS OAM and processes the Keepalive packets of these protocols through the hardware OAM engine but not the CPU. Therefore, on the S5700HI, Keepalive packet processing of 802.1ag, BFD, and MPLS OAM is not affected by the CPU load. If the CPU is too busy to receive and send protocol packets on time, protocols flap and service traffic forwarding is adversely affected.

  • Eth-Trunk of LACP type flaps.

    LACP is maintained by the CPU. If the CPU is too busy to receive and send LACP packets on time, Eth-Trunk shuts down the link, leading to link flapping.

  • A switch cannot respond to normal management requests.

    • Telnet or SSH sessions cannot be set up, causing a failure to manage the device, slow device response, or delay in executing commands.
    • SNMP times out.
    • MAC/IP ping lasts a long time or even times out.
  • A switch cannot forward or respond to client requests on time, causing DHCP or IEEE 802.1x failures.
  • Packets software-forwarded through the CPU are discarded or the delay in forwarding packets is increased.
  • More memory resources are used.

7.4  Procedure for Handling a High CPU Usage Situation

This topic describes the procedure for handling a high CPU usage situation, including possible causes and methods to solve the problem.

When CPU usage becomes high, determine the symptom, clarify the problem, confirm the root cause, and rectify the problem. For example, consider the following points:
  • When does CPU usage become high?
  • What is the system doing when CPU usage becomes high?
  • What factors could cause a high CPU usage?
  • Is the high CPU usage normal and does it need to be rectified? How can it be rectified?

7.4.1  Obtaining CPU Usage Information

CPU usage is the amount of time the CPU spends processing non-idle tasks expressed as a percentage. It has the following characteristics:
  • Constantly changing: System CPU usage varies according to the system operation and external environment changes.
  • Not in real time: System CPU usage reflects CPU usage within specific periods.
  • Entity-related: CPU usage is calculated based on the physical CPU. Generally, each physical entity has an independent physical CPU. Therefore, each member switch in a stack has its own CPU usage.

Obtaining Device Information

Run the display device command to obtain device information, including the device model, whether the device is in a stack, and what member switches the stack has.

<HUAWEI> display device
S5701-28TP-PWR-LI-AC's Device status:
Slot Sub  Type                Online    Power      Register     Status   Role
-------------------------------------------------------------------------------
0    -    S5701-28TP-PWR-LI   Present   PowerOn    Registered   Normal   Master
1    -    S5701-28TP-PWR-LI   Present   PowerOn    Registered   Normal   Standby
2    -    S5701-28TP-PWR-LI   Present   PowerOn    Registered   Normal   Slave
3    -    S5701-28TP-PWR-LI   Present   PowerOn    Registered   Normal   Slave

Obtaining CPU Usage Statistics

Run the display cpu-usage [ slave | slot slot-id ] command to view CPU usage statistics on a device with high CPU usage. slot-id indicates the stack ID of a member switch in a stack.

<HUAWEI> display cpu-usage slot 0
CPU Usage Stat. Cycle: 60 (Second)
CPU Usage            : 99% Max: 100%
CPU Usage Stat. Time : 2014-06-05 15:19 
CPU utilization for five seconds: 99%: one minute: 75%: five minutes: 42%
Max CPU Usage Stat. Time : 2014-06-05 15:19.

TaskName             CPU  Runtime(CPU Tick High/Tick Low)  Task Explanation
ARP                  30%         0/bda2b23b       ARP                          
OS                   30%         0/b2d02f1f       Operation System             
L2IF                 21%         0/8448bf54       L2IF                         
IFPD                  4%         0/1e575090       IFPD Ifnet Product Adapter   
L2_P                  3%         0/1a777526       L2_PR                        
FTS                   2%         0/13ed6c3e       FTS                          
IPCQ                  2%         0/1256ab6f       IPCQIPC task for single queue
STP                   2%         0/175350b9       STP                          
VPR                   2%         0/16254e6f       VPR VP Receive               
mv_rx7                2%         0/123d908c       mv_rx7                       
VIDL                  1%         0/ 5f5df6f       DOPRA IDLE                   
mv_rx6                1%         0/ db73d34       mv_rx6                       
AAA                   0%         0/   1d5c6       AAA  Authen Account Authorize
ACL                   0%         0/  5fa8c7       ACL Access Control List      
ADPT                  0%         0/       0       ADPT Adapter                 
AGNT                  0%         0/       0       AGNTSNMP agent task          
AGT6                  0%         0/       0       AGT6SNMP AGT6 task           
ALM                   0%         0/       0       ALM  Alarm Management        
ALS                   0%         0/ 3c2c178       ALS  Loss of Signal          
AM                    0%         0/  155db9       AM   Address Management      
APP                   0%         0/       0       APP                          
ASFI                  0%         0/       0       ASFI                         
ASFM                  0%         0/       0       ASFM                         
BATT                  0%         0/       0       BATT Main Task               
BFD                   0%         0/  3d8a91       BFD Bidirection Forwarding
                                                  Detect                       
BOX                   0%         0/       0       BOX Output                   
BPDU                  0%         0/   1f13d       BPDU Adapter                 
BTRC                  0%         0/    6295       BTRC                         
CAPM                  0%         0/       0       CAPM Capture Packet          
......

Obtaining Alarm Information About a High CPU Usage

If CPU usage exceeds the alarm threshold, the system sends an alarm to the NMS. You can obtain high CPU usage records through alarm information and log information.
  • View alarm information about high CPU usage.

    You can check whether a high CPU usage alarm is generated on the device through the NMS or using the display trapbuffer command. The related alarm information is as follows:

    ENTITYTRAP_1.3.6.1.4.1.2011.5.25.219.2.14.1 hwCPUUtilizationRising
    ENTITYTRAP/4/ENTITYCPUALARM:OID [oid] CPU utilization exceeded the pre-alarm threshold.(Index=[INTEGER], 
    EntityPhysicalIndex=[INTEGER], PhysicalName=[OCTET], EntityThresholdType=[INTEGER], EntityThresholdValue=[INTEGER], 
    EntityThresholdCurrent=[INTEGER], EntityTrapFaultID=[INTEGER].)

7.4.2  Identifying Device Behaviors

After collecting CPU usage of a device, analyze device behaviors when CPU usage becomes high. Generally, a high CPU usage is related to service processing or abnormal network environments. When system CPU usage becomes high, you can collect tasks with a high CPU usage to analyze device behaviors.

Collecting the Tasks with a High CPU Usage

Obtain the tasks with a high CPU usage based on the command output in 7.4.1 Obtaining CPU Usage Information or collected alarm and log information. You are advised to focus on the first three tasks with the highest CPU usage.

Analyzing Device Behaviors Based on the Task Type

The system provides service functions through tasks. CPU usage of tasks is an indicator of service functions and an important metric for analyzing device behaviors. In most cases, you can focus on the following types of tasks based on service deployment:

  • System idle task

    This type of task is a special task in the system. It is called VIDL, has the lowest priority, and occupies the CPU only when all the other tasks are idle. When a non-idle task needs to occupy the CPU, the VIDL task cannot occupy the CPU.

    CPU usage is the amount of time the CPU spends processing non-idle tasks expressed as a percentage. The system calculates the device CPU usage based on the time during which the VIDL task occupies the CPU. A higher CPU usage of the VIDL task indicates a lower system CPU usage and idler system.

  • System management task

    This type of task manages system resources and provides the basic mechanism for operating systems, such as timers and information centers. The following describes common system management tasks that may cause a high CPU usage:
    • Information center task

      Includes the BOX task and INFO task. The BOX task outputs the information stored in black boxes, and the INFO task receives and outputs the logs and alarms generated by service modules.

      These tasks provide operating systems with basic information center functions such as recording logs, alarms, exceptions, and infinite loops and outputting debugging information.

      When the device displays a large amount of debugging information or log information, CPU usage of this type of task becomes high.

    • Device management task

      Includes the DEV task, HOTT task, and SRMI task. The DEV task manages hardware modules on the device, the HOTT task manages hot swap of cards, and the SRMI task processes external interruptions related to device components.

      These tasks process a variety of device change events and may result in short-term high CPU usage during the configuration recovery, active/standby switchover, addition of new stack members, and installation of subcards. These situations will not affect services.

      When device components become faulty, a large number of interruptions are reported, which may cause CPU usage of the SRMI task to become high.

    • Inter-device communication task

      Includes the IPCR task, IPCQ task, and RPCQ task. The IPCR task sends, receives, and distributes Inter-device communication messages, the IPCQ task retransmits Inter-device communication messages that fail to be transmitted, and the RPCQ task provides the remote procedure calling function.

      These tasks implement Inter-device management message communication.

      CPU usage of this type of task becomes high when a large number of Inter-device management messages are generated. For example, when a large number of routes flap, a large number of users get online concurrently, or a ring network flaps.

    • Interface management task

      Includes the IFNT task, IFPD task, and linkscan task. The IFNT task processes interface status change events, the IFPD task maintains the interface database and processes interface status change events, and the linkscan task detects the interface link status.

      These tasks maintain information about current interfaces and peripheral components (such as optical modules) and interface statuses, and report interface events to service modules for processing.

      CPU usage of this type of task may become high when a large number of interfaces exist, the interface link status flaps, or optical modules become faulty.

  • Network management task

    This type of task provides the network management GUI and also monitors and manages the network status. Common tasks that may result in a high CPU usage include:
    • Network management task

      Includes the AGNT task, AGT6 task, VTx task, and FTPS task. The AGNT task provides the IPv4 SNMP function, and the AGT6 task provides the IPv6 SNMP function. The VTx task is also called VTY user task, which provides VTY users with the login, authentication, and man-machine interaction functions. x indicates the login sequence of a user. For example, the task name of the first user is VT0. The FTPS task provides the FTP service function.

      These tasks provide the capability to manage the device through the network.

      CPU usage of this type of task may become high for a short time when a large amount of data is displayed on a user terminal, multiple FTP processes download files simultaneously, or the network management software frequently accesses the device to traverse MIB object information.

    • Network monitoring task

      Includes the NSA task, NQAS task, and NQAC task. The NSA task provides the NetStream function to monitor service traffic on a network. Both the NQAS and NQAC tasks provide the NQA function to perform simulation tests on service packets on the live network.

      These tasks provide network monitoring capabilities and will not result in a high CPU usage in most situations.

  • Packet receiving/sending task

    Packets on a network can be classified into control packets and data packets based on the function. Because the control plane and forwarding plane are separated on Huawei switches, control packets and some data packets (such as ARP Miss and multicast RPF-Fail packets) need to be processed on the control plane where the processing core is the CPU.

    Packets sent from the forwarding plane to the CPU are resolved and distributed by a series of packet receiving/sending tasks and then processed and forwarded by the device. In this process, tasks such as BCMR, BCMT, MV0 to MV7, FTS, VP, VPR, VPS, and SOCK need to participate. When a large number of packets are sent to the control plane, CPU usages of these tasks increase significantly. This is a major cause for high system CPU usage.

  • Service protocol task

    Service protocol tasks provide most protocol functions on switches. When a network is stable, service protocol interaction and processing do not cause great fluctuations in CPU usage. When the network frequently changes or flaps, service protocols need to perform frequent interactions and calculations to adapt to network environment changes. In this situation, CPU usage may become high.

    Common tasks that may result in a high CPU usage include routing management tasks (such as ROUT and FIB), MAC management tasks (such as frag_add, frag_del, and MSYN), user management tasks (include DHCP, EAP, and SAM), and protocol tasks with frequent interaction (such as ARP). The ROUT task provides routing protocol functions such as BGP, IS-IS, OSPF, and RIP.

7.4.3  Analyzing the Root Cause

Understanding Major Network Events

A high CPU usage is often caused by internal or external events such as the service configuration, NMS synchronization, network environment, and component fault. Before determining the root cause of the high CPU usage, check the network O&M information to see if any major network events have occurred. Major network events include service migration, link status change, service adjustment, spare part replacement, NMS synchronization, login of multiple users, device alarms, and network flapping.

Analyzing the Cause Based on Device Behaviors

Analyze device behaviors when the CPU usage becomes high to determine the immediate cause. Locate the root cause of the high CPU usage through analysis of the network deployment and network environment. Different tasks have different processes, so the root causes of the high CPU usage are also different.

  • System management task

    A system management task manages system components and provides basic functions for other service modules. A high CPU usage of a system management task is often caused by internal events (such as hardware faults) or triggered by other service modules.

    When high CPU usage is triggered by a service module, analyze the fault based on information about the service module.

  • Network management task

    A high CPU usage of a network management task is caused by network management events such as NMS synchronization. The high CPU usage lasts only for a short time and services are not affected. Analyze the fault together with network management events.

  • Protocol receiving/sending task and service protocol task

    The two tasks that cause a high CPU usage often occur simultaneously. Oftentimes, the CPU usage becomes high because many protocol packets were sent to the CPU for processing.

    Identify the cause according to the following roadmap:
    1. Determine the packet type.

      Different switch models collect statistics on packets sent to the CPU in different modes. You can identify the type of packets using the following methods:
      • Analyze the type of sent packets according to statistics on packets sent to the CPU (supported by only the S5700EI, S5710EI, S5720EI, S5700HI, S5710HI, S5720HI, S6700EI, S6720EI, and S6720S-EI).

        You can run the display cpu-defend statistics all command multiple times to check statistics on all packets sent to the CPU. The statistical value is accumulated continuously. If the rate of a certain type of packets sent to the CPU increases or the rate exceeds the rate limit, these packets will cause a high CPU usage. (You can run the display cpu-defend rate all command to check the collection rate. Packets that exceed the rate limit are discarded.)

        You can run the reset cpu-defend statistics command to clear statistics. CPU attack defense technology monitors packets sent to the CPU at 10 minute intervals. If the number of packets sent to the CPU within this period exceeds the threshold, the system logs important information including the packet type, quantity of dropped packets, and occurrence time. The log format is as follows:

        DEFD/4/CPCAR_DROP_MPU:Rate of packets to cpu exceeded the CPCAR limit on the MPU. (Protocol=[STRING], CIR/CBS=[ULONG]/[ULONG], 
        ExceededPacketCount=[STRING])
      • Determine the type of packets sent to the CPU according to the service module usage.

        When many protocol packets are sent to the CPU, the CPU usage of some protocol tasks becomes high. Determine the packet type according to the CPU usage information of protocol tasks. The following describes common important protocol tasks.

        Task Name

        Description

        ARP

        Implements ARP protocol stack processing, manages the protocol state machine, and maintains the ARP-specific database.

        DHCP

        Implements DHCP protocol stack processing and provides the DHCP snooping and DHCP relay functions.

        SNPG

        Implements IGMP snooping/MLD snooping protocol stack processing, and listens to and processes IGMP and MLD messages.

        ROUT

        Is responsible for route selection and learning of a routing protocol, selects the optimal route, and delivers routes to the FIB.

        STP

        Implements the STP protocol stack, manages the protocol state machine, and maintains the STP-specific database.

    2. (Optional) Determine packet characteristics.

      If the cause cannot be determined based on the packet type and network management events, obtain packet information through port mirroring to analyze the characteristics of packets sent to the CPU.

      You can directly obtain packet information through port mirroring. This mode does not affect the CPU usage of the device. It is recommended that mirroring be configured on the inbound interface of packets sent to the CPU. For details on how to configure port mirroring on a switch, see "Mirroring Configuration" in the "Configuration Guide - Network Management and Monitoring".

    3. Analyze the root cause.

      You can obtain the immediate cause of the high CPU usage according to the packet type and characteristics. Further analyze the root cause and take troubleshooting measures based on the immediate cause. Common root causes include protocol flapping, network loops, network attacks, and concurrent services. For detailed troubleshooting measures, see 7.4.4 Common Causes of and Solutions to a High CPU Usage.

7.4.4  Common Causes of and Solutions to a High CPU Usage

Hardware Fault

When the switch hardware is faulty, components may report several interruption messages. As a result, the CPU usage becomes high.

Fault Location

If a hardware fault occurs, the tasks that process interruption messages (for example, SRMI, SRMR, and BCMDPC tasks) occupy high CPU usage. That is, if the CPU usage is high and the preceding tasks occupy the most CPU resources, a hardware fault may have occurred.

Suggestions

Manually reset the device that has high CPU usage. You are advised to turn the power off and then on. If the fault persists after reset, contact technical support personnel.

Network Environment

Network environment factors such as network flapping, loops, and attacks often cause a high CPU usage. Take different measures depending on the cause:

  • Network flapping

    When network flapping occurs, the network topology changes frequently. The device is busy processing network switching events, causing high CPU usage. Common network flapping includes STP flapping and routing protocol flapping:

    • STP flapping

      STP flapping occurs on Layer 2 networks. When STP flapping occurs frequently, the device continuously performs STP calculation. The forwarding tables such as MAC address tables and ARP tables are updated accordingly, causing high CPU usage.

      Fault Location

      1. When you doubt that frequent STP flapping occurs on a network, run the display stp topology-change command to check STP topology change information.
      2. When you determine that there are frequent network topology changes, run the display stp tc-bpdu statistics command to check the statistics on received TC BPDUs to determine the source of the TC BPDUs.
      3. Find the device that sends TC BPDUs according to the source of the TC BPDUs, and analyze the cause of the STP topology change using network management events and system logs on the device.

      Suggestions

      1. If a user-side interface Up/Down event causes the STP topology change, run the stp edged-port enable command in the interface view to configure the user-side interface as the edge port and run the stp bpdu-protection command to enable BPDU protection.
      2. If the root bridge is preempted, run the stp root-protection command on the expected root port to enable root protection and ensure that the STP topology is correct.
      3. If TC BPDUs are used to attack a network, run the stp tc-protection command on the attacked port to enable TC protection and reduce the impact of the attack on the device.
      4. If the topology change cause cannot be located or the fault persists after the preceding measures are taken, contact technical support personnel.
    • Routing protocol flapping

      Routing protocol flapping will cause routing information re-advertisement and routing table recalculation. This affects the CPU usage. In practice, OSPF is often used on the switch to manage dynamic routing information.

      Fault Location

      Check the cause for the OSPF neighbor Down event using the logs. Run the display logbuffer command to check the following log:

      OSPF/3/NBR_DOWN_REASON:Neighbor state leaves full or changed to Down. (ProcessId=[USHORT], NeighborRouterId=[IPADDR], 
      NeighborAreaId=[ULONG], NeighborInterface=[STRING],NeighborDownImmediate reason=[STRING], NeighborDownPrimeReason=[STRING], 
      NeighborChangeTime=[STRING])

      The NeighborDownImmediate reason parameter indicates the cause for the OSPF neighbor Down event. Possible causes are as follows:

      • Neighbor Down Due to Inactivity

        The device does not receive Hello packets during the deadtime from the neighbor.

      • Neighbor Down Due to Kill Neighbor

        The device interface used to establish the OSPF neighbor relationship is Down, the BFD session is Down, or the reset ospf process command is executed.

        You can view the NeighborDownPrimeReason parameter to determine the detailed cause.

      • Neighbor Down Due to 1-Wayhello Received or Neighbor Down Due to SequenceNum Mismatch

        The OSPF status of the remote device first goes Down and the remote device sends a 1-Wayhello packet to the local device. As a result, the OSPF status of the local device also becomes Down. In this situation, check whether the fault is caused by the remote device.

      Suggestions

      Interface link flapping and flooding of many LSAs are common causes for the OSPF neighbor Down event. Take different measures depending on the cause.

      • Interface link flapping

        The interface link flapping causes the OSPF neighbor relationship flapping. Check the interface Up/Down event in logs. If interface link flapping occurs, check the link of the interface.

      • Flooding of many LSAs

        When many LSAs are flooded, many LS UPDATE messages are generated on the network. The device is busy processing LS UPDATE messages and as a result, Hello packets cannot be processed in a timely manner, causing the OSPF status to becomes Down. You are advised to perform the following operations:
        • If the deadtime of the OSPF neighbor relationship is less than 20s, run the ospf timer dead interval command to change the dead time to a value greater than 20s.

        • Run the sham-hello enable command in the OSPF view to enable the Sham-Hello function. That is, the device is allowed to maintain the OSPF neighbor relationship by sending non-Hello packets such as LSUs.

      • If the fault persists after the preceding measures are taken, contact technical support personnel.

  • Network loops

    When network loops occur, MAC address flapping frequently occurs and many protocol packets are sent to the device for processing due to broadcast storms. As a result, the CPU usage becomes high.

    Fault Location

    Network loops cause broadcast storms and may also lead to the following problems:

    • Users cannot log in to the device remotely.
    • The display interface command output shows a large number of broadcast packets received on one or more interfaces.
    • It takes a long time to log in to the device from the serial port.
    • The device CPU usage exceeds 70%.
    • A large number of ICMP packets are lost in ping tests.
    • Indicators of interfaces in the VLAN where a loop has occurred blink at a higher frequency than usual.
    • PCs receive many broadcast packets.
    • MAC address flapping frequently occurs.
    • Loop alarms are generated when loop detection is enabled.

    Suggestions

    1. Determine the interface where broadcast storms occur according to the interface indicator status and traffic.
    2. Check devices where loops occur hop by hop according to the topology.
    3. Locate the interface where a loop occurs and remove the loop.
    4. If the fault persists after the preceding measures are taken, contact technical support personnel.
  • Network attacks

    Network hosts or devices send many abnormal exchange requests to attack other network devices, affecting security and services. When network attacks occur, the device is busy in processing abnormal exchange requests from the attack source. As a result, the CPU usage becomes high.

    Fault Location

    The network attacks that cause high CPU usage include ARP packet attacks, ARP Miss packet attacks, DHCP attacks, and TC BPDU attacks. In these attacks, many protocol packets are sent to the device, so you can see large statistics on such protocol packets on the device.

    • ARP packet attacks and ARP Miss packet attacks

      Run the display arp packet statistics command to check ARP packets statistics. Determine the network attack type by using these statistics, especially the values of ARP Pkt Received and ARP-Miss Msg Received.
      a462f39e22304633878a596768e1e6c2 NOTE:

      In a stack scenario, the display arp packet statistics command displays only the statistics on ARP packets on the master switch.

      Run the debugging arp packet command to enable ARP packet debugging. Check the source of a large number of sent ARP or ARP Miss packets.

    • DHCP attacks

      Run the display dhcp statistics command to check the statistics on DHCP packets. If DHCP packets are being sent at a higher speed, then a DHCP attack may be occurring.

    • TC BPDU attacks

      See "Fault Location" under "STP flapping."

    Suggestions

    • If ARP packet attacks, ARP Miss packet attacks, and DHCP attacks occur, enable automatic attack source tracing to promptly detect attacks on time.
    • If TC BPDU attacks occur, see "Suggestions" under "STP flapping."

Concurrent Services

The impact of many concurrent services on the CPU usage is similar to the impact of network attacks. The fault scenario is also similar (many users go online and many ARP and DHCP packets are exchanged). However, protocol packets for concurrent services are normal whereas protocol packets for network attacks are malicious ones. Therefore, fault location is similar, but the process is different.

Fault Location

See "Fault Location" in "Network attacks."

Suggestions

  • Adjust service deployment and migrate some hosts or services to other devices.
  • Reduce the CPCAR value of some protocol packets. This adjustment may reduce the user login rate. Exercise caution when you perform this operation.
  • If the fault persists after the preceding measures are taken, contact technical support personnel.

User Operations

When NMS synchronization operations are performed or many command outputs are delivered to terminals, the CPU usage often becomes high. These operations are typically performed for network management.

Fault Location

Collect the CPU usage of each task in the case of a high CPU usage. When AGNT or AGT6 tasks cause high CPU usage, NMS synchronization operations will result in the fault. When VT tasks cause high CPU usage, delivering many command outputs to terminals will result in this fault.

Suggestions

The high CPU usage caused by user operations is temporary and services are not affected. If user network management operations are appropriate and do not affect services, this situation can be ignored. If CPU usage remains high or services are affected, contact technical support personnel.

7.5  Recommended Configuration

This topic describes the recommended configuration to avoid high CPU usage for specific scenarios.

  • Port groups: When a port group has more than 40 member ports, adding these member ports to 4096 VLANs in batches may cause CPU usage to quickly exceed 80%. Therefore, you are advised to add the member ports to no more than 500 VLANs in batches.
  • LNP feature: When the type of more than 20 ports is changed in batches, CPU usage may exceed 80% in a short period. Therefore, you are advised to change the type of ports one by one.
  • MAC addresses: Frequent MAC address flapping may result in a high CPU usage. When MAC address flapping could potentially occur, you are advised to run the mac-address flapping action error-down command to set the action performed on the interface where MAC address flapping occurs to error-down.
  • Loopback detection: When the ports on which loopback detection is enabled are added to a total of more than 1024 VLANs, you are advised to run the loopback-detect action shutdowncommand to shut down the ports on which loops are detected. The VLAN counter increases by 1 every time a port is added to a VLAN, even when multiple ports are added to the same VLAN.

7.6  Common Information

This topic describes commands, CPU tasks, and network management OID related to CPU usage.

Related Information

S Switch High CPU Usage Troubleshooting ---- High CPU Usage Typical Cases

Command

Syntax

Function

display cpu-usage [ slave | slot slot-id ]

Display CPU usage.

display logbuffer [ size value | slot slot-id | modulemodule-name | security | level { severity | level } ] *

Display log information on the device.

display trapbuffer [ size value ]

Display alarm information on the device.

display cpu-defend rate [ packet-type packet-type ] [ slot slot-id | all ]

Display the rate at which protocol packets are sent to the CPU.

display cpu-defend statistics [ packet-type packet-type ] [ slot slot-id | all ]

Display statistics on the protocol packets sent to the CPU.

display stp process process-id ] [ instance instance-idtopology-change

Display STP topology changes.

display stp [ process process-id ] [ instance instance-id] [ interface interface-type interface-number | slot slot-id ] tc-bpdu statistics

Display STP TC BPDU statistics.

display arp packet statistics

Display ARP packet statistics.

display dhcp statistics

Display DHCP packet statistics.

CPU Task Names and Functions

  • BUFM: Output debugging information.
  • 1731: Implement the Y.1731 protocol stack, manage the protocol state machine, and maintain protocol databases.
  • _EXC: Process system exception events.
  • _TIL: Monitor and process deadloops caused by software errors.
  • AAA: Interact with modules such as the UCM and RADIUS modules, process user authentication messages, and maintain authentication and authorization entries.
  • ACL: Access control list.
  • ADPG: Maintain dynamic VLAN chip entries at the adaptation layer.
  • ADPT: Implement the EFM protocol, manage the protocol state machine, and maintain protocol databases.
  • age_task: Age MAC entries.
  • AGNT: Implement the IPv4 SNMP protocol.
  • AGT6: Implement the IPv6 SNMP protocol.
  • ALM: Add, clear, and manage alarm information.
  • ALS: Implement automatic laser shutdown.
  • AM: Manage IP address pools and addresses and manage IP addresses for the DHCP module.
  • APP: Schedule Layer 3 services in a unified manner.
  • ARP: Implement the ARP protocol, manage the protocol state machine, and maintain protocol databases.
  • au_msg_hnd: Process AU messages. MAC entry learning and issuing are implemented using AU messages.
  • bcmC: Collect packet statistics on chip interface.
  • bcmD: Process asynchronous message of chip's driver software.
  • bcmR: Receive packets from the chip.
  • bcmT: Send packets to the chip.
  • bcmX: Send packets asynchronously to the chip of a certain type.
  • bcmL2MOD.0: Learn MAC address entries.
  • BEAT: Send and receive heartbeat packets between boards to monitor inter-board communication.
  • BFD: Implement the BFD protocol, manage the protocol state machine, and maintain protocol databases.
  • bmLI: Scan port status and notify the application modules of status changes.
  • BOX: Output the data stored in a black box. A black box stores the error and exception information generated during device operation.
  • BULK_CLASS: Manage the USB flash drive (operating system task).
  • BULK_CLASS_IRP: Manage USB I/O request packets (operating system task).
  • BusM A: Manage USB bus (operating system task).
  • CCTL: Collect and schedule performance data in a batch.
  • CDM: Manage configuration data.
  • CFM: Restore configurations.
  • CHAL: Adapt to hardware.
  • CKDV: Control and manage clock card.
  • CMD_Switching: Listen to the socket.
  • CMDA: Execute commands in a batch.
  • cmdExec: Execute commands.
  • CSBR: Check the configuration consistency between master and slave boards.
  • CSPF: Implement the CSPF protocol and calculate paths.
  • CssC: Process stack events.
  • CSSM: Implement the stack protocol and manage the stack status.
  • DEFD: Monitor traffic sent to the CPU and maintain CPU protection data.
  • DELM: Enable STP to delete MAC entries.
  • DEV: Manage hardware modules on the device.
  • DEVA: Process hot swapping events of subcards.
  • DFSU: Load logic files.
  • DHCP: Process the DHCP protocol and implement DHCP snooping and DHCP relay.
  • DLDP: Implement the DLDP protocol, manage the protocol state machine, and maintain protocol databases.
  • DSMS: Process environment alarms generated by the environment monitoring system.
  • EAP: Implement 802.1x authentication, MAC address authentication, and MAC address bypass authentication, manage the protocol state machine, and maintain protocol databases.
  • Ecm: Manage communication between low-level boards.
  • EFMT: Send 802.3ah test packets.
  • EHCD_IH: USB host controller driver task (operating system task).
  • ELAB: Manage device electronic labels.
  • EOAM: Implement the EOAM 802.1ag protocol, manage the protocol state machine, and maintain protocol databases.
  • Eout: Output debugging information about the ECM task.
  • FBUF: Send packets.
  • FCAT: Capture the packets sent or received by the CPU for fault location.
  • FECD: Process MOD synchronization messages.
  • FIB: Generate IPv4 FIB entries, maintain software entries, and request the application layer to maintain chip entries.
  • FIB6: Manage IPv6 FIB entries, maintain software entries, and request the application layer to maintain chip entries.
  • FM93: Output fault information.
  • FMAT: Manage faults.
  • FMCK: Detect device faults.
  • FMON: Monitor logic card faults.
  • frag_add: Synchronize MAC entries from the hardware table to the software table, walk through the hardware table, and add the MAC entries that do not exist in the software table to the software table.
  • frag_del: Synchronize MAC entries from the hardware table to the software table, walk through the software table, and delete the MAC entries that do not exist in the hardware table from the software table.
  • FTPS: Provide FTP service.
  • FTS: Receive packets. This task is created by FECD. After the driver receives packets, it sends the packets to the FTS task for processing if these packets are not sent to the super task for processing.
  • GREP: Manage chip's GRE forwarding entries at the adaptation layer.
  • GTL: Manage common data such as memory and character strings.
  • GVRP: Implement the GVRP protocol, manage the protocol state machine, and maintain protocol databases.
  • HACK: Process HA response messages.
  • HOTT: Manage hot swapping events of LPUs.
  • HS2M: Synchronize data between the master and slave switches to ensure high reliability.
  • IFNT: Process interface status changes.
  • IFPD: Implement interface management, maintain interface data, and process interface status changes.
  • INFO: Receive and send logs, alarms, and debugging information generated by service modules.
  • IP: Schedule IP protocol tasks in a unified manner.
  • IPCQ: Retransmit IPC messages upon message transmission failures.
  • IPCR: Send, receive, and distribute IPC messages to related service modules.
  • IPMC: Adapt to Layer 3 multicast protocols, monitor the control plane changes, and issue forwarding entries.
  • ITSK: Send, receive, and distribute various protocol packets.
  • L2: Schedule Layer 2 services in a unified manner.
  • L2MC: Listen to IGMP/MLD packets and implement fast join/leave of channels.
  • L3I4: Issue IPv4 unicast forwarding entries on the device.
  • L3IO: Issue entries of Layer 3 protocols, such as URPF and VRRP, on the device.
  • L3M4: Adapt to the ARP protocol on the device, issue IPv4 unicast forwarding entries, and respond to the changes at the control plane.
  • L3MB: Adapt to Layer 3 protocols on the device such as URPF and VRRP, and issue forwarding entries.
  • LACP: Implement the LACP protocol stack, manage the protocol state machine, and maintain protocol databases.
  • LDRV: Synchronize software versions between master and slave boards.
  • LDT: Implement the LDT protocol, manage the protocol state machine, and maintain protocol databases.
  • LHAL: Provide hardware adaptation for the device to shield hardware difference.
  • LINK: Schedule link layer tasks in a unified manner.
  • linkscan: Detect the port link status.
  • LLDP: Implement the LLDP protocol, manage the protocol state machine, and maintain protocol databases.
  • LOAD: Load mirrored version files and patch packages on the device.
  • LSPA: Maintain LSP forwarding entries and request the application-layer to maintain chip entries.
  • LSPM: Create, update, and delete LSPs.
  • MCSW: Adapt to the Layer 3 multicast protocol, respond to the changes at the control plane, and issue forwarding entries.
  • MERX: Process the packets received on the management interface.
  • MFF: Implement MFF.
  • MFIB: Manage Layer 3 multicast forwarding entries.
  • MIRR: Implement port mirroring.
  • MOD: Manage, allocate, and reclaim board numbers.
  • MSYN: Synchronize MAC address entries between boards.
  • MTR: Implement scheduled statistics on memory usage.
  • mv_rx0: Process the packets in queue 0 of the CPU.
  • mv_rx1: Process the packets in queue 1 of the CPU.
  • mv_rx2: Process the packets in queue 2 of the CPU.
  • mv_rx3: Process the packets in queue 3 of the CPU.
  • mv_rx4: Process the packets in queue 4 of the CPU.
  • mv_rx5: Process the packets in queue 5 of the CPU.
  • mv_rx6: Process the packets in queue 6 of the CPU.
  • mv_rx7: Process the packets in queue 7 of the CPU.
  • NDIO: Issue IPv6 unicast forwarding entries on the device.
  • NDMB: Adapt to the ND protocol on the device, issue IPv6 unicast forwarding entries, and respond to the changes at the control plane.
  • NQAC: Respond to and process NQA packets as an NQA client.
  • NQAS: Respond to and process NQA events and packets as an NQA server.
  • NSA: Manage chip entries at the VRP NetStream adaptation layer.
  • NTPT: Implement the NTP protocol, manage the protocol state machine, and maintain protocol databases.
  • OAM1: Adapt to the OAM 802.1ag protocol, respond to protocol-layer changes, and process the changes at the forwarding plane.
  • OAMI: Process packets received from logic cards.
  • OAMT: This is a task at the adaptation layer. Respond to protocol changes and maintain chip entries.
  • OS: Operating system.
  • PING: Quickly respond to ping packets.
  • PNGI: Provide the fast ping operation on the device and fast respond to the ping operation.
  • PNGM: Provide the fast ping operation on the device and fast respond to the ping operation.
  • Port: Process chip debugging commands.
  • port_statistics: Collect port statistics.
  • PPI: This is a task at the adaptation layer. Maintain chip interface status.
  • PTAL: Implement redirection authentication, authentication and authorization, manage the protocol state machine, and maintain protocol databases.
  • QOSA: Manage QoS configurations and maintain chip entries.
  • QOSB: Issue QoS entries on the device and maintain issued QoS entries.
  • RACL: Create session table entries based on TCP/UDP/ICMP initial packet, and monitor and age out session table entries.
  • RDS: Implement the RADIUS protocol, manage the protocol state machine, and maintain protocol databases.
  • RMON: Monitor system remotely.
  • root: System root task.
  • ROUT: Implement routing and route learning, select the optimal route, and issue FIB entries.
  • RPCQ: Invoke remote procedures.
  • RRPP: Implement the RRPP protocol on the device, detect interface status quickly, and issue hardware entries.
  • RSA: Calculate the RSA key.
  • RTMR: Manage scheduled tasks.
  • SAM: Issue service entries to the device and maintain issued entries.
  • SAPP: Manage application layer's protocol dictionary and whitelist, maintain software entries and request the adaptation layer to set chip status.
  • SDKD: Detect the status of the interface connected to the backplane and collect the packet rate on the interface.
  • SDKE: Display related LSW chip entries.
  • SECB: Issue security entries to the device and maintain issued security entries.
  • SECE: Implement functions such as ARP, IP, and CPU security functions, manage the protocol state machine, and maintain protocol databases.
  • SERVER: TCP/IP server task.
  • SFPM: Query manufacturer information and digital diagnosis information of optical modules.
  • SLAG: Implement E-Trunk.
  • SMAG: Smart link agent. Fast detect and process port status changes.
  • SMLK: Implement the SmartLink protocol, manage the protocol state machine, and maintain protocol databases.
  • smsL: Load the environment monitoring module.
  • smsR: Send environment monitoring requests.
  • smsT: Send packets for the environment monitoring system.
  • SNPG: Listen to and process IGMP and MLD protocol packets.
  • SOCK: Schedule and process IP packets.
  • SRM: Manage devices.
  • SRMI: Process external interruptions.
  • SRMT: Device management timer.
  • SRVC: Process DHCP packets related to IP sessions, and interact with the user access and authentication module to carry out authorization and accounting.
  • STFW: Implement super forwarding and maintain forwarding entries in the trunk memory.
  • STND: Help the operating system to schedule tasks and events.
  • STP: Implement the STP protocol stack, manage the protocol state machine, and maintain protocol databases.
  • STRA: Monitor and identify attack traffic and punish attack source.
  • STRB: Monitor the device and identify attack traffic.
  • SUPP: Process interruption messages and timer messages in the device management module.
  • t1: Implement the temporary task (operating system task).
  • TACH: Implement the HWTACACS protocol, manage the protocol state machine, and maintain protocol databases.
  • TAD: Transmit alarms.
  • TARP: Process alarm information.
  • tBulkClnt: USB insertion and removal driver management task (operating system task).
  • TCPKEEPALIVE: Keep the TCP connection.
  • TCTL: Control the upload of performance data collected in a batch.
  • tDcacheUpd: Update the disk cache (operating system task).
  • tExcTask: Process exceptions (operating system task).
  • TICK: Process the system timer task.
  • tLogTask: Process log tasks (operating system task).
  • TM: Maintain access service entries and chip entries.
  • tNetTask: Process network-related tasks (operating system task).
  • TNLM: Manage tunnels.
  • TNQA: Schedule NQA client tasks in a unified manner.
  • TRAP: Process alarm information.
  • tRlogind: Log in to the virtual terminal remotely (operating system task).
  • tTelnetd: Process the Telnet server task (operating system task).
  • TTNQ: Schedule NQA server tasks in a unified manner.
  • tUsbPgs: USB insertion and removal device management task (operating system task).
  • tWdbTask: Debugging task (operating system task).
  • U 34: Process user's commands.
  • UCM: Interact with the AAA module, process user status, and maintain user tables.
  • UDPH: UDP helper.
  • USB: Upgrade the version using the USB flash drive.
  • usbPegasusLib: USB hot LIB (operating system task).
  • usbPegasusLib_IRP: USB host I/O request LIB (operating system task).
  • UTSK: Optimize protocol processing and ensure the high priority of protocol packets.
  • VCON: Redirect traffic at the device's serial port.
  • VFS: Manage the virtual file system.
  • VIDL: Collect statistics on CPU usage of idle tasks.
  • VMON: Monitor task execution.
  • VP: Receive and sent VP packets between boards.
  • VPR: Receive VP packets between boards.
  • VPRE: Process VP messages.
  • VPS: Send VP packets between boards.
  • VRPT: Timer test task.
  • VRRP: Implement the VRRP protocol stack, manage the protocol state machine, and maintain protocol databases.
  • VT: Process the virtual terminal task.
  • VT0: Authenticate the first login user and process the user's commands.
  • VTRU: Process Vtrunk Up/Down events.
  • VTYD: Process login requests of all users.
  • WEB: Implement Web authentication.
  • WEBS: Allow users to log in to the device through Web.
  • XMON: Monitor system task running.
  • XQOS: QoS task.

Network Management OIDs Related to CPU Usage

Object Name

Object OID

Data Type

Description

Implemented Specifications

hwEntityCpuUsage1.3.6.1.4.1.2011.5.25.31.1.1.1.1.5

Integer32

This object indicates CPU usage.

The value ranges from 2 to 100.

read-only
hwEntityCpuUsageThreshold1.3.6.1.4.1.2011.5.25.31.1.1.1.1.6

Integer32

This object indicates the CPU usage threshold.

The value ranges from 2 to 100.

The default value is 95.

read-only

From group: Switch
  • x
  • convention:

WoodWood     Created Sep 26, 2017 11:32:47 Helpful(0) Helpful(0)

S1720&S2700&S3700&S5700&S6700 Series Switches Product Troubleshooting - Preliminary-7 High CPU Usage
  • x
  • convention:

Reply

Reply
You need to log in to reply to the post Login | Register

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " Privacy."
If the attachment button is not available, update the Adobe Flash Player to the latest version!
Fast reply Scroll to top