The direct time asynchrony between the main OMS node and the agent node presents the problem of not available
Conclusion:
Due to the time asynchrony between the main OMS node and the agent node, the problem of not available is caused.
Location keywords:
It appears in the controller log
... 6 more
Caused by: java.io.IOException: Couldn't setup connection for oms/manager@HADOOP.COM to/160.165.0.83:20025
At org. apache. hadoop. ipc. Client $Connection $1. run (Client. java: 690)
At java. security. AccessController. doPrivileged (Native Method)
At javax. security. auth. Subject. doAs (Subject. java: 422)
At org. apache. hadoop. security. UserGroupInformation. doAs (UserGroupInformation. java: 1672)
At org. apache. hadoop. ipc. Client $Connection. handleSaslConnection Failure (Client. java: 661)
At org. apache. hadoop. ipc. Client $Connection. setupIOstreams (Client. java: 748)
At org. apache. hadoop. ipc. Client $Connection. access $2800 (Client. java: 370)
At org. apache. hadoop. ipc. Client. getConnection (Client. java: 1545)
At org. apache. hadoop. ipc. Client. call (Client. java: 1462)
... 8 more
Caused by: javax. security. sasl. SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Clock skew too big (37)]
At com. sun. security. sasl. gsskerb. GssKrb5Client. evaluateChallenge (GssKrb5Client. java: 211)
At org. apache. hadoop. security. SaslRpcClient. saslConnect (SaslRpcClient. java: 413)
At org. apache. hadoop. ipc. Client $Connection. setupSaslConnection (Client. java: 557)
At org. apache. hadoop. ipc. Client $Connection. access $1800 (Client. java: 370)
At org. apache. hadoop. ipc. Client $Connection $2. run (Client. java: 740)
At org. apache. hadoop. ipc. Client $Connection $2. run (Client. java: 736)
At java. security. AccessController. doPrivileged (Native Method)
At javax. security. auth. Subject. doAs (Subject. java: 422)
At org. apache. hadoop. security. UserGroupInformation. doAs (UserGroupInformation. java: 1672)
At org. apache. hadoop. ipc. Client $Connection. setupIOstreams (Client. java: 735)
... 11 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Clock skew too big (37))
At sun. security. jgss. krb5. Krb5Context. initSecContext (Krb5Context. java: 770)
At sun. security. jgss. GSSContextImpl. initSecContext (GSSContextImpl. java: 248)
At sun. security. jgss. GSSContextImpl. initSecContext (GSSContextImpl. java: 179)
At com. sun. security. sasl. gsskerb. GssKrb5Client. evaluateChallenge (GssKrb5Client. java: 192)
... 20 more
Caused by: KrbException: Clock skew too big (37)
At sun. security. krb5. KrbKdcRep. check (KrbKdcRep. java: 88)
At sun. security. krb5. KrbTgsRep. <init> (KrbTgsRep. java: 87)
At sun. security. krb5. KrbTgsReq. getReply (KrbTgsReq. java: 259)
At sun. security. krb5. KrbTgsReq. sendAndGetCreds (KrbTgsReq. java: 270)
At sun. security. krb5. internal. Credentials Util. service Creds (Credentials Util. java: 302)
At sun. security. krb5. internal. Credentials Util. acquireService Creds (Credentials Util. java: 120)
At sun. security. krb5. Credentials. acquireService Creds (Credentials. java: 458)
At sun. security. jgss. krb5. Krb5Context. initSecContext (Krb5Context. java: 693)
The red part is the key word.