Updating Task Suspended

Problem Symptom

No progress is displayed on the microservice updating page of DeployManager for more than 30 minutes.

Problem Diagnosis

1. Log in to the active node of DeployManager as user dfvmanager using SSH, and then switch to user root.

2. Continuously observe the scheduled task logs of celery to check whether Sending due task query_doing_task_result is updated. Example:

   tailf /var/log/servicetool/CloudAutoDeploy/sdk/celerybeat.log
   

   

3. If Sending due task query_doing_task_result is not updated for five minutes, the updating task is suspended. Otherwise, no further action is required.

Causes

Time hopping occurs on the node.

Solution

1. Log in to the active node of DeployManager as user dfvmanager using SSH, and then switch to user root.

2. Delete the celerybeat-schedule file.

   rm /var/log/servicetool/CloudAutoDeploy/sdk/celerybeat-schedule
   

3. Restart the celery process.

   sh /opt/admin/CloudAutoDeploy/deploy/sdk/deployscripts/clyservice stopsh /opt/admin/CloudAutoDeploy/deploy/sdk/deployscripts/clyservice start
   

Check After Recovery

1. Log in to DeployManager again.

2. Go to the updating page and check whether the updating task is recovered.

Suggestion and Summary

N/A

Applicable Versions

All