k8s pod的Evicted状态处理

查询Evicted状态pod

使用命令查看Pod的状态，发现很多pod的状态为Evicted

1	$ kubectl get pods -n <Namespace>

使用describe查看pod日志

$ kubectl describe -n qa po wagepay-in-6447f7db46-fxxdl

  Warning  Evicted              14m   kubelet, node2  The node was low on resource: memory. Container wagepay-in was using 640456Ki, which exceeds its request of 64Mi.
  Normal   Killing              14m   kubelet, node2  Stopping container wagepay-in
  Warning  ExceededGracePeriod  13m   kubelet, node2  Container runtime did not kill the podwithin specified grace period.

问题原因

Eviction 即驱赶的意思，当节点出现异常时，kubernetes将有相应的机制驱赶该节点上的Pod。从日志上看是因为内存不足导致的驱赶。

解决方法

排查资源和异常原因，防止新的驱赶产生。处理资源问题后，删除Evicted状态pod

1	$ kubectl get pods -n qa \| grep Evicted \| awk '{print $1}' \| xargs kubectl -n qa delete pod

批量删除指定状态的 Pods
命令如下：

kubectl get pods -n <Namespace> | grep Evicted | awk ‘{print $1}’ | xargs kubectl delete pod -n <Namespace>