sai sasank:
Hi Team
I have 3 nodes in one cluster, where only two nodes are scheduling pods.
while the third node is auto tainted with “DeletionCandidateOfClusterAutoscaler=1662896133:PreferNoSchedule”
But I need this node to be available to create pods.
what’s the process to achieve it?
sai sasank:
Thanks for response @Ibrahim.
I tried to remove taint manually by giving cmd like
“Kubectl taint node nodename deletioncadidateofclusterautoscaler”. Taint removes for a second and adding again.
No use of it. Any other process?
unnivkn:
taint on node node1.
kubectl taint nodes node1 key1=value1:NoSchedule
##To remove the taint added by the command above, you can run:
kubectl taint nodes node1 key1=value1:NoSchedule-
Alistair Mackay:
@sai sasank that taint is not there by accident.
It means you have Cluster Autoscaler running in your cluster and it has determined that the cluster is overprovisioned and has marked the node for termination. It placed the taint to prevent new pods being scheduled while it gradually evicts any remaining pods. If the autoscaler is working properly and you want to deploy more pods, then it will add more nodes as necessary.
If for some reason the cluster autoscaler deployment was deleted or scaled to zero, then that taint will remain until manually removed. If it is still active, then it will replace the taint if you delete it.
Cluster autoscaler should be found as a deployment in the kube-system namespace.
sai sasank:
Thanks @Ali Al Idrees. This really helped me. I have made autoscaler deployment to zero. which helped me to remove taint and now pods are getting generated on the node. Thanks a lot 
Alistair Mackay:
But is that what you really want to do? The cluster autoscaler was deployed for a reason. If you’ve scaled it to zero, then it will not function and if you deploy more workloads, the cluster cannot add new nodes and you will be stuck with pods in “Pending” state.
sai sasank:
as it is just Dev environment, there wont be much workload on it. so, hopefully wont get any issue.
So, If I made autoscaler replicas to 3, will that allow 3 nodes to have pods?
Alistair Mackay:
Depending on which cloud provider you use, select the correct folder for it in here https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler/cloudprovider and see the README within. This details the parameters you can set to tune autoscaler for your environment.
Alistair Mackay:
It is well worth tuning the autoscaler in your lower environments using performance/load/stress workloads so you can know how it should be set in the production environment.
sai sasank:
yes. I will have a practice on it and apply.