Unable to work with labs due to docker pull limit

Trying to prepare for CKA certification and working on labs in Certified Kubernetes Administrator – Full Mock Exam Series course.
Since yesterday I started to face issues on lab, when public docker images from docker registry is unable to be pulled properly, which messed my results in the lab since states and statuses were not as expected.
From today, my lab can’t even be properly provisioned, since it can’t pull image for first task.

Please fix this somehow, add a caching mirror like docker harbor for that use at least, since I really like your service and wanted to continue studying

This is also happening to me

1 Like

Our problem with access to Docker Hub is system-wide right now, and won’t be resolved in the next few hours :frowning: I don’t expect a change in status in the next 12 hours, since while we understand the fix, it will take time to do.

Hey Rob,

Yeah that’s totally understandable, we faced the same issue on our project right when docker applied such a limitation.

One of the possible solutions to this might be installation of a mirroring registry, that will be capable of providing a mirror for public registry, but will use a pre-paid token to authorize there.

So with this approach you will have an endpoint e.g. reg.kodekloud.com/dockerio that will be able to provide any docker.io image on request. Different solutions can solve that, we used harbor-registry by bitnami since it does the job.

Then, the tricky part is labs, since old labs configs will still use old image paths that will attempt to pull from the public docker registry facing the same limit.

For that you can configure registry mirrors for docker environments Registry as a pull through cache | Docker Docs, this will force docker search images with unspecified registry address on your selected mirror, e.g. requesting nginx:1.16 will be served from reg.kodekloud.com/dockerio/nginx:1.16, which in turn will request or provide cached image from docker.io configured mirror.

For kubernetes environment such change need to be done for containerd configs - containerd/docs/cri/registry.md at main · containerd/containerd · GitHub

That’s for sure will require to change a lot of servers configs for underlying labs but this will solve the issue once and forever (unless paid token expires, we faced that too :smiley: )

You basically understand our problem. There are a couple of other techniques that can be used, but yeah, that’s the general idea: we need to use a cache, and we need to use an authenticated account to access dockerhub. This will require some low level changes in how many of our labs are configured, so yeah, not a trivial change.

Hi Rob, please let us know when the docker pull limit is resolved! Also if this issue CKA Lightning Labs issue is also related to the same thing!

Our lab team has cut over new code that should fix this issue. If you’re still seeing this problem, make sure I know which course (and lab) you’re seeing the problem in, but our Kubernetes labs should now be working, and at least some of our Docker courses are working as well.

@rob_kodekloud it seems that this lab “Set Up Time Check Pod in Kubernetes” in engineer.kodekloud still has the same issue.

We’re still cutting over labs. We know that this issue affects engineer.kodekloud.com, so we we’ll get to it pretty quickly.

1 Like