How to Fix Failed Datadog Agent Image Pulls from DockerHub
Posted on 02 March 2022 by Ben Limmer
Today, my client’s ECS tasks suddenly failed at startup. This posed challenges for deploying new code and autoscaling.
Our tasks were failing to start with this error message:
CannotPullContainerError: inspect image has been retried 1 time(s): failed to resolve ref "docker.io/datadog/agent:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization fa...
It appears that we started getting throttled pulling the datadog/agent:latest image from DockerHub.
The Fix
To work around this problem, you’ll need to do one of the following.
Authenticate with DockerHub
If you have a paid user through DockerHub, you can authenticate with your credentials to avoid the rate limit. This documentation explains how.
Use the Google Container Registry (GCR) Mirror
Datadog has quietly updated their documentation to recommend pulling the image from GCR to avoid the DockerHub rate limit.
If you want to pull from GCR, change:
datadog/agent:latest
to
gcr.io/datadoghq/agent:latest
Use AWS Public Elastic Container Registry (ECR)
In addition to GCR, Datadog publishes verified images to AWS Public ECR.
If you want to pull from AWS Public ECR, change:
datadog/agent:latest
to
public.ecr.aws/datadog/agent:latest