How to Fix Failed Datadog Agent Image Pulls from DockerHub

Posted on

Today, my client’s ECS tasks suddenly failed at startup. This posed challenges for deploying new code and autoscaling.

Our tasks were failing to start with this error message:

CannotPullContainerError: inspect image has been retried 1 time(s): failed to resolve ref "docker.io/datadog/agent:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization fa...

It appears that we started getting throttled pulling the datadog/agent:latest image from DockerHub.

The Fix

To work around this problem, you’ll need to do one of the following.

Authenticate with DockerHub

If you have a paid user through DockerHub, you can authenticate with your credentials to avoid the rate limit. This documentation explains how.

Use the Google Container Registry (GCR) Mirror

Datadog has quietly updated their documentation to recommend pulling the image from GCR to avoid the DockerHub rate limit.

If you want to pull from GCR, change:

datadog/agent:latest

to

gcr.io/datadoghq/agent:latest

Use AWS Public Elastic Container Registry (ECR)

In addition to GCR, Datadog publishes verified images to AWS Public ECR.

If you want to pull from AWS Public ECR, change:

datadog/agent:latest

to

public.ecr.aws/datadog/agent:latest
Find an issue?
Open a pull request against my blog on GitHub.
Ben Limmer