-
Notifications
You must be signed in to change notification settings - Fork 3
Description
Problem
We are seeing "rate pull limit" errors which prevents us from compiling ReCiterDB and other applications.
Cause
According to this site: "If you use a cloud service (such as CodeBuild) to build your Docker images, this new rate limiting will likely affect you since the cloud build service will likely exceed Docker’s new rate limit thresholds. The service is building images for thousands of other customers, so Docker sees thousands of pull requests coming from the build service and they no idea who the requests belong to."
Supposedly, this is how you manually check this data.
TOKEN=$(curl https://auth.docker.io/token?service=registry.docker.io&scope=665[image name]:ratelimitpreview/test:pull | jq -r .token)
curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest
Goal
Our goal should be to authenticate into Docker as part of the build so that only our builds are credited to our account.
Requirements
As per the above link:
Update buildspec.yml like so...
aws s3 cp s3://mybucket/mypath/.docker.key .docker.key
docker login -u myusername --password-stdin < .docker.key
rm .docker.key
If this works, we should do this for our other builds.
- ReCiter
- ReCiter Publication Manager
- ReCiter PubMed Retrieval Tool
- ReCiter Scopus Retrieval Tool
- ReCiter Connect
Credentials
- Username for Docker = wcmcReciter
- Password for Docker = I have this
- .Docker.key = I have this file
Additional info
https://stackoverflow.com/questions/65806330/toomanyrequests-you-have-reached-your-pull-rate-limit-you-may-increase-the-lim
https://aws.amazon.com/blogs/containers/advice-for-customers-dealing-with-docker-hub-rate-limits-and-a-coming-soon-announcement/
Instructions
Here's how you can authenticate to Docker as part of your build process:
- Create a Docker Hub Account: If you don't already have a Docker Hub account, you need to create one at Docker Hub.
- Login Locally: Before building your image, you should login to Docker Hub from your command line using the following command:
docker login
-
This command will prompt you to enter your Docker Hub username and password. Once authenticated, this login session will be used for subsequent image pulls.
-
Using Docker Login in CI/CD Pipelines: If you are running your builds in a CI/CD pipeline, you can use environment variables to store your Docker Hub credentials. Most CI/CD platforms provide a way to securely store such secrets. You can then use these secrets in your pipeline script to log in to Docker Hub before the build starts.
For example, in a CI/CD script, you could use:
echo $DOCKER_PASSWORD | docker login --username $DOCKER_USERNAME --password-stdin
Here, DOCKER_USERNAME and DOCKER_PASSWORD are environment variables that store your Docker Hub credentials.
- Configure Docker for Automated Builds: If you are using automated build systems, you may need to configure your system to use a Docker credentials store or to automatically log in to Docker Hub using a script.