Limit IAM policies

[keyvaann]

It seems like ecr_access and ecr_pull_through_cache policies allow access to all ECR repositories. I think it's safer to limit them to specific resources. Also, I don't see an ECR repository to be defined in the Terraform code, so I'm not why it's being defined.

[baixiac]

It was added for the plan to host RADAR images in ECR in order to get around the issues of rate limiting by DockerHub. Maybe there is a general/not-cloud-specific solution now and if so, these policies can be removed.

[keyvaann]

It's also good to add the ECR pull through cache to our Terraform code since the issue is still there. I'm making changes to Helm charts to make it easier to define an alternative registry. RADAR-base/radar-helm-charts#310

[baixiac]

I have a GH workflow set up (currently disabled to minimise the cost) to sync ECR images with RADAR DockerHub images. Let me know if you guys are interested.

[keyvaann]

I haven't used ECR pull through cache yet but I think it is not needed to mirror images? My impression is that you change the image registry in Helm chart and then it should get the image first time from Dockerhub and next times from the cache.
https://docs.aws.amazon.com/AmazonECR/latest/userguide/pull-through-cache-working-pulling.html

[baixiac]

The ECR pull-through cache only supports "official" public DockerHub images and https://hub.docker.com/r/radarbase is not one of them (as of the last time I checked at least).

[keyvaann]

It looks like you can use regular images as well:

For all other Docker Hub images:
docker pull aws_account_id.dkr.ecr.region.amazonaws.com/docker-hub/repository_name/image_name:tag

[baixiac]

Can you test that in your own account and confirm if images from the radarbase community organisation can be cached?

[keyvaann]

Sure

[keyvaann]

The pull through cache works with custom images as well.

  Normal  Scheduled  88s   default-scheduler  Successfully assigned default/radar-jdbc-connector-5c9cdf46fd-x8sgx to
  Normal  Pulling    87s   kubelet            Pulling image "...amazonaws.com/docker-hub/radarbase/radar-jdbc-connector:10.5.2"
  Normal  Pulled     62s   kubelet            Successfully pulled image "....amazonaws.com/docker-hub/radarbase/radar-jdbc-connector:10.5.2" in 24.973s (24.973s including waiting). Image size: 1084429937 bytes.

[baixiac]

Thanks for testing it. This means ECR has relaxed their rule since this announcement. When last time I checked their PTC only supports officials.

[keyvaann]

Would it make sense to add the terraform code to create a ECR pull through cache to this repository?

[baixiac]

Yes, will TAL. Looks like ECR PTC requires the DockerHub user credentials to be set as an SM secret, which is a less straightforward solution than I thought it would be.

[baixiac]

PTAL at #44, @keyvaann.