Skip to content

Remove dependency on undocumented binaries in crate docker imageΒ #710

@mfussenegger

Description

@mfussenegger

Problem

Currently the operator uses various tools like hostname or curl:

# Availability zone retrieval at pod launch time
if config.CLOUD_PROVIDER == CloudProvider.AWS:
aws_cmd = (
"curl -s -X PUT 'http://169.254.169.254/latest/api/token' "
"-H 'X-aws-ec2-metadata-token-ttl-seconds: 120' | "
"xargs -I {} curl -s "
"'http://169.254.169.254/latest/meta-data/placement/availability-zone'"
" -H 'X-aws-ec2-metadata-token: {}'"
)
settings["-Cnode.attr.zone"] = f"$({aws_cmd})"
elif config.CLOUD_PROVIDER == CloudProvider.AZURE:
url = "http://169.254.169.254/metadata/instance/compute/zone?api-version=2020-06-01&format=text" # noqa
settings["-Cnode.attr.zone"] = f"$(curl -s '{url}' -H 'Metadata: true')"
elif config.CLOUD_PROVIDER == CloudProvider.GCP:
url = "http://169.254.169.254/computeMetadata/v1/instance/zone" # noqa
# projects/<account-id>/zones/us-central1-a
settings[
"-Cnode.attr.zone"
] = f"$(curl -s '{url}' -H 'Metadata-Flavor: Google' | rev | cut -d '/' -f 1 | rev)" # noqa

These are not documented or tested to be available in docker-crate.
Furthermore, the logic depends on bin/crate allowing shell expansion - which is also not officially supported but rather working by accident.

Options

  1. Operator stops using these commands and shell expansion.
  2. The commands get official support with tests and documentation. Same for the shell expansion.

As for 1)

  • One way to do that could be to inject a wrapper script into the container - that could even be a python script because that's available for crash.
  • We could look into what the commands are used for and see if/where it is appropriate to add first class support in CrateDB. E.g. have hostname inferred from the system-hostname. Or via some env variable. Detect that it is running in a container and infer container name. Things like that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    importantNot a bug, still important.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions