Skip to content

Curator restore command doesn't take into account the rename pattern #1375

@piclemx

Description

@piclemx

To submit a bug or report an issue

(⎈ |nonprod3.us-east-1:default) ➜  ~ curator --config ~/.curator/prod_curator.yml ~/.curator/restore.yml --dry-run
/usr/local/lib/python3.7/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  return yaml.load(read_file(path))
2019-04-06 19:49:56,530 INFO      Preparing Action ID: 1, "restore"
2019-04-06 19:49:56,648 INFO      Trying Action ID: 1, "restore": Restore all indices in the most recent snapshot with state SUCCESS.  Wait for the restore to complete before continuing.  Do not skip the repository filesystem access check.  Use the other options to define the index/shard settings for the restore.
2019-04-06 19:49:56,939 INFO      DRY-RUN MODE.  No changes will be made.
2019-04-06 19:49:56,939 INFO      DRY-RUN: restore: Repository: migration Snapshot name: curator-20190406232030 Arguments: {'wait_for_completion': True, 'body': {'indices': ['padre_payloads-2019.04.05', 'eventbase_logs-2019.04.05', 'offer_payloads-2019.04.05', 'client_setup_payloads-2019.04.05', 'client_setup_envoy_logs-2019.04.05', 'venuelayout_payloads-2019.04.05', 'fees_envoy_logs-2019.04.05', 'prd130-prod1-aurora_rds_logs-2019.04.05', 'offer_accesslog-2019.04.05', 'venuelayout_logs-2019.04.05', 'offer_pxy_logs-2019.04.05', 'venuelayout_accesslog-2019.04.05', 'eventbase_uilogs-2019.04.05', 'eventbase_pxy_logs-2019.04.05', 'offer_logs-2019.04.05', 'eventbase_payloads-2019.04.05', 'hic_logs-2019.04.05', 'hic_envoy_logs-2019.04.05', 'padre_logs-2019.04.05', 'dic_logs-2019.04.05', 'offer_envoy_logs-2019.04.05', 'eventbase_accesslog-2019.04.05', 'hic_payloads-2019.04.05', 'fees_payloads-2019.04.05', 'dic_payloads-2019.04.05', 'liquibaser_logs-2019.04.05', 'fees_logs-2019.04.05', 'client_setup_logs-2019.04.05'], 'include_aliases': False, 'ignore_unavailable': False, 'include_global_state': False, 'partial': False, 'rename_pattern': '([^-]*)-(\\\\d)', 'rename_replacement': '$1-restored-$2'}}
2019-04-06 19:49:56,939 INFO      DRY-RUN: restore: Index padre_payloads-2019.04.05 as padre_payloads-2019.04.05
2019-04-06 19:49:56,939 INFO      DRY-RUN: restore: Index eventbase_logs-2019.04.05 as eventbase_logs-2019.04.05
2019-04-06 19:49:56,939 INFO      DRY-RUN: restore: Index offer_payloads-2019.04.05 as offer_payloads-2019.04.05
2019-04-06 19:49:56,939 INFO      DRY-RUN: restore: Index client_setup_payloads-2019.04.05 as client_setup_payloads-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index client_setup_envoy_logs-2019.04.05 as client_setup_envoy_logs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index venuelayout_payloads-2019.04.05 as venuelayout_payloads-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index fees_envoy_logs-2019.04.05 as fees_envoy_logs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index prd130-prod1-aurora_rds_logs-2019.04.05 as prd130-prod1-aurora_rds_logs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index offer_accesslog-2019.04.05 as offer_accesslog-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index venuelayout_logs-2019.04.05 as venuelayout_logs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index offer_pxy_logs-2019.04.05 as offer_pxy_logs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index venuelayout_accesslog-2019.04.05 as venuelayout_accesslog-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index eventbase_uilogs-2019.04.05 as eventbase_uilogs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index eventbase_pxy_logs-2019.04.05 as eventbase_pxy_logs-2019.04.05
2019-04-06 19:49:56,940 INFO      DRY-RUN: restore: Index offer_logs-2019.04.05 as offer_logs-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index eventbase_payloads-2019.04.05 as eventbase_payloads-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index hic_logs-2019.04.05 as hic_logs-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index hic_envoy_logs-2019.04.05 as hic_envoy_logs-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index padre_logs-2019.04.05 as padre_logs-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index dic_logs-2019.04.05 as dic_logs-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index offer_envoy_logs-2019.04.05 as offer_envoy_logs-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index eventbase_accesslog-2019.04.05 as eventbase_accesslog-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index hic_payloads-2019.04.05 as hic_payloads-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index fees_payloads-2019.04.05 as fees_payloads-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index dic_payloads-2019.04.05 as dic_payloads-2019.04.05
2019-04-06 19:49:56,941 INFO      DRY-RUN: restore: Index liquibaser_logs-2019.04.05 as liquibaser_logs-2019.04.05
2019-04-06 19:49:56,942 INFO      DRY-RUN: restore: Index fees_logs-2019.04.05 as fees_logs-2019.04.05
2019-04-06 19:49:56,942 INFO      DRY-RUN: restore: Index client_setup_logs-2019.04.05 as client_setup_logs-2019.04.05
2019-04-06 19:49:56,942 INFO      Action ID: 1, "restore" completed.
2019-04-06 19:49:56,942 INFO      Job completed.

Expected Behavior

I will have expected that the restore will rename the index on the left.. When I removed the --dry-run the job failed because I already have a index with that name (expected)

Steps to Reproduce the Problem

actions:
  1:
    action: restore
    description: >-
      Restore all indices in the most recent snapshot with state SUCCESS.  Wait
      for the restore to complete before continuing.  Do not skip the repository
      filesystem access check.  Use the other options to define the index/shard
      settings for the restore.
    options:
      repository: migration
      # If name is blank, the most recent snapshot by age will be selected
      name:
      # If indices is blank, all indices in the snapshot will be restored
      indices:
      rename_pattern: '([^-]*)-(\\d)'
      rename_replacement: '$1-restored-$2'
      wait_for_completion: True
      max_wait: -1
      wait_interval: 10
    filters:
    - filtertype: state
      state: SUCCESS
      exclude:

Specifications

  • Version: 5.6.0
  • Platform: MAC OSX
  • Subsystem:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions