CloudElastic - restore index from prod to dev

Production Deployment

Get snapshot storage details

GET /_snapshot/found-snapshots
{
  "found-snapshots": {
    "type": "gcs",
    "uuid": "tVWAY9PxRiGvVueoQPJw5Q",
    "settings": {
      "bucket": "bucketname",
      "aws_account": "operations-1-europe-west3",
      "use_for_peer_recovery": "true",
      "client": "elastic-internal-7547ec",
      "base_path": "snapshots/7547ec6fc9394d3582bd1611ee7e953c",
      "region": "europe-west3",
      "email": "bucketname@cloud-production-storage.iam.gserviceaccount.com"
    }
  }
}

Development Deployment

Attach production storage

PUT /_snapshot/production
{
  "type": "gcs",
  uuid" : "tVWAY9PxRiGvVueoQPJw5Q",
  "settings": {
    "bucket": "bucketname",
    "aws_account": "operations-1-europe-west3",
    "use_for_peer_recovery": "true",
    "client": "elastic-internal-cb03d5",
    "base_path": "snapshots/7547ec....",
    "region": "europe-west3",
    "email": "u358@cloud-production-storage.iam.gserviceaccount.com"
  }
}

Notes:

  • it should fail with error “Unknown client name [elastic-internal-7547ec]. Existing client configs: elastic-internal-167eab,default“ just replace client from elastic-internal-cb03d5 to elastic-internal-167eab.
  • just in case add "readonly": true to settings, even if forgotten just rerun command with this addition

Now you should see available production snapshots

GET /_snapshot/production

GET /_snapshot/production/_all

GET /_cat/snapshots/production

Restore index

Details and docs can be found here: https://www.elastic.co/guide/en/elasticsearch/reference/7.17/snapshots-restore-snapshot.html

In general there are two cases: restoring of existing and non existing index, second one requires additional steps.

Restore non existing index

POST _snapshot/production/cloud-snapshot-2023.08.22-59zqlj4fshogwahszpa5jg/_restore
{
  "indices": "user_data_viewed_vacancy_09082023"
}

restore existing index

GET /user_data_viewed_vacancy_03032022/_search


POST _snapshot/production/cloud-snapshot-2023.08.22-59zqlj4fshogwahszpa5jg/_restore
{
  "indices": "user_data_viewed_vacancy_03032022",
  "rename_pattern": "(.+)",
  "rename_replacement": "restored_$1"
}

GET /user_data_viewed_vacancy_03032022/_search
GET /restored_user_data_viewed_vacancy_03032022/_search

DELETE /user_data_viewed_vacancy_03032022

POST _reindex
{
  "source": {
    "index": "restored_user_data_viewed_vacancy_03032022"
  },
  "dest": {
    "index": "user_data_viewed_vacancy_03032022",
    "op_type": "create"
  }
}

GET /user_data_viewed_vacancy_03032022/_search
GET /restored_user_data_viewed_vacancy_03032022/_search
DELETE /restored_user_data_viewed_vacancy_03032022

Notes: