Retrieve data from AWS S3 Glacier to AWS S3 Standard

Here we explain the steps required to retrieve data from Glacier to S3 Standard format.

Steps:

Restore the objects from glacier (temporarily) and then overwrite them using the COPY operation which essentially creates new objects and they’ll stay in standard format. This would be done using awscli on terminal.

1- Get List of Object in Glacier (Optional)

If you need to restore all objects from a bucket in glacier format then store that list to a text file.

using the following command :

aws s3api list-objects-v2 --bucket <bucket_name> --prefix <required_prefix> --query "Contents[?StorageClass=='GLACIER']" --output text | awk -F '\t' '{print $2}' > glacier-restore.txt

In case you have to restore a single file or you already know the list of files then you can skip this step.

Restore Objects:

Trigger glacier restore job in S3 using the restore command on the files that are in Glacier ,this might take 2–3 hrs to complete

aws s3api restore-object --restore-request Days=7 --bucket <bucket_name> --key "<required_prefix>"

If you have multiple files you can create a shell script to run command on all of them

#!/bin/sh
IFS=$'\n'
for x in `cat glacier-restore.txt`
  do
    echo "Begin restoring ${x}"
    aws s3api restore-object --restore-request Days=7 --bucket <bucketName> --key "${x}"
    echo "Done restoring ${x}"
  done

Copy for permanent restore to standard S3 format:

Following command for a permanent restore to standard S3 format

aws s3 cp s3://<bucket_name>/"<required_prefix>" s3://<bucket_name>/"<required_prefix>" --force-glacier-transfer --storage-class STANDARD

For multiple files you can use the following shell script in the similar mannar as the previous step

#!/bin/sh
IFS=$'\n'
for x in `glacier-restore.txt`
do
    echo "Begin restoring ${x}"
    aws s3 cp s3://<bucket_name>/"${x}" s3://<bucket_name>/"${x}" --force-glacier-transfer --storage-class STANDARD
    echo "Done restoring ${x}"
done

For more details you can visit following stackoverflow page as well

https://stackoverflow.com/questions/51670231/how-do-i-restore-from-aws-glacier-back-to-s3-permanently