Introducing Elastigroup’s Data Volume Persistency
Data volume persistency maintains the data volumes during Spot instance replacement. All data volumes that were attached at the time of the previous instance termination will be automatically re-attached using the same BlockDeviceMapping configuration upon instance replacement.
Configuring Data Volume Persistency
To configure data volume persistency head to the Elastigroup’s Creation Wizard, select Compute and open the Stateful option. Under Persist Data Volumes select a persistence method.
Elastigroup provides 2 different methods for data volume persistency:
- Reattach Volumes (recommended for large data volumes):The same EBS volume is detached from the original instance and reattached to the newly launched instance. If the new instance is launched in a different AZ, we create a new volume and attach it to the new instance (as volumes can’t be migrated between AZ’s).Volumes can be created based on the AMI’s Block Device Mapping on the first resume of the Stateful instance or can be attached through AWS console. We maintain the same volumes as long as the instance is launched in the same Availability Zone.
- Snapshot Backups: Periodic snapshots of the data volume are taken while the instance is running. For each data volume, 3 snapshots are kept. Upon Spot instance replacement, a new EBS volume is created from the latest snapshot and is attached to the new instance by updating the AMI’s Block Device Mapping configuration.
Suspending User Data Execution Until Volumes are Available
When using the Reattach option for data volume persistency, the instances are launched, followed by the attachment of the volumes. To prevent the User Data from executing before the volumes are attached add the following script:
#!/bin/bash EC2_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id) EC2_AVAIL_ZONE=`curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone` EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`" DEVICE_NAME="/dev/xvdb" DATA_STATE="unknown" until [ "$DATA_STATE" == "attached" ]; do DATA_STATE=$(aws ec2 describe-volumes \ --region $EC2_REGION \ --filters \ Name=attachment.instance-id,Values=$EC2_INSTANCE_ID \ Name=attachment.device,Values=$DEVICE_NAME \ --query Volumes.Attachments.State \ --output text) echo $DATA_STATE sleep 5 done echo "volume is ready"