When thinking about stateful applications, there are three main things to keep in mind:

  1. Volumes – root and data.
  2. IP’s –  Private and Public
  3. Instance type to match them.

At Spotinst, in order to run production workloads on spot instances, it is important to give flexibility around the instance types and families clusters are built from. For that to work with stateful instances, we had to do a couple things.

How Stateful configuration works

Provision a new Stateful Spot instance from the Spotinst Console or API. This will be just like provisioning a new instance from the EC2 console. Spotinst will also take regular backup snapshots of your instances over time.

  1. If a Spot Interruption occurs, the instance will be shut down and terminated.
  2. The EBS volumes associated with the instance will become detached.
  3. The Original root volume and data volumes will become available and the last EBS snapshot will be taken. A new AMI will be created from this snapshot.
  4. A new Recovery clone instance will be launched to replace the previous instance.
    1. The same data, private-IP, security groups, load balancers, and other metadata will be available to you.
    2. The recovery time can take as little as three minutes.
  5. New EBS volumes will be created and attached from the newly created AMI
  6. The new instance is launched and becomes healthy. From the standpoint of this single server, it will appear as if it were shut down for a period of time.