Skip to content

States

A "State" represents a collection of persistent volumes that together form the persistent state of a Kubernetes workload. Each state (i.e., all the data stored in its volumes) can be available any of your Kubernetes clusters anywhere in the world, so that you can instantly move your application between geographical regions and cloud vendors.

An example of a workload and its corresponding state would be an web application consisting of:

  1. A stateful set that has 3 replicas, running a MongoDB container, claiming a persistent volume of 200GiB for each replica.
  2. A stateful set that has 3 replicas, running an Nginx container, claiming a persistent volume of 100GiB for each replica
  3. Other pods, deployment, services and stateless components that constitute the application's business logic and microservices, but do not require persistent storage.

This state for this workload would consist of:

  1. 3 volumes of 200GiB (for MongoDB)
  2. 3 volumes of 100GiB (for Nginx)

Depending on your infrastructure's topology and your application requirements, you can create multiple states for unrelated workloads.

Creating a State

You can create a state by defining its name, and optionally specifiying clusters the state needs to be visible to, its locations, and the owner cluster.

Since adding location to a state takes time, it's recommended to specify the clusters you intend to use the state with, if those clusters are already registered with your Statehub organization. Statehub will figure out which locations need be added to the state according to the locations of the clusters you've specified. If one or some of the clusters are not registered, or you plan to install them "on demand" at a later time, you can specify the location or locations where you intend to have clusters accessing the state in the future.

To create a state:

  • Using the Statehub CLI:

    To create a state without defining locations, clusters or owner, run:

    statehub create-state my-state
    

    To specify clusters and have Statehub figure out the locations for the state, use the -c flag:

    statehub create-state my-state -c cluster1 -c cluster2
    

    To specify the locations explicitly, use the -l flag:

    statehub create-state my-state -l aws/us-west-2 -l azure/eastus2
    

    To specify the cluster that will be the owner of the state upon its creation, use the -o flag:

    statehub create-state my-state -c cluster1 -c cluster2 -o cluster1
    

    The -c, -l, and -o flags are not mutually exclusive.

  • Using the Statehub Management Console:

    1. Go to https://console.statehub.io/states/
    2. Click the "Create State" button
    3. Select a name for the cluster, and optionally the state's owner and locations
    4. Click the "Create" button

Getting State Information

A state has the following properties:

  • Name – the name of the state
  • ID – a unique ID generated by Statehub for the state
  • Storage class properties – the configuration of the storage class in all the cluster the state is available to, including:
    • Storage class name
    • Default file system type
    • Volume binding mode
    • Mount options
  • Owner cluster – the cluster that is currently designed as owner for the states, has access to the data and can perform state operations
  • Creation and modification dates
  • A list of locations the state spans – each location has the following attributes:
    • Statusok, provisioning, recovering, deleting, or error
    • Private link service – the name of the private link service the Statehub controller that runs on your cluster needs to connect to in order to give your cluster access to the volumes
  • A list of volumes the state contains – each volume has the following attributes:
    • Name – the name of the volume, identical to the persistent volume claim used by your applications that access the volume's data
    • Size – the size of the volume in GiB
    • Status of the volume in each locationok, provisioning, recovering, deleting, or error
  • The volume's condition – an overall indicator of the state's health. Can be one of:
    • Green – All operation is normal, there are no failures or problems. Some locations or volumes may be provisioning as per the user's or cluster's request.
    • Yellow – Failures detected. Recovery in progress or pending. Normal operation is possible in at least one location.
    • Red – Failures detected. Recovery is currently impossible due to insufficient healthy copies of the data.

To get a list of all states:

  • Using the Statehub CLI:

    statehub list-states
    

    Example Output:

    ☘        default 🟢 (2 volumes)  [aws/us-east-1 🆗] (🔒 aws-us-east)
    ☘       my-state 🟢 (3 volumes)  [aws/us-east-2 🆗, azure/eastus2 🆗] (🔓)
    

    In this example, we see two states:

    • default, condition green, has 2 volumes, has 1 location: aws/us-east-1, owned by cluster aws-us-east
    • my-state, condition green, has 3 volume, has 2 locations: aws/us-east-2 and azure/eastus2, not owned by a cluster at the moment
  • Using the Statehub Management Console:

    1. Go to https://clonsole.statehub.io/states/

To show detailed information about a state:

  • Using the Stateub CLI:

    statehub show-state my-state
    

    Example Output:

    State:         my-state
    Id:            daa15549-28a0-46e9-97ae-f5b9761670b2
    Storage Class: my-state.demo.statehub.io (ext4)
    Owner:         🔒 azure-us-east
    Created:       4 hours ago
    Modified:      7 minutes ago
    Condition:     🟢
    Locations:
    aws/us-east-2:
      Status: 🆗
      PLS   : vpce-svc-0abe4be88b588a937 / com.amazonaws.vpce.us-east-2.vpce-svc-0abe4be88b588a937
    azure/eastus2:
      Status: 🆗
      PLS   : /subscriptions/2c437161-bc95-4456-b842-038c1fbf1a38/resourceGroups/demo/providers/Microsoft.Network/privateLinkServices/0324e86b-b090-437b-8e04-738a4ac6d336-pls
    Volumes:
      dbdata-mysql-0:
        azure/eastus2: 🆗, aws/us-east-2: 🆗
      dbdata-mysql-1:
        azure/eastus2: 🆗, aws/us-east-2: 🆗
      dbdata-mysql-2:
        azure/eastus2: 🆗, aws/us-east-2: 🆗
    Visible clusters:
        aws-us-east azure-us-east
    
  • Using the Statehub Management Console:

    1. Go to https://clonsole.statehub.io/states/{state-name}

State Locations

State locations are defined by a cloud vendor and region. For example, a location can be aws/us-west-1 or azure/eastus2. The locations determine where the volumes of the state are availabe. A state can be configured to span multiple locations, making all of the state's volumes' data replicated in real time between these locations. This allows you to start your workloads on a cluster located at these locations at any time.

If you want the state's data to be available at a new location, you need to add the location to the state. This will synchronize the state's volumes in the new location.

Adding a Location

When adding a location to a state, all of the volumes in the state will begin synchronizing to the new location, until the all locations are brought no near synchronicity. From that point on, all data written by the owner cluster to the state's volumes is replicated in real time to all locations.

Initial synchronization of a state with existing volumes to a new location takes a while to complete, depending on the total size of the volumes.

To add a location to the state:

  • Using the Statehub CLI: You can add a location by specifying it explicitly:

    statehub add-location my-state aws/east-us-1
    

    Or, you can add a location by specifying a cluster you want to make the state available for by using the -c flag:

    statehub add-location my-state -c my-cluster
    

    Since adding a location is a time-consuming operation, you can specify you'd like the CLI to wait for it to complete by adding the --wait flag.

  • Using the Statehub Management Console:

  • Go to https://console.statehub.io/states/

  • Click on the state to which you want to add the location
  • Depending on the cloud vendor for the location click either "Add AWS Location" or "Add Azure Location", select the necessary cloud region, and click "Add"

Removing a Location

When a state location is no longer needed (i.e., you no longer need clusters in this location to access the state's volumes), you can remove the location from the state.

To remove a location from a state:

  • Using the Statehub CLI: Specify the state and the location you wish to remove from it:

    statehub remove-location my-state aws/us-west-2
    
  • Using the Statehub Management Console:

    1. Go to https://console.statehub.io/states/
    2. Click on the state to which you want to add the location
    3. Click on the "Remove" button next to the loation you want to remove, and confirm

The Default State

When you create your account, and with it your Statehub organization, a default state (called default) will be created. Upon its initial creation, it has no locations. However, the default behavior of the Statehub CLI is to extend its locations to the locations where your clusters are located.

In other words, whenever you registed a cluster using the statehub register-cluster command, Statehub will figure out where the cluster is located, and will add this location to the default state so that you can start using it immediately in the cluster you've just registered.

This is a convenient in configurations in which you have one main workload that you want to be able to move between your clusters. Just by registering your clusters you'll be able to enable stateful application mobility between them. However, for advanced configurations, this feature that can be disabled.

States and Your Clusters

Visibility and Access

A state will be visible to all clusters that have nodes located in one or more of the state's locations. For example, a state has three locations: aws/us-west-1, aws/us-east-1 and azure/eastus2 will be visible to all clusters that have nodes in these locations, but not visible to clusters with nodes only in aws/eu-central-1.

When a state is visible to a cluster, a storage class object is automatically created for it. The storage class name has the following format:

{stateName}.{YourStatehubOrgId}.statehub.io

Since the default behavior for the Statehub CLI when registering a cluster is to extend the default state to the location of the cluster, and to set it as the default storage class, clusters registered with the default arguments will have a default storage class called default.YourStatehubOrgId.statehub.io.

In terms of networking and storage protocol, states' volumes are accessed by your clusters via iSCSI, through your cloud vendor's Private Link service (in AWS and Azure). However, the necessary network and iSCSI configuration is taken care of entierly by the Statehub Controller running on your clusters.

State Ownership

A state can be owned by one cluster at any given time. A cluster that owns the volume can:

  1. Mount the volumes when running applications that use them.
  2. Provision new volumes on the state.

Changing the ownership of the state is possible by using the Statehub CLI, Management Console and the REST API.

To change the owner of the state:

  • Using the Statehub CLI: Specify the state and the cluster you want to be its owner:

    statehub set-owner my-state cluster2
    

    In this example, the cluster cluster2 will become the owner of the state my-state.

    ℹ️ Note:

    Clusters can only own states they have access to. In other words, a cluster and the state must share the same location.