mirror of
https://github.com/optim-enterprises-bv/vault.git
synced 2025-10-30 18:17:55 +00:00
Docs: add Integrated Storage migration checklist (#28995)
- Adds a new migration checklist from archived tutorial content
This commit is contained in:
@@ -0,0 +1,180 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Migration checklist
|
||||
description: Use this checklist for decision making related to migrating your Vault deployment to Integrated Storage.
|
||||
---
|
||||
|
||||
# Migration checklist
|
||||
|
||||
<Tip title="This is a decision-making checklist">
|
||||
|
||||
The purpose of this checklist is not to walk you through the storage
|
||||
migration steps. This content provides a quick self-check whether it is your
|
||||
best interest to migrate your Vault storage from an external system to
|
||||
Integrated Storage.
|
||||
|
||||
</Tip>
|
||||
|
||||
## Who should use this checklist?
|
||||
|
||||
Integrated Storage is a recommended storage option, made available in
|
||||
Vault 1.4. Vault continues to also support other storage solutions
|
||||
like Consul.
|
||||
|
||||
You should use this checklist if you are operating a Vault deployment backed
|
||||
by external storage like Consul, and you are considering migration to
|
||||
Integrated Storage.
|
||||
|
||||
## Understand architectural differences
|
||||
|
||||
It is important that you understand the differences between operating Vault
|
||||
with external storage and operating with Integrated Storage. The following
|
||||
sections detail key differences in architecture between Vault with Consul
|
||||
storage, and Vault with Integrated Storage to help inform your decision.
|
||||
|
||||
### Reference architecture with Consul
|
||||
|
||||
The recommended number of Vault instances is **3** in a cluster which connects
|
||||
to a Consul cluster which may have **5** or more nodes as shown in the diagram.
|
||||
|
||||
A total of 8 virtual machines hosts this Vault highly available architecture.
|
||||
|
||||
<ImageConfig hideBorder>
|
||||
|
||||

|
||||
|
||||
</ImageConfig>
|
||||
|
||||
The processing requirements depend on the encryption and messaging workloads.
|
||||
Memory requirements are dependant on the total size of secrets stored in
|
||||
memory. The Vault server itself has minimal storage requirements, but
|
||||
the Consul nodes should have a high-performance physical storage system.
|
||||
|
||||
### Reference architecture with Integrated Storage
|
||||
|
||||
The recommended number of Vault instances is **5** in a cluster. In a single HA
|
||||
cluster, all Vault nodes share the data while an active node holds the lock;
|
||||
therefore, only the active node has write access. To achieve n-2 redundancy,
|
||||
(meaning that the cluster can still function after losing 2 nodes),
|
||||
an ideal size for a Vault HA cluster is 5 nodes.
|
||||
|
||||
<Tip title="More deployment details in the documentation">
|
||||
|
||||
Refer to the [Integrated
|
||||
Storage](/vault/docs/internals/integrated-storage#deployment-table)
|
||||
documentation for more deployment details.
|
||||
|
||||
</Tip>
|
||||
|
||||
<ImageConfig hideBorder>
|
||||
|
||||

|
||||
|
||||
</ImageConfig>
|
||||
|
||||
Because the data gets persisted on the same host, the Vault server should be
|
||||
hosted on a relatively high-performance hard disk system.
|
||||
|
||||
## Consul vs. Integrated Storage
|
||||
|
||||
The Integrated Storage eliminates the need for external storage; therefore,
|
||||
Vault is the only software you need to stand up a cluster. This indicates that
|
||||
the host machine must have disk capacity in an amount equal or
|
||||
greater to that of the existing external storage backend.
|
||||
|
||||
### System requirements comparison
|
||||
|
||||
The fundamental difference between Vault's Integrated Storage and Consul is
|
||||
that the Integrated Storage stores everything on disk while [Consul
|
||||
KV](/consul/docs/dynamic-app-config/kv) stores everything in its memory
|
||||
which impacts the host's RAM.
|
||||
|
||||
#### Machine sizes for Vault - Consul as its storage backend
|
||||
|
||||
It is recommended to avoid hosting Consul on an instance with burstable CPU.
|
||||
|
||||
| Size | CPU | Memory | Disk | Typical Cloud Instance Types |
|
||||
| ----- | -------- | ------------ | ----- | ----------------------------------------- |
|
||||
| Small | 2 core | 4-8 GB RAM | 25 GB | **AWS:** m5.large |
|
||||
| | | | | **Azure:** Standard_D2_v3 |
|
||||
| | | | | **GCE:** n1-standard-2, n1-standard-4 |
|
||||
| Large | 4-8 core | 16-32 GB RAM | 50 GB | **AWS:** m5.xlarge, m5.2xlarge |
|
||||
| | | | | **Azure:** Standard_D4_v3, Standard_D8_v3 |
|
||||
| | | | | **GCE:** n1-standard-8, n1-standard-16 |
|
||||
|
||||
#### Machine sizes for Vault with Integrated Storage
|
||||
|
||||
| Size | CPU | Memory | Disk | Typical Cloud Instance Types |
|
||||
| ----- | -------- | ------------ | ------ | ------------------------------------------ |
|
||||
| Small | 2 core | 8-16 GB RAM | 100 GB | **AWS:** m5.large, m5.xlarge |
|
||||
| | | | | **Azure:** Standard_D2_v3, Standard_D4_v3 |
|
||||
| | | | | **GCE:** n2-standard-2, n2-standard-4 |
|
||||
| Large | 4-8 core | 32-64 GB RAM | 200 GB | **AWS:** m5.2xlarge, m5.4xlarge |
|
||||
| | | | | **Azure:** Standard_D8_v3, Standard_D16_v3 |
|
||||
| | | | | **GCE:** n2-standard-8, n2-standard-16 |
|
||||
|
||||
If many secrets are being generated or rotated frequently, this information will
|
||||
need to be flushed to the disk often. Therefore, the infrastructure should have
|
||||
a relatively high-performance hard disk system when using the integrated
|
||||
storage.
|
||||
|
||||
<Note title="A note about the importance of IOPS">
|
||||
|
||||
Vault's Integrated Storage is disk-bound; therefore, care should be taken when planning storage volume size and performance. For cloud providers, IOPS can be dependent on volume size and/or provisioned IOPS. It is recommended to provision IOPS and avoid burstable IOPS. Monitoring of IOPS performance should be implemented in order to tune the storage volume to the IOPS load.
|
||||
|
||||
</Note>
|
||||
|
||||
### Performance considerations
|
||||
|
||||
Because Consul KV is memory-bound, it is necessary to take a snapshot frequently.
|
||||
However, Vault's Integrated Storage persists everything on the disk which eliminates
|
||||
the need for such frequent snapshot operations. Take snapshots to back up the data
|
||||
so that you can restore them in case of data loss. This reduces the performance cost
|
||||
introduced by the frequent snapshot operations.
|
||||
|
||||
In considering disk performance, since Vault data changes are immediately written to disk,
|
||||
rather than in batched snapshots as Consul does, it is important to monitor IOPS as well
|
||||
as disk queues to limit storage bottlenecks.
|
||||
|
||||
|
||||
### Inspect Vault data
|
||||
|
||||
Inspection of Vault data differs considerably from the `consul kv` commands used
|
||||
to inspect Consul's KV store.
|
||||
Consult the [Inspect Data in Integrated Storage](/vault/tutorials/monitoring/inspect-data-integrated-storage)
|
||||
tutorial to learn more about querying Integrated Storage data.
|
||||
|
||||
### Summary
|
||||
|
||||
The table below highlights the differences between Consul and integrated
|
||||
storage.
|
||||
|
||||
| Consideration | Consul as storage backend | Vault Integrated Storage |
|
||||
| ------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| System requirement | Memory optimized machine | Storage optimized high IOPS machine |
|
||||
| Data snapshot | Frequent snapshots | Normal data backup strategy |
|
||||
| Snapshot automation | Snapshot agent (**Consul Enterprise only**) | Automatic snapshot (**Vault Enterprise v1.6.0 and later**) |
|
||||
| Data inspection | [Online, use `consul kv` command](/vault/tutorials/monitoring/inspecting-data-consul) | [Offline, requires using recovery mode](/vault/tutorials/monitoring/inspect-data-integrated-storage) |
|
||||
| Autopilot | Supported | Supported (**Vault 1.7.0 and later**) |
|
||||
|
||||
## Self-check questions
|
||||
|
||||
- [ ] Where is the product expertise?
|
||||
- [ ] Do you already have Consul expertise?
|
||||
- [ ] Are you concerned about lack of Consul knowledge?
|
||||
- [ ] Do you experience any technical issues with Consul?
|
||||
- [ ] What motivates the data migration from the current storage to Integrated Storage?
|
||||
- [ ] Reduce the operational overhead?
|
||||
- [ ] Reduce the number of machines to run?
|
||||
- [ ] Reduce the cloud infrastructure cost?
|
||||
- [ ] Do you have a staging environment where you can run production loads and verify that everything works as you expect?
|
||||
- [ ] Have you thought through the storage backup process or workflow after migrating to the Integrated Storage?
|
||||
- [ ] Do you currently rely heavily on using Consul to inspect Vault data?
|
||||
|
||||
## Tutorials
|
||||
|
||||
If you are ready to migrate the current storage backend to Integrated Storage,
|
||||
refer to the [Storage Migration Tutorial - Consul to Integrated Storage](/vault/tutorials/raft/raft-migration).
|
||||
|
||||
To deploy a new cluster with Integrated Storage, refer to the [Vault HA Cluster
|
||||
with Integrated Storage](/vault/tutorials/raft/raft-storage) tutorial.
|
||||
@@ -251,6 +251,10 @@
|
||||
{
|
||||
"title": "Autopilot",
|
||||
"path": "concepts/integrated-storage/autopilot"
|
||||
},
|
||||
{
|
||||
"title": "Migration checklist",
|
||||
"path": "concepts/integrated-storage/migration-checklist"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
BIN
website/public/img/diagram-vault-integrated-ra-3_az.png
Normal file
BIN
website/public/img/diagram-vault-integrated-ra-3_az.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
BIN
website/public/img/diagram-vault-ra-3-az.png
Executable file
BIN
website/public/img/diagram-vault-ra-3-az.png
Executable file
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
Reference in New Issue
Block a user