mirror of
				https://github.com/optim-enterprises-bv/vault.git
				synced 2025-11-01 02:57:59 +00:00 
			
		
		
		
	Docs: add Integrated Storage migration checklist (#28995)
- Adds a new migration checklist from archived tutorial content
This commit is contained in:
		| @@ -0,0 +1,180 @@ | |||||||
|  | --- | ||||||
|  | layout: docs | ||||||
|  | page_title: Migration checklist | ||||||
|  | description: Use this checklist for decision making related to migrating your Vault deployment to Integrated Storage. | ||||||
|  | --- | ||||||
|  |  | ||||||
|  | # Migration checklist | ||||||
|  |  | ||||||
|  | <Tip title="This is a decision-making checklist"> | ||||||
|  |  | ||||||
|  | The purpose of this checklist is not to walk you through the storage | ||||||
|  | migration steps. This content provides a quick self-check whether it is your | ||||||
|  | best interest to migrate your Vault storage from an external system to | ||||||
|  | Integrated Storage. | ||||||
|  |  | ||||||
|  | </Tip> | ||||||
|  |  | ||||||
|  | ## Who should use this checklist? | ||||||
|  |  | ||||||
|  | Integrated Storage is a recommended storage option, made available in | ||||||
|  | Vault 1.4. Vault continues to also support other storage solutions | ||||||
|  | like Consul. | ||||||
|  |  | ||||||
|  | You should use this checklist if you are operating a Vault deployment backed | ||||||
|  | by external storage like Consul, and you are considering migration to | ||||||
|  | Integrated Storage. | ||||||
|  |  | ||||||
|  | ## Understand architectural differences | ||||||
|  |  | ||||||
|  | It is important that you understand the differences between operating Vault | ||||||
|  | with external storage and operating with Integrated Storage. The following | ||||||
|  | sections detail key differences in architecture between Vault with Consul | ||||||
|  | storage, and Vault with Integrated Storage to help inform your decision. | ||||||
|  |  | ||||||
|  | ### Reference architecture with Consul | ||||||
|  |  | ||||||
|  | The recommended number of Vault instances is **3** in a cluster which connects | ||||||
|  | to a Consul cluster which may have **5** or more nodes as shown in the diagram. | ||||||
|  |  | ||||||
|  | A total of 8 virtual machines hosts this Vault highly available architecture. | ||||||
|  |  | ||||||
|  | <ImageConfig hideBorder> | ||||||
|  |  | ||||||
|  |  | ||||||
|  |  | ||||||
|  | </ImageConfig> | ||||||
|  |  | ||||||
|  | The processing requirements depend on the encryption and messaging workloads. | ||||||
|  | Memory requirements are dependant on the total size of secrets stored in | ||||||
|  | memory. The Vault server itself has minimal storage requirements, but  | ||||||
|  | the Consul nodes should have a high-performance physical storage system. | ||||||
|  |  | ||||||
|  | ### Reference architecture with Integrated Storage | ||||||
|  |  | ||||||
|  | The recommended number of Vault instances is **5** in a cluster. In a single HA | ||||||
|  | cluster, all Vault nodes share the data while an active node holds the lock; | ||||||
|  | therefore, only the active node has write access. To achieve n-2 redundancy, | ||||||
|  | (meaning that the cluster can still function after losing 2 nodes), | ||||||
|  | an ideal size for a Vault HA cluster is 5 nodes. | ||||||
|  |  | ||||||
|  | <Tip title="More deployment details in the documentation"> | ||||||
|  |  | ||||||
|  | Refer to the [Integrated | ||||||
|  | Storage](/vault/docs/internals/integrated-storage#deployment-table) | ||||||
|  | documentation for more deployment details. | ||||||
|  |  | ||||||
|  | </Tip> | ||||||
|  |  | ||||||
|  | <ImageConfig hideBorder> | ||||||
|  |  | ||||||
|  |  | ||||||
|  |  | ||||||
|  | </ImageConfig> | ||||||
|  |  | ||||||
|  | Because the data gets persisted on the same host, the Vault server should be | ||||||
|  | hosted on a relatively high-performance hard disk system. | ||||||
|  |  | ||||||
|  | ## Consul vs. Integrated Storage | ||||||
|  |  | ||||||
|  | The Integrated Storage eliminates the need for external storage; therefore, | ||||||
|  | Vault is the only software you need to stand up a cluster. This indicates that | ||||||
|  | the host machine must have disk capacity in an amount equal or | ||||||
|  | greater to that of the existing external storage backend. | ||||||
|  |  | ||||||
|  | ### System requirements comparison | ||||||
|  |  | ||||||
|  | The fundamental difference between Vault's Integrated Storage and Consul is | ||||||
|  | that the Integrated Storage stores everything on disk while [Consul | ||||||
|  | KV](/consul/docs/dynamic-app-config/kv) stores everything in its memory | ||||||
|  | which impacts the host's RAM. | ||||||
|  |  | ||||||
|  | #### Machine sizes for Vault - Consul as its storage backend | ||||||
|  |  | ||||||
|  | It is recommended to avoid hosting Consul on an instance with burstable CPU. | ||||||
|  |  | ||||||
|  | | Size  | CPU      | Memory       | Disk  | Typical Cloud Instance Types              | | ||||||
|  | | ----- | -------- | ------------ | ----- | ----------------------------------------- | | ||||||
|  | | Small | 2 core   | 4-8 GB RAM   | 25 GB | **AWS:** m5.large                         | | ||||||
|  | |       |          |              |       | **Azure:** Standard_D2_v3                 | | ||||||
|  | |       |          |              |       | **GCE:** n1-standard-2, n1-standard-4     | | ||||||
|  | | Large | 4-8 core | 16-32 GB RAM | 50 GB | **AWS:** m5.xlarge, m5.2xlarge            | | ||||||
|  | |       |          |              |       | **Azure:** Standard_D4_v3, Standard_D8_v3 | | ||||||
|  | |       |          |              |       | **GCE:** n1-standard-8, n1-standard-16    | | ||||||
|  |  | ||||||
|  | #### Machine sizes for Vault with Integrated Storage | ||||||
|  |  | ||||||
|  | | Size  | CPU      | Memory       | Disk   | Typical Cloud Instance Types               | | ||||||
|  | | ----- | -------- | ------------ | ------ | ------------------------------------------ | | ||||||
|  | | Small | 2 core   | 8-16 GB RAM  | 100 GB | **AWS:** m5.large, m5.xlarge               | | ||||||
|  | |       |          |              |        | **Azure:** Standard_D2_v3, Standard_D4_v3  | | ||||||
|  | |       |          |              |        | **GCE:** n2-standard-2, n2-standard-4      | | ||||||
|  | | Large | 4-8 core | 32-64 GB RAM | 200 GB | **AWS:** m5.2xlarge, m5.4xlarge            | | ||||||
|  | |       |          |              |        | **Azure:** Standard_D8_v3, Standard_D16_v3 | | ||||||
|  | |       |          |              |        | **GCE:** n2-standard-8, n2-standard-16     | | ||||||
|  |  | ||||||
|  | If many secrets are being generated or rotated frequently, this information will | ||||||
|  | need to be flushed to the disk often. Therefore, the infrastructure should have | ||||||
|  | a relatively high-performance hard disk system when using the integrated | ||||||
|  | storage. | ||||||
|  |  | ||||||
|  | <Note title="A note about the importance of IOPS"> | ||||||
|  |  | ||||||
|  |  Vault's Integrated Storage is disk-bound; therefore, care should be taken when planning storage volume size and performance. For cloud providers, IOPS can be dependent on volume size and/or provisioned IOPS. It is recommended to provision IOPS and avoid burstable IOPS. Monitoring of IOPS performance should be implemented in order to tune the storage volume to the IOPS load. | ||||||
|  |  | ||||||
|  | </Note> | ||||||
|  |  | ||||||
|  | ### Performance considerations | ||||||
|  |  | ||||||
|  | Because Consul KV is memory-bound, it is necessary to take a snapshot frequently. | ||||||
|  | However, Vault's Integrated Storage persists everything on the disk which eliminates | ||||||
|  | the need for such frequent snapshot operations. Take snapshots to back up the data | ||||||
|  | so that you can restore them in case of data loss. This reduces the performance cost | ||||||
|  | introduced by the frequent snapshot operations. | ||||||
|  |  | ||||||
|  | In considering disk performance, since Vault data changes are immediately written to disk, | ||||||
|  | rather than in batched snapshots as Consul does, it is important to monitor IOPS as well | ||||||
|  | as disk queues to limit storage bottlenecks. | ||||||
|  |  | ||||||
|  |  | ||||||
|  | ### Inspect Vault data | ||||||
|  |  | ||||||
|  | Inspection of Vault data differs considerably from the `consul kv` commands used | ||||||
|  | to inspect Consul's KV store. | ||||||
|  | Consult the [Inspect Data in Integrated Storage](/vault/tutorials/monitoring/inspect-data-integrated-storage) | ||||||
|  | tutorial to learn more about querying Integrated Storage data. | ||||||
|  |  | ||||||
|  | ### Summary | ||||||
|  |  | ||||||
|  | The table below highlights the differences between Consul and integrated | ||||||
|  | storage. | ||||||
|  |  | ||||||
|  | | Consideration       | Consul as storage backend                                                  | Vault Integrated Storage                                                                     | | ||||||
|  | | ------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | ||||||
|  | | System requirement  | Memory optimized machine                                                   | Storage optimized high IOPS machine                                                          | | ||||||
|  | | Data snapshot       | Frequent snapshots                                                         | Normal data backup strategy                                                                  | | ||||||
|  | | Snapshot automation | Snapshot agent (**Consul Enterprise only**)                                | Automatic snapshot (**Vault Enterprise v1.6.0 and later**)                                   | | ||||||
|  | | Data inspection     | [Online, use `consul kv` command](/vault/tutorials/monitoring/inspecting-data-consul) | [Offline, requires using recovery mode](/vault/tutorials/monitoring/inspect-data-integrated-storage) | | ||||||
|  | | Autopilot           | Supported                                                                  | Supported (**Vault 1.7.0 and later**)                                  | | ||||||
|  |  | ||||||
|  | ## Self-check questions | ||||||
|  |  | ||||||
|  | - [ ] Where is the product expertise? | ||||||
|  |   - [ ] Do you already have Consul expertise? | ||||||
|  |   - [ ] Are you concerned about lack of Consul knowledge? | ||||||
|  | - [ ] Do you experience any technical issues with Consul? | ||||||
|  | - [ ] What motivates the data migration from the current storage to Integrated Storage? | ||||||
|  |   - [ ] Reduce the operational overhead? | ||||||
|  |   - [ ] Reduce the number of machines to run? | ||||||
|  |   - [ ] Reduce the cloud infrastructure cost? | ||||||
|  | - [ ] Do you have a staging environment where you can run production loads and verify that everything works as you expect? | ||||||
|  | - [ ] Have you thought through the storage backup process or workflow after migrating to the Integrated Storage? | ||||||
|  | - [ ] Do you currently rely heavily on using Consul to inspect Vault data? | ||||||
|  |  | ||||||
|  | ## Tutorials | ||||||
|  |  | ||||||
|  | If you are ready to migrate the current storage backend to Integrated Storage, | ||||||
|  | refer to the [Storage Migration Tutorial - Consul to Integrated Storage](/vault/tutorials/raft/raft-migration). | ||||||
|  |  | ||||||
|  | To deploy a new cluster with Integrated Storage, refer to the [Vault HA Cluster | ||||||
|  | with Integrated Storage](/vault/tutorials/raft/raft-storage) tutorial. | ||||||
| @@ -251,6 +251,10 @@ | |||||||
|           { |           { | ||||||
|             "title": "Autopilot", |             "title": "Autopilot", | ||||||
|             "path": "concepts/integrated-storage/autopilot" |             "path": "concepts/integrated-storage/autopilot" | ||||||
|  |           }, | ||||||
|  |           { | ||||||
|  |             "title": "Migration checklist", | ||||||
|  |             "path": "concepts/integrated-storage/migration-checklist" | ||||||
|           } |           } | ||||||
|         ] |         ] | ||||||
|       }, |       }, | ||||||
|   | |||||||
							
								
								
									
										
											BIN
										
									
								
								website/public/img/diagram-vault-integrated-ra-3_az.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										
											BIN
										
									
								
								website/public/img/diagram-vault-integrated-ra-3_az.png
									
									
									
									
									
										Normal file
									
								
							
										
											Binary file not shown.
										
									
								
							| After Width: | Height: | Size: 22 KiB | 
							
								
								
									
										
											BIN
										
									
								
								website/public/img/diagram-vault-ra-3-az.png
									
									
									
									
									
										Executable file
									
								
							
							
						
						
									
										
											BIN
										
									
								
								website/public/img/diagram-vault-ra-3-az.png
									
									
									
									
									
										Executable file
									
								
							
										
											Binary file not shown.
										
									
								
							| After Width: | Height: | Size: 39 KiB | 
		Reference in New Issue
	
	Block a user
	 Brian Shumate
					Brian Shumate