Upgrade guidance updates from VLT-172 (#13327)

* Upgrade guidance updates from VLT-172

Trying to clarify some upgrade questions. Learn update to follow in
separate PR.

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

website/content/api-docs/system/replication/replication-dr.mdx

@@ -340,6 +340,12 @@ docs](#generate-disaster-recovery-operation-token) for more information.
 !> Only one performance primary should be active at a given time. Multiple primaries may
 result in data loss!
 
+~> It is not safe to replicate from a newer version of Vault to an older version.
+When upgrading replicated clusters, ensure that upstream clusters are always
+on older versions of Vault than downstream clusters. See
+[Upgrading Vault](/docs/upgrading#replication-installations) for an example.
+
+
 | Method | Path                                    |
 | :----- | :-------------------------------------- |
 | `POST` | `/sys/replication/dr/secondary/promote` |

website/content/docs/internals/replication.mdx

@@ -144,6 +144,12 @@ Vault Replication for operators.
 
 # Caveats
 
+~> **Mismatched Cluster Versions**: It is not safe to replicate from a newer
+version of Vault to an older version. When upgrading replicated clusters,
+ensure that upstream clusters are always on older version of Vault than
+downstream clusters. See
+[Upgrading Vault](/docs/upgrading#replication-installations) for an example.
+
 - **Read-After-Write Consistency**: All write requests are forwarded from
   secondaries to the primary cluster in order to avoid potential conflicts.
   While replication is near real-time, it is not instantaneous, meaning there

website/content/docs/upgrading/index.mdx

@@ -20,11 +20,25 @@ structure that make the data incompatible with a downgrade. If you need to roll
 back to a previous version of Vault, you should roll back your data store as
 well.
 
+Vault upgrades are designed such that large jumps (ie 1.3.10 -> 1.7.x) are
+supported. The upgrade notes for each intervening version must be reviewed. The
+upgrade notes may describe additional steps or configuration to update before,
+during, or after the upgrade.
+
 ## Learn
 
 Refer to the [Vault Upgrade Standard Procedure](https://learn.hashicorp.com/tutorials/vault/sop-upgrade)
 guide for step-by-step examples of upgrading Vault Enterprise.
 
+## Agent
+
+The Vault Agent is an API client of the Vault Server. Vault APIs are almost
+always backwards compatible. When they are not, this is called out in the
+upgrade guide for the new Vault version, and there is a lengthy deprecation
+period. The Vault Agent version can lag behind the Vault Server version, though
+we recommend keeping all Vault instances up to date with the most recent minor Vault version
+to the extent possible.
+
 ## Testing the Upgrade
 
 It's always a good idea to try to ensure that the upgrade will be successful in
@@ -115,8 +129,11 @@ Upgrading installations of Vault which participate in [Enterprise Replication](/
 - When satisfied with functionality of upgraded secondary instances, upgrade
   the primary instance
 
-Upgrades of multiple replicated clusters can be complicated. Here is an
+~> **Note:** It is not safe to replicate from a newer version of Vault to an
-example.
+older version. When upgrading replicated clusters, ensure that upstream clusters
+are always on older versions of Vault than downstream clusters.
+
+Here is an example of upgrading four Vault replicated Vault clusters:
 
 ![Upgrading multiple replicated clusters](/img/vault-replication-upgrade.png)