mirror of
https://github.com/optim-enterprises-bv/vault.git
synced 2025-11-01 19:17:58 +00:00
CE: Remove RequestLimiter config plumbing (#28592)
This PR provides some plumbing for the enterprise counterpart PR: hashicorp/vault-enterprise#6791
This commit is contained in:
Mike Palmiotto
@@ -27,8 +27,7 @@ func entGetFIPSInfoKey() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func entGetRequestLimiterStatus(coreConfig vault.CoreConfig) string {
|
||||
return ""
|
||||
func entCheckRequestLimiter(_cmd *ServerCommand, _config *server.Config) {
|
||||
}
|
||||
|
||||
func entExtendAddonHandlers(handlers *vaultHandlers) {}
|
||||
|
||||
@@ -448,6 +448,8 @@ func (c *ServerCommand) parseConfig() (*server.Config, []configutil.ConfigError,
|
||||
config.Entropy = nil
|
||||
}
|
||||
|
||||
entCheckRequestLimiter(c, config)
|
||||
|
||||
return config, configErrors, nil
|
||||
}
|
||||
|
||||
@@ -1431,12 +1433,6 @@ func (c *ServerCommand) Run(args []string) int {
|
||||
info["HCP resource ID"] = config.HCPLinkConf.Resource.ID
|
||||
}
|
||||
|
||||
requestLimiterStatus := entGetRequestLimiterStatus(coreConfig)
|
||||
if requestLimiterStatus != "" {
|
||||
infoKeys = append(infoKeys, "request limiter")
|
||||
info["request limiter"] = requestLimiterStatus
|
||||
}
|
||||
|
||||
infoKeys = append(infoKeys, "administrative namespace")
|
||||
info["administrative namespace"] = config.AdministrativeNamespacePath
|
||||
|
||||
|
||||
@@ -1,81 +0,0 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: 'Request Limiter'
|
||||
description: >-
|
||||
Vault provides an adaptive concurrency limiter to protect the Vault server
|
||||
from overload.
|
||||
---
|
||||
|
||||
# Request Limiter
|
||||
|
||||
@include 'alerts/enterprise-only.mdx'
|
||||
|
||||
<Warning title="Beta (Deprecated)">
|
||||
|
||||
The request limiter was released in Vault 1.16 as a Beta
|
||||
feature. During Beta evaluation we found an alternative approach better met
|
||||
the needs of our users. This feature will be removed from Vault in a future
|
||||
release. It is replaced with [adaptive overload protection](/vault/docs/concepts/adaptive-overload-protection).
|
||||
|
||||
</Warning>
|
||||
|
||||
This document contains conceptual information about the **Request Limiter** and
|
||||
its user-facing effects.
|
||||
|
||||
## Preventing overload
|
||||
|
||||
The Request Limiter aims to prevent overload by proactively detecting latency
|
||||
deviation from a baseline and adapting the number of allowed in-flight requests.
|
||||
|
||||
This is done in two phases at the beginning of an HTTP request:
|
||||
|
||||
1. Consult the current number of allowed in-flight requests. If the new request
|
||||
would exceed this limit, immediately reject it, indicating that the client
|
||||
should retry later.
|
||||
|
||||
2. If the request is allowed, begin a measurement of its latency, allowing the
|
||||
Request Limiter to calculate a new limit.
|
||||
|
||||
## Resource constraints
|
||||
|
||||
The Request Limiter intentionally focuses on preventing overload derived from
|
||||
resource-constrained operations on the Vault server. Vault focuses on two
|
||||
specific types of resource constraints which commonly cause issues in production
|
||||
workloads:
|
||||
|
||||
1. Write latency in the storage backend, resulting in a growing queue of updates
|
||||
to be flushed. These writes originate primarily from `Write`-based HTTP methods.
|
||||
|
||||
2. CPU utilization caused by computationally expensive PKI issue requests
|
||||
(generally for RSA keys). Large numbers of these requests can consume all CPU
|
||||
resources, preventing timely processing of other requests such as heartbeats and
|
||||
health checks.
|
||||
|
||||
Storage constraints can be accounted for by limiting logical requests according
|
||||
to their `http.Method`. We only measure and limit requests with `Write`-based
|
||||
HTTP methods. Read requests do not generally cause storage updates, meaning that
|
||||
their latencies are unlikely to be correlated with storage constraints.
|
||||
|
||||
CPU constraints are accounted for using the same underlying library and
|
||||
technique; however, they require special treatment. The maximum number of
|
||||
concurrent pki/issue requests found in testing (again, specifically for RSA
|
||||
keys) is far lower than the minimum tolerable write request rate.
|
||||
|
||||
In both cases, utilization will be effectively throttled before Vault reaches
|
||||
any degraded state. The resulting `503 - Service Unavailable` is a retryable
|
||||
HTTP response code, which can be handled to gracefully retry and eventually
|
||||
succeed. Clients should handle this by retrying with jitter and exponential
|
||||
backoff. This is done within Vault's API `Client` implementation, using the
|
||||
go-retryablehttp library.
|
||||
|
||||
## Read requests
|
||||
|
||||
HTTP methods such as `GET` and `LIST` are not subject to write request
|
||||
limiting. This allows operators to continue querying server state without
|
||||
needing to retry.
|
||||
|
||||
## Vault server overloaded
|
||||
|
||||
When Vault has reached capacity, new requests will be immediately rejected with a
|
||||
retryable `503 - Service Unavailable`
|
||||
[error](/vault/docs/concepts/adaptive-overload-protection/vault-server-temporarily-overloaded).
|
||||
@@ -224,14 +224,14 @@ can have a negative effect on performance due to the tracking of each lock attem
|
||||
This can also be specified via the `VAULT_LOG_LEVEL` environment variable.
|
||||
|
||||
<Note>
|
||||
|
||||
|
||||
On SIGHUP (`sudo kill -s HUP` _pid of vault_), if a valid value is specified, Vault will update the existing log level,
|
||||
overriding (even if specified) both the CLI flag and environment variable.
|
||||
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
|
||||
|
||||
Not all parts of Vault's logging can have its log level be changed dynamically this way; in particular,
|
||||
secrets/auth plugins are currently not updated dynamically.
|
||||
|
||||
@@ -257,9 +257,6 @@ can have a negative effect on performance due to the tracking of each lock attem
|
||||
When `imprecise_lease_role_tracking` is set to true and a new role-based quota is enabled, subsequent lease counts start from 0.
|
||||
`imprecise_lease_role_tracking` affects role-based lease count quotas, but reduces latencies when not using role based quotas.
|
||||
|
||||
- `request_limiter` `([Request Limiter][request-limiter]: <none>)` – Allows
|
||||
operators to enable Vault's Request Limiter functionality.
|
||||
|
||||
### High availability parameters
|
||||
|
||||
The following parameters are used on backends that support [high availability][high-availability].
|
||||
@@ -304,7 +301,7 @@ The following parameters are only used with Vault Enterprise
|
||||
provided via the environment variable `VAULT_LICENSE_PATH`, or the license
|
||||
itself can be provided in the environment variable `VAULT_LICENSE`.
|
||||
|
||||
- `administrative_namespace_path` `(string: "")` - Specifies the absolute path
|
||||
- `administrative_namespace_path` `(string: "")` - Specifies the absolute path
|
||||
to the Vault namespace to be used as an [Administrative namespace](/vault/docs/enterprise/namespaces/create-admin-namespace).
|
||||
|
||||
[storage-backend]: /vault/docs/configuration/storage
|
||||
@@ -315,4 +312,3 @@ The following parameters are only used with Vault Enterprise
|
||||
[sentinel]: /vault/docs/configuration/sentinel
|
||||
[high-availability]: /vault/docs/concepts/ha
|
||||
[plugins]: /vault/docs/plugins
|
||||
[request-limiter]: /vault/docs/concepts/request-limiter
|
||||
|
||||
@@ -296,10 +296,6 @@ default value in the `"/sys/config/ui"` [API endpoint](/vault/api-docs/system/co
|
||||
- `disable_replication_status_endpoints` `(bool: false)` - Disables replication
|
||||
status endpoints for the configured listener when set to `true`.
|
||||
|
||||
- `disable_request_limiter` `(bool: false)` - Disables the request limiter for
|
||||
this listener. The default configuration will honor the global
|
||||
[configuration](/vault/docs/configuration/request-limiter).
|
||||
|
||||
### `telemetry` parameters
|
||||
|
||||
- `unauthenticated_metrics_access` `(bool: false)` - If set to true, allows
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Request Limiter - Configuration
|
||||
description: |-
|
||||
The Request Limiter mitigates overload scenarios in Vault by adaptively
|
||||
limiting in-flight requests based on latency measurements.
|
||||
---
|
||||
|
||||
# `request_limiter`
|
||||
|
||||
@include 'alerts/enterprise-only.mdx'
|
||||
|
||||
<Warning title="Deprecated beta feature">
|
||||
|
||||
Vault 1.16 included the request limiter as a Beta feature. During the beta, we
|
||||
found an alternative approach that better meets user needs. The request limiter
|
||||
has been deprecated in favor of [adaptive overload
|
||||
protection](/vault/docs/concepts/adaptive-overload-protection).
|
||||
|
||||
</Warning>
|
||||
|
||||
The `request_limiter` stanza allows operators to turn on the adaptive
|
||||
concurrency limiter, which is off by default. This is a reloadable config.
|
||||
|
||||
```hcl
|
||||
request_limiter {
|
||||
disable = false
|
||||
}
|
||||
```
|
||||
|
||||
~> **Warning** This feature is still in Tech Preview. Turning the Request
|
||||
Limiter *on* may have negative effects on request success rates. Please test
|
||||
your workloads before turning this on in production.
|
||||
@@ -97,3 +97,35 @@ kubectl exec -ti <NAME> -- wget https://github.com/moparisthebest/static-curl/re
|
||||
```
|
||||
|
||||
**NOTE:** When using this option you'll want to verify that the static binary comes from a trusted source.
|
||||
|
||||
### Request limiter configuration removal
|
||||
|
||||
Vault 1.16.0 included an experimental request limiter. The limiter was disabled
|
||||
by default with an opt-in `request_limiter` configuration.
|
||||
|
||||
Further testing indicated that an alternative approach improves performance and
|
||||
reduces risk for many workloads. Vault 1.17.0 included a new [adaptive overload
|
||||
protection](/vault/docs/concepts/adaptive-overload-protection) feature that
|
||||
prevents outages when Vault is overwhelmed by write requests.
|
||||
|
||||
Adaptive overload protection was a beta feature in 1.17.0.
|
||||
|
||||
As of Vault 1.18.0, the adaptive overload protection feature for writes is
|
||||
now GA and enabled by default for the integrated storage backend.
|
||||
|
||||
The beta `request_limiter` configuration stanza is officially removed in Vault 1.18.0.
|
||||
|
||||
Vault will output two types of warnings if the `request_limiter` stanza is
|
||||
detected in your Vault config.
|
||||
|
||||
1. A UI warning message printed to `stderr`:
|
||||
|
||||
```text
|
||||
WARNING: Request Limiter configuration is no longer supported; overriding server configuration to disable
|
||||
```
|
||||
|
||||
2. A log line with level `WARN`, appearing in Vault's logs:
|
||||
|
||||
```text
|
||||
... [WARN] unknown or unsupported field request_limiter found in configuration at config.hcl:22:1
|
||||
```
|
||||
@@ -321,25 +321,6 @@
|
||||
"color": "neutral"
|
||||
}
|
||||
},
|
||||
{
|
||||
"title": "Request Limiter",
|
||||
"badge": {
|
||||
"text": "ENTERPRISE | DEPRECATED",
|
||||
"type": "outlined",
|
||||
"color": "neutral"
|
||||
},
|
||||
"routes": [
|
||||
{
|
||||
"title": "Overview",
|
||||
"path": "concepts/request-limiter",
|
||||
"badge": {
|
||||
"text": "BETA",
|
||||
"type": "outlined",
|
||||
"color": "highlight"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Adaptive overload protection",
|
||||
"badge": {
|
||||
@@ -595,10 +576,6 @@
|
||||
"title": "<code>telemetry</code>",
|
||||
"path": "configuration/telemetry"
|
||||
},
|
||||
{
|
||||
"title": "<code>Request Limiter</code>",
|
||||
"path": "configuration/request-limiter"
|
||||
},
|
||||
{
|
||||
"title": "Adaptive overload protection",
|
||||
"path": "configuration/adaptive-overload-protection"
|
||||
|
||||
Reference in New Issue
Block a user