I have an upcoming PR for event notifications that needs similar
exponential backoff logic, and I prefer the API and logic in the
auto-auth exponential backoff rather than that of
github.com/cenkalti/backoff/v3.
This does have a small behavior change: the auto-auth min backoff
will now be randomly reduced by up to 25% on the first call. This is
a desirable property to avoid thundering herd problems, where a bunch
of agents won't all try have the same retry timeout.
* VAULT-21427 change ui references from K/V to KV
* references in docs/
* website json data
* go command errors
* replace Key/Value with Key Value
* add changelog
* update test
* update secret list header badge
* two more test updates
It is not sufficient to check that function setSeal in server.go does not return
an "unwrap seal". For migrations away from a Shamir seal, NewCore constructor
sets up an unwrap seal by calling method adjustForSealMigration.
Factor out new method checkForSealMigration out of adjustForSealMigration so
that NewCore can verify that there won't be a migration when returning early due
to running in recovery mode.
* Refactor plugin catalog into its own package
* Fix some unnecessarily slow tests due to accidentally running multiple plugin processes
* Clean up MakeTestPluginDir helper
* Move getBackendVersion tests to plugin catalog package
* Use corehelpers.MakeTestPlugin consistently
* Fix semgrep failure: check for nil value from logical.Storage
With the introduction of the Seal High Availability feature, the presence of
multiple seals in configuration does not necessarily mean that the configuration
entails a seal migration.
Instead of checking for multiple seals, check for the presence on an "unwrap"
seal, which is only used for seal migrations.
@mitchellh suggested we fork `cli` and switch to that.
Since we primarily use the interfaces in `cli`, and the new
fork has not changed those, this is (mostly) a drop-in replacement.
A small fix will be necessary for Vault Enterprise, I believe.
* fix -log-file so that it uses the correct name and only adds timestamps on rotation
* added some tests for naming/rotation
* changelog
* revert to previous way of getting created time
* remove unused stat
* comment shuffle
* Update changelog/24297.txt
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* Update website/content/docs/agent-and-proxy/agent/index.mdx
Update 'agent' docs page
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* Update website/content/docs/agent-and-proxy/proxy/index.mdx
Update 'proxy' docs page
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* Update website/content/docs/commands/server.mdx
Update 'server' docs page
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* fix typos
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* reload seals on SIGHUP
* add lock in SetSeals
* move lock
* use stubmaker and change wrapper finalize call
* change finalize logic so that old seals will be finalized after new seals are configured
* add changelog
* run make fmt
* fix fmt
* fix panic when reloading seals errors out
* Fix non-JSON log messages when using -log-format JSON
Removed the call to consul-template's logging.Setup inside the created of config for the Runner. Instead we call it when we assign the logger to the Agent command.
* The elusive extra line
* Adjust the approach
* changelog
* Infer levels *with* timestamp prefix
* InferLeveslWithTimestamp required InferLevels
* Test to show -log-format and -log-file working in consul-template generated messages
* classic typo
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* Stop supporting vault plugin info and deregister without a type argument
* Make a best-effort attempt to report whether a plugin was actually deregistered and give more descriptive errors
* Fix error message for vault plugin reload
* Pulls in github.com/go-secure-stdlib/plugincontainer@v0.3.0 which exposes a new `Config.Rootless` option to opt in to extra container configuration options that allow establishing communication with a non-root plugin within a rootless container runtime.
* Adds a new "rootless" option for plugin runtimes, so Vault needs to be explicitly told whether the container runtime on the machine is rootless or not. It defaults to false as rootless installs are not the default.
* Updates `run_config.go` to use the new option when the plugin runtime is rootless.
* Adds new `-rootless` flag to `vault plugin runtime register`, and `rootless` API option to the register API.
* Adds rootless Docker installation to CI to support tests for the new functionality.
* Minor test refactor to minimise the number of test Vault cores that need to be made for the external plugin container tests.
* Documentation for the new rootless configuration and the new (reduced) set of restrictions for plugin containers.
* As well as adding rootless support, we've decided to drop explicit support for podman for now, but there's no barrier other than support burden to adding it back again in future so it will depend on demand.
* wip
* Work on the tuneable allowance and some bugs
* Call handleCancellableRequest instead, which gets the audit order more correct and includes the preauth response
* Get rid of no longer needed operation
* Phew, this wasn't necessary
* Add auth error handling by the backend, and fix a bug with handleInvalidCredentials
* Cleanup req/resp naming
* Use the new form, and data
* Discovered that tokens werent really being checked because isLoginRequest returns true for the re-request into the backend, when it shouldnt
* Add a few more checks in the delegated request handler for bad inputs
- Protect the delegated handler from bad inputs from the backend such
as an empty accessor, a path that isn't registered as a login request
- Add similar protections for bad auth results as we do in the normal
login request paths. Technically not 100% needed but if somehow the
handleCancelableRequest doesn't use the handleLoginRequest code path
we could get into trouble in the future
- Add delegated-auth-accessors flag to the secrets tune command and
api-docs
* Unit tests and some small fixes
* Remove transit preauth test, rely on unit tests
* Cleanup and add a little more commentary in tests
* Fix typos, add another failure use-case which we reference a disabled auth mount
* PR Feedback
- Use router to lookup mount instead of defining a new lookup method
- Enforce auth table types and namespace when mount is found
- Define a type alias for the handleInvalidCreds
- Fix typos/grammar
- Clean up globals in test
* Additional PR feedback
- Add test for delegated auth handler
- Force batch token usage
- Add a test to validate failures if a non-batch token is used
- Check for Data member being nil in test cases
* Update failure error message around requiring batch tokens
* Trap MFA requests
* Reword some error messages
* Add test and fixes for delegated response wrapping
* Move MFA test to dedicated mount
- If the delegated auth tests were running in parallel, the MFA test
case might influence the other tests, so move the MFA to a dedicated
mount
* PR feedback: use textproto.CanonicalMIMEHeaderKey
- Change the X-Vault-Wrap-Ttl constant to X-Vault-Wrap-TTL
and use textproto.CanonicalMIMEHeaderKey to format it
within the delete call.
- This protects the code around changes of the constant typing
* PR feedback
- Append Error to RequestDelegatedAuth
- Force error interface impl through explicit nil var assignment on
RequestDelegatedAuthError
- Clean up test factory and leverage NewTestSoloCluster
- Leverage newer maps.Clone as this is 1.16 only
---------
Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
- Instead of relying on the initial call to import to generate the
wrapping key, generate it within the test setup with a longer
dedicated timeout.
- This hopefully is enough of a timeout for the 32 bit nightly runner
* VAULT-19239 create disable static secret caching config
* VAULT-19239 missed file
* VAULT-19239 didn't finish a log line
* VAULT-19239 adjust test to use new option
* Fix typo
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
---------
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
* add vault operator inspect command
* tidy
* add tests
* add changelog
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* remove test.snapshot file and create snapshot during test
* fix help text and add custom format flag
* Update changelog/23457.txt
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* add check that kvdepth is not less than zero
* remove kvprefix from flag names
* set details flag to default to true
* remove total size, we don't use it
* fix test that referenced old flag names
* add size calculations
* return all keys if depth set to 0
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update command/operator_raft_snapshot_inspect.go
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update command/operator_raft_snapshot_inspect.go
* revert go.mod
* update comment on temp file
* update comment on closing writer
* remove temporary file, and process state.bin in memory
* return metadata like info
* remove unnused function
* remove ReadSnapshot func, and unnecessary goroutine
* remove handler, and zero values
* move defer, and return error instead of logging
* add subcommand doc
* remove -format flag
* remove comments
* change copy to copyN
* remove formatters
* remove formatters
* remove duplicate check
* fix error checks
* remove unused consts
---------
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Fix for undelete for mount paths
* Pulled in a fix from kv_delete.go that adds partialPath to
the mountPath to support paths that are multiple levels deep.
* Added undelete tests to validate that KV secrets can be successfully
undeleted when mounted at a multi-level mount path.
* Added changelog txt file
* Update changelog to specify KV impact
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
This causes the registry to now contain ent plugins on ent; previously it did not, though that appears to have been the intention. I believe this is because of the order in which inits were run.
Having changed this, various tests broke that were relying on the incorrect behaviour. Several tests were changed to rely less on opaque counts of expected plugins, instead they're now using explicit comparison by name.
* VAULT-19255 first pass at structure for event updater
* VAULT-19255 some more work, committign before rebase
* VAULT-19255 Mostly finish event updating scaffolding
* VAULT-19255 some additional coverage, clean-up, etc
* VAULT-19255 some clean-up
* VAULT-19255 fix tests
* VAULT-19255 more WIP event system integration
* VAULT-19255 More WIP
* VAULT-19255 more discovery
* VAULT-19255 add new test, some clean up
* VAULT-19255 fix bug, extra clean-up
* VAULT-19255 fix bugs, and clean up
* VAULT-19255 clean imports, add more godocs
* VAULT-19255 add config for test
* VAULT-19255 typo
* VAULT-19255 don't do the kv refactor in this PR
* VAULT-19255 update docs
* VAULT-19255 PR feedback
* VAULT-19255 More specific error messages
* redaction should only work for TCP listeners, also fix bug that allowed custom response headers for unix listeners
* fix failing test
* updates from PR feedback
* Correct the post-unseal meaning of the seal status type
And at the same time add a RecoverySealType to the response which preserves the old meaning.
Updated the CLI to display both when relevant.
* changelog
* no longer needed
* Don't need this field either, which fixes unit tests
* fix unit tests
* implement user lockout logger
* formatting
* make user lockout log interval configurable
* create func to get locked user count, and fix potential deadlock
* fix test
* fix test
* add changelog
* VAULT-19237 Add mount_type to secret response
* VAULT-19237 changelog
* VAULT-19237 make MountType generic
* VAULT-19237 clean up comment
* VAULT-19237 update changelog
* VAULT-19237 update test, remove mounttype from wrapped responses
* VAULT-19237 fix a lot of tests
* VAULT-19237 standby test
* ensure -log-level is added to core config (#23017)
* Feature/document tls servername (#22714)
* Add Raft TLS Helm examples
Co-authored-by: Pascal Reeb <pascal.reeb@adfinis.com>
---------
* Clean up unused CRL entries when issuer is removed (#23007)
* Clean up unused CRL entries when issuer is removed
When a issuer is removed, the space utilized by its CRL was not freed,
both from the CRL config mapping issuer IDs to CRL IDs and from the
CRL storage entry. We thus implement a two step cleanup, wherein
orphaned CRL IDs are removed from the config and any remaining full
CRL entries are removed from disk.
This relates to a Consul<->Vault interop issue (#22980), wherein Consul
creates a new issuer on every leadership election, causing this config
to grow. Deleting issuers manually does not entirely solve this problem
as the config does not fully reclaim space used in this entry.
Notably, an observation that when deleting issuers, the CRL was rebuilt
on secondary clusters (due to the invalidation not caring about type of
the operation); for consistency and to clean up the unified CRLs, we
also need to run the rebuild on the active primary cluster that deleted
the issuer as well.
This approach does allow cleanup on existing impacted clusters by simply
rebuilding the CRL.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case on CRL removal
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* UI: Handle control group error on SSH (#23025)
* Handle control group error on SSH
* Add changelog
* Fix enterprise failure of TestCRLIssuerRemoval (#23038)
This fixes the enterprise failure of the test
```
=== FAIL: builtin/logical/pki TestCRLIssuerRemoval (0.00s)
crl_test.go:1456:
Error Trace: /home/runner/actions-runner/_work/vault-enterprise/vault-enterprise/builtin/logical/pki/crl_test.go:1456
Error: Received unexpected error:
Global, cross-cluster revocation queue cannot be enabled when auto rebuilding is disabled as the local cluster may not have the certificate entry!
Test: TestCRLIssuerRemoval
Messages: failed enabling unified CRLs on enterprise
```
* fix LDAP auto auth changelog (#23027)
* VAULT-19233 First part of caching static secrets work
* VAULT-19233 update godoc
* VAULT-19233 invalidate cache on non-GET
* VAULT-19233 add locking to proxy cache writes
* VAULT-19233 add caching of capabilities map, and some additional test coverage
* VAULT-19233 Additional testing
* VAULT-19233 namespaces for cache ids
* VAULT-19233 cache-clear testing and implementation
* VAULT-19233 adjust format, add more tests
* VAULT-19233 some more docs
* VAULT-19233 Add RLock holding for map access
* VAULT-19233 PR comments
* VAULT-19233 Different table for capabilities indexes
* VAULT-19233 keep unique for request path
* VAULT-19233 passthrough for non-v1 requests
* VAULT-19233 some renames/PR comment updates
* VAULT-19233 remove type from capabilities index
* VAULT-19233 remove obsolete capabilities
* VAULT-19233 remove erroneous capabilities
* VAULT-19233 woops, missed a test
* VAULT-19233 typo
* VAULT-19233 add custom error for cachememdb
* VAULT-19233 fix cachememdb test
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
Co-authored-by: Andreas Gruhler <andreas.gruhler@adfinis.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Co-authored-by: Chelsea Shaw <82459713+hashishaw@users.noreply.github.com>
* add redaction config settings to listener
* sys seal redaction + test modification for default handler properties
* build date should be redacted by 'redact_version' too
* sys-health redaction + test fiddling
* sys-leader redaction
* added changelog
* Lots of places need ListenerConfig
* Renamed options to something more specific for now
* tests for listener config options
* changelog updated
* updates based on PR comments
* updates based on PR comments - removed unrequired test case field
* fixes for docker tests and potentially server dev mode related flags
