* Support reloading database plugins across multiple mounts
* Add clarifying comment to MountEntry.Path field
* Tests: Replace non-parallelisable t.Setenv with plugin env settings
* VAULT-22481: Audit filter node (#24465)
* Initial commit on adding filter nodes for audit
* tests for audit filter
* test: longer filter - more conditions
* copywrite headers
* Check interface for the right type
* Add audit filtering feature (#24554)
* Support filter nodes in backend factories and add some tests
* More tests and cleanup
* Attempt to move control of registration for nodes and pipelines to the audit broker (#24505)
* invert control of the pipelines/nodes to the audit broker vs. within each backend
* update noop audit test code to implement the pipeliner interface
* noop mount path has trailing slash
* attempting to make NoopAudit more friendly
* NoopAudit uses known salt
* Refactor audit.ProcessManual to support filter nodes
* HasFiltering
* rename the pipeliner
* use exported AuditEvent in Filter
* Add tests for registering and deregistering backends on the audit broker
* Add missing licence header to one file, fix a typo in two tests
---------
Co-authored-by: Peter Wilson <peter.wilson@hashicorp.com>
* Add changelog file
* update bexpr datum to use a strong type
* go docs updates
* test path
* PR review comments
* handle scenarios/outcomes from broker.send
* don't need to re-check the complete sinks
* add extra check to deregister to ensure that re-registering non-filtered device sets sink threshold
* Ensure that the multierror is appended before attempting to return it
---------
Co-authored-by: Peter Wilson <peter.wilson@hashicorp.com>
* update node and pipeline registration to prevent overwriting, strip some unused bits of NewTestCluster, tweak to prevent auditing on a test that is flaking
* tidy imports
* PKI refactoring to start breaking apart monolith into sub-packages
- This was broken down by commit within enterprise for ease of review
but would be too difficult to bring back individual commits back
to the CE repository. (they would be squashed anyways)
- This change was created by exporting a patch of the enterprise PR
and applying it to CE repository
* Fix TestBackend_OID_SANs to not be rely on map ordering
* Refactor plugin catalog into its own package
* Fix some unnecessarily slow tests due to accidentally running multiple plugin processes
* Clean up MakeTestPluginDir helper
* Move getBackendVersion tests to plugin catalog package
* Use corehelpers.MakeTestPlugin consistently
* Fix semgrep failure: check for nil value from logical.Storage
- Noticed that our documentation was out of date, we allow 8192
bit RSA keys to be used as an argument to the various PKI
issuer/key creation APIs.
- Augument some unit tests to verify this continues to work
* allow to skip TLS check in acme http-01 challenge
* remove configurable logic, just ignore TLS
* add changelog
* Add test case
---------
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
* Address a panic export RSA public keys in transit
- When attempting to export the public key for an RSA key that
we only have a private key for, the export panics with a nil
deference.
- Add additional tests around Transit key exporting
* Add cl
- Try to avoid these build failures as our proxy does seem to have
issues around pulling images with the 'latest' tag at times.
```
acme_test.go:206:
Error Trace: /home/runner/actions-runner/_work/vault-enterprise/vault-enterprise/builtin/logical/pkiext/pkiext_binary/acme_test.go:206
/home/runner/actions-runner/_work/vault-enterprise/vault-enterprise/builtin/logical/pkiext/pkiext_binary/acme_test.go:75
Error: Received unexpected error:
container create failed: Error response from daemon: No such image: docker.mirror.hashicorp.services/curlimages/curl:latest
Test: Test_ACME/group/caddy_http_eab
Messages: could not start cURL container
```
* wip
* more pruning
* Integrate OCSP into binary paths PoC
- Simplify some of the changes to the router
- Remove the binary test PKI endpoint
- Switch OCSP to use the new binary paths backend variable
* Fix proto generation and test compilation
* Add unit test for binary request handling
---------
Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
* fix lease revocation when config token exists in one namespace but can create tokens in another
* add test
* Add similar check for admin partition
* Add admin partition test
---------
Co-authored-by: robmonte <17119716+robmonte@users.noreply.github.com>
* Forbid setting auto_rotate_period on transit managed keys
- Prevent and guard against auto-rotating managed keys as we
generate an invalid key version without the uuid field set.
- Hook in the datakey generation api into managed key encryption.
* Add cl
* auth/aws: fix panic in IAM-based login when client config doesn't exist
* add changelog
* adds known issue for 1.15.0
* fixes up known issue with workaround
* fix link
* maintain behavior of client config not needing to exist for IAM login
* update changelog
- Do not load existing ACME challenges persisted within storage on non-active nodes. This was the main culprit of the issues, secondary nodes would load existing persisted challenges trying to resolve them but writes would fail leading to the excessive logging.
- We now handle this by not starting the ACME background thread on non-active nodes, while also checking within the scheduling loop and breaking out. That will force a re-reading of the Closing channel that should have been called by the PKI plugin's Cleanup method.
- If a node is stepped down from being the active node while it is actively processing a verification, we could get into an infinite loop due to an ErrReadOnly error attempting to clean up a challenge entry
- Add a maximum number of retries for errors around attempting to decode,fetch challenge/authorization entries from disk. We use double the number of "normal" max attempts for these types of errors, than we would for normal ACME retry attempts to avoid collision issues. Note that these additional retry attempts are not persisted to disk and will restart on every node start
- Add a 1 second backoff to any disk related error to not immediately spin on disk/io errors for challenges.
* secrets/db: add rotation error path test
We add a test to verify that failed rotations can successfully recover
and that they do not occur outside of a rotation window. Additionally,
we remove registering some external plugins in getCluster() that shaves
off about 5 minutes the database package tests.
* remove dead code and add test comment
* revert to original container helper after refactor
This fixes the enterprise failure of the test
```
=== FAIL: builtin/logical/pki TestCRLIssuerRemoval (0.00s)
crl_test.go:1456:
Error Trace: /home/runner/actions-runner/_work/vault-enterprise/vault-enterprise/builtin/logical/pki/crl_test.go:1456
Error: Received unexpected error:
Global, cross-cluster revocation queue cannot be enabled when auto rebuilding is disabled as the local cluster may not have the certificate entry!
Test: TestCRLIssuerRemoval
Messages: failed enabling unified CRLs on enterprise
```
* Clean up unused CRL entries when issuer is removed
When a issuer is removed, the space utilized by its CRL was not freed,
both from the CRL config mapping issuer IDs to CRL IDs and from the
CRL storage entry. We thus implement a two step cleanup, wherein
orphaned CRL IDs are removed from the config and any remaining full
CRL entries are removed from disk.
This relates to a Consul<->Vault interop issue (#22980), wherein Consul
creates a new issuer on every leadership election, causing this config
to grow. Deleting issuers manually does not entirely solve this problem
as the config does not fully reclaim space used in this entry.
Notably, an observation that when deleting issuers, the CRL was rebuilt
on secondary clusters (due to the invalidation not caring about type of
the operation); for consistency and to clean up the unified CRLs, we
also need to run the rebuild on the active primary cluster that deleted
the issuer as well.
This approach does allow cleanup on existing impacted clusters by simply
rebuilding the CRL.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case on CRL removal
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Ignore nonces when encrypting without convergence or with convergence versions > 1
* Honor nonce use warning in non-FIPS modes
* Revert "Honor nonce use warning in non-FIPS modes"
This reverts commit 2aee3dbdc11c4e333ecb20503539c7993b24ee57.
* Add a test func that removes a nonce when not needed
* err out rather than ignore the nonce
* Alter unit test to cover, also cover convergent version 3
* More unit test work
* Fix test 14
* changelog
* tests not already in a nonce present path
* Update unit test to not assume warning when nonce provided incorrectly
* remove unused test field
* Fix auto-squash events experiments
When #22835 was merged, it was auto-squashed, so the `experiments`
import was removed, but the test still referenced it.
This removes the (now unnecessary) experiment from the test.
* Allow nonces for managed keys, because we have no way of knowing if the backing cipher/mode needs one
---------
Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
The recent additions to the transit secret engine have created two new
endpoints which both have the incorrect (and duplicate) operation ID of
just `"update"`.
Amend to unique meaningful values.
* add rotation_schedule field to db backend
* add cron schedule field
* use priority queue with scheduled rotation types
* allow marshalling of cron schedule type
* return warning on use of mutually exclusive fields
* handle mutual exclusion of rotation fields (#22306)
* handle mutual exclusion of rotation fields
* fix import
* adv ttl mgmt: add rotation_window field (#22303)
* adv ttl mgmt: add rotation_window field
* do some rotation_window validation and add unit tests
* adv ttl mgmt: Ensure initialization sets appropriate rotation schedule (#22341)
* general cleanup and refactor rotation type checks
* make NextRotationTime account for the rotation type
* add comments
* add unit tests to handle mutual exclusion (#22352)
* add unit tests to handle mutual exclusion
* revert rotation_test.go and add missing test case to path_roles_test.go
* adv ttl mgmt: add tests for init queue (#22376)
* Vault 18908/handle manual rotation (#22389)
* support manual rotation for schedule based roles
* update description and naming
* adv ttl mgmt: consider rotation window (#22448)
* consider rotation window
ensure rotations only occur within a rotation window for schedule-based
rotations
* use helper method to set priority in rotateCredential
* fix bug with priority check
* remove test for now
* add and remove comments
* add unit tests for manual rotation (#22453)
* adv ttl mgmt: add tests for rotation_window
* adv ttl mgmt: refactor window tests (#22472)
* Handle GET static-creds endpoint (#22476)
* update read static-creds endpoint to include correct resp data
* return rotation_window if set
* update
* add changelog
* add unit test for static-creds read endpoint (#22505)
* Add the ability to set seconds in cron schedule for testing purposes
* update test so we don't use global var
* update with suggestions
---------
Co-authored-by: JM Faircloth <jmfaircloth@hashicorp.com>
Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
