Commit Graph

1748 Commits

Author SHA1 Message Date
Alexander Scheel
cfc6b43607 Don't include issuers on delta CRLs (#17463)
When revoking an issuer, we immediately force a full rebuild of all CRLs
(complete and delta). However, we had forgotten to guard the delta CRL's
inclusion of augmented issuers, resulting in double-listing the issuer's
serial number on both the complete and the delta CRL. This isn't
necessary as the delta's referenced complete CRL number has incremented
to the point where the issuer itself was included on the complete CRL.

Avoid this double reference and don't include issuers on delta CRLs;
they should always appear only on the complete CRL.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-07 12:36:22 -04:00
Steven Clark
e9914734e1 Unit tests that validate OCSP signatures leverage revocation signature algo (#17452)
- Add some unit tests around the OCSP response validation that we
   are using the proper signature algorithms.
 - Add in test cases as well to validate SHA384 and SHA512 requested hash support
2022-10-07 12:33:17 -04:00
Ruben De Visscher
4a2e014087 Fix for duplicate SANs in signed certificates (#16700)
* Fix for duplicate SANs in signed certificates when othernames are present in the CSR SAN extension and UseCSRValues is true.

When UseCSRValues is true (as is the case on the sign-verbatim endpoint), all extensions including Subject Alternative Names are copied from the CSR to the final certificate.
If the Subject Alternative Name in question contains any othernames (such as a Microsoft UPN) the SAN extension is added again as a workaround for an encoding issue (in function HandleOtherSANs).
Having duplicate x509v3 extensions is invalid and is rejected by openssl on Ubuntu 20.04, and also by Go since https://github.com/golang/go/issues/50988 (including in Go 1.19).

In this fix I do not add the extension from the CSR if it will be added during HandleOtherSANs.

* Added unittest and changelog entry.
2022-10-07 12:19:08 -04:00
Alexander Scheel
32f7e8ba28 Fix RevocationSigAlg provisioning in GCP (#17449)
* Fix RevocationSigAlg provisioning in GCP

GCP restricts keys to a certain type of signature, including hash
algorithm, so we must provision our RevocationSigAlg from the root
itself unconditionally in order for GCP to work.

This does change the default, but only for newly created certificates.

Additionally, we clarify that CRL building is not fatal to the import
process.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add inverse mapping for SignatureAlgorithm

By default we'd use .String() on x509.SignatureAlgorithm, but this
doesn't round-trip. Switch to a custom map that is round-trippable
and matches the constant name as there is no other way to get this info
presently.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test to ensure root creation sets rev_sig_alg

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Test round-tripping of SigAlgoNames, InvSigAlgoNames

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix failing Default Update test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-06 17:50:49 -04:00
Alexander Scheel
6f1bc04624 Add warning on missing entity information (#17428)
When requesting a SSH certificate with default_extension templating
enabled, if the request lacks entity information and a particular
extension requires templating, just these extensions will be elided.
Other extensions (if present) will still be on the final certificate.

Add a warning in the event of missing entity information and at least
one extension that was skipped as a result.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-06 14:00:56 -04:00
Alexander Scheel
95860c6a69 Fix RevocationSigAlgo support in OCSP (#17436)
* Allow OCSP to use issuer's RevocationSigAlgo

When an issuer specifies a RevocationSigAlgo, we should largely follow
this for both CRLs and OCSP. However, x/crypto/ocsp lacks support for
PSS signatures, so we drop these down to PKCS#1v1.5 instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning when issuer has PSS-based RevSigAlgo

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add note about OCSP and PSS support

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-06 12:01:12 -04:00
Steven Clark
71fa60481f PKI: Add support for signature_bits param to the intermediate/generate api (#17388)
* PKI: Add support for signature_bits param to the intermediate/generate api

 - Mainly to work properly with GCP backed managed keys, we need to
   issue signatures that would match the GCP key algorithm.
 - At this time due to https://github.com/golang/go/issues/45990 we
   can't issue PSS signed CSRs, as the libraries in Go always request
   a PKCS1v15.
 - Add an extra check in intermediate/generate that validates the CSR's
   signature before providing it back to the client in case we generated
   a bad signature such as if an end-user used a GCP backed managed key
   with a RSA PSS algorithm.
   - GCP ignores the requested signature type and always signs with the
     key's algorithm which can lead to a CSR that says it is signed with
     a PKCS1v15 algorithm but is actually a RSA PSS signature

* Add cl

* PR feedback
2022-10-03 12:39:54 -04:00
Steven Clark
aea2844c5f PKI: Do not load revoked certificates if CRL has been disabled (#17385)
* PKI: Do not load revoked certificates if CRL has been disabled

 - Restore the prior behavior of not reading in all revoked certificates
   if the CRL has been disabled as there might be performance issues
   if a customer had or is still revoking a lot of certificates.

* Add cl
2022-10-03 10:04:32 -04:00
Alexander Scheel
c2d427e7d2 Remove delta indicator on main CRL (#17334)
When adding delta CRL support, we unconditionally added the delta
indicator extension to the main CRL. We shouldn't have done this, and
instead only added it conditionally when we were building delta CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-27 17:44:38 -04:00
Christopher Swenson
0b34b73c47 Check if plugin version matches running version (#17182)
Check if plugin version matches running version

When registering a plugin, we check if the request version matches the
self-reported version from the plugin. If these do not match, we log a
warning.

This uncovered a few missing pieces for getting the database version
code fully working.

We added an environment variable that helps us unit test the running
version behavior as well, but only for approle, postgresql, and consul
plugins.

Return 400 on plugin not found or version mismatch

Populate the running SHA256 of plugins in the mount and auth tables (#17217)
2022-09-21 12:25:04 -07:00
Kit Haines
410ecba6af Fix non-atomic read of atomic value fix (#17255)
* Always load to access certCount

* Test-reads of the atomic value.
2022-09-21 11:24:34 -04:00
Alexander Scheel
b2d6fc0a92 Prevent PSS with Go-incompatible CAs, CSRs, Private Keys (#17223)
* Fix interoperability concerns with PSS

When Go parses a certificate with rsaPSS OID, it will accept this
certificate but not parse the SubjectPublicKeyInfo, leaving the
PublicKeyAlgorithm and PublicKey fields blank, but otherwise not erring.
The same behavior occurs with rsaPSS OID CSRs.

On the other hand, when Go parses rsaPSS OID PKCS8 private keys, these
keys will fail to parse completely.

Thus, detect and fail on any empty PublicKey certs and CSRs, warning the
user that we cannot parse these correctly and thus refuse to operate.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Run more PKI tests in parallel

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add notes about PSS shortcomings to considerations

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-20 17:30:58 -04:00
Steven Clark
60e65ae710 Fix fmt error (#17241) 2022-09-20 13:33:01 -07:00
Kit Haines
06097d8c83 Basics of Cert-Count Non-Locking Telemetry (#16676)
Basics of Cert-Count Telemetry, changelog,  "best attempt" slice to capture (and test for) duplicates, Move sorting of possibleDoubleCountedRevokedSerials to after compare of entries. Add values to counter when still initializing.
Set lists to nil after use, Fix atomic2 import, Delay reporting metrics until after deduplication has completed, 
The test works now, Move string slice to helper function; Add backendUUID to gauge name.
2022-09-20 10:32:20 -07:00
vinay-gopalan
011b3d1136 Re-initialize v5 backend after a plugin crash (#17140) 2022-09-19 16:48:45 -07:00
Alexander Scheel
9f3955bdbd Don't race for CRL rebuilding capability check (#17185)
* Don't race for CRL rebuilding capability check

Core has recently seen some data races during SystemView/replication
updates between them and the PKI subsystem. This is because this
SystemView access occurs outside of a request (during invalidation
handling) and thus the proper lock isn't held.

Because replication status cannot change within the lifetime of a plugin
(and instead, if a node switches replication status, the entire plugin
instance will be torn down and recreated), it is safe to cache this
once, at plugin startup, and use it throughout its lifetime.

Thus, we replace this SystemView access with a stored boolean variable
computed ahead of time.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update builtin/logical/pki/backend.go

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-19 11:41:32 -04:00
Christopher Swenson
70278c2787 Add plugin version to GRPC interface (#17088)
Add plugin version to GRPC interface

Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.

I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:

If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.

* Placeholder backend should be external

We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.

So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
2022-09-15 16:37:59 -07:00
Kit Haines
6c9db68844 Add "plumbing" for surfacing warnings, and warning overwriting ttl (#17073)
* Add "plumbing" for surfacing warnings, and add warning about TTL > maxTTL when issuing a cert.
2022-09-15 12:38:33 -07:00
Steven Clark
e61aa5df91 make fmt (#17131) 2022-09-14 07:45:50 -05:00
Scott Miller
5d8791631c Implement partial_failure_response_code_override for batch requests (#17118)
* Implement partial_failure_response_code_override for batch requests

* docs

* changelog

* one more test case
2022-09-13 12:51:09 -05:00
Steven Clark
71cb0a2ae5 A PKI test to verify our defaults are the same for creates and update apis (#17094) 2022-09-12 09:22:56 -04:00
Alexander Scheel
43443db98d Update issuer usage with ocsp-signing by default (#17087)
This option was elided from the default value for the usage field. This
results in issuers "losing" ocsp-signing when they're POST updated. Most
issuers will want OCSP signing by default, so it makes sense to add this
as the default.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-09 13:28:45 -04:00
Tom Proctor
688469b66d Support version selection for database plugins (#16982)
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
2022-09-09 17:32:28 +01:00
Josh Black
03d2be4cb9 update gofumpt to 0.3.1 and reformat the repo (#17055)
* update gofumpt to 0.3.1 and reformat the repo

* output the version of the formatter we're using
2022-09-07 17:31:20 -07:00
Matt Schultz
00eaf0727f Account for a possible error in Transit BYOK. (#17025) 2022-09-06 11:03:24 -07:00
Steven Clark
41f78c66d8 Address failures in FIPS builds around new transit RSA PSS tests (#17024)
- When we added new tests that validate the RSA PSS feature, they
   work properly on normal Go builds, but tests underneath the Boring
   Crypto fips implementations fail due to a lack of SHA3 support in
   FIPS 140-2.
2022-09-06 13:40:13 -04:00
Scott Miller
d6a1ce2e7b Add support for a dedicated HMAC type in Transit. (#16668)
* Get import correct

* limits, docs

* changelog

* unit tests

* And fix import for hmac unit test

* typo

* Update website/content/api-docs/secret/transit.mdx

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

* Update builtin/logical/transit/path_keys.go

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

* Validate key sizes a bit more carefully

* Update sdk/helper/keysutil/policy.go

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
2022-09-06 10:17:58 -05:00
Steven Clark
9cbd80b51e Fix various trivial warnings from staticcheck in the PKI plugin (#16946)
* Fix up simple warnings in production code

* Address warnings from static check in the PKI test classes
2022-08-31 16:25:14 -04:00
Alexander Scheel
3fd7e53dd5 Add ability to request manual rebuild of Delta CRLs (#16964)
* Add path to manually rebuild delta CRLs

The crl/rotate-delta path behaves like crl/rotate, triggering a
cluster-local rebuild of just the delta CRL. This is useful for when
delta CRLs are enabled with a longer-than-desired auto-rebuild period
after some high-profile revocations occur.

In the event delta CRLs are not enabled, this becomes a no-op.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for Delta CRL rebuilding

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update documentation about Delta CRLs

Also fixes a omission in the If-Modified-Since docs to mention that the
response header should probably also be passed through.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-31 12:42:59 -07:00
Alexander Scheel
76d89fd45f Add ability to cancel PKI tidy operations, pause between tidying certs (#16958)
* Allow tidy operations to be cancelled

When tidy operations take a long time to execute (and especially when
executing them automatically), having the ability to cancel them becomes
useful to reduce strain on Vault clusters (and let them be rescheduled
at a later time).

To this end, we add the /tidy-cancel write endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing auto-tidy synopsis / description

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add a pause duration between tidying certificates

By setting pause_duration, operators can have a little control over the
resource utilization of a tidy operation. While the list of certificates
remain in memory throughout the entire operation, a pause is added
between processing certificates and the revocation lock is released.
This allows other operations to occur during this gap and potentially
allows the tidy operation to consume less resources per unit of time
(due to the sleep -- though obviously consumes the same resources over
the time of the operation).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for cancellation, pause

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add API docs on pause_duration, /tidy-cancel

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add lock releasing around tidy pause

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Reset cancel guard, return errors

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-31 11:36:12 -07:00
Trishank Karthik Kuppusamy
754c11995a Allow configuring the possible salt lengths for RSA PSS signatures (#16549)
* accommodate salt lengths for RSA PSS

* address feedback

* generalise salt length to an int

* fix error reporting

* Revert "fix error reporting"

This reverts commit 8adfc15fe3303b8fdf9f094ea246945ab1364077.

* fix a faulty check

* check for min/max salt lengths

* stringly-typed HTTP param

* unit tests for sign/verify HTTP requests

also, add marshaling for both SDK and HTTP requests

* randomly sample valid salt length

* add changelog

* add documentation
2022-08-31 12:27:03 -04:00
Steven Clark
0636467e5d Add remove_roots_from_chain to sign and issue pki apis (#16935)
* Add remove_roots_from_chain flag to sign and issue pki apis

 - Add a new flag to allow end-users to control if we return the
   root/self-signed CA certificate within the list of certificates in
   ca_chain field on issue and sign api calls.

* Add cl

* PR feedback
2022-08-31 09:51:26 -04:00
Alexander Scheel
a36f4a0a03 Fix comment of default value of tidy interval (#16948)
We switch these fields to use the explicit default value (computing the
time in seconds appropriately).

As reported by @beornf, thanks!

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-31 09:23:17 -04:00
Alexander Scheel
f0c318e4e7 Add ability to perform automatic tidy operations (#16900)
* Add ability to perform automatic tidy operations

This enables the PKI secrets engine to allow tidy to be started
periodically by the engine itself, avoiding the need for interaction.
This operation is disabled by default (to avoid load on clusters which
don't need tidy to be run) but can be enabled.

In particular, a default tidy configuration is written (via
/config/auto-tidy) which mirrors the options passed to /tidy. Two
additional parameters, enabled and interval, are accepted, allowing
auto-tidy to be enabled or disabled and controlling the interval
(between successful tidy runs) to attempt auto-tidy.

Notably, a manual execution of tidy will delay additional auto-tidy
operations. Status is reported via the existing /tidy-status endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on auto-tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for auto-tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prevent race during parallel testing

We modified the RollbackManager's execution window to allow more
faithful testing of the periodicFunc. However, the TestAutoRebuild and
the new TestAutoTidy would then race against each other for modifying
the period and creating their clusters (before resetting to the old
value).

This changeset adds a lock around this, preventing the races.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use tidyStatusLock to gate lastTidy time

This prevents a data race between the periodic func and the execution of
the running tidy.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add read lock around tidyStatus gauges

When reading from tidyStatus for computing gauges, since the underlying
values aren't atomics, we really should be gating these with a read lock
around the status access.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-30 15:45:54 -04:00
Alexander Scheel
82174f0c0d Clean up behavior of If-Modified-Since header (#16929)
* Issuer renames should invalidate CRL cache times

When an issuer is renamed (or rather, two issuers' names are swapped in
quick succession), this is akin to the earlier identified default issuer
update condition. So, when any issuer is updated, go ahead and trigger
the invalidation logic.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix handling of delta CRL If-Modified-Since

The If-Modified-Since PR was proposed prior to the Delta CRL changes and
thus didn't take it into account. This follow-up commit fixes that,
addressing If-Modified-Since semantics for delta CRL fetching and
ensuring an accurate number is stored.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-30 13:25:36 -04:00
Gabriel Santos
a805ccbf2a PKI - Honor header If-Modified-Since if present (#16249)
* honor header if-modified-since if present

* pathGetIssuerCRL first version

* check if modified since for CA endpoints

* fix date comparison for CA endpoints

* suggested changes and refactoring

* add writeIssuer to updateDefaultIssuerId and fix error

* Move methods out of storage.go into util.go

For the most part, these take a SC as param, but aren't directly storage
relevant operations. Move them out of storage.go as a result.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use UTC timezone for storage

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rework path_fetch for better if-modified-since handling

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Invalidate all issuers, CRLs on default write

When the default is updated, access under earlier timestamps will not
work as we're unclear if the timestamp is for this issuer or a previous
issuer. Thus, we need to invalidate the CRL and both issuers involved
(previous, next) by updating their LastModifiedTimes.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for If-Modified-Since

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly invalidate default issuer changes

When the default issuer changes, we'll have to mark the invalidation on
PR secondary clusters, so they know to update their CRL mapping as well.
The swapped issuers will have an updated modification time (which will
eventually replicate down and thus be correct), but the CRL modification
time is cluster-local information and thus won't be replicated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* make fmt

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor sendNotModifiedResponseIfNecessary

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on if-modified-since

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 15:28:47 -04:00
Alexander Scheel
041d8372a5 Support for generating Delta CRLs (#16773)
* Allow generation of up-to-date delta CRLs

While switching to periodic rebuilds of CRLs alleviates the constant
rebuild pressure on Vault during times of high revocation, the CRL
proper becomes stale. One response to this is to switch to OCSP, but not
every system has support for this. Additionally, OCSP usually requires
connectivity and isn't used to augment a pre-distributed CRL (and is
instead used independently).

By generating delta CRLs containing only new revocations, an existing
CRL can be supplemented with newer revocations without requiring Vault
to rebuild all complete CRLs. Admins can periodically fetch the delta
CRL and add it to the existing CRL and applications should be able to
support using serials from both.

Because delta CRLs are emptied when the next complete CRL is rebuilt, it
is important that applications fetch the delta CRL and correlate it to
their complete CRL; if their complete CRL is older than the delta CRL's
extension number, applications MUST fetch the newer complete CRL to
ensure they have a correct combination.

This modifies the revocation process and adds several new configuration
options, controlling whether Delta CRLs are enabled and when we'll
rebuild it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for delta CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on delta CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address review feedback: fix several bugs

Thanks Steve!

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly invoke periodic func on active nodes

We need to ensure we read the updated config (in case of OCSP request
handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL
re-building.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 11:37:09 -04:00
Kit Haines
5cbe743c3a Fix CodeQL Errors - check allocation is smaller than 2^30 (#16869)
* Fix CodeQL Errors - check allocation is smaller than 32 bits.

* make fmt.
2022-08-26 13:26:11 -04:00
Alexander Scheel
6e6914547e Let PKI tidy associate revoked certs with their issuers (#16871)
* Refactor tidy steps into two separate helpers

This refactors the tidy go routine into two separate helpers, making it
clear where the boundaries of each are: variables are passed into these
method and concerns are separated. As more operations are rolled into
tidy, we can continue adding more helpers as appropriate. Additionally,
as we move to make auto-tidy occur, we can use these as points to hook
into periodic tidying.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor revInfo checking to helper

This allows us to validate whether or not a revInfo entry contains a
presently valid issuer, from the existing mapping. Coupled with the
changeset to identify the issuer on revocation, we can begin adding
capabilities to tidy to update this association, decreasing CRL build
time and increasing the performance of OCSP.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor issuer fetching for revocation purposes

Revocation needs to gracefully handle using the old legacy cert bundle,
so fetching issuers (and parsing them) needs to be done slightly
differently than other places. Refactor this from revokeCert into a
common helper that can be used by tidy.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow tidy to associate revoked certs, issuers

When revoking a certificate, we need to associate the issuer that signed
its certificate back to the revInfo entry. Historically this was
performed during CRL building (and still remains so), but when running
without CRL building and with only OCSP, performance will degrade as the
issuer needs to be found each time.

Instead, allow the tidy operation to take over this role, allowing us to
increase the performance of OCSP and CRL in this scenario, by decoupling
issuer identification from CRL building in the ideal case.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for tidy updates

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on new tidy parameter, metrics

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor tidy config into shared struct

Finish adding metrics, status messages about new tidy operation.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-26 10:13:45 -07:00
Brian Shumate
7ad1559489 Transit: update documentation strings (#10027)
- Update descriptions to match field content
  (actually key name, not policy name)
2022-08-26 09:25:02 -07:00
Steven Clark
84f130871f Add ocsp_expiry configuration field to PKI crl config (#16888)
* Add ocsp_expiry configuration field to PKI crl config

 - Add a new configurable duration field to the crl configuration to
   allow operator control of how long an OCSP response can be cached
   for.
 - This is useful for how long a server like NGINX/Apache is
   allowed to cache the response for OCSP stapling.
 - A value of 0 means no one should cache the response.
 - Address an issue discovered that we did not upgrade existing crl
   configurations properly

* PR feedback
2022-08-25 16:01:39 -04:00
Steven Clark
c5c016fc72 Add maximums on how much data we will read for OCSP requests (#16879)
* Add maximums on how much data we will read for OCSP requests

* Update max size to 2048
2022-08-25 12:59:49 -04:00
Alexander Scheel
f06a6f71d3 Cleanup changes around issuer revocation (#16874)
* Refactor CRL tests to use /sys/mounts

Thanks Steve for the approach! This also address nits from Kit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Skip CRL building steps when disabled

This skips a number of steps during CRL build when it is disabled (and
forceNew is not set). In particular, we avoid fetching issuers, we avoid
associating issuers with revocation entries (and building that in-memory
mapping), making CRL building more efficient.

This means that there'll again be very little overhead on clusters with
the CRL disabled.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prevent revoking roots from appearing on own CRLs

This change ensures that when marking a root as revoked, it no longer
appears on its own CRL. Very few clients support this event (as
generally only leaves/intermediates are checked for presence on a
parent's CRL) and it is technically undefined behavior (if the root is
revoked, its own CRL should be untrusted and thus including it on its
own CRL isn't a safe/correct distribution channel).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure stability of revInfo issuer identification

As mentioned by Kit, iterating through each revInfoEntry and associating
the first issuer which matches it can cause churn when many (equivalent)
issuers are in the system and issuers come and go (via CRLSigning usage,
which has been modified in this release as well). Because we'd not
include issuers without CRLSigning usage, we'd cause our verification
helper, isRevInfoIssuerValid, to think the issuer ID is no longer value
(when instead, it just lacks crlSigning bits).

We address this by pulling in all issuers we know of for the
identification. This allows us to keep valid-but-not-for-signing
issuers, and use other representatives of their identity set for
signing/building the CRL (if they are enabled for such usage).

As a side effect, we now no longer place these entries on the default
CRL in the event all issuers in the CRL set are without the usage.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

This is only for the last commit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-25 11:36:37 -04:00
Tom Proctor
247a019be0 Pin MongoDB test container images pre-v6 (#16880)
v6 was released in the last 24h, and our tests fail to connect to the db when v6 is used.
Using v6 needs investigating, but for now I'm pinning to the last known good version.
2022-08-25 08:14:37 -07:00
Alexander Scheel
319dc82bb8 Identify issuer on revocation (#16763)
* Identify issuer on revocation

When we attempt to revoke a leaf certificate, we already parse all of
the issuers within the mount (to x509.Certificate) to ensure we don't
accidentally revoke an issuer via the leaf revocation endpoint. We can
reuse this information to associate the issuer (via issuer/subject
comparison and signature checking) to the revoked cert in its revocation
info. This will help OCSP, avoiding the case where the OCSP handler
needs to associate a certificate to its issuer.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test to ensure issuers are identified

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-24 12:23:27 -04:00
Alexander Scheel
7f90f83d3d Don't allow crl-signing issuer usage without CRLSign KeyUsage (#16865)
* Allow correct importing of certs without CRL KU

When Vault imports certificates without KU for CRLSign, we shouldn't
provision CRLUsage on the backing issuer; otherwise, we'll attempt to
build CRLs and Go will cause us to err out. This change makes it clear
(at issuer configuration time) that we can't possibly support this
operation and hopefully prevent users from running into the more cryptic
Go error.

Note that this does not apply for OCSP EKU: the EKU exists, per RFC 6960
Section 2.6 OCSP Signature Authority Delegation, to allow delegation of
OCSP signing to a child certificate. This EKU is not necessary on the
issuer itself, and generally assumes issuers are allowed to issue OCSP
responses regardless of KU/EKU.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add docs to clarify issue with import, CRL usage

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update website/content/api-docs/secret/pki.mdx

* Add additional test assertion

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-24 07:45:54 -07:00
Steven Clark
14cd842d50 Handle multiple matching issuers in OCSP requests (#16848)
* Handle multiple matching issuers in OCSP requests

 - Select the first issuer that matches our request hashes and has
   the OCSP signing usage enabled. This might not match the exact
   issuer id that issued the certificate but the signatures will be
   okay.

* PR feedback
2022-08-24 09:00:40 -04:00
Alexander Scheel
986e43b7f2 Finish refactor to remove global crlLifetime (#16835)
Previously we used the global backend-set crlLifetime as a default
value. However, this was refactored into a new defaultCrlConfig instead,
which we should reply with when the CRL configuration has not been set
yet. In particular, the 72h default expiry (and new 12h auto-rebuild
grace period) was added and made explicit.

This fixes the broken UI test.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-23 15:19:11 -04:00
Alexander Scheel
dccc749c31 Enable periodic, automatic rebuilding of CRLs (#16762)
* Allow automatic rebuilding of CRLs

When enabled, periodic rebuilding of CRLs will improve PKI mounts in two
way:

 1. Reduced load during periods of high (new) revocations, as the CRL
    isn't rebuilt after each revocation but instead on a fixed schedule.
 2. Ensuring the CRL is never stale as long as the cluster remains up,
    by checking for next CRL expiry and regenerating CRLs before that
    happens. This may increase cluster load when operators have large
    CRLs that they'd prefer to let go stale, rather than regenerating
    fresh copies.

In particular, we set a grace period before expiration of CRLs where,
when the periodic function triggers (about once a minute), we check
upcoming CRL expirations and check if we need to rebuild the CRLs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on periodic rebuilding

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow modification of rollback period for testing

When testing backends that use the periodic func, and specifically,
testing the behavior of that periodic func, waiting for the usual 1m
interval can lead to excessively long test execution. By switching to a
shorter period--strictly for testing--we can make these tests execute
faster.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for auto-rebuilding of CRLs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove non-updating getConfig variant

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Avoid double reload of config

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-23 13:27:15 -04:00
Kit Haines
51d8cdca4b Add _remaining tidy metrics. (#16702)
* Add _remaining tidy metrics.

* Add two extra metrics during tidy.

* Update test and documentation for remaining tidy metrics.
2022-08-23 12:17:17 -04:00