* Limit ACME issued certificates to a max of 90 days
- If the normal NotAfter date that is calculated from the mount/issuer/role TTL values is greater than 90 days, override the TTL value to a 90-day value.
* Add changelog
* Generate ACME EAB tokens that do not start with -
- To avoid people having issues copying EAB tokens and using them on command lines when they start with - from the base64 encoded values, append a prefix.
- Remove the key_bits data from the eab api, not really useful and now technically wrong
- Fix up some issues with tests not running in parallel.
- Update docs to reflect new EAB apis.
* Add ACME directory to the various EAB output APIs
* Update EAB token prefix to be divisable by 3
- Our decoded prefix was not divisable by 3, which meant the last
character might be tweaked by the rest of the input
* Attempt to resolve flaky test TestAcmeBasicWorkflow test
- Since we can't control the challenge engine, flush the validation records it leverages prior to manually updating the authorization/challenge statuses
```
path_acme_test.go:261: csr: &{[] [] [] [] 0 [] 0 0 <nil> CN=*.localdomain [] [] [] [localhost.localdomain *.localdomain] [] [] []}
path_acme_test.go:300:
Error Trace: /home/runner/actions-runner/_work/vault-enterprise/vault-enterprise/builtin/logical/pki/path_acme_test.go:300
Error: Received unexpected error:
403 urn:ietf:params:acme:error:orderNotReady: The request attempted to finalize an order that is not ready to be finalized: order is status pending, needs to be in ready state
Test: TestAcmeBasicWorkflow/role
Messages: failed finalizing order
```
* make fmt
* Change from default_role to default_directory_policy to allow future improvements.
* Helper functions
* Use the helper function and make fmt.
* Do not allow the zero-length role "".
* Semgrep doesn't like shadowing errors that are impossible to hit, so fix that.
* Add default to switch branches.
* Add/fix docs.
* Fix wrong requestedRole
* Fix ACME computed order status
* Return validation errors and status updates for authorizations
- We now populate the error field within challenges with the error results from the challenge
- Update the status of the challenge and authorizations to invalid when we give up on the challenge
- Verify that only a single challenge within a given authorization can be accepted to avoid race conditions.
- Saw a test failure when we generated an EAB key that started with -
```
acme_test.go:249: Certbot Issue Command: [certbot certonly
--no-eff-email --email certbot.client@dadgarcorp.com --eab-kid
0246913b-4382-10fc-bf57-b05f2dad0f13 --eab-hmac-key
-Avt5q_KUWWWL8slYJn_MdmiCA-jzvif6Tpt45gQNR0 --agree-tos --no-verify-ssl
--standalone --non-interactive --server
...
certbot: error: argument --eab-hmac-key: expected one argument
```
* Fix export of HMAC typed keys
When initially implemented, exporting HMAC keys resulted in returning
the unused, internal HMACKey value rather than the main Key value that
is used for HMAC operations.
This is a breaking change.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Consistently handle HMAC keys in keysutil
When generating HMAC-typed keys, set HMACKey = Key consistently, to
allow users of HMAC-typed keys to use them backwards compatibly.
Notably, this could discard the (unused) HMACKey field set today.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test proving export of HMAC keys work
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
This should help to prevent the issue of missing tidy configurations
in the future, by placing all related configuration options at the
top with common validation logic.
However, short from this approach is ensuring that each config option
can be specified independently. Thus, the test allows (for any added
and properly tracked tidy operations) verifying that we have enabled
proper storage/retention of that attribute.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add WrapKey capabilities to keysutil
This allows one keysutil to wrap another key, assuming that key has an
type matching one of keysutil's allowed KeyTypes. This allows completing
the BYOK import loop with Transit, allowing imported wrapping keys to
export (wrap) other keys in transit, without having them leave in
plaintext.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /byok-export/:dst/:src[/:version] to Transit
Still respecting exportable, we allow encrypted-only export of transit
keys to another cluster using the BYOK semantics.
In particular, this allows an operator to securely establish key
material between two separate Transit installations. This potentially
allows one cluster to be used as a source cluster (encrypting a large
amount of data) and a second cluster to decrypt this data later. This
might be useful in hybrid or site-specific deployments of Vault for
instance.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing dependency to sdk/, vault/
Also updates to a newer version while we're here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on BYOK export
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for BYOK export/import
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/transit.mdx
* Update builtin/logical/transit/path_byok.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Add Certbot unregister integration test
- Make sure we can unregister ACME accounts through Certbot to provide an additional validation.
* Add vault-crypto as a code owner for builtin/logical/pkiext
* poc: client cert credential type
* go mod tidy
* fix typo
* fix newUserReqToProto
* add changelog
* add newline
* fix changelog
* add test for the client cert generator
* Fix formatting
* unset signing bundle URLs
* set BasicConstraintsValidForNonCA to false
* backdate cert by 30s
* remove empty creation params URLs
* check cert BasicConstraintsValid
* set default key bits in newClientCertificateGenerator
* fix client cert gen test with default values
* Add default for key_type
* fix default key_type
* update test with default key type
* update test
* Update changelog/20425.txt
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
* set default key bits and sig bits
* remove the default for key type ad fix the test
* make fmt + add comments for each exported field
* restart test
---------
Co-authored-by: Austin Gebauer <agebauer@hashicorp.com>
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
* Update EAB management urls underneath pki/eab
- It was decided that for ease of ACLing, the management
paths for EAB apis should be outside of the acme path
prefix
- Delete duplicated tests, rely on the proper cluster
based tests for EAB management.
* Update ACME EAB creation paths to be directory specific
- Make the EAB creation APIs directory specific.
- This commit is still missing the enforcement that
they can be redeemed on a specific path.
* Enforce EAB tokens per ACME directory context like accounts
- Do not allow an EAB from one ACME directory to be used
in another.
- Rework the ACME directory function to simply get the path from the request instead of parsing out the role/issuer name.
- Add some commentary around expectations if operators change issuer names
* Add an EAB certbot integration test
- Verify with the 3rd party certbot cli that our EAB workflow works as expected.
* Fix unit test
- Unit test wasn't setting up r.Path within the request
that we now use to determine the acme directory.
* ACME override issuer's leaf_not_after_behavior to truncate
- To provide a better ACME experience as we don't allow clients to specify TTL times, we will override the issuer's leaf_not_after_behavior setting to 'truncate' if set to the default of 'err' and issue the certificate truncated to the issuer's NotAfter time.
* Only allow ServerAuth ExtKeyUsage from ACME certificates
- Add an enforcement to ACME issued certificates that the only ExtKeyUsage we currently allow is the ServerAuth usage.
* Force ServerAuth as the ExtKeyUsage in ACME roles
- Override a role's values related to ExtKeyUsage when
running in ACME mode to only return the ServerAuth usage.
- We do this as the majority of roles out there will most likely have the ClientAuth set to true which will cause friction using ACME.
* Fix error handling in ACME
- If we don't match a specific ACME error, use ErrServerInternal instead of the last error type from the internal map
- Logger parameters need two params
* Enforce cluster local path is set when enabling ACME
* Add a warning on ACME config read api if enabled but path not set
- This might help expose that the local path configuration on a secondary cluster was not set which would prevent ACME from running.
* Fix various EAB related issues
- List API wasn't plumbed through properly so it did not work as expected
- Use random 32 bytes instead of an EC key for EAB key values
- Update OpenAPI definitions
* Clean up unused EAB keys within tidy
* Move Vault EAB creation path to pki/acme/new-eab
* Update eab vault responses to match up with docs
* Refine documentation for public_key
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support additional key types in importing version
This originally left off the custom support for Ed25519 and RSA-PSS
formatted keys that we've added manually.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for Ed25519 keys
Here, we prevent importing public-key only keys with derived Ed25519
keys. Notably, we still allow import of derived Ed25519 keys via private
key method, though this is a touch weird: this private key must have
been packaged in an Ed25519 format (and parseable through Go as such),
even though it is (strictly) an HKDF key and isn't ever used for Ed25519.
Outside of this, importing non-derived Ed25519 keys works as expected.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key only export method to Transit
This allows the existing endpoints to retain private-key only, including
empty strings for versions which lack private keys. On the public-key
endpoint, all versions will have key material returned.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tests for exporting via public-key interface
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key export option to docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix race in PKI's runUnifiedTransfer
During this race, we'll sometimes start (or fail to start) an additional
unified transfer if the updated last run timestamp was written at the
same time as another thread was reading it.
Instead, delay this check until we're holding the CAS guard; this will
occasionally result in more messages saying that an existing process is
already running, but otherwise shouldn't impact the functionality at
all.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix tidy with maintain_stored_certificate_counts == publish_stored_certificate_count_metrics == false
The logic around the check to set both to false was wrong, and should
be validated independently.
Additionally, these fields should only exist on auto-tidy and not on the
manual tidy endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pki/path_tidy.go
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
* Respond with cache size on config write
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Respond with key policy on write
This includes creating a key, but also trimming or rotating an
existing key.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly handle locking around policy formatting
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate that responses are non-empty
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* pki: add subject key identifier to read key response
This will be helpful for the Terraform Vault Provider to detect
migration of pre-1.11 exported keys (from CA generation) into post-1.11
Vault.
* add changelog
* Update builtin/logical/pki/path_fetch_keys.go
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* check for managed key first
* Validate the SKID matches on root CAs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate SKID matches on int CAs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix formatting of tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Do not set use_csr_values when issuing ACME certs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure CSRs with Basic Constraints are rejected
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test to ensure CA certificates cannot be issued
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pkiext/pkiext_binary/acme_test.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update builtin/logical/pkiext/pkiext_binary/acme_test.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update acme_test.go to include certutil
* Update acme_test.go - unused imports, reformat
* Update acme_test.go - hex really was used
This is why I can't use the GH web editor. :-)
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add stub ACME billing interfaces
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add initial implementation of client count
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly attribute to mount, namespace
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor adding entities of custom types
This begins to add custom types of events; presently these are counted
as non-entity tokens, but prefixed with a custom ClientID prefix.
In the future, this will be the basis for counting these events
separately (into separate buckets and separate storage segments).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of ACME mounts
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case for billing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better support managed key system view casting
Without an additional parameter, SystemView could be of a different
internal implementation type that cannot be directly casted to in OSS.
Use a separate parameter for the managed key system view to use instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of mounts for enterprise
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate mounts in ACME billing tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a hopefully unique separator for encoded identifiers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use mount accesor, not path
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename AddEventToFragment->AddActivityToFragment
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
* Disable requiring EAB in ACME by default
- After an internal meeting it was decided that enabling EAB support by default was probably not the right decision.
- The main motivating factor being ease of use by end-users as the majority of implementations aren't expecting EAB to be required by default.
* Leverage function isPublicACMEDisabledByEnv and log parsing error
- Add logging to the new isPublicACMEDisabledByEnv function if we fail to parse the env var
- Leverage the function within the isAcmeDisabled function in acme_wrappers.go to not duplicate the env getting logic in two places.
* Fail closed when VAULT_DISABLE_PUBLIC_ACME is un-parsable.
* Add a last issued date on ACME accounts
- When we issue a new ACME certificate, attempt to update the account's last issued field
- Within ACME account tidy, use both account creation and last issue date to provide a buffer before we mark the account as revoked.
- Cleanup the cert serial to account tracker
- Misc formatting fixes in JSON objects
* Move account max-cert-expiry updates within tidy
- Perform the account update of max-cert-expiry within
the tidy operation as it has the account write lock
and is already iterating over all orders.
- With this the order path does not need any account
level locks
* Prefix ACME account status constants with AccountStatusX
