Commit Graph

18722 Commits

Author SHA1 Message Date
Ben Ash
091aea8379 auth/kubernetes: upgrade to v0.14.1 (#18716) 2023-01-13 19:00:18 -05:00
Ben Ash
41012b48ef Revert "auth/kubernetes: upgrade to v0.14.1 (#18711)" (#18715)
This reverts commit ed244a9263.
2023-01-13 18:17:12 -05:00
Ben Ash
ed244a9263 auth/kubernetes: upgrade to v0.14.1 (#18711) 2023-01-13 17:15:35 -05:00
Anton Averchenkov
9997cf5cdd Add AppRole response schema validation tests (#18636)
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role  successful responses align with the declared response schema.

It also introduces a test helper in `sdk/helper/testhelpers`:

```go
func FindResponseSchema(t *testing.T, ...)
```

This test helper will be useful for all plugins that require similar response schema validation in tests.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
sdk/v0.7.0
2023-01-13 15:23:36 -05:00
Anton Averchenkov
6a2586747c Add response schema validation methods & test helpers (#18635)
This pull request adds 3 functions (and corresponding tests):

`testhelpers/response_validation.go`:

  - `ValidateResponse`
  - `ValidateResponseData`
  
field_data.go:

  - `ValidateStrict` (has the "strict" validation logic)

The functions are primarily meant to be used in tests to ensure that the responses are consistent with the defined response schema. An example of how the functions can be used in tests can be found in #18636.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 14:55:56 -05:00
Yoko Hyakuna
1700d1c1d3 Add VAULT_LOG_LEVEL to the environment var list (#18694) 2023-01-13 12:54:51 -05:00
Jaymala
148bc6ca27 [QT-19] Enable Enos replication scenario (#17748)
* Add initial replication scenario config

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add support for replication with different backend and seal types

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Update Consul versions

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Additional config for replicaiton scenario

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Update replication scenario modules

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Refactor replication modules

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add more steps for replication

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Work in progress with unsealing followers on secondary cluster

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add more replication scenario steps

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* More updates

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Working shamir scenario

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Update to unify get Vault IP module

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Remove duplicate module

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Fix race condition for secondary followers unseal

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Use consistent naming for module directories

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Update replication scenario with latest test matrix

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Verify replication with awskms

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add write and retrive data support for all scenarios

* Update all scenarios to verify write and read kv data

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Fix write and read data modules

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add comments explaining the module run

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Address review feedback and update consul version

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Address more review feedback

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Remove vault debug logging

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Exclude ent.fips1402 and ent.hsm.fips1402 packages from Enos test matrix

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add verification for replication connection status

* Currently this verification fails on Consul due to VAULT-12332

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Add replication scenario to Enos README

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Update README as per review suggesstions

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* [QT-452] Add recovery keys to scenario outputs

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Fix replication output var

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

* Fix autopilot scenario deps and add retry for read data

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>

Signed-off-by: Jaymala Sinha <jaymala@hashicorp.com>
2023-01-13 11:43:26 -05:00
Steven Clark
a14d5c90b4 Refactor the PKI revocation handler to prep for unified revocation (#18685)
* Rename revokeCert variable to identify serial number formatting

* Refactor out lease specific behavior out of revokeCert

 - Isolate the specific behavior regarding revoking lease specific
   certificates outside of the revokeCert function and into the only
   caller that leveraged used it.
 - This allows us to simplify revokeCert a little bit and keeps the
   function purely about revoking a certificate

* Within revokeCert short circuit the already revoked use-case

 - Make the function a little easier to process by exiting early
   if the certificate has already been revoked.

* Do not load certificates from storage multiple times during revocation

 - Isolate the loading of a certificate and parsing of a certificate
   into a single attempt, either when provided the certificate for BYOC
   revocation or strictly from storage for the other revocation types.

* With BYOC write certificate entry using dashes not the legacy colon char
2023-01-13 10:31:03 -05:00
Violet Hynes
819c9e0e2e VAULT-12542 Add info encouraging users to upgrade if agent version is different to server (#18684)
* VAULT-12542 Add info encouraging users to upgrade if agent version is different to server

* VAULT-12542 Changelog

* VAULT-12542 Language update
2023-01-13 09:49:36 -05:00
Jordan Reimer
5fd1b88fbc PKI Issuer Edit (#18687)
* adds pki issuer edit view

* updates pki issuer details test and fixes styling issue in issuer edit form

* addresses feedback
2023-01-12 16:33:14 -07:00
claire bontempo
8bd3541346 ui: create object list input component (#18671)
* build object-list-component WIP

* finish component functionality

* refactor so handleChange is separate function and does not send empty last object to parent

* add inputValue as an arg to pre-fill component

* finish tests

* final cleanup, add comments and assertions

* add aria-labels

* Update ui/lib/core/addon/components/object-list-input.js

* Update ui/lib/core/addon/components/object-list-input.hbs

Co-authored-by: Chelsea Shaw <82459713+hashishaw@users.noreply.github.com>

Co-authored-by: Chelsea Shaw <82459713+hashishaw@users.noreply.github.com>
2023-01-12 23:20:45 +00:00
akshya96
1af6bc2b58 Vault 8308 Background thread to update locked user entries (#18673)
* background thread changes

* adding changelog

* fix changelog typo
2023-01-12 14:09:33 -08:00
claire bontempo
9752352fbd ui: unload auth method when navigating away from form (#18651)
* unload record instead of rollback

* unload record instead of rollback

* add changelog

* add rollback attrs back if record is not new
2023-01-12 13:03:22 -08:00
Chris Capurso
418082d3d3 VAULT-12548: Fix data races in Link status reporting (#18674)
* pin hcp-scada-provider

* fix link reportStatus data race

* go get hcp-scada-provider@v0.2.1
2023-01-12 15:10:34 -05:00
Austin Gebauer
280c7dbbd9 docs/oidc: change user type recommendation for Google workspace integration (#18676)
* docs/oidc: change user type recommendation for Google workspace integration

* Update website/content/docs/auth/jwt/oidc-providers/google.mdx

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2023-01-12 00:44:01 +00:00
Josh Brand
4768e74a35 Add automated CI account cleanup & monitoring (#18659)
This uses aws-nuke and awslimitchecker to monitor the new vault CI account to clean up and prevent resource quota exhaustion.  AWS-nuke will scan all regions of the accounts for lingering resources enos/terraform didn't clean up, and if they don't match exclusion criteria, delete them every night.  By default, we exclude corp-sec created resources, our own CI resources, and when possible, anything created within the past 72 hours. Because this account is dedicated to CI, users should not expect resources to persist beyond this without additional configuration.
2023-01-11 17:24:08 -05:00
Max Bowsher
aeb1b1e72f Add option 'elide_list_responses' to audit backends (#18128)
This PR relates to a feature request logged through HashiCorp commercial
support.

Vault lacks pagination in its APIs. As a result, certain list operations
can return **very** large responses.  The user's chosen audit sinks may
experience difficulty consuming audit records that swell to tens of
megabytes of JSON.

In our case, one of the systems consuming audit log data could not cope,
and failed.

The responses of list operations are typically not very interesting, as
they are mostly lists of keys, or, even when they include a "key_info"
field, are not returning confidential information. They become even less
interesting once HMAC-ed by the audit system.

Some example Vault "list" operations that are prone to becoming very
large in an active Vault installation are:

    auth/token/accessors/
    identity/entity/id/
    identity/entity-alias/id/
    pki/certs/

In response, I've coded a new option that can be applied to audit
backends, `elide_list_responses`. When enabled, response data is elided
from audit logs, only when the operation type is "list".

For added safety, the elision only applies to the "keys" and "key_info"
fields within the response data - these are conventionally the only
fields present in a list response - see logical.ListResponse, and
logical.ListResponseWithInfo. However, other fields are technically
possible if a plugin author writes unusual code, and these will be
preserved in the audit log even with this option enabled.

The elision replaces the values of the "keys" and "key_info" fields with
an integer count of the number of entries. This allows even the elided
audit logs to still be useful for answering questions like "Was any data
returned?" or "How many records were listed?".
2023-01-11 16:15:52 -05:00
Peter Wilson
8abcde7cbb VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Ellie
49da2544ce add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Austin Gebauer
5564be70ea docs/plugin: adds note on leases and tokens during upgrades (#18669)
* docs/plugin: adds note on leases and tokens during upgrades

* Update website/content/docs/upgrading/plugins.mdx

Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
2023-01-11 18:24:54 +00:00
Anton Averchenkov
7304d5424e Fix AppRole / path_role response schema (#18637) 2023-01-11 12:15:29 -05:00
Alexander Scheel
8ea8598b63 Allow tidy to backup legacy CA bundles (#18645)
* Allow tidy to backup legacy CA bundles

With the new tidy_move_legacy_ca_bundle option, we'll use tidy to move
the legacy CA bundle from /config/ca_bundle to /config/ca_bundle.bak.
This does two things:

 1. Removes ca_bundle from the hot-path of initialization after initial
    migration has completed. Because this entry is seal wrapped, this
    may result in performance improvements.
 2. Allows recovery of this value in the event of some other failure
    with migration.

Notably, this cannot occur during migration in the unlikely (and largely
unsupported) case that the operator immediately downgrades to Vault
<1.11.x. Thus, we reuse issuer_safety_buffer; while potentially long,
tidy can always be run manually with a shorter buffer (and only this
flag) to manually move the bundle if necessary.

In the event of needing to recover or undo this operation, it is
sufficient to use sys/raw to read the backed up value and subsequently
write it to its old path (/config/ca_bundle).

The new entry remains seal wrapped, but otherwise isn't used within the
code and so has better performance characteristics.

Performing a fat deletion (DELETE /root) will again remove the backup
like the old legacy bundle, preserving its wipe characteristics.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation about new tidy parameter

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for migration scenarios

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clean up time comparisons

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-11 12:12:53 -05:00
Alexander Scheel
c7a696457d Add pki health-check docs (#18517)
* Add documentation on vault pki health-check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refer users to online docs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-11 11:46:30 -05:00
Divya Pola
11dbc16b7e Add documentation for KMIP features implemented in 1.13 (#18613)
* Add documentation for KMIP features implemented in 1.13

* Add release version for key format types

* Fix syntax

* Add supported hashing algorithms and padding methods

* Fix formatting

* Add  nit picks from review feedback
2023-01-11 20:33:05 +05:30
Chelsea Shaw
29533caa17 UI: PKI config refactor (#18639) 2023-01-10 16:13:20 -06:00
Ellie
36d68656f4 docs: note that env vars must be set on the vault process and can be checked in 1.13 (#18652) 2023-01-10 15:35:45 -06:00
John-Michael Faircloth
7517e39f53 db plugin: support multiline revoke stmt in postgres (#18632)
* db plugin: support multiline revoke stmt in postgres

* add changelong
2023-01-10 15:27:00 -06:00
Max Bowsher
4758cc8f86 Fix HelpOperation on sudo-protected paths (#18568)
* Fix HelpOperation on sudo-protected paths

Fixes #18566

* Add changelog
2023-01-10 12:17:16 -06:00
Peter Wilson
1fca38a4ae VAULT-9883: Agent Reloadable Config (#18638)
* Update command/agent.go
* Attempt to only reload log level and certs
* Mimicked 'server' test for cert reload in 'agent'

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Left out the `c.config` tweak that meant changes to lots of lines of code within the `Run` function of Agent command. :)
2023-01-10 17:45:34 +00:00
claire bontempo
582a64598c UI: Remove LearnLink, use DocLink! (#18641)
* delete learn link, update links

* update documentation

* update test

* Update ui/app/templates/components/wizard/replication-setup.hbs
2023-01-10 17:07:08 +00:00
Anton Averchenkov
3f073ae3ce Revert "Add mount path into the default generated openapi.json spec (#17926)" (#18617)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"

This reverts commit db8efac708.

* Revert "Remove `generic_mount_paths` field (#18558)"

This reverts commit 79c8f626c5.
2023-01-10 11:16:59 -05:00
Alexander Scheel
1e356c9c7d Correctly distinguish empty issuer names in PKI (#18466)
* Correctly distinguish empty issuer names

When using client.Logical().JSONMergePatch(...) with an empty issuer
name, patch incorrectly reports:

> issuer name contained invalid characters

In this case, both the error in getIssuerName(...) is incorrect and
patch should allow setting an empty issuer name explicitly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-10 10:04:30 -05:00
Alexander Scheel
822fba38d3 Add cluster_aia_path templating variable (#18493)
* Add cluster_aia_path templating variable

Per discussion with maxb, allow using a non-Vault distribution point
which may use an insecure transport for RFC 5280 compliance.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address feedback from Max

Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
2023-01-10 09:51:37 -05:00
Alexander Scheel
ab415325ab Add vault pki command website documentation (#18514)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-10 09:39:41 -05:00
claire bontempo
64628739ed UI: pki import issuer (#18634)
* create pki ca import component

* add serial number to cert parser

* convert to ts

* remove comments

* reset yarn.lock

* fixed yarn lock

* fix comment

* add declaration for base cert
2023-01-09 16:46:02 -08:00
Josh Black
d641bbc28e Add new clients into the monthly breakdown (#18629)
* Add new clients into the monthly breakdown

* add changelog
2023-01-09 15:26:11 -08:00
Violet Hynes
4b8bd15e28 VAULT-12491 Add docs for group policy config (#18616)
* VAULT-12491 Add docs for group policy config

* VAULT-12491 typo

* VAULT-12491 typo

* VAULT-12491 Update language a bit

* VAULT-12491 Update language a bit

* VAULT-12491 Update language a bit
2023-01-09 12:50:16 -05:00
Mike Palmiotto
31772c930b Mark deprecated builtins Removed (#18039)
* Remove logical database builtins

* Drop removed builtins from registry keys

* Update plugin prediction test

* Remove app-id builtin

* Add changelog
2023-01-09 09:16:35 -05:00
Chris Capurso
40e87ae4c0 VAULT-11830: Expand NodeStatusReporter with new fields (#18302)
* expand NodeStatusReporter with new fields

* only call IsRaftVoter if using raft storage

* add changelog entry

* fix listeners

* return LogLevel as enum

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* bump github.com/hashicorp/vault/vault/hcp_link/proto

* go mod tidy
2023-01-06 20:53:09 -05:00
Chris Capurso
9247459dc3 VAULT-11829: Add cluster status handler (#18351)
* go get link proto @vault-11829-meta-get-cluster-status

* add HA status

* add HAEnabled method

* add raft config

* allocate HA nodes based on actual count

* add raft autopilot status

* add raft quorum warnings

* add ClusterID method

* add StorageType

* add ClusterID

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* fix raft config panic

* remove "Warning" quorum message prefix

* add error wrapping

* add Core.HAStateWithLock method

* reduce quorum warnings to single string

* fix HCP_API_HOST test env var check

* Revert "fix HCP_API_HOST test env var check"

This reverts commit 97c73c4798b77b84aea84f341f2c63c4d657914d.
2023-01-06 17:06:54 -05:00
claire bontempo
a8c8260634 extend overview route from issuers index (#18623) 2023-01-06 21:01:52 +00:00
Chris Capurso
9eaf267163 change quorum warning to a singular string (#18619) 2023-01-06 15:31:27 -05:00
Max Bowsher
901876f27a Fix error in changelog template (#18572)
Too many newlines are stripped, which is responsible for the `FEATURES:`
heading in the current in-progress 1.13.0 changelog entry being
erroneously appended to the end of the last bullet point of the previous
`CHANGES:` section.
2023-01-06 14:32:22 +00:00
Chris Capurso
048241517f VAULT-11829: Add GetClusterStatus rpc to meta capability (#18316)
* add GetClusterStatus rpc to meta capability

* consolidate HA-related info

* add storage type
2023-01-05 13:33:08 -05:00
Violet Hynes
78bb0757e3 VAULT-12489 OSS Changes (#18607) 2023-01-05 13:00:55 -05:00
Christopher Swenson
2a469d2c59 docs: Update PKCS#11 provider docs for XKS and RNG (#18597)
Better IV random generation is supported with XKS in the latest version
of the provider (0.1.3).
2023-01-05 09:42:52 -08:00
Josh Black
cb61488b9a Account for mount counts when de-duplicating current and historical month data (#18598)
* Account for mount counts when de-duplicating current and historical month data

* add changelog
2023-01-05 09:34:05 -08:00
Chris Capurso
334a4fb401 VAULT-11827: Add new Link node level fields (#18203)
* add addition Link node-level status fields

* pin protoc to 3.21.9

* make proto

* change LogLevel to be a string

* add RaftStatus; IsActive -> Activetime

* use an enum for LogLevel
2023-01-05 11:16:48 -05:00
John-Michael Faircloth
e73baa4a76 docs: db plugin add link to lease docs (#18605) 2023-01-05 16:14:54 +00:00
Max Bowsher
774c6fa802 Fix duplicate definition of path sys/internal/specs/openapi (#18553)
This was accidentally duplicated in #5687.

Remove the second definition, which was shadowed by the first, and move
the documentation that was part of the second to the surviving version.
2023-01-04 22:48:40 -05:00