Commit Graph

1667 Commits

Author SHA1 Message Date
hc-github-team-secure-vault-core
c496011eed backport of commit d08bf5616d (#19347)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-02-24 14:11:43 -05:00
hc-github-team-secure-vault-core
6a73f37ba4 backport of commit 20b347e3cd (#19315)
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
2023-02-23 19:29:52 +01:00
hc-github-team-secure-vault-core
d21564e539 backport of commit 7d52daf715 (#19308)
Co-authored-by: Peter Wilson <peter.wilson@hashicorp.com>
2023-02-23 17:13:14 +00:00
hc-github-team-secure-vault-core
871dd067e3 backport of commit fe7eedafc3 (#19286)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-02-22 14:32:18 +00:00
hc-github-team-secure-vault-core
8cad3a3426 backport of commit 1b33b99253 (#19275)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-02-21 15:15:16 -05:00
hc-github-team-secure-vault-core
88e9f55199 backport of commit 9c4e65986f (#19273)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-02-21 19:24:22 +00:00
hc-github-team-secure-vault-core
8ffa334946 backport of commit 4ea5c581ad (#19268)
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-02-21 14:18:22 +00:00
hc-github-team-secure-vault-core
e53ac2633b backport of commit 100ec9a700 (#19203)
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-20 20:03:17 +00:00
hc-github-team-secure-vault-core
02bc25417a Trap errors related to vault pki list-intermediate issuer reading (#19165) (#19177)
* Rename files to match test suite and existing pattern

* Factor out issuer loading into a dedicated function

 - Add a little more checks/validation when loading the a PKI issuer
 - Factor out the issuer loading into a dedicated function
 - Leverage existing health check code to parse issuer certificates

* Read parent issuer once instead of reloading it for every child

 - Read in our parent issuer once instead of running it for every child
   we want to compare against
 - Provides clearer error message that we have failed reading from which
   path to the end user

* PR Feedback

 - Rename a variable for clarity
 - Use readIssuer in the validation of the parent issuer within
   pkiIssuer
 - Add some missing return 1 statements in error handlers that had been
   missed

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-02-14 09:51:12 -05:00
hc-github-team-secure-vault-core
3e4710d7ad backport of commit d08de3e78e (#19178)
Co-authored-by: Kit Haines <khaines@mit.edu>
2023-02-14 14:23:50 +00:00
Christopher Swenson
6e233e567b events: Add websockets and command (#19057)
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.

This can be used by either `wscat` or the new
`vault events subscribe`:

e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

and

```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-09 13:18:58 -08:00
mickael-hc
2eb11c0109 update diagnose command to no longer use docker (#19102)
docker dependency should no longer be included in the binary
2023-02-09 11:59:46 -05:00
Steven Clark
e32b4dccf1 Stop using title capitalization for PKI help (#19104)
- Match the existing vault kv capitalization scheme for Synopsis help of each sub-command.
 - A few small tweaks as well to the messages text in a few cases
2023-02-09 16:40:26 +00:00
miagilepner
13caa0842e VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
Nick Cabatoff
192baa88db Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Scott Miller
bf58843b0a Revert #18683 (#18942)
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"

This reverts commit 990d3bacc2.

* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"

This reverts commit 2ffe49aab0.
2023-02-01 13:34:53 -06:00
Violet Hynes
68dbb925b7 VAULT-13056 fix leasecache usage, add test coverage (#18922)
* VAULT-13056 fix leasecache usage, add test coverage

* VAULT-13056 remove deprecated ioutil functions

* VAULT-13056 some test clean-up

* VAULT-13056 re-add environment variable thing

* VAULT-13056 add comment for clarity
2023-02-01 11:40:20 -05:00
Nick Cabatoff
970ed07b04 Vault test cluster helper refactorings, mostly audit related (#18928)
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package.  Consolidate on a single "noop audit" implementation.
2023-02-01 08:33:16 -05:00
Alexander Scheel
9042643d63 Fix command.RunCustom(...) correctly (#18904)
* Revert "Remove t.Parallel() due to initialization race (#18751)"

This reverts commit ebcd653102.

We're going to fix this properly, running initCommands exactly once.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prevent parallel testing racing in initCommands(...)

When running initCommands(...) from multiple tests, they can potentially
race, causing a panic. Test callers needing to set formatting
information must use RunCustom(...) instead of directly invoking the
test backend directly. When using t.Parallel(...) in these top-level
tests, we thus could race.

This removes the Commands global variable, making it a local variable
instead as nothing else appears to use it. We'll update Enterprise to
add in the Enterprise-specific commands to the existing list.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-31 08:18:21 -05:00
Scott Miller
cd70976945 Transit Import Key CLI functionality (#18887)
* wip

* Transit byok cli

* It works!

* changelog

* document return codes

* Update command/transit_import_key.go

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* make fmt

---------

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-01-30 12:46:57 -06:00
miagilepner
097bd046e0 VAULT-12833 Update prompts for the rekey command (#18892)
* update prompts for rekey command

* cleanup additional places with unseal/recovery keys
2023-01-30 16:51:01 +00:00
Kit Haines
e8c5f2aea0 Switch ed25519 keytype to ec keytype to make the fips build happy. (#18889) 2023-01-30 09:08:25 -05:00
Kit Haines
958a1c2a18 Vault 11798 vault cli issue intermediate (#18467)
* The verify-sign command in it's cleanest existing form.

* Working state

* Updates to proper verification syntax

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>

* make fmt

* Git CI caught some stuff.

* Base functionality.

* make fmt; changelog

* pki issue command.

* Make fmt. Changelog.

* Error Handling Is Almost A Tutorial

* What I thought empty issuers response fix would be.

* Some tests

* PR-review updates.

* make fmt.

* Fix null response data for listing empty issuers causing a crash.

* Update command/pki_list_children_command.go

Fix double specifier

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Add test for pki_list_children.

* Fix tests.

* Update descriptions for correctness based on PR reviews.

* make fmt.

* Updates based on PR feedback.

* Allow multiple arguements (space separated)

* Remove bad merge-thing.

* White-space hell fix change.

* Tests, and return information for issue ca

* Fix make fmt error introduced here: https://github.com/hashicorp/vault/pull/18876

* Update command/pki_issue_intermediate.go

Puncutation.

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove smart quotes for standard quotes.

* More information as part of the help text.

* Better help text.

* Add missing "/" into error message.

---------

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-01-27 16:41:16 -05:00
Alexander Scheel
722c1c1d7d Small cleanup pki commands (#18877)
* Reflow pki list-intermediates help text, add args

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Reflow pki verify-sign help text, add args

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Simplify boolean checks across PKI commands

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clean up pki list-intermediate arg text

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rename list_children->list_intermediate to align with command

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-27 16:37:42 +00:00
Kit Haines
416f8bd898 Vault 11796 vault cli list intermediates (#18463)
* Base functionality.

* make fmt; changelog

* What I thought empty issuers response fix would be.

* Fix null response data for listing empty issuers causing a crash.

* Update command/pki_list_children_command.go

Fix double specifier

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Add test for pki_list_children.

* Fix tests.

* Update descriptions for correctness based on PR reviews.

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-01-27 10:34:31 -05:00
Alexander Scheel
9d17887dce Shadow globally defined var to avoid race (#18856)
* Shadow globally defined var to avoid race

output is defined in format_test.go as a global variable, which the
login_test.go tests were unknowingly using. Since these tests execute in
parallel, under the correct circumstances, they'd race to write/read
from the same variable in separate tests.

Shadow to avoid the race.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove global output variable from tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-26 16:47:42 +00:00
Kit Haines
be967f26f6 Vault 11795 vault cli verify s ign (#18437)
* The verify-sign command in it's cleanest existing form.

* Working state

* Updates to proper verification syntax

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>

* make fmt

* Git CI caught some stuff.

* Some tests

* PR-review updates.

* make fmt.

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
2023-01-26 10:21:13 -05:00
Peter Wilson
37b1ab12d3 Parallel migration (#18815) (#18817)
* Parallel migration (#18815)
* flagParallel sanity check
* Attempt to use ErrGroups
* Updated docs
* Allow 'start' and 'max-parallel' together
* parallel flag renamed to max-parallel
* tests for start + parallel
* Removed permit pool
* Updated docs to make it clearer that a high setting might not be honored based on storage backend setting
* System dependent max int size
* Default max-parallel 1 => 10
* Test folder/paths updated

Co-authored-by: Tomasz Pawelczak <10206601+gites@users.noreply.github.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
2023-01-25 15:19:45 +00:00
Violet Hynes
17be1024e4 VAULT-12564 Add new token_file auto-auth method (#18740)
* VAULT-12564 Work so far on token file auto-auth

* VAULT-12564 remove lifetime watcher struct modifications

* VAULT-12564 add other config items, and clean up

* VAULT-12564 clean-up and more tests

* VAULT-12564 clean-up

* VAULT-12564 lookup-self and some clean-up

* VAULT-12564 safer client usage

* VAULT-12564 some clean-up

* VAULT-12564 changelog

* VAULT-12564 some clean-ups

* VAULT-12564 batch token warning

* VAULT-12564 remove follow_symlink reference

* VAULT-12564 Remove redundant stat, change temp file creation

* VAULT-12564 Remove ability to delete token after auth
2023-01-24 16:09:32 -05:00
Scott Miller
2ffe49aab0 Add the ability to unseal using recovery keys via an explicit seal option. (#18683)
* wip

* wip

* Got it 'working', but not happy about cleanliness yet

* Switch to a dedicated defaultSeal with recovery keys

This is simpler than trying to hijack SealAccess as before.  Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal.  Then everything proceeds
as if you had a shamir seal to begin with.

* Handle recovery rekeying

* changelog

* Revert go.mod redirect

* revert multi-blob info

* Dumb nil unmarshal target

* More comments

* Update vault/seal.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* pr feedback

* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split

* Better comment on recovery seal during adjustSealMigration

* Make it possible to migrate from an auto-seal in recovery mode to shamir

* Fix sealMigrated to account for a recovery seal

* comments

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Address PR feedback

* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate

* Don't shortcut the reast of seal migration

* get rid of redundant transit server cleanup

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-24 14:57:56 -06:00
Hamid Ghaf
e18fd32946 named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Alexander Scheel
ebcd653102 Remove t.Parallel() due to initialization race (#18751)
Using RunCommand(...) to set format to JSON for PKI HC tests results in
multiple initCommands(...) being called, overwriting the same global
variable. Nobody else calls the test suite in this way, so remove
t.Parallel() as the CLI isn't really meant to be called in parallel and
there might be other issues.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-18 10:40:35 -05:00
Anton Averchenkov
ef3e3eace2 Remove timeout logic from ReadRaw functions and add ReadRawWithContext (#18708)
Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:

- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`

The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):

> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.

Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.

Resolves: #18658
2023-01-17 15:41:59 -05:00
Tom Proctor
e36690e067 Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Violet Hynes
819c9e0e2e VAULT-12542 Add info encouraging users to upgrade if agent version is different to server (#18684)
* VAULT-12542 Add info encouraging users to upgrade if agent version is different to server

* VAULT-12542 Changelog

* VAULT-12542 Language update
2023-01-13 09:49:36 -05:00
Peter Wilson
8abcde7cbb VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Ellie
49da2544ce add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Alexander Scheel
c7a696457d Add pki health-check docs (#18517)
* Add documentation on vault pki health-check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refer users to online docs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-11 11:46:30 -05:00
Peter Wilson
1fca38a4ae VAULT-9883: Agent Reloadable Config (#18638)
* Update command/agent.go
* Attempt to only reload log level and certs
* Mimicked 'server' test for cert reload in 'agent'

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Left out the `c.config` tweak that meant changes to lots of lines of code within the `Run` function of Agent command. :)
2023-01-10 17:45:34 +00:00
Mike Palmiotto
31772c930b Mark deprecated builtins Removed (#18039)
* Remove logical database builtins

* Drop removed builtins from registry keys

* Update plugin prediction test

* Remove app-id builtin

* Add changelog
2023-01-09 09:16:35 -05:00
Violet Hynes
c2abcccb94 VAULT-12095 Support multiple config files for Vault Agent (#18403)
* VAULT-12095 Code changes for multi-config

* VAULT-12095 typo

* VAULT-12095 make vault non-nil during update

* VAULT-12095 docs

* VAULT-12095 small refactor

* VAULT-12095 typos
2023-01-03 12:50:19 -05:00
Alexander Scheel
c2620e136d Add PKI base command (#18512)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-21 10:01:57 -05:00
divyaac
8ad46e0875 OSS PR for Config Changes PR (#18418)
* OSS PR for Config Changes PR

* Edited tests

* typo

* Added changelog

* Remove changelog
2022-12-15 12:19:19 -08:00
Mike Palmiotto
82f998f071 plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Mike Palmiotto
57017b626e command/audit: Recommend multiple audit devices (#18348)
* command/audit: Add note about enabling multiple audit devices

* docs: Recommend multiple audit devices
2022-12-13 17:51:03 -05:00
Chris Capurso
082e04d199 Add HCP connectivity info to vault server startup logs (#18315)
* add HCP org, proj, and resource ID to server startup logs

* add server config test for cloud stanza

* add changelog entry
2022-12-12 14:01:02 -05:00
Sanad Haj Yahya
835e3ed78d Server: add and support unix listener (UDS) (#18227)
Co-authored-by: shaj13 <hajsanad@gamil.com>
2022-12-09 12:28:18 -08:00
Chris Capurso
186ee31b10 Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Nick Cabatoff
35df9489c7 Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Violet Hynes
672cdc0fdb VAULT-11510 Vault Agent can start listeners without caching (#18137)
* VAULT-11510 Vault Agent can start listeners without caching

* VAULT-11510 fix order of imports

* VAULT-11510 changelog

* VAULT-11510 typo and better switch

* VAULT-11510 update name

* VAULT-11510 New api_proxy stanza to configure API proxy

* VAULT-11510 First pass at API Proxy docs

* VAULT-11510 nav data

* VAULT-11510 typo

* VAULT-11510 docs update
2022-12-05 10:51:03 -05:00