Commit Graph

692 Commits

Author SHA1 Message Date
Vishal Nayak
0beaa439d6 Remove unneeded fields from state output (#11073) 2021-03-10 12:08:12 -05:00
Vishal Nayak
415890e79c Autopilot: Server Stabilization, State and Dead Server Cleanup (#10856)
* k8s doc: update for 0.9.1 and 0.8.0 releases (#10825)

* k8s doc: update for 0.9.1 and 0.8.0 releases

* Update website/content/docs/platform/k8s/helm/configuration.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Autopilot initial commit

* Move autopilot related backend implementations to its own file

* Abstract promoter creation

* Add nil check for health

* Add server state oss no-ops

* Config ext stub for oss

* Make way for non-voters

* s/health/state

* s/ReadReplica/NonVoter

* Add synopsis and description

* Remove struct tags from AutopilotConfig

* Use var for config storage path

* Handle nin-config when reading

* Enable testing autopilot by using inmem cluster

* First passing test

* Only report the server as known if it is present in raft config

* Autopilot defaults to on for all existing and new clusters

* Add locking to some functions

* Persist initial config

* Clarify the command usage doc

* Add health metric for each node

* Fix audit logging issue

* Don't set DisablePerformanceStandby to true in test

* Use node id label for health metric

* Log updates to autopilot config

* Less aggressively consume config loading failures

* Return a mutable config

* Return early from known servers if raft config is unable to be pulled

* Update metrics name

* Reduce log level for potentially noisy log

* Add knob to disable autopilot

* Don't persist if default config is in use

* Autopilot: Dead server cleanup (#10857)

* Dead server cleanup

* Initialize channel in any case

* Fix a bunch of tests

* Fix panic

* Add follower locking in heartbeat tracker

* Add LastContactFailureThreshold to config

* Add log when marking node as dead

* Update follower state locking in heartbeat tracker

* Avoid follower states being nil

* Pull test to its own file

* Add execution status to state response

* Optionally enable autopilot in some tests

* Updates

* Added API function to fetch autopilot configuration

* Add test for default autopilot configuration

* Configuration tests

* Add State API test

* Update test

* Added TestClusterOptions.PhysicalFactoryConfig

* Update locking

* Adjust locking in heartbeat tracker

* s/last_contact_failure_threshold/left_server_last_contact_threshold

* Add disabling autopilot as a core config option

* Disable autopilot in some tests

* s/left_server_last_contact_threshold/dead_server_last_contact_threshold

* Set the lastheartbeat of followers to now when setting up active node

* Don't use config defaults from CLI command

* Remove config file support

* Remove HCL test as well

* Persist only supplied config; merge supplied config with default to operate

* Use pointer to structs for storing follower information

* Test update

* Retrieve non voter status from configbucket and set it up when a node comes up

* Manage desired suffrage

* Consider bucket being created already

* Move desired suffrage to its own entry

* s/DesiredSuffrageKey/LocalNodeConfigKey

* s/witnessSuffrage/recordSuffrage

* Fix test compilation

* Handle local node config post a snapshot install

* Commit to storage first; then record suffrage in fsm

* No need of local node config being nili case, post snapshot restore

* Reconcile autopilot config when a new leader takes over duty

* Grab fsm lock when recording suffrage

* s/Suffrage/DesiredSuffrage in FollowerState

* Instantiate autopilot only in leader

* Default to old ways in more scenarios

* Make API gracefully handle 404

* Address some feedback

* Make IsDead an atomic.Value

* Simplify follower hearbeat tracking

* Use uber.atomic

* Don't have multiple causes for having autopilot disabled

* Don't remove node from follower states if we fail to remove the dead server

* Autopilot server removals map (#11019)

* Don't remove node from follower states if we fail to remove the dead server

* Use map to track dead server removals

* Use lock and map

* Use delegate lock

* Adjust when to remove entry from map

* Only hold the lock while accessing map

* Fix race

* Don't set default min_quorum

* Fix test

* Ensure follower states is not nil before starting autopilot

* Fix race

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-03-03 13:59:50 -05:00
Andrej van der Zee
77a5ca59e0 Respect VAULT_MAX_RETRIES from environment in DefaultConfig() (#10883) 2021-03-02 10:39:20 -08:00
Scott Miller
e5316a9e2f OSS side barrier encryption tracking and automatic rotation (#11007)
* Automatic barrier key rotation, OSS portion

* Fix build issues

* Vendored version

* Add missing encs field, not sure where this got lost.
2021-02-25 14:27:25 -06:00
Ace Eldeib
c2c6ccd764 don't panic on connection errors in DefaultRetryPolicy (#11002)
fixes #11001
2021-02-25 13:16:17 -05:00
Nick Cabatoff
22b486b651 OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Vishal Nayak
405eced084 Revert "Read-replica instead of non-voter (#10875)" (#10890)
This reverts commit fc745670cf.
2021-02-10 16:41:58 -05:00
Vishal Nayak
fc745670cf Read-replica instead of non-voter (#10875) 2021-02-10 09:58:18 -05:00
Josh Black
a9191bcb36 Only set the namespace if the env var isn't present (#1519) (#10556) 2020-12-14 11:40:48 -08:00
Brian Kassouf
9f5babf584 core: Record the time a node became active (#10489)
* core: Record the time a node became active

* Update vault/core.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Add omitempty field

* Update vendor

* Added CL entry and fixed test

* Fix test

* Fix command package tests

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2020-12-11 16:50:19 -08:00
Michel Vocks
26943b7317 Add AllowedHeader and fix AllowedOrigins in the API (#10482)
* Add AllowedHeader and fix AllowedOrigins in the API

* Update changelog entry

* Fix wrong response conversion
2020-12-03 14:31:25 +01:00
Josh Black
0fcbea5140 Fix client.Clone() to include the address. (#10077) 2020-11-06 11:27:35 -08:00
Aleksandr Bezobchuk
e4421691da Merge PR #10192: Auto-Join: Configurable Scheme & Port (and add k8s provider) 2020-10-23 16:13:09 -04:00
Aleksandr Bezobchuk
b67da26279 Merge PR #10095: Integrated Storage Cloud Auto-Join 2020-10-13 16:26:39 -04:00
Josh Black
245e4b428b Add README.md to api (#10088) 2020-10-06 14:27:59 -07:00
Scott Miller
7a875e2454 Revert unnecessary changes to api/go.mod (#10097)
* restore api's go.mod

* sdk too

* root level go mod vendor
2020-10-06 14:25:46 -05:00
Scott Miller
bf8d7efb07 Expose generic versions of KDF and symmetric crypto (#10076)
* Support salt in DeriveKey

* Revert "Support salt in DeriveKey"

This reverts commit b295ae42673308a2d66d66b53527c6f9aba92ac9.

* Refactor out key derivation, symmetric encryption, and symmetric decryption into generic functions

* comments

* comments

* go mod vendor

* bump both go.mods

* This one too

* bump

* bump

* bump

* Make the lesser used params of symmetric ops a struct

* go fmt

* Call GetKey instead of DeriveKey

* Address feedback

* Wrong rv

* Rename calls

* Assign the nonce field

* trivial change

* Check nonce len instead

* go mod vendor
2020-10-01 21:04:36 -05:00
Hridoy Roy
d5c20be2dc Normalize format output for vault status [VAULT-508] (#9976)
* normalize format output for vault status

* interim commit

* interim commit

* make formatting idiomatic

* clean up comments

* added formatting test

* updated comments in format test to match godocs

Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MBP.hitronhub.home>
Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MacBook-Pro.local>
2020-09-23 10:30:01 -07:00
ncabatoff
49f2a79a53 Upgrade to newer okta lib for pagination, fetch all groups using it (#9580) 2020-07-24 09:05:08 -04:00
Jeff Mitchell
69d50406e9 Revert "Revert "Get rid of sdk internal reloadutil package for now. (#9515)" (#9521)" (#9522)
This reverts commit f687267c80.
2020-07-17 15:42:41 -04:00
Jeff Mitchell
f687267c80 Revert "Get rid of sdk internal reloadutil package for now. (#9515)" (#9521)
This reverts commit fa39e7bce0.
2020-07-17 15:18:44 -04:00
ncabatoff
fa39e7bce0 Get rid of sdk internal reloadutil package for now. (#9515) 2020-07-17 15:16:15 -04:00
ncabatoff
99564f52d0 Update sentinel dependencies. (#9427) 2020-07-08 15:04:11 -04:00
Scott Miller
65dd5cfd27 Address feedback on Plugin Reload: OSS Side (#9350)
* just use an error string

* Switch command to use new struct
2020-06-30 16:26:38 -05:00
Scott Miller
0be92206d7 Fix wrong err return value in plugin reload status command (#9348)
* Fix wrong return value (discovered when merging to ENT)

* go.mod

* go mod vendor

* Add setup plugin reload hook

* All reloads return something now
2020-06-30 13:33:30 -05:00
Scott Miller
10ef01b1bf Global Plugin Reload: OSS Changes Take II (#9347)
* Carefully move changes from the plugin-cluster-reload branch into this clean branch off master.

* Don't test this at this level, adequately covered in the api level tests

* Change PR link

* go.mod

* Vendoring

* Vendor api/sys_plugins.go
2020-06-30 10:26:52 -05:00
Scott Miller
d0bbb081c3 Revert global plugin reload commits (#9344)
* Revert "Some of the OSS changes were clobbered when merging with quotas out of, master (#9343)"

This reverts commit 8719a9b7c4.

* Revert "OSS side of Global Plugin Reload (#9340)"

This reverts commit f98afb998a.
2020-06-29 17:36:22 -05:00
Scott Miller
8719a9b7c4 Some of the OSS changes were clobbered when merging with quotas out of, master (#9343)
* OSS side of Global Plugin Reload
2020-06-29 16:58:51 -05:00
Scott Miller
f98afb998a OSS side of Global Plugin Reload (#9340)
* OSS side of Global Plugin Reload

* changelog++
2020-06-29 16:23:28 -05:00
Vishal Nayak
c68e270863 Resource Quotas: Rate Limiting (#9330) 2020-06-26 17:13:16 -04:00
Clint
e926896fc7 VLT091 plugin testing framework stepwise (#9270)
* Resolve merge conflicts and updates from running a test

* move testing/_test.go over to legacy

* updates

* Add core of plugin test framework Stepwise  (#9166)

* adding stepwise testing, but there are protocol buff error :/

* move file and update sdk/go.mo

* update/sync modules

* update from other branch

* update sdk/go.mod

* some cleanups after feedback

* remove enviornments from this PR

* update vendor

* change from running go mod tidy

* change from go mod tidy

* Update sdk/testing/stepwise/helpers.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Update sdk/testing/stepwise/helpers.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* change panic to error

* Update sdk/testing/stepwise/helpers.go

return `nil` and not `err` at the end

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* Defer close() on successful Open of a file

* document the re-creation of steps

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* remove unused BarrierKeys()

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* updates from feedback

* fix return with bad arguments

* Rename things:

- StepOperation -> Operation
- StepwiseEnvironment -> Environment
- StepCheckFunc -> AssertionFunc
- step.Check -> step.Assert

* document the environment interface methods

* rename EnvironmentOptions to MountOptions

* rename Name to RegistryName

* remove ExpectError because it's redundant

* minor doc update

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* add checkShouldRun function

* remove redundant return

* remove vestigial PreCheck function

* add tt.Helper() to makeRequest

* minor code formatting and document 1-based index for log output of Steps

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>

* minor updates

* update sdk

* use local reference for api, vault dep

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk/testing/stepwise/stepwise.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* cleanup some defer functions

* call fatal if environment setup fails, and don't call teardown

* defer re-setting client token in makeRequest

* Move legacy logicaltest back to testhelpers

* update mods and test files with go mod tidy

* go mod vendor

* remove relative replace directives

* restore old logical test location

* move declaration to main stepwise file

* remove index var and use i+1

* add testing for write, delete paths of makeRequest

* update stepwise core testing to do request counting

* remove unused methods

* Update sdk/testing/stepwise/stepwise.go

remove dead line

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update sdk/testing/stepwise/stepwise.go

fix capitalization in code comment

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* update code comments for SkipTeardown to clarify its use

* update stepwise

Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
2020-06-23 06:01:39 -05:00
Michael Golowka
1508a3b12b Move sdk/helper/random -> helper/random (#9226)
* This package is new for 1.5 so this is not a breaking change.
* This is being moved because this code was originally intended to be used
within plugins, however the design of password policies has changed such
that this is no longer needed. Thus, this code doesn't need to be in the
public SDK.
2020-06-17 14:24:38 -06:00
Austin Gebauer
f0b848ed96 Update oracle cloud infrastructure auth plugin to v0.5.5 (#9210) 2020-06-15 10:11:20 -07:00
Jim Kalafut
eb666e94bd Add namespace parameter to ssh helper config (#9160) 2020-06-08 08:16:03 -07:00
Billie Cleek
62d0ecff3d document response wrapping behavior (#8156)
Document response wrapping behavior so that it's clear how
WrappingLookupFuncs should behave.
2020-06-08 10:50:48 -04:00
Michael Golowka
ad90e0b39d Add user configurable password policies available to secret engines (#8637)
* Add random string generator with rules engine

This adds a random string generation library that validates random
strings against a set of rules. The library is designed for use as generating
passwords, but can be used to generate any random strings.
2020-05-27 12:28:00 -06:00
Josh Black
af5338b485 Add a new "vault monitor" command (#8477)
Add a new "vault monitor" command

Co-authored-by: ncabatoff <ncabatoff@hashicorp.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2020-05-21 13:07:50 -07:00
Jeff Mitchell
e0cfd64bc2 Bump api's sdk 2020-05-19 18:18:38 -04:00
Brian Kassouf
b8799b3358 storage/raft: Add committed and applied indexes to the status output (#9011)
* storage/raft: Add committed and applied indexes to the status output

* Update api vendor

* changelog++

* Update http/sys_leader.go

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-05-18 16:07:27 -07:00
Jeff Mitchell
89b1987e38 Update sdk in go.mod files 2020-05-14 12:42:14 -04:00
Jeff Mitchell
91b09c09b5 Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Jeff Mitchell
81afb3fdd0 Update comment in client about canceling the WithTimeout context 2020-05-08 14:48:26 -04:00
Vladimir Dimitrov
52cadd32d9 Added plugin reload function to api (#8777)
* Added plugin reload function to api

* Apply suggestions from code review

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* go mod vendor

* addressing comments

* addressing comments

* add docs

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-05-04 11:14:23 -07:00
Scott Miller
8571221f03 Handle request errors during raft snapshot (#8861)
* Handle request errors during raft snapshot
2020-04-27 13:39:06 -05:00
Daniel Spangenberg
8dd6180e31 Fix SRV Lookups (#8520)
* Pin HTTP Host header for all client requests
* Drop port map scheme
* Add SRV Lookup environment var
* Lookup SRV records only when env var is specified
* Add docs

Co-Authored-By: Michel Vocks <michelvocks@gmail.com>
2020-03-11 14:22:58 +01:00
Jeff Mitchell
26a747400a Bump API's SDK 2020-02-15 14:57:06 -05:00
Jeff Mitchell
c39f5634b3 Update api's sdk 2020-02-14 17:27:43 -05:00
Jeff Mitchell
b215d1c322 Update API's sdk 2020-02-13 10:40:35 -05:00
Michel Vocks
b52049a749 Add Consul TLS options to access API endpoint (#8253) 2020-01-29 09:44:35 +01:00
Michel Vocks
267665ecc0 Fix redoing redirect response raft snapshot cli (#8211)
* Fix redoing redirect response raft snapshot cli

* Removed unnecessary lines of code

* go mod vendor
2020-01-27 11:25:52 +01:00