Commit Graph

620 Commits

Author SHA1 Message Date
Alexander Scheel
d234111801 Start counting ACME certificate issuance as client activity (#20520)
* Add stub ACME billing interfaces

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add initial implementation of client count

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly attribute to mount, namespace

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor adding entities of custom types

This begins to add custom types of events; presently these are counted
as non-entity tokens, but prefixed with a custom ClientID prefix.

In the future, this will be the basis for counting these events
separately (into separate buckets and separate storage segments).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor creation of ACME mounts

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case for billing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Better support managed key system view casting

Without an additional parameter, SystemView could be of a different
internal implementation type that cannot be directly casted to in OSS.
Use a separate parameter for the managed key system view to use instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor creation of mounts for enterprise

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Validate mounts in ACME billing tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use a hopefully unique separator for encoded identifiers

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use mount accesor, not path

Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rename AddEventToFragment->AddActivityToFragment

Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
2023-05-17 16:12:04 +00:00
Violet Hynes
6b4b0f7aaf VAULT-15547 First pass at agent/proxy decoupling (#20548)
* VAULT-15547 First pass at agent/proxy decoupling

* VAULT-15547 Fix some imports

* VAULT-15547 cases instead of string.Title

* VAULT-15547 changelog

* VAULT-15547 Fix some imports

* VAULT-15547 some more dependency updates

* VAULT-15547 More dependency paths

* VAULT-15547 godocs for tests

* VAULT-15547 godocs for tests

* VAULT-15547 test package updates

* VAULT-15547 test packages

* VAULT-15547 add proxy to test packages

* VAULT-15547 gitignore

* VAULT-15547 address comments

* VAULT-15547 Some typos and small fixes
2023-05-17 09:38:34 -04:00
Gabriel Santos
dae5cf13c4 Provide public key encryption via transit engine (#17934)
* import rsa and ecdsa public keys

* allow import_version to update public keys - wip

* allow import_version to update public keys

* move check key fields into func

* put private/public keys in same switch cases

* fix method in UpdateKeyVersion

* move asymmetrics keys switch to its own method - WIP

* test import public and update it with private counterpart

* test import public keys

* use public_key to encrypt if RSAKey is not present and failed to decrypt
if key version does not have a private key

* move key to KeyEntry parsing from Policy to KeyEntry method

* move extracting of key from input fields into helper function

* change back policy Import signature to keep backwards compatibility and
add new method to import private or public keys

* test import with imported public rsa and ecdsa keys

* descriptions and error messages

* error messages, remove comments and unused code

* changelog

* documentation - wip

* suggested changes - error messages/typos and unwrap public key passed

* fix unwrap key error

* fail if both key fields have been set

* fix in extractKeyFromFields, passing a PolicyRequest wouldn't not work

* checks for read, sign and verify endpoints so they don't return errors when a private key was not imported and tests

* handle panic on "export key" endpoint if imported key is public

* fmt

* remove 'isPrivateKey' argument from 'UpdateKeyVersion' and
'parseFromKey' methods

also: rename 'UpdateKeyVersion' method to 'ImportPrivateKeyForVersion' and 'IsPublicKeyImported' to 'IsPrivateKeyMissing'

* delete 'RSAPublicKey' when private key is imported

* path_export: return public_key for ecdsa and rsa when there's no private key imported

* allow signed data validation with pss algorithm

* remove NOTE comment

* fix typo in EC public key export where empty derBytes was being used

* export rsa public key in pkcs8 format instead of pkcs1 and improve test

* change logic on how check for is private key missing is calculated

---------

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-11 11:56:46 +00:00
Pratyoy Mukhopadhyay
dee7fd839e add ns path to granting_policies (#20522) 2023-05-04 15:08:22 -07:00
Hamid Ghaf
f8ccaac856 adding support for four cluster docker based test scenario (#20492) 2023-05-03 10:49:45 -07:00
Alexander Scheel
a5bca6a122 Add dns resolver to PKI Binary Cluster (#20485)
* Export DockerAPI for use by other consumers

As usage of DockerCluster gets more advanced, some users may want to
interact with the container nodes of the cluster. While, if you already
have a DockerAPI instance lying around you can reuse that safely, for
use cases where an existing e.g., docker/testhelpers's runner instance
is not available, reusing the existing cluster's DockerAPI is easiest.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to exec commands without runner

When modifying DockerTestCluster's containers manually, we might not
have a Runner instance; instead, expose the ability to run commands via
a DockerAPI instance directly, as they're awfully convenient.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add DNS resolver into ACME tests

This updates the pkiext_binary tests to use an adjacent DNS resolver,
allowing these tests to eventually be extended to solve DNS challenges,
as modifying the /etc/hosts file does not allow this.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix loading DNS resolver onto network

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix bug with DNS configuration validation

Both conditionals here were inverted: address being empty means a bad
specification was given, and the parse being nil means that it was not a
valid IP address.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix specifying TXT records, allow removing records

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-03 17:32:39 +00:00
Nick Cabatoff
781d43e91c Simple perf replication test using docker containers (#20393) 2023-04-28 19:43:30 +00:00
Nick Cabatoff
a65597aa59 Add support for restarting docker nodes, and for providing args (#20409) 2023-04-28 14:03:14 -04:00
Alexander Scheel
37852757b0 Add custom DNS resolver to ACME configuration (#20400)
* Handle caching of ACME config

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add DNS resolvers to ACME configuration

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add custom DNS resolver to challenge verification

This required plumbing through the config, reloading it when necessary,
and creating a custom net.Resolver instance.

Not immediately clear is how we'd go about building a custom DNS
validation mechanism that supported multiple resolvers. Likely we'd need
to rely on meikg/dns and handle the resolution separately for each
container and use a custom Dialer that assumes the address is already
pre-resolved.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Improvements to Docker harness

 - Expose additional service information, allowing callers to figure out
   both the local address and the network-specific address of the
   service container, and
 - Allow modifying permissions on uploaded container files.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add infrastructure to run Bind9 in a container for tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Validate DNS-01 challenge works

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-27 15:30:29 -04:00
Nick Cabatoff
f4c881726f Don't require every test to specify a VaultLicense field (#20372) 2023-04-26 18:39:48 +00:00
Nick Cabatoff
445e2e91c6 Docker testing: handle licensing, different images per node (#20347) 2023-04-25 17:11:46 -04:00
Nick Cabatoff
2f0929f3ab Add support for docker testclusters (#20247) 2023-04-24 14:25:50 -04:00
Nick Cabatoff
980f1e0aae Add tests based on vault binary (#20224)
First steps towards docker-based tests: tests using vault binary in -dev or -dev-three-node modes.
2023-04-24 09:57:37 -04:00
Nick Cabatoff
bbc1b4cb44 Bump the sdk go version to 1.20, document when go versions should be updated (#20278) 2023-04-24 08:44:39 -04:00
Luis (LT) Carbonell
7f2deb1420 Add Configurable LDAP Max Page Size (#19032)
* Add config flag for LDAP max page size

* Add changelog

* move changelog to correct file

* cleanup

* Default to non-paged searching for with -1

* Update website/content/api-docs/auth/ldap.mdx

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update tests

---------

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2023-04-20 20:39:27 +00:00
Josh Black
d51ae039b2 OSS changes for new update-primary API endpoint (#20277)
* OSS changes for new update-primary API endpoint

* remove ENT specific piece

* remove another ENT specific field
2023-04-20 13:13:37 -07:00
Alexander Scheel
14ba8b292a Move OCSP/PKI interop test to PKI engine (#20273)
This prevents SDK from having a circular import on the main Vault
package.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-20 14:57:20 +00:00
claire bontempo
9afac14f08 UI: remove references to comma separation for string array edit types (#20163)
* remove intercepting helpText

* add subtext directly to StringList input component

* update tests and add coverage for new openapi-attrs util

* update test

* add warning validation to input

* lol is this right i dont know go

* literally no idea what im doing

* add Description to display attrs struct

* update struct comment

* add descriptions to remaining go fields

* add missing comma

* remaining commas..."

* add description to display attrs

* update tests

* update tests

* add changelog;

* Update ui/app/utils/openapi-to-attrs.js

* update tests following backend changes

* clearly name variable

* format files

* no longer need to test for modified tooltip since coming from backend now
2023-04-19 16:16:30 +00:00
Jason O'Donnell
d5584b614a sdk/ldap: update interface to use DialURL (#20200)
* sdk/ldap: update interface to use DialURL

* Fix scheme

* Fix race condition

* Add tls config dialopt
2023-04-17 16:34:10 -04:00
Alexander Scheel
17a2827894 Add fix for Go x/crypto/ocsp failure case (#20181)
* Add fix for Go x/crypto/ocsp failure case

When calling ocsp.ParseRequest(req, issue) with a non-nil issuer on a
ocsp request which _unknowingly_ contains an entry in the
BasicOCSPResponse's certs field, Go incorrectly assumes that the issuer
is a direct parent of the _first_ certificate in the certs field,
discarding the rest.

As documented in the Go issue, this is not a valid assumption and thus
causes OCSP verification to fail in Vault with an error like:

> bad OCSP signature: crypto/rsa: verification error

which ultimately leads to a cert auth login error of:

> no chain matching all constraints could be found for this login certificate

We address this by using the unsafe issuer=nil argument, taking on the
task of validating the OCSP response's signature as best we can in the
absence of full chain information on either side (both the trusted
certificate whose OCSP response we're verifying and the lack of any
additional certs the OCSP responder may have sent).

See also: https://github.com/golang/go/issues/59641

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case with Vault PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-17 11:48:19 -04:00
Anton Averchenkov
c6fc0033ee openapi: Better comments for OperationPrefix/Verb/Suffix (#20162) 2023-04-13 18:47:14 -04:00
Jason O'Donnell
2f7f0d2db9 sdk/ldaputil: add connection_timeout configurable (#20144)
* sdk/ldaputil: add connection_timeout configurable

* changelog

* Update doc

* Fix test

* Change default to 30s
2023-04-13 12:43:28 -04:00
Tom Proctor
8dca0f3767 Simplify tracking of external plugins (#20009) 2023-04-12 18:34:35 +01:00
Mike Palmiotto
08aa30ad72 Fix RFC3339 comment typo (#20100) 2023-04-11 20:11:05 +00:00
Thy Ton
0e9b3b061a feat: add plugin metadata to audit logging (#19814) 2023-04-06 00:41:07 -07:00
Anton Averchenkov
31d7bd3b70 Fix OpenAPI-related tests (#19983) 2023-04-04 18:27:20 -04:00
Anton Averchenkov
3fdb09a500 openapi: Improve operationId/request/response naming strategy (#19319) 2023-04-04 13:14:40 -04:00
Anton Averchenkov
f674f0ea32 Fix response structure validation tests for non-2xx responses (#19726) 2023-03-23 16:33:44 -04:00
Alexander Scheel
38fc3e79f2 Add support for HEAD operations (#19520)
* Add header operation to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for routing HEAD operations

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-21 18:37:17 -04:00
Anton Averchenkov
09d58d1858 openapi: Fix logic for labeling unauthenticated/sudo paths (#19600) 2023-03-20 13:25:09 -04:00
Tom Proctor
e6427b2b30 Suppress event broker not started log warning (#19593) 2023-03-20 11:14:14 +00:00
Hamid Ghaf
e55c18ed12 adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Anton Averchenkov
ce420de231 Ignore special HTTP fields in response validation tests (#19530) 2023-03-14 13:21:20 -04:00
Alexander Scheel
9e188973d9 Add support for importing RSA-PSS keys into Transit (#19519)
* Add support for importing RSA-PSS keys in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-13 17:03:01 +00:00
Tom Proctor
ea33318b4b sdk: Fix fmt + add FieldType test (#19493)
* sdk: Fix fmt + add FieldType test

* Add test comment
2023-03-10 18:07:48 +00:00
valli_0x
0bbeba1028 bug: correct sdk handling of the zero int64 value (#18729)
* bug: correct handling of the zero int64 value

* Update changelog/18729.txt

---------

Co-authored-by: valli_0x <personallune@mail.ru>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-03-09 11:25:45 +00:00
Austin Gebauer
4b9eb233ca ldaputil: adds comment on available text/template functions (#19469) 2023-03-06 19:38:42 -08:00
Jakob Beckmann
39f9e5e775 Allow alias dereferencing in LDAP searches (#18230)
* impr(auth/ldap): allow to dereference aliases in searches

* docs: add documentation for LDAP alias dereferencing

* chore(auth/ldap): add changelog entry for PR 18230

* chore: run formatter

* fix: update default LDAP configuration with new default

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

* docs(ldap): add alias dereferencing to API docs for LDAP

---------

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
2023-02-24 13:49:17 -05:00
Alexander Scheel
3adb416da1 Refactor OCSP client to support better retries (#19345)
Mirror NSS's GET-vs-POST selection criteria, wherein GET is preferred
over POST (as the former might be a response from a cached CDN entry,
whereas the latter might hit a live responder). However, only accept it
if it definitively says "Good" or "Revoked" -- trigger a POST request
when an unknown or failure status is seen.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-24 13:18:37 -05:00
Tom Proctor
4df7c64d05 Update x/net and x/crypto/ssh (#19282)
* Update x/net and x/crypto/ssh in api and sdk

* go mod tidy in root go module
2023-02-22 13:46:12 +00:00
Christopher Swenson
4944581a9c events: WS protobuf messages should be binary (#19232)
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.

We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.

We also bump the eventlogger to fix a race condition that this code triggers.
2023-02-17 11:38:03 -08:00
Scott Miller
add3659f39 Update go-kms-wrapping and it's gcpc wrapper to pick up google-cloud-go migration (#19219) 2023-02-16 11:21:25 -06:00
Daniel Huckins
ede32d5599 Validate response schema for integration tests (#19043)
* add RequestResponseCallback to core/options

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* pass in router and apply function on requests

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add callback

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* cleanup

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Update vault/core.go

* bad typo...

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use pvt interface, can't downcast to child struct

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* finer grained errors

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* trim path for backend

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* remove entire mount point instead of just the first part of url

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/testing.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* add doc string

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update docstring

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* reformat

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-15 14:57:57 -05:00
Tom Proctor
34b3d0406d Convert events metadata type to google.protobuf.Struct (#19130) 2023-02-10 18:58:03 +00:00
Christopher Swenson
6e233e567b events: Add websockets and command (#19057)
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.

This can be used by either `wscat` or the new
`vault events subscribe`:

e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

and

```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-09 13:18:58 -08:00
Scott Miller
91b458d8a6 Update specific wrappers to v2.0.7 (#19101)
* Update go-kms-wrapping to v2.0.7

* Update specific wrappers to v2.0.7

* Attempt to fix aead
2023-02-09 12:19:50 -06:00
Nick Cabatoff
e0d726ac22 Remove the last vestiges of sdk/version. (#19068) 2023-02-08 12:30:27 -05:00
Nick Cabatoff
192baa88db Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Christopher Swenson
80485f927b Add events sending routed from plugins (#18834)
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.

This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
2023-02-03 13:24:16 -08:00
Anton Averchenkov
6a9ca261d1 openapi: Add default values to the thing_mount_path parameters (#18935) 2023-01-31 19:37:16 -05:00