Updates the transitive dependency github.com/cloudflare/circl which will help address scanning noise related to CVE-2023-1732.
Vault is not affected by this issue as it does not use the vulnerable functionality.
The operation ID for `GET auth/token/lookup` has been overridden in
a way which *implies* it can only perform lookup-self operations.
This is not true, as shown:
1) Here is my current token:
```
$ vault read -field=display_name auth/token/lookup
userpass-admin
```
2) I create a second token, with a different display name to distinguish
it:
```
$ othertoken=$(vault token create -field=token -display-name=testing123)
```
3) I use this operation, still authenticated with the first token, to
lookup the second token, proving it does not have lookup-self nature:
```
$ vault read -field=display_name auth/token/lookup token=$othertoken
token-testing123
```
I have set the new operation ID suffix to `"2"` adhering to current
conventions in `vault-client-go` and `vault-client-dotnet` for secondary
versions of APIs which should be excluded from the generated library
code.
* Fix `vault path-help` for selected paths with bad regexps
See the comment being added in `sdk/framework/path.go` for the
explanation of why this change is needed.
* Grammar fix and add changelog
* Also fix hardcoded expectations in a new test
* Add a couple more testcases, and some comments.
* Tweak spelling in comment
* address all h1 title is-3 headings
* fix mult-factor and tests
* qa'ing
* clean up
* clean up
* clean up
* fix tests
* fix test
* one more
* okay this is the last one.
* blah
* Fix OpenAPI spec definitions for PKI EAB APIs
- Do not generate duplicate operation ids for the various new-eab apis
- Fill out proper operation verb for eab delete call
- Pluralize operation verb for list-eab-keys api
- Fill out proper response data for new-eab and list-eab-keys
* Add cl
Add an updated `target_ec2_instances` module that is capable of
dynamically splitting target instances over subnet/az's that are
compatible with the AMI architecture and the associated instance type
for the architecture. Use the `target_ec2_instances` module where
necessary. Ensure that `raft` storage scenarios don't provision
unnecessary infrastructure with a new `target_ec2_shim` module.
After a lot of trial, the state of Ec2 spot instance capacity, their
associated APIs, and current support for different fleet types in AWS
Terraform provider, have proven to make using spot instances for
scenario targets too unreliable.
The current state of each method:
* `target_ec2_fleet`: unusable due to the fact that the `instant` type
does not guarantee fulfillment of either `spot` or `on-demand`
instance request types. The module does support both `on-demand` and
`spot` request types and is capable of bidding across a maximum of
four availability zones, which makes it an attractive choice if the
`instant` type would always fulfill requests. Perhaps a `request` type
with `wait_for_fulfillment` option like `aws_spot_fleet_request` would
make it more viable for future consideration.
* `target_ec2_spot_fleet`: more reliable if bidding for target instances
that have capacity in the chosen zone. Issues in the AWS provider
prevent us from bidding across multiple zones succesfully. Over the
last 2-3 months target capacity for the instance types we'd prefer to
use has dropped dramatically and the price is near-or-at on-demand.
The volatility for nearly no cost savings means we should put this
option on the shelf for now.
* `target_ec2_instances`: the most reliable method we've got. It is now
capable of automatically determing which subnets and availability
zones to provision targets in and has been updated to be usable for
both Vault and Consul targets. By default we use the cheapest medium
instance types that we've found are reliable to test vault.
* Update .gitignore
* enos/modules/create_vpc: create a subnet for every availability zone
* enos/modules/target_ec2_fleet: bid across the maximum of four
availability zones for targets
* enos/modules/target_ec2_spot_fleet: attempt to make the spot fleet bid
across more availability zones for targets
* enos/modules/target_ec2_instances: create module to use
ec2:RunInstances for scenario targets
* enos/modules/target_ec2_shim: create shim module to satisfy the
target module interface
* enos/scenarios: use target_ec2_shim for backend targets on raft
storage scenarios
* enos/modules/az_finder: remove unsed module
Signed-off-by: Ryan Cragun <me@ryan.ec>
* openapi: Fix response schema for PKI Issue requests
* tests
* changelog
* another expiration for generate/rotate root
* more type fixes from @stevendpclark
Vault API endpoints are defined using regexes in instances of the SDK's
framework.Path structure. However, OpenAPI does not use regexes, so a
translation is performed. It is technically possible that this
translation produces colliding OpenAPI paths from multiple
framework.Path structures. When this happens, there has formerly been no
diagnostic, and one result silently overwrites the other in a map.
As a result of this, several operations are currently accidentally
missing from the Vault OpenAPI, which is also the trigger for
https://github.com/hashicorp/vault-client-go/issues/180.
This PR adds a log message, to help catch such accidents so that they
can be fixed. Much of the PR is propagating a logger to the point where
it is needed, and adjusting tests for the API change.
With current Vault, this will result in the following being logged each
time a request is made which triggers OpenAPI generation:
```
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/totp
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/okta
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/duo
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/pingid
```
I will submit a further PR to fix the issue - this one is just to add
the diagnostic.
* Refactor `identity/mfa/method/*` endpoints to fix bad OpenAPI
There is a problem with how the `identity/mfa/method/*` endpoints are
defined, resulting in incorrect OpenAPI generation.
I raised hashicorp/vault-client-go#180 to track a consequence, and
opened #20873 which explains the problem and adds a log message to
detect it.
This PR is now the fix.
It's actually quite an interesting problem, that has come about through
some particular implementation choices, in Vault's first/only case where
REST API objects are created by writing to the collection URL, and have
their ID allocated by the server, instead of the client.
The triggering cause of the malfunction was trying to have a single
framework.Path struct instance which optionally includes or excludes the
method_id path parameter, and also another framework.Path struct
instance handling list operations.
The fix is to simplify the path regexes, and have one framework.Path
which handles the method_id being present, and one that handles it being
absent.
The diff is somewhat large, because the affected code had been
copy/pasted four times (TOTP, Okta, Duo, PingID) - so I took the
opportunity to fix the duplication, creating appropriate helper methods
so that the quadruplicated code could be re-unified.
* Revise documentation
This update refactors how the documentation presents these endpoints to
users, both for clarity, and to align with the new structure of the
code.
From a user perspective, it clears up some unclear presentation of when
the `method_id` parameter should and should not be present, adds
a missing description of the response to create requests, and changes
the `method_id` parameter name to be used consistently (rather than `id`
in some cases, unlike the actual code/OpenAPI).
* Fix incorrect acronym (review fix)
* Accept suggestion of tweaked grammar in documentation
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Add changelog
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
We seem to hit occasional capacity issues when attempting to launch spot
fleets with arm64 instance types. After checking pricing in the regions
that we use, it appears that current and older generation amd64 t2 and
t3 instance types are running at quite a discount whereas t4 arm64
instances are barely under on-demand price, suggesting limited capacity
for arm64 spot instances at this time. We'll change our default backend
instance architecture to amd64 to bid for the cheaper t2 and t3
instances and increase our `max_price` globally to that of a RHEL
machine running on-demand with a t3.medium.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Begin the process of migrating away from the "strongly encouraged not to
use"[0] Ec2 spot fleet API to the more modern `ec2:CreateFleet`.
Unfortuantely the `instant` type fleet does not guarantee fulfillment
with either on-demand or spot types. We'll need to add a feature similar
to `wait_for_fulfillment` on the `spot_fleet_request` resource[1] to
`ec2_fleet` before we can rely on it.
We also update the existing target fleets to support provisioning generic
targets. This has allowed us to remove our usage of `terraform-enos-aws-consul`
and replace it with a smaller `backend_consul` module in-repo.
We also remove `terraform-enos-aws-infra` and replace it with two smaller
in-repo modules `ec2_info` and `create_vpc`. This has allowed us to simplify
the vpc resources we use for each scneario, which in turn allows us to
not rely on flaky resources.
As part of this refactor we've also made it possible to provision
targets using different distro versions.
[0] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html#which-spot-request-method-to-use
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/spot_fleet_request#wait_for_fulfillment
* enos/consul: add `backend_consul` module that accepts target hosts.
* enos/target_ec2_spot_fleet: add support for consul networking.
* enos/target_ec2_spot_fleet: add support for customizing cluster tag
key.
* enos/scenarios: create `target_ec2_fleet` which uses a more modern
`ec2_fleet` API.
* enos/create_vpc: replace `terraform-enos-aws-infra` with smaller and
simplified version. Flatten the networking to a single route on the
default route table and a single subnet.
* enos/ec2_info: add a new module to give us useful ec2 information
including AMI id's for various arch/distro/version combinations.
* enos/ci: update service user role to allow for managing ec2 fleets.
Signed-off-by: Ryan Cragun <me@ryan.ec>
