diff --git a/website/data/api-navigation.js b/website/data/api-navigation.js index 9a70627a6a..9115fa4d96 100644 --- a/website/data/api-navigation.js +++ b/website/data/api-navigation.js @@ -60,6 +60,7 @@ export default [ { category: 'rabbitmq' }, { category: 'ssh' }, { category: 'totp' }, + { category: 'transform' }, { category: 'transit' }, '-----------------------', { category: 'cassandra' }, diff --git a/website/data/docs-navigation.js b/website/data/docs-navigation.js index beaedfc33b..e6652385ec 100644 --- a/website/data/docs-navigation.js +++ b/website/data/docs-navigation.js @@ -254,6 +254,7 @@ export default [ ] }, { category: 'totp' }, + { category: 'transform' }, { category: 'transit' }, { category: 'venafi' }, '------------------------', diff --git a/website/pages/api-docs/secret/transform/index.mdx b/website/pages/api-docs/secret/transform/index.mdx new file mode 100644 index 0000000000..eba63f2813 --- /dev/null +++ b/website/pages/api-docs/secret/transform/index.mdx @@ -0,0 +1,694 @@ +--- +layout: api +page_title: Transform - Secrets Engines - HTTP API +sidebar_title: Transform ENTERPRISE +description: This is the API documentation for the Transform secrets engine. +--- + +# Transform Secrets Engine (API) + +This is the API documentation for the Transform secrets engine. For general +information about the usage and operation of the secrets engine, please see the +[Transform secrets engine documentation](/docs/secrets/transform). + +This documentation assumes the transform secrets engine is enabled at the +`/transform` path in Vault. Since it is possible to enable secrets engines at any +location, please update your API calls accordingly. + +## Create/Update Role + +This endpoint creates or updates the role with the given `name`. If a role with +the name does not exist, it will be created. If the role exists, it will be +updated with the new attributes. + +| Method | Path | +| :----- | :---------------------- | +| `POST` | `/transform/role/:name` | + + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the role to create. This is part of the request URL. + +- `transformations` (`list: []`) - + Specifies the transformations that can be used with this role. + +### Sample Payload + +```json +{ + "transformations": [ + "creditcard-fpe", + "creditcard-masking" + ] +} +``` + +### Sample Request + +```text +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + https://127.0.0.1:8200/v1/transform/role/example-role +``` + +## Read Role + +This endpoint queries an existing role by the given name. + +| Method | Path | +| :----- | :----------------------- | +| `GET` | `/transform/role/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the role to read. This is part of the request URL. + +### Sample Request + +``` +$ curl \ + --header "X-Vault-Token: ..." \ + http://127.0.0.1:8200/v1/transform/role/example-role +``` + +### Sample Response + +```json +{ + "data": { + "transformations": [ + "creditcard-fpe", + "creditcard-masking" + ] + } +} +``` + +## List Roles + +This endpoint lists all existing roles in the secrets engine. + +| Method | Path | +| :----- | :----------------- | +| `LIST` | `/transform/role` | + +### Sample Request + +```shell +$ curl + --header "X-Vault-Token: ..." \ + --request LIST \ + http://127.0.0.1:8200/v1/transform/role +``` + +### Sample Response + +```json +{ + "data": { + "keys": [ + "example-role" + ] + } +} +``` + +## Delete Role + +This endpoint deletes an existing role by the given name. + +| Method | Path | +| :------- | :---------------------- | +| `DELETE` | `/transform/role/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the role to delete. This is part of the request URL. + +### Sample Request + +```shell +$ curl \ + --header "X-Vault-Token: ..." \ + --request DELETE \ + http://127.0.0.1:8200/v1/transform/role/example-role +``` + +## Create/Update Transformation + +This endpoint creates or updates a transformation with the given `name`. If a +transformation with the name does not exist, it will be created. If the +transformation exists, it will be updated with the new attributes. + +| Method | Path | +| :----- | :-------------------------------- | +| `POST` | `/transform/transformation/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the transformation to create or update. This is part of + the request URL. + +- `type` `(string: )` - + Specifies the type of transformation to perform. The types currently supported + by this backend are `fpe` and `masking`. This value cannot be modified + by an update operation after creation. + +- `template` `(string: )` - + Specifies the template name to use for matching value on encode and decode + operations when using this transformation. + +- `tweak_source` `(string: "supplied")` - + Specifies the source of where the tweak value comes from. Valid sources are + `supplied`, `generated`, and `internal`. Only used when the type is FPE. + +- `masking_character` `(string: "*")` - + Specifies the character to use for masking. If multiple characters are + provided, only the first one is used and the rest is ignored. Only used when + the type is masking. + +- `allowed_roles` `(list: [])` - + Specifies a list of allowed roles that this transformation can be assigned to. + A role using this transformation must exist in this list in order for + encode and decode operations to properly function. + +### Sample Payload + +```json +{ + "type": "fpe", + "template": "builtin/creditcardnumber", + "tweak_source": "internal", + "allowed_roles": [ + "example-role" + ] +} +``` + +### Sample Request + +```text +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + https://127.0.0.1:8200/v1/transform/transformation/example-transformation +``` + +## Read Transformation + +This endpoint queries an existing transformation by the given name. + +| Method | Path | +| :----- | :-------------------------------- | +| `GET` | `/transform/transformation/:name` | + +- `name` `(string: )` –  + Specifies the name of the role to read. This is part of the request URL. + +### Sample Request + +``` +$ curl \ + --header "X-Vault-Token: ..." \ + http://127.0.0.1:8200/v1/transform/transformation/example-transformation +``` + +### Sample Response + +```json +{ + "data": { + "allowed_roles": [ + "example-role" + ], + "templates": [ + "builtin/creditcardnumber" + ], + "tweak_source": "internal", + "type": "fpe" + } +} +``` + +## List Transformation + +This endpoint lists all existing transformations in the secrets engine. + +| Method | Path | +| :----- | :--------------------------- | +| `LIST` | `/transform/transformation` | + +### Sample Request + +```shell +$ curl + --header "X-Vault-Token: ..." \ + --request LIST \ + http://127.0.0.1:8200/v1/transform/transformation +``` + +### Sample Response + +```json +{ + "data": { + "keys": [ + "example-transformation" + ] + } +} +``` + +## Delete Transformation + +This endpoint deletes an existing transformation by the given name. + +| Method | Path | +| :------- | :-------------------------------- | +| `DELETE` | `/transform/transformation/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the transformation to delete. This is part of the + request URL. + +### Sample Request + +```shell +$ curl \ + --header "X-Vault-Token: ..." \ + --request DELETE \ + http://127.0.0.1:8200/v1/transform/transformation/example-transformation +``` + +## Create/Update Template + +This endpoint creates or updates a template with the given `name`. If a +template with the name does not exist, it will be created. If the +template exists, it will be updated with the new attributes. + +| Method | Path | +| :----- | :-------------------------------- | +| `POST` | `/transform/template/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the template to create. This is part of the + request URL. + +- `type` `(string: )` - + Specifies the type of pattern matching to perform. The ony type currently supported + by this backend is `regex`. + +- `pattern` `(string: )` - + Specifies the pattern used to match a particular value. For regex type + matching, capture group determines the set of character that should be matched + against. Any matches outside of capture groups are retained + post-transformation. + +- `alphabet` `(string)` - + Specifies the name of the alphabet to use when this template is used for FPE + encoding and decoding operations. + +### Sample Payload + +```json +{ + "type": "regex", + "pattern": "(\\d{9})", + "alphabet": "builtin/numeric" +} +``` + +### Sample Request + +```text +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + https://127.0.0.1:8200/v1/transform/template/example-template +``` + +## Read Template + +This endpoint queries an existing template by the given name. + +| Method | Path | +| :----- | :-------------------------------- | +| `GET` | `/transform/template/:name` | + +- `name` `(string: )` –  + Specifies the name of the role to read. This is part of the request URL. + +### Sample Request + +``` +$ curl \ + --header "X-Vault-Token: ..." \ + http://127.0.0.1:8200/v1/transform/template/example-template +``` + +### Sample Response + +```json +{ + "data": { + "alphabet": "builtin/numeric", + "pattern": "(\\d{9})", + "type": "regex" + } +} +``` + +## List Template + +This endpoint lists all existing templates in the secrets engine. + +| Method | Path | +| :----- | :--------------------------- | +| `LIST` | `/transform/transformation` | + +### Sample Request + +```shell +$ curl + --header "X-Vault-Token: ..." \ + --request LIST \ + http://127.0.0.1:8200/v1/transform/template +``` + +### Sample Response + +```json +{ + "data": { + "keys": [ + "example-template" + ] + } +} +``` + +## Delete Template + +This endpoint deletes an existing template by the given name. + +| Method | Path | +| :------- | :-------------------------------- | +| `DELETE` | `/transform/template/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the template to delete. This is part of the + request URL. + +### Sample Request + +```shell +$ curl \ + --header "X-Vault-Token: ..." \ + --request DELETE \ + http://127.0.0.1:8200/v1/transform/template/example-template +``` + +## Create/Update Alphabet + +This endpoint creates or updates an alphabet with the given `name`. If an +alphabet with the name does not exist, it will be created. If the +alphabet exists, it will be updated with the new attributes. + +| Method | Path | +| :----- | :-------------------------------- | +| `POST` | `/transform/alphabet/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the transformation to create. This is part of the + request URL. + +- `alphabet` `(string: )` –  + Specifies the set of characters that can exist within the provided value + and the encoded or decoded value for a FPE transformation. + +### Sample Payload + +```json +{ + "alphabet": "abc" +} +``` + +### Sample Request + +```text +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + https://127.0.0.1:8200/v1/transform/alphabet/example-alphabet +``` + +## Read Alphabet + +This endpoint queries an existing alphabet by the given name. + +| Method | Path | +| :----- | :-------------------------------- | +| `GET` | `/transform/alphabet/:name` | + +- `name` `(string: )` –  + Specifies the name of the role to read. This is part of the request URL. + +### Sample Request + +``` +$ curl \ + --header "X-Vault-Token: ..." \ + http://127.0.0.1:8200/v1/transform/alphabet/example-alphabet +``` + +### Sample Response + +```json +{ + "data": { + "alphabet": "abc" + } +} +``` + +## List Alphabets + +This endpoint lists all existing alphabets in the secrets engine. + +| Method | Path | +| :----- | :--------------------------- | +| `LIST` | `/transform/alphabet` | + +### Sample Request + +```shell +$ curl + --header "X-Vault-Token: ..." \ + --request LIST \ + http://127.0.0.1:8200/v1/transform/alphabet +``` + +### Sample Response + +```json +{ + "data": { + "keys": [ + "example-alphabet" + ] + } +} +``` + +## Delete Alphabet + +This endpoint deletes an existing alphabet by the given name. + +| Method | Path | +| :------- | :-------------------------------- | +| `DELETE` | `/transform/alphabet/:name` | + +### Parameters + +- `name` `(string: )` –  + Specifies the name of the alphabet to delete. This is part of the request URL. + +### Sample Request + +```shell +$ curl \ + --header "X-Vault-Token: ..." \ + --request DELETE \ + http://127.0.0.1:8200/v1/transform/alphabet/example-alphabet +``` + +## Encode + +This endpoint encodes the provided value using a named role. + +| Method | Path | +| :----- | :----------------------------- | +| `POST` | `/transform/encode/:role_name` | + + +### Parameters + +- `role_name` `(string: )` – + Specifies the role name to use for this operation. This is specified as part + of the URL. + +- `value` `(string: )` – + Specifies the value to be encoded. + +- `transformation` `(string)` – + Specifies the transformation within the role that should be used for this + encode operation. If a single transformation exists for role, this parameter + may be skipped and will be inferred. If multiple transformations exist, one + must be specified. + +- `tweak` `(string)` – + Specifies the **base64 encoded** tweak to use. Only applicable for FPE + transformations with `supplied` as the tweak source. + +- `batch_input` `(array: nil)` - + Specifies a list of items to be encoded in a single batch. When this + parameter is set, the 'value', 'transformation' and 'tweak' parameters are + ignored. Instead, the aforementioned parameters should be provided within + each object in the list. + + ```json + [ + { + "value": "1111-1111-1111-1111", + "transformation": "ccn-fpe" + }, + { + "value": "2222-2222-2222-2222", + "transformation": "ccn-masking" + } + ] + ``` + +**NOTE:** The response payload may return a tweak along with the encoded value +if the `tweak_source` for the specified transformation is set to `generated`. +The resource owner should properly store this tweak, which must be supplied back +when decrypting the encoded value. + +### Sample Payload + +```json +{ + "value": "1111-2222-3333-4444", + "transformation": "ccn-fpe" +} +``` + +### Sample Request + +``` +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + http://127.0.0.1:8200/v1/transform/encode/example-role +``` + +### Sample Response + +```json +{ + "data": { + "encoded_value": "5682-4613-6822-8064" + } +} +``` + +## Decode + +This endpoint decodes the provided value using a named role. + +| Method | Path | +| :----- | :----------------------------- | +| `POST` | `/transform/decode/:role_name` | + + +### Parameters + +- `role_name` `(string: )` – + Specifies the role name to use for this operation. This is specified as part + of the URL. + +- `value` `(string: )` – + Specifies the value to be decoded. + +- `transformation` `(string)` – + Specifies the transformation within the role that should be used for this + decode operation. If a single transformation exists for role, this parameter + may be skipped and will be inferred. If multiple transformations exist, one + must be specified. + +- `tweak` `(string)` – + Specifies the **base64 encoded** tweak to use. Only applicable for FPE + transformations with `supplied` or `generated` as the tweak source. + +- `batch_input` `(array: nil)` - + Specifies a list of items to be decoded in a single batch. When this + parameter is set, the 'value', 'transformation' and 'tweak' parameters are + ignored. Instead, the aforementioned parameters should be provided within + each object in the list. + + ```json + [ + { + "value": "5682-4613-6822-8064", + "transformation": "ccn-fpe" + } + ] + ``` +### Sample Payload + +```json +{ + "value": "5682-4613-6822-8064", + "transformation": "ccn-fpe" +} +``` + +### Sample Request + +``` +$ curl \ + --header "X-Vault-Token: ..." \ + --request POST \ + --data @payload.json \ + http://127.0.0.1:8200/v1/transform/encode/example-role +``` + +### Sample Response + +```json +{ + "data": { + "decoded_value": "1111-2222-3333-4444" + } +} +``` diff --git a/website/pages/docs/secrets/transform/index.mdx b/website/pages/docs/secrets/transform/index.mdx new file mode 100644 index 0000000000..e5a365b0ff --- /dev/null +++ b/website/pages/docs/secrets/transform/index.mdx @@ -0,0 +1,185 @@ +--- +layout: docs +page_title: Transform - Secrets Engines +sidebar_title: Transform ENTERPRISE +description: >- + The Transform secrets engine for Vault performs secure data transformation. +--- + +# Transform Secrets Engine + +-> **Note**: This secret engine requires [Vault +Enterprise](https://www.hashicorp.com/products/vault/) with the Advanced Data +Protection Module. + +The Transform secrets engine handles secure data transformation and tokenization +against provided input value. Transformation methods may encompass NIST vetted +cryptographic standards such as [format-preserving encryption +(FPE)](https://en.wikipedia.org/wiki/Format-preserving_encryption) via +[FF3-1](https://csrc.nist.gov/publications/detail/sp/800-38g/rev-1/draft), but +can also be pseudonymous transformations of the data through other means, such +as masking. + +The secret engine currently supports `fpe` and `masking` as data transformation +types. + +## Setup + +Most secrets engines must be configured in advance before they can perform their +functions. These steps are usually completed by an operator or configuration +management tool. + +1. Enable the Transform secrets engine: + + ```text + $ vault secrets enable transform + Success! Enabled the transform secrets engine at: transform/ + ``` + + By default, the secrets engine will mount at the name of the engine. To enable + the secrets engine at a different path, use the -path argument. + +1. Create a named role: + + ```text + $ vault write transform/role/payments transformations=ccn-fpe + Success! Data written to: transform/role/payments + ``` + +1. Create a transformation: + + ```text + $ vault write transform/transformation/ccn-fpe \ + type=fpe \ + template=ccn \ + tweak_source=internal \ + allowed_roles=payments + Success! Data written to: transform/transformation/ccn-fpe + ``` + +1. Optionally, create a template: + + ```text + $ vault write transform/template/ccn \ + type=regex \ + pattern='(\d{4})-(\d{4})-(\d{4})-(\d{4})' \ + alphabet=numerics + Success! Data written to: transform/template/ccn + ``` + +1. Optionally, create an alphabet: + + ```text + $ vault write transform/alphabet/numerics \ + alphabet="0123456789" + Success! Data written to: transform/alphabet/numerics + ``` + +## Usage + +After the secrets engine is configured and a user/machine has a Vault token with +the proper permission, it can use this secrets engine to encode and decode input +values. + +1. Encode some input value using the `/encode` endpoint with a named role: + + ```text + $ vault write transform/encode/payments value=1111-2222-3333-4444 + Key Value + --- ----- + encoded_value 9300-3376-4943-8903 + ``` + + A transformation must be provided if the role contains more than one + transformation. A tweak must be provided if the tweak source for the + transformation is "supplied". + +1. Decode some input value using the `/decode` endpoint with a named role: + + ```text + $ vault write transform/decode/payments value=9300-3376-4943-8903 + Key Value + --- ----- + decoded_value 1111-2222-3333-4444 + ``` + + A transformation must be provided if the role contains more than one + transformation. A tweak must be provided if the tweak source for the + transformation is "supplied" or "generated". + +## Roles, Transformations, Templates, and Alphabets + +The Transform secrets engine contains several types of resources that +encapsulate different aspects of the information required in order to perform +data transformation. + +- **Roles** are the basic high-level construct that holds the set of +transformation that it is allowed to performed. The role name is provided when +performing encode and decode operations. + +- **Transformations** hold information about a particular transformation. It +contains information about the type of transformation that we want to perform, +the template that it should use for value detection, and other +transformation-specific values such as the tweak source or the masking character +to use. + +- **Templates** allow us to determine what and how to capture the value that we +want to transform. + +- **Alphabets** provide the set of valid UTF-8 character contained within both +the input and transformed value on FPE transformations. + +## Deletion Behavior + +The deletion of resources, aside from roles, is guarded by checking whether any +other related resources are currently using it in order to avoid accidental data +loss of any encoded value that may depend on these bits of information to +decode and reconstruct the original value. Role deletion can be safely done +since the information related to the transformation itself is contained within +transformation object and its related resources. + +The following rules applies when it comes to deleting a resource: + +- A transformation cannot be deleted if it's in use by a role +- A template cannot be deleted if it's in use by a transformation +- An alphabet cannot be deleted if it's in use by a template + +## Provided Builtin Resources + +The secret engine provides a set of builtin templates and alphabets that are +considered common. Builtin templates cannot be deleted, and the prefix +"builtin/" on template and alphabet names is a reserved keyword. + +### Templates + +The following builtin templates are available for use in the secret engine: + +- builtin/creditcardnumber +- builtin/socialsecuritynumber + +Note that these templates only check for the matching pattern(s), and not the +validity of the value itself. For instance, the builtin credit card number +template can determine whether the provided value is in the format of commonly +issued credit cards, but not whether the credit card is a valid number from a +particular issuer. + +### Alphabets + +The following builtin alphabets are available for use in the secret engine: + +- builtin/numeric +- builtin/alphalower +- builtin/alphaupper +- builtin/alphanumericlower +- builtin/alphanumericupper +- builtin/alphanumeric + +## Learn + +Refer to the [Transform Secrets Engine](https://learn.hashicorp.com/vault/adp/transform) guide for a step-by-step tutorial. + +## API + +The Transform secrets engine has a full HTTP API. Please see the +[Transform secrets engine API](/api/secret/transform) for more +details.