Add docs for password policies (#8974)

* Add docs for password policies

website/data/api-navigation.js

@@ -124,6 +124,7 @@ export default [
       'plugins-catalog',
       'policy',
       'policies',
+      'policies-password',
       'pprof',
       'raw',
       'rekey',

website/data/docs-navigation.js

@@ -33,6 +33,7 @@ export default [
       'tokens',
       'response-wrapping',
       'policies',
+      'password-policies',
       'ha',
       'integrated-storage',
       'pgp-gpg-keybase',

website/pages/api-docs/system/policies-password.mdx

@@ -0,0 +1,157 @@
+---
+layout: api
+page_title: /sys/policies/password - HTTP API
+sidebar_title: <code>/sys/policies/password</code>
+description: >-
+  The `/sys/policies/password` endpoints are used to manage password generation policies in Vault.
+---
+
+# `/sys/policies/password/`
+
+The `/sys/policies/password/` endpoints are used to manage password generation policies in Vault.
+Not all secret engines utilize password policies, so check the documentation for the engine you
+are using for compatibility.
+
+~> Password policies are only available in Vault version 1.5+.
+
+See [Password Policies](/docs/concepts/password-policies) for details of how password policies work
+as well as the syntax of the policies themselves.
+
+## Create/Update Password Policy
+
+This endpoint adds a new or updates an existing password policy. Once a policy is updated,
+it takes effect immediately to all associated secret engines.
+
+Prior to Vault saving the password policy, it will attempt to generate a number of passwords
+from the policy. This helps prevent creating password policies that are impossible to satisfy
+as well as prevent password policies that are overly restrictive which prevents both a poor
+security posture for the policy as well as preventing performance problems due to slow
+generation times.
+
+| Method | Path                           |
+| :----- | :----------------------------- |
+| `PUT`  | `/sys/policies/password/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` – Specifies the name of the password policy to create.
+  This is specified as part of the request URL.
+
+- `policy` `(string: <required>)` - Specifies the password policy document. This can be
+  base64-encoded to avoid string escaping. See [Password Policy Syntax](#password-policy-syntax)
+  for details on password policy definitions.
+
+### Sample Payload
+
+```json
+{
+  "policy": "length = 20\nrule \"charset\" { ..."
+}
+```
+
+### Sample Request
+**cURL:**
+```shell
+$ cat payload.json
+{
+  "policy": "length = 20\nrule \"charset\" {\n  charset = \"abcde\"\n}\n"
+}
+
+$ curl \
+    --header "X-Vault-Token: ..." \
+    --request PUT \
+    --data @payload.json \
+    http://127.0.0.1:8200/v1/sys/policies/password/my-policy
+```
+
+**Vault CLI:**
+```shell
+$ cat my-policy.hcl
+length = 20
+rule "charset" {
+    charset = "abcde"
+}
+
+$ vault write sys/policies/password/my-policy policy=@my-policy.hcl
+```
+
+## Read Password Policy
+
+This endpoint retrieves information about the named password policy.
+
+| Method | Path                           |
+| :----- | :----------------------------- |
+| `GET`  | `/sys/policies/password/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` – Specifies the name of the password policy to retrieve.
+  This is specified as part of the request URL.
+
+### Sample Request
+
+```shell
+$ curl \
+    --header "X-Vault-Token: ..." \
+    http://127.0.0.1:8200/v1/sys/policies/password/my-policy
+```
+
+### Sample Response
+```json
+{
+  "policy": "length = 20\nrule \"charset\" { ..."
+}
+```
+
+## Delete Password Policy
+
+This endpoint deletes the password policy with the given name. This does not check if any
+secret engines are using it prior to deletion, so you should ensure that any engines that
+are utilizing this password policy are changed to a different policy (or to that engines'
+default behavior).
+
+| Method | Path                           |
+| :----- | :----------------------------- |
+| `DELETE`  | `/sys/policies/password/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` – Specifies the name of the password policy to delete.
+  This is specified as part of the request URL.
+
+### Sample Request
+
+```shell
+$ curl \
+    --header "X-Vault-Token: ..." \
+    --request DELETE
+    http://127.0.0.1:8200/v1/sys/policies/password/my-policy
+```
+
+## Generate Password from Password Policy
+
+This endpoint generates a password from the specified existing password policy.
+
+| Method | Path                           |
+| :----- | :----------------------------- |
+| `GET`  | `/sys/policies/password/:name/generate` |
+
+### Parameters
+
+- `name` `(string: <required>)` – Specifies the name of the password policy to generate
+  a password from. This is specified as part of the request URL.
+
+### Sample Request
+
+```shell
+$ curl \
+    --header "X-Vault-Token: ..." \
+    http://127.0.0.1:8200/v1/sys/policies/password/my-policy/generate
+```
+
+### Sample Response
+```json
+{
+  "password": "..."
+}
+```

website/pages/docs/concepts/password-policies.mdx

@@ -0,0 +1,346 @@
+---
+layout: docs
+page_title: Password Policies
+sidebar_title: Password Policies
+description: >-
+  Password policies are used in some secret engines to allow users to define how passwords are generated
+  for dynamic & static users within those engines.
+---
+
+# Password Policies
+A password policy is a set of instructions on how to generate a password, similar to other password
+generators. These password policies are used in a subset of secret engines to allow you to configure
+how a password is generated for that engine. Not all secret engines utilize password policies, so check
+the documentation for the engine you are using for compatibility.
+
+**Note:** Password policies are unrelated to [Policies](/docs/concepts/policies) other than sharing similar names.
+
+Password policies are available in Vault version 1.5+.
+
+!> Password policies are an advanced usage of Vault. This generates credentials for external systems
+   (databases, LDAP, AWS, etc.) and should be used with caution.
+
+## Design
+Password policies fundamentally have two parts: a length, and a set of rules that a password must
+adhere to. Passwords are randomly generated from the de-duplicated union of charsets found in all rules
+and then checked against each of the rules to determine if the candidate password is valid according
+to the policy. See [Candidate Password Generation](#candidate-password-generation) for details on how
+passwords are generated prior to being checked against the rule set.
+
+A rule is an assertion upon a candidate password string that indicates whether or not
+the password is acceptable. For example: a "charset" rule states that a password must have at least one
+lowercase letter in it. This rule will reject any passwords that do not have any lowercase letters in it.
+
+Multiple rules may be specified within a policy to create more complex rules, such as requiring at least
+one lowercase letter, at least one uppercase letter, and at least one number.
+
+The flow looks like:
+
+[![Vault Password Policy Flow](/img/vault-password-policy-flow.svg)](/img/vault-password-policy-flow.svg)
+
+## Candidate Password Generation
+How a candidate password is generated is extremely important. Great care must be placed to ensure that
+passwords aren't created in a way that can be exploited by threat actors. This section describes how we
+generate passwords within password policies to ensure that passwords are generated as securely as possible.
+
+To generate a candidate password, three things are needed:
+
+1. A [cryptographically secure random number generator](https://golang.org/pkg/crypto/rand/) (RNG).
+2. A character set (charset) to select characters from.
+3. The length of the password.
+
+At a high level, we use our RNG to generate N numbers that correspond to indices into the charset
+array where N is the length of the password we wish to create. Each value returned from the RNG is then
+used to extract a character from the charset into the password.
+
+For example, let's generate a password of length 8 from the charset `abcdefghij`:
+
+The RNG is used to generate 8 random values. For our example let's say those values are:
+
+`[3, 2, 0, 8, 7, 3, 5, 1]`
+
+Each of these values is an index into the charset array:
+
+`[3, 2, 0, 8, 7, 3, 5, 1]` => `[d, c, a, i, h, d, f, b]`
+
+This gives us our candidate password: `dcaihdfb` which can then be run through the rules of the policy.
+
+In a real world scenario, the values in the random array will be between `[0-255]` as that is the range of
+values that a single byte can be. The value is restricted to the size of the charset array by using the
+[modulo operation](https://en.wikipedia.org/wiki/Modulo_operation) to prevent referencing a character
+outside the bounds of the charset. However this can introduce a problem with bias.
+
+### Preventing Bias
+When using the [modulo operation](https://en.wikipedia.org/wiki/Modulo_operation) to generate a password,
+you must be very careful to prevent the introduction of bias. When generating a random number between
+[0-255] for a charset that has a length that isn't evenly divisible into 256, some of the first characters
+in the charset may be selected more frequently than the remaining characters.
+
+To demonstrate this, let's simplify the math. Assume that we have a charset of length 10: `abcdefghij`.
+Let's also assume that our RNG generates values `[0-25]`. The first 10 values `0-9` correspond to each
+character in our charset. The next 10 values `10-19` also correspond to each character in our charset.
+However, the next 6 values `20-25` correspond to only the first 6 characters in the charset. This means
+that those 6 characters `abcdef` can be selected more often than the last 4 characters `ghij`.
+
+In order to prevent this from happening, we calculate the maximum value that we can allow an index to be.
+This is based on the length of the charset we are selecting from. In the example above, the maximum index
+value we should allow is 19 as that represents the largest integer multiple of the length of the charset
+array that is less than the maximum value that our RNG can generate. When our RNG generates any values
+larger than our maximum allowed value, that number is ignored and we continue to the next number. Passwords
+do not lose any length because we continue generating numbers until the password is fully filled in to the
+length requested.
+
+## Performance Characteristics
+Characterizing password generation performance with this model is heavily dependent on the policy
+configuration. In short, the more restrictive the policy, the longer it will take to generate a password.
+This generalization isn't always true, but is a general guideline. The performance curve can be generalized:
+
+`(time to generate a candidate password) * (number of candidate passwords generated)`
+
+Where the number of times a candidate password needs to be generated is a function of how likely a given
+candidate password does not pass all of the rules.
+
+Here are some example policy configurations with their performance characteristics below. Each of these
+policies have the same charset that candidate passwords are generated from (94 characters). The only
+difference is the minimum number of characters for various character subsets.
+
+<details>
+<summary>No Minimum Characters</summary>
+
+```hcl
+rule "charset" {
+  charset = "abcdefghijklmnopqrstuvwxyz"
+}
+rule "charset" {
+  charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+}
+rule "charset" {
+  charset = "0123456789"
+}
+rule "charset" {
+  charset = "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
+}
+```
+</details>
+
+<details>
+<summary>1 uppercase, 1 lowercase, 1 numeric</summary>
+
+```hcl
+rule "charset" {
+  charset = "abcdefghijklmnopqrstuvwxyz"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "0123456789"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
+}
+```
+</details>
+
+<details>
+<summary>1 uppercase, 1 lowercase, 1 numeric, 1 from all ASCII characters</summary>
+
+```hcl
+rule "charset" {
+  charset = "abcdefghijklmnopqrstuvwxyz"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "0123456789"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
+  min-chars = 1
+}
+```
+</details>
+
+<details>
+<summary>1 uppercase, 1 lowercase, 1 numeric, 1 from <code>!@#$</code></summary>
+
+```hcl
+rule "charset" {
+  charset = "abcdefghijklmnopqrstuvwxyz"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "0123456789"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "!@#$"
+  min-chars = 1
+}
+# Fleshes out the rest of the symbols but doesn't add any required characters
+rule "charset" {
+  charset = "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
+}
+```
+</details>
+
+[![Password Policy Performance](/img/vault-password-policy-performance.svg)](/img/vault-password-policy-performance.svg)
+
+As more characters are generated, the amount of time increases (as seen in `No Minimum Characters`).
+This upward trend can be dwarfed by restricting charsets. When a password is short, the chances of a character
+being selected from a subset is smaller. For instance, if you have a 1 character password from the charset
+`abcde` the chances of selecting `c` from it is 1/5. However if you have a 2 character password, the chances
+of selecting `c` at least once are greater than 1/5 because you have a second chance to select `c` from
+the charset.
+
+In these examples, as the length of the password increases, the amount of time to generate a password trends
+down, levels off, and then slowly increases. This is a combination of the two effects listed above: increasing
+time to generate more characters vs the chances of the character subsets being selected. When a single subset is
+very small (such as `!@#$`) the chances of it being selected are much smaller (4/94) than if the subset is larger
+(26/94 for lowercase characters). This can result in a dramatic loss of performance.
+
+<details>
+<summary><b>Click here for more details on password generation probabilities</b></summary>
+
+In the examples above, the charset being used to generate candidate passwords is 94 characters long.
+Randomly choosing a given character from the 94 character charset has a 1/94 chance. Choosing a single
+character from it after N tries (where N is the length of the password) is `1-(1-1/94)^N`.
+
+If we expand this to look at a subset of characters (such as lowercase characters) the chances of selecting
+a character from that subset is `1-(1-L/94)^N` where `L` is the length of the subset. For lowercase
+characters, we get a probability of `1-(1-26/94)^N`.
+
+If we do this for uppercase letters as well as numbers, then we get a combined probability curve:
+
+`p = (1-(1-26/94)^N) * (1-(1-26/94)^N) * (1-(1-10/94)^N)`
+
+[![Chance of Generating a Good Password - 1](/img/vault-password-policy-chance.svg)](/img/vault-password-policy-chance.svg)
+
+It should be noted that this probability curve only applies to this specific policy. To understand the
+performance characteristics of a given policy, you should run your policy with the
+[`generate`](/api-docs/system/policies-password.mdx) endpoint to see how much time the policy takes to
+produce passwords.
+
+</details>
+
+## Password Policy Syntax
+Password policies are defined in [HCL](https://github.com/hashicorp/hcl) or JSON which defines
+the length of the password and a set of rules a password must adhere to.
+
+Here is a very simple policy which generates 20 character passwords from lowercase characters:
+
+```hcl
+length = 20
+rule "charset" {
+  charset = "abcdefghijklmnopqrstuvwxyz"
+}
+```
+
+Multiple rules may be specified, including multiple rules of the same type. For instance, the following
+policy will generate a 20 character password with at least one lowercase letter, at least one uppercase
+letter, at least one number, and at least one symbol from the set `!@#$%^&*`:
+
+```hcl
+length = 20
+rule "charset" {
+  charset = "abcdefghijklmnopqrstuvwxyz"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "0123456789"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "!@#$%^&*"
+  min-chars = 1
+}
+```
+
+At least one charset must be specified for a policy to be valid. In order to generate a password, a charset
+must be available to select characters from and password policies do not have a default charset.
+The following policy is **NOT** valid and will be rejected:
+
+```hcl
+length = 20
+```
+
+## Configuration & Available Rules
+
+### `length` Parameter
+- `length` `(int: <required>)` - Specifies how long the generated password will be. Must be >= 4.
+
+Length is **not** a rule. It is the only part of the configuration that does not adhere to the guess-
+and-check approach of rules.
+
+### Rule `charset`
+Allows you to specify a minimum number of characters from a given charset. For instance: a password must
+have at least one lowercase letter. This rule also helps construct the charset that the password generation
+utilizes. In order to generate a password, a charset must be specified.
+
+If multiple charsets are specified, all of the charsets will be combined and de-duplicated prior to
+generating any candidate passwords. Each individual `charset` rule will still need to be adhered to in
+order to successfully generate passwords.
+
+~> After combining and de-duplicating charsets, the length of the charset that candidate passwords
+   are generated from must be no longer than 256 characters.
+
+#### Parameters
+- `charset` `(string: <required>)` – A string representation of the character set that this rule observes.
+  Accepts UTF-8 compatible strings. All characters within the string must be printable.
+- `min-chars` `(int: 0)` - Specifies a minimum number of characters required from the charset specified in
+  this rule. For example: if `min-chars = 2`, the password must have at least 2 characters from `charset`.
+
+
+#### Example
+
+```hcl
+length = 20
+rule "charset" {
+  charset = "abcde"
+  min-chars = 1
+}
+rule "charset" {
+  charset = "01234"
+  min-chars = 1
+}
+```
+
+This policy will generate passwords from the charset `abcde01234`. However, the password must have at
+least one character that is from `abcde` and at least one character from `01234`. If charsets overlap
+between rules, the charsets will be de-duplicated to prevent bias towards the overlapping set.
+For instance: if you have two charset rules: `abcde` & `cdefg`, the charset `abcdefg` will be used to
+generate candidate passwords, but a least one character from each `abcde` & `cdefg` must still appear
+in the password.
+
+If `min-chars` is not specified (or set to `0`) then this charset will not have a minimum required number
+of characters, but it will be used to select characters from. Example:
+
+```hcl
+length = 8
+rule "charset" {
+  charset = "abcde"
+}
+rule "charset" {
+  charset = "01234"
+  min-chars = 1
+}
+```
+
+This policy generates 8 character passwords from the charset `abcde01234` and requires at least one
+character from `01234` to be in it, but does not require any characters from `abcde`. The password
+`04031945` may result from this policy, even though no alphabetical characters are in it.