707111b78f
By adding the link flags `-s -w` we can reduce the Vault binary size from 204 MB to 167 MB (about 18% reduction in size). This removes the DWARF section of the binary. i.e., before: ``` $ objdump --section-headers vault-debug vault-debug: file format mach-o arm64 Sections: Idx Name Size VMA Type 0 __text 03a00340 0000000100001000 TEXT 1 __symbol_stub1 00000618 0000000103a01340 TEXT 2 __rodata 00c18088 0000000103a01960 DATA 3 __rodata 015aee18 000000010461c000 DATA 4 __typelink 0004616c 0000000105bcae20 DATA 5 __itablink 0000eb68 0000000105c10fa0 DATA 6 __gosymtab 00000000 0000000105c1fb08 DATA 7 __gopclntab 02a5b8e0 0000000105c1fb20 DATA 8 __go_buildinfo 00008c10 000000010867c000 DATA 9 __nl_symbol_ptr 00000410 0000000108684c10 DATA 10 __noptrdata 000fed00 0000000108685020 DATA 11 __data 0004e1f0 0000000108783d20 DATA 12 __bss 00052520 00000001087d1f20 BSS 13 __noptrbss 000151b0 0000000108824440 BSS 14 __zdebug_abbrev 00000129 000000010883c000 DATA, DEBUG 15 __zdebug_line 00651374 000000010883c129 DATA, DEBUG 16 __zdebug_frame 001e1de9 0000000108e8d49d DATA, DEBUG 17 __debug_gdb_scri 00000043 000000010906f286 DATA, DEBUG 18 __zdebug_info 00de2c09 000000010906f2c9 DATA, DEBUG 19 __zdebug_loc 00a619ea 0000000109e51ed2 DATA, DEBUG 20 __zdebug_ranges 001e94a6 000000010a8b38bc DATA, DEBUG ``` And after: ``` $ objdump --section-headers vault-no-debug vault-no-debug: file format mach-o arm64 Sections: Idx Name Size VMA Type 0 __text 03a00340 0000000100001000 TEXT 1 __symbol_stub1 00000618 0000000103a01340 TEXT 2 __rodata 00c18088 0000000103a01960 DATA 3 __rodata 015aee18 000000010461c000 DATA 4 __typelink 0004616c 0000000105bcae20 DATA 5 __itablink 0000eb68 0000000105c10fa0 DATA 6 __gosymtab 00000000 0000000105c1fb08 DATA 7 __gopclntab 02a5b8e0 0000000105c1fb20 DATA 8 __go_buildinfo 00008c20 000000010867c000 DATA 9 __nl_symbol_ptr 00000410 0000000108684c20 DATA 10 __noptrdata 000fed00 0000000108685040 DATA 11 __data 0004e1f0 0000000108783d40 DATA 12 __bss 00052520 00000001087d1f40 BSS 13 __noptrbss 000151b0 0000000108824460 BSS ``` The only side effect I have been able to find is that it is no longer possible to use [delve](https://github.com/go-delve/delve) to run the Vault binary. Note, however, that running delve and other debuggers requires access to the full source code, which isn't provided for the Enterprise, HSM, etc. binaries, so it isn't possible to debug those anyway outside of people who have the full source. * panic traces * `vault debug` * error messages * Despite what the documentation says, these flags do *not* delete the function symbol table (so it is not the same as having a `strip`ped binary). It contains mappings between the compiled binary and functions, paramters, and variables in the source code. Using `llvm-dwarfdump`, it looks like: ``` 0x011a6d85: DW_TAG_subprogram DW_AT_name ("github.com/hashicorp/vault/api.(*replicationStateStore).recordState") DW_AT_low_pc (0x0000000000a99300) DW_AT_high_pc (0x0000000000a99419) DW_AT_frame_base (DW_OP_call_frame_cfa) DW_AT_decl_file ("/home/swenson/vault/api/client.go") DW_AT_external (0x01) 0x011a6de1: DW_TAG_formal_parameter DW_AT_name ("w") DW_AT_variable_parameter (0x00) DW_AT_decl_line (1735) DW_AT_type (0x00000000001e834a "github.com/hashicorp/vault/api.replicationStateStore *") DW_AT_location (0x009e832a: [0x0000000000a99300, 0x0000000000a9933a): DW_OP_reg0 RAX [0x0000000000a9933a, 0x0000000000a99419): DW_OP_call_frame_cfa) 0x011a6def: DW_TAG_formal_parameter DW_AT_name ("resp") DW_AT_variable_parameter (0x00) DW_AT_decl_line (1735) DW_AT_type (0x00000000001e82a2 "github.com/hashicorp/vault/api.Response *") DW_AT_location (0x009e8370: [0x0000000000a99300, 0x0000000000a9933a): DW_OP_reg3 RBX [0x0000000000a9933a, 0x0000000000a99419): DW_OP_fbreg +8) 0x011a6e00: DW_TAG_variable DW_AT_name ("newState") DW_AT_decl_line (1738) DW_AT_type (0x0000000000119f32 "string") DW_AT_location (0x009e83b7: [0x0000000000a99385, 0x0000000000a99385): DW_OP_reg0 RAX, DW_OP_piece 0x8, DW_OP_piece 0x8 [0x0000000000a99385, 0x0000000000a993a4): DW_OP_reg0 RAX, DW_OP_piece 0x8, DW_OP_reg3 RBX, DW_OP_piece 0x8 [0x0000000000a993a4, 0x0000000000a993a7): DW_OP_piece 0x8, DW_OP_reg3 RBX, DW_OP_piece 0x8) ``` This says that the particular binary section is the function `github.com/hashicorp/vault/api.(*replicationStateStore).recordState`, from the file `/home/swenson/vault/api/client.go`, containing the `w` parameter on line 1735 mapped to certain registers and memory, the `resp` paramter on line 1735 mapped to certain reigsters and memory, and the `newState` variable on line 1738, mapped to certain registers, and memory. It's really only useful for a debugger. Anyone running the code in a debugger will need full access the source code anyway, so presumably they will be able to run `make dev` and build the version with the DWARF sections intact, and then run their debugger.
Vault 
Please note: We take Vault's security and our users' trust very seriously. If you believe you have found a security issue in Vault, please responsibly disclose by contacting us at security@hashicorp.com.
- Website: https://www.vaultproject.io
- Announcement list: Google Groups
- Discussion forum: Discuss
- Documentation: https://www.vaultproject.io/docs/
- Tutorials: HashiCorp's Learn Platform
- Certification Exam: Vault Associate
Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, and more. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.
A modern system requires access to a multitude of secrets: database credentials, API keys for external services, credentials for service-oriented architecture communication, etc. Understanding who is accessing what secrets is already very difficult and platform-specific. Adding on key rolling, secure storage, and detailed audit logs is almost impossible without a custom solution. This is where Vault steps in.
The key features of Vault are:
-
Secure Secret Storage: Arbitrary key/value secrets can be stored in Vault. Vault encrypts these secrets prior to writing them to persistent storage, so gaining access to the raw storage isn't enough to access your secrets. Vault can write to disk, Consul, and more.
-
Dynamic Secrets: Vault can generate secrets on-demand for some systems, such as AWS or SQL databases. For example, when an application needs to access an S3 bucket, it asks Vault for credentials, and Vault will generate an AWS keypair with valid permissions on demand. After creating these dynamic secrets, Vault will also automatically revoke them after the lease is up.
-
Data Encryption: Vault can encrypt and decrypt data without storing it. This allows security teams to define encryption parameters and developers to store encrypted data in a location such as a SQL database without having to design their own encryption methods.
-
Leasing and Renewal: All secrets in Vault have a lease associated with them. At the end of the lease, Vault will automatically revoke that secret. Clients are able to renew leases via built-in renew APIs.
-
Revocation: Vault has built-in support for secret revocation. Vault can revoke not only single secrets, but a tree of secrets, for example, all secrets read by a specific user, or all secrets of a particular type. Revocation assists in key rolling as well as locking down systems in the case of an intrusion.
Documentation, Getting Started, and Certification Exams
Documentation is available on the Vault website.
If you're new to Vault and want to get started with security automation, please check out our Getting Started guides on HashiCorp's learning platform. There are also additional guides to continue your learning.
For examples of how to interact with Vault from inside your application in different programming languages, see the vault-examples repo. An out-of-the-box sample application is also available.
Show off your Vault knowledge by passing a certification exam. Visit the certification page for information about exams and find study materials on HashiCorp's learning platform.
Developing Vault
If you wish to work on Vault itself or any of its built-in systems, you'll first need Go installed on your machine.
For local dev first make sure Go is properly installed, including setting up a
GOPATH. Ensure that $GOPATH/bin is in
your path as some distributions bundle the old version of build tools. Next, clone this
repository. Vault uses Go Modules,
so it is recommended that you clone the repository outside of the GOPATH.
You can then download any required build tools by bootstrapping your environment:
$ make bootstrap
...
To compile a development version of Vault, run make or make dev. This will
put the Vault binary in the bin and $GOPATH/bin folders:
$ make dev
...
$ bin/vault
...
To compile a development version of Vault with the UI, run make static-dist dev-ui. This will
put the Vault binary in the bin and $GOPATH/bin folders:
$ make static-dist dev-ui
...
$ bin/vault
...
To run tests, type make test. Note: this requires Docker to be installed. If
this exits with exit status 0, then everything is working!
$ make test
...
If you're developing a specific package, you can run tests for just that
package by specifying the TEST variable. For example below, only
vault package tests will be run.
$ make test TEST=./vault
...
Importing Vault
This repository publishes two libraries that may be imported by other projects:
github.com/hashicorp/vault/api and github.com/hashicorp/vault/sdk.
Note that this repository also contains Vault (the product), and as with most Go
projects, Vault uses Go modules to manage its dependencies. The mechanism to do
that is the go.mod file. As it happens, the presence of that file
also makes it theoretically possible to import Vault as a dependency into other
projects. Some other projects have made a practice of doing so in order to take
advantage of testing tooling that was developed for testing Vault itself. This
is not, and has never been, a supported way to use the Vault project. We aren't
likely to fix bugs relating to failure to import github.com/hashicorp/vault
into your project.
Acceptance Tests
Vault has comprehensive acceptance tests covering most of the features of the secret and auth methods.
If you're working on a feature of a secret or auth method and want to verify it is functioning (and also hasn't broken anything else), we recommend running the acceptance tests.
Warning: The acceptance tests create/destroy/modify real resources, which may incur real costs in some cases. In the presence of a bug, it is technically possible that broken backends could leave dangling data behind. Therefore, please run the acceptance tests at your own risk. At the very least, we recommend running them in their own private account for whatever backend you're testing.
To run the acceptance tests, invoke make testacc:
$ make testacc TEST=./builtin/logical/consul
...
The TEST variable is required, and you should specify the folder where the
backend is. The TESTARGS variable is recommended to filter down to a specific
resource to test, since testing all of them at once can sometimes take a very
long time.
Acceptance tests typically require other environment variables to be set for things such as access keys. The test itself should error early and tell you what to set, so it is not documented here.
For more information on Vault Enterprise features, visit the Vault Enterprise site.