scottk/vault
1
0
Fork 0
mirror of https://github.com/optim-enterprises-bv/vault.git synced 2025-10-30 10:12:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Speed up dynamodb List() by only getting keys (#21159)

Browse Source 
* Speed up ddb list by only getting keys

* Add same optimization to has_children

* Add changeling

---------

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
This commit is contained in:
Thomas Hargrove
2024-06-07 12:11:00 -07:00
committed by GitHub
parent 8fd63b0a60
commit 27563039db
2 changed files with 14 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
changelog/21159.txt Normal file
View File

@@ -0,0 +1,4 @@
```release-note:improvement
storage/dynamodb: Speed up list and delete of large directories by only requesting keys from DynamoDB
```

10
physical/dynamodb/dynamodb.go
View File

@@ -410,6 +410,11 @@ func (d *DynamoDBBackend) List(ctx context.Context, prefix string) ([]string, er
}}, }},
}, },
}, },
ProjectionExpression: aws.String("#key, #path"),
ExpressionAttributeNames: map[string]*string{
"#key": aws.String("Key"),
"#path": aws.String("Path"),
},
} }
d.permitPool.Acquire() d.permitPool.Acquire()
@@ -453,6 +458,11 @@ func (d *DynamoDBBackend) hasChildren(prefix string, exclude []string) (bool, er
}}, }},
}, },
}, },
ProjectionExpression: aws.String("#key, #path"),
ExpressionAttributeNames: map[string]*string{
"#key": aws.String("Key"),
"#path": aws.String("Path"),
},
// Avoid fetching too many items from DynamoDB for performance reasons. // Avoid fetching too many items from DynamoDB for performance reasons.
// We want to know if there are any children we don't expect to see. // We want to know if there are any children we don't expect to see.
// Answering that question requires fetching a minimum of one more item // Answering that question requires fetching a minimum of one more item