Speed up dynamodb List() by only getting keys (#21159)

* Speed up ddb list by only getting keys

* Add same optimization to has_children

* Add changeling

---------

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
This commit is contained in:
Thomas Hargrove
2024-06-07 12:11:00 -07:00
committed by GitHub
parent 8fd63b0a60
commit 27563039db
2 changed files with 14 additions and 0 deletions

4
changelog/21159.txt Normal file
View File

@@ -0,0 +1,4 @@
```release-note:improvement
storage/dynamodb: Speed up list and delete of large directories by only requesting keys from DynamoDB
```

View File

@@ -410,6 +410,11 @@ func (d *DynamoDBBackend) List(ctx context.Context, prefix string) ([]string, er
}},
},
},
ProjectionExpression: aws.String("#key, #path"),
ExpressionAttributeNames: map[string]*string{
"#key": aws.String("Key"),
"#path": aws.String("Path"),
},
}
d.permitPool.Acquire()
@@ -453,6 +458,11 @@ func (d *DynamoDBBackend) hasChildren(prefix string, exclude []string) (bool, er
}},
},
},
ProjectionExpression: aws.String("#key, #path"),
ExpressionAttributeNames: map[string]*string{
"#key": aws.String("Key"),
"#path": aws.String("Path"),
},
// Avoid fetching too many items from DynamoDB for performance reasons.
// We want to know if there are any children we don't expect to see.
// Answering that question requires fetching a minimum of one more item