175 Commits

Author SHA1 Message Date
carlory
94bf8fc8a9 Promoted API VolumeAttributesClass and VolumeAttributesClassList to storage.k8s.io/v1.
Promoted feature-gate `VolumeAttributesClass` to GA (on by default)

Signed-off-by: carlory <baofa.fan@daocloud.io>
2025-07-25 01:53:59 +08:00
Eddie Torres
6f97f5bbf8 Use Mock CSI Driver for MutableCSINodeAllocatableCount storage e2e test (#132373)
* Refactor MutableCSINodeAllocatableCount storage e2e test

Signed-off-by: Eddie Torres <torredil@amazon.com>

* Update var names and formatting

Signed-off-by: Eddie Torres <torredil@amazon.com>

* Update function names

Signed-off-by: Eddie Torres <torredil@amazon.com>

---------

Signed-off-by: Eddie Torres <torredil@amazon.com>
2025-06-18 14:02:57 -07:00
Kubernetes Prow Robot
d50a18731a Merge pull request #130942 from torredil/storage-e2e-51231
Implement CSI e2e test for MutableCSINodeAllocatableCount
2025-06-11 20:20:55 -07:00
Jan Safranek
b206264ffb Shorten long directory names with e2e pod logs
In a downstream test job, I've seen an e2e test failing to create a
directory for Pod logs when running an e2e test. Shorten them to 255
characters.

For example, consider this test: "[sig-storage] CSI Mock selinux on mount metrics and SELinuxWarningController SELinuxMount metrics [LinuxOnly] [Feature:SELinux] [Serial] error is not bumped on two Pods with a different policy RWX volume (MountOption + MountOption) [FeatureGate:SELinuxMountReadWriteOncePod] [Beta] [FeatureGate:SELinuxChangePolicy] [Beta] [FeatureGate:SELinuxMount] [Beta] [Feature:OffByDefault] [sig-storage, Feature:SELinux, Serial, FeatureGate:SELinuxMountReadWriteOncePod, Beta, FeatureGate:SELinuxChangePolicy, FeatureGate:SELinuxMount, Feature:OffByDefault, BetaOffByDefault]"

During execution, the test will create a directory
`error_is_not_bumped_on_two_Pods_with_Recursive_policy_and_a_different_context_on_RWX_volume_FeatureGate_SELinuxMountReadWriteOncePod_Beta_FeatureGate_SELinuxChangePolicy_Beta_FeatureGate_SELinuxMount_Beta_Feature_OffByDefault_`,
which has 226 characters and it's close to the limit (256).
2025-04-09 15:45:31 +02:00
torredil
88a3c000e5 Implement CSI e2e test for MutableCSINodeAllocatableCount 2025-03-20 14:33:38 +00:00
Jordan Liggitt
8090db5dcf Switch to private instances of rand for seeding for tests 2025-02-26 11:27:10 +01:00
carlory
c68fc01b77 return an error instead of recording a test failure
Signed-off-by: carlory <baofa.fan@daocloud.io>
2025-01-21 23:43:48 +08:00
carlory
aecf3238bf fix lint errors
Signed-off-by: carlory <baofa.fan@daocloud.io>
2025-01-19 20:46:37 +08:00
carlory
8b4eae24ab Move some exec helper functions from framework/volume to framework/pod 2025-01-18 21:42:42 +08:00
Kubernetes Prow Robot
b358cf3a79 Merge pull request #128972 from manishym/beta-e2e
Add crd from external snapshotter
2025-01-06 11:24:16 +01:00
Manish Yathnalli
446252b0c3 Add crd from external snapshotter
Signed-off-by: Manish Yathnalli <myathnal@redhat.com>
2024-12-20 15:59:43 +05:30
yaroslavborbat
e2f6232258 preserve options after remount for bind mounting
Signed-off-by: yaroslavborbat <yaroslav.752@gmail.com>
2024-10-23 19:35:57 +03:00
Kubernetes Prow Robot
d598a3ec0f Merge pull request #126326 from manishym/group_snapshot_e2e
Add end-to-end tests for Volume Group Snapshot
2024-10-08 22:58:22 +01:00
Manish
88f9f5250d Add end-to-end tests for Volume Group Snapshot
This commit introduces comprehensive e2e tests for the
Volume Group Snapshot feature to ensure robust validation and functionality.
Addresses issue [kubernetes#1080 ](kubernetes-csi/external-snapshotter#1080).

Signed-off-by: Manish <myathnal@redhat.com>
2024-10-08 16:24:29 +05:30
Jan Safranek
02101df58f Revert "fix test"
This reverts commit 62c5fd9438.
2024-10-07 10:00:18 +02:00
yaroslavborbat
62c5fd9438 fix test
Signed-off-by: yaroslavborbat <yaroslav.752@gmail.com>
2024-10-03 18:39:28 +03:00
Kubernetes Prow Robot
9fe06620f3 Merge pull request #124028 from huww98/kubelet-speedup
e2e/storage: speed up kubelet commands
2024-08-26 08:13:32 +01:00
carlory
0260c7d023 Promote VolumeAttributesClass to beta 2024-07-23 13:58:14 +08:00
Connor Catlett
ea58abfd99 Add Happy Path VolumeAttributesClass CSI E2E Tests
Signed-off-by: Connor Catlett <conncatl@amazon.com>
2024-06-10 20:03:54 +00:00
胡玮文
c916f5e755 e2e/framework/node: use gomega.Eventually to poll 2024-05-21 18:07:19 +08:00
胡玮文
f3f44f70bf e2e/storage: speed up kubectl commands
Speed up stopping by not waiting for Node not ready, `systemctl` will ensure
kubelet process stopped before return. This should save 40s per case.

Since stop command does not wait for not ready, start command needs to wait for
the next heartbeat to ensure we are checking status from new process.

implement restart by stop then start, to get heartbeat time when kubelet is
down. And we do not need to sleep 30s now. The sleep is moved to callers, since
they still need them to ensure the volume does not disappear.

Dropped support for non-systemd system.
2024-05-19 02:51:32 +08:00
liyuerich
f55588fa0b e2e_storage:stop using deprecated framework.ExpectError
Signed-off-by: liyuerich <yue.li@daocloud.io>
2024-05-09 18:10:33 +08:00
carlory
2bd7f4f8e0 storage e2e: update hostpath and mock images 2024-03-05 10:18:54 +08:00
Patrick Ohly
f2d34426f8 e2e: enhance SIGDescribe
framework.SIGDescribe is better because:
- Ginkgo uses the source code location of the test, not of the wrapper,
  when reporting progress.
- Additional annotations can be passed.

To make this a drop-in replacement, framework.SIGDescribe generates a function
that can be used instead of the former SIGDescribe functions.

windows.SIGDescribe contained some additional code to ensure that tests are
skipped when not running with a suitable node OS. This gets moved into a
separate wrapper generator, to allow using framework.SIGDescribe as intended.
To ensure that all callers were modified, the windows.sigDescribe isn't
exported anymore (wasn't necessary in the first place!).
2023-10-10 18:15:49 +02:00
宋文杰
8a368afa27 e2e_storage: stop using deprecated framework.ExpectEqual 2023-08-30 14:00:34 +08:00
Davanum Srinivas
90d185b7e1 Drop AWS kubelet credential provider and cleanup AWS storage e2e tests
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2023-03-07 09:00:12 -05:00
Patrick Ohly
136f89dfc5 e2e: use error wrapping with %w
The recently introduced failure handling in ExpectNoError depends on error
wrapping: if an error prefix gets added with `fmt.Errorf("foo: %v", err)`, then
ExpectNoError cannot detect that the root cause is an assertion failure and
then will add another useless "unexpected error" prefix and will not dump the
additional failure information (currently the backtrace inside the E2E
framework).

Instead of manually deciding on a case-by-case basis where %w is needed, all
error wrapping was updated automatically with

    sed -i "s/fmt.Errorf\(.*\): '*\(%s\|%v\)'*\",\(.* err)\)/fmt.Errorf\1: %w\",\3/" $(git grep -l 'fmt.Errorf' test/e2e*)

This may be unnecessary in some cases, but it's not wrong.
2023-02-06 15:39:13 +01:00
Antonio Ojea
7f5ae1c0c1 Revert "e2e: wait for pods with gomega" 2023-02-06 12:08:22 +01:00
Patrick Ohly
222f655062 e2e: use error wrapping with %w
The recently introduced failure handling in ExpectNoError depends on error
wrapping: if an error prefix gets added with `fmt.Errorf("foo: %v", err)`, then
ExpectNoError cannot detect that the root cause is an assertion failure and
then will add another useless "unexpected error" prefix and will not dump the
additional failure information (currently the backtrace inside the E2E
framework).

Instead of manually deciding on a case-by-case basis where %w is needed, all
error wrapping was updated automatically with

    sed -i "s/fmt.Errorf\(.*\): '*\(%s\|%v\)'*\",\(.* err)\)/fmt.Errorf\1: %w\",\3/" $(git grep -l 'fmt.Errorf' test/e2e*)

This may be unnecessary in some cases, but it's not wrong.
2023-01-31 13:01:39 +01:00
Kubernetes Prow Robot
3244ebd532 Merge pull request #113973 from my-git9/invalidurl
fix invalid url
2023-01-30 18:42:48 -08:00
David Porter
6cb022ff4a test: Update Host Exec to support node e2e
Node E2E tests do not run a scheduler, so the host exec pod must have
the `spec.nodeName` set explicitly.

Signed-off-by: David Porter <david@porter.me>
2023-01-26 00:37:01 -08:00
Patrick Ohly
2f6c4f5eab e2e: use Ginkgo context
All code must use the context from Ginkgo when doing API calls or polling for a
change, otherwise the code would not return immediately when the test gets
aborted.
2022-12-16 20:14:04 +01:00
Hemant Kumar
3d33178dab Start adding e2e for recovery from expansion failure 2022-12-14 15:28:05 -05:00
Patrick Ohly
d4729008ef e2e: simplify test cleanup
ginkgo.DeferCleanup has multiple advantages:
- The cleanup operation can get registered if and only if needed.
- No need to return a cleanup function that the caller must invoke.
- Automatically determines whether a context is needed, which will
  simplify the introduction of context parameters.
- Ginkgo's timeline shows when it executes the cleanup operation.
2022-12-13 08:09:01 +01:00
xin.li
5aeb9db52f fix invalid url
Signed-off-by: xin.li <xin.li@daocloud.io>
2022-11-17 18:24:54 +08:00
Jan Safranek
3fdd4ca7a4 Fix subpath disruptive tests
Add volumePath parameter to all disruptive checks, so subpath tests can use
"/test-volume" and disruptive tests can use "/mnt/volume1" for their
respective Pods.
2022-11-11 14:03:33 +01:00
Jan Safranek
d6c36736d5 Add mock CSI driver test for SELinux mount 2022-11-08 13:37:09 +01:00
Jan Safranek
802979c295 Add SELinux disruptive test 2022-11-08 12:42:20 +01:00
Patrick Ohly
dfdf88d4fa e2e: adapt to moved code
This is the result of automatically editing source files like this:

    go install golang.org/x/tools/cmd/goimports@latest
    find ./test/e2e* -name "*.go" | xargs env PATH=$GOPATH/bin:$PATH ./e2e-framework-sed.sh

with e2e-framework-sed.sh containing this:

sed -i \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainer(/e2epod.ExecCommandInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainerWithFullOutput(/e2epod.ExecCommandInContainerWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInContainer(/e2epod.ExecShellInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPod(/e2epod.ExecShellInPod(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPodWithFullOutput(/e2epod.ExecShellInPodWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecWithOptions(/e2epod.ExecWithOptions(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.MatchContainerOutput(/e2eoutput.MatchContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClient(/e2epod.NewPodClient(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClientNS(/e2epod.PodClientNS(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutput(/e2eoutput.TestContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutputRegexp(/e2eoutput.TestContainerOutputRegexp(\1, /" \
    -e "s/framework.AddOrUpdateLabelOnNode\b/e2enode.AddOrUpdateLabelOnNode/" \
    -e "s/framework.AllNodes\b/e2edebug.AllNodes/" \
    -e "s/framework.AllNodesReady\b/e2enode.AllNodesReady/" \
    -e "s/framework.ContainerResourceGatherer\b/e2edebug.ContainerResourceGatherer/" \
    -e "s/framework.ContainerResourceUsage\b/e2edebug.ContainerResourceUsage/" \
    -e "s/framework.CreateEmptyFileOnPod\b/e2eoutput.CreateEmptyFileOnPod/" \
    -e "s/framework.DefaultPodDeletionTimeout\b/e2epod.DefaultPodDeletionTimeout/" \
    -e "s/framework.DumpAllNamespaceInfo\b/e2edebug.DumpAllNamespaceInfo/" \
    -e "s/framework.DumpDebugInfo\b/e2eoutput.DumpDebugInfo/" \
    -e "s/framework.DumpNodeDebugInfo\b/e2edebug.DumpNodeDebugInfo/" \
    -e "s/framework.EtcdUpgrade\b/e2eproviders.EtcdUpgrade/" \
    -e "s/framework.EventsLister\b/e2edebug.EventsLister/" \
    -e "s/framework.ExecOptions\b/e2epod.ExecOptions/" \
    -e "s/framework.ExpectNodeHasLabel\b/e2enode.ExpectNodeHasLabel/" \
    -e "s/framework.ExpectNodeHasTaint\b/e2enode.ExpectNodeHasTaint/" \
    -e "s/framework.GCEUpgradeScript\b/e2eproviders.GCEUpgradeScript/" \
    -e "s/framework.ImagePrePullList\b/e2epod.ImagePrePullList/" \
    -e "s/framework.KubectlBuilder\b/e2ekubectl.KubectlBuilder/" \
    -e "s/framework.LocationParamGKE\b/e2eproviders.LocationParamGKE/" \
    -e "s/framework.LogSizeDataTimeseries\b/e2edebug.LogSizeDataTimeseries/" \
    -e "s/framework.LogSizeGatherer\b/e2edebug.LogSizeGatherer/" \
    -e "s/framework.LogsSizeData\b/e2edebug.LogsSizeData/" \
    -e "s/framework.LogsSizeDataSummary\b/e2edebug.LogsSizeDataSummary/" \
    -e "s/framework.LogsSizeVerifier\b/e2edebug.LogsSizeVerifier/" \
    -e "s/framework.LookForStringInLog\b/e2eoutput.LookForStringInLog/" \
    -e "s/framework.LookForStringInPodExec\b/e2eoutput.LookForStringInPodExec/" \
    -e "s/framework.LookForStringInPodExecToContainer\b/e2eoutput.LookForStringInPodExecToContainer/" \
    -e "s/framework.MasterAndDNSNodes\b/e2edebug.MasterAndDNSNodes/" \
    -e "s/framework.MasterNodes\b/e2edebug.MasterNodes/" \
    -e "s/framework.MasterUpgradeGKE\b/e2eproviders.MasterUpgradeGKE/" \
    -e "s/framework.NewKubectlCommand\b/e2ekubectl.NewKubectlCommand/" \
    -e "s/framework.NewLogsVerifier\b/e2edebug.NewLogsVerifier/" \
    -e "s/framework.NewNodeKiller\b/e2enode.NewNodeKiller/" \
    -e "s/framework.NewResourceUsageGatherer\b/e2edebug.NewResourceUsageGatherer/" \
    -e "s/framework.NodeHasTaint\b/e2enode.NodeHasTaint/" \
    -e "s/framework.NodeKiller\b/e2enode.NodeKiller/" \
    -e "s/framework.NodesSet\b/e2edebug.NodesSet/" \
    -e "s/framework.PodClient\b/e2epod.PodClient/" \
    -e "s/framework.RemoveLabelOffNode\b/e2enode.RemoveLabelOffNode/" \
    -e "s/framework.ResourceConstraint\b/e2edebug.ResourceConstraint/" \
    -e "s/framework.ResourceGathererOptions\b/e2edebug.ResourceGathererOptions/" \
    -e "s/framework.ResourceUsagePerContainer\b/e2edebug.ResourceUsagePerContainer/" \
    -e "s/framework.ResourceUsageSummary\b/e2edebug.ResourceUsageSummary/" \
    -e "s/framework.RunHostCmd\b/e2eoutput.RunHostCmd/" \
    -e "s/framework.RunHostCmdOrDie\b/e2eoutput.RunHostCmdOrDie/" \
    -e "s/framework.RunHostCmdWithFullOutput\b/e2eoutput.RunHostCmdWithFullOutput/" \
    -e "s/framework.RunHostCmdWithRetries\b/e2eoutput.RunHostCmdWithRetries/" \
    -e "s/framework.RunKubectl\b/e2ekubectl.RunKubectl/" \
    -e "s/framework.RunKubectlInput\b/e2ekubectl.RunKubectlInput/" \
    -e "s/framework.RunKubectlOrDie\b/e2ekubectl.RunKubectlOrDie/" \
    -e "s/framework.RunKubectlOrDieInput\b/e2ekubectl.RunKubectlOrDieInput/" \
    -e "s/framework.RunKubectlWithFullOutput\b/e2ekubectl.RunKubectlWithFullOutput/" \
    -e "s/framework.RunKubemciCmd\b/e2ekubectl.RunKubemciCmd/" \
    -e "s/framework.RunKubemciWithKubeconfig\b/e2ekubectl.RunKubemciWithKubeconfig/" \
    -e "s/framework.SingleContainerSummary\b/e2edebug.SingleContainerSummary/" \
    -e "s/framework.SingleLogSummary\b/e2edebug.SingleLogSummary/" \
    -e "s/framework.TimestampedSize\b/e2edebug.TimestampedSize/" \
    -e "s/framework.WaitForAllNodesSchedulable\b/e2enode.WaitForAllNodesSchedulable/" \
    -e "s/framework.WaitForSSHTunnels\b/e2enode.WaitForSSHTunnels/" \
    -e "s/framework.WorkItem\b/e2edebug.WorkItem/" \
    "$@"

for i in "$@"; do
    # Import all sub packages and let goimports figure out which of those
    # are redundant (= already imported) or not needed.
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2edebug "k8s.io/kubernetes/test/e2e/framework/debug"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2ekubectl "k8s.io/kubernetes/test/e2e/framework/kubectl"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2enode "k8s.io/kubernetes/test/e2e/framework/node"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eoutput "k8s.io/kubernetes/test/e2e/framework/pod/output"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2epod "k8s.io/kubernetes/test/e2e/framework/pod"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eproviders "k8s.io/kubernetes/test/e2e/framework/providers"' "$i"
    goimports -w "$i"
done
2022-10-06 08:19:47 +02:00
Patrick Ohly
5614a9d064 e2e framework: eliminate interim sub packages
The "todo" packages were necessary while moving code around to avoid hitting
cyclic dependencies. Now that any sub package can depend on the framework, they
are no longer needed and the code can be moved into the normal sub packages.
2022-10-06 08:16:47 +02:00
Patrick Ohly
a46fea53e6 e2e framework: move pod exec and create into sub package 2022-10-06 08:16:47 +02:00
Humble Chirammal
9e9fc2be88 various corrections in test/e2e package
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-16 18:59:30 +05:30
Patrick Ohly
24230b4307 e2e storage: support creating ReplicaSet
This is useful for running a driver on a subset of all ready nodes:

- use e2enode.GetBoundedReadySchedulableNodes with a suitable
  maximum number of nodes to determine how much nodes are available
  for a test
- define pod anti-affinity in the PodTemplate:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/instance: xxxxxxx
            topologyKey: kubernetes.io/hostname
- set the ReplicaSetSpec.Replicas value to the number of nodes
2022-09-12 14:38:28 +02:00
Joe Betz
7be77f9ecc changes code to improve output for files under test/e2e/storage 2022-09-02 17:43:41 +05:30
Fabio Bertinatto
e13c9be88a e2e: use custom timeouts in GetSnapshotContentFromSnapshot() 2022-08-26 15:16:35 -03:00
Davanum Srinivas
a9593d634c Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-07-26 13:14:05 -04:00
Abirdcfly
00b9ead02c cleanup: remove duplicate import
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-14 11:25:19 +08:00
Dave Chen
fd4b5b629b Stop using the deprecated method CurrentGinkgoTestDescription
Besides, the using of method might lead to a `concurrent map writes`
issue per the discussion here: https://github.com/onsi/ginkgo/issues/970

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
ece0bb3522 Adapt to new type of GinkgoWriter
Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:53 +08:00
Dave Chen
857458cfa5 update ginkgo from v1 to v2 and gomega to 1.19.0
- update all the import statements
- run hack/pin-dependency.sh to change pinned dependency versions
- run hack/update-vendor.sh to update go.mod files and the vendor directory
- update the method signatures for custom reporters

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:46 +08:00