Commit Graph

137 Commits

Author SHA1 Message Date
Patrick Ohly
dfdf88d4fa e2e: adapt to moved code
This is the result of automatically editing source files like this:

    go install golang.org/x/tools/cmd/goimports@latest
    find ./test/e2e* -name "*.go" | xargs env PATH=$GOPATH/bin:$PATH ./e2e-framework-sed.sh

with e2e-framework-sed.sh containing this:

sed -i \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainer(/e2epod.ExecCommandInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecCommandInContainerWithFullOutput(/e2epod.ExecCommandInContainerWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInContainer(/e2epod.ExecShellInContainer(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPod(/e2epod.ExecShellInPod(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecShellInPodWithFullOutput(/e2epod.ExecShellInPodWithFullOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.ExecWithOptions(/e2epod.ExecWithOptions(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.MatchContainerOutput(/e2eoutput.MatchContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClient(/e2epod.NewPodClient(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.PodClientNS(/e2epod.PodClientNS(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutput(/e2eoutput.TestContainerOutput(\1, /" \
    -e "s/\(f\|fr\|\w\w*\.[fF]\w*\)\.TestContainerOutputRegexp(/e2eoutput.TestContainerOutputRegexp(\1, /" \
    -e "s/framework.AddOrUpdateLabelOnNode\b/e2enode.AddOrUpdateLabelOnNode/" \
    -e "s/framework.AllNodes\b/e2edebug.AllNodes/" \
    -e "s/framework.AllNodesReady\b/e2enode.AllNodesReady/" \
    -e "s/framework.ContainerResourceGatherer\b/e2edebug.ContainerResourceGatherer/" \
    -e "s/framework.ContainerResourceUsage\b/e2edebug.ContainerResourceUsage/" \
    -e "s/framework.CreateEmptyFileOnPod\b/e2eoutput.CreateEmptyFileOnPod/" \
    -e "s/framework.DefaultPodDeletionTimeout\b/e2epod.DefaultPodDeletionTimeout/" \
    -e "s/framework.DumpAllNamespaceInfo\b/e2edebug.DumpAllNamespaceInfo/" \
    -e "s/framework.DumpDebugInfo\b/e2eoutput.DumpDebugInfo/" \
    -e "s/framework.DumpNodeDebugInfo\b/e2edebug.DumpNodeDebugInfo/" \
    -e "s/framework.EtcdUpgrade\b/e2eproviders.EtcdUpgrade/" \
    -e "s/framework.EventsLister\b/e2edebug.EventsLister/" \
    -e "s/framework.ExecOptions\b/e2epod.ExecOptions/" \
    -e "s/framework.ExpectNodeHasLabel\b/e2enode.ExpectNodeHasLabel/" \
    -e "s/framework.ExpectNodeHasTaint\b/e2enode.ExpectNodeHasTaint/" \
    -e "s/framework.GCEUpgradeScript\b/e2eproviders.GCEUpgradeScript/" \
    -e "s/framework.ImagePrePullList\b/e2epod.ImagePrePullList/" \
    -e "s/framework.KubectlBuilder\b/e2ekubectl.KubectlBuilder/" \
    -e "s/framework.LocationParamGKE\b/e2eproviders.LocationParamGKE/" \
    -e "s/framework.LogSizeDataTimeseries\b/e2edebug.LogSizeDataTimeseries/" \
    -e "s/framework.LogSizeGatherer\b/e2edebug.LogSizeGatherer/" \
    -e "s/framework.LogsSizeData\b/e2edebug.LogsSizeData/" \
    -e "s/framework.LogsSizeDataSummary\b/e2edebug.LogsSizeDataSummary/" \
    -e "s/framework.LogsSizeVerifier\b/e2edebug.LogsSizeVerifier/" \
    -e "s/framework.LookForStringInLog\b/e2eoutput.LookForStringInLog/" \
    -e "s/framework.LookForStringInPodExec\b/e2eoutput.LookForStringInPodExec/" \
    -e "s/framework.LookForStringInPodExecToContainer\b/e2eoutput.LookForStringInPodExecToContainer/" \
    -e "s/framework.MasterAndDNSNodes\b/e2edebug.MasterAndDNSNodes/" \
    -e "s/framework.MasterNodes\b/e2edebug.MasterNodes/" \
    -e "s/framework.MasterUpgradeGKE\b/e2eproviders.MasterUpgradeGKE/" \
    -e "s/framework.NewKubectlCommand\b/e2ekubectl.NewKubectlCommand/" \
    -e "s/framework.NewLogsVerifier\b/e2edebug.NewLogsVerifier/" \
    -e "s/framework.NewNodeKiller\b/e2enode.NewNodeKiller/" \
    -e "s/framework.NewResourceUsageGatherer\b/e2edebug.NewResourceUsageGatherer/" \
    -e "s/framework.NodeHasTaint\b/e2enode.NodeHasTaint/" \
    -e "s/framework.NodeKiller\b/e2enode.NodeKiller/" \
    -e "s/framework.NodesSet\b/e2edebug.NodesSet/" \
    -e "s/framework.PodClient\b/e2epod.PodClient/" \
    -e "s/framework.RemoveLabelOffNode\b/e2enode.RemoveLabelOffNode/" \
    -e "s/framework.ResourceConstraint\b/e2edebug.ResourceConstraint/" \
    -e "s/framework.ResourceGathererOptions\b/e2edebug.ResourceGathererOptions/" \
    -e "s/framework.ResourceUsagePerContainer\b/e2edebug.ResourceUsagePerContainer/" \
    -e "s/framework.ResourceUsageSummary\b/e2edebug.ResourceUsageSummary/" \
    -e "s/framework.RunHostCmd\b/e2eoutput.RunHostCmd/" \
    -e "s/framework.RunHostCmdOrDie\b/e2eoutput.RunHostCmdOrDie/" \
    -e "s/framework.RunHostCmdWithFullOutput\b/e2eoutput.RunHostCmdWithFullOutput/" \
    -e "s/framework.RunHostCmdWithRetries\b/e2eoutput.RunHostCmdWithRetries/" \
    -e "s/framework.RunKubectl\b/e2ekubectl.RunKubectl/" \
    -e "s/framework.RunKubectlInput\b/e2ekubectl.RunKubectlInput/" \
    -e "s/framework.RunKubectlOrDie\b/e2ekubectl.RunKubectlOrDie/" \
    -e "s/framework.RunKubectlOrDieInput\b/e2ekubectl.RunKubectlOrDieInput/" \
    -e "s/framework.RunKubectlWithFullOutput\b/e2ekubectl.RunKubectlWithFullOutput/" \
    -e "s/framework.RunKubemciCmd\b/e2ekubectl.RunKubemciCmd/" \
    -e "s/framework.RunKubemciWithKubeconfig\b/e2ekubectl.RunKubemciWithKubeconfig/" \
    -e "s/framework.SingleContainerSummary\b/e2edebug.SingleContainerSummary/" \
    -e "s/framework.SingleLogSummary\b/e2edebug.SingleLogSummary/" \
    -e "s/framework.TimestampedSize\b/e2edebug.TimestampedSize/" \
    -e "s/framework.WaitForAllNodesSchedulable\b/e2enode.WaitForAllNodesSchedulable/" \
    -e "s/framework.WaitForSSHTunnels\b/e2enode.WaitForSSHTunnels/" \
    -e "s/framework.WorkItem\b/e2edebug.WorkItem/" \
    "$@"

for i in "$@"; do
    # Import all sub packages and let goimports figure out which of those
    # are redundant (= already imported) or not needed.
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2edebug "k8s.io/kubernetes/test/e2e/framework/debug"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2ekubectl "k8s.io/kubernetes/test/e2e/framework/kubectl"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2enode "k8s.io/kubernetes/test/e2e/framework/node"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eoutput "k8s.io/kubernetes/test/e2e/framework/pod/output"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2epod "k8s.io/kubernetes/test/e2e/framework/pod"' "$i"
    sed -i -e '/"k8s.io.kubernetes.test.e2e.framework"/a e2eproviders "k8s.io/kubernetes/test/e2e/framework/providers"' "$i"
    goimports -w "$i"
done
2022-10-06 08:19:47 +02:00
Patrick Ohly
5614a9d064 e2e framework: eliminate interim sub packages
The "todo" packages were necessary while moving code around to avoid hitting
cyclic dependencies. Now that any sub package can depend on the framework, they
are no longer needed and the code can be moved into the normal sub packages.
2022-10-06 08:16:47 +02:00
Patrick Ohly
a46fea53e6 e2e framework: move pod exec and create into sub package 2022-10-06 08:16:47 +02:00
Humble Chirammal
9e9fc2be88 various corrections in test/e2e package
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-16 18:59:30 +05:30
Patrick Ohly
24230b4307 e2e storage: support creating ReplicaSet
This is useful for running a driver on a subset of all ready nodes:

- use e2enode.GetBoundedReadySchedulableNodes with a suitable
  maximum number of nodes to determine how much nodes are available
  for a test
- define pod anti-affinity in the PodTemplate:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app.kubernetes.io/instance: xxxxxxx
            topologyKey: kubernetes.io/hostname
- set the ReplicaSetSpec.Replicas value to the number of nodes
2022-09-12 14:38:28 +02:00
Joe Betz
7be77f9ecc changes code to improve output for files under test/e2e/storage 2022-09-02 17:43:41 +05:30
Fabio Bertinatto
e13c9be88a e2e: use custom timeouts in GetSnapshotContentFromSnapshot() 2022-08-26 15:16:35 -03:00
Davanum Srinivas
a9593d634c Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-07-26 13:14:05 -04:00
Abirdcfly
00b9ead02c cleanup: remove duplicate import
Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-07-14 11:25:19 +08:00
Dave Chen
fd4b5b629b Stop using the deprecated method CurrentGinkgoTestDescription
Besides, the using of method might lead to a `concurrent map writes`
issue per the discussion here: https://github.com/onsi/ginkgo/issues/970

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:46:11 +08:00
Dave Chen
ece0bb3522 Adapt to new type of GinkgoWriter
Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:53 +08:00
Dave Chen
857458cfa5 update ginkgo from v1 to v2 and gomega to 1.19.0
- update all the import statements
- run hack/pin-dependency.sh to change pinned dependency versions
- run hack/update-vendor.sh to update go.mod files and the vendor directory
- update the method signatures for custom reporters

Signed-off-by: Dave Chen <dave.chen@arm.com>
2022-07-08 10:44:46 +08:00
Kubernetes Prow Robot
313d9fb0b1 Merge pull request #109311 from my-git9/creatego
Modify function parameters
2022-05-05 13:50:41 -07:00
Jordan Liggitt
410ac59c0d Remove PodSecurityPolicy admission plugin 2022-05-04 16:00:56 -04:00
xin.li
e0496cee62 Modify function parameters
Signed-off-by: xin.li <xin.li@daocloud.io>
2022-04-05 22:05:39 +08:00
Masaki Kimura
cd634b59ac Implement initial E2E test for AnyVolumeDataSource 2022-04-04 16:10:39 +00:00
Patrick Ohly
ff5558edc3 test: support storage tests with non-standard kubelet root directory
Some storage tests deploy DaemonSets which hard-code /var/lib/kubelet as root
directory for kubelet registration and pod directory. There was already a
parameter which allowed specifying the root directory, just with a very
confusing name ("--volume-dir") and matching field name. A --kubelet-root-dir
parameters gets added because this may make it easier to find the parameter,
with the old name preserved as an alias for the same field for backwards
compatibility.
2022-03-03 12:34:18 +01:00
Kubernetes Prow Robot
1f8b1b84fb Merge pull request #102886 from gnufied/add-local-expansion
Add support for expanding local volumes
2021-08-12 11:55:50 -07:00
Hemant Kumar
d166cabd9e Add e2e for local volume expansion 2021-08-06 15:08:33 -04:00
lzj
7cf30d5690 fix/remove-errors-pkg 2021-06-24 16:09:59 +08:00
Patrick Ohly
53af002742 storage e2e: capture driver pod changes, including all events
As seen in https://github.com/kubernetes/kubernetes/issues/102452, we
currently don't have pod events for the CSI driver pods because of the
different namespace and would need them to determine whether the
driver gets evicted.

Previously, only changes of the pods where logged. Perhaps even more
interesting are events in the namespace.
2021-06-02 17:37:03 +02:00
Masaki Kimura
382f37142c storage e2e: auto detect sector size 2021-05-11 22:32:05 +00:00
drfish
244d7a5d67 Remove suffixes for VolumeSnapshotClasses in E2E tests 2021-03-23 21:24:28 +08:00
Benjamin Elder
56e092e382 hack/update-bazel.sh 2021-02-28 15:17:29 -08:00
Jiawei Wang
c4dfee6262 Fix storage e2e snapshot test deletion order 2021-02-17 16:17:08 -08:00
Patrick Ohly
d27c541b45 test: bump CSI container versions for mock testing
These are the latest stable releases. We should test with those.
The newer external-provisioner no longer needs (and doesn't support)
the --provisioner parameter.
2021-02-05 21:08:39 +01:00
Jiawei Wang
356bea6c9f Add storage framework and address comments 2020-12-10 22:48:06 -08:00
Jiawei Wang
988563f8f5 Extract testsuite api to a separate package
Extract TestSuite, TestDriver, TestPattern, TestConfig
and VolumeResource, SnapshotVolumeResource from testsuite
package and put them into a new package called api.

The ultimate goal here is to make the testsuites as clean
as possible. And only testsuites in the package.
2020-12-10 11:12:51 -08:00
Fabio Bertinatto
ee082985c2 e2e: promote use of functions that support custom timeouts in storage tests.
WaitForPodSuccessInNamespace[Slow] are replaced by WaitForPodSuccessInNamespaceTimeout(),
so that custom timeouts are used instead of the hardcoded ones.
2020-12-02 16:14:13 -03:00
Fabio Bertinatto
c82626f96f e2e: use custom timeouts in all storage E2E tests 2020-12-02 15:57:58 -03:00
Christian Huffman
38071e74cf Correct rebase issues 2020-11-12 17:09:49 -05:00
Christian Huffman
4d2d063635 Included e2e test for CSIDriver FSGroupPolicy 2020-11-12 16:30:38 -05:00
Kubernetes Prow Robot
0b1d94a849 Merge pull request #96479 from jingxu97/nov/testwindows
Update storage test suits for Windows
2020-11-12 13:17:29 -08:00
Shihang Zhang
d2859cd89b plumb service account token down to csi driver 2020-11-12 09:26:43 -08:00
Jing Xu
56373aa10d Update storage test suits for Windows
Update storage test suits to enable some tests on Windows

Change-Id: I1f596bb9f7a0f41ae398aec43406ee5c753cdce3
2020-11-11 15:22:47 -08:00
Saikat Roychowdhury
a07096952b FsgroupChange policy test suite 2020-11-10 08:36:07 +00:00
Kubernetes Prow Robot
d39562fd93 Merge pull request #95863 from RaunakShah/snap_e2e
Volume snapshot e2e test to validate PVC finalizer
2020-11-05 10:53:06 -08:00
shahra
e95af138b5 Volume snapshot e2e test to validate
VolumeSnapshotContent and PVC finalizer
2020-11-04 14:08:24 -08:00
Jan Safranek
96e3bc1bfa Log PodExec stdout + stderr
e2e tests should log stdout / stderr of failed commands executed via
PodExec. "command terminated with exit code 1" is not really useful.
2020-10-26 13:22:18 +01:00
Niels de Vos
f2bf2ab76e e2e/storage: disable caching when writing/reading to block PVCs
By passing "oflag=nocache" and "iflag=direct", caching should be
disabled while writing/reading with "dd" to a block device. The
TestConcurrentAccessToSingleVolume() test is known to fail with certain
storage backends (like Ceph RBD) when caching is enabled.

The default BusyBox image used for testing does not support the required
options for "dd". So instead of running with BusyBox, run the test with
a Debian image.
2020-09-25 14:38:32 +02:00
Andi Li
7918f69055 Remove custom hash function, generate a random UUID for snapshot and snapcontent name instead 2020-08-28 12:36:15 -04:00
Andi Li
a4498846b1 Quick commit 2020-08-28 12:36:15 -04:00
Andi Li
ed85129831 Minor cleanup in snapshot test suite. 2020-08-28 12:29:07 -04:00
Jiawei Wang
d030c64743 [e2e]Bind csi driver namespace to test namespace to avoid namespace collision 2020-08-12 10:20:30 -07:00
Hemant Kumar
f91d448e1b Add tests for out of band attachments 2020-08-07 08:23:25 -04:00
Patrick Ohly
567ce87aee CSIStorageCapacity: E2E test with mock driver
We can create CSIStorageCapacity objects manually, therefore we don't
need the updated external-provisioner for these tests.
2020-07-08 08:02:26 +02:00
Grant Griffiths
e1f0e4cd9f Add retain policy test and refactor snapshottable tests
Signed-off-by: Grant Griffiths <grant@portworx.com>
2020-06-17 19:53:53 -07:00
Hemant Kumar
da941d8d3e Create mock CSI driver resources in different namespace 2020-05-13 11:16:00 -04:00
Hemant Kumar
708261e06c Make AfterSuite hooks ordered
ginkgo has a weird bug that - AfterEach does not get called when
testsuite exits with certain kind of interrupt (Ctrl-C for example).
More info - https://github.com/onsi/ginkgo/issues/222

We workaround this issue in Kubernetes by adding a special hook into
AfterSuite call, but AfterSuite can not be used to peforms certain
kind of cleanup because it can race with AfterEach hook and
framework.AfterEach hook will set framework.ClientSet to nil.

This presents a problem in cleaning up CSI driver and testpods. This
PR removes cleanup of driver manifest via CleanupAction because that
is not safe and racy (such as f.ClientSet may disappear!) and makes
AfterSuite hooks run in a ordered fashion
2020-05-13 11:15:27 -04:00
drfish
dfab6b637f Update .import-aliases for e2e test framework 2020-03-25 11:40:02 +08:00