Commit Graph

2867 Commits

Author SHA1 Message Date
NickrenREN
81e360d20f update volume plugins accordingly 2018-08-14 11:12:54 +08:00
NickrenREN
c7e4466873 attacher/detacher refactor 2018-08-14 11:12:41 +08:00
Kenjiro Nakayama
32117163e4 Support mount options for cephfs with ceph-fuse mount
When cephfs uses ceph-fuse for the mount command, mount option and
readOnly options are disregarded. This patch adds to ceph-fuse as
well.
2018-08-13 18:09:28 +09:00
Kubernetes Submit Queue
1dfe2e8bff Merge pull request #67140 from NetApp/multipath-race-fix
Automatic merge from submit-queue (batch tested with PRs 67017, 67190, 67110, 67140, 66873). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add wait loop for multipath devices to appear

It takes a variable amount of time for the multipath daemon
to create /dev/dm-XX in response to new LUNs being discovered.
The old iscsi_util code only discovered the multipath device
if it was created quickly enough, but in a significant number
of cases, kubelet would grab one of the individual paths and
put a filesystem it on before multipathd could construct a
multipath device.

This change waits for the multipath device to get created for
up to 10 seconds, but only if the PV actually had more than
one portal.

fixes #60894

```release-note
Dynamic provisions that create iSCSI PVs can ensure that multipath is used by specifying 2 or more target portals in the PV, which will cause kubelet to wait up to 10 seconds for the multipath device. PVs with just one portal continue to work as before, with kubelet not waiting for the multipath device and just using the first disk it finds.
```
2018-08-10 17:01:16 -07:00
Kubernetes Submit Queue
032a096d86 Merge pull request #67110 from verult/kubelet-nodeid
Automatic merge from submit-queue (batch tested with PRs 67017, 67190, 67110, 67140, 66873). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

CSI plugin now calls NodeGetInfo() to get driver's node ID

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #67040

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
/sig storage
@sbezverk @vladimirvivien @saad-ali
2018-08-10 17:01:13 -07:00
Kubernetes Submit Queue
6696ae39ee Merge pull request #66332 from jianglingxia/jlx-k8s-201807181704
Automatic merge from submit-queue (batch tested with PRs 66602, 67178, 67207, 67125, 66332). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add UT to RBD volume test of TestGetAccessModes and TestRequiresRemount

**What this PR does / why we need it**:
Add UT to RBD volume test of TestGetAccessModes and TestRequiresRemount
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-08-10 14:13:19 -07:00
Kubernetes Submit Queue
6c77dce9be Merge pull request #67220 from verult/vsphere-panic
Automatic merge from submit-queue (batch tested with PRs 67058, 67083, 67220, 67222, 67209). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixed unsafe type cast in vSphere volume plugin

**What this PR does / why we need it**: Fixes the controller manager panic caused by vSphere volumes being used on the wrong cloud provider.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #67218

**Release note**:

```release-note
NONE
```

/assign @saad-ali
2018-08-09 20:58:10 -07:00
Cheng Xing
85c3de40e5 Fixed vsphere volume plugin unsafe type cast; added unit tests 2018-08-09 16:15:59 -07:00
Kubernetes Submit Queue
facc84752b Merge pull request #67184 from linyouchong/pr-0809
Automatic merge from submit-queue (batch tested with PRs 67195, 67184). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove redundant code in aws_ebs_block.go

**What this PR does / why we need it**:
Remove redundant code in aws_ebs_block.go
There is the same code in aws_ebs.go

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE

**Special notes for your reviewer**:
NONE

**Release note**:
```release-note
NONE
```

/sig storage
2018-08-09 12:18:21 -07:00
Masaki Kimura
d03434e370 Fix provision fail issue for aws provisioner if fsType specified
fixes: #67146
2018-08-09 13:32:34 +00:00
linyouchong
5472f445dc Remove redundant code in aws_ebs_block.go 2018-08-09 15:57:05 +08:00
Pengfei Ni
7962954053 Parse zoned first before using it 2018-08-09 10:23:53 +08:00
Cheng Xing
7fa120c18c CSI plugin now calls NodeGetInfo() to get driver's node ID 2018-08-08 13:15:43 -07:00
Pengfei Ni
30fe79d63f Add DynamicProvisioningScheduling and VolumeScheduling support for AzureDisk 2018-08-08 17:05:46 +08:00
Ben Swartzlander
39e52ddae5 Add wait loop for multipath devices to appear
It takes a variable amount of time for the multipath daemon
to create /dev/dm-XX in response to new LUNs being discovered.
The old iscsi_util code only discovered the multipath device
if it was created quickly enough, but in a significant number
of cases, kubelet would grab one of the individual paths and
put a filesystem it on before multipathd could construct a
multipath device.

This change waits for the multipath device to get created for
up to 10 seconds, but only if the PV actually had more than
one portal.
2018-08-08 00:44:45 -04:00
Chakri Nelluri
93a19fce28 Ignore EIO error in unmount path 2018-08-07 21:04:39 -04:00
Kubernetes Submit Queue
14bcbb9450 Merge pull request #66553 from feiskyer/azure-disk-availablity-zone
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add availability zones support to Azure managed disks

**What this PR does / why we need it**:

Continue of [Azure Availability Zone feature](https://github.com/kubernetes/features/issues/586).

This PR adds availability zone support for Azure managed disks and its storage class. Zoned managed disks is enabled by default if there are zoned nodes in the cluster.

The zone could also be customized by `zone` or `zones` parameter, e.g.

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
  name: managed-disk-zone-1
parameters:
  zone: "southeastasia-1"
  # zones: "southeastasia-1,"southeastasia-2"
  cachingmode: None
  kind: Managed
  storageaccounttype: Standard_LRS
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Delete
volumeBindingMode: Immediate
```

All zoned AzureDisk PV will also be labeled with its availability zone, e.g.

```sh
$ kubectl get pvc pvc-azuredisk-az-1
NAME                 STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
pvc-azuredisk-az-1   Bound     pvc-5ad0c7b8-8f0b-11e8-94f2-000d3a07de8c   5Gi        RWO            managed-disk-zone-1   2h

$ kubectl get pv pvc-5ad0c7b8-8f0b-11e8-94f2-000d3a07de8c --show-labels
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                        STORAGECLASS          REASON    AGE       LABELS
pvc-5ad0c7b8-8f0b-11e8-94f2-000d3a07de8c   5Gi        RWO            Delete           Bound     default/pvc-azuredisk-az-1   managed-disk-zone-1             2h        failure-domain.beta.kubernetes.io/region=southeastasia,failure-domain.beta.kubernetes.io/zone=southeastasia-1
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

See also the [KEP](https://github.com/kubernetes/community/pull/2364).

DynamicProvisioningScheduling feature would be added in a following PR.

**Release note**:

```release-note
Azure managed disks now support availability zones and new parameters `zoned`, `zone` and `zones` are added for AzureDisk storage class.
```

/kind feature
/sig azure
/assign @brendandburns @khenidak @andyzhangx
2018-08-06 02:02:54 -07:00
Kubernetes Submit Queue
cb1ef9f7e8 Merge pull request #64815 from dixudx/hostname_empty
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

error out empty hostname

**What this PR does / why we need it**:
For linux, the hostname is read from file `/proc/sys/kernel/hostname` directly, which can be overwritten with whitespaces.

Should error out such invalid hostnames.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubernetes/kubeadm#835

**Special notes for your reviewer**:
/cc luxas timothysc 

**Release note**:

```release-note
nodes: improve handling of erroneous host names
```
2018-08-03 17:13:32 -07:00
Kubernetes Submit Queue
55d6a013e6 Merge pull request #66675 from humblec/log-file
Automatic merge from submit-queue (batch tested with PRs 61389, 66817, 66903, 66675, 66965). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Provide an option to supply log-file mount option for gluster plugin.

At present, `log-file` location of glusterfs client mount is decided
by the plugin, however at times, users/admin want to configure log-file
of gluster fuse client process on their own in some setups. Even though they can pass log-file
mount option through storage class, before this patch glusterfs plugin
always discard it. This patch enable/give preference
to admin supplied log-file mount option if specified in storage class.
If the log-file mount option is incomplete or wrong, plugin fallback to the
location which is carved out by the combination of pvc and pod name.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-08-03 07:33:16 -07:00
Kubernetes Submit Queue
b95ae9312c Merge pull request #61389 from linyouchong/linyouchong-20180320
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

csiAttacher: check deviceMountPath before hasStageUnstageCapability

csiAttacher#MountDevice: it is better to check `deviceMountPath` before `hasStageUnstageCapability`

**Release note**:
```
NONE
```
/release-note-none
/kind cleanup
/sig storage
2018-08-03 07:02:46 -07:00
Humble Chirammal
9e29ab985b Provide an option to supply log-file mount option for gluster plugin.
At present, `log-file` location of glusterfs client mount is decided
by the plugin, however at times, users/admin want to configure log-file
of gluster fuse client process on their own in some setups. Even though they can pass log-file
mount option through storage class, before this patch glusterfs plugin
always discard it. This patch enable/give preference
to admin supplied log-file mount option if specified in storage class.
If the log-file mount option is incomplete or wrong, plugin fallback to the
location which is carved out by the combination of pvc and pod name.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-08-02 22:09:49 +05:30
Zhou Peng
92b7810e12 pkg/volume/cephfs: fix doc typo, nfs -> cephfs 2018-08-02 16:11:02 +08:00
Kubernetes Submit Queue
d3c09656d7 Merge pull request #66832 from msau42/udev
Automatic merge from submit-queue (batch tested with PRs 64645, 66832). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Detect if GCE PD udev link is wrong and try to correct it

**What this PR does / why we need it**:
udev can miss scsi events and not update device links properly when disks are detached and reattached to the same node in a different order.  This PR workarounds the issue by comparing the udev link name, which is the PD disk name, with the scsi disk serial number returned from scsi_id, and prevents mounting of the disk if the link is wrong.  It will also run `udevadm trigger` on the disk to try to correct the bad link.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
This fix prevents a GCE PD volume from being mounted if the udev device link is stale and tries to correct the link.
```
2018-08-01 17:13:03 -07:00
Kubernetes Submit Queue
22d0ef2a8e Merge pull request #65730 from ddebroy/ebs-affinity1
Automatic merge from submit-queue (batch tested with PRs 65730, 66615, 66684, 66519, 66510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add DynamicProvisioningScheduling support for EBS

**What this PR does / why we need it**:

This PR adds support for the DynamicProvisioningScheduling feature in EBS. With this in place, if VolumeBindingMode: WaitForFirstConsumer is specified in a EBS storageclass and DynamicProvisioningScheduling is enabled, EBS provisioner will use the selected node's LabelZoneFailureDomain as the zone to provision the EBS volume in.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
Related to #63232

Sample `describe pv` output with NodeAffinity populated:

```
~$ kubectl describe pv pvc-f9d2138b-7e3e-11e8-a4ea-064124617820
Name:              pvc-f9d2138b-7e3e-11e8-a4ea-064124617820
Labels:            failure-domain.beta.kubernetes.io/region=us-west-2
                   failure-domain.beta.kubernetes.io/zone=us-west-2a
Annotations:       kubernetes.io/createdby=aws-ebs-dynamic-provisioner
                   pv.kubernetes.io/bound-by-controller=yes
                   pv.kubernetes.io/provisioned-by=kubernetes.io/aws-ebs
Finalizers:        [kubernetes.io/pv-protection]
StorageClass:      slow3
Status:            Bound
Claim:             default/pvc3
Reclaim Policy:    Delete
Access Modes:      RWO
Capacity:          6Gi
Node Affinity:     
  Required Terms:  
    Term 0:        failure-domain.beta.kubernetes.io/zone in [us-west-2a]
                   failure-domain.beta.kubernetes.io/region in [us-west-2]
Message:           
Source:
    Type:       AWSElasticBlockStore (a Persistent Disk resource in AWS)
    VolumeID:   aws://us-west-2a/vol-0fc1cdae7d10860f6
    FSType:     ext4
    Partition:  0
    ReadOnly:   false
Events:         <none>
```

**Release note**:
```release-note
none
```

/sig storage
/assign @msau42 @jsafrane
2018-08-01 15:52:07 -07:00
Michelle Au
cf107159b4 Detect if GCE PD udev link is wrong and try to correct it 2018-08-01 10:04:06 -07:00
Deep Debroy
217a3d8902 Add DynamicProvisioningScheduling support for EBS
Signed-off-by: Deep Debroy <ddebroy@docker.com>
2018-08-01 09:00:03 -07:00
Di Xu
b3dfe0c652 nodes: improve handling of erroneous host names 2018-08-01 14:57:25 +08:00
Pengfei Ni
6bfd2be2ea Add documentation and unit tests 2018-07-30 13:32:57 +08:00
Pengfei Ni
74813d0d26 Add availability zone support for dynamic provisioning Azure managed disks 2018-07-30 13:31:12 +08:00
Kubernetes Submit Queue
c2cbe86f7d Merge pull request #66648 from feiskyer/azure-sdk-update
Automatic merge from submit-queue (batch tested with PRs 66225, 66648, 65799, 66630, 66619). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update Azure Go SDK to v19.0.0 and get availability zone for VirtualMachineScaleSetVM

**What this PR does / why we need it**:

Continue of #66242. This PR updates Azure Go SDK to v19.0.0 (with compute API 2018-04-01) and gets availability zones for VirtualMachineScaleSetVM.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Azure Go SDK has been upgraded to v19.0.0 and VirtualMachineScaleSetVM now supports availability zones.
```

/sig azure
/assign @brendandburns @khenidak @andyzhangx
2018-07-27 16:42:08 -07:00
Ardalan Kangarlou
ee747b8649 Changed admission controller to allow volume expansion for all volume plugins
There are two motivations for this change:
(1) CSI plugins are soon going to support volume expansion. For such
plugins, admission controller doesn't know whether the plugins are
capabale of supporting volume expansion or not.
(2) Currently, admission controller rejects PVC updates for in-tree plugins
that don't support volume expansion (e.g., NFS, iSCSI). This change allows
external controllers to expand volumes similar to how external provisioners
operate.
2018-07-27 03:06:48 -04:00
jianglingxia
aeef7fdca2 Add UT to RBD volume test of TestGetAccessModes and TestRequiresRemount 2018-07-26 10:00:44 +08:00
Kubernetes Submit Queue
845a55dbbd Merge pull request #63176 from NetApp/bug/59946
Automatic merge from submit-queue (batch tested with PRs 64844, 63176). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix discovery/deletion of iscsi block devices

This PR modifies the iSCSI attach/detatch codepaths in the following
ways:
1) After unmounting a filesystem on an iSCSI block device, always
flush the multipath device mapper entry (if it exists) and delete
all block devices so the kernel forgets about them.
2) When attaching an iSCSI block device, instead of blindly
attempting to scan for the new LUN, first determine if the target
is already logged into, and if not, do the login first. Once every
portal is logged into, the scan is done.
3) Scans are now done for specific devices, instead of the whole
bus. This avoids discovering LUNs that kubelet has no interest in.
4) Additions to the underlying utility interfaces, with new tests
for the new functionality.
5) Some existing code was shifted up or down, to make the new logic
work.
6) A typo in an existing exec call on the attach path was fixed.

Fixes #59946

```release-note
When attaching iSCSI volumes, kubelet now scans only the specific
LUNs being attached, and also deletes them after detaching. This avoids
dangling references to LUNs that no longer exist, which used to be the
cause of random I/O errors/timeouts in kernel logs, slowdowns during
block-device related operations, and very rare cases of data corruption.
```
2018-07-25 16:19:01 -07:00
Kubernetes Submit Queue
158998b8b2 Merge pull request #66612 from humblec/enable-mountoptions
Automatic merge from submit-queue (batch tested with PRs 66373, 66612). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove `auto_unmount` mount option from pv spec annotation.

At present, `auto_unmount` option is recorded at PV annotation of glusterfs PV.
Due to the preference given in MountOptionFromSpec() for annotation mount options
over sc supplied mount options(Ref PR# https://github.com/kubernetes/kubernetes/pull/66576),
the sc supplied mount options are not honoured in glusterfs plugin
eventhough the driver returns `true` for storage class mountoptions
support at probe.

This patch removes `auto_unmount` option from annotation of the pv spec

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-07-25 14:23:03 -07:00
Kubernetes Submit Queue
4378a999e1 Merge pull request #66373 from jianglingxia/jlx-k8s-201807191640
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add UT Test to cephfs

**What this PR does / why we need it**:
Add UT Test to cephfs
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-07-25 13:25:39 -07:00
Humble Chirammal
d4480d4f32 Remove auto_unmount mount option from pv spec annotation to setup() func.
At present, `auto_unmount` option is recorded at PV annotation of glusterfs PV.
Due to the preference given in MountOptionFromSpec() for annotation mount options
over sc supplied mount options(Ref PR# https://github.com/kubernetes/kubernetes/pull/66576),
the sc supplied mount options are not honoured in glusterfs plugin
eventhough the driver returns `true` for storage class mountoptions
support at probe.

This patch removes `auto_unmount` option from annotation of the pv spec.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2018-07-25 22:27:40 +05:30
Pengfei Ni
5bf3b2119b Update compute API to 2018-04-01 2018-07-25 17:39:32 +08:00
Kubernetes Submit Queue
69176615e5 Merge pull request #66202 from tanshanshan/fixformat
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix wrong description 

**What this PR does / why we need it**:
fix wrong description 
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-07-24 21:24:30 -07:00
Ben Swartzlander
6d23d8edbb Avoid deleted iSCSI LUNs in the kernel
This change ensures that iSCSI block devices are deleted after
unmounting, and implements scanning of individual LUNs rather
than scanning the whole iSCSI bus.

In cases where an iSCSI bus is in use by more than one attachment,
detaching used to leave behind phantom block devices, which could
cause I/O errors, long timeouts, or even corruption in the case
when the underlying LUN number was recycled. This change makes
sure to flush references to the block devices after unmounting.

The original iSCSI code scanned the whole target every time a LUN
was attached. On storage controllers that export multiple LUNs on
the same target IQN, this led to a situation where nodes would
see SCSI disks that they weren't supposed to -- possibly dozens or
hundreds of extra SCSI disks. This caused 3 significant problems:

1) The large number of disks wasted resources on the node and
caused a minor drag on performance.
2) The scanning of all the devices caused a huge number of uevents
from the kernel, causing udev to bog down for multiple minutes in
some cases, triggering timeouts and other transient failures.
3) Because Kubernetes was not tracking all the "extra" LUNs that
got discovered, they would not get cleaned up until the last LUN
on a particular target was detached, causing a logout. This led
to significant complications:

In the time window between when a LUN was unintentially scanned,
and when it was removed due to a logout, if it was deleted on the
backend, a phantom reference remained on the node. In the best
case, the phantom LUN would cause I/O errors and timeouts in the
udev system. In the worst case, the backend could reuse the LUN
number for a new volume, and if that new volume were to be
scheduled to a pod with a phantom reference to the old LUN by the
same number, the initiator could get confused and possibly corrupt
data on that volume.

To avoid these problems, the new implementation only scans for
the specific LUN number it expects to see. It's worth noting that
the default behavior of iscsiadm is to automatically scan the
whole bus on login. That behavior can be disabled by setting
node.session.scan = manual
in iscsid.conf, and for the reasons mentioned above, it is
strongly recommended to set that option. This change still works
regardless of the setting in iscsid.conf, and while automatic
scanning will cause some problems, this change doesn't make the
problems any worse, and can make things better in some cases.
2018-07-24 23:58:19 -04:00
jianglingxia
6a75ec264c Add UT Test to cephfs 2018-07-25 10:40:16 +08:00
Kubernetes Submit Queue
9e0c4a6095 Merge pull request #66488 from linyouchong/pr-0723-csi-labelmanager
Automatic merge from submit-queue (batch tested with PRs 66464, 66488). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use glog instead of fmt

**What this PR does / why we need it**:
Use glog instead of fmt

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```

/sig storage
2018-07-24 19:03:04 -07:00
Kubernetes Submit Queue
35c3764bbb Merge pull request #66464 from wongma7/round-overflow
Automatic merge from submit-queue (batch tested with PRs 66464, 66488). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Avoid overflowing int64 in RoundUpSize and return error if overflow int

**What this PR does / why we need it**:
There are many places in plugins (some I may have missed) that we naively convert a resource.Quantity.Value() which is an int64, to an int, which may be only 32 bits long.

Background, optional to read :): Kubernetes canonicalizes resource.Quantities, and from what I have seen testing creating PVCs, decimalSI is the default. If a quantity is in `decimalSI` format and its value in bytes would overflow an int64, e.g. `10E`, nothing happens. If it is in binarySI and its value in bytes would overflow an int64, e.g. `10Ei`, it is set down to 2^63-1 and there's no overflow of the field value. But there may be overflow later in the code which is what this PR is addressing.

* Change `RoundUpSize` implementation to avoid overflowing `int64`
* Add `RoundUp*Int` functions for use when an `int` is expected instead of an `int64`, because `int` may be 32bits and naively doing `int($INT64_VALUE)` can lead to silent overflow. These functions return an error if overflow has occurred.
* Rename `*GB` variables to `*GiB` where appropriate for maximum clarity
* Use `RoundUpToGiB` instead of `RoundUpSize` where possible

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**: please review carefully as we don't have e2e tests for most plugins!

**Release note**:

```release-note
NONE
```
edit: remove 'we do not need to worry about...'. yes we do, i worded that badly :))
2018-07-24 19:03:01 -07:00
linyouchong
f2e92776bc Use glog instead of fmt 2018-07-24 09:46:56 +08:00
Matthew Wong
093e231289 Avoid overflowing int64 in RoundUpSize and return error if overflow int 2018-07-23 13:48:45 -04:00
Kubernetes Submit Queue
53ee0c8652 Merge pull request #65660 from mtaufen/incremental-refactor-kubelet-node-status
Automatic merge from submit-queue (batch tested with PRs 66152, 66406, 66218, 66278, 65660). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Refactor kubelet node status setters, add test coverage

This internal refactor moves the node status setters to a new package, explicitly injects dependencies to facilitate unit testing, and adds individual unit tests for the setters.

I gave each setter a distinct commit to facilitate review.

Non-goals:
- I intentionally excluded the class of setters that return a "modified" boolean, as I want to think more carefully about how to cleanly handle the behavior, and this PR is already rather large.
- I would like to clean up the status update control loops as well, but that belongs in a separate PR.

```release-note
NONE
```
2018-07-20 12:12:24 -07:00
Seungcheol Ko
43f805b7bd Fix a typo in csiPlugin comment 2018-07-19 21:01:09 +09:00
linyouchong
51e81d2585 csiAttacher: check deviceMountPath before hasStageUnstageCapability 2018-07-19 11:26:45 +08:00
Kubernetes Submit Queue
78082a6a28 Merge pull request #66174 from ddebroy/scaleio1
Automatic merge from submit-queue (batch tested with PRs 64690, 66174). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Check presence of /dev/disk/by-id before reading it in ScaleIO

**What this PR does / why we need it**:
In certain OS environments, like RHEL 7.4 using Xen PV driver `xen_blkfront` for boot and data disks, the path `/dev/disk/by-id` does not exist in the OS post boot. When trying to dynamically provision PVs from a ScaleIO storageclass in such an environment, ScaleIO fails when trying to create the PV with:
```
E0711 08:02:50.441964   25246 sio_client.go:342] scaleio: failed to ReadDir /dev/disk/by-id: open /dev/disk/by-id: no such file or directory
E0711 08:02:50.441989   25246 sio_volume.go:123] scaleio: setup of volume k8svol-282bd4fa84d:  open /dev/disk/by-id: no such file or directory
```
This PR avoids the above failure by first checking for the presence of `/dev/disk/by-id` before attempting to read from it as done in other drivers like gce_pd. 


**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
/sig storage
2018-07-18 03:32:59 -07:00
Deep Debroy
7392ed4ba0 Check presence of sioDiskIDPath before reading it
Signed-off-by: Deep Debroy <ddebroy@docker.com>
2018-07-17 09:55:46 -07:00
Kubernetes Submit Queue
77205b90a0 Merge pull request #66172 from krunaljain/bugfix/revert_pd_size_rounding_to_GB
Automatic merge from submit-queue (batch tested with PRs 66172, 66254). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Reverting commit #56600 as GCE PD is allocated in chunks of GiB inste…

**What this PR does / why we need it:**
This PR reverts the changes made in commit https://github.com/kubernetes/kubernetes/pull/56600 which considered GCE PDs are allocated in chunks of GBs. The following set of operations demonstrate the allocation is in GiBs. 

Manually create a PD in GB, and manually attach it to a node:
```
$ gcloud compute disks create msau-test --zone=us-central1-b --size=1GB
```
Run lsblk on it, and it shows the bytes of the disk are 1GiB:
```
$ lsblk -b
sdc       8:32   0  1073741824  0 disk 
```

**Which issue(s) this PR fixes**:
[65285](https://github.com/kubernetes/kubernetes/issues/65285)
**Special notes for your reviewer**:
```release-note
none
```
2018-07-17 02:13:03 -07:00