Commit Graph

2408 Commits

Author SHA1 Message Date
andyzhangx
8a7198b036 use new account generation method for blob disk
fix comments

change azureDiskSharedAccountNamePrefix var

rename sharedDiskAccountNamePrefix

use default vhd container name as "vhds"

use one commaon func: SearchStorageAccount

fix comments
2018-02-12 13:09:29 +00:00
Pengfei Ni
8c382649ba Add unit tests for mapLoadBalancerNameToVMSet 2018-02-12 16:07:11 +08:00
Pengfei Ni
d3e0280c3a Map correct vmset name for internal load balancers 2018-02-12 15:56:57 +08:00
Pengfei Ni
1976983e79 Fix unit tests for vmss 2018-02-12 14:07:05 +08:00
Pengfei Ni
890b7efb1d Add azure disk support of vmss 2018-02-12 14:07:05 +08:00
Pengfei Ni
5042cea857 Use new clients for vmss cache 2018-02-12 14:07:05 +08:00
Pengfei Ni
3b7cc3dd5e Update Azure GO SDK to v12.4.0-beta 2018-02-12 14:07:05 +08:00
Pengfei Ni
5d16067ddd Update vmss fake clients 2018-02-12 14:07:05 +08:00
Pengfei Ni
829e0946e5 Update vmss client to new version 2018-02-12 14:07:05 +08:00
Pengfei Ni
11e5399dde Abstract disk operation interfaces in VMSet 2018-02-12 14:07:05 +08:00
Kubernetes Submit Queue
69324f90e6 Merge pull request #59652 from feiskyer/vmss-cache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add generic cache for Azure VMSS

**What this PR does / why we need it**:

This PR adds a generic cache for VMSS and removes old list-based cache.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

Continue of ##58770.

**Special notes for your reviewer**:

Depends on #59520.

**Release note**:

```release-note
Add generic cache for Azure VMSS
```
2018-02-11 21:22:45 -08:00
Pengfei Ni
bde2989c7a Add unit tests for extractVmssVMName 2018-02-12 11:21:08 +08:00
Kubernetes Submit Queue
cfa6774540 Merge pull request #56557 from andyzhangx/azurefile-createaccount
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix the create azure file pvc failure if there is no storage account in current resource group

**What this PR does / why we need it**:
When create an azure file PVC, there will be error if there is no storage account in current resource group.
With this PR, a storage account will be created if there is no storage account in current resource group.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56556

**Special notes for your reviewer**:
1. rephrase the code logic of `CreateFileShare` func.
```
if accountName is empty, then 
    find a storage account that matches accountType
    if no storage account found, then
        create a new account
else
    we only use user specified storage account

create a file share according to found storage account
```
2. Use func `getStorageAccountName` to get a unique storage account name by UUID, a storage account for azure file would be like `f0b2b0bd40c010112e897fa`. And in next PR, I will use this function to create storage account for azure disk, the storage account for azure disk would be like `d8f3ad8ad92000f1e1e88bd`.

**Release note**:

```
fix the create azure file pvc failure if there is no storage account in current resource group
```
/sig azure
/assign @rootfs
2018-02-11 05:03:32 -08:00
andyzhangx
aa21bef677 create storage account if necessary when create azure file pvc
use new storage account name generation method

use uuid to generate account name

change azure file account prefix

use uniqueID to generate a storage account name

fix comments

fix comments

fix comments

fix a storage account matching bug

only use UUID in getStorageAccountName func

use shorter storage account prefix for azure file

fix comments

fix comments

fix comments

fix rebase build error

rewrite CreateFileShare code logic

fix gofmt issue

fix test error

fix comments

fix a location matching bug
2018-02-11 06:29:05 +00:00
Kubernetes Submit Queue
317853c90c Merge pull request #59464 from dixudx/fix_all_typos
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix all the typos across the project

**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.

This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy 
/assign @fejta 

**Release note**:

```release-note
None
```
2018-02-10 22:12:45 -08:00
Davanum Srinivas
7ef4f207c2 New github id - FengyunPan -> FengyunPan2
PanFengyun <pan_feng_yun@163.com>'s previous github id was @FengyunPan

Due to some problem with github, he lost access to @FengyunPan and
is not using @FengyunPan2. So let's switch over to the new id. Github
has promised to release the previous id back in 6 months, so we may
have to switch it back later.
2018-02-10 22:28:51 -05:00
Pengfei Ni
f32bd6a081 Adjust unit tests for vmss 2018-02-11 11:09:22 +08:00
Pengfei Ni
4d5e7b7cfb Use generic cache for vmss 2018-02-11 11:09:22 +08:00
Kubernetes Submit Queue
f0e573d6d5 Merge pull request #59520 from feiskyer/new-cache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add generic cache for Azure VM/LB/NSG/RouteTable

**What this PR does / why we need it**:

Part of #58770. This PR adds a generic cache of Azure VM/LB/NSG/RouteTable for reducing ARM calls.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #58770

**Special notes for your reviewer**:

**Release note**:

```release-note
Add generic cache for Azure VM/LB/NSG/RouteTable
```
2018-02-10 19:06:02 -08:00
Di Xu
48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
Jesse Haka
6665fa7144 taint also node controller
fix function

fix gofmt

fix function return value

fix tests

skip notimplemented error

remove factory unused

in openstack we should try to find instanceid from all states instead of ACTIVE, all other cloudproviders do this already

fix tests and lint

fix gofmt

fix nodelifecycletest

fix lint errors
2018-02-10 15:41:24 +02:00
Kubernetes Submit Queue
a35acccaf8 Merge pull request #59447 from verult/pd-alpha
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixes the regression of GCEPD not provisioning correctly on GKE alpha clusters.

Fixes the regression by better distinguishing between single-zone and multi-zone PDs.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59047

**Special notes for your reviewer**: All PD and dynamic provisioning e2e tests pass, manually verified provision, delete, attach, and detach of both single-zone and multi-zone PDs. Will create e2e tests for multizone PDs in a separate PR.

**Release note**:


/sig storage
/assign @saad-ali
2018-02-09 11:55:10 -08:00
Pengfei Ni
7634eacb4f Add error handling and new tests 2018-02-09 20:38:29 +08:00
Kubernetes Submit Queue
5d8f6ed452 Merge pull request #59619 from jianliao82/patch-1
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update azure_loadbalancer.md to fix typo

fix typo 
incase -> in case
selction -> selection



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
fix typo 
incase -> in case
selction -> selection
```
2018-02-08 21:49:54 -08:00
Kubernetes Submit Queue
0b6a0a6977 Merge pull request #59490 from agau4779/gce-unittest
Automatic merge from submit-queue (batch tested with PRs 58437, 59490, 55684). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[GCE] Unit test ExternalLoadBalancer

**What this PR does / why we need it**:

- Unit tests for `pkg/cloudprovider/providers/gce/gce_loadbalancer_external.go`
- Tests creating, updating, and deleting an external LoadBalancer

**Future Improvements**
In order to further test `gce_loadbalancer_external.go`, we should add tests for the following cases:

- Network Tiers - when the current/desired network tier doesn't match, existing resources with the wrong tier should be torn down
- Expect an error when the TargetPool does not exist
- Expect an error when the LoadBalancer Firewall does not exist
- Case when TargetPool needs to be recreated 
- Case when IP needs to be released (calls gce.DeleteRegionAddress)
```release-note
NONE
```
2018-02-08 21:34:37 -08:00
liaoj
953cbb11a1 Update azure_loadbalancer.md
fix typo 
incase -> in case
selction -> selection
2018-02-09 12:52:59 +08:00
Kubernetes Submit Queue
30f54b4028 Merge pull request #56974 from gnufied/speed-up-attach-detach
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Make AWS attach/detach operations faster

Most attach/detach operations on AWS finish within 1-4seconds.
By using a shorter time interval and higher exponetial
factor  we can shorten time taken for attach and detach to complete. 

After this change retry interval looks like:

```
[1, 1.8, 3.24, 5.832000000000001, 10.4976]
```

Before it was:
```
[10, 12.0, 14.399999999999999, 17.279999999999998]
```


/sig aws

```release-note
AWS: Make attach/detach operations faster. from 10-12s to 2-6s
```
2018-02-08 20:24:10 -08:00
Pengfei Ni
daec2bd745 Add cache for route tables 2018-02-09 09:09:25 +08:00
Pengfei Ni
21c8a63689 Add cache for network security groups 2018-02-09 09:09:25 +08:00
Pengfei Ni
d22b6d9ebe Add cache for load balancer 2018-02-09 09:09:25 +08:00
Pengfei Ni
2badf1ff55 Add cache for virtual machines 2018-02-09 09:09:25 +08:00
Pengfei Ni
035c8da63d New unit tests for timedCache 2018-02-09 09:09:25 +08:00
Pengfei Ni
259dbf8da7 Make azure cache general for all objects 2018-02-09 09:09:25 +08:00
Kubernetes Submit Queue
2172a2a806 Merge pull request #59569 from tsmetana/fix-aws-detach
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

AWS: Do not ignore errors from EC2::DescribeVolume in DetachDisk

The DetachDisk method of AWS cloudprovider indirectly calls
EC2::DescribeVolume AWS API function to check if the volume
being detached is really attached to the specified node.

The AWS API call may fail and return error which is logged however
the DetachDisk then finishes successfully. This may cause the AWS
volumes to remain attached to the instances forever because the
attach/detach controller will mark the volume as attached. The PV
controller will never be able to delete those disks and they need
to be detached manually.

This patch ensures the error from DescribeVolume is propagated to
attach/detach controller and the detach operation is re-tried.

cc: @gnufied, @jsafrane 

```release-note
NONE
```
2018-02-08 17:02:18 -08:00
Ashley Gau
5e38d785a2 make context the first arg in AddInstanceHook/RemoveInstanceHook 2018-02-08 11:04:39 -08:00
Ashley Gau
4b398a6d8d generate mocked methods with context as the first arg, because golint 2018-02-08 11:02:48 -08:00
Cheng Xing
02352460f6 Fixes the regression of GCEPD not provisioning correctly on alpha clusters. 2018-02-08 10:46:06 -08:00
Kubernetes Submit Queue
c3a92d0b9b Merge pull request #59497 from dougm/vclib-tests
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

vclib: enable VM disk attach test

**What this PR does / why we need it**:

Follow up to PR #58534 , where this test was disabled due to a limitation in
govmomi/simulator.  The test passes as expected with godeps update of govmomi.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

This PR is 1-line update to the vSphere Cloud Provider tests and godep update of the vendor'd vmware/govmomi repo.

**Release note**:

```release-note
NONE
```
2018-02-08 10:40:27 -08:00
Kubernetes Submit Queue
c0a337d4cc Merge pull request #59519 from vmware/vm_uuid_provider_id
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Report InstanceID for vSphere Cloud Provider as UUID obtained from product_serial file 

**What this PR does / why we need it**:
vSphere Cloud Provider is not able to find the nodes for VMs created on vSphere v1.6.5. Kubelet fetches SystemUUID from file ```/sys/class/dmi/id/product_uuid```. vSphere Cloud Provider uses this uuid as VM identifier to get node information from vCenter. vCenter v1.6.5 doesn't recognize this uuids, as a result, nodes are not found. 

UUID present in file ```/sys/class/dmi/id/product_serial``` is recognized by vCenter. Yet,  Kubelet doesn't report this. Therefore, in this PR InstanceID is reported as UUID which is fetched from file 
```/sys/class/dmi/id/product_serial```.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/58927

**Special notes for your reviewer**:
Internally review here: https://github.com/vmware/kubernetes/pull/452

Tested:
Launched K8s cluster using kubeadm (Used Ubuntu VM compatible with vSphere version 6.5.)
_**Note: Installed Ubuntu from ISO**_
Observed following:
```
Master
> cat /sys/class/dmi/id/product_uuid
743F0E42-84EA-A2F9-7736-6106BB5DBF6B

> cat /sys/class/dmi/id/product_serial
VMware-42 0e 3f 74 ea 84 f9 a2-77 36 61 06 bb 5d bf 6b

Node
> cat /sys/class/dmi/id/product_uuid
956E0E42-CC9D-3D89-9757-F27CEB539B76

> cat /sys/class/dmi/id/product_serial
VMware-42 0e 6e 95 9d cc 89 3d-97 57 f2 7c eb 53 9b 76
```
With this fix controller manager was able to find the nodes.
**controller manager logs**
```
{"log":"I0205 22:43:00.106416       1 nodemanager.go:183] Found node ubuntu-node as vm=VirtualMachine:vm-95 in vc=10.161.120.115 and datacenter=vcqaDC\n","stream":"stderr","time":"2018-02-05T22:43:00.421010375Z"}
```


**Release note**:

```release-note
vSphere Cloud Provider supports VMs provisioned on vSphere v1.6.5
```
2018-02-08 09:43:08 -08:00
Davanum Srinivas
18f1486e97 Skip TestRoutes when there are no vm(s)
TestRoutes assumes that there is at least one vm in the OpenStack it
is connecting to. So let's limit this test to run properly only when
we are running in a VM or one was created already outside of the
test harness
2018-02-08 08:54:04 -05:00
Tomas Smetana
c879e531f9 AWS: Do not ignore errors from EC2::DescribeVolumee in DetachDisk
The DetachDisk method of AWS cloudprovider indirectly calls
EC2::DescribeVolume AWS API function to check if the volume
being detached is really attached to the specified node.

The AWS API call may fail and return error which is logged however
the DetachDisk then finishes successfully. This may cause the AWS
volumes to remain attached to the instances forever because the
attach/detach controller will mark the volume as attached. The PV
controller will never be able to delete those disks and they need
to be detached manually.

This patch ensures on error from DescribeVolume is propagated to
attach/detach controller and the detach operation is re-tried.
2018-02-08 13:51:53 +01:00
Jesse Haka
3cf5b172fa add node shutdown taint
shutdowned -> stopped

use shutdown everywhere

use patch in taints api call

use notimplemented in clouds use AddOrUpdateTaintOnNode

correct log text

add fake cloud

try to fix bazel

add shutdown tests

add context
2018-02-08 12:56:06 +02:00
Pengfei Ni
20de29ae6f Set instanceID to azure resource ID format while useInstanceMetadata is enabled 2018-02-08 15:12:36 +08:00
Pengfei Ni
4e4fde93a7 Add useInstanceMetadata param back in Azure cloud provider
This reverts commit bb1e797b28.
2018-02-08 12:40:35 +08:00
Abrar Shivani
918e47c438 Change provider ID to uuid 2018-02-07 17:08:47 -08:00
Ashley Gau
b7fe9fe02f run update-bazel.sh, lint on mock.go 2018-02-07 15:27:35 -08:00
Ashley Gau
5c8e7e3707 remove newline before err checks. address pr comments 2018-02-07 14:38:31 -08:00
Ashley Gau
41070a2f58 use getInstanceByName to check for node presence, instead of DeleteInstance) 2018-02-07 14:36:08 -08:00
Ashley Gau
378b177f02 test updateExternalLoadBalancer removes nodes 2018-02-07 12:42:57 -08:00
Ashley Gau
60ed92b16d check firewall creation + deletion for healthcheck firewall 2018-02-07 11:38:24 -08:00