Commit Graph

357 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
f2db955b9d Merge pull request #64363 from idealhack/sub-benchmarks/scheduler/schedulercache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

scheduler: update tests to use sub-benchmarks (pkg/scheduler/cache)

**What this PR does / why we need it**:

Go 1.7 added the subtest feature which can make table-driven tests much easier to run and debug. Some tests are not using this feature.

Further reading: [Using Subtests and Sub-benchmarks](https://blog.golang.org/subtests)

/kind cleanup

**Release note**:

```release-note
NONE
```
2018-07-01 19:04:19 -07:00
Yang Li
d7e12ce453 scheduler: update tests to use sub-benchmarks (pkg/scheduler/cache) 2018-07-01 00:51:42 +08:00
Kubernetes Submit Queue
ea3451f83e Merge pull request #65188 from aveshagarwal/master-rhbz-1555057
Automatic merge from submit-queue (batch tested with PRs 65188, 65541, 65534). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Increase glog level of some scheduling errors.

In our production environments, we are noticing that for every scheduling error, we are logging 3 errors at following lines:

1. https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/scheduler.go#L194

2. https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/factory/factory.go#L1416

3. https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/factory/factory.go#L1323

This PR increases log levels of the last 2 errors to V(3).Infof. We can discuss if it would be helpful to increase the log level of the first error too.

@kubernetes/sig-scheduling-pr-reviews 
@bsalamat @k82cn @liggitt @sjenning 

```release-note
None.
```
2018-06-29 21:42:07 -07:00
Dr. Stefan Schimanski
f8de7cea40 Update generated files 2018-06-29 20:36:17 +02:00
Avesh Agarwal
c0cffb8a34 Increase glog level of some scheduling errors. 2018-06-28 23:34:29 -04:00
Kubernetes Submit Queue
a13fe4d15d Merge pull request #65424 from liggitt/scheduler-config
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix scheduler config decoding

Fixes #65413

Implements a custom unmarshaler for a single scheduler config type which did not correctly specify JSON tags until https://github.com/kubernetes/kubernetes/issues/65414 is resolved

Adds missing compatibility tests for scheduler extenders back to 1.7

```release-note
Fixes incompatibility with custom scheduler extender configurations specifying `bindVerb`
```
2018-06-25 00:21:35 -07:00
Jordan Liggitt
fcaaf59359 Fix scheduler config decoding 2018-06-24 23:28:56 -04:00
Kubernetes Submit Queue
f0311d8232 Merge pull request #65396 from bsalamat/sched_no_sort
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve scheduler's performance by eliminating sorting of nodes by their score

**What this PR does / why we need it**:
Profiling scheduler, I noticed that scheduler spends a significant amount of time in sorting the nodes after we score them to find nodes with the highest score. Finding nodes with the highest score does not need sorting the array. This PR replaces the sort with a linear scan.

Eliminating the sort results in over 10% improvement in throughput of the scheduler.

Before (3 runs for 5000 nodes, scheduling 1000 pods in a cluster running 2000 pods):
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  20682552 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  20464729 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  21188906 ns/op

After:
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  18485866 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  18457749 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  18418200 ns/op

**Release note**:

```release-note
Improve scheduler's performance by eliminating sorting of nodes by their score.
```
2018-06-23 20:12:01 -07:00
Bobby (Babak) Salamat
ffc8cc2f50 Improve scheduler's performance by eliminating sorting when finding the host with the highest score 2018-06-23 11:24:43 -07:00
Kubernetes Submit Queue
582b88c879 Merge pull request #64995 from bsalamat/preempt_opt
Automatic merge from submit-queue (batch tested with PRs 65388, 64995). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add more conditions to the list of predicate failures that won't be resolved by preemption

**What this PR does / why we need it**:
Adds more conditions to the list of predicate failures that won't be resolved by preemption. This change can potentially improve performance of preemption by avoiding the nodes that won't be able to schedule the pending pod no matter how many other pods are removed from them.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Add more conditions to the list of predicate failures that won't be resolved by preemption.
```

/sig scheduling
2018-06-23 05:52:07 -07:00
Bobby (Babak) Salamat
8cdf83ed1e Add tests to cover newly added unresolvable failures 2018-06-22 17:06:19 -07:00
Bobby (Babak) Salamat
fab26e470c Add more unresolvable conditions to optimize preemption logic 2018-06-22 17:04:55 -07:00
Jeff Grafton
23ceebac22 Run hack/update-bazel.sh 2018-06-22 16:22:57 -07:00
Jeff Grafton
a725660640 Update to gazelle 0.12.0 and run hack/update-bazel.sh 2018-06-22 16:22:18 -07:00
Kubernetes Submit Queue
b45ba959c0 Merge pull request #64693 from xiechengsheng/fix-typos
Automatic merge from submit-queue (batch tested with PRs 65024, 65287, 65345, 64693, 64941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix some typos in code comments.

Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>



**What this PR does / why we need it**:
Fix some typos in code comments.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-06-22 06:10:21 -07:00
xiechengsheng
66e0b53c3c fix some typos
Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>
2018-06-22 09:26:14 +08:00
Kubernetes Submit Queue
23b4690d00 Merge pull request #65306 from shyamjvs/fine-grained-scheduler-metric
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Split scheduler latency metric to fine-grained steps

This splits the summary metric we recently added into finer steps. It should be very useful for performance experiments.

/cc @wojtek-t 
fyi - @bsalamat @misterikkit 

Strictly speaking this is a breaking change, but since this metric was added only ~week ago I think it should fine (we should port this change to 1.11).

```release-note
Split 'scheduling_latency_seconds' metric into finer steps (predicate, priority, premption)
```
2018-06-21 09:11:58 -07:00
Shyam Jeedigunta
b9ae20c99e Split scheduler latency metric to fine-grained steps 2018-06-21 14:19:39 +02:00
Kubernetes Submit Queue
d1f5cb2348 Merge pull request #65050 from sttts/sttts-deepcopy-update
Automatic merge from submit-queue (batch tested with PRs 64895, 64938, 63700, 65050, 64957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump gengo to include uniform pointer deepcopy

This bumps k8s.io/gengo with uniform pointer support in deepcopy-gen.

Fixes https://github.com/kubernetes/code-generator/issues/45.
2018-06-21 04:15:16 -07:00
Kubernetes Submit Queue
15902d9113 Merge pull request #63662 from xchapter7x/pkg-scheduler-algorithmprovider
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler/algorithmprovider)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:26 -07:00
Kubernetes Submit Queue
58574021a7 Merge pull request #63661 from xchapter7x/pkg-scheduler
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:22 -07:00
Kubernetes Submit Queue
1712073165 Merge pull request #63660 from xchapter7x/pkg-scheduler-algorithm-predicates
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler/algorithm/predicates)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:18 -07:00
Jonathan Basseri
56b941f3df scheduler: fix equiv. cache logging.
Change the log messages to accurately reflect the cache behavior.
2018-06-20 10:52:33 -07:00
Jonathan Basseri
c24842d806 Add a package docstring to core/equivalence. 2018-06-20 10:52:33 -07:00
Jonathan Basseri
b571065bc4 Clean up names in equivalence package.
Remove stutter from names and provide more idiomatic patterns.

This makes call sites that use equivalence cache easier to read.
2018-06-20 10:52:33 -07:00
Jonathan Basseri
31c746d960 Move equivalence cache into new package.
This moves the equivalence cache implementation out of the 'core'
package and into k8s.io/kubernetes/pkg/scheduler/core/equivalence.

Separating the equiv. cache from the genericScheduler implementation
make their interaction points easier to follow, and prevents us from
accidentally accessing unexported fields.
2018-06-20 10:52:32 -07:00
Kubernetes Submit Queue
bb6270bd92 Merge pull request #64692 from adohe/scheduler_cache
Automatic merge from submit-queue (batch tested with PRs 64882, 64692, 64389, 60626, 64840). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

update pod state of scheduler cache when UpdatePod

update pod state map in scheduler cache when call UpdatePod. @k82cn @bsalamat 

```release-note
keep pod state consistent when scheduler cache UpdatePod
```
2018-06-20 10:03:23 -07:00
Kubernetes Submit Queue
dd040d6010 Merge pull request #63942 from misterikkit/ecache-cleanup
Automatic merge from submit-queue (batch tested with PRs 64142, 64426, 62910, 63942, 64548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

scheduler: further cleanup of equivalence cache

**What this PR does / why we need it**:
This improves comments and simplifies some names/logic in equivalence_cache.go, as well as changing the order of some items in the file.


**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
/kind cleanup
2018-06-20 00:05:18 -07:00
Kubernetes Submit Queue
53d03c58cd Merge pull request #64179 from wgliang/master.scheduler-unused-para
Automatic merge from submit-queue (batch tested with PRs 64252, 64307, 64163, 64378, 64179). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove unused parameter (pod) in `pkg/scheduler/core/generic_scheduler`

**What this PR does / why we need it**:

Remove unused parameter (pod) in `pkg/scheduler/core/generic_scheduler`

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-06-19 21:45:21 -07:00
Kubernetes Submit Queue
e486e8f1f9 Merge pull request #63659 from xchapter7x/pkg-scheduler-algorithm-priorities-util
Automatic merge from submit-queue (batch tested with PRs 65230, 57355, 59174, 63698, 63659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg-scheduler-algorithm-priorities-util)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-19 20:19:25 -07:00
Krzysztof Siedlecki
e32910a544 Readding summary metrics 2018-06-14 15:05:12 +02:00
Krzysztof Siedlecki
0547bbf744 Revert "Fixing scheduling latency metrics"
This reverts commit 0e833bfc83.
2018-06-14 14:50:12 +02:00
Dr. Stefan Schimanski
1208437f84 Update generated files 2018-06-13 12:35:13 +02:00
John Calabrese
f415558c30 use subtest for table units
implement PR feedback

replace errorf + return with fatalf

  https://github.com/kubernetes/kubernetes/pull/63662#discussion_r192540370
  https://github.com/kubernetes/kubernetes/pull/63662#discussion_r192540457
2018-06-08 11:44:24 -04:00
John Calabrese
7735dd6843 use subtest for table units
apply consistent format to name strings

  - #63661 (comment)
  - #63661 (comment)

inline subtest logic

  https://github.com/kubernetes/kubernetes/pull/63661/files#r192540031

remove duplicate messaging in subtest errors
2018-06-08 10:52:28 -04:00
AdoHe
6116c64f64 keep pod state consistent when UpdatePod 2018-06-06 21:13:12 +08:00
xiechengsheng
cd65afdf05 fix some typos
Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>
2018-06-06 13:11:26 +08:00
John Calabrese
8bbf182695 use subtest for table units
apply subtest to predicates
2018-06-05 09:07:57 -04:00
Silvery Fu
8792f99bf8 Fix the handling of untagged images 2018-06-04 13:57:09 -07:00
Silvery Fu
73970a5027 Update generated bazel 2018-06-04 13:56:51 -07:00
Łukasz Osipiuk
98041d0925 Allow parametrization of RequestedToCapacityRatio priority function via policy config 2018-06-04 12:28:54 +02:00
Łukasz Osipiuk
4684d5aac6 Register RequestedToCapacityRatioPriority priority function 2018-06-04 11:30:57 +02:00
Hemant Kumar
a0a9ccfa87 Implement scheduler changes for volume limits 2018-06-01 19:17:31 -04:00
Guoliang Wang
761cf41427 Move pkg/scheduler/schedulercache -> pkg/scheduler/cache 2018-05-31 22:55:34 +08:00
Kubernetes Submit Queue
b5d21a92ad Merge pull request #64339 from liztio/pronouns
Automatic merge from submit-queue (batch tested with PRs 61803, 64305, 64170, 64361, 64339). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove some unnecessarily gendered pronouns in comments

**What this PR does / why we need it**:

A bunch of comments are unnecessarily gendered. I've changed them to gender-neutral they/theirs.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-30 17:34:25 -07:00
Krzysztof Siedlecki
0e833bfc83 Fixing scheduling latency metrics 2018-05-30 11:20:12 +02:00
Jonathan Basseri
ba08b05e28 Rename equiv. class invalidation functions.
Change the invalidation functions to have cleaner and more consistent
names.
2018-05-29 11:23:13 -07:00
Jonathan Basseri
5d13798e5c Change the return of EquivalenceClass.lookupResult.
This makes the lookup behave like a normal map lookup, so it is easier
for readers to follow the logic. It also inverts the "invalid" bool to
an "ok" bool because `!invalid` is a double negative.
2018-05-29 11:22:02 -07:00
Jonathan Basseri
9b06870620 Clean up names and comments in equivalence cache. 2018-05-29 11:22:02 -07:00
John Calabrese
d1abc5c824 use subtest for table units
employ consitent table var naming

  - https://github.com/kubernetes/kubernetes/pull/63659#discussion_r187736533
2018-05-28 09:33:42 -04:00