mirror of
https://github.com/optim-enterprises-bv/kubernetes.git
synced 2025-11-04 12:18:16 +00:00
DAG Workflow with Workflow resource
This commit is contained in:
Salvatore Dario Minonne
255
docs/proposals/workflow.md
Normal file
255
docs/proposals/workflow.md
Normal file
@@ -0,0 +1,255 @@
|
|||||||
|
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||||
|
|
||||||
|
<!-- BEGIN STRIP_FOR_RELEASE -->
|
||||||
|
|
||||||
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||||
|
width="25" height="25">
|
||||||
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||||
|
width="25" height="25">
|
||||||
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||||
|
width="25" height="25">
|
||||||
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||||
|
width="25" height="25">
|
||||||
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||||
|
width="25" height="25">
|
||||||
|
|
||||||
|
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
|
||||||
|
|
||||||
|
If you are using a released version of Kubernetes, you should
|
||||||
|
refer to the docs that go with that version.
|
||||||
|
|
||||||
|
<strong>
|
||||||
|
The latest release of this document can be found
|
||||||
|
[here](http://releases.k8s.io/release-1.1/docs/proposals/workflow.md).
|
||||||
|
|
||||||
|
Documentation for other releases can be found at
|
||||||
|
[releases.k8s.io](http://releases.k8s.io).
|
||||||
|
</strong>
|
||||||
|
--
|
||||||
|
|
||||||
|
<!-- END STRIP_FOR_RELEASE -->
|
||||||
|
|
||||||
|
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||||
|
|
||||||
|
|
||||||
|
## Abstract
|
||||||
|
|
||||||
|
A proposal to introduce [workflow](https://en.wikipedia.org/wiki/Workflow_management_system)
|
||||||
|
functionality in kubernetes.
|
||||||
|
Workflows (aka [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph) workflows
|
||||||
|
since _tasks_ are organized in a Directed Acyclic Graph) are ubiquitous
|
||||||
|
in modern [job schedulers](https://en.wikipedia.org/wiki/Job_scheduler), see for example:
|
||||||
|
|
||||||
|
* [luigi](https://github.com/spotify/luigi)
|
||||||
|
* [ozie](http://oozie.apache.org/)
|
||||||
|
* [azkaban](https://azkaban.github.io/)
|
||||||
|
|
||||||
|
Most of the [job schedulers](https://en.wikipedia.org/wiki/List_of_job_scheduler_software) offer
|
||||||
|
workflow functionality to some extent.
|
||||||
|
|
||||||
|
|
||||||
|
## Use Cases
|
||||||
|
|
||||||
|
* As a user I want to be able to define a workflow.
|
||||||
|
* As a user I want to compose workflows.
|
||||||
|
* As a user I want to delete a workflow (eventually cascading to running _tasks_).
|
||||||
|
* As a user I want to debug a workflow (ability to track failure).
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Initializers
|
||||||
|
|
||||||
|
In order to implement `Workflow`, one needs to introduce the concept of _dependency_ between resources.
|
||||||
|
Dependecies are _edges_ of the graph.
|
||||||
|
_Dependency_ is introduced by the [initializers proposal #17305](https://github.com/kubernetes/kubernetes/pull/17305), and we would like to use this as the foundation for workflows.
|
||||||
|
An _initializer_ is a dynamically registered object which implements a custom policy.
|
||||||
|
The policy could be based on some dependencies. The policy is applied before the resource is
|
||||||
|
created (even API validated).
|
||||||
|
By modifying the policy one may defer creation of the resource until prerequisites are satisfied.
|
||||||
|
Even if not completed [#17305](https://github.com/kubernetes/kubernetes/pull/17305) already introduces a
|
||||||
|
_dependency_ concept ([see this comment](https://github.com/kubernetes/kubernetes/pull/17305#discussion_r45007826))
|
||||||
|
which could be reused to implement `Workflow`.
|
||||||
|
|
||||||
|
```go
|
||||||
|
type ObjectDependencies struct {
|
||||||
|
Initializers map[string]string `json:"initializers,omitempty"`
|
||||||
|
Finalizers map[string]string `json:"finalizers,omitempty"`
|
||||||
|
ExistenceDependencies []ObjectReference `json:"existenceDependencies,omitempty"`
|
||||||
|
ControllerRef *ObjectReference `json:"controllerRef,omitempty"`
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Recurring `Workflow` and `ScheduledJob`
|
||||||
|
|
||||||
|
One of the major functionalities missing here is the ability to set a recurring `Workflow` (cron-like),
|
||||||
|
similar to the `ScheduledJob` [#11980](https://github.com/kubernetes/kubernetes/pull/11980) for `Job`.
|
||||||
|
If the scheduled job is able
|
||||||
|
to support [various resources](https://github.com/kubernetes/kubernetes/pull/11980#discussion_r46729699)
|
||||||
|
`Workflow` will benefit from the _schedule_ functionality of `Job`.
|
||||||
|
|
||||||
|
|
||||||
|
### Graceful and immediate termination
|
||||||
|
|
||||||
|
`Workflow` should support _graceful and immediate termination_ [#1535](https://github.com/kubernetes/kubernetes/issues/1535).
|
||||||
|
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
This proposal introduces a new REST resource `Workflow`. A `Workflow` is represented as a
|
||||||
|
[graph](https://en.wikipedia.org/wiki/Graph_(mathematics)), more specifically as a DAG.
|
||||||
|
Vertices of the graph represent steps of the workflow. The workflow steps are represented via a
|
||||||
|
`WorkflowStep` resource.
|
||||||
|
The edges of the graph are not represented explicitly - rather they are stored as a list of
|
||||||
|
predecessors in each `WorkflowStep` (i.e. each node).
|
||||||
|
|
||||||
|
|
||||||
|
### Workflow
|
||||||
|
|
||||||
|
A new resource will be introduced in the API. A `Workflow` is a graph.
|
||||||
|
In the simplest case it's a graph of `Job`s but it can also
|
||||||
|
be a graph of other entities (for example cross-cluster objects or other `Workflow`s).
|
||||||
|
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Workflow is a directed acyclic graph
|
||||||
|
type Workflow struct {
|
||||||
|
unversioned.TypeMeta `json:",inline"`
|
||||||
|
|
||||||
|
// Standard object's metadata.
|
||||||
|
// More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata.
|
||||||
|
api.ObjectMeta `json:"metadata,omitempty"`
|
||||||
|
|
||||||
|
// Spec defines the expected behavior of a Workflow. More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status.
|
||||||
|
Spec WorkflowSpec `json:"spec,omitempty"`
|
||||||
|
|
||||||
|
// Status represents the current status of the Workflow. More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status.
|
||||||
|
Status WorkflowStatus `json:"status,omitempty"`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
#### `WorkflowSpec`
|
||||||
|
|
||||||
|
```go
|
||||||
|
// WorkflowSpec contains Workflow specification
|
||||||
|
type WorkflowSpec struct {
|
||||||
|
|
||||||
|
// Optional duration in seconds the workflow needs to terminate gracefully. May be decreased in delete request.
|
||||||
|
// Value must be non-negative integer. The value zero indicates delete immediately.
|
||||||
|
// If this value is nil, the default grace period will be used instead.
|
||||||
|
// Set this value longer than the expected cleanup time for your workflow.
|
||||||
|
// If downstream resources (job, pod, etc.) define their TerminationGracePeriodSeconds
|
||||||
|
// the biggest is taken.
|
||||||
|
TerminationGracePeriodSeconds *int64 `json:"terminationGracePeriodSeconds,omitempty"`
|
||||||
|
|
||||||
|
// Steps contains the vertices of the workflow graph.
|
||||||
|
Steps []WorkflowStep `json:"steps,omitempty"`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
* `spec.steps`: is an array of `WorkflowStep`s.
|
||||||
|
* `spec.terminationGracePeriodSeconds`: is the terminationGracePeriodSeconds.
|
||||||
|
|
||||||
|
### `WorkflowStep`<sup>1</sup>
|
||||||
|
|
||||||
|
The `WorkflowStep` resource acts as a [union](https://en.wikipedia.org/wiki/Tagged_union) of `JobSpec` and `ObjectReference`.
|
||||||
|
|
||||||
|
```go
|
||||||
|
// WorkflowStep contains necessary information to identifiy the node of the workflow graph
|
||||||
|
type WorkflowStep struct {
|
||||||
|
// Id is the identifier of the current step
|
||||||
|
Id string `json:"id,omitempty"`
|
||||||
|
|
||||||
|
// Spec contains the job specificaton that should be run in this Workflow.
|
||||||
|
// Only one between External and Spec can be set.
|
||||||
|
Spec JobSpec `json:"jobSpec,omitempty"`
|
||||||
|
|
||||||
|
// Dependecies represent dependecies of the current workflow step
|
||||||
|
Dependencies ObjectDependencies `json:"dependencies,omitempty"`
|
||||||
|
|
||||||
|
// External contains a reference to another schedulable resource.
|
||||||
|
// Only one between ExternalRef and Spec can be set.
|
||||||
|
ExternalRef api.ObjectReference `json:"externalRef,omitempty"`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
* `workflowStep.id` is a string to identify the current `Workflow`. The `workfowStep.id` is injected
|
||||||
|
as a label in `metadata.annotations` in the `Job` created in the current step.
|
||||||
|
* `workflowStep.jobSpec` contains the specification of the job to be executed.
|
||||||
|
* `workflowStep.externalRef` contains a reference to external resources (for example another `Workflow`).
|
||||||
|
|
||||||
|
```go
|
||||||
|
type ObjectDependencies struct {
|
||||||
|
...
|
||||||
|
ExistenceDependencies []ObjectReference `json:"existenceDependencies,omitempty"`
|
||||||
|
ControllerRef *ObjectReference `json:"controllerRef,omitempty"`
|
||||||
|
...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
* `dependencies.controllerRef`: will contain the policy to trigger current `WorkflowStep`
|
||||||
|
|
||||||
|
For `Workflow` basic scenario 2 controllers should be implemented: `AllDependenciesRunToCompletion`,
|
||||||
|
`AtLeastOneDependencyRunToCompletion`. This approach permits implementation of other kinds of triggering
|
||||||
|
policies, like for example data availability or other external event.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### `WorkflowStatus`
|
||||||
|
|
||||||
|
```go
|
||||||
|
// WorkflowStatus contains the current status of the Workflow
|
||||||
|
type WorkflowStatus struct {
|
||||||
|
Statuses []WorkflowStepStatus `json:statuses`
|
||||||
|
}
|
||||||
|
|
||||||
|
// WorkflowStepStatus contains the status of a WorkflowStep
|
||||||
|
type WorkflowStepStatus struct {
|
||||||
|
// Job contains the status of Job for a WorkflowStep
|
||||||
|
JobStatus Job `json:"jobsStatus,omitempty"`
|
||||||
|
|
||||||
|
// External contains the
|
||||||
|
ExternalRefStatus api.ObjectReference `json:"externalRefStatus,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// WorkflowList implements list of Workflow.
|
||||||
|
type WorkflowList struct {
|
||||||
|
unversioned.TypeMeta `json:",inline"`
|
||||||
|
// Standard list metadata
|
||||||
|
// More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata
|
||||||
|
unversioned.ListMeta `json:"metadata,omitempty"`
|
||||||
|
|
||||||
|
// Items is the list of Workflow
|
||||||
|
Items []Workflow `json:"items"`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
* `workflowStepStatus.jobStatus`: contains the `Job` information to report current status of the _step_.
|
||||||
|
|
||||||
|
## Events
|
||||||
|
|
||||||
|
The events associated to `Workflow`s will be:
|
||||||
|
|
||||||
|
* WorkflowCreated
|
||||||
|
* WorkflowStarted
|
||||||
|
* WorkflowEnded
|
||||||
|
* WorkflowDeleted
|
||||||
|
|
||||||
|
## Relevant use cases out of scope of this proposal
|
||||||
|
|
||||||
|
* As an admin I want to set quota on workflow resources
|
||||||
|
[#13567](https://github.com/kubernetes/kubernetes/issues/13567).
|
||||||
|
* As an admin I want to re-assign a workflow resource to another namespace/user<sup>2</sup>.
|
||||||
|
* As a user I want to set an action when a workflow ends/start
|
||||||
|
[#3585](https://github.com/kubernetes/kubernetes/issues/3585)
|
||||||
|
|
||||||
|
<sup>1</sup>Something about naming: literature is full of different names, a commonly used
|
||||||
|
name is _task_, but since we plan to compose `Workflow`s (i.e. a task can execute
|
||||||
|
another whole `Workflow`) we have chosen the more generic word `Step`.
|
||||||
|
|
||||||
|
<sup>2</sup>A very common feature in industrial strength workflow tools.
|
||||||
|
|
||||||
|
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||||
|
[]()
|
||||||
|
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||||
Reference in New Issue
Block a user