core: define generic command core type for all kinds of tasks

Previously the Command core type was only useful for Validators and was
a bit hacky in that Holos appends the directory to the end of the
argument vector.

This patch changes the Command type to represent a generic command for
use as a Generator, Transformer, or Validator.  The type is extended to
support the semantics of reading output from a file, directory, or
standard output pipe.

Go templates are used to fill in Holos managed data values, for example
the temporary directory associated with the pipeline task.  The TaskData
structu represents these values passed into the template engine.

api/core/v1alpha5/types.go

@@ -256,12 +256,46 @@ type Validator struct {
 	Command Command `json:"command,omitempty" yaml:"command,omitempty"`
 }
 
-// Command represents a command vetting one or more artifacts.  Holos appends
-// fully qualified input file paths to the end of the args list, then executes
-// the command.  Inputs are written into a temporary directory prior to
-// executing the command and removed afterwards.
+// Command represents a generic command for use as a Generator, Transformer, or
+// Validator.  Holos uses the Go template engine to render the Args field using
+// data provided by the TaskData field.  For example to fill in the fully
+// qualified temporary directory used to provide inputs to the task.
 type Command struct {
+	// DisplayName represents a friendly display name for the command.
+	DisplayName string `json:"displayName,omitempty" yaml:"displayName,omitempty"`
+	// Args represents the complete command argument vector as a go template.
 	Args []string `json:"args,omitempty" yaml:"args,omitempty"`
+	// OutputRef references the source of the output data.
+	OutputRef OutputRef `json:"outputRef,omitempty" yaml:"outputRef,omitempty"`
+	// TaskData populated by Holos for template rendering.
+	TaskData TaskData `json:"taskData,omitempty" yaml:"taskData,omitempty"`
+	// TODO(jjm): add command environment variable support similar to args.
+}
+
+// TaskData represents data values associated with a pipeline task necessary to
+// execute the task.  For example, the randomly generated temporary directory
+// used to read and write artifact files when executing user defined task
+// commands.  Values of this struct are intended for the Go template engine.
+//
+// Holos populates this struct as needed.  Holos may treat user provided values
+// as an error condition.
+type TaskData struct {
+	// TempDir represents the temp directory holos manages for task artifacts.
+	TempDir string `json:"tempDir,omitempty" yaml:"tempDir,omitempty"`
+}
+
+// OutputRef represents a reference to the data source used as the output of a
+// task.  For example, a Generator output may be sourced from standard output or
+// a file path.
+type OutputRef struct {
+	// Kind represents the kind of output produced.
+	Kind string `json:"kind,omitempty" yaml:"kind,omitempty" cue:"string | *\"Pipe\" | \"Path\""`
+	// Pipe represents stdout or stderr.  Ignored unless kind is Pipe.
+	Pipe string `json:"pipe,omitempty" yaml:"pipe,omitempty" cue:"string | *\"stdout\" | \"stderr\""`
+	// Path represents an artifact path relative to the task temp directory.
+	// Ignored unless kind is Path.
+	Path string `json:"path,omitempty" yaml:"path,omitempty"`
+	// TODO(jjm): support jsonpath or cel references to the output data maybe?
 }
 
 // InternalLabel is an arbitrary unique identifier internal to holos itself.

doc/md/api/core.md

@@ -36,11 +36,13 @@ Package core contains schemas for a [Platform](<#Platform>) and [BuildPlan](<#Bu
 - [type Kustomization](<#Kustomization>)
 - [type Kustomize](<#Kustomize>)
 - [type Metadata](<#Metadata>)
+- [type OutputRef](<#OutputRef>)
 - [type Platform](<#Platform>)
 - [type PlatformSpec](<#PlatformSpec>)
 - [type Repository](<#Repository>)
 - [type Resource](<#Resource>)
 - [type Resources](<#Resources>)
+- [type TaskData](<#TaskData>)
 - [type Transformer](<#Transformer>)
 - [type Validator](<#Validator>)
 - [type ValueFile](<#ValueFile>)
@@ -151,11 +153,18 @@ type Chart struct {
 <a name="Command"></a>
 ## type Command {#Command}
 
-Command represents a command vetting one or more artifacts. Holos appends fully qualified input file paths to the end of the args list, then executes the command. Inputs are written into a temporary directory prior to executing the command and removed afterwards.
+Command represents a generic command for use as a Generator, Transformer, or Validator. Holos uses the Go template engine to render the Args field using data provided by the TaskData field. For example to fill in the fully qualified temporary directory used to provide inputs to the task.
 
 ```go
 type Command struct {
+    // DisplayName represents a friendly display name for the command.
+    DisplayName string `json:"displayName,omitempty" yaml:"displayName,omitempty"`
+    // Args represents the complete command argument vector as a go template.
     Args []string `json:"args,omitempty" yaml:"args,omitempty"`
+    // OutputRef references the source of the output data.
+    OutputRef OutputRef `json:"outputRef,omitempty" yaml:"outputRef,omitempty"`
+    // TaskData populated by Holos for template rendering.
+    TaskData TaskData `json:"taskData,omitempty" yaml:"taskData,omitempty"`
 }
 ```
 
@@ -387,6 +396,23 @@ type Metadata struct {
 }
 ```
 
+<a name="OutputRef"></a>
+## type OutputRef {#OutputRef}
+
+OutputRef represents a reference to the data source used as the output of a task. For example, a Generator output may be sourced from standard output or a file path.
+
+```go
+type OutputRef struct {
+    // Kind represents the kind of output produced.
+    Kind string `json:"kind,omitempty" yaml:"kind,omitempty" cue:"string | *\"Pipe\" | \"Path\""`
+    // Pipe represents stdout or stderr.  Ignored unless kind is Pipe.
+    Pipe string `json:"pipe,omitempty" yaml:"pipe,omitempty" cue:"string | *\"stdout\" | \"stderr\""`
+    // Path represents an artifact path relative to the task temp directory.
+    // Ignored unless kind is Path.
+    Path string `json:"path,omitempty" yaml:"path,omitempty"`
+}
+```
+
 <a name="Platform"></a>
 ## type Platform {#Platform}
 
@@ -457,6 +483,20 @@ Resources represents Kubernetes resources. Most commonly used to mix resources i
 type Resources map[Kind]map[InternalLabel]Resource
 ```
 
+<a name="TaskData"></a>
+## type TaskData {#TaskData}
+
+TaskData represents data values associated with a pipeline task necessary to execute the task. For example, the randomly generated temporary directory used to read and write artifact files when executing user defined task commands. Values of this struct are intended for the Go template engine.
+
+Holos populates this struct as needed. Holos may treat user provided values as an error condition.
+
+```go
+type TaskData struct {
+    // TempDir represents the temp directory holos manages for task artifacts.
+    TempDir string `json:"tempDir,omitempty" yaml:"tempDir,omitempty"`
+}
+```
+
 <a name="Transformer"></a>
 ## type Transformer {#Transformer}
 

internal/generate/platforms/cue.mod/gen/github.com/holos-run/holos/api/core/v1alpha5/types_go_gen.cue

@@ -284,12 +284,49 @@ package core
 	command?: #Command @go(Command)
 }
 
-// Command represents a command vetting one or more artifacts.  Holos appends
-// fully qualified input file paths to the end of the args list, then executes
-// the command.  Inputs are written into a temporary directory prior to
-// executing the command and removed afterwards.
+// Command represents a generic command for use as a Generator, Transformer, or
+// Validator.  Holos uses the Go template engine to render the Args field using
+// data provided by the TaskData field.  For example to fill in the fully
+// qualified temporary directory used to provide inputs to the task.
 #Command: {
+	// DisplayName represents a friendly display name for the command.
+	displayName?: string @go(DisplayName)
+
+	// Args represents the complete command argument vector as a go template.
 	args?: [...string] @go(Args,[]string)
+
+	// OutputRef references the source of the output data.
+	outputRef?: #OutputRef @go(OutputRef)
+
+	// TaskData populated by Holos for template rendering.
+	taskData?: #TaskData @go(TaskData)
+}
+
+// TaskData represents data values associated with a pipeline task necessary to
+// execute the task.  For example, the randomly generated temporary directory
+// used to read and write artifact files when executing user defined task
+// commands.  Values of this struct are intended for the Go template engine.
+//
+// Holos populates this struct as needed.  Holos may treat user provided values
+// as an error condition.
+#TaskData: {
+	// TempDir represents the temp directory holos manages for task artifacts.
+	tempDir?: string @go(TempDir)
+}
+
+// OutputRef represents a reference to the data source used as the output of a
+// task.  For example, a Generator output may be sourced from standard output or
+// a file path.
+#OutputRef: {
+	// Kind represents the kind of output produced.
+	kind?: string & (string | *"Pipe" | "Path") @go(Kind)
+
+	// Pipe represents stdout or stderr.  Ignored unless kind is Pipe.
+	pipe?: string & (string | *"stdout" | "stderr") @go(Pipe)
+
+	// Path represents an artifact path relative to the task temp directory.
+	// Ignored unless kind is Path.
+	path?: string @go(Path)
 }
 
 // InternalLabel is an arbitrary unique identifier internal to holos itself.