Implement post-processor component

[sudomateo]

Create a post-processor component that can promote an image from a project image to a silo image.

[sudomateo]

Going to defer on this for the initial Packer plugin work. Perhaps this can be an argument on the builder instead. Going to feel that out a bit before committing to a post-processor.

[lgfa29]

Thinking a bit more about this with @sudomateo and a post-processor may give users more control over their builds since they can be selected per source.

Suppose an environment where a LTS base image is available at silo-level, but a more bleeding edge image is available at project-level.

With a post-processor it would be possible to only promote the LTS image:

data "oxide-image" "lts" {
  name = "lts"
}

data "oxide-image" "latest" {
  name = "latest"
}

source "oxide-instance" "lts" {
  boot_disk_image_id = data.oxide-image.lts.image_id
  # ..
}

source "oxide-instance" "latest" {
  boot_disk_image_id = data.oxide-image.latest.image_id
  # ..
}

build {
  sources = [
    "source.oxide-instance.lts",
    "source.oxide-instance.latest",
  ]
  # ...
  post-processor "oxide-promote" {
    only = ["source.oxide-instance.lts"]
    keep_input_artifact = true
  }
}

A few caveats to note:

• Post-processors run for every build, so when building mixed images (such as an AWS AMI and an Oxide image) the post-processor needs to be gated to only run for the Oxide image. This is a similar pattern when using the docker plugin for example.
• The Packer docs notes that "The input 'artifact' received by a post-processor will be automatically deleted by default.". User can set keep_input_artifact = true to avoid this, but we should probably investigate exactly what this means at the plugin level. We probably don't want to delete the image after promotion.

[sudomateo]

Nice writeup! I think I agree that this should be a post-processor.

• The Packer docs notes that "The input 'artifact' received by a post-processor will be automatically deleted by default.". User can set keep_input_artifact = true to avoid this, but we should probably investigate exactly what this means at the plugin level. We probably don't want to delete the image after promotion.

Looks like this might be up to the implementation to respect. In our case, an artifact is an image ID that's stored in Oxide. Promoting an image does not change its ID so it doesn't make sense to destroy the previous artifact. This is notably different than, say, a container image tag that's different than the previous tag or an image moved to a new region with a different ID. In those cases you would want to destroy the previous artifact after you've processed it into a new artifact.

The PostProcessor interface looks like this.

type PostProcessor interface {
	HCL2Speccer

	// Configure is responsible for setting up configuration, storing
	// the state for later, and returning and errors, such as validation
	// errors.
	Configure(...interface{}) error

	// PostProcess takes a previously created Artifact and produces another
	// Artifact. If an error occurs, it should return that error. If `keep` is
	// true, then the previous artifact defaults to being kept if user has not
	// given a value to keep_input_artifact. If forceOverride is true, then any
	// user input for keep_input_artifact is ignored and the artifact is either
	// kept or discarded according to the value set in `keep`.
	// PostProcess is cancellable using context
	PostProcess(context.Context, Ui, Artifact) (a Artifact, keep bool, forceOverride bool, err error)
}

The Artifact interface has a Destroy method that Packer will call to destroy previous artifacts.

type Artifact interface {
	// ... other methods.

	// Destroy deletes the artifact. Packer calls this for various reasons,
	// such as if a post-processor has processed this artifact and it is
	// no longer needed.
	Destroy() error
}

We currently have that unimplemented for our builder.Artifact.

packer-plugin-oxide/component/builder/instance/artifact.go

Lines 51 to 54 in 38a556c

	// Destroy deletes the artifact when it is determined to no longer be needed.
	func (a *Artifact) Destroy() error {
	return nil
	}

We should probably.

• Implement Destroy on our builder.Artifact.
• Implement this image-promote post-processor.
• Do a sanity check to ensure all the IDs for the components are good. Packer uses things like artifact IDs to ensure a given builder's artifact is accepted by the post-processor. I gave this some thought initially but admittedly I was learning the Packer idioms from a development perspective.

[linear]

SSE-152