Steward.Controller behaviour (Steward v0.1.0)

Copy Markdown View Source

A controller groups resources of one type and defines the workers to reconcile for them.

The centralized load/1 callback is used to load domain data for reconciliation and worker runs.

Using this module

defmodule MyApp.OrderController do
  use Steward.Controller,
    steward: MyApp.Steward,
    reconcile_interval: :timer.hours(24)

  @impl Steward.Controller
  def workers, do: [MyApp.Steward.FulfillOrderWorker]

  @impl Steward.Controller
  def load(%Steward.Resource{} = resource) do
    case MyApp.Orders.get!(resource.id) do
      # Returning :cancel stops reconciliation of the resource object.
      %{state: :cancelled} -> :cancel
      # Fill the virtual `object` field with arbitrary data.
      order -> {:ok, %{resource | object: order}}
    end
  end
end

use Steward.Controller also defines shorthands for managing the controller's resources, e.g. MyController.schedule_resource!(id) for schedule_resource!(MyController, id). See create_resource/3, get_resource/2, list_resources_by_meta/2, schedule_resource!/2, process_resource/4, cancel_resource!/2 and resume_resource!/2.

Options

  • :steward - the module defined with use Steward, which stores this controller's resources. Required.

  • :reconcile_interval - how long after a reconciliation a resource is reconciled again, in milliseconds. Defaults to :infinity, i.e. resources are only reconciled after their jobs ran and when scheduled on demand, see below.

Recurring reconciliation vs. on-demand reconciliation

Use the :reconcile_interval option with caution. A frequently recurring reconciliation spends compute on resources that haven't changed, up to the point where individual "poller workers" require less resources. It is typically preferable to explicitly schedule your resource for reconciliation when external state is modified, hence in your domain operations.

See the Steward.Controller.process_resource/4 function for a common integration pattern.

However, a recurring but infrequent reconciliation may serve as a safety net to protect against accidentally forgetting about the process_resource/4 or schedule_resource!/2 in a code path.

Summary

Types

Function passed to process_resource/4. Returns {schedule?, result}: when schedule? is true, the resource is scheduled for reconciliation; result is passed through.

Callbacks

Called with the resource before reconciliation and processing.

Called to fetch the set of related workers for this controller.

Functions

Cancels the resource, ending its lifecycle.

Creates a new resource and schedules it for immediate reconciliation.

Lists the resources whose :meta contains meta.

Loads a resource, processes it through a handler, and optionally schedules it for reconciliation.

Resumes a cancelled resource, scheduling it for reconciliation.

Schedules the given resource for asynchronous reconciliation.

Types

processor()

(since 0.1.0)
@type processor() :: (Steward.Resource.t() -> {boolean(), term()})

Function passed to process_resource/4. Returns {schedule?, result}: when schedule? is true, the resource is scheduled for reconciliation; result is passed through.

Callbacks

load(t)

(since 0.1.0)
@callback load(Steward.Resource.t()) :: {:ok, Steward.Resource.t()} | :cancel

Called with the resource before reconciliation and processing.

This function allows to load arbitrary domain data before calls to Steward.Worker.spec/3, Steward.Worker.perform/2, and within client calls to Steward.Controller.process_resource/4.

Fill the virtual :object field in the Steward.Resource.t struct with your data.

Return :cancel if the given resource is not relevant anymore and should be removed from the reconciliation loop. Its pending jobs are deleted, yet currently executing jobs continue to run.

Implementation should ideally be pure and must be free of side effects on Steward resources, e.g. no calls to schedule_resource!/2.

workers()

(since 0.1.0)
@callback workers() :: [module()]

Called to fetch the set of related workers for this controller.

Functions

cancel_resource!(controller, resource_or_id)

(since 0.1.0)
@spec cancel_resource!(module(), Steward.Resource.t() | Steward.Resource.id()) ::
  Steward.Resource.t()

Cancels the resource, ending its lifecycle.

The resource is reconciled one last time, which deletes its pending jobs. Executing jobs run to completion. Afterwards, the resource is no longer reconciled.

create_resource(controller, user_attrs \\ %{}, opts \\ [])

(since 0.1.0)
@spec create_resource(module(), map(), keyword()) ::
  {:ok, Steward.Resource.t()} | {:error, Ecto.Changeset.t()}

Creates a new resource and schedules it for immediate reconciliation.

Associating a resource with a domain entity

To associate a resource with a domain entity of your choice, you have the following options:

  • The :meta field (map) can carry arbitrary domain information, like foreign keys. Look up resources by it with list_resources_by_meta/2.
  • You may force the (UUID) :id attribute to associate the record with a domain entity.
  • You can store the resource's id on your domain entity.

Example

create_resource(MyController, %{id: "1d7a1e1c-6a52-4f0e-9a8e-2f4c1b5d3e9a"})

or

create_resource(MyController, %{meta: %{order_id: 123}})

Last argument is passed as options to Ecto.Repo.insert/2.

get_resource(controller, resource_id)

(since 0.1.0)
@spec get_resource(module(), Steward.Resource.id()) ::
  {:ok, Steward.Resource.t()} | {:error, :not_found}

Fetches a Steward.Resource by id.

list_resources_by_meta(controller, meta)

(since 0.1.0)
@spec list_resources_by_meta(module(), map()) :: [Steward.Resource.t()]

Lists the resources whose :meta contains meta.

Matches by JSON containment: a resource matches if its :meta has all keys of meta with equal values. Nested maps match partially as well, lists match if they contain all given elements.

Values are compared in their JSON form, e.g. atoms match strings.

Uses the GIN index on :meta created by Steward.Migrations.up/1. If you chose not to create this index in your migration, this function's query will result in a full table scan.

Example

list_resources_by_meta(MyController, %{"order_id" => 123})

process_resource(controller, resource, fun, opts \\ [])

(since 0.1.0)
@spec process_resource(module(), Steward.Resource.t(), processor(), keyword()) ::
  :cancelled | term()

Loads a resource, processes it through a handler, and optionally schedules it for reconciliation.

fun receives the loaded resource and returns {schedule?, result}. When schedule? is true, the resource is scheduled for reconciliation. result is returned.

If Steward.Controller.load/1 returns :cancel, fun is not called and this function returns :cancelled. The resource is cancelled and the resource's next reconciliation will delete its pending jobs.

If the resource is already cancelled (based on the given struct), again fun is not called and this function returns :cancelled.

Exception handling

If fun raises, throws or exits, the resource is scheduled for reconciliation before the error is re-raised, to avoid silently dropping it from its reconciliation loop. Pass schedule_on_error: false to disable this.

Transactions

Must not be called within a database transaction, and raises if it is. Scheduling on error would be rolled back together with the transaction, silently dropping the resource from its reconciliation loop. fun may open transactions of its own.

Integration example

defmodule MyApp.OrderController do
  [...]

  def process_order(order, fun) do
    # See `create_resource/3` for options for how to connect a resource to a domain entity.
    {:ok, resource} = get_resource(order.steward_resource_id)

    process_resource(resource, fn resource ->
      # You could have more centralized boilerplate here, like transaction & locking.

      # Reconcile after the function exits.
      {true, fun.(resource.object)}
    end)
  end
end

# In operations:
def cancel_order(order) do
  OrderController.process_order(order, fn loaded_order ->
    # ...
  end)
end

resume_resource!(controller, resource_or_id)

(since 0.1.0)
@spec resume_resource!(module(), Steward.Resource.t() | Steward.Resource.id()) ::
  Steward.Resource.t()

Resumes a cancelled resource, scheduling it for reconciliation.

Reconciliation recreates the jobs its workers' specs desire. Jobs that completed before the cancellation are kept until the resource is pruned, so they are not enqueued again.

Raises Steward.ResourceAlreadyPrunedError if the resource's jobs were already pruned (see the :prune_max_age option of Steward.Plugin), as its workers' jobs could run again. Raises Steward.ResourceNotFoundError if the resource doesn't exist.

Resuming a resource that isn't cancelled only schedules it.

schedule_resource!(controller, resource_or_id)

(since 0.1.0)
@spec schedule_resource!(module(), Steward.Resource.t() | Steward.Resource.id()) ::
  Steward.Resource.t()

Schedules the given resource for asynchronous reconciliation.