A controller groups resources of one type and defines the workers to reconcile for them.
The centralized load/1 callback is used to load domain data for
reconciliation and worker runs.
Using this module
defmodule MyApp.OrderController do
use Steward.Controller,
steward: MyApp.Steward,
reconcile_interval: :timer.hours(24)
@impl Steward.Controller
def workers, do: [MyApp.Steward.FulfillOrderWorker]
@impl Steward.Controller
def load(%Steward.Resource{} = resource) do
case MyApp.Orders.get!(resource.id) do
# Returning :cancel stops reconciliation of the resource object.
%{state: :cancelled} -> :cancel
# Fill the virtual `object` field with arbitrary data.
order -> {:ok, %{resource | object: order}}
end
end
enduse Steward.Controller also defines shorthands for managing the controller's resources,
e.g. MyController.schedule_resource!(id) for schedule_resource!(MyController, id). See
create_resource/3, get_resource/2, list_resources_by_meta/2, schedule_resource!/2,
process_resource/4, cancel_resource!/2 and resume_resource!/2.
Options
:steward- the module defined withuse Steward, which stores this controller's resources. Required.:reconcile_interval- how long after a reconciliation a resource is reconciled again, in milliseconds. Defaults to:infinity, i.e. resources are only reconciled after their jobs ran and when scheduled on demand, see below.
Recurring reconciliation vs. on-demand reconciliation
Use the :reconcile_interval option with caution. A frequently recurring reconciliation spends
compute on resources that haven't changed, up to the point where individual "poller workers"
require less resources. It is typically preferable to explicitly schedule your resource for
reconciliation when external state is modified, hence in your domain operations.
See the Steward.Controller.process_resource/4 function for a common integration pattern.
However, a recurring but infrequent reconciliation may serve as a safety net to protect against
accidentally forgetting about the process_resource/4 or schedule_resource!/2 in a code path.
Summary
Types
Function passed to process_resource/4. Returns {schedule?, result}: when schedule? is
true, the resource is scheduled for reconciliation; result is passed through.
Callbacks
Called with the resource before reconciliation and processing.
Called to fetch the set of related workers for this controller.
Functions
Cancels the resource, ending its lifecycle.
Creates a new resource and schedules it for immediate reconciliation.
Fetches a Steward.Resource by id.
Lists the resources whose :meta contains meta.
Loads a resource, processes it through a handler, and optionally schedules it for reconciliation.
Resumes a cancelled resource, scheduling it for reconciliation.
Schedules the given resource for asynchronous reconciliation.
Types
@type processor() :: (Steward.Resource.t() -> {boolean(), term()})
Function passed to process_resource/4. Returns {schedule?, result}: when schedule? is
true, the resource is scheduled for reconciliation; result is passed through.
Callbacks
@callback load(Steward.Resource.t()) :: {:ok, Steward.Resource.t()} | :cancel
Called with the resource before reconciliation and processing.
This function allows to load arbitrary domain data before calls to Steward.Worker.spec/3,
Steward.Worker.perform/2, and within client calls to Steward.Controller.process_resource/4.
Fill the virtual :object field in the Steward.Resource.t struct with your data.
Return :cancel if the given resource is not relevant anymore and should be removed from
the reconciliation loop. Its pending jobs are deleted, yet currently executing jobs continue
to run.
Implementation should ideally be pure and must be free of side effects on Steward resources,
e.g. no calls to schedule_resource!/2.
@callback workers() :: [module()]
Called to fetch the set of related workers for this controller.
Functions
@spec cancel_resource!(module(), Steward.Resource.t() | Steward.Resource.id()) :: Steward.Resource.t()
Cancels the resource, ending its lifecycle.
The resource is reconciled one last time, which deletes its pending jobs. Executing jobs run to completion. Afterwards, the resource is no longer reconciled.
@spec create_resource(module(), map(), keyword()) :: {:ok, Steward.Resource.t()} | {:error, Ecto.Changeset.t()}
Creates a new resource and schedules it for immediate reconciliation.
Associating a resource with a domain entity
To associate a resource with a domain entity of your choice, you have the following options:
- The
:metafield (map) can carry arbitrary domain information, like foreign keys. Look up resources by it withlist_resources_by_meta/2. - You may force the (UUID)
:idattribute to associate the record with a domain entity. - You can store the resource's id on your domain entity.
Example
create_resource(MyController, %{id: "1d7a1e1c-6a52-4f0e-9a8e-2f4c1b5d3e9a"})or
create_resource(MyController, %{meta: %{order_id: 123}})Last argument is passed as options to Ecto.Repo.insert/2.
@spec get_resource(module(), Steward.Resource.id()) :: {:ok, Steward.Resource.t()} | {:error, :not_found}
Fetches a Steward.Resource by id.
@spec list_resources_by_meta(module(), map()) :: [Steward.Resource.t()]
Lists the resources whose :meta contains meta.
Matches by JSON containment: a resource matches if its :meta has all keys of meta with
equal values. Nested maps match partially as well, lists match if they contain all given
elements.
Values are compared in their JSON form, e.g. atoms match strings.
Uses the GIN index on :meta created by Steward.Migrations.up/1. If you chose not to create
this index in your migration, this function's query will result in a full table scan.
Example
list_resources_by_meta(MyController, %{"order_id" => 123})
@spec process_resource(module(), Steward.Resource.t(), processor(), keyword()) :: :cancelled | term()
Loads a resource, processes it through a handler, and optionally schedules it for reconciliation.
fun receives the loaded resource and returns {schedule?, result}. When schedule?
is true, the resource is scheduled for reconciliation. result is returned.
If Steward.Controller.load/1 returns :cancel, fun is not called and this function
returns :cancelled. The resource is cancelled and the resource's next reconciliation
will delete its pending jobs.
If the resource is already cancelled (based on the given struct), again fun is not called
and this function returns :cancelled.
Exception handling
If fun raises, throws or exits, the resource is scheduled for reconciliation before the
error is re-raised, to avoid silently dropping it from its reconciliation loop. Pass
schedule_on_error: false to disable this.
Transactions
Must not be called within a database transaction, and raises if it is. Scheduling on
error would be rolled back together with the transaction, silently dropping the resource
from its reconciliation loop. fun may open transactions of its own.
Integration example
defmodule MyApp.OrderController do
[...]
def process_order(order, fun) do
# See `create_resource/3` for options for how to connect a resource to a domain entity.
{:ok, resource} = get_resource(order.steward_resource_id)
process_resource(resource, fn resource ->
# You could have more centralized boilerplate here, like transaction & locking.
# Reconcile after the function exits.
{true, fun.(resource.object)}
end)
end
end
# In operations:
def cancel_order(order) do
OrderController.process_order(order, fn loaded_order ->
# ...
end)
end
@spec resume_resource!(module(), Steward.Resource.t() | Steward.Resource.id()) :: Steward.Resource.t()
Resumes a cancelled resource, scheduling it for reconciliation.
Reconciliation recreates the jobs its workers' specs desire. Jobs that completed before the cancellation are kept until the resource is pruned, so they are not enqueued again.
Raises Steward.ResourceAlreadyPrunedError if the resource's jobs were already pruned (see
the :prune_max_age option of Steward.Plugin), as its workers' jobs could run again.
Raises Steward.ResourceNotFoundError if the resource doesn't exist.
Resuming a resource that isn't cancelled only schedules it.
@spec schedule_resource!(module(), Steward.Resource.t() | Steward.Resource.id()) :: Steward.Resource.t()
Schedules the given resource for asynchronous reconciliation.