Steward.Worker behaviour (Steward v0.1.0)

Copy Markdown View Source

A wrapper around Oban.Worker integrating it with Steward's reconciliation engine.

Using this module

defmodule MyApp.Steward.FulfillOrderWorker do
  use Steward.Worker, queue: :default, max_attempts: 5

  @impl Steward.Worker
  def spec(spec, %{object: order} = _resource, _existing_resource_jobs) do
    if order.fulfilled do
      spec
    else
      schedule(spec, %{some: :arg}, scheduled_at: DateTime.add(order.inserted_at, 2, :day))
    end
  end

  @impl Steward.Worker
  def perform(_oban_job, %{object: order}) do
    # Invalidates the order.fulfilled precondition in spec/3.
    MyApp.Orders.mark_fulfilled!(order)
    :ok
  end
end

Summary

Callbacks

Callback replacing Oban.Worker.perform/1 in your workers.

Called as part of the resource reconciliation to calculate the desired execution plan.

Callbacks

perform(t, t)

(since 0.1.0)
@callback perform(Oban.Job.t(), Steward.Resource.t()) :: Oban.Worker.result()

Callback replacing Oban.Worker.perform/1 in your workers.

Steward.Worker implements the default Oban.Worker.perform/1. It loads the resource's object by calling Steward.Controller.load/1 on its controller and passes it as the second argument to Steward.Worker.perform/2. Afterwards, it schedules the resource for reconciliation if the job is about to enter a terminal state, i.e. it succeeded, was discarded or cancelled, or failed its last attempt.

Terminal states outside of perform/2

Reconciliation is only scheduled when perform/2 returns or raises. Jobs entering a terminal state by other means don't trigger it, e.g. jobs

  • cancelled with Oban.cancel_job/2 or through Oban Web,
  • killed after exceeding their Oban.Worker.timeout/1 in their last attempt,
  • or discarded by Oban's lifeline after their node crashed during their last attempt.

Their resource isn't reconciled until it's scheduled otherwise. Use the :reconcile_interval option of Steward.Controller as a safety net, or call Steward.Controller.schedule_resource!/2 when you cancel jobs yourself.

spec(t, t, list)

(since 0.1.0)

Called as part of the resource reconciliation to calculate the desired execution plan.

Callback implementation must be pure and deterministic. Repeated runs of the reconciliation should result in the same job schedule, hence the implementation must schedule the same set of jobs in every run as long as application state hasn't changed.

The reconciliation engine consolidates the desired execution plan with jobs in Oban's queues.

Scheduling jobs

When your worker's preconditions are fulfilled, you can enqueue a job for immediate execution.

schedule(spec, %{some: :args})

The :scheduled_at option schedules a job into the future. To avoid "pushing" the job further into the future in repeated reconciliation runs, Oban's :schedule_in convenience option is not allowed.

schedule(spec, %{some: :args}, scheduled_at: tomorrow)

Job uniqueness

In order to reconcile the desired job in the spec and actual jobs in Oban's queues, Steward needs to be able to uniquely identify each job it manages. The reconciliation process identifies jobs based on their worker, controller, and the id of the given Steward.Resource, meaning a worker can only enqueue a single job per resource by default.

If you wish to enqueue multiple jobs from a single worker, the :steward_unique_key option distinguishes them by an arbitrary value.

schedule(spec, %{some: :args}, steward_unique_key: my_job_id)

Steward tries its best to ensure job uniqueness by controlling the job's :unique and :replace options. Passing either to schedule/3 will raise an ArgumentError. Uniqueness holds over all job states (including :completed) for an infinite amount of time.

Oban.Pruner

If you enable Oban's pruner with its :pruner option, make sure you do not rely on Steward's job uniqueness in your workers. Instead, we advise that your worker's Steward.Worker.perform/2 implementation mutates application state in a way that Steward.Worker.spec/3 preconditions are invalidated.

Batched inserts

Also note that Steward uses Oban.insert_all to insert jobs in a batch, which when combined with Oban's Basic engine does not verify uniqueness constraints. However, as Steward's reconciler checks uniqueness within a spec, the only risk of non-unique jobs is manual inserts of duplicates while a resource is being reconciled.

Changes outside of Steward

Scheduled jobs may be changed outside of Steward, e.g. rescheduled through Oban Web. Steward keeps such changes per field until the desired value for that field changes, i.e. until the spec asks for something new.