Pools and autoscaling

Pools

A pool is a group of worker slots with one executor (or one set of remote workers), one autoscaling policy and its own queue bounds. A slot runs one attempt at a time.

The minimum bounds the autoscaler's desired capacity: desired is never below it. It does not set capacity itself. In the library, initial_workers is the capacity at start (at most the maximum, and not raised to the minimum); in the server, capacity comes only from connected workers.

Use several pools to isolate workloads with different executors, hardware or scaling needs; use resources to share limits between them.

Capacity

• In-process (Rust or Python library): a slot is a concurrently running task. The default SlotScaler sets capacity to whatever the autoscaler asks for, immediately.
• Server: capacity is the sum of slots of the workers connected to the pool. The scaler only requests capacity (External: export a metric; Webhook: POST the number), and capacity changes when workers connect or disconnect.
• You can report capacity yourself with Scheduler::set_capacity (Rust) or set_capacity (Python) for externally managed pools.

The queue-pressure policy

Every interval (1 s) each autoscaled pool computes:

λ        arrival rate (EWMA, α = 0.3) of tasks becoming ready
S        mean service time (EWMA of observed run times; default_duration before any)
B        backlog per class: ready tasks plus tasks waiting for a retry
         (fixed-start tasks that are due count as critical)

base     = ceil(λ × S / target_utilization)             target_utilization = 0.75
             (0 when below 0.01: the smoothed rate decays toward zero but never reaches it)
drain    = ceil(ΣB × S / drain_time)                    drain_time = 30 s
slo      = max over classes c with backlog and an SLO:
             ceil(B_≤c × S / slo[c])                    B_≤c = backlog in classes c and higher
reserved = peak concurrent reservations in [now, now + reservation_horizon]   horizon = 60 s

desired  = clamp(max(base + drain, slo, busy) + reserved, min_workers, max_workers)

Then:

• Scale up immediately, by at most max(ceil(current × max_step_up_ratio), max_step_up_abs) per evaluation (defaults: +100% or +4).
• Scale down only to the highest desired value seen over the stabilization_window (60 s), by at most max_step_down (1) per cooldown (30 s).
• Running tasks are never interrupted: when capacity shrinks below the busy count, slots retire as their attempts finish.

Cost

Set cost_per_worker_second and each pool reports worker_seconds and cost, accumulated since start, in snapshots. Cost is also exported as a metric (cronoq_pool_cost_total in Prometheus, cronoq.pool.cost in OpenTelemetry); worker_seconds is in snapshots only.

Configuration reference

The policy is implemented in cronoq-core and is fully deterministic, which makes it testable with simulated time.