Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Keeper Bot

The keeper is an off-chain bot that cranks settlement, lending, mode transitions, and withdrawals on a schedule. Mode transitions are automatic: the bot opens or closes the basis when the on-chain smoothed funding signal and guardrails permit, with no manual intervention. The position, NAV-attestation, and lending instructions require the vault's authorized_keeper to sign — so the keeper is a permissioned executor, bounded by the on-chain rules, not a decision-maker. settle, process_withdrawal, and emergency_close are permissionless, so anyone can keep the vault safe and redeemable if the keeper goes offline.

v1 hedges on Phoenix Perps (USDC margin via Ember). The keeper assembles the Phoenix and Ember account groups with the Rise SDK. Mode transitions run automatically off the on-chain funding signal; the keeper bot executes them and cranks settlement, lending, and withdrawals.

Reference implementation: the keeper — index.ts is the loop, duties/ are the individual cranks, shared plumbing is in its shared libraries. TypeScript + Anchor + Jupiter API. Run with npm run keeper (KEEPER_MODE=dry-run to plan only, KEEPER_MODE=simulate to build and simulate without sending). npm run config:check validates the configuration first and refuses anything that would run but not work.

Cadence

The keeper ticks every KEEPER_INTERVAL seconds — 300 by default, and hard capped at 900 on mainnet, because deposits and withdrawals revert NavStale once NAV is 30 minutes old. That is a liveness constraint, not a preference: the keeper refuses to start with a longer interval.

Instruction
Frequency
Trigger

settle

Every tick

Refresh funding EMA, drawdown peak, LST depeg check, settle_pnl if open. While parked it is the only thing keeping NAV fresh.

rebalance_hedge

When |delta| / NAV clears delta_band_bps (25 = ~every other week)

Sweep the unhedged jitoSOL accrual back onto the short. Bounded by max_rebalance_clip_lots, which admin must set — see the clip.

attest_nav(new_nav_usdc)

After settle

Snap NAV to live position value when on-chain reads can't capture unrealized PnL. Bounded by max_nav_change_bps_per_hour.

open_position

When the smoothed funding signal clears the threshold and dwell has elapsed

One-shot per mode entry

close_position

When the signal falls back through the threshold

One-shot per mode exit

lend_idle_usdc / unlend_usdc

Whenever idle vault USDC exceeds the buffer target by > a margin (or buffer is short)

Keep USDC productive without breaching liquidity_buffer_target

lend_reserve / unlend_reserve

Same, for the reserve fund

Reserve fund stays in Kamino unless an admin draw is queued

process_withdrawal

When the queue has unprocessed requests AND vault holds enough idle USDC for the next request (FIFO)

Permissionless — anyone can crank

emergency_close

When drawdown guard trips OR vault is paused with a position open

Permissionless once tripped

add_margin

Every tick while a position is open, when margin health drops below 2.5

The keeper moves funds on its own here — see below

reduce_position

When the withdrawal queue still falls short after redeeming the whole Kamino position, and that shortfall persists across ticks

De-lever both legs to fund the queue: buy back part of the short and sell the matching slice of jitoSOL, so net exposure never moves. It waits deliberately — tearing down part of the hedge realises slippage on both legs and permanently shrinks the position, so a queue one deposit would satisfy must not trigger it. Also the margin backstop, and permissionless below hard_margin_floor_bps

Every perp fill is priced against the oracle, whoever sent it. open_position, close_position, rebalance_hedge, reduce_position and emergency_close all band the realized fill against the pinned SOL/USD mark by max_swap_slippage_bps, computed from the position delta the trader account records. The keeper's min_base_lots_to_fill / min_quote_lots_to_fill are its own floor on top; the program's bound does not depend on the keeper getting them right, which matters most on emergency_close, where the caller is a stranger.

Margin only ever moves onto the venue to defend the position. Nothing the keeper does reduces collateral on a short that stays the same size — the only way collateral comes back is if the short itself gets smaller, which reduce_position does by shrinking both legs together. Surplus margin raises a margin-surplus warning so an operator can judge whether the drag is worth closing early, and when idle USDC and the Kamino balance can no longer fund a top-up, reduce_position de-levers both legs proportionally rather than pulling collateral out from under the short.

The rebalance clip

initialize turns rebalancing on — delta_band_bps = 25 — but leaves max_rebalance_clip_lots at 0, which means unclipped: one order may take the entire correction. That is deliberate, because the right ceiling is a function of the venue's book depth and the program never sees the book. It is also the one thing that must not be left as it ships.

It matters because rebalance_hedge places an IOC market order with min_base_lots_to_fill = 0 and no price_in_ticks. The fill price is bounded after the fact — assert_fill_price_within bands it against the pinned SOL/USD mark — so the clip is what bounds how much size one order pushes into the book, and how far that order walks the book.

The keeper knows and will warn (clip-unset, clip-oversized) but cannot fix it: update_params is admin-gated. Measure and set it:

Re-measure after any large move in Phoenix volume — the clip is 1% of a typical day, and a book that halves makes yesterday's clip twice as aggressive.

Signal pipeline (open / close decisions)

The normal↔parked transition is decided by the on-chain funding signal; the keeper bot assembles and submits the switch automatically whenever the rules below permit. The pipeline is that funding logic.

  1. Read funding. settle parses the last funding rate + timestamp from the perp venue's SOL-PERP market and updates the on-chain EMA (funding_apr_smoothed_bps). Sanity-bounded — absurd values revert with FundingRateInsane.

  2. Stale check. If now - funding_smooth_last_ts > funding_max_staleness_seconds, opens revert with FundingSignalStale. Run settle first.

  3. Threshold + dwell + drawdown latency.

  • If funding_apr_smoothed_bps ≥ funding_threshold_normal_bps AND now − last_mode_change_ts ≥ min_dwell_seconds → call open_position.

  • Else if in the active mode but the signal decayed below threshold + hysteresis → call close_position.

  • Drawdown trips require consecutive_dd_settles_observed ≥ consecutive_dd_settles_required (default 2) before emergency_close can fire on a NAV breach — single-tick noise is rejected with DrawdownTriggerLatent.

  1. Build the transaction. Assemble per-instruction remaining_accounts groups (see Instructions) and the Jupiter swap data via the Jupiter /swap-instructions endpoint.

  2. Submit. Wrap with priority-fee compute budget instructions. Retry on slot-out / blockhash-not-found.

Assembling remaining_accounts

  • Strategy instructions pack multiple CPIs into one transaction

  • Each group needs a precise account list — wrong order or missing accounts reverts the CPI

  • Use the venue's official client / API to build each group:

  • Phoenix margin (Ember) / place_perp_order / settle funding — use the Rise SDK (github.com/Ellipsis-Labs/rise-public) instruction builders and extract the keys array.

  • Kamino lending (lend_idle_usdc / unlend_usdc / lend_reserve / unlend_reserve) — use the Kamino IDL and PDA derivation utilities to build the USDC reserve deposit/withdraw account groups.

  • Jupiter route_swap — call Jupiter's /swap-instructions HTTP endpoint with userPublicKey = vaultPda, then pass the returned swapInstruction.data as jupiter_swap_data and the returned keys as the relevant remaining-accounts group.

Pass per-group counts as the u8 instruction args (jupiter_account_count, phoenix_deposit_account_count, etc.) so the on-chain handler can split safely.

Process-withdrawal cranking

Strict FIFO:

  • Next-in-line request_id is vault.queue_processed_through itself (both counters start at 0, and process_withdrawal requires request_id == queue_processed_through)

  • Any other request errors with WithdrawalNotNextInQueue

  • Anyone can crank (including the original requester)

  • Rent refunds to the original requester even if a different wallet pays the tx

Liquidity shortfall:

  • Reverts with InsufficientLiquidityBuffer if vault USDC is short of the next usdc_owed

  • Keeper should unlend_usdc first if Kamino USDC can cover the shortfall

  • Otherwise wait for the next close_* to free up USDC

Failure modes

Symptom
Cause
Resolution

MinDwellNotElapsed

Tried to switch modes too soon

Wait for last_mode_change_ts + min_dwell_seconds

FundingThresholdNotMet

Smoothed funding hasn't crossed the threshold

Re-read funding_apr_smoothed_bps

FundingSignalStale

EMA older than funding_max_staleness_seconds

Run settle first

SlippageExceeded

Jupiter route filled at worse price than expected

Refresh the quote, narrow the bound, retry

PhoenixCpiFailed

Phoenix returned an error — usually margin or oracle

Inspect program logs; may need to settle funding first

InsufficientLiquidityBuffer (process_withdrawal)

Not enough idle USDC for the next request

unlend_usdc to refill, or wait for next close_*

LendingBreachesBuffer

lend_idle_usdc would drop the buffer below liquidity_buffer_target

Reduce amount or run unlend_usdc instead

NavChangeExceedsCap

attest_nav delta > max_nav_change_bps_per_hour

Stage with smaller deltas over multiple hours, or admin raises the cap if the position genuinely moved that much

DrawdownTriggerLatent

Drawdown observed but not yet across consecutive settles

Wait one more settle cycle; this is by design

LstDepeg

jitoSOL/SOL deviation > lst_depeg_bps

settle auto-pauses; admin investigates and either unpauses or emergency-closes

Idempotency

  • All open / close instructions assert position_mode invariants on entry

  • Duplicate calls during a retry storm revert cleanly without state damage

  • settle is fully idempotent within an hour (no state change beyond the EMA refresh)

The keeper moves margin autonomously

Unlike every other duty, margin-health can move vault funds without a human. That is deliberate: liquidation on a delta-hedged short can arrive in minutes, and an alert you read an hour later is not a defence.

The mechanism it defends against is unintuitive — the jitoSOL leg is in the vault and the margin is on Phoenix, and Phoenix can only see its own side. On a rally the short loses margin while the offsetting jitoSOL gain sits somewhere Phoenix cannot count, so a hedged position gets liquidated anyway. emergency_close will not save it, because NAV doesn't fall in a hedged rally.

Bounds on that autonomy:

  • it can never draw on the withdrawal buffer

  • total posted margin can never exceed half of effective NAV (enforced on-chain, not just in the keeper)

  • collateral never falls on a short that stays the same size; de-levering closes part of the position and the matching spot leg together

  • KEEPER_AUTO_MARGIN=0 reduces it to alert-only — but then you must be reachable within minutes

If the keeper is down, or auto-margin is off, the thin-margin policy (min_margin_bps = 900) is no longer justified — close the position rather than leave it sitting. Note the 10× this implies is the venue's view of the perp leg, not portfolio leverage: the vault's total exposure equals its NAV.

Related

  • Admin operations

  • Monitoring

  • Deploying it — scripts/keeper/deploy/README.md: Railway setup, dead-man's switch, kill switch (KEEPER_READ_ONLY=1), key rotation, and the keeper-down procedure

  • Instructions reference — every entry point and who may call it

  • Errors — what a revert means

PreviousProtocol Integrations
NextAdmin Operations

Last updated