Reading the mirror
Your instance’s tables are a mirror of the source system, not a copy of its UI. Most of the surprises below share one shape: the query succeeds, the page renders, and the data is absent or subtly wrong. Nothing errors. This page is the list of things worth checking before you trust a result.
Reading rows always runs as a person
Section titled “Reading rows always runs as a person”Rows are never read by PostgREST under a token. A personal access token
reaches schema that way — /v1/catalog, types generation, the authoring MCP —
but PostgREST rejects it outright, because a PAT is an opaque bearer rather than
a JWT:
{"code":"PGRST301","message":"JWSError (CompactDecodeError Invalid number of parts: Expected 3 parts; got 1)"}Until 1.3.x a PAT carrying a data:read scope could be traded for a session at
POST /auth/pat-exchange. Both are gone and neither is coming back. The session
it returned was the token owner’s ordinary session — write included, for an
admin owner — so a long-lived token in a project file was standing write access
to production data.
Four ways to read rows, each running under a real person’s identity and RLS:
Inside a hosted app, you need nothing: the app is served same-origin and
rides the visitor’s session cookie. select() through src/lib/turbobase.ts
and the signed-in user’s row policies apply.
From a native app you build — iOS, Android or desktop — sign the person in
through the instance’s own OAuth server and read /rest/v1 with the token it
returns. That token is their session, so the same row policies fire as in a
hosted app; it lasts eight hours and renews quietly, for thirty days from the
sign-in, while their approval stands. On day thirty-one the person signs in
again. See building a native app.
From your own editor, connect the data MCP as
yourself. You sign in once over OAuth and your client gets list_records,
get_record and search_fields as tools — scoped to you, expiring with your
session. Connect it from Integrations settings, and revoke it from the
Connected apps tab of your account settings.
From a script or an unattended agent that is not an AI client — a label
printer, a warehouse script, an overnight integration — present a PAT carrying
the rest:read permission as a bearer at
POST /v1/batch. You send PostgREST query strings and get their rows back, so
your script keeps the read grammar it would use at /rest/v1, and the request
runs as the administrator who minted the token, under that person’s row-level
permissions. rest:write beside it reaches POST /v1/data/<recordType>, the
upload route and DELETE /v1/data/<recordType> — the same write boundary an app
goes through, so your hooks, guardrails and expectedSyncVersion all apply.
/rest/v1 itself takes a browser session only and does not accept a token,
which is why a script reads through the batch endpoint.
From an AI client without a person signed in, present a PAT carrying the
mcp:data permission as a bearer at the same data MCP
endpoint. Reads run as the administrator who minted the token, under that
person’s row-level permissions, and the session behind them is minted inside
each request and never handed to your client.
Neither credential is granted by default — an administrator ticks it
deliberately in the mint dialog — and each read permission is refused the
matching write on its own. A token carrying mcp:data alone cannot change
anything, whatever the instance’s MCP write setting says; changing data through
that endpoint takes mcp:write as well, and the instance’s MCP write setting
on. Neither pair implies the other, so a token connected to the data MCP is not
thereby allowed on the web interface. From an app itself, writes still go
through POST /v1/data as a signed-in person.
permission denied usually means “not signed in”
Section titled “permission denied usually means “not signed in””{"code":"42501","message":"permission denied for table cr_acme_incoming"}This reads as your role cannot see this table. Far more often it means there was no session at all: PostgREST fell back to the anonymous role, which is denied everything. Check the session before you go looking at grants.
In local development this is almost always a dev-proxy gap — the sign-in page
loads its assets from /_app, so if that path isn’t proxied the login form
renders but never hydrates, and you stay signed out with no error. The scaffolded
vite.config.ts proxies it; don’t remove that entry.
Five shapes that return nothing
Section titled “Five shapes that return nothing”1. A jsonb custom field is a bare id
Section titled “1. A jsonb custom field is a bare id”Every mirrored custom field is reported as jsonb, which says nothing about what
is inside. It is the bare source internal id:
"cf_ns_custrecord_vendor": 1074Not {"value": 1074, "text": "FedEx"}. There is no display text and — unlike
standard columns such as entity_id / entity_name — no display twin.
Resolve the id against its lookup table yourself. An app built on the wrapper
assumption renders empty cells, which look exactly like an empty table.
An empty custom field holds the JSON literal null, not SQL NULL. So
is.null matches zero rows on these columns, and lt.0 matches blanks as well
as negatives (jsonb sorts null below every number).
2. The obvious lookup table may be the wrong one
Section titled “2. The obvious lookup table may be the wrong one”A List/Record field named after a custom record does not necessarily point at
it — a carrier field can point at entity (vendors) while a
customrecord_acme_carrier table also exists and holds something else entirely.
Confirm the target by checking that real ids resolve to sensible rows, not by
matching names.
3. Some tables mix kinds in one id space
Section titled “3. Some tables mix kinds in one id space”entity holds customers and vendors — the same id is a valid row of each.
order holds every connector’s orders. GET /v1/catalog/records marks these
with a sharedTable note naming the discriminator. Apply it:
/rest/v1/entity?entity_type=eq.vendor&external_id=eq.1604/rest/v1/order?source_system=eq.netsuiteSkip it and you will label a vendor with a customer’s name, and both queries will have succeeded.
The note only names filters that genuinely narrow, so read it rather than
assuming one exists. discriminators is empty where nothing separates the kinds
at all — tax_code holds NetSuite’s salesTaxItem and taxGroup, so filtering
by source excludes nothing — and there the hint tells you to match on
pg_row_id and check the row you read. Where a filter narrows only partly, the
hint names the record types it leaves behind.
4. Five timestamps are the mirror’s, and two of them are worth rendering
Section titled “4. Five timestamps are the mirror’s, and two of them are worth rendering”created_at, updated_at, synced_at, last_modified_pg and
source_confirmed_at are all on Turbobase’s own clock, not the source’s, and
the catalog marks all five provenance: "mirror". After a bulk sync
created_at, synced_at and last_modified_pg are identical across the whole
table, so ordering by one of those gives you sync batch order, not business
order.
updated_at is when the row last changed — stamped whenever a change
arrives from the connected source or one of your own writes edits the record. It
is the column to order a “recently touched” list by, and the one updated_since
filters on. Nothing else moves it: a write-back completing, a freshness confirm,
a source-id backfill and a platform maintenance backfill all leave it alone, so
a sweep that re-reads a record and finds it unchanged does not make it look
edited. Use created_at when you want import order — that is the one updated_at
used to answer by accident.
source_confirmed_at is the other one worth putting on a screen: it is when the
source was last asked about this row and agreed with it, whether or not a
value moved, so it answers “is this figure current” in the same request that
returns the row. It is not updated_at — a row re-read thirty seconds ago and
found identical has a four-hour-old updated_at and a thirty-second-old
source_confirmed_at. NULL means never confirmed, not just-confirmed, so
render it as “not confirmed yet” rather than as age zero.
For real business dates use source_modified (provenance: "source") or the
source’s own date field inside raw.
5. A column can exist and be empty on every row
Section titled “5. A column can exist and be empty on every row”currency_name and subsidiary_name are frequently NULL while currency_id /
subsidiary_id carry the data. A display contract that points at an unpopulated
column renders every total unlabelled — worse than wrong when not everything is
in one currency. Check one real row before rendering a column, not just the
schema.
Follow links with embeds, not a second read
Section titled “Follow links with embeds, not a second read”One request can pull related rows in alongside the row they hang off. Two kinds of link are embeddable, and the first is worth far more than the second:
- A header and its lines. A real foreign key. Works on every instance today, needs nothing declared, and is where nearly all the wasted bytes are.
- A
jsonbcustom-field link your instance has declared. Resolved on the server overraw->>'<key>'.
Read a transaction’s lines as an embed — never out of the header’s raw
Section titled “Read a transaction’s lines as an embed — never out of the header’s raw”Every line table carries a real foreign key to its header
(item_receipt_line.parent_pg_row_id → item_receipt.pg_row_id), so PostgREST
embeds the lines directly. Name the columns you actually render:
/rest/v1/order?select=external_id,tran_date,order_line(line_index,line_order,item_id,quantity)Order the lines on line_order, not on line_index. line_index is the
line’s identity — the key that does not move when the document is re-read — and
it is what you match on when you join one document’s lines to another’s. Treat
it as opaque, and never as a position. For a source whose lines are separate
records (Salesforce, HubSpot) it is a hash of the line’s own id and carries no
order at all; for NetSuite it is the line’s own line id, which is not in display
order either — a sales order whose lines are shown 1st, 2nd, 3rd can carry ids
8, 4 and 9. Sorting on line_index scrambles the document in both cases.
line_order is the position a person sees, counting from 1, and twenty line
tables carry it. It can be null — on a NetSuite line whose position could
not be read from the document it was synced with, until that document next
changes in NetSuite, and on a source that states no order of its own — so the
portable form falls back: order=line_order.asc.nullslast,line_index.asc.
Six of the fifteen header/line pairs need a constraint hint — the !… in the
next example — because two foreign keys join them. The table below marks which;
read that before you copy a line from here.
/rest/v1/item_receipt?select=external_id,tran_date,item_receipt_line!item_receipt_line_parent_pg_row_id_fkey(line_number,item_id,quantity)The shape this replaces is pulling lines out of the header’s raw — select=raw
(or select=lines:raw->item->items) and then walking the array in the browser.
That is the most expensive query shape on the platform. A header’s raw is
the entire source document, line array included; we have measured single requests
of that shape at ~10 MB on live instances, and a first paint doing it for
three record types moves tens of megabytes and takes seconds. The embed above
moves kilobytes, because no raw column crosses the wire at all.
Header / line pairs that embed this way. Nine embed bare; six need the hint — copy the exact embed text from the right-hand column:
| Header | Line embed |
|---|---|
order |
order_line(…) |
order_fulfillment |
order_fulfillment_line(…) |
invoice |
invoice_line(…) |
cash_sale |
cash_sale_line(…) |
estimate |
estimate_line(…) |
opportunity |
opportunity_line(…) |
credit_memo |
credit_memo_line(…) |
purchase_order |
purchase_order_line(…) |
vendor_bill |
vendor_bill_line(…) |
item_receipt |
item_receipt_line!item_receipt_line_parent_pg_row_id_fkey(…) |
return_authorization |
return_authorization_line!return_authorization_line_parent_pg_row_id_fkey(…) |
vendor_return_authorization |
vendor_return_authorization_line!vendor_return_authorization_line_parent_pg_row_id_fkey(…) |
journal_entry |
journal_entry_line!journal_entry_line_parent_pg_row_id_fkey(…) |
check |
check_line!check_line_parent_pg_row_id_fkey(…) |
transfer_order |
transfer_order_line!transfer_order_line_parent_pg_row_id_fkey(…) |
The hint is always the line table’s own name plus
!<line_table>_parent_pg_row_id_fkey, so you can derive it rather than memorise
the six.
Why the hint, and what happens without it
Section titled “Why the hint, and what happens without it”Those six line tables carry two foreign keys to their header — the
parent_pg_row_id one above, and an older one on the source id
(tran_external_id / tran_internal_id / je_internal_id). PostgREST refuses to
guess which to traverse and answers 300 with PGRST201, listing both
candidates. The response is helpful, but it is an error: no rows come back.
Hint the parent_pg_row_id constraint, not the source-id one. Both resolve
the ambiguity, but parent_pg_row_id is NOT NULL on every line while the
source-id column is nullable — a record created in Turbobase and not yet synced
to the source has no source id yet, so hinting the older constraint silently drops
its lines. That failure looks like “the header has no lines”, which is far harder
to spot than a 300.
You can filter, order and cap the embedded rows without touching the parent query, by prefixing the parameter with the embed name (one line, wrapped here for readability):
/rest/v1/purchase_order ?select=external_id,purchase_order_line(line_index,line_order,item_id,quantity) &purchase_order_line.quantity=gt.0 &purchase_order_line.order=line_order.asc.nullslast,line_index.asc &purchase_order_line.limit=200Those parameters filter the lines, not the purchase orders — a PO with no matching lines still comes back, with an empty array. Selecting parents by what their lines contain is a different problem an embed can’t solve; ask for a view.
If your app reads a *_line table separately and joins it back to the header on
tran_external_id in memory, or reads lines out of raw, replace it with the
embed. Nothing has to be declared and nothing has to be enabled.
Embedding across a declared jsonb link
Section titled “Embedding across a declared jsonb link”Some records point at another record through a jsonb custom field — an item
receipt carries the internal id of the purchase order it was created from in
raw->>'createdFrom'. Where your instance has declared that link, embed it:
/rest/v1/item_receipt?select=*,created_from_po(*)created_from_po(*) pulls each receipt’s originating purchase order into the same
response. It is:
- one round trip — no second read of
purchase_order, no in-memory join; - resolved on the server — the join runs in Postgres over
raw->>'createdFrom', not in the browser; - RLS-scoped — the embed is filtered by the same row policy as the base read, so it can never surface a row your session couldn’t already see on its own.
The links declared on the item-receipt / purchase-order pair:
| From | Embed | Pulls in |
|---|---|---|
item_receipt |
select=*,created_from_po(*) |
the purchase order the receipt was created from |
item_receipt_line |
select=*,source_po(*) |
the purchase order a receipt line traces back to |
purchase_order |
select=*,receipt_lines(*) |
the receipt lines booked against the PO |
Relationships are declared, not discovered
Section titled “Relationships are declared, not discovered”Your instance does not grow a resolver for every link field it can see. A
source account has hundreds of them and a given app uses a handful, so a
relationship exists because something asked for it: your app declares the ones it
embeds in public/turbobase.json, or someone creates one in the instance UI.
Declaring is what causes it to be materialized — discovery on its own does
nothing.
{ "name": "ops", "relationships": [ { "name": "inbound_shipment", "sourceTable": "item_receipt", "sourceJsonbKey": "custbody_acme_inbound_shipment", "targetTable": "cr_inbound_shipment" } ]}The list your app can rely on is the list in your own manifest — plus the header→line embeds above, which are structural and always present.
A declared embed works from the moment you declare it. Materializing the fast
path (the resolver function and its index) is a background pass, but you never
wait on it and never gate a screen on it: select() and selectPage() try the
native embed first, and while it isn’t armed yet the instance resolves the same
embed for you server-side and returns the same rows. Arming changes how fast the
read answers, not what it answers — and your code is identical either way, because
the fallback reuses the query string you already wrote.
One constraint holds only during that window: the instance-resolved path
reproduces a straight rel(columns) embed exactly, and refuses what it can’t
reproduce — nested embeds, !inner, spread embeds, embed-scoped filters — with a
501 naming the alias instead of quietly returning different rows. Keep a newly
declared embed simple and add nested embeds or !inner once it’s armed.
(Header→line embeds are never affected; they’re native from the start.)
Spread embeds (...rel(col)) are the exception — don’t use them at all. They
aren’t merely unsupported while unarmed: on PostgREST 12 they’re rejected outright,
and on 14 the request succeeds but the spread column comes back under its own name
alongside the parent’s column of the same name, so the parent’s value is the one
you lose. That’s a silent wrong-value bug, so it’s the one shape to avoid armed or
not. Select the embedded column normally and read it from the array.
A declared relationship’s embed is always an array
Section titled “A declared relationship’s embed is always an array”Even when the link is to-one. created_from_po comes back as a one-element array,
not an object — row.created_from_po[0]?.tran_id, never
row.created_from_po.tran_id. The second form is undefined on every row and
renders as an empty column rather than an error, so it survives review easily.
This differs from a plain foreign-key embed, where PostgREST follows the
cardinality of the FK: a header’s lines come back as an array
(row.item_receipt_line), but a to-one FK embed like currency:currency_id(symbol)
comes back as an object or null. Declared relationships are uniform because they
resolve through a set-returning function, so index into them and you’re right in
both directions.
A missing embed means “not synced yet”, not “no link”
Section titled “A missing embed means “not synced yet”, not “no link””Embeds over a jsonb link are eventually consistent. The resolver matches the
link id (raw->>'createdFrom') against the target’s external_id. If the target
row hasn’t synced yet — a receipt landed before its purchase order — the embed
comes back empty (created_from_po: []) even though the link id is present on the
row. That is a timing gap, not a signal that the relationship doesn’t exist.
To tell the two apart, read the raw link id itself
(select=external_id,createdFrom:raw->>createdFrom): if it’s populated but the
embed is empty, the target simply hasn’t landed yet and the embed will fill in on
its next sync. Don’t render an empty embed as “orphaned”, and don’t write a
migration/repair on the strength of one.
Header→line embeds don’t have this failure mode — a header and its lines are written by the same sync, so the FK is never dangling.
A custom record’s children are not lines
Section titled “A custom record’s children are not lines”The third kind of parent/child, and the one that looks most like the first while behaving like the second.
In Oracle NetSuite, a native transaction owns a real sublist: its lines have no identity of their own, they arrive in the same save as the header, and they get the real foreign key that the whole first half of this section relies on.
A custom record has no sublist. What you see rendered as one on its form is a second custom record type whose rows each carry a List/Record field pointing back at the parent. Each child is a first-class record with its own internal id, its own permissions, and its own place in the mirror. NetSuite draws them under the parent as a convenience; nothing else about them is line-like.
So in your instance they are two independent tables with no foreign key between
them. The link is the promoted column that the parent field was projected into,
holding the parent’s external_id:
turbobase.cr_acme_inbound_shipment -- parent external_id = '162'
turbobase.cr_acme_inbound_shipment_item -- child cf_ns_custrecord_acme_isi_parent = '162' ← the linkThree consequences:
-
There is no
parent_pg_row_id. Reaching for it is the single most common mistake here; that column belongs to native<table>_linetables only. -
A bare embed does not work, because PostgREST embeds traverse foreign keys and there is no constraint to traverse. Read the children with a filter on the link column — the child’s own table, filtered by the parent’s
external_id:cr_<child>?<link column>=eq.<parent external_id>To get a parent and its children in one request instead of two, the pair has to be declared as a computed relationship — the same mechanism as the
jsonblinks above, and subject to the same eventual-consistency caveat. Ask your Turbobase administrator to declare it; it is not discovered automatically. -
The link is an id, not a row. Like every other custom-field reference it is a bare external id (see “A
jsonbcustom field is a bare id”), so joining it to the parent means matching it againstexternal_id— never againstpg_row_id.
Writing follows the same shape, in the other direction: you cannot POST lines
to a custom record (it is a 422 LINES_NOT_SUPPORTED). Create the parent, read
its external_id off the response, then POST each child to its own record
type with the link column set to that id.
Querying efficiently
Section titled “Querying efficiently”Project keys out of raw, never select it whole
Section titled “Project keys out of raw, never select it whole”raw is the entire source document. Selecting it across a wide table can produce
a response large enough to fail at the edge (502). Pull out the one key you need:
/rest/v1/vendor_bill?select=external_id,total,inbound:raw->custbody_acme_inbound_shipment-> keeps the JSON type, ->> gives text. Deep paths use #>/#>>:
raw#>'{inventory,items}'.
If the key you’re reaching into raw for is the line array, stop — embed the
line table instead (see Follow links with embeds above). raw->item->items is
the single most expensive read shape on the platform, and the line table holds the
same rows in typed columns.
One bad column loses the whole table
Section titled “One bad column loses the whole table”PostgREST rejects an unknown column with 42703 and fails the entire
request — one wrong name in a select returns nothing for every row. Record
types are not uniform: memo, total, currency_name, created_from_id and
even pg_row_id are absent from some transaction tables.
Check each table’s real column list (GET /v1/catalog/records/:type) rather than
assuming a shared transaction shape, and assert your selected columns against it
in a test if you query several tables the same way.
Page large reads with selectPage(), don’t drain
Section titled “Page large reads with selectPage(), don’t drain”PostgREST caps an unbounded read at 1000 rows. A select() that comes back at
exactly 1000 rows was almost certainly cut short — a partial page that looks
identical to a complete one. The generated client makes that loud: an unbounded
select() that hits the cap logs a TurbobaseTruncationError telling you to
paginate. (A select() you gave an explicit limit opted into a bounded read and
stays quiet.)
To read a full result set, page it with selectPage() — a keyset seek, not
offset (which slows on deep pages and can skip or repeat rows under concurrent
writes):
import { selectPage } from './lib/turbobase';
let { rows, nextCursor } = await selectPage('item_receipt?select=*', { pageSize: 100 });render(rows);while (nextCursor) { ({ rows, nextCursor } = await selectPage('item_receipt?select=*', { pageSize: 100, cursor: nextCursor })); render(rows);}Pass the SAME path+query you’d give select(); selectPage() supplies the order,
limit, and seek. nextCursor is null on the last page. The cursor is opaque and
pinned to the exact query that minted it — replaying it against a changed
filter/select throws TurbobaseCursorError rather than silently mixing two result
sets.
A seek key must be NOT NULL and UNIQUE
Section titled “A seek key must be NOT NULL and UNIQUE”pg_row_id is the default and is right for almost every table. Read-only master
tables that have no pg_row_id need { sortKey: 'external_id' } — and only
those.
Get this wrong and nothing errors. A keyset seek walks <key>=gt.<lastValue>, so
a shared boundary value skips every row that shares it, and a NULL sorts to one
end and is never passed. The list simply comes back short.
Don’t hard-code the column — read it from your generated types. turbobase project types emits a SEEK_KEYS map derived from the live schema:
import { SEEK_KEYS } from './types';
const { rows } = await selectPage('vendor_credit?select=*', { sortKey: SEEK_KEYS.vendor_credit,});That matters because the right answer changes. Five transaction headers
(vendor_credit, credit_card_charge, credit_card_refund — now a
record_kind = 'refund' row on credit_card_charge — inventory_adjustment,
inventory_status_change) had no pg_row_id, so
external_id was the documented fallback — then they gained one, and their
external_id became nullable and unique only per connection in the same release.
Every app carrying that override started dropping rows, on a column that still
existed. Reading SEEK_KEYS means regenerating types is the fix; hard-coding
means there isn’t one.
If you do hard-code it, selectPage() now throws when it can see you’re wrong —
when the rows coming back carry a pg_row_id and you passed something else, and
when the seek key is null on the last row of a page that has more rows.
A SEEK_KEYS entry may be a tuple, and it is passed exactly the same way. A
line table is keyed (parent_pg_row_id, line_index) — line_number on a few of
them — and has no correct single column: every line of one transaction shares
parent_pg_row_id, and the line’s identity is unique only within a parent
(and line_order, the position, is not in the key at all). Pass both,
in key order, and selectPage() seeks the pair with a row-wise comparison, so a
page boundary that falls mid-parent neither drops nor repeats a row:
const { rows, nextCursor } = await selectPage('vendor_bill_line?select=*', { sortKey: ['parent_pg_row_id', 'line_index'], pageSize: 100,});Add { descending: true } to walk any key — single column or tuple — backwards.
The direction is pinned into the cursor, so one can’t be replayed against the
other. A table absent from SEEK_KEYS has no keyset-safe key at all; page it by
filtering to a bounded subset instead.
Cursors do not survive a client upgrade. A cursor now carries every column of
the seek key, and one minted by an older client is refused with a
TurbobaseCursorError rather than reinterpreted — reading it as a one-column
tuple would silently drop rows. If you persist a cursor to storage or a URL,
re-read from the first page after upgrading.
Embeds page fine: pageSize counts parent rows, and each parent carries its
embedded rows whole, so
selectPage('order?select=*,order_line(*)') pages orders and brings every
order’s lines with it.
Keep concurrency low
Section titled “Keep concurrency low”Your instance is metered in two places, and they count differently. The one that governs ordinary reads is the edge rule below; a handful of endpoints carry their own budget inside the instance as well, and those are covered at the end of this section.
The edge rule, in front of everything
Section titled “The edge rule, in front of everything”One rule covers everything your instance serves, applied before a request reaches it: 1000 requests per 10 seconds, per client IP.
Three things about how it counts:
- It’s one budget for the whole instance. Every request that reaches your
instance shares it:
/rest/v1reads,/v1/*calls, the Realtime connection, and any page or asset served fresh rather than from cache. - Only requests that actually reach your instance count. Assets answered from the edge cache don’t, so a page reload is usually far cheaper than its request count in devtools suggests.
- It’s per client IP — one browser tab’s fan-out, not your app’s total traffic. Users behind one shared NAT egress share a budget.
Every /rest/v1 response restates the budget so you can pace against it
instead of hard-coding a number:
X-RateLimit-Limit: 1000X-RateLimit-Window: 10X-RateLimit-Scope: instance-subdomainX-RateLimit-Limit is requests, X-RateLimit-Window is the seconds they’re
counted over, and X-RateLimit-Scope is the reminder that the count is shared
across everything your instance serves. These are advisory: they describe
the budget, they aren’t a live meter.
On this edge budget there is deliberately no X-RateLimit-Remaining,
X-RateLimit-Reset or Retry-After. The counter lives at the edge, not in the
response path, so any number in those headers would be a guess — and a guessed
budget is worse than none. Pace against the limit and back off when you’re
refused. The in-instance budgets at the end of this section are the opposite —
their counter is in the response path, so they report all three.
What a refusal looks like. Exceed the budget and the edge — not your instance — answers:
HTTP/1.1 429 Too Many Requests
<!-- Cloudflare "error 1015 — You are being rate limited", as an HTML page -->Two consequences worth coding for:
- It isn’t JSON. Nothing in the body matches the
{"code","message","hint"}error shape your reads normally return, sores.json()throws on it. Checkres.status === 429before parsing. If a JSON parser suddenly chokes on markup, this is what happened. - It’s a flat 10-second block. Treat it as exactly that rather than reading a retry hint out of the response: wait ten seconds before the next attempt, and don’t retry in a tight loop, which only re-arms the block.
So: keep a small number of requests in flight (a handful, briefly spaced), back
off for 10s on a 429, and load what the first screen needs while fetching the
rest on demand. This matters most on dashboards that open by querying every
record type at once.
Reads your instance makes for you while server-rendering a page don’t count — they never leave the instance. The budget is about requests arriving from a browser or an external client.
Four endpoints carry their own budget as well
Section titled “Four endpoints carry their own budget as well”The edge rule is not the only limit. A few endpoints do far more work per request than a read does, so they carry their own counter inside the instance, on top of the edge rule — exceed either and you’re refused:
/mcp, the MCP server an external client or agent talks to. Its allowance is published on that page; read it there rather than fitting a delay by trial and error./v1/batch, the multiplexed read described in the next section.- The auth endpoints (
/auth/callbackand the token exchanges), where every attempt costs real crypto and database work, so the budget is deliberately small. - The webhook ingress your source systems post to.
Unlike the edge budget, these report themselves. The counter is in the
response path, so every response from one of these endpoints carries
X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset, and a
refusal adds Retry-After in whole seconds — a live meter rather than the
advisory figures above. Pace against those numbers, and on a 429 wait exactly
what Retry-After says: retrying sooner neither extends the window nor
shortens it.
A refusal here comes from your instance rather than the edge, so it arrives in
the format that endpoint normally speaks — JSON for /v1/batch and the auth
routes, and for /mcp a JSON-RPC error whose error.data repeats the limit,
what’s left and the seconds to wait as numbers.
These budgets are counted per instance process, not per client, so on a busy instance you can be refused before your own request count reaches the published figure.
A request also has a deadline of its own
Section titled “A request also has a deadline of its own”Separately from how MANY requests you may send, there is a limit on how long one
may take to arrive. Your instance gives a request 120 seconds to finish
arriving, and answers a 503 carrying statusCode, error, message and
Retry-After when it does not — rather than closing the socket with nothing on
it, which is what a client sees as a request that vanished.
That deadline sits a whole minute above the front door’s own 60-second patience
with one request, deliberately: the front door is always the first of the two to
give up, so a large apps push that is still making progress is never the thing
this cuts. If you do meet it, the cause is a stalled connection rather than the
size of what you sent — send it again.
An unexpected failure answers the same shape: a 5xx whose body carries a
correlationId. The cause is in your instance’s log under that id; quote it when
you ask an administrator. Nothing about the inside of the instance is in the
body, so there is nothing to parse out of the message.
Multiplex the cold-load fan-out with batch()
Section titled “Multiplex the cold-load fan-out with batch()”When the first paint genuinely needs several reads — a dashboard that queries
every record type, or a header table plus the small lookup tables (currency,
location, subsidiary) you map ids against — don’t fire them as many parallel
select() calls. Issue them as one request with batch():
import { batch } from './lib/turbobase';
const [orders, invoices, customers] = await batch([ 'order?select=external_id,total,entity_id&order=tran_date.desc&limit=100', 'invoice?select=external_id,total,status&limit=100', 'entity?select=external_id,display_name&entity_type=eq.customer&limit=500',]);
if (orders.ok) renderOrders(orders.rows); // rows is the parsed result setif (!invoices.ok) note(`invoices unavailable (${invoices.status})`);Each element is the same path-and-query string you’d pass select(). Results
come back in the same order, and each carries its own ok / status, so a
429 or 5xx on one sub-query does not fail the others — the other result
sets still render. Because it’s a single HTTP request rather than 50, it spends
one request of your rate-limit budget instead of fifty: a ~50-read cold load
collapses into one or two batch() calls (up to 50 queries each).
batch() is reads only (writes still go through write()), it’s RLS-scoped
exactly like select() (same session, same rows), and the server bounds its own
fan-out to PostgREST so a big batch can’t overload the instance. Reserve
individual select() for on-demand / drill-in reads after first paint.
Each result is
{ ok, status, rows?, error?, count?, contentRange?, truncated?, notRun? } —
rows holds the parsed set on success, error the message on failure. Under the
hood it’s a single POST /v1/batch with a
{ queries: (string | { query, count })[] } body; if you aren’t using the
generated client, call that endpoint directly with the same session cookie and
read results[i].body /
results[i].count from the response. The batch itself returns 200
whenever your session is valid — a 401 means you’re not signed in, a 400
means a malformed body, a sub-query whose path isn’t a bare resource name (the
error names the offending index), or too many exact counts (below). The cap is
50 queries per call.
Check each result’s own ok / status before you read its rows. The
envelope stays 200 for anything that goes wrong with an individual
sub-query, which includes a 403 for a session that ended mid-batch — the
person signed out elsewhere, was suspended, or had their role changed. Reading
rows without looking at status hands a refusal to a screen as an empty set.
See App auth wiring for how to tell that 403 from a
permissions one.
batch() is for cold fan-out across unrelated record types, and for the small
lookup tables (entity_id → entity) you map ids against. It is not the
answer to a header and its lines, or to a declared link — embed those instead (see
Follow links with embeds above) and the server resolves the join in one hop.
Reaching for batch() first is how a two-query page becomes a fifty-query page.
One page per sub-query — 1000 rows, and it says when it cut
Section titled “One page per sub-query — 1000 rows, and it says when it cut”Each sub-query returns at most 1000 rows. Ask for more — limit=5000, or no
limit at all — and the instance serves the first 1000 and sets truncated on
that result:
const [people] = await batch(['entity?select=external_id,email&entity_type=eq.customer']);
if (people.truncated) { // `rows` is a PAGE, not the set. Page on, or narrow the query. const [more] = await batch([ 'entity?select=external_id,email&entity_type=eq.customer&limit=1000&offset=1000', ]);}Three things to know:
truncatedis absent, notfalse, when the page is whole. So the check isif (r.truncated). A sub-query whose ownlimitis 1000 or less can never carry it — if you asked for 25 rows and got 25, that is your page and nothing was cut.- Page with
offset, or narrow the read. If you need to know how far, ask that sub-query for a total ({ query, count: 'exact' }, below) — the cap applies to the page, never to the count, socountis still the whole filtered set. - A cut page is one sub-query’s problem. Its siblings in the same batch are
unaffected, exactly as with a
429or a5xxon one of them.
Why the ceiling exists: a sub-query with no limit used to stream the whole
filtered set, and on a busy instance one screen doing that put every other read
behind it in the queue for the database connections your instance has. If a
surface genuinely needs tens of thousands of rows, page them with selectPage()
rather than asking for them in one shot — a browser cannot render them at once
either.
A wide batch has a deadline, and says which sub-queries never ran
Section titled “A wide batch has a deadline, and says which sub-queries never ran”At most six sub-queries read at once across your whole instance, so a batch of 50 slow reads spends most of its time queueing. The whole call therefore has a wall-clock deadline of its own, set below the front door’s patience with one request: past it, the instance stops starting sub-queries and answers with what it has.
A sub-query that never got its turn comes back ok: false, status: 504 — the
same as a read that timed out — plus notRun:
const results = await batch(queries);
// Ask again for just the slots that did not run, and merge them in.const missed = results.flatMap((r, i) => (r.notRun ? [i] : []));if (missed.length) { const rest = await batch(missed.map((i) => queries[i]!)); for (const [j, i] of missed.entries()) results[i] = rest[j]!;}
render(results.filter((r) => r.ok));Three things to know:
notRunis absent, notfalse, on every other result. So the check isif (r.notRun), exactly as withtruncated. A batch that finishes inside the deadline carries it nowhere, and nothing about such a call changed.- Ask again for the slots that did not run, not for the whole batch. Re-sending all of it re-runs the sub-queries that already came back and re-enters the same queue that ran out of time.
- Meeting it regularly means the batch is too wide, or the reads are too
slow. Split the first paint into two narrower calls, replace
select=*with the columns you render, and make sure each sub-query’s filter is one the instance can serve from an index.
Why it exists: without it a wide fan-out could run for minutes, and the front door gave up first — so the request failed with no body at all, which told you neither what came back nor which sub-queries to ask for again.
Ask for a total on the one query that needs it
Section titled “Ask for a total on the one query that needs it”A batch result’s count is the number of rows matching your filter ignoring
limit/offset — the denominator in “1–25 of 1,340”. It is opt-in per
sub-query, because it isn’t free:
const [page, currencies] = await batch([ { query: 'order?select=external_id,total&order=tran_date.desc&limit=25', count: 'exact' }, 'currency?select=external_id,symbol',]);
if (page.ok) { render(page.rows); showPager(page.count); // 1340 — the whole filtered set, not this page}// currencies.count is null: it never asked for one, and never paid for one.Three things to know:
- A counted sub-query comes back
status: 206, and that is success. PostgREST answers206 Partial Contentwhen the page it returned is shorter than the total it computed. Branch onok, never onstatus === 200— a=== 200check turns every counted page into a phantom failure. countisnullunless you asked. Withoutcount: 'exact'the server sends no count header at all and PostgREST reports the total as*(contentRange: "0-24/*").rows.lengthis the page, never the total.- Only a few per request. An exact count makes the database scan your whole
filtered set rather than the page, and nothing cancels it once it starts — so
the instance caps how many sub-queries in one request may ask for one
(
BATCH_READ_MAX_EXACT_COUNTS, 4 by default) and answers400for the whole request over the cap.batch()chunks count-aware, so it never sends a request over the cap however many counted queries you pass — but each is still a full scan, and the extra requests go out one after another. Posting to/v1/batchyourself, keep it to four. Count the one table you’re paging; leave the lookups as plain strings.
If an estimate is good enough — a rough “about 40k orders” — don’t ask for a
count at all: page with selectPage() and show what you have. The exact count is
for a pager that has to be right.
include_total is an estimate
Section titled “include_total is an estimate”The MCP list_records tool’s include_total returns PostgREST’s
count=planned — a planner estimate from table statistics, not a count. On a
recently-synced table it can be off by a factor of two or more. The response
labels it (estimated_total_note). Never quote it to a user as a total; page
through with next_cursor when the exact number matters.
A file’s bytes come from one route, never from a column
Section titled “A file’s bytes come from one route, never from a column”A mirrored file — a blob in a connected Azure Blob Storage container, a file on a
mirrored repository branch — has a row in source_file carrying its path, size,
digest, mime type and modification time. The row never carries the bytes, on
any source, and it never will. Ask for them:
GET /v1/data/file/<externalId>/contentSeveral things about that route are worth knowing before you build on it.
Let the shared client build the URL. fileUrl(row) in src/lib/turbobase.ts
takes the mirrored-file row and returns exactly the path above, encoded and with
the connection on it. Build it by hand and the part you will leave out is the
connection — which works on an instance with one container connected and stops
working on one with two.
import { fileUrl } from './lib/turbobase';
<img src={fileUrl(row)} alt={row.path} /><a href={fileUrl(row, { download: true })}>Download</a>URL-encode the external id. It carries slashes — <container>:<folder>/<name>
for Azure Blob Storage — so encodeURIComponent it or the path segments split.
fileUrl does this for you.
Name the connection, and store this URL rather than the source’s. The route
takes ?connection=<id> — the connection_id on the row — and that is what
makes the URL address the FILE rather than whichever container the viewer’s own
session defaults to. Omit it and the caller’s own connection is assumed, which
is right on a single-container instance and wrong as soon as a second one is
connected. Because the URL carries no host and no version marker, it is the
string to save on any record that points at a file: it keeps resolving after the
file is replaced, and it opens for every user who may read the file.
An image, a video or a PDF renders in place; everything else downloads. The
response is content-disposition: inline for raster images, video/* and
audio/* containers a browser plays, and application/pdf. Every other type —
including HTML, SVG, XML, JavaScript and a file whose type the connector never
recorded — is an attachment with nosniff. The list is an allowlist, so a type
nobody has catalogued downloads rather than rendering. Add ?download=1 to force
the attachment answer for any type.
Seek with Range, and revisit with If-None-Match. A request carrying
Range: bytes=… answers 206 with content-range and accept-ranges: bytes,
and the window is fetched from the source as a window — so setting a <video src> to this URL gives you a seekable player without moving the whole file. A
malformed or out-of-range request answers 416, never the whole body. Every
answer carries the file’s etag, so a second view sending If-None-Match
answers 304 without re-reading the connected account. cache-control is
private, no-cache: a browser may keep its own copy and revalidate it, and
nothing is ever shared between users.
A person opening the link while signed out lands on sign-in, then on the
file. A request whose Accept asks for HTML — a browser navigating — is
302-redirected to /login and returned here afterwards, which is what makes
this URL safe to paste into a field somebody clicks from another system. Your
own fetch calls ask for JSON, */* or an image type and still get the
unchanged 401 envelope, so nothing that parses that envelope changes.
The answer is the same whichever way the connection stores its files. A
customer connecting a container chooses whether the files are copied into their
instance or left in Azure and fetched when one is opened. Either way this route
answers identically — same status, same disposition, same mime_type, same
etag, same 206 behaviour — so an app never learns which mode a connection
uses and must not branch on it. It asks for the same read:source_file
capability the row does, and who may see which file is decided by Turbobase
permissions: nobody needs access of their own to the storage account, and no
account host, signature or redirect to it ever reaches the browser.
404 and 409 both say “no bytes”, and they are not the same thing. A file
whose contents have not arrived yet answers 404 content_unavailable. One that is
there and can no longer be fetched — a credential withdrawn, the blob moved in the
source — answers 409 content_unavailable with a sentence saying which. Render a
409 as “we can’t reach this right now”, never as “deleted”: a 409 read as deleted
is how a customer concludes their files are gone.
A file your own app has just saved is readable as soon as the write settles,
so you can post its link somewhere the moment the create comes back synced:
the write records where the bytes went, and a file whose record somehow missed
that is served by working out where it must be. Keep a retry on 404 content_unavailable for a file you created moments ago, though — one case
remains. A connection that copies its files into the instance holds no bytes for
a brand-new file until the next container sweep, and as the paragraph above says
your app cannot tell which kind of connection it is on, so a retry rather than a
branch is what covers it. Before this, a link shared straight after a save
showed that error for hours on every connection.
For a file you did NOT just create, 404 content_unavailable means what it says
— the contents were genuinely never mirrored, which today is a file the instance
declined to hold because it is over the connection’s size ceiling.
Adding a file is two requests, and removing one is not available yet
Section titled “Adding a file is two requests, and removing one is not available yet”Writing a file is only open on sources whose write path has been switched on for the instance — today, a connected Azure Blob Storage container. Send the bytes first and name the handle you get back on the create:
POST /v1/data/file/uploads content-type: application/octet-stream → 201 {"ok":true,"upload":{"upload_id":…,"size_bytes":…}}POST /v1/data/file {"path":"legal/2026/addendum.md","raw":{"upload_id":…}}Bytes go in the body, never base64 inside JSON and never in a column. The ceiling
is 8 MiB (413 UPLOAD_TOO_LARGE); an empty body is 422 EMPTY_UPLOAD. A create
missing either input is refused at the boundary — 422 NO_FILE_CONTENTS for the
bytes, 422 NO_TARGET_PATH for the destination — and both are published on the
record’s required_inputs, so list_record_types tells you what a create needs
rather than making you discover it from a refusal. path is the full path inside
the connected container, folders and all; folders need no creating.
Expect three more refusals and handle them distinctly: BIDIRECTIONAL_DISABLED
until staff switch writing on for the instance, CONTAINER_OUT_OF_SCOPE for a
path outside the one container the connection names, and
CONTAINER_NOT_RECOVERABLE when overwriting existing bytes in a storage account
that has neither versioning nor soft delete turned on. Creating a file at a path
that does not exist destroys nothing and is not held to that last rule.
Do not build a remove-a-file flow. Deleting is built and deliberately switched off pending one verification against a real storage account, so a delete is refused before the adapter is reached. Adding and overwriting are available; removing is not.