Source ingestion contract¶

Connection enablement and connector evidence are governed by the source capability contract. Payload classification, access, retention, deletion, and offboarding are governed by the data governance contract.

Ingestion profiles¶

Profile	Freshness path	Completeness path	Pilot status
Shopify	Verified webhooks	Airbyte incremental sync plus scheduled lookback	Required
Amazon SP-API	Notifications where supported	Reports and API extraction	Deferred
eBay	Notifications where supported	API extraction	Deferred
Payment provider	Provider webhooks	Balance/transaction extraction	Provider-specific

Airbyte is not a low-latency event bus. Shopify webhooks are acknowledged quickly and processed asynchronously; duplicates and out-of-order delivery are expected. Amazon also recommends a retrieval fallback when notification delivery is delayed or unavailable. Source adapters must therefore combine event-driven freshness with periodic source reconciliation.

Trusted tenant attribution¶

Raw source content never chooses its own tenant. The control plane resolves source_connection_id to one active tenant before accepting a record. The landing writer stamps:

tenant_id
source_connection_id
source_platform
airbyte_connection_id or webhook_subscription_id
ingestion_run_id
source_stream
source_primary_key
source_cursor
source_emitted_at
observed_at
payload_sha256
payload_ref
adapter_version

The tuple (tenant_id, source_connection_id, source_stream, source_primary_key, payload_sha256) is unique for snapshot records. Webhook deliveries are unique on (tenant_id, source_connection_id, provider_delivery_id).

Control tables¶

create table source_connections (
  tenant_id uuid not null,
  id uuid not null default gen_random_uuid(),
  platform text not null check (platform in ('shopify','amazon','ebay','stripe','paypal','other')),
  external_account_id text not null,
  secret_ref text not null,
  status text not null check (status in ('provisioning','active','degraded','disabled')),
  adapter_version text not null,
  created_at timestamptz not null default now(),
  primary key (tenant_id, id),
  unique (tenant_id, platform, external_account_id)
);

create table ingestion_runs (
  tenant_id uuid not null,
  id uuid not null default gen_random_uuid(),
  source_connection_id uuid not null,
  mechanism text not null check (mechanism in ('airbyte','webhook','manual_backfill','reconciliation')),
  provider_job_id text,
  status text not null check (status in ('queued','running','succeeded','failed','cancelled')),
  started_at timestamptz,
  completed_at timestamptz,
  min_source_cursor text,
  max_source_cursor text,
  records_read bigint not null default 0,
  records_accepted bigint not null default 0,
  records_rejected bigint not null default 0,
  error_code text,
  primary key (tenant_id, id),
  foreign key (tenant_id, source_connection_id)
    references source_connections(tenant_id, id)
);

create table ingestion_inbox (
  tenant_id uuid not null,
  id uuid not null default gen_random_uuid(),
  source_connection_id uuid not null,
  ingestion_run_id uuid not null,
  source_stream text not null,
  source_primary_key text not null,
  provider_delivery_id text,
  payload_sha256 char(64) not null,
  payload_ref text not null,
  source_emitted_at timestamptz,
  observed_at timestamptz not null,
  status text not null check (status in ('accepted','processing','processed','rejected','quarantined')),
  reject_code text,
  primary key (tenant_id, id),
  foreign key (tenant_id, source_connection_id)
    references source_connections(tenant_id, id),
  foreign key (tenant_id, ingestion_run_id)
    references ingestion_runs(tenant_id, id)
);

create unique index ingestion_webhook_delivery_uq
  on ingestion_inbox (tenant_id, source_connection_id, provider_delivery_id)
  where provider_delivery_id is not null;

create unique index ingestion_snapshot_payload_uq
  on ingestion_inbox (
    tenant_id, source_connection_id, source_stream,
    source_primary_key, payload_sha256
  );

Every table uses forced RLS and composite tenant foreign keys as defined by the platform specification.

Shopify pilot stream contract¶

The Airbyte Shopify connector supports incremental extraction for relevant commerce streams, but namespaces are not supported, deletion capture is limited to selected streams, and closed fulfillment orders require explicit connector configuration. The provisioning check records connector version and verifies the actual configured catalog against the connector documentation.

Capability	Required source evidence	Failure behavior
Orders and lines	Orders stream plus `orders/create`, `orders/updated`, `orders/cancelled` webhooks	Tenant cannot enable posting
Transactions	Transactions stream	Payment posting disabled
Refunds	Order Refunds stream plus refund/update webhook coverage	Refund automation disabled
Fulfillments	Fulfillments and closed fulfillment orders enabled	Delivery automation disabled
Products/SKUs	Products, variants, inventory items	Unmapped line enters exception
Inventory	Locations and inventory levels where Odoo owns operational stock	Inventory reconciliation disabled
Payouts/fees	Shopify Payments payouts and balance transactions, if applicable	Settlement close disabled
Disputes	Disputes stream, if applicable	Dispute automation disabled

Webhook receipt¶

Resolve the host/subscription to one active source connection.
Read the unmodified request body and enforce a maximum payload size.
Verify the provider signature with a constant-time comparison and a secret from the secrets manager.
Validate timestamp/replay limits where the provider supplies signed time evidence.
Persist the inbox row and encrypted payload reference in one transaction.
Return a success response only after durable receipt; do not canonicalize or call Odoo in the request.
Publish source.delivery.accepted.v1 through the transactional outbox.

Invalid signatures return 401. Unknown or disabled connections return a concealed 404. Duplicate delivery IDs return the original receipt outcome without creating downstream work.

Airbyte run contract¶

Use incremental append or append-deduped only when the stream exposes a suitable cursor and primary key.
Treat every run as at least once. Downstream uniqueness is Accountify's responsibility.
Record the connector image/version, configured catalog hash, stream cursor, run/job ID, row counts, and destination checkpoint.
Apply a configurable lookback window to mutable streams. The Shopify pilot starts with 72 hours and is tuned from observed late changes.
Run a daily count/amount reconciliation and a weekly bounded backfill for the pilot.
Quarantine schema-breaking records; do not silently drop unknown required values.
A successful Airbyte job means extraction completed, not that canonicalization or accounting posting succeeded.

Freshness and completeness¶

Measure	Pilot target	Alert
Webhook durable receipt	p95 under 2 seconds	5 minutes with no successful receipts while source traffic exists
Webhook to canonical entity	p95 under 5 minutes	p95 over 15 minutes
Airbyte incremental completion	Every 60 minutes	Two consecutive failed or missed runs
Source completeness comparison	Daily	Count or amount mismatch above zero after lookback
Bounded backfill	Weekly	Backfill fails or cannot advance cursor

Freshness alerts never prove data loss. Completeness checks compare source counts, IDs, currencies, and monetary control totals by business date.

Payload handling¶

Payload bodies are encrypted at rest and addressed by opaque payload_ref; URLs, bucket names, and keys are not exposed to clients.
Logs contain IDs, hashes, sizes, statuses, and error codes, never raw payloads or secrets.
Customer and address fields are tokenized or omitted from analytical marts.
Retention is policy-driven by the classes in the data governance contract. The pilot default is 90 days for raw replay payloads and 30 days for processed webhook bodies; financial evidence has no automatic universal default and follows the approved tenant/jurisdiction policy.
Deletion requests create a legal-basis review. Records required for accounting evidence are restricted rather than blindly deleted.

Replay¶

Replay creates a new ingestion_run with mechanism manual_backfill or reconciliation and references the original payload. It may rerun canonicalization under a new adapter version, but it cannot reuse a prior posting idempotency key when the resulting posting intent differs. A dry-run diff is required before reprocessing records that already have confirmed Odoo mappings.