Skip to main content
If your inventory lives in S3, this recipe wires an agent to watch a bucket prefix, pull new files on a schedule, upload them through the standard inventory pipeline, and trigger a mapping job automatically.

​
Prerequisites

  • Pro Max workspace.
  • An S3 bucket with CSVs you control.
  • IAM access keys with s3:GetObject + s3:ListBucket on the bucket/prefix.

​
Steps

1

Create the agent

Name: “S3 inventory puller”. Goal:
Watch s3://acme-inventory/daily-uploads/. On every cron tick, pull new CSVs, upload them, and trigger a mapping job per file.
2

Add the inbound data source

Data sources → Add → S3 (inbound).
  • name: “Daily uploads”
  • bucket: acme-inventory
  • prefix: daily-uploads/
  • region: us-east-1
  • format: CSV
  • accessKeyId / secretAccessKey: paste IAM key pair (stored in vault, never echoed back).
  • scheduleCron: 0 */6 * * * (every 6 hours).
3

Test the connection

Click Test — we run a read-only ListObjectsV2 against the prefix and report what we’d pull on the next sync.
4

Set agent rules

5

Activate

The next cron tick triggers the first pull. Watch Runs for the cycle and Data sources → Syncs for the per-file manifest.

​
What gets stored

Per sync: agent_data_source_sync.manifest_json looks like:
The mapping job(s) are then triggered via InventoryUploadCompletedEvent, which the agent also subscribes to.

​
Equivalent API call

​
Failure handling

After 3 consecutive failed pulls (any cause — credential rotation, network blip, bucket permissions), the source is auto-paused, a DATA_SOURCE_PULL_FAILED pending item is filed, and configured channels are notified.