Skip to main content
If your inventory lives in S3, this recipe wires an agent to watch a bucket prefix, pull new files on a schedule, upload them through the standard inventory pipeline, and trigger a mapping job automatically.

​
Prerequisites

  • Pro Max workspace.
  • An S3 bucket with CSVs you control.
  • IAM access keys with s3:GetObject + s3:ListBucket on the bucket/prefix.

​
Steps

1

Create the agent

Name: “S3 inventory puller”. Goal:
Watch s3://acme-inventory/daily-uploads/. On every cron tick, pull new CSVs, upload them, and trigger a mapping job per file.
2

Add the inbound data source

Data sources → Add → S3 (inbound).
  • name: “Daily uploads”
  • bucket: acme-inventory
  • prefix: daily-uploads/
  • region: us-east-1
  • format: CSV
  • accessKeyId / secretAccessKey: paste IAM key pair (stored in vault, never echoed back).
  • scheduleCron: 0 */6 * * * (every 6 hours).
3

Test the connection

Click Test — we run a read-only ListObjectsV2 against the prefix and report what we’d pull on the next sync.
4

Set agent rules

{
  "data_sources": {
    "auto_pull_on_schedule": true,
    "pause_source_after_failures": 3
  },
  "mapping_jobs": {
    "trigger_on_new_inventory_upload": true
  }
}
5

Activate

The next cron tick triggers the first pull. Watch Runs for the cycle and Data sources → Syncs for the per-file manifest.

​
What gets stored

Per sync: agent_data_source_sync.manifest_json looks like: 
[
  { "key": "daily-uploads/2026-05-25/booking.csv", "size": 2451231, "status": "uploaded", "inventoryId": "inv_01J..." },
  { "key": "daily-uploads/2026-05-25/expedia.csv", "size": 1782991, "status": "uploaded", "inventoryId": "inv_01J..." }
]
The mapping job(s) are then triggered via InventoryUploadCompletedEvent, which the agent also subscribes to.

​
Equivalent API call

curl -X POST https://api.mapping.travel/api/v1/agents/{id}/data-sources \
  -H 'Authorization: Bearer mt_...' \
  -H 'Content-Type: application/json' \
  -d '{
    "role": "INVENTORY_SOURCE",
    "type": "S3",
    "name": "Daily uploads",
    "config": { "bucket": "acme-inventory", "prefix": "daily-uploads/", "region": "us-east-1" },
    "secret": { "accessKeyId": "AKIA...", "secretAccessKey": "..." },
    "format": "CSV",
    "scheduleCron": "0 */6 * * *"
  }'

​
Failure handling

After 3 consecutive failed pulls (any cause — credential rotation, network blip, bucket permissions), the source is auto-paused, a DATA_SOURCE_PULL_FAILED pending item is filed, and configured channels are notified.