Amazon S3

Import files from S3 buckets, or query Parquet on S3 directly. Also works with S3-compatible storage such as MinIO, DigitalOcean Spaces and Backblaze B2.

---

Prerequisites

• Access Key ID and Secret Access Key with read access

• Bucket name and region

---

Setting up the connection

1. In the Import Manager, click the S3 card

2. Enter your connection details:

Field	Description	Example
Access Key ID	AWS access key	AKIAIOSFODNN7EXAMPLE
Secret Access Key	AWS secret key
Region	AWS region of the bucket	us-east-1
Default Bucket	Bucket to use for imports	my-data-bucket
Endpoint URL	S3 API endpoint, for S3-compatible storage	https://minio.example.com

Leave Endpoint URL empty for Amazon S3: the endpoint follows the region. Only S3-compatible storage needs one, pointing at your provider rather than AWS.

1. Click Test connection to verify, then Save & continue

---

Importing data

1. Browse to the file or folder under Select Bucket, Folder, and File. If browsing is unavailable, type the bucket and the object path, for example data/sales.parquet or data/2024/.

2. Enable Include files in sub-folders for nested directories.

3. Pick the Format: Auto-detect, CSV, Parquet or JSON.

Browsing a bucket down to a single Parquet file, with format auto-detection.

An import matching several files creates a folder source holding one child source per file, named after its path relative to the import path.

---

Import modes

Full Import, Full Import + Live, Recent Rows, Time Period and Query Directly. S3 cannot resolve rolling windows at the source, so a rolling window re-reads the path on each scheduled sync, and it needs a single object rather than a folder. Query Directly reads Parquet only. See Connectors.

---

IAM policy

Create an IAM policy with minimum required permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::your-bucket-name",
        "arn:aws:s3:::your-bucket-name/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "*"
    }
  ]
}

s3:ListAllMyBuckets is only needed if you browse for the bucket in RootCause rather than typing bucket and path by hand. It cannot be scoped to a bucket ARN, so it has to be a separate statement with "Resource": "*".

Attach this policy to a dedicated IAM user for RootCause.

---

Troubleshooting

Access denied

• Verify GetObject and ListBucket, and that no bucket policy denies access

No files found

• Check the path prefix and that the format matches the files

S3-compatible store fails

• Confirm the Endpoint URL points at the provider, not AWS