tigris-s3-migration

The full skill.

Original instructions from the publisher’s SKILL.md

# Migrate to Tigris from S3/GCS/Azure

Migrate your object storage to Tigris with zero downtime. Tigris is S3-compatible, so most apps need only an endpoint and credential swap. Shadow buckets enable transparent migration without moving data upfront.

## Prerequisites

**Before doing anything else**, install the Tigris CLI if it's not already available:

```bash
tigris help || npm install -g @tigrisdata/cli
```

If you need to install it, tell the user: "I'm installing the Tigris CLI (`@tigrisdata/cli`) so we can work with Tigris object storage."

## Migration Strategies

| Strategy             | Downtime | Best For                                                                |
| -------------------- | -------- | ----------------------------------------------------------------------- |
| **Shadow bucket**    | Zero     | Production apps — Tigris reads from S3 on miss, backfills automatically |
| **Bulk copy**        | Brief    | Small datasets, clean cutover                                           |
| **Incremental sync** | Zero     | Large datasets, gradual migration                                       |

---

## Shadow Bucket (Recommended)

Tigris reads from your existing S3 bucket on cache miss and gradually backfills data. No upfront data movement needed.

```bash
# Create a Tigris bucket
tigris buckets create my-app-uploads
# Point the Tigris bucket at the source S3 bucket
tigris buckets set-migration my-app-uploads \
  --bucket my-existing-s3-bucket \
  --endpoint https://s3.us-east-1.amazonaws.com \
  --region us-east-1 \
  --access-key AKIA_YOUR_AWS_KEY \
  --secret-key YOUR_AWS_SECRET
```

Add `--write-through` to also sync new writes back to the source bucket — useful when you want a safe rollback path during the migration window.

**How it works:**

1. Requests go to Tigris
2. If the object exists in Tigris, it's served directly
3. If not, Tigris fetches it from S3, serves it, and caches it

**Migrate everything up front (optional):**

Lazy migration only copies objects when they're requested. To actively migrate every remaining object server-side, run:

```bash
# Migrate every unmigrated object in the bucket
tigris buckets migrate my-app-uploads

# Or scope to a key prefix
tigris buckets migrate my-app-uploads/images/
```

The command runs in the foreground and reports progress as it goes, so you can fully drain the source before cutting over.

**After migration completes:**

```bash
# Disable migration (makes Tigris the sole source)
tigris buckets set-migration my-app-uploads --disable
```

---

## Bulk Copy

For smaller datasets or when you want a clean cutover:

### From Google Cloud Storage

```bash
gsutil -m cp -r gs://my-gcs-bucket /tmp/migration/
tigris cp /tmp/migration/ t3://my-app-uploads/ -r
```

### From Azure Blob Storage

```bash
az storage blob download-batch -d /tmp/migration/ -s my-container
tigris cp /tmp/migration/ t3://my-app-uploads/ -r
```

---

## SDK Code Changes

Read the resource file for your language to see before/after migration examples:

- **Node.js / TypeScript** — Read `./resources/sdk-nodejs.md` for AWS SDK → Tigris SDK migration
- **Go** — Read `./resources/sdk-go.md` for AWS SDK → Tigris SDK migration
- **Python** — Read `./resources/sdk-python.md` for boto3 → tigris-boto3-ext migration, or use the **tigris-python-sdk** skill for the post-migration developer experience (Django, snapshots, forks, Bundle API)
- **Ruby** — Read `./resources/sdk-ruby.md` for aws-sdk-s3 endpoint swap
- **PHP** — Read `./resources/sdk-php.md` for aws-sdk-php endpoint swap

---

## Environment Variable Changes

```bash
# Before (AWS)
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_REGION=us-east-1
S3_BUCKET=my-bucket

# After (Tigris)
AWS_ACCESS_KEY_ID=tid_xxx
AWS_SECRET_ACCESS_KEY=tsec_yyy
AWS_ENDPOINT_URL_S3=https://t3.storage.dev
AWS_REGION=auto
S3_BUCKET=my-bucket  # bucket name can stay the same
```

For frameworks with specific Tigris env vars:

```bash
TIGRIS_STORAGE_ACCESS_KEY_ID=tid_xxx
TIGRIS_STORAGE_SECRET_ACCESS_KEY=tsec_yyy
TIGRIS_STORAGE_ENDPOINT=https://t3.storage.dev
TIGRIS_STORAGE_BUCKET=my-bucket
```

---

## Verification Checklist

- [ ] Object count matches between source and Tigris
- [ ] Spot-check files: download a few and verify content/checksums
- [ ] Test presigned URL generation (endpoint must point to Tigris)
- [ ] Test CORS if using browser uploads (reconfigure on Tigris bucket)
- [ ] Test all upload/download code paths in staging
- [ ] Update CDN origin if using CloudFront/similar (point to Tigris)
- [ ] Update DNS if using custom domains

---

## Rollback Strategy

1. Keep source bucket read-only during migration (don't delete data)
2. Run both systems in parallel during verification
3. Only delete source data after confirming Tigris works fully
4. If using shadow bucket, removing the shadow source is the point of no return

---

## Common Mistakes

| Mistake                                 | Fix                                                                                |
| --------------------------------------- | ---------------------------------------------------------------------------------- |
| Forgot to update presigned URL endpoint | Presigned URLs must use Tigris endpoint, not S3                                    |
| CORS not configured on Tigris           | Re-create CORS rules: `tigris buckets cors set`                                    |
| Region hardcoded to `us-east-1`         | Use `auto` for Tigris                                                              |
| `path_style` not set                    | Add `force_path_style: true` (Ruby/Rails) or `use_path_style_endpoint: true` (PHP) |
| Custom domain DNS still points to S3    | Update CNAME to point to Tigris                                                    |

---

## Related Skills

- **file-storage** — CLI setup and SDK reference
- **tigris-security-access-control** — CORS and access key setup

## Official Documentation

- S3 Compatibility: https://www.tigrisdata.com/docs/sdks/s3/
- Shadow Buckets & migration: https://www.tigrisdata.com/docs/migration/
- CLI: `tigris buckets set-migration` — https://www.tigrisdata.com/docs/cli/buckets/set-migration
- CLI: `tigris buckets migrate` — https://www.tigrisdata.com/docs/cli/buckets/migrate