Image Migration to Supabase Storage #

This document describes the process of migrating images from the Git repository to Supabase Storage to reduce repository size and improve performance.

Overview #

Previous State:

Current State:

Netlify Build Caching #

To avoid re-downloading images on every Netlify build:

  1. Netlify Cache Configuration (netlify.toml):

    [[build.cache]]
  paths = ["img/"]

    This caches the /img directory between builds.

  2. Smart Download Script:

  3. Cache Invalidation:

Result: Images are downloaded once, then reused from cache on subsequent builds.

Adding New Images #

Automatic Upload (Recommended) #

New images are automatically uploaded to Supabase Storage via pre-push hook:

# 1. Add image to /img directory
cp new-image.jpg img/

# 2. Create content that references the image
echo '![Description](/img/new-image.jpg)' >> content/posts/my-post.md

# 3. Commit and push (upload happens automatically)
git add .
git commit -m "Add new post with image"
git push  # ← Image automatically uploaded to Supabase

How it works:

Branch-aware behavior:

# On feature branch
git push  # → Uploads to Preview Supabase

# On main branch
git push  # → Syncs Preview to Production, then uploads local changes

GitHub Actions Backup:

Requirements:

Manual Upload #

If you prefer manual control or the automatic upload fails:

# Upload to production
npm run migrate:images

# Upload to preview environment
npm run migrate:images:preview

# Dry-run to see what would be uploaded
npm run migrate:images:dry-run

Prerequisites #

  1. Supabase Service Role Key

  2. Dependencies

Migration Process #

Step 1: Test Migration (Dry Run) #

First, test the migration without actually uploading files:

# Test with preview branch
npm run migrate:images:dry-run -- --preview

# Or test with production
npm run migrate:images:dry-run

This will show:

Step 2: Upload Images to Supabase Storage #

Once you've verified the dry run, perform the actual upload:

# Upload to preview branch for testing
npm run migrate:images:preview

# Or upload to production
npm run migrate:images

The script will:

Step 3: Update Image URLs (Dry Run) #

Test URL replacement without modifying files:

# Test with preview branch
npm run migrate:update-urls:dry-run -- --preview

# Or test with production
npm run migrate:update-urls:dry-run

This will show:

Step 4: Update Image URLs in Content Files #

Perform the actual URL replacement:

# Update URLs for preview branch
npm run migrate:update-urls:preview

# Or update URLs for production
npm run migrate:update-urls

The script will:

Step 5: Test Locally #

Build and test the site with new image URLs:

npm run build
npm run serve

Verify:

Step 6: Deploy to Staging #

Push changes to staging branch for verification:

git add .
git commit -m "Migrate images to Supabase Storage"
git push origin feature/migrate-images-to-supabase

Create a pull request and deploy to staging environment to verify in production-like conditions.

Step 7: Remove Images from Git (After Verification) #

Once verified on staging:

# Remove /img directory
git rm -r img/

# Commit removal
git commit -m "Remove images from repository (now in Supabase Storage)"

# Add to .gitignore to prevent re-adding
echo "img/" >> .gitignore
git add .gitignore
git commit -m "Add img/ to gitignore"

Step 8: Clean Git History (Optional) #

To reduce repository size by removing images from Git history:

# Install BFG Repo-Cleaner
brew install bfg

# Create backup
git clone --mirror https://github.com/cadextcp/RetroComputerDD.git

# Remove /img folder from history
bfg --delete-folders img --no-blob-protection RetroComputerDD.git

# Clean up
cd RetroComputerDD.git
git reflog expire --expire=now --all
git gc --prune=now --aggressive

# Force push (coordinate with team!)
git push --force

⚠️ Warning: Force pushing rewrites history. Coordinate with all team members before doing this.

Supabase Storage Structure #

Bucket Configuration:

Folder Structure:
The migration preserves the original folder structure:

images/
├── favicon/
│   ├── favicon.ico
│   └── ...
├── arcadeBuild/
│   ├── IMG_0130.JPG
│   └── ...
├── retroparty2024/
├── petRepair/
└── ...

URL Format #

Before:

![Alt text](/img/arcadeBuild/IMG_2400.JPG)

After:

![Alt text](/img/arcadeBuild/IMG_2400.JPG)

Rollback Plan #

If issues occur after migration:

  1. Revert URL changes:

    git revert <commit-hash>

  2. Images are still in Git history (until Step 8), so they can be restored:

    git checkout <previous-commit> -- img/

  3. Supabase Storage files remain and can be deleted if needed via Supabase Dashboard

Benefits #

Troubleshooting #

Upload fails with "Bucket not found" #

Ensure the images bucket exists in Supabase Storage. Create it via:

"Missing SUPABASE_SERVICE_ROLE_KEY" #

Set the service role key in your environment:

# For preview
echo "SUPABASE_SERVICE_ROLE_KEY_PREVIEW=your_key" >> .env.preview

# For production
echo "SUPABASE_SERVICE_ROLE_KEY=your_key" >> .env

Images don't load after migration #

Check:

  1. Bucket is public in Supabase Dashboard
  2. URLs are correctly formatted
  3. Files were successfully uploaded (check Supabase Storage)
  4. No CORS issues (Supabase Storage should handle this automatically)

Some images still reference /img/ #

Run the update script again:

npm run migrate:update-urls:dry-run

Check which files weren't updated and why.

Notes #