shell/immich/README.md

# Immich Management Scripts

This directory contains scripts for managing and backing up Immich photo management system.

## Scripts

### backup-immich.sh

Complete backup script for Immich installation that creates backups of:

- PostgreSQL database (using pg_dumpall as recommended by Immich)
- User upload directories (photos, videos, and metadata)

**Requirements:**

- `.env` file in the parent directory (`/home/acedanger/shell/.env`) with:
  - `DB_USERNAME` - PostgreSQL username
  - `DB_DATABASE_NAME` - Database name  
  - `UPLOAD_LOCATION` - Path to Immich upload directory
- Docker containers: `immich_postgres` and `immich_server`

**Usage:**

```bash
./backup-immich.sh [OPTIONS]
```

**Command-Line Options:**

- `--help, -h` - Show help message and exit
- `--dry-run` - Show what would be backed up without performing actual backup
- `--no-upload` - Skip B2 upload (local backup only)
- `--verbose` - Enable verbose logging

**Examples:**

```bash
# Standard backup (default behavior)
./backup-immich.sh

# Show help and usage information
./backup-immich.sh --help

# Preview what would be backed up without executing
./backup-immich.sh --dry-run

# Backup locally only (skip B2 upload)
./backup-immich.sh --no-upload

# Run with verbose logging
./backup-immich.sh --verbose

# Combine options
./backup-immich.sh --no-upload --verbose
```

**Backup Location:**

- Database: `../immich_backups/immich_db_backup_YYYYMMDD_HHMMSS.sql.gz`
- Uploads: `../immich_backups/immich_uploads_YYYYMMDD_HHMMSS.tar.gz`

**Features:**

- Command-line options for flexible operation (--help, --dry-run, --no-upload, --verbose)
- Dry-run mode to preview operations without executing
- Option to skip B2 upload for local-only backups
- Automatic container pausing/resuming during backup
- Comprehensive error handling and cleanup
- Backup validation and health checks
- Automatic compression
- Old backup cleanup (configurable)
- Centralized logging to `/home/acedanger/shell/logs/`
- Detailed progress reporting and timestamped logs
- 🔔 **Webhook notifications** to notify.peterwood.rocks/lab
- ☁️ **Backblaze B2 integration** for off-site backup storage
- 📊 **File size reporting** in notifications

## Usage Examples

### Basic Operations

**Standard Backup (Default)**

```bash
./backup-immich.sh
# Performs complete backup with all default settings:
# - Backs up database and upload directory
# - Uploads to B2 if configured
# - Sends webhook notifications
# - Logs to /home/acedanger/shell/logs/immich-backup.log
```

**Getting Help**

```bash
./backup-immich.sh --help
# Shows complete usage information including:
# - All available command-line options
# - Configuration requirements
# - Examples and restore instructions
```

### Preview and Testing

**Dry Run (Preview Mode)**

```bash
./backup-immich.sh --dry-run
# Shows what would be backed up without executing:
# - Checks all prerequisites and container status
# - Displays backup file paths and estimated sizes
# - Validates B2 configuration if present
# - Reports any issues that would prevent backup
# - No files are created or modified
```

Example dry-run output:

```text
=== DRY RUN MODE - NO ACTUAL BACKUP WILL BE PERFORMED ===

Configuration:
  - Database: immich
  - Username: postgres
  - Upload Location: /opt/immich/upload
  - Container: immich_postgres
  - Backup Directory: /home/acedanger/shell/immich_backups

Would create:
  - Database backup: /home/acedanger/shell/immich_backups/immich_db_backup_20250527_140000.sql.gz
  - Upload backup: /home/acedanger/shell/immich_backups/immich_uploads_20250527_140000.tar.gz

Container Status Check:
  ✓ immich_server: Running (would pause during backup)
  ✓ immich_postgres: Running
  ✓ Upload directory: /opt/immich/upload (42GB)

B2 Upload Configuration:
  ✓ B2 configured - would upload to bucket: my-immich-backups
  ✓ B2 CLI found at: /home/acedanger/shell/immich/b2-linux

=== DRY RUN COMPLETE - No files were created or modified ===
```

### Local Backup Only

**Skip B2 Upload**

```bash
./backup-immich.sh --no-upload
# Performs backup but skips B2 upload:
# - Creates local backup files
# - Validates backup integrity
# - Sends notifications (without B2 status)
# - Useful for testing or when B2 is unavailable
```

### Verbose Logging

**Detailed Output**

```bash
./backup-immich.sh --verbose
# Enables detailed logging for troubleshooting:
# - Shows additional progress information
# - Includes Docker command output
# - Provides more detailed error messages
# - Helpful for debugging issues
```

### Combined Options

**Local Backup with Verbose Output**

```bash
./backup-immich.sh --no-upload --verbose
# Combines multiple options:
# - Creates local backup only (no B2 upload)
# - Shows detailed progress and logging
# - Useful for testing or troubleshooting
```

**Preview with Verbose Details**

```bash
./backup-immich.sh --dry-run --verbose
# Shows detailed preview information:
# - Extended configuration validation
# - More detailed container status
# - Comprehensive B2 configuration check
# - Additional filesystem checks
```

### Automation Examples

**Scheduled Backup (Crontab)**

```bash
# Daily backup at 2:00 AM with logging
0 2 * * * /home/acedanger/shell/immich/backup-immich.sh >> /home/acedanger/shell/logs/immich-backup.log 2>&1

# Weekly local-only backup (no B2 upload) at 3:00 AM on Sundays
0 3 * * 0 /home/acedanger/shell/immich/backup-immich.sh --no-upload

# Daily validation run (dry-run) at 1:55 AM to check system health
55 1 * * * /home/acedanger/shell/immich/backup-immich.sh --dry-run >> /home/acedanger/shell/logs/immich-validation.log 2>&1
```

**Manual Backup Scripts**

```bash
#!/bin/bash
# emergency-backup.sh - Quick local backup without B2
echo "Starting emergency Immich backup..."
/home/acedanger/shell/immich/backup-immich.sh --no-upload --verbose

#!/bin/bash
# weekly-validation.sh - Comprehensive system check
echo "Validating Immich backup system..."
/home/acedanger/shell/immich/backup-immich.sh --dry-run --verbose
```

### Troubleshooting Examples

**Check System Status**

```bash
# Quick system validation without backup
./backup-immich.sh --dry-run

# If containers are not running:
docker ps | grep immich  # Check container status
docker start immich_server immich_postgres  # Start if needed

# If upload directory missing:
ls -la /opt/immich/upload  # Verify path exists
```

**Test B2 Configuration**

```bash
# Backup without B2 to test local functionality
./backup-immich.sh --no-upload

# Check B2 CLI manually
./b2-linux version  # Verify B2 CLI works
./b2-linux authorize-account YOUR_KEY_ID YOUR_KEY  # Test authorization
```

**Debug Backup Issues**

```bash
# Run with maximum detail for troubleshooting
./backup-immich.sh --verbose --no-upload

# Check logs for errors
tail -f /home/acedanger/shell/logs/immich-backup.log

# Validate backup files
ls -la /home/acedanger/shell/immich_backups/
```

## Configuration

The scripts expect a `.env` file in the parent directory with the following variables:

```bash
# Database configuration
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
UPLOAD_LOCATION=/path/to/immich/uploads

# Notification settings
WEBHOOK_URL="https://notify.peterwood.rocks/lab"

# Backblaze B2 settings (optional)
# Get these from your B2 account: https://secure.backblaze.com/app_keys.htm
# B2_APPLICATION_KEY_ID=your_key_id_here
# B2_APPLICATION_KEY=your_application_key_here
# B2_BUCKET_NAME=your_bucket_name_here

# Optional: Backup retention (days)
BACKUP_RETENTION_DAYS=30
```

## Backup Strategy

Based on Immich's official backup recommendations:

1. **Database Backup**: Uses `pg_dumpall` with `--clean` and `--if-exists` flags
2. **Upload Directory**: Complete archive of upload location including:
   - upload/ - Original photos and videos
   - profile/ - User profile images
   - thumbs/ - Generated thumbnails
   - encoded-video/ - Transcoded videos
   - library/ - Library metadata
   - backups/ - Existing backup files (excluded from new backups)

## Notifications 🔔

The backup script sends notifications to your webhook URL with:

- 🚀 **Start notification**: When backup begins
- ✅ **Success notification**: When backup completes successfully with file sizes
- ⚠️ **Warning notification**: When backup succeeds but B2 upload fails
- 🚨 **Error notification**: When backup fails

Example notification:

```text
📦 Database: immich_db_backup_20250526_215913.sql.gz (150MB)
📁 Uploads: immich_uploads_20250526_215913.tar.gz (25GB)
☁️ Successfully uploaded to B2 bucket: my-immich-backups
```

## Backblaze B2 Integration ☁️

### Setup B2 Account

1. Create a [Backblaze B2 account](https://www.backblaze.com/b2/cloud-storage.html)
2. Create a new bucket for Immich backups
3. Generate application keys:
   - Go to: <https://secure.backblaze.com/app_keys.htm>
   - Create new key with read/write access to your bucket

### Configure B2 in .env

Add these variables to your `.env` file:

```bash
B2_APPLICATION_KEY_ID=your_key_id_here
B2_APPLICATION_KEY=your_application_key_here
B2_BUCKET_NAME=your_bucket_name_here
```

### B2 Features

- **Automatic upload**: Backup files are uploaded to B2 after creation
- **Organized storage**: Files stored in `immich-backups/` folder in your bucket
- **Error handling**: Script continues if B2 upload fails (local backup preserved)
- **Progress tracking**: Upload status included in notifications

The B2 CLI tool (`b2-linux`) is included in this directory and doesn't require separate installation.

## Restore Process

For complete restore instructions, see: <https://immich.app/docs/administration/backup-and-restore/>

1. **Database Restore:**

   ```bash
   docker exec -i immich_postgres psql -U postgres < immich_db_backup.sql
   ```

2. **Upload Directory Restore:**

   ```bash
   tar -xzf immich_uploads_backup.tar.gz -C /target/location
   ```

## Logs

Backup logs and performance metrics are stored in the main shell logs directory (`/home/acedanger/shell/logs/`).

## Monitoring

The backup script includes:

- Progress indicators for long-running operations
- Size validation for backup files
- Container status monitoring
- Automatic cleanup procedures
- Comprehensive error reporting

## Automation

To automate backups, add to crontab:

```bash
# Daily Immich backup at 2:00 AM
0 2 * * * /home/acedanger/shell/immich/backup-immich.sh >> /home/acedanger/shell/logs/immich-backup.log 2>&1
```