peterwood/shell
1
0
Fork 0
mirror of https://github.com/acedanger/shell.git synced 2025-12-05 22:50:18 -08:00
Code Issues 19 Packages Projects Releases Wiki Activity

Commit local changes before merging with remote

Browse Source
This commit is contained in:
Peter Wood
2025-05-29 11:25:02 -04:00
parent 868b340fb5
commit be4f6a8d8c
75 changed files with 14107 additions and 562 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
immich/.prompts/immich-backup.prompt.md Normal file
View File

@@ -0,0 +1,19 @@
# Requirements
Immich backup script. This script should be run daily and should backup the following immich objects. Based on recommendations from https://immich.app/docs/administration/backup-and-restore/
1. Postgres database (using pg_dumpall as recommended by Immich)
2. User upload directories (photos, videos, and metadata)
## Notification
- Send a notification using the curl command to the <https://notify.peterwood.rocks/lab> when the backup starts and when it completes.
- Add the file names and sizes to the message.
- Ensure that the timestamps aren't appended.
- Emojis are fun in those kinds of notifications - success, failure, or warning.
- Follow the same methodology as is present in the plex backup script `plex/backup-plex.sh`.
## Backblaze
- Backblaze B2 bucket for off-site storage - once the archived file is created, I want to send it transfer it to a backblaze b2 bucket.
- Backblaze has a CLI app available for download here <https://github.com/Backblaze/B2_Command_Line_Tool/releases/download/v4.3.2/b2-linux>.
- Download it if it does not exist already and place it in the immich directory.
- Their quickstart guide is here <https://b2-command-line-tool.readthedocs.io/en/master/quick_start.html>

382
immich/README.md Normal file
View File

@@ -0,0 +1,382 @@
# Immich Management Scripts
This directory contains scripts for managing and backing up Immich photo management system.
## Scripts
### backup-immich.sh
Complete backup script for Immich installation that creates backups of:
- PostgreSQL database (using pg_dumpall as recommended by Immich)
- User upload directories (photos, videos, and metadata)
**Requirements:**
- `.env` file in the parent directory (`/home/acedanger/shell/.env`) with:
- `DB_USERNAME` - PostgreSQL username
- `DB_DATABASE_NAME` - Database name
- `UPLOAD_LOCATION` - Path to Immich upload directory
- Docker containers: `immich_postgres` and `immich_server`
**Usage:**
```bash
./backup-immich.sh [OPTIONS]
```
**Command-Line Options:**
- `--help, -h` - Show help message and exit
- `--dry-run` - Show what would be backed up without performing actual backup
- `--no-upload` - Skip B2 upload (local backup only)
- `--verbose` - Enable verbose logging
**Examples:**
```bash
# Standard backup (default behavior)
./backup-immich.sh
# Show help and usage information
./backup-immich.sh --help
# Preview what would be backed up without executing
./backup-immich.sh --dry-run
# Backup locally only (skip B2 upload)
./backup-immich.sh --no-upload
# Run with verbose logging
./backup-immich.sh --verbose
# Combine options
./backup-immich.sh --no-upload --verbose
```
**Backup Location:**
- Database: `../immich_backups/immich_db_backup_YYYYMMDD_HHMMSS.sql.gz`
- Uploads: `../immich_backups/immich_uploads_YYYYMMDD_HHMMSS.tar.gz`
**Features:**
- Command-line options for flexible operation (--help, --dry-run, --no-upload, --verbose)
- Dry-run mode to preview operations without executing
- Option to skip B2 upload for local-only backups
- Automatic container pausing/resuming during backup
- Comprehensive error handling and cleanup
- Backup validation and health checks
- Automatic compression
- Old backup cleanup (configurable)
- Centralized logging to `/home/acedanger/shell/logs/`
- Detailed progress reporting and timestamped logs
- 🔔 **Webhook notifications** to notify.peterwood.rocks/lab
- ☁️ **Backblaze B2 integration** for off-site backup storage
- 📊 **File size reporting** in notifications
## Usage Examples
### Basic Operations
**Standard Backup (Default)**
```bash
./backup-immich.sh
# Performs complete backup with all default settings:
# - Backs up database and upload directory
# - Uploads to B2 if configured
# - Sends webhook notifications
# - Logs to /home/acedanger/shell/logs/immich-backup.log
```
**Getting Help**
```bash
./backup-immich.sh --help
# Shows complete usage information including:
# - All available command-line options
# - Configuration requirements
# - Examples and restore instructions
```
### Preview and Testing
**Dry Run (Preview Mode)**
```bash
./backup-immich.sh --dry-run
# Shows what would be backed up without executing:
# - Checks all prerequisites and container status
# - Displays backup file paths and estimated sizes
# - Validates B2 configuration if present
# - Reports any issues that would prevent backup
# - No files are created or modified
```
Example dry-run output:
```text
=== DRY RUN MODE - NO ACTUAL BACKUP WILL BE PERFORMED ===
Configuration:
- Database: immich
- Username: postgres
- Upload Location: /opt/immich/upload
- Container: immich_postgres
- Backup Directory: /home/acedanger/shell/immich_backups
Would create:
- Database backup: /home/acedanger/shell/immich_backups/immich_db_backup_20250527_140000.sql.gz
- Upload backup: /home/acedanger/shell/immich_backups/immich_uploads_20250527_140000.tar.gz
Container Status Check:
✓ immich_server: Running (would pause during backup)
✓ immich_postgres: Running
✓ Upload directory: /opt/immich/upload (42GB)
B2 Upload Configuration:
✓ B2 configured - would upload to bucket: my-immich-backups
✓ B2 CLI found at: /home/acedanger/shell/immich/b2-linux
=== DRY RUN COMPLETE - No files were created or modified ===
```
### Local Backup Only
**Skip B2 Upload**
```bash
./backup-immich.sh --no-upload
# Performs backup but skips B2 upload:
# - Creates local backup files
# - Validates backup integrity
# - Sends notifications (without B2 status)
# - Useful for testing or when B2 is unavailable
```
### Verbose Logging
**Detailed Output**
```bash
./backup-immich.sh --verbose
# Enables detailed logging for troubleshooting:
# - Shows additional progress information
# - Includes Docker command output
# - Provides more detailed error messages
# - Helpful for debugging issues
```
### Combined Options
**Local Backup with Verbose Output**
```bash
./backup-immich.sh --no-upload --verbose
# Combines multiple options:
# - Creates local backup only (no B2 upload)
# - Shows detailed progress and logging
# - Useful for testing or troubleshooting
```
**Preview with Verbose Details**
```bash
./backup-immich.sh --dry-run --verbose
# Shows detailed preview information:
# - Extended configuration validation
# - More detailed container status
# - Comprehensive B2 configuration check
# - Additional filesystem checks
```
### Automation Examples
**Scheduled Backup (Crontab)**
```bash
# Daily backup at 2:00 AM with logging
0 2 * * * /home/acedanger/shell/immich/backup-immich.sh >> /home/acedanger/shell/logs/immich-backup.log 2>&1
# Weekly local-only backup (no B2 upload) at 3:00 AM on Sundays
0 3 * * 0 /home/acedanger/shell/immich/backup-immich.sh --no-upload
# Daily validation run (dry-run) at 1:55 AM to check system health
55 1 * * * /home/acedanger/shell/immich/backup-immich.sh --dry-run >> /home/acedanger/shell/logs/immich-validation.log 2>&1
```
**Manual Backup Scripts**
```bash
#!/bin/bash
# emergency-backup.sh - Quick local backup without B2
echo "Starting emergency Immich backup..."
/home/acedanger/shell/immich/backup-immich.sh --no-upload --verbose
#!/bin/bash
# weekly-validation.sh - Comprehensive system check
echo "Validating Immich backup system..."
/home/acedanger/shell/immich/backup-immich.sh --dry-run --verbose
```
### Troubleshooting Examples
**Check System Status**
```bash
# Quick system validation without backup
./backup-immich.sh --dry-run
# If containers are not running:
docker ps | grep immich # Check container status
docker start immich_server immich_postgres # Start if needed
# If upload directory missing:
ls -la /opt/immich/upload # Verify path exists
```
**Test B2 Configuration**
```bash
# Backup without B2 to test local functionality
./backup-immich.sh --no-upload
# Check B2 CLI manually
./b2-linux version # Verify B2 CLI works
./b2-linux authorize-account YOUR_KEY_ID YOUR_KEY # Test authorization
```
**Debug Backup Issues**
```bash
# Run with maximum detail for troubleshooting
./backup-immich.sh --verbose --no-upload
# Check logs for errors
tail -f /home/acedanger/shell/logs/immich-backup.log
# Validate backup files
ls -la /home/acedanger/shell/immich_backups/
```
## Configuration
The scripts expect a `.env` file in the parent directory with the following variables:
```bash
# Database configuration
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
UPLOAD_LOCATION=/path/to/immich/uploads
# Notification settings
WEBHOOK_URL="https://notify.peterwood.rocks/lab"
# Backblaze B2 settings (optional)
# Get these from your B2 account: https://secure.backblaze.com/app_keys.htm
# B2_APPLICATION_KEY_ID=your_key_id_here
# B2_APPLICATION_KEY=your_application_key_here
# B2_BUCKET_NAME=your_bucket_name_here
# Optional: Backup retention (days)
BACKUP_RETENTION_DAYS=30
```
## Backup Strategy
Based on Immich's official backup recommendations:
1. **Database Backup**: Uses `pg_dumpall` with `--clean` and `--if-exists` flags
2. **Upload Directory**: Complete archive of upload location including:
- upload/ - Original photos and videos
- profile/ - User profile images
- thumbs/ - Generated thumbnails
- encoded-video/ - Transcoded videos
- library/ - Library metadata
- backups/ - Existing backup files (excluded from new backups)
## Notifications 🔔
The backup script sends notifications to your webhook URL with:
- 🚀 **Start notification**: When backup begins
- ✅ **Success notification**: When backup completes successfully with file sizes
- ⚠️ **Warning notification**: When backup succeeds but B2 upload fails
- 🚨 **Error notification**: When backup fails
Example notification:
```text
📦 Database: immich_db_backup_20250526_215913.sql.gz (150MB)
📁 Uploads: immich_uploads_20250526_215913.tar.gz (25GB)
☁️ Successfully uploaded to B2 bucket: my-immich-backups
```
## Backblaze B2 Integration ☁️
### Setup B2 Account
1. Create a [Backblaze B2 account](https://www.backblaze.com/b2/cloud-storage.html)
2. Create a new bucket for Immich backups
3. Generate application keys:
- Go to: <https://secure.backblaze.com/app_keys.htm>
- Create new key with read/write access to your bucket
### Configure B2 in .env
Add these variables to your `.env` file:
```bash
B2_APPLICATION_KEY_ID=your_key_id_here
B2_APPLICATION_KEY=your_application_key_here
B2_BUCKET_NAME=your_bucket_name_here
```
### B2 Features
- **Automatic upload**: Backup files are uploaded to B2 after creation
- **Organized storage**: Files stored in `immich-backups/` folder in your bucket
- **Error handling**: Script continues if B2 upload fails (local backup preserved)
- **Progress tracking**: Upload status included in notifications
The B2 CLI tool (`b2-linux`) is included in this directory and doesn't require separate installation.
## Restore Process
For complete restore instructions, see: <https://immich.app/docs/administration/backup-and-restore/>
1. **Database Restore:**
```bash
docker exec -i immich_postgres psql -U postgres < immich_db_backup.sql
```
2. **Upload Directory Restore:**
```bash
tar -xzf immich_uploads_backup.tar.gz -C /target/location
```
## Logs
Backup logs and performance metrics are stored in the main shell logs directory (`/home/acedanger/shell/logs/`).
## Monitoring
The backup script includes:
- Progress indicators for long-running operations
- Size validation for backup files
- Container status monitoring
- Automatic cleanup procedures
- Comprehensive error reporting
## Automation
To automate backups, add to crontab:
```bash
# Daily Immich backup at 2:00 AM
0 2 * * * /home/acedanger/shell/immich/backup-immich.sh >> /home/acedanger/shell/logs/immich-backup.log 2>&1
```

510
immich/backup-immich.sh Executable file
View File

@@ -0,0 +1,510 @@
#!/bin/bash
# Immich Complete Backup Script
# This script creates a complete backup of the Immich installation including:
# 1. Postgres database (using pg_dumpall as recommended by Immich)
# 2. User upload directories (photos, videos, and metadata)
# Based on recommendations from https://immich.app/docs/administration/backup-and-restore/
# Set up error handling
set -e
# Function to ensure server is unpaused even if script fails
cleanup() {
local exit_code=$?
echo "Running cleanup..."
# Check if immich_server is paused and unpause it if needed
if [ "${IMMICH_SERVER_RUNNING:-true}" = true ] && docker inspect --format='{{.State.Status}}' immich_server 2>/dev/null | grep -q "paused"; then
echo "Unpausing immich_server container during cleanup..."
docker unpause immich_server 2>/dev/null || true
fi
if [ $exit_code -ne 0 ]; then
echo "Script failed with exit code $exit_code"
send_notification "🚨 Immich Backup Failed" "Backup process encountered an error (exit code: $exit_code)" "error"
fi
exit $exit_code
}
# Set up trap to call cleanup function on script exit (normal or error)
trap cleanup EXIT SIGINT SIGTERM
# Load environment variables from the .env file
ENV_FILE="$(dirname "$0")/../.env"
if [ -f "$ENV_FILE" ]; then
echo "Loading environment variables from $ENV_FILE"
source "$ENV_FILE"
else
echo "Error: .env file not found in $(dirname "$0")/.."
exit 1
fi
# Verify required environment variables are set
if [ -z "$DB_USERNAME" ] || [ -z "$DB_DATABASE_NAME" ] || [ -z "$UPLOAD_LOCATION" ]; then
echo "Error: Required environment variables (DB_USERNAME, DB_DATABASE_NAME, UPLOAD_LOCATION) not found in .env file"
echo "Please ensure your .env file contains:"
echo " - DB_USERNAME=<database_username>"
echo " - DB_DATABASE_NAME=<database_name>"
echo " - UPLOAD_LOCATION=<path_to_upload_directory>"
exit 1
fi
# Help function
show_help() {
cat << EOF
Immich Complete Backup Script
This script creates a complete backup of the Immich installation including:
1. Postgres database (using pg_dumpall as recommended by Immich)
2. User upload directories (photos, videos, and metadata)
USAGE:
$(basename "$0") [OPTIONS]
OPTIONS:
--help, -h Show this help message and exit
--dry-run Show what would be backed up without performing actual backup
--no-upload Skip B2 upload (local backup only)
--verbose Enable verbose logging
CONFIGURATION:
This script requires a .env file in the parent directory with:
- DB_USERNAME=<database_username>
- DB_DATABASE_NAME=<database_name>
- UPLOAD_LOCATION=<path_to_upload_directory>
OPTIONAL B2 CONFIGURATION:
- B2_APPLICATION_KEY_ID=<your_b2_app_key_id>
- B2_APPLICATION_KEY=<your_b2_app_key>
- B2_BUCKET_NAME=<your_b2_bucket_name>
OPTIONAL WEBHOOK CONFIGURATION:
- WEBHOOK_URL=<your_notification_webhook_url>
EXAMPLES:
$(basename "$0") # Run full backup
$(basename "$0") --help # Show this help
$(basename "$0") --dry-run # Preview backup without executing
$(basename "$0") --no-upload # Backup locally only (skip B2)
RESTORE INSTRUCTIONS:
https://immich.app/docs/administration/backup-and-restore/
EOF
}
# Parse command line arguments
DRY_RUN=false
NO_UPLOAD=false
VERBOSE=false
while [[ $# -gt 0 ]]; do
case $1 in
--help|-h)
show_help
exit 0
;;
--dry-run)
DRY_RUN=true
shift
;;
--no-upload)
NO_UPLOAD=true
shift
;;
--verbose)
VERBOSE=true
shift
;;
*)
echo "Error: Unknown option $1"
echo "Use --help for usage information"
exit 1
;;
esac
done
# B2 CLI tool path
B2_CLI="$(dirname "$0")/b2-linux"
# Notification function
send_notification() {
local title="$1"
local message="$2"
local status="${3:-info}" # success, error, warning, info
local hostname=$(hostname)
# Console notification
log_message "$title: $message"
# Webhook notification
if [ -n "$WEBHOOK_URL" ]; then
local tags="backup,immich,${hostname}"
[ "$status" == "error" ] && tags="${tags},errors"
[ "$status" == "warning" ] && tags="${tags},warnings"
# Clean message without newlines or timestamps for webhook
local webhook_message="$message"
curl -s \
-H "tags:${tags}" \
-d "$webhook_message" \
"$WEBHOOK_URL" 2>/dev/null || log_message "Warning: Failed to send webhook notification"
fi
}
# Function to upload to Backblaze B2
upload_to_b2() {
local file_path="$1"
local filename=$(basename "$file_path")
# Check if B2 is configured
if [ -z "$B2_APPLICATION_KEY_ID" ] || [ -z "$B2_APPLICATION_KEY" ] || [ -z "$B2_BUCKET_NAME" ]; then
log_message "B2 upload skipped: B2 credentials not configured in .env file"
return 0
fi
# Check if B2 CLI exists
if [ ! -f "$B2_CLI" ]; then
log_message "Error: B2 CLI not found at $B2_CLI"
return 1
fi
log_message "Uploading $filename to B2 bucket: $B2_BUCKET_NAME"
# Authorize B2 account
if ! "$B2_CLI" authorize-account "$B2_APPLICATION_KEY_ID" "$B2_APPLICATION_KEY" 2>/dev/null; then
log_message "Error: Failed to authorize B2 account"
return 1
fi
# Upload file to B2
if "$B2_CLI" upload-file "$B2_BUCKET_NAME" "$file_path" "immich-backups/$filename" 2>/dev/null; then
log_message "✅ Successfully uploaded $filename to B2"
return 0
else
log_message "❌ Failed to upload $filename to B2"
return 1
fi
}
# Initialize container status variables
IMMICH_SERVER_RUNNING=true
# Set up logging to central logs directory
LOG_DIR="$(dirname "$0")/../logs"
mkdir -p "$LOG_DIR"
LOG_FILE="${LOG_DIR}/immich-backup.log"
# Function to log with timestamp
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}
# Function to log without timestamp (for progress/status)
log_status() {
echo "$1" | tee -a "$LOG_FILE"
}
# Create backup directory if it doesn't exist
BACKUP_DIR="$(dirname "$0")/../immich_backups"
mkdir -p "$BACKUP_DIR"
# Generate timestamp for the backup filename
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
DB_BACKUP_FILENAME="immich_db_backup_${TIMESTAMP}.sql"
DB_BACKUP_PATH="${BACKUP_DIR}/${DB_BACKUP_FILENAME}"
UPLOAD_BACKUP_PATH="${BACKUP_DIR}/immich_uploads_${TIMESTAMP}.tar.gz"
# Handle dry-run mode
if [ "$DRY_RUN" = true ]; then
echo ""
echo "=== DRY RUN MODE - NO ACTUAL BACKUP WILL BE PERFORMED ==="
echo ""
echo "Configuration:"
echo " - Database: ${DB_DATABASE_NAME}"
echo " - Username: ${DB_USERNAME}"
echo " - Upload Location: ${UPLOAD_LOCATION}"
echo " - Container: immich_postgres"
echo " - Backup Directory: ${BACKUP_DIR}"
echo ""
echo "Would create:"
echo " - Database backup: ${DB_BACKUP_PATH}.gz"
echo " - Upload backup: ${UPLOAD_BACKUP_PATH}"
echo ""
# Check container status in dry-run
echo "Container Status Check:"
if docker ps -q --filter "name=immich_server" | grep -q .; then
echo " ✓ immich_server: Running (would pause during backup)"
else
echo " ! immich_server: Not running or not found"
fi
if docker ps -q --filter "name=immich_postgres" | grep -q .; then
echo " ✓ immich_postgres: Running"
else
echo " ✗ immich_postgres: Not running - backup would fail!"
exit 1
fi
# Check upload directory
if [ -d "${UPLOAD_LOCATION}" ]; then
UPLOAD_SIZE=$(du -sh "${UPLOAD_LOCATION}" 2>/dev/null | cut -f1 || echo "unknown")
echo " ✓ Upload directory: ${UPLOAD_LOCATION} (${UPLOAD_SIZE})"
else
echo " ✗ Upload directory: ${UPLOAD_LOCATION} does not exist - backup would fail!"
exit 1
fi
# Check B2 configuration
echo ""
echo "B2 Upload Configuration:"
if [ "$NO_UPLOAD" = true ]; then
echo " ! B2 upload disabled by --no-upload flag"
elif [ -n "$B2_APPLICATION_KEY_ID" ] && [ -n "$B2_APPLICATION_KEY" ] && [ -n "$B2_BUCKET_NAME" ]; then
echo " ✓ B2 configured - would upload to bucket: ${B2_BUCKET_NAME}"
if [ -f "$B2_CLI" ]; then
echo " ✓ B2 CLI found at: ${B2_CLI}"
else
echo " ✗ B2 CLI not found at: ${B2_CLI} - upload would fail!"
fi
else
echo " ! B2 not configured - would skip upload"
fi
echo ""
echo "=== DRY RUN COMPLETE - No files were created or modified ==="
exit 0
fi
log_message "Starting complete backup of Immich installation..."
log_message "Using settings from .env file:"
log_message " - Database: ${DB_DATABASE_NAME}"
log_message " - Username: ${DB_USERNAME}"
log_message " - Upload Location: ${UPLOAD_LOCATION}"
log_message " - Container: immich_postgres"
log_message " - Backup Directory: ${BACKUP_DIR}"
if [ "$NO_UPLOAD" = true ]; then
log_message " - B2 Upload: DISABLED (--no-upload flag)"
fi
if [ "$VERBOSE" = true ]; then
log_message " - Verbose logging: ENABLED"
fi
# Send start notification
send_notification "🚀 Immich Backup Started" "Starting complete backup of Immich database and uploads directory" "info"
# Check if the Immich server container exists and is running
log_status "Checking immich_server container status..."
if docker ps -q --filter "name=immich_server" | grep -q .; then
log_message "Pausing immich_server container to minimize database writes..."
if ! docker pause immich_server; then
log_message "Failed to pause immich_server container."
# Continue with backup instead of exiting
fi
else
log_message "Note: immich_server container not found or not running. Continuing with backup anyway."
# Set a flag so we don't try to unpause it later
IMMICH_SERVER_RUNNING=false
fi
# Check if the Postgres container exists and is running
log_status "Checking postgres container status..."
if ! docker ps -q --filter "name=immich_postgres" | grep -q .; then
log_message "Error: immich_postgres container is not running. Cannot proceed with backup."
exit 1
fi
# Check if the Immich server container exists and is running
log_status "Checking immich_server container status..."
if docker ps -q --filter "name=immich_server" | grep -q .; then
log_message "Pausing immich_server container to minimize database writes during backup..."
if ! docker pause immich_server; then
log_message "Failed to pause immich_server container."
# Continue with backup instead of exiting
fi
else
log_message "Note: immich_server container not found or not running. Continuing with backup anyway."
# Set a flag so we don't try to unpause it later
IMMICH_SERVER_RUNNING=false
fi
echo ""
echo "=== PHASE 1: DATABASE BACKUP ==="
log_message "Taking database backup using pg_dumpall as recommended by Immich documentation..."
# Use pg_dumpall with recommended flags: --clean and --if-exists
docker exec -t immich_postgres pg_dumpall \
--clean \
--if-exists \
--username="${DB_USERNAME}" \
> "${DB_BACKUP_PATH}"
# Check if the dump was successful
if [ $? -ne 0 ] || [ ! -s "${DB_BACKUP_PATH}" ]; then
log_message "Error: Database backup failed or created an empty file."
exit 1
fi
log_message "Database backup completed successfully!"
# Compress the database backup file
log_message "Compressing database backup file..."
if ! gzip -f "${DB_BACKUP_PATH}"; then
log_message "Warning: Failed to compress database backup file."
fi
echo ""
echo "=== PHASE 2: UPLOAD DIRECTORY BACKUP ==="
log_message "Backing up user upload directory: ${UPLOAD_LOCATION}"
# Verify the upload location exists
if [ ! -d "${UPLOAD_LOCATION}" ]; then
log_message "Error: Upload location ${UPLOAD_LOCATION} does not exist!"
exit 1
fi
# Create compressed archive of the upload directory
# According to Immich docs, we need to backup the entire UPLOAD_LOCATION
# which includes: upload/, profile/, thumbs/, encoded-video/, library/, backups/
log_message "Creating compressed archive of upload directory..."
log_message "This may take a while depending on the size of your media library..."
# Use tar with progress indication and exclude any existing backup files in the upload location
if ! tar --exclude="${UPLOAD_LOCATION}/backups/*.tar.gz" \
--exclude="${UPLOAD_LOCATION}/backups/*.sql.gz" \
-czf "${UPLOAD_BACKUP_PATH}" \
-C "$(dirname "${UPLOAD_LOCATION}")" \
"$(basename "${UPLOAD_LOCATION}")"; then
log_message "Error: Failed to create upload directory backup."
exit 1
fi
log_message "Upload directory backup completed successfully!"
# Resume the Immich server only if it was running and we paused it
if [ "${IMMICH_SERVER_RUNNING:-true}" = true ]; then
log_status "Resuming immich_server container..."
if ! docker unpause immich_server 2>/dev/null; then
log_message "Note: No need to unpause immich_server container."
fi
fi
# Resume the Immich server only if it was running and we paused it
if [ "${IMMICH_SERVER_RUNNING:-true}" = true ]; then
log_status "Resuming immich_server container..."
if ! docker unpause immich_server 2>/dev/null; then
log_message "Note: No need to unpause immich_server container."
fi
fi
echo ""
echo "=== BACKUP COMPLETED SUCCESSFULLY! ==="
echo "Database backup saved to: ${DB_BACKUP_PATH}.gz"
echo "Upload directory backup saved to: ${UPLOAD_BACKUP_PATH}"
# Calculate backup sizes
DB_BACKUP_SIZE=$(du -h "${DB_BACKUP_PATH}.gz" 2>/dev/null | cut -f1 || echo "Unknown")
UPLOAD_BACKUP_SIZE=$(du -h "${UPLOAD_BACKUP_PATH}" 2>/dev/null | cut -f1 || echo "Unknown")
echo ""
echo "=== BACKUP SUMMARY ==="
echo "Database backup size: ${DB_BACKUP_SIZE}"
echo "Upload directory backup size: ${UPLOAD_BACKUP_SIZE}"
# Upload to B2 (if configured and not disabled)
echo ""
if [ "$NO_UPLOAD" = true ]; then
echo "=== SKIPPING B2 UPLOAD (--no-upload flag) ==="
log_message "B2 upload skipped due to --no-upload flag"
B2_UPLOAD_SUCCESS="skipped"
else
echo "=== UPLOADING TO BACKBLAZE B2 ==="
B2_UPLOAD_SUCCESS=true
# Upload database backup
if ! upload_to_b2 "${DB_BACKUP_PATH}.gz"; then
B2_UPLOAD_SUCCESS=false
fi
# Upload uploads backup
if ! upload_to_b2 "${UPLOAD_BACKUP_PATH}"; then
B2_UPLOAD_SUCCESS=false
fi
fi
# Prepare notification message
DB_FILENAME=$(basename "${DB_BACKUP_PATH}.gz")
UPLOAD_FILENAME=$(basename "${UPLOAD_BACKUP_PATH}")
NOTIFICATION_MESSAGE="📦 Database: ${DB_FILENAME} (${DB_BACKUP_SIZE})
📁 Uploads: ${UPLOAD_FILENAME} (${UPLOAD_BACKUP_SIZE})"
if [ "$B2_UPLOAD_SUCCESS" = "skipped" ]; then
NOTIFICATION_MESSAGE="${NOTIFICATION_MESSAGE}
💾 Local backup only (B2 upload skipped)"
send_notification "✅ Immich Backup Completed (Local Only)" "$NOTIFICATION_MESSAGE" "success"
elif [ "$B2_UPLOAD_SUCCESS" = true ] && [ -n "$B2_BUCKET_NAME" ]; then
NOTIFICATION_MESSAGE="${NOTIFICATION_MESSAGE}
☁️ Successfully uploaded to B2 bucket: ${B2_BUCKET_NAME}"
send_notification "✅ Immich Backup Completed" "$NOTIFICATION_MESSAGE" "success"
elif [ -n "$B2_BUCKET_NAME" ]; then
NOTIFICATION_MESSAGE="${NOTIFICATION_MESSAGE}
⚠️ B2 upload failed - files saved locally only"
send_notification "⚠️ Immich Backup Completed (B2 Upload Failed)" "$NOTIFICATION_MESSAGE" "warning"
else
send_notification "✅ Immich Backup Completed" "$NOTIFICATION_MESSAGE" "success"
fi
# Show backup information
echo ""
echo "=== BACKUP INVENTORY ==="
find "${BACKUP_DIR}" -name "*.gz" | wc -l | xargs echo "Total number of backup files:"
du -sh "${BACKUP_DIR}" | cut -f1 | xargs echo "Total backup directory size:"
# List recent backups
echo ""
echo "Recent backups:"
find "${BACKUP_DIR}" -name "*.gz" -mtime -7 | sort
# Health check: Verify backup file sizes
DB_BACKUP_SIZE_KB=$(du -k "${DB_BACKUP_PATH}.gz" 2>/dev/null | cut -f1 || echo "0")
UPLOAD_BACKUP_SIZE_KB=$(du -k "${UPLOAD_BACKUP_PATH}" 2>/dev/null | cut -f1 || echo "0")
echo ""
echo "=== BACKUP VALIDATION ==="
if [ "${DB_BACKUP_SIZE_KB}" -lt 100 ]; then
echo "WARNING: Database backup file is smaller than expected (${DB_BACKUP_SIZE_KB}KB). Please verify its integrity."
else
echo "✓ Database backup size appears normal (${DB_BACKUP_SIZE_KB}KB)"
fi
if [ "${UPLOAD_BACKUP_SIZE_KB}" -lt 1024 ]; then
echo "WARNING: Upload directory backup file is smaller than expected (${UPLOAD_BACKUP_SIZE_KB}KB). Please verify its integrity."
else
echo "✓ Upload directory backup size appears normal (${UPLOAD_BACKUP_SIZE_KB}KB)"
fi
# Optional: Remove old backups (older than 30 days)
echo ""
echo "=== CLEANUP ==="
OLD_BACKUPS=$(find "${BACKUP_DIR}" -name "*.gz" -mtime +30 | wc -l)
if [ "${OLD_BACKUPS}" -gt 0 ]; then
echo "Found ${OLD_BACKUPS} backup files older than 30 days."
echo "To remove them automatically, uncomment the cleanup line in this script."
# Uncomment the next line to automatically remove old backups
# find "${BACKUP_DIR}" -name "*.gz" -mtime +30 -delete
else
echo "No old backup files found (older than 30 days)."
fi
echo ""
echo "=== RESTORE INSTRUCTIONS ==="
echo "To restore from this backup:"
echo "1. Database restore instructions: https://immich.app/docs/administration/backup-and-restore/#database"
echo "2. Upload directory: Extract ${UPLOAD_BACKUP_PATH} to your UPLOAD_LOCATION"
echo ""
echo "IMPORTANT: For a complete restore, you need BOTH the database backup AND the upload directory backup."
echo "The database contains metadata, while the upload directory contains your actual photos and videos."

141
immich/restore-immich.sh Executable file
View File

@@ -0,0 +1,141 @@
#!/bin/bash
# Immich Restore Script
# This script restores an Immich installation from backups created by backup-immich.sh
# Based on recommendations from https://immich.app/docs/administration/backup-and-restore/
# Set up error handling
set -e
# Load environment variables from the .env file
ENV_FILE="$(dirname "$0")/../.env"
if [ -f "$ENV_FILE" ]; then
echo "Loading environment variables from $ENV_FILE"
source "$ENV_FILE"
else
echo "Error: .env file not found in $(dirname "$0")/.."
exit 1
fi
# Set up logging to central logs directory
LOG_DIR="$(dirname "$0")/../logs"
mkdir -p "$LOG_DIR"
LOG_FILE="${LOG_DIR}/immich-restore.log"
# Function to log with timestamp
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}
# Function to display usage
usage() {
echo "Usage: $0 --db-backup <path> --uploads-backup <path> [options]"
echo ""
echo "Required arguments:"
echo " --db-backup PATH Path to database backup file (.sql.gz)"
echo " --uploads-backup PATH Path to uploads backup file (.tar.gz)"
echo ""
echo "Optional arguments:"
echo " --dry-run Show what would be restored without making changes"
echo " --skip-db Skip database restoration"
echo " --skip-uploads Skip uploads restoration"
echo " --help Show this help message"
echo ""
echo "Example:"
echo " $0 --db-backup ./immich_backups/immich_db_backup_20250526_120000.sql.gz \\"
echo " --uploads-backup ./immich_backups/immich_uploads_20250526_120000.tar.gz"
}
# Parse command line arguments
DB_BACKUP=""
UPLOADS_BACKUP=""
DRY_RUN=false
SKIP_DB=false
SKIP_UPLOADS=false
while [[ $# -gt 0 ]]; do
case $1 in
--db-backup)
DB_BACKUP="$2"
shift 2
;;
--uploads-backup)
UPLOADS_BACKUP="$2"
shift 2
;;
--dry-run)
DRY_RUN=true
shift
;;
--skip-db)
SKIP_DB=true
shift
;;
--skip-uploads)
SKIP_UPLOADS=true
shift
;;
--help)
usage
exit 0
;;
*)
echo "Unknown option: $1"
usage
exit 1
;;
esac
done
# Validate required arguments
if [ -z "$DB_BACKUP" ] && [ "$SKIP_DB" = false ]; then
echo "Error: --db-backup is required unless --skip-db is specified"
usage
exit 1
fi
if [ -z "$UPLOADS_BACKUP" ] && [ "$SKIP_UPLOADS" = false ]; then
echo "Error: --uploads-backup is required unless --skip-uploads is specified"
usage
exit 1
fi
# Validate backup files exist
if [ "$SKIP_DB" = false ] && [ ! -f "$DB_BACKUP" ]; then
echo "Error: Database backup file not found: $DB_BACKUP"
exit 1
fi
if [ "$SKIP_UPLOADS" = false ] && [ ! -f "$UPLOADS_BACKUP" ]; then
echo "Error: Uploads backup file not found: $UPLOADS_BACKUP"
exit 1
fi
echo "=== IMMICH RESTORE OPERATION ==="
echo "Database backup: ${DB_BACKUP:-SKIPPED}"
echo "Uploads backup: ${UPLOADS_BACKUP:-SKIPPED}"
echo "Dry run mode: $DRY_RUN"
echo ""
if [ "$DRY_RUN" = true ]; then
echo "DRY RUN MODE - No changes will be made"
echo ""
fi
# TODO: Implement restore logic
echo "⚠️ RESTORE SCRIPT TEMPLATE ⚠️"
echo ""
echo "This is a template script. Implementation needed:"
echo ""
echo "1. Stop Immich containers"
echo "2. Restore database (if not skipped):"
echo " - Decompress $DB_BACKUP"
echo " - Execute SQL restore commands"
echo "3. Restore uploads (if not skipped):"
echo " - Extract $UPLOADS_BACKUP to $UPLOAD_LOCATION"
echo " - Set proper ownership and permissions"
echo "4. Restart Immich containers"
echo "5. Verify restoration"
echo ""
echo "For detailed restore instructions, see:"
echo "https://immich.app/docs/administration/backup-and-restore/"

177
immich/validate-immich-backups.sh Executable file
View File

@@ -0,0 +1,177 @@
#!/bin/bash
# Immich Backup Validation Script
# This script validates Immich backup files for integrity and completeness
# Set up error handling
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Function to display usage
usage() {
echo "Usage: $0 [backup_directory]"
echo ""
echo "Arguments:"
echo " backup_directory Directory containing backup files (default: ../immich_backups)"
echo ""
echo "This script validates:"
echo " - Database backup file integrity"
echo " - Upload archive integrity"
echo " - File sizes and timestamps"
echo " - Backup completeness"
}
# Parse command line arguments
BACKUP_DIR="${1:-$(dirname "$0")/../immich_backups}"
# Set up logging to central logs directory
LOG_DIR="$(dirname "$0")/../logs"
mkdir -p "$LOG_DIR"
LOG_FILE="${LOG_DIR}/immich-validation.log"
# Function to log validation results
log_validation() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
# Verify backup directory exists
if [ ! -d "$BACKUP_DIR" ]; then
echo -e "${RED}Error: Backup directory not found: $BACKUP_DIR${NC}"
log_validation "Error: Backup directory not found: $BACKUP_DIR"
exit 1
fi
log_validation "Starting backup validation for directory: $BACKUP_DIR"
echo "=== IMMICH BACKUP VALIDATION ==="
echo "Backup directory: $BACKUP_DIR"
echo ""
# Find backup files
DB_BACKUPS=$(find "$BACKUP_DIR" -name "immich_db_backup_*.sql.gz" -type f | sort -r)
UPLOAD_BACKUPS=$(find "$BACKUP_DIR" -name "immich_uploads_*.tar.gz" -type f | sort -r)
TOTAL_ERRORS=0
echo "=== DATABASE BACKUPS ==="
if [ -z "$DB_BACKUPS" ]; then
echo -e "${YELLOW}Warning: No database backup files found${NC}"
log_validation "Warning: No database backup files found"
((TOTAL_ERRORS++))
else
for backup in $DB_BACKUPS; do
echo "Validating: $(basename "$backup")"
# Check file size
SIZE=$(stat -c%s "$backup" 2>/dev/null || echo "0")
if [ "$SIZE" -lt 1024 ]; then
echo -e " ${RED}✗ File is too small (${SIZE} bytes)${NC}"
log_validation "Error: File is too small (${SIZE} bytes) - $(basename "$backup")"
((TOTAL_ERRORS++))
else
echo -e " ${GREEN}✓ File size OK ($(du -h "$backup" | cut -f1))${NC}"
fi
# Check if it's a valid gzip file
if gzip -t "$backup" 2>/dev/null; then
echo -e " ${GREEN}✓ Gzip file integrity OK${NC}"
else
echo -e " ${RED}✗ Gzip file corruption detected${NC}"
log_validation "Error: Gzip file corruption detected - $(basename "$backup")"
((TOTAL_ERRORS++))
fi
# Check if SQL content looks valid (basic check)
if zcat "$backup" 2>/dev/null | head -n 10 | grep -q "PostgreSQL database dump"; then
echo -e " ${GREEN}✓ SQL content appears valid${NC}"
else
echo -e " ${YELLOW}? Cannot verify SQL content format${NC}"
fi
echo ""
done
fi
echo "=== UPLOAD BACKUPS ==="
if [ -z "$UPLOAD_BACKUPS" ]; then
echo -e "${YELLOW}Warning: No upload backup files found${NC}"
log_validation "Warning: No upload backup files found"
((TOTAL_ERRORS++))
else
for backup in $UPLOAD_BACKUPS; do
echo "Validating: $(basename "$backup")"
# Check file size
SIZE=$(stat -c%s "$backup" 2>/dev/null || echo "0")
if [ "$SIZE" -lt 1024 ]; then
echo -e " ${RED}✗ File is too small (${SIZE} bytes)${NC}"
log_validation "Error: File is too small (${SIZE} bytes) - $(basename "$backup")"
((TOTAL_ERRORS++))
else
echo -e " ${GREEN}✓ File size OK ($(du -h "$backup" | cut -f1))${NC}"
fi
# Check if it's a valid tar.gz file
if tar -tzf "$backup" >/dev/null 2>&1; then
echo -e " ${GREEN}✓ Tar.gz file integrity OK${NC}"
# Count files in archive
FILE_COUNT=$(tar -tzf "$backup" 2>/dev/null | wc -l)
echo -e " ${GREEN}✓ Archive contains ${FILE_COUNT} files/directories${NC}"
else
echo -e " ${RED}✗ Tar.gz file corruption detected${NC}"
log_validation "Error: Tar.gz file corruption detected - $(basename "$backup")"
((TOTAL_ERRORS++))
fi
echo ""
done
fi
echo "=== BACKUP PAIRING VALIDATION ==="
# Check if we have matching pairs of backups (same timestamp)
DB_TIMESTAMPS=$(echo "$DB_BACKUPS" | sed 's/.*immich_db_backup_\([0-9_]*\)\.sql\.gz/\1/' | sort)
UPLOAD_TIMESTAMPS=$(echo "$UPLOAD_BACKUPS" | sed 's/.*immich_uploads_\([0-9_]*\)\.tar\.gz/\1/' | sort)
echo "Database backup timestamps: $(echo "$DB_TIMESTAMPS" | tr '\n' ' ')"
echo "Upload backup timestamps: $(echo "$UPLOAD_TIMESTAMPS" | tr '\n' ' ')"
# Find matching pairs
MATCHED_PAIRS=0
for db_ts in $DB_TIMESTAMPS; do
if echo "$UPLOAD_TIMESTAMPS" | grep -q "^${db_ts}$"; then
echo -e "${GREEN}✓ Complete backup set found for timestamp: $db_ts${NC}"
((MATCHED_PAIRS++))
else
echo -e "${YELLOW}? Incomplete backup set for timestamp: $db_ts (missing upload backup)${NC}"
log_validation "Warning: Incomplete backup set for timestamp: $db_ts (missing upload backup)"
fi
done
for upload_ts in $UPLOAD_TIMESTAMPS; do
if ! echo "$DB_TIMESTAMPS" | grep -q "^${upload_ts}$"; then
echo -e "${YELLOW}? Incomplete backup set for timestamp: $upload_ts (missing database backup)${NC}"
log_validation "Warning: Incomplete backup set for timestamp: $upload_ts (missing database backup)"
fi
done
echo ""
echo "=== VALIDATION SUMMARY ==="
echo "Complete backup pairs: $MATCHED_PAIRS"
echo "Total validation errors: $TOTAL_ERRORS"
log_validation "Validation summary: $MATCHED_PAIRS complete backup pairs, $TOTAL_ERRORS errors"
if [ "$TOTAL_ERRORS" -eq 0 ]; then
echo -e "${GREEN}✓ All backup validations passed${NC}"
log_validation "Success: All backup validations passed"
exit 0
else
echo -e "${RED}✗ Backup validation failed with $TOTAL_ERRORS errors${NC}"
log_validation "Error: Backup validation failed with $TOTAL_ERRORS errors"
exit 1
fi