backup-media.sh: Enterprise-grade media backup script with parallel processing, comprehensive logging, and verification features.
backup-plex.sh: Enhanced Plex backup script with integrity verification, incremental backups, and advanced features.
restore-plex.sh: Script to restore Plex data from backups with safety checks.
validate-plex-backups.sh: Script to validate backup integrity and monitor backup health.

Management Scripts

plex.sh: Script to manage the Plex Media Server (start, stop, restart, status).
folder-metrics.sh: Script to calculate disk usage and file count for a directory and its subdirectories.

Testing Scripts

test-setup.sh: Validates the bootstrap and setup process.
run-docker-tests.sh: Runner script that executes tests in Docker containers.

Enhanced Media Backup System

This repository includes enterprise-grade backup solutions for both general media files and Plex Media Server with comprehensive features for reliability, performance, and monitoring.

Media Backup Script (`backup-media.sh`)

The enhanced media backup script provides enterprise-grade features for backing up large media collections:

Key Features

Parallel Processing: Multi-threaded operations with configurable worker pools
Comprehensive Logging: Multiple formats (text, JSON, markdown) with detailed metrics
Backup Verification: SHA-256 checksum validation and integrity checks
Performance Monitoring: Real-time progress tracking and transfer statistics
Automatic Cleanup: Configurable retention policies with space management
Smart Notifications: Detailed completion reports with statistics
Safety Features: Dry-run mode, pre-flight checks, and graceful error handling
Interactive Mode: Manual control with real-time feedback

Usage Examples

# Standard parallel backup (recommended)
./backup-media.sh

# Sequential backup for better compatibility
./backup-media.sh --sequential

# Test run without making changes
./backup-media.sh --dry-run

# Interactive mode with manual control
./backup-media.sh --interactive

# Verbose logging with performance metrics
./backup-media.sh --verbose

# Custom source and destination
./backup-media.sh --source /path/to/media --destination /path/to/backup

Configuration Options

The script includes configurable parameters:

PARALLEL_JOBS=4: Number of parallel rsync processes
MAX_BACKUP_AGE_DAYS=90: Retention period for old backups
BACKUP_ROOT: Default backup destination
LOG_ROOT: Location for backup logs

Performance Features

Progress Tracking: Real-time file transfer progress
Transfer Statistics: Bandwidth, file counts, and timing metrics
Resource Monitoring: CPU and memory usage tracking
Optimization: Intelligent file handling and compression options

Advanced Plex Backup System

Specialized backup system for Plex Media Server with database-aware features:

Components

backup-plex.sh: Advanced backup script with integrity verification, incremental backups, and automatic cleanup
restore-plex.sh: Safe restoration script with dry-run mode and current data backup
validate-plex-backups.sh: Backup validation and health monitoring script

Plex-Specific Features

Incremental backups: Only backs up files that have changed since last backup
Database integrity verification: Uses MD5 checksums to verify backup integrity
Automatic cleanup: Configurable retention policies for old backups
Disk space monitoring: Checks available space before starting backup
Safe restoration: Backs up current data before restoring from backup
Comprehensive logging: Detailed logs with color-coded output
Service management: Safely stops/starts Plex during backup operations

Backup Usage Examples

Media Backup Operations

# Quick media backup with default settings
./backup-media.sh

# High-performance parallel backup
./backup-media.sh --parallel --workers 8

# Test backup strategy without making changes
./backup-media.sh --dry-run --verbose

# Custom backup with specific paths
./backup-media.sh --source /mnt/movies --destination /backup/movies

Advanced Plex Operations

# Run enhanced Plex backup (recommended)
./backup-plex.sh

# Validate all backups and generate report
./validate-plex-backups.sh --report

# Quick validation check
./validate-plex-backups.sh

# Test restore without making changes (dry run)
./restore-plex.sh plex-backup-20250125_143022.tar.gz --dry-run

# Restore from specific backup archive
./restore-plex.sh plex-backup-20250125_143022.tar.gz

Automation and Scheduling

Daily Media Backup

# Add to crontab for daily media backup at 2 AM
0 2 * * * /home/acedanger/shell/backup-media.sh --parallel

# Alternative: Sequential backup for systems with limited resources
0 2 * * * /home/acedanger/shell/backup-media.sh --sequential

Automated Plex Backup with Validation

Daily Plex Backup with Validation

# Add to crontab for daily Plex backup at 3 AM
0 3 * * * /home/acedanger/shell/backup-plex.sh

# Add daily validation at 7 AM
0 7 * * * /home/acedanger/shell/validate-plex-backups.sh --fix

Weekly Comprehensive Validation Report

# Generate detailed weekly report (Sundays at 8 AM)
0 8 * * 0 /home/acedanger/shell/validate-plex-backups.sh --report

Backup Configuration and Strategy

Media Backup Configuration

The enhanced media backup script includes configurable parameters at the top of the file:

PARALLEL_JOBS=4: Number of parallel rsync processes
MAX_BACKUP_AGE_DAYS=90: Remove backups older than 90 days
LOG_RETENTION_DAYS=30: Keep logs for 30 days
BACKUP_ROOT: Default location for backup storage
LOG_ROOT: Location for backup logs and reports

Plex Backup Configuration

The Plex backup script configuration parameters:

MAX_BACKUP_AGE_DAYS=30: Remove backups older than 30 days
MAX_BACKUPS_TO_KEEP=10: Keep maximum of 10 backup archives
BACKUP_ROOT: Location for compressed backup archives
LOG_ROOT: Location for backup logs

Final Backup Directory Structure

The enhanced Plex backup system creates a streamlined archive-only structure:

/mnt/share/media/backups/plex/
├── plex-backup-20250125_143022.tar.gz    # Latest backup
├── plex-backup-20250124_143011.tar.gz    # Previous backup
├── plex-backup-20250123_143008.tar.gz    # Older backup
└── logs/
    ├── backup_log_20250125_143022.md
    ├── plex-backup-performance.json
    └── plex-backup.json

Key Benefits:

Direct Archive Storage: No intermediate directories required
Efficient Space Usage: Only compressed files stored permanently
Easy Management: Timestamp-based naming for clear identification
Automatic Cleanup: Legacy dated directories removed automatically

Recommended Backup Strategy

Both systems implement a robust backup strategy following industry best practices:

For Media Files:

Daily incremental backups with parallel processing for speed
Weekly verification of backup integrity
Monthly cleanup of old backups based on retention policies
Quarterly offsite sync for disaster recovery

For Plex Database:

Daily full backups with service-aware operations
Immediate validation after each backup
Weekly comprehensive reports on backup health
Monthly testing of restore procedures

Offsite Backup Integration

For comprehensive disaster recovery, sync backups to remote locations:

# Sync media backups to remote server daily at 5 AM
0 5 * * * rsync -av /mnt/share/media/backups/media/ user@remote-server:/backups/media/

# Sync Plex backups to remote server daily at 6 AM
0 6 * * * rsync -av /mnt/share/media/backups/plex/ user@remote-server:/backups/plex/

Documentation

Backup System Documentation

Enhanced Media Backup Documentation: Comprehensive guide for the enhanced backup-media.sh script with enterprise features.
Media Backup Enhancement Summary: Summary of enhancements and feature comparisons.
Plex Backup Script Documentation: Detailed documentation for the backup-plex.sh script.

Script Documentation

Plex Management Script Documentation: Detailed documentation for the plex.sh script.
Folder Metrics Script Documentation: Detailed documentation for the folder-metrics.sh script.
Testing Framework Documentation: Detailed documentation for the Docker-based testing system.

Dotfiles

The repository includes dotfiles for system configuration in the dotfiles directory. These can be automatically set up using the bootstrap script:

curl -fsSL https://raw.githubusercontent.com/acedanger/shell/main/bootstrap.sh | bash

For more information about the dotfiles, see Dotfiles README.

Testing

This repository includes Docker-based testing to validate the setup process across different environments:

test-setup.sh: Script that validates the bootstrap and setup process
run-docker-tests.sh: Runner script that executes tests in Docker containers
Dockerfile: Defines test environments (Ubuntu, Debian)

Running Tests

Test your setup in isolated Docker containers with:

# Test in Ubuntu container
./run-docker-tests.sh ubuntu

# Test in Debian container
./run-docker-tests.sh debian

# Run full bootstrap test in Ubuntu
./run-docker-tests.sh full-ubuntu

# Run full bootstrap test in Debian
./run-docker-tests.sh full-debian

# Test in both Ubuntu and Debian
./run-docker-tests.sh all

When to Use Each Testing Option

Use ./run-docker-tests.sh ubuntu or ./run-docker-tests.sh debian when you want to:
- Quickly validate if packages in packages.list are available and installed
- Test if your test-setup.sh script is working correctly
- Check for issues with specific components without performing a full setup
Use ./run-docker-tests.sh full-ubuntu or ./run-docker-tests.sh full-debian when you want to:
- Test the complete bootstrap installation process end-to-end
- Validate that all installation steps work correctly on a fresh system
- Simulate what users will experience when running the bootstrap script
Use ./run-docker-tests.sh all when you want to:
- Ensure your test-setup.sh works across both Ubuntu and Debian
- Run comprehensive checks before committing changes

The test environment checks:

Package availability and installation
Core components (git, curl, wget, etc.)
Additional packages from setup/packages.list
Oh My Zsh and plugin installation
Dotfile symlinks

Tests will continue even when some packages fail to install, reporting all issues in a comprehensive summary.

plex.sh

This script is used to manage the Plex Media Server service on a systemd-based Linux distribution. It provides the following functionalities:

start: Starts the Plex Media Server.
stop: Stops the Plex Media Server.
restart: Restarts the Plex Media Server.
status: Displays the current status of the Plex Media Server.

Usage

Note that these commands will run as root.

./shell/plex.sh {start|stop|restart|status}

Enhanced Crontab System Migration

System Status

The crontab system has been migrated from a universal crontab.txt to system-specific files:

crontab-europa.txt - Media server configuration
crontab-io.txt - Download/acquisition server configuration
crontab-racknerd.txt - Backup server configuration

Legacy File Status

The original crontab.txt has been:

Backed up to crontab.txt.bak
Role: Previously served as fallback, now obsolete
Content: Contains mixed configuration that was split into system-specific files

Current Management

Use the system-specific approach:

# Install system-specific crontab
./manage-enhanced-crontab.sh install

# The script automatically detects hostname and uses appropriate file
# europa -> crontab-europa.txt
# io -> crontab-io.txt  
# racknerd -> crontab-racknerd.txt

Fallback Behavior

If a system-specific file is missing, the management script will:

Warning: Display that system-specific file is not found
Recommendation: Create appropriate system-specific file

Creating New System Files

For new systems, create system-specific files based on the templates:

# Copy and customize for new system
cp crontab-europa.txt crontab-newsystem.txt
# Edit for system-specific tasks

Benefits of System-Specific Approach

Clear separation of concerns between systems
Reduced conflicts from universal configurations
System-appropriate task scheduling
Better maintenance and troubleshooting
Scalable for additional systems

Migration Notes

Migration completed: May 26, 2025

Languages

Shell 84.6%

PowerShell 5.3%

Go 3.2%

HTML 3.2%

Python 2.2%

Other 1.5%

README.md

Shell Scripts and Dotfiles Repository

Available Scripts

Backup Scripts