shell/README.md

Shell Scripts and Dotfiles Repository

This repository contains various shell scripts for managing media-related tasks and dotfiles for system configuration.

Quick Navigation

• Backup Scripts - Enterprise-grade backup solutions
• Management Scripts - System and service management
• Documentation - Complete guides and references
• Testing - Docker-based validation framework
• Dotfiles - System configuration files

Available Scripts

Backup Scripts

• backup-media.sh: Enterprise-grade media backup script with parallel processing, comprehensive logging, and verification features.
• backup-plex.sh: Enhanced Plex backup script with integrity verification, incremental backups, and advanced features.
• restore-plex.sh: Script to restore Plex data from backups with safety checks.
• validate-plex-backups.sh: Script to validate backup integrity and monitor backup health.

Management Scripts

• plex.sh: Script to manage the Plex Media Server (start, stop, restart, status).
• folder-metrics.sh: Script to calculate disk usage and file count for a directory and its subdirectories.

Development Projects

• Telegram Backup Monitoring Bot: Comprehensive Telegram bot project for monitoring and managing all backup systems with real-time notifications and control capabilities.

Testing Scripts

• test-setup.sh: Validates the bootstrap and setup process.
• run-docker-tests.sh: Runner script that executes tests in Docker containers.

Enhanced Media Backup System

This repository includes enterprise-grade backup solutions for both general media files and Plex Media Server with comprehensive features for reliability, performance, and monitoring.

Media Backup Script (backup-media.sh)

The enhanced media backup script provides enterprise-grade features for backing up large media collections:

Key Features

• Parallel Processing: Multi-threaded operations with configurable worker pools
• Comprehensive Logging: Multiple formats (text, JSON, markdown) with detailed metrics
• Backup Verification: SHA-256 checksum validation and integrity checks
• Performance Monitoring: Real-time progress tracking and transfer statistics
• Automatic Cleanup: Configurable retention policies with space management
• Smart Notifications: Detailed completion reports with statistics
• Safety Features: Dry-run mode, pre-flight checks, and graceful error handling
• Interactive Mode: Manual control with real-time feedback

Usage Examples

# Standard parallel backup (recommended)
./backup-media.sh

# Sequential backup for better compatibility
./backup-media.sh --sequential

# Test run without making changes
./backup-media.sh --dry-run

# Interactive mode with manual control
./backup-media.sh --interactive

# Verbose logging with performance metrics
./backup-media.sh --verbose

# Custom source and destination
./backup-media.sh --source /path/to/media --destination /path/to/backup

Configuration Options

The script includes configurable parameters:

• PARALLEL_JOBS=4: Number of parallel rsync processes
• MAX_BACKUP_AGE_DAYS=90: Retention period for old backups
• BACKUP_ROOT: Default backup destination
• LOG_ROOT: Location for backup logs

Performance Features

• Progress Tracking: Real-time file transfer progress
• Transfer Statistics: Bandwidth, file counts, and timing metrics
• Resource Monitoring: CPU and memory usage tracking
• Optimization: Intelligent file handling and compression options

Advanced Plex Backup System

Specialized backup system for Plex Media Server with database-aware features. For complete documentation, see Plex Scripts Documentation.

Backup Usage Examples

Media Backup Operations

# Quick media backup with default settings
./backup-media.sh

# High-performance parallel backup
./backup-media.sh --parallel --workers 8

# Test backup strategy without making changes
./backup-media.sh --dry-run --verbose

# Custom backup with specific paths
./backup-media.sh --source /mnt/movies --destination /backup/movies

Advanced Plex Operations

# Run enhanced Plex backup (recommended)
./backup-plex.sh

# Validate all backups and generate report
./validate-plex-backups.sh --report

# Quick validation check
./validate-plex-backups.sh

# Test restore without making changes (dry run)
./restore-plex.sh plex-backup-20250125_143022.tar.gz --dry-run

# Restore from specific backup archive
./restore-plex.sh plex-backup-20250125_143022.tar.gz

Automation and Scheduling

Daily Media Backup

# Add to crontab for daily media backup at 2 AM
0 2 * * * /home/acedanger/shell/backup-media.sh --parallel

# Alternative: Sequential backup for systems with limited resources
0 2 * * * /home/acedanger/shell/backup-media.sh --sequential

Automated Plex Backup with Validation

Daily Plex Backup with Validation

# Add to crontab for daily Plex backup at 3 AM
0 3 * * * /home/acedanger/shell/backup-plex.sh

# Add daily validation at 7 AM
0 7 * * * /home/acedanger/shell/validate-plex-backups.sh --fix

Weekly Comprehensive Validation Report

# Generate detailed weekly report (Sundays at 8 AM)
0 8 * * 0 /home/acedanger/shell/validate-plex-backups.sh --report

Backup Configuration and Strategy

For detailed configuration information and backup strategies, see:

• Plex Backup Configuration: Plex-specific settings and options
• Immich Backup Configuration: Database and upload directory settings
• Enhanced Media Backup: Media backup script configuration
• Crontab Management: Automated scheduling and system-specific configurations

Documentation

Component-Specific Documentation

• Plex Scripts Documentation: Comprehensive documentation for Plex backup, restoration, validation, and management scripts
• Immich Scripts Documentation: Complete guide for Immich backup and management system with PostgreSQL and B2 integration
• Crontab Management Documentation: Complete guide for crontab management, backup systems, and multi-system administration
• Dotfiles Documentation: System configuration files and setup instructions

Technical Documentation

• Enhanced Media Backup Documentation: Comprehensive guide for the enhanced backup-media.sh script with enterprise features
• Basic Media Backup Documentation: Documentation for the original backup-media.sh script
• Media Backup Enhancement Summary: Summary of enhancements and feature comparisons
• Immich Backup Enhancement Summary: Details of Immich backup system improvements
• Docker Bootstrap Testing Framework: Detailed documentation for the Docker-based bootstrap validation system
• Folder Metrics Script Documentation: Detailed documentation for the folder-metrics.sh script
• Production Deployment Guide: Complete deployment and operations guide
• Project Completion Summary: Overview of completed features and project milestones

Dotfiles

The repository includes dotfiles for system configuration in the dotfiles directory. These can be automatically set up using the bootstrap script:

curl -fsSL https://raw.githubusercontent.com/acedanger/shell/main/bootstrap.sh | bash

For more information about the dotfiles, see Dotfiles README.

Testing

This repository includes Docker-based testing to validate the setup process across different environments. For complete testing documentation, see Docker Bootstrap Testing Framework.

Quick Testing

# Test in Ubuntu container
./setup/run-docker-tests.sh ubuntu

# Test in Debian container  
./setup/run-docker-tests.sh debian

# Test in both environments
./setup/run-docker-tests.sh all

plex.sh

This script is used to manage the Plex Media Server service on a systemd-based Linux distribution. It provides the following functionalities:

• start: Starts the Plex Media Server.
• stop: Stops the Plex Media Server.
• restart: Restarts the Plex Media Server.
• status: Displays the current status of the Plex Media Server.

Usage

Note that these commands will run as root.

./shell/plex.sh {start|stop|restart|status}

Enhanced Crontab System Migration

For complete crontab management documentation, see Crontab Management Documentation.

Current Status

The crontab system uses hostname-specific configuration files:

• crontab-europa.txt - Media server configuration
• crontab-io.txt - Download/acquisition server configuration
• crontab-racknerd.txt - Backup server configuration

Quick Management

# Install system-specific crontab (auto-detects hostname)
./crontab/manage-enhanced-crontab.sh install

# View current system's crontab configuration
./crontab/manage-enhanced-crontab.sh show