# Shell Scripts and Dotfiles Repository This repository contains various shell scripts for managing media-related tasks and dotfiles for system configuration. ## Quick Navigation - **[Backup Scripts](#backup-scripts)** - Enterprise-grade backup solutions - **[Management Scripts](#management-scripts)** - System and service management - **[Documentation](#documentation)** - Complete guides and references - **[Testing](#testing)** - Docker-based validation framework - **[Dotfiles](#dotfiles)** - System configuration files ## Available Scripts ### Backup Scripts - **`backup-media.sh`**: Enterprise-grade media backup script with parallel processing, comprehensive logging, and verification features. - **`backup-plex.sh`**: Enhanced Plex backup script with integrity verification, incremental backups, and advanced features. - **`restore-plex.sh`**: Script to restore Plex data from backups with safety checks. - **`validate-plex-backups.sh`**: Script to validate backup integrity and monitor backup health. ### Management Scripts - **`plex.sh`**: Script to manage the Plex Media Server (start, stop, restart, status). - **`folder-metrics.sh`**: Script to calculate disk usage and file count for a directory and its subdirectories. ### Development Projects - **[Telegram Backup Monitoring Bot](./telegram/github-issues/README.md)**: Comprehensive Telegram bot project for monitoring and managing all backup systems with real-time notifications and control capabilities. ### Testing Scripts - **`test-setup.sh`**: Validates the bootstrap and setup process. - **`run-docker-tests.sh`**: Runner script that executes tests in Docker containers. ## Enhanced Media Backup System This repository includes enterprise-grade backup solutions for both general media files and Plex Media Server with comprehensive features for reliability, performance, and monitoring. ### Media Backup Script (`backup-media.sh`) The enhanced media backup script provides enterprise-grade features for backing up large media collections: #### Key Features - **Parallel Processing**: Multi-threaded operations with configurable worker pools - **Comprehensive Logging**: Multiple formats (text, JSON, markdown) with detailed metrics - **Backup Verification**: SHA-256 checksum validation and integrity checks - **Performance Monitoring**: Real-time progress tracking and transfer statistics - **Automatic Cleanup**: Configurable retention policies with space management - **Smart Notifications**: Detailed completion reports with statistics - **Safety Features**: Dry-run mode, pre-flight checks, and graceful error handling - **Interactive Mode**: Manual control with real-time feedback #### Usage Examples ```bash # Standard parallel backup (recommended) ./backup-media.sh # Sequential backup for better compatibility ./backup-media.sh --sequential # Test run without making changes ./backup-media.sh --dry-run # Interactive mode with manual control ./backup-media.sh --interactive # Verbose logging with performance metrics ./backup-media.sh --verbose # Custom source and destination ./backup-media.sh --source /path/to/media --destination /path/to/backup ``` ### Configuration Options The script includes configurable parameters: - `PARALLEL_JOBS=4`: Number of parallel rsync processes - `MAX_BACKUP_AGE_DAYS=90`: Retention period for old backups - `BACKUP_ROOT`: Default backup destination - `LOG_ROOT`: Location for backup logs ### Performance Features - **Progress Tracking**: Real-time file transfer progress - **Transfer Statistics**: Bandwidth, file counts, and timing metrics - **Resource Monitoring**: CPU and memory usage tracking - **Optimization**: Intelligent file handling and compression options ### Advanced Plex Backup System Specialized backup system for Plex Media Server with database-aware features. For complete documentation, see [Plex Scripts Documentation](./plex/README.md). ## Backup Usage Examples ### Media Backup Operations ```bash # Quick media backup with default settings ./backup-media.sh # High-performance parallel backup ./backup-media.sh --parallel --workers 8 # Test backup strategy without making changes ./backup-media.sh --dry-run --verbose # Custom backup with specific paths ./backup-media.sh --source /mnt/movies --destination /backup/movies ``` ### Advanced Plex Operations ```bash # Run enhanced Plex backup (recommended) ./backup-plex.sh # Validate all backups and generate report ./validate-plex-backups.sh --report # Quick validation check ./validate-plex-backups.sh # Test restore without making changes (dry run) ./restore-plex.sh plex-backup-20250125_143022.tar.gz --dry-run # Restore from specific backup archive ./restore-plex.sh plex-backup-20250125_143022.tar.gz ``` ## Automation and Scheduling ### Daily Media Backup ```bash # Add to crontab for daily media backup at 2 AM 0 2 * * * /home/acedanger/shell/backup-media.sh --parallel # Alternative: Sequential backup for systems with limited resources 0 2 * * * /home/acedanger/shell/backup-media.sh --sequential ``` ### Automated Plex Backup with Validation ### Daily Plex Backup with Validation ```bash # Add to crontab for daily Plex backup at 3 AM 0 3 * * * /home/acedanger/shell/backup-plex.sh # Add daily validation at 7 AM 0 7 * * * /home/acedanger/shell/validate-plex-backups.sh --fix ``` ### Weekly Comprehensive Validation Report ```bash # Generate detailed weekly report (Sundays at 8 AM) 0 8 * * 0 /home/acedanger/shell/validate-plex-backups.sh --report ``` ## Backup Configuration and Strategy For detailed configuration information and backup strategies, see: - **[Plex Backup Configuration](./plex/README.md#configuration)**: Plex-specific settings and options - **[Immich Backup Configuration](./immich/README.md#configuration)**: Database and upload directory settings - **[Enhanced Media Backup](./docs/enhanced-media-backup.md)**: Media backup script configuration - **[Crontab Management](./crontab/README.md)**: Automated scheduling and system-specific configurations ## Documentation ### Component-Specific Documentation - **[Plex Scripts Documentation](./plex/README.md)**: Comprehensive documentation for Plex backup, restoration, validation, and management scripts - **[Immich Scripts Documentation](./immich/README.md)**: Complete guide for Immich backup and management system with PostgreSQL and B2 integration - **[Crontab Management Documentation](./crontab/README.md)**: Complete guide for crontab management, backup systems, and multi-system administration - **[Dotfiles Documentation](./dotfiles/README.md)**: System configuration files and setup instructions ### Technical Documentation - **[Enhanced Media Backup Documentation](./docs/enhanced-media-backup.md)**: Comprehensive guide for the enhanced `backup-media.sh` script with enterprise features - **[Basic Media Backup Documentation](./docs/backup-media.md)**: Documentation for the original `backup-media.sh` script - **[Media Backup Enhancement Summary](./docs/backup-media-enhancement-summary.md)**: Summary of enhancements and feature comparisons - **[Immich Backup Enhancement Summary](./docs/immich-backup-enhancement-summary.md)**: Details of Immich backup system improvements - **[Docker Bootstrap Testing Framework](./docs/docker-bootstrap-testing-framework.md)**: Detailed documentation for the Docker-based bootstrap validation system - **[Folder Metrics Script Documentation](./docs/folder-metrics.md)**: Detailed documentation for the `folder-metrics.sh` script - **[Production Deployment Guide](./docs/production-deployment-guide.md)**: Complete deployment and operations guide - **[Project Completion Summary](./docs/project-completion-summary.md)**: Overview of completed features and project milestones ## Dotfiles The repository includes dotfiles for system configuration in the `dotfiles` directory. These can be automatically set up using the bootstrap script: ```bash curl -fsSL https://raw.githubusercontent.com/acedanger/shell/main/bootstrap.sh | bash ``` For more information about the dotfiles, see [Dotfiles README](./dotfiles/README.md). ## Testing This repository includes Docker-based testing to validate the setup process across different environments. For complete testing documentation, see [Docker Bootstrap Testing Framework](./docs/docker-bootstrap-testing-framework.md). ### Quick Testing ```bash # Test in Ubuntu container ./setup/run-docker-tests.sh ubuntu # Test in Debian container ./setup/run-docker-tests.sh debian # Test in both environments ./setup/run-docker-tests.sh all ``` ## plex.sh This script is used to manage the Plex Media Server service on a systemd-based Linux distribution. It provides the following functionalities: - **start**: Starts the Plex Media Server. - **stop**: Stops the Plex Media Server. - **restart**: Restarts the Plex Media Server. - **status**: Displays the current status of the Plex Media Server. ## Usage Note that these commands will run as `root`. ```shell ./shell/plex.sh {start|stop|restart|status} ``` ## Enhanced Crontab System Migration For complete crontab management documentation, see [Crontab Management Documentation](./crontab/README.md). ### Current Status The crontab system uses hostname-specific configuration files: - **crontab-europa.txt** - Media server configuration - **crontab-io.txt** - Download/acquisition server configuration - **crontab-racknerd.txt** - Backup server configuration ### Quick Management ```bash # Install system-specific crontab (auto-detects hostname) ./crontab/manage-enhanced-crontab.sh install # View current system's crontab configuration ./crontab/manage-enhanced-crontab.sh show ```