peterwood/shell
1
0
Fork 0
mirror of https://github.com/acedanger/shell.git synced 2025-12-06 07:50:11 -08:00
Code Issues 19 Packages Projects Releases Wiki Activity
Files
a49d3743fb632e05718a836fb0c0a92f1ebcc7a2
shell/README.md
Peter Wood 3ce2b687ac feat: Add Docker deployment manager and stack assignment helper scripts 
- Introduced `docker-deployment-manager.sh` for managing Docker stack deployments across multiple servers, including initialization, deployment, and status checks.
- Added `stack-assignment-helper.sh` to analyze Docker stacks and suggest server assignments based on predefined patterns.
- Removed outdated `SETUP_COMPLETE.md` file as it is no longer relevant to the current setup process.
- Ref - Documentation review #11
2025-05-29 18:19:37 -04:00

18 KiB
Raw Blame History

Shell Scripts and Dotfiles Repository

This repository contains various shell scripts for managing media-related tasks and dotfiles for system configuration.

Quick Navigation

Available Scripts

Backup Scripts

  • backup-media.sh: Enterprise-grade media backup script with parallel processing, comprehensive logging, and verification features.
  • backup-plex.sh: Enhanced Plex backup script with integrity verification, incremental backups, and advanced features.
  • restore-plex.sh: Script to restore Plex data from backups with safety checks.
  • validate-plex-backups.sh: Script to validate backup integrity and monitor backup health.

Management Scripts

  • plex.sh: Script to manage the Plex Media Server (start, stop, restart, status).
  • folder-metrics.sh: Script to calculate disk usage and file count for a directory and its subdirectories.

AI Integration

This repository includes a complete AI development environment with Ollama and Fabric integration for AI-assisted development tasks.

Ollama + Fabric Setup

The system includes:

  • Ollama Docker container running on port 11434 with phi3:mini model (3.8B parameters)
  • Fabric v1.4.195 with 216+ AI patterns for text processing
  • Google Gemini 2.5 Pro as primary AI provider
  • Local Ollama models as secondary AI provider
  • Custom shell aliases for easy container management

Basic Fabric Usage

# List all available patterns
fabric -l

# Use a pattern (uses Gemini 2.5 Pro by default)
echo "Your text here" | fabric -p summarize

# Use with specific model
echo "Your text here" | fabric -p summarize -m gemini-2.0-flash-exp

# Use local Ollama models
echo "Your text here" | fabric -p summarize -m ollama:phi3:mini

# Update patterns
fabric -U

Ollama Management Aliases

ollama-start      # Start Ollama container
ollama-stop       # Stop Ollama container
ollama-restart    # Restart Ollama container
ollama-logs       # View container logs
ollama-status     # Check container status

Popular AI Patterns

  • summarize - Summarize text content
  • explain_code - Explain code snippets and logic
  • improve_writing - Enhance writing quality and clarity
  • extract_wisdom - Extract key insights from content
  • create_quiz - Generate quiz questions from text
  • analyze_claims - Analyze and fact-check claims

Configuration Files

  • Fabric config: ~/.config/fabric/.env - AI provider settings and API keys
  • Ollama aliases: ~/.oh-my-zsh/custom/ollama-aliases.zsh - Container management commands
  • Shell config: ~/.zshrc - Main shell configuration

For complete setup instructions, see the setup documentation.

Development Projects

  • Telegram Backup Monitoring Bot: Comprehensive Telegram bot project for monitoring and managing all backup systems with real-time notifications and control capabilities.
  • Docker Deployment Management: Centralized Docker stack deployment system for managing containers across multiple servers with intelligent assignment and environment integration.

Testing Scripts

  • test-setup.sh: Validates the bootstrap and setup process.
  • run-docker-tests.sh: Runner script that executes tests in Docker containers.

Enhanced Media Backup System

This repository includes enterprise-grade backup solutions for both general media files and Plex Media Server with comprehensive features for reliability, performance, and monitoring.

Media Backup Script (backup-media.sh)

The enhanced media backup script provides enterprise-grade features for backing up large media collections:

Key Features

  • Parallel Processing: Multi-threaded operations with configurable worker pools
  • Comprehensive Logging: Multiple formats (text, JSON, markdown) with detailed metrics
  • Backup Verification: SHA-256 checksum validation and integrity checks
  • Performance Monitoring: Real-time progress tracking and transfer statistics
  • Automatic Cleanup: Configurable retention policies with space management
  • Smart Notifications: Detailed completion reports with statistics
  • Safety Features: Dry-run mode, pre-flight checks, and graceful error handling
  • Interactive Mode: Manual control with real-time feedback

Usage Examples

# Standard parallel backup (recommended)
./backup-media.sh

# Sequential backup for better compatibility
./backup-media.sh --sequential

# Test run without making changes
./backup-media.sh --dry-run

# Interactive mode with manual control
./backup-media.sh --interactive

# Verbose logging with performance metrics
./backup-media.sh --verbose

# Custom source and destination
./backup-media.sh --source /path/to/media --destination /path/to/backup

Configuration Options

The script includes configurable parameters:

  • PARALLEL_JOBS=4: Number of parallel rsync processes
  • MAX_BACKUP_AGE_DAYS=90: Retention period for old backups
  • BACKUP_ROOT: Default backup destination
  • LOG_ROOT: Location for backup logs

Performance Features

  • Progress Tracking: Real-time file transfer progress
  • Transfer Statistics: Bandwidth, file counts, and timing metrics
  • Resource Monitoring: CPU and memory usage tracking
  • Optimization: Intelligent file handling and compression options

Advanced Plex Backup System

Specialized backup system for Plex Media Server with database-aware features. For complete documentation, see Plex Scripts Documentation.

Backup Usage Examples

Media Backup Operations

# Quick media backup with default settings
./backup-media.sh

# High-performance parallel backup
./backup-media.sh --parallel --workers 8

# Test backup strategy without making changes
./backup-media.sh --dry-run --verbose

# Custom backup with specific paths
./backup-media.sh --source /mnt/movies --destination /backup/movies

Advanced Plex Operations

# Run enhanced Plex backup (recommended)
./backup-plex.sh

# Validate all backups and generate report
./validate-plex-backups.sh --report

# Quick validation check
./validate-plex-backups.sh

# Test restore without making changes (dry run)
./restore-plex.sh plex-backup-20250125_143022.tar.gz --dry-run

# Restore from specific backup archive
./restore-plex.sh plex-backup-20250125_143022.tar.gz

Automation and Scheduling

Daily Media Backup

# Add to crontab for daily media backup at 2 AM
0 2 * * * /home/acedanger/shell/backup-media.sh --parallel

# Alternative: Sequential backup for systems with limited resources
0 2 * * * /home/acedanger/shell/backup-media.sh --sequential

Automated Plex Backup with Validation

Daily Plex Backup with Validation

# Add to crontab for daily Plex backup at 3 AM
0 3 * * * /home/acedanger/shell/backup-plex.sh

# Add daily validation at 7 AM
0 7 * * * /home/acedanger/shell/validate-plex-backups.sh --fix

Weekly Comprehensive Validation Report

# Generate detailed weekly report (Sundays at 8 AM)
0 8 * * 0 /home/acedanger/shell/validate-plex-backups.sh --report

Backup Configuration and Strategy

For detailed configuration information and backup strategies, see:

Comprehensive Documentation

This repository includes extensive documentation organized by component and functionality:

📁 Documentation Structure

📁 docs/          - Technical documentation and guides
📁 setup/         - Setup and configuration files  
📁 plex/          - Plex Media Server scripts and docs
📁 immich/        - Immich photo management scripts
📁 crontab/       - Crontab management system
📁 dotfiles/      - System configuration files
📁 completions/   - Tab completion scripts
📁 telegram/      - Telegram bot integration

🛠️ Component Documentation

Backup Systems

Media Server Management

Photo Management

System Administration

Development & Deployment

📊 Project Documentation

Documentation

Component-Specific Documentation

Technical Documentation

Environment & System Documentation

Project Reviews & Summaries

Setup and Getting Started

Quick Setup

The repository includes automated setup via bootstrap script:

curl -fsSL https://raw.githubusercontent.com/acedanger/shell/main/bootstrap.sh | bash

This installs:

  • Zsh with Oh My Zsh and custom plugins
  • All packages from setup/packages.list
  • Dotfiles and system configuration
  • Tab completion for all scripts
  • Development tools (Node.js via nvm, VS Code, etc.)

AI Development Environment

For AI-assisted development, the system includes:

  • Ollama running in Docker with local models
  • Fabric with 216+ AI patterns for text processing
  • Google Gemini integration as primary AI provider
  • Custom aliases for easy management

Test the AI setup:

# Test Gemini integration
echo "Test text" | fabric -p summarize

# Test local Ollama integration  
echo "Test text" | fabric -p summarize -m ollama:phi3:mini

Dotfiles

The repository includes dotfiles for system configuration in the dotfiles directory. These can be automatically set up using the bootstrap script:

curl -fsSL https://raw.githubusercontent.com/acedanger/shell/main/bootstrap.sh | bash

For more information about the dotfiles, see Dotfiles README.

Tab Completion

This repository includes intelligent bash completion for all backup scripts, providing tab completion for command-line flags and options.

Features

  • Intelligent flag completion: Tab completion for all backup script options
  • Webhook URL suggestions: Auto-complete common webhook endpoints
  • Path-aware completion: Works with relative, absolute, and PATH-based script execution
  • Cross-shell support: Compatible with both bash and zsh

Supported Scripts

  • backup-immich.sh: --help, --dry-run, --no-upload, --verbose
  • backup-plex.sh: --help, --auto-repair, --check-integrity, --non-interactive, --no-parallel, --no-performance, --webhook, --email
  • backup-media.sh: --help, --dry-run, --no-verify, --sequential, --interactive, --webhook
  • Generic backup scripts: Common flags for all backup utilities

Usage Examples

# Tab completion for backup script flags
~/shell/immich/backup-immich.sh --<TAB>
# Shows: --help --dry-run --no-upload --verbose

# Tab completion for webhook URLs
~/shell/plex/backup-plex.sh --webhook <TAB>
# Shows: https://notify.peterwood.rocks/lab

# Works with any script path
backup-immich.sh --<TAB>          # From PATH
./backup-immich.sh --<TAB>        # Relative path
/full/path/backup-immich.sh --<TAB>  # Absolute path

Installation

Tab completion is automatically installed when you run the setup scripts:

./setup/bootstrap.sh  # Installs completion automatically

For manual installation or more details, see Completions README.

Testing

This repository includes Docker-based testing to validate the setup process across different environments. For complete testing documentation, see Docker Bootstrap Testing Framework.

Quick Testing

# Test in Ubuntu container
./setup/run-docker-tests.sh ubuntu

# Test in Debian container
./setup/run-docker-tests.sh debian

# Test in both environments
./setup/run-docker-tests.sh all

plex.sh

This script is used to manage the Plex Media Server service on a systemd-based Linux distribution. It provides the following functionalities:

  • start: Starts the Plex Media Server.
  • stop: Stops the Plex Media Server.
  • restart: Restarts the Plex Media Server.
  • status: Displays the current status of the Plex Media Server.

Usage

Note that these commands will run as root.

./shell/plex.sh {start|stop|restart|status}

Enhanced Crontab System Migration

For complete crontab management documentation, see Crontab Management Documentation.

Current Status

The crontab system uses hostname-specific configuration files:

  • crontab-europa.txt - Media server configuration
  • crontab-io.txt - Download/acquisition server configuration
  • crontab-racknerd.txt - Backup server configuration

Quick Management

# Install system-specific crontab (auto-detects hostname)
./crontab/manage-enhanced-crontab.sh install

# View current system's crontab configuration
./crontab/manage-enhanced-crontab.sh show
Reference in New Issue View Git Blame Copy Permalink