shell/plex/README.md

Plex Backup and Management Scripts

Author: Peter Wood peter@peterwood.dev

This directory contains a streamlined suite of scripts for Plex Media Server backup, monitoring, and management operations. After identifying and resolving the root cause of database corruption issues (aggressive auto-repair schedules), the toolkit has been simplified to focus on reliable, safe operations.

🚨 Important Notes

• Database corruption issues resolved (June 21, 2025)
• Custom backup system DISABLED (June 21, 2025) - Replaced by Plex built-in backups
• Plex built-in backups active - Automatic database backups every 3 days
• Manual intervention required for database repairs
• Redundant scripts moved to deprecated/ folder

⚙️ Backup System Status

Current State: Using Plex Built-in Scheduled Backups

• ✅ Plex Native Backups: Active (every 3 days)
• ✅ Backup Location: /mnt/share/media/backups/plex (NAS)
• ❌ Custom backup scripts: DISABLED (June 21, 2025)
• ✅ Health monitoring tools: Available for manual use

Why the change? Plex's built-in scheduled backup system is more reliable and integrated with the application lifecycle. The custom backup system has been disabled to avoid conflicts and redundancy.

🎯 Quick Start

With Plex Built-in Backups (Current Setup):

# Check database health (safe, read-only)
./plex-db-manager.sh check

# Basic Plex service management
./plex.sh status

# Emergency nuclear recovery (LAST RESORT ONLY)
sudo ./nuclear-plex-recovery.sh --dry-run    # Check what would happen
sudo ./nuclear-plex-recovery.sh --auto       # Actual recovery

# Check built-in Plex backups
./check-plex-builtin-backups.sh

Recommended Operations:

# Weekly health check (manual)
./plex-db-manager.sh check && echo "✅ Databases healthy"

# Check Plex built-in backup status
./check-plex-builtin-backups.sh

# Monthly validation (if needed)
./validate-plex-backups.sh --latest

🚫 Disabled Operations:

# These are NO LONGER NEEDED (Plex handles backups automatically)
# ./backup-plex.sh --non-interactive --disable-auto-repair
# ./validate-plex-backups.sh --fix

📁 Active Scripts

🔧 Core Management

plex-db-manager.sh ⭐ NEW

Consolidated database management tool - SAFE operations only

Features:

• Read-only database integrity checking (currently active)
• Proper service management with synchronization
• Manual repair intervention (currently disabled for safety)
• Comprehensive logging and error handling

Current Status:

• ✅ Integrity checking: ACTIVE and safe
• ⚠️ Repair functionality: DISABLED (safety measure)
• ✅ Service management: ACTIVE

Usage:

./plex-db-manager.sh check              # Safe integrity check
./plex-db-manager.sh repair             # Shows disabled message
./plex-db-manager.sh help               # Show all options
./plex-db-manager.sh                    # Show help (no args)

Example Output:

$ ./plex-db-manager.sh check
+================================================================+
|              PLEX DATABASE MANAGEMENT TOOL                    |
+================================================================+

[2025-06-21 14:30:15] INFO: Checking database integrity for all databases
[2025-06-21 14:30:16] SUCCESS: Database integrity check passed for main database
[2025-06-21 14:30:17] SUCCESS: Database integrity check passed for blobs database
[2025-06-21 14:30:18] SUCCESS: All database integrity checks completed successfully

🔄 Backup & Recovery

backup-plex.sh ⚠️ DISABLED

Enhanced Plex backup script - NOW DISABLED

Status: ❌ DISABLED (June 21, 2025) Reason: Replaced by Plex's built-in scheduled backup system

Previous Features:

• Database integrity checking (auto-repair disabled by default)
• WAL file handling for SQLite databases
• Performance monitoring with JSON-based logging
• Multi-channel notifications (console, webhook, email)
• Safe service management with extended timeouts
• Automated cleanup of old backups

Current Recommendation: Use Plex's built-in scheduled backups instead. Available for emergency manual use only.

Manual Usage (Emergency Only):

./backup-plex.sh --help                     # Show available options
./backup-plex.sh --check-integrity          # Database health check only
./backup-plex.sh --non-interactive --disable-auto-repair  # Manual backup

Plex Built-in Backup Status:

# Check current built-in backups
ls -la "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.*

# Recent built-in backup
ls -lath /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/*.backup.* | head -1

nuclear-plex-recovery.sh

Last-resort complete database replacement

⚠️ WARNING: Completely replaces databases with backup copies. All data since backup will be lost.

Features:

• Complete database replacement from backups
• Automatic service management
• Pre-recovery validation and safety checks
• Rollback capability if recovery fails

Usage:

sudo ./nuclear-plex-recovery.sh         # Interactive recovery
sudo ./nuclear-plex-recovery.sh --auto  # Automated recovery
sudo ./nuclear-plex-recovery.sh --dry-run  # Show what would be done
sudo ./nuclear-plex-recovery.sh --help  # Show all options

Example Workflow:

# First, check what would be restored
$ sudo ./nuclear-plex-recovery.sh --dry-run
Nuclear Recovery Plan:
- Latest backup: plex-backup-20250621.tar.gz
- Databases to replace: 2 files
- Service downtime: ~5 minutes

# If satisfied, proceed with recovery
$ sudo ./nuclear-plex-recovery.sh --auto
[WARNING] This will DESTROY current databases
[SUCCESS] Recovery completed successfully

🔍 Validation & Monitoring

validate-plex-backups.sh ⚠️ LIMITED USE

Backup validation - Now limited to legacy backup validation

Status: ⚠️ LIMITED USE (June 21, 2025) Reason: Mainly applies to old custom backup files, not Plex built-in backups

Current Functionality:

• Can validate legacy custom backup archives
• Database integrity checking within old backups
• Historical backup analysis for troubleshooting

Limited Usage:

./validate-plex-backups.sh --latest      # Validate latest custom backup (if any)
./validate-plex-backups.sh --help        # Show available options

For Plex Built-in Backups:

# Plex built-in backups don't need external validation
# They're automatically managed and verified by Plex
ls -la "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.*

monitor-plex-backup.sh ⚠️ LIMITED USE

System monitoring - Limited functionality with built-in backups

Status: ⚠️ LIMITED USE (June 21, 2025) Reason: Custom backup monitoring less relevant with Plex built-in system

Available Functionality:

• General system health monitoring
• Service status verification
• Disk space monitoring

Usage:

./monitor-plex-backup.sh --summary   # Basic system summary
./monitor-plex-backup.sh --help      # Show available options

Alternative: Check Plex Built-in Backup Status

# Check when last backup was created
stat "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.* 2>/dev/null | grep Modify | tail -1

# Check backup file sizes
ls -lah "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.* 2>/dev/null

🎛️ Service Management

plex.sh

Basic Plex service management and system operations

Features:

• Service start/stop/restart/status operations
• Interactive menu system for easy management
• Process monitoring and health checks
• Integration with other backup/repair tools
• NEW: Library scanner integration

Usage:

./plex.sh start                     # Start Plex service
./plex.sh stop                      # Stop Plex service
./plex.sh restart                   # Restart Plex service
./plex.sh status                    # Show service status
./plex.sh scan                      # Launch library scanner
./plex.sh repair                    # Launch repair menu (calls other tools)
./plex.sh nuclear                   # Nuclear recovery option
./plex.sh                           # Interactive menu

Example Usage:

# Check current service status
$ ./plex.sh status
Plex Media Server Status: RUNNING
Uptime: 2 days, 14 hours, 23 minutes

# Launch library scanner
$ ./plex.sh scan
[*] Launching Plex Library Scanner...
[i] Launching scanner in interactive mode...

# Interactive menu
$ ./plex.sh
=== Plex Management Menu ===
1) Start Service
2) Stop Service
3) Restart Service
4) Service Status
5) Database Check
6) Exit
Enter your choice [1-6]:

scan-plex-libraries.sh ⭐ NEW

Comprehensive Plex library scanning and management

Features:

• List all Plex library sections with IDs
• Scan libraries for new media files
• Refresh metadata with force options
• Deep media analysis capabilities
• Generate thumbnails and fanart
• Show library tree structures
• Interactive and command-line modes
• Comprehensive error handling and logging

Status: ✅ ACTIVE (June 26, 2025)

Integration:

• Integrated into plex.sh as the scan command
• Available via aliases: plex-scan, plex-scanner
• Direct command-line access available

Usage:

# Via plex.sh (recommended)
./plex.sh scan                      # Interactive scanner
./plex.sh scan list                 # List all libraries
./plex.sh scan scan 29              # Scan Movies library
./plex.sh scan refresh "" true      # Force refresh all libraries

# Direct usage
./scan-plex-libraries.sh list       # List all library sections
./scan-plex-libraries.sh scan       # Scan all libraries for new media
./scan-plex-libraries.sh scan 29    # Scan specific library (ID 29)
./scan-plex-libraries.sh refresh 29 true  # Force refresh specific library
./scan-plex-libraries.sh analyze 29 true  # Deep analyze library
./scan-plex-libraries.sh generate 29      # Generate thumbnails
./scan-plex-libraries.sh tree 29          # Show library structure
./scan-plex-libraries.sh -v scan 29       # Verbose scanning

# Via aliases
plex-scan                           # Launch via plex.sh
plex-scanner list                   # Direct scanner access

Example Operations:

# List all available libraries
$ ./scan-plex-libraries.sh list
Available Library Sections:
=========================
  29: Movies
  30: TV Shows
  31: Music

# Scan Movies library for new content
$ ./scan-plex-libraries.sh scan 29
[>] Scanning library section 29 for new media...
[✓] Library section 29 scan completed

# Force refresh all libraries
$ ./scan-plex-libraries.sh refresh "" true
[~] Refreshing metadata for all libraries...
[i] Refreshing section 29...
[✓] Section 29 refreshed successfully
[i] Refreshing section 30...
[✓] Section 30 refreshed successfully
[✓] All libraries refreshed successfully

Interactive Mode:

$ ./scan-plex-libraries.sh
🎬 Plex Library Scanner - Interactive Mode
Select an operation to perform:

Available Operations:
1) List all libraries
2) Scan libraries for new media
3) Refresh library metadata
4) Analyze library media
5) Generate thumbnails
6) Show library tree
q) Quit

Choose an option [1-6,q]:

### 📊 Utility Scripts

#### `check-plex-builtin-backups.sh` ⭐ **NEW**
**Monitor Plex's built-in backup system status**

**Features:**
- Check status of Plex's automatic backups
- Display latest backup information and frequency
- Validate backup location accessibility
- Show legacy custom backup file counts

**Usage:**
```bash
./check-plex-builtin-backups.sh          # Basic backup status
./check-plex-builtin-backups.sh --detailed # Detailed file information
./check-plex-builtin-backups.sh --help   # Show help

Example Output:

$ ./check-plex-builtin-backups.sh
✅ Found 1 built-in backup file(s)

📅 Latest backup:
   File: com.plexapp.plugins.library.db.backup.20250621_063833
   Date: 2025-06-21 06:38:56
   Size: 149M

📊 Summary:
✅ Custom backup system: DISABLED (June 21, 2025)
✅ Plex built-in backups: ACTIVE
ℹ️  Backup frequency: Every 3 days (automatic)

plex-recent-additions.sh

Recent media additions reporting

Features:

• Recent additions reporting
• Media library analysis
• Usage statistics

Usage:

./plex-recent-additions.sh          # Generate recent additions report

cleanup-plex-databases.sh ⭐ NEW

Safe cleanup of temporary and corrupted database files

Features:

• Removes temporary files, recovery files, and corrupted directories
• Preserves active databases and legitimate backups
• Dry-run mode for safe preview
• Comprehensive safety checks and validation

Usage:

./cleanup-plex-databases.sh --dry-run   # Preview what will be removed
sudo ./cleanup-plex-databases.sh        # Perform actual cleanup
./cleanup-plex-databases.sh --help      # Show all options

What it removes:

• Temporary files (*-tmp, *-2025-*-tmp)
• MD5 checksum files (*.md5)
• Recovery files (*recovery*)
• Repaired files (*repaired*)
• Empty backup files (*empty-backup*)
• Repair log files (DBRepair.log)
• Corrupted database directories (corrupted-*)

What it preserves:

• Active database files (*.db, *.db-shm, *.db-wal)
• Plex built-in backups (*.backup.*)
• Historical dated backups (*.db.YYYYMMDD)
• EPG databases (tv.plex.providers.epg.cloud-*.db)

🧪 Testing & Integration

test-plex-backup.sh

Comprehensive testing suite

Features:

• Unit testing for individual backup components
• Integration testing for full backup workflows
• Database integrity test scenarios
• Performance benchmarking
• Error condition simulation and recovery testing

Usage:

./test-plex-backup.sh              # Run full test suite
./test-plex-backup.sh --unit       # Unit tests only
./test-plex-backup.sh --integration # Integration tests only
./test-plex-backup.sh --quick      # Quick smoke tests
./test-plex-backup.sh --help       # Show all options

Example Run:

$ ./test-plex-backup.sh --quick
[TEST] Quick smoke test suite
[PASS] Database access test
[PASS] Service management test
[PASS] Backup directory access test
[PASS] All 3 tests passed in 15 seconds

integration-test-plex.sh

End-to-end integration testing

Features:

• Complete backup/restore workflow testing
• Multi-system compatibility testing
• Performance regression testing
• Stress testing and load simulation

Usage:

./integration-test-plex.sh         # Full integration test suite
./integration-test-plex.sh --stress # Stress testing

validate-plex-recovery.sh

Recovery validation and verification

Features:

• Database integrity verification
• Service functionality testing
• Library accessibility checks
• Performance validation

Usage:

./validate-plex-recovery.sh        # Full validation suite
./validate-plex-recovery.sh --quick # Quick validation checks

📁 Deprecated Scripts

Scripts moved to deprecated/ folder (June 21, 2025):

• plex-database-repair.sh → Use plex-db-manager.sh
• recover-plex-database.sh → Use nuclear-plex-recovery.sh
• restore-plex.sh → Use nuclear-plex-recovery.sh
• icu-aware-recovery.sh → Functionality built into other scripts

See deprecated/README.md for details and emergency access.

🏗️ Current System Architecture

Active Script Relationships

graph TD
    A[plex-db-manager.sh] --> B[Database Health Checks]
    C[backup-plex.sh] --> D[validate-plex-backups.sh]
    C --> E[monitor-plex-backup.sh]
    F[nuclear-plex-recovery.sh] --> G[validate-plex-recovery.sh]
    H[plex.sh] --> A
    H --> C
    H --> F
    I[test-plex-backup.sh] --> C
    I --> A
    J[integration-test-plex.sh] --> C
    K[plex-recent-additions.sh]

    style A fill:#e1f5fe
    style C fill:#f3e5f5
    style F fill:#ffebee
    style H fill:#e8f5e8

Legend:

• 🔵 Blue (plex-db-manager.sh): Safe database operations
• 🟣 Purple (backup-plex.sh): Backup operations
• 🔴 Red (nuclear-plex-recovery.sh): Emergency recovery
• 🟢 Green (plex.sh): Service management hub

Current Data Flow

With Plex Built-in Backups (Current System):

1. Automated Operations:

   • Plex Built-in Backups: Every 3 days (automatic, no user intervention)
   • Service Monitoring: plex.sh status for operational checks (manual)

2. Manual Health Monitoring (as needed):

   • Database Health: plex-db-manager.sh check for manual assessment
   • Service Status: plex.sh status for service verification

3. Emergency Recovery Workflow (unchanged):

   • Issue Detection: Service failures or manual observation
   • Initial Assessment: plex-db-manager.sh check to diagnose problems
   • Service Recovery: plex.sh restart for service-level issues
   • Nuclear Recovery: nuclear-plex-recovery.sh --auto for database corruption
   • Post-Recovery Verification: validate-plex-recovery.sh confirms recovery

Built-in Backup Management:

# Check latest Plex built-in backup
ls -lath "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.* | head -1

# Monitor backup frequency (should be every 3 days)
ls -la "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.*

🚫 Disabled Workflows:

• Daily custom backup creation → Replaced by Plex built-in
• Backup validation and monitoring → Not needed for built-in backups
• Performance metrics collection → Plex handles internally

🔧 Configuration & Setup

Cron Schedule (Current)

Plex Built-in Backup System Active (June 21, 2025):

# Plex automatically creates backups every 3 days
# Location: /var/lib/plexmediaserver/.../Databases/*.backup.*
# No manual cron jobs needed for backups

# Optional: Weekly database health check (currently disabled)
# 0 6 * * 0 /home/acedanger/shell/plex/plex-db-manager.sh check

# DISABLED - Custom backup system (replaced by Plex built-in)
# 15 4 * * * /home/acedanger/shell/plex/backup-plex.sh --non-interactive --disable-auto-repair
# 0 7 * * * /home/acedanger/shell/plex/validate-plex-backups.sh --fix

✅ Current Active Schedule:

• Plex Built-in Backups: Every 3 days (automatic)
• Custom Scripts: Disabled (manual use only)

⚠️ Previous schedule (DISABLED):

• Daily integrity check → No longer needed
• Daily custom backup → Replaced by Plex built-in
• Daily backup validation → Not applicable to built-in backups
• Every 30 minutes auto-repair → Fixed: Caused database corruption

Environment Configuration

• Backup Location: /mnt/share/media/backups/plex
• Log Location: ./logs/ (local) and /mnt/share/media/backups/logs (shared)
• Plex Database Path: /var/lib/plexmediaserver/Library/Application Support/Plex Media Server/
• Service Name: plexmediaserver

Notification Configuration

Scripts support multiple notification channels:

• Webhook notifications: Custom webhook URL support
• Email notifications: Via sendmail (if configured)
• Console output: Color-coded status messages

📊 Monitoring & Health Checks

Current Health Status

✅ Database Corruption Issues Resolved (June 21, 2025)

• Root cause: 30-minute auto-repair cron job
• Solution: Disabled aggressive auto-repair schedule + switched to Plex built-in backups
• Current status: All databases healthy, Plex built-in backups active

✅ Backup System Status (June 21, 2025)

• Plex Built-in Backups: ✅ Active (every 3 days)
• Custom Backup Scripts: ❌ Disabled (available for manual use)
• Backup Location: /mnt/share/media/backups/plex (NAS mounted)

Latest Built-in Backup Check:

# Check when Plex last created a backup
ls -lath "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.* | head -1

Key Metrics Tracked

• Backup success/failure rates
• Database integrity status
• Service uptime and performance
• Disk space utilization
• Recovery operation success

🚨 Emergency Procedures

Database Issues

Step-by-step troubleshooting:

1. Quick Health Check (safe)

   ./plex-db-manager.sh check
   

2. Review Service Status

   ./plex.sh status
   systemctl status plexmediaserver
   

3. Check Recent Logs

   tail -f logs/plex-backup-$(date +%Y-%m-%d).log
   journalctl -u plexmediaserver -f
   

4. Service Recovery (if needed)

   ./plex.sh restart
   ./plex-db-manager.sh check  # Verify after restart
   

5. Nuclear Recovery (LAST RESORT)

   # Always dry-run first
   sudo ./nuclear-plex-recovery.sh --dry-run
   
   # If satisfied with plan:
   sudo ./nuclear-plex-recovery.sh --auto
   
   # Validate recovery
   ./validate-plex-recovery.sh
   

Service Issues

Diagnostic steps:

1. Basic Status Check

   ./plex.sh status
   ps aux | grep -i plex
   

2. Port and Network Check

   netstat -tlnp | grep :32400
   curl -I http://localhost:32400/web/
   

3. Safe Service Restart

   ./plex.sh stop    # Graceful shutdown
   sleep 5           # Allow complete shutdown
   ./plex.sh start   # Clean start
   

4. Database Verification (after service issues)

   ./plex-db-manager.sh check
   

📚 Documentation Files

Available Guides

• BACKUP-SYSTEM-TRANSITION.md - Transition from custom to Plex built-in backups
• CORRUPTION-RESOLUTION-SUMMARY.md - Root cause analysis and resolution
• plex-backup.md - Detailed backup system documentation (legacy)
• plex-management.md - Service management procedures
• deprecated/README.md - Information about retired scripts

Key Documentation

• Primary Tool: plex-db-manager.sh - New consolidated database management
• Backup Strategy: Conservative approach with manual repair intervention
• Recovery Method: Nuclear recovery for emergency situations
• Monitoring: Comprehensive logging and health tracking

🏷️ Version Information

• Script Suite Version: 3.0 (Streamlined)
• Major Update: June 21, 2025 - Corruption issues resolved
• Author: Peter Wood peter@peterwood.dev
• Compatibility: Ubuntu 20.04+, Debian 11+
• Plex Version: Compatible with Plex Media Server 1.25+

📞 Support & Changes

Recent Major Changes (June 21, 2025):

• ✅ Fixed database corruption root cause (30-minute auto-repair loop)
• ✅ DISABLED custom backup system - Replaced with Plex built-in backups
• ✅ Retired redundant repair scripts to deprecated/ folder
• ✅ Implemented Plex native scheduled backups (every 3 days)
• ✅ Created consolidated database manager (plex-db-manager.sh)
• ✅ Enhanced service management with safer shutdown procedures

💡 Best Practices

Current Recommended Workflow (Updated)

1. Let Plex Handle Backups:

   • ✅ Plex built-in backups run automatically every 3 days
   • ✅ No manual intervention needed for routine backups
   • ✅ Backups are stored in NAS location: /mnt/share/media/backups/plex

2. Manual Health Monitoring (as needed):

   • Use plex-db-manager.sh check for occasional health checks
   • Use plex.sh status for service status verification
   • Monitor system logs if issues are suspected

3. When Issues Occur:

   • Always start with read-only checks (plex-db-manager.sh check)
   • Service issues first: plex.sh restart before database interventions
   • Nuclear recovery as last resort with proper validation

4. Backup Verification:

   # Check Plex built-in backup status
   ls -lath "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/"*.backup.*
   
   # Verify backup location on NAS
   ls -lah /mnt/share/media/backups/plex/
   

What NOT to Do

❌ Don't re-enable daily custom backup scripts (conflicts with Plex built-in) ❌ Don't enable 30-minute auto-repair schedules (causes corruption) ❌ Don't use force-kill on Plex processes (corrupts databases) ❌ Don't bypass safety checks in scripts ❌ Don't run multiple repair operations simultaneously ❌ Don't ignore failed service shutdowns (wait for graceful exit)

For issues or questions:

• Author: Peter Wood peter@peterwood.dev
• Repository: Part of comprehensive shell script collection