mirror of
https://github.com/acedanger/shell.git
synced 2025-12-06 07:50:11 -08:00
9b83924597
- Introduced `recover-plex-database.sh` for comprehensive database recovery with multiple strategies, logging, and rollback capabilities. - Added `restore-plex.sh` for safe restoration of Plex backups, including validation and dry-run options. - Created `plex-db-manager.sh` to consolidate database management functionalities, including integrity checks and service management. - Enhanced logging and error handling across all scripts for better user feedback and troubleshooting. - Implemented safety measures to prevent running scripts as root and ensure proper service management during operations.
131 lines
4.1 KiB
Markdown
131 lines
4.1 KiB
Markdown
# Plex Database Corruption Resolution Summary
|
|
|
|
## ✅ ISSUE RESOLVED: Auto-Repair Cycle Causing Corruption
|
|
|
|
### Root Cause Identified
|
|
The primary cause of your Plex database corruption was an **aggressive auto-repair schedule** running every 30 minutes via cron:
|
|
|
|
```bash
|
|
# PROBLEMATIC (FIXED):
|
|
*/30 * * * * /home/acedanger/shell/plex/backup-plex.sh --check-integrity --auto-repair
|
|
```
|
|
|
|
This caused:
|
|
- 48+ service stops/starts per day
|
|
- WAL file manipulation conflicts
|
|
- Repair cascading failures
|
|
- Race conditions during service transitions
|
|
|
|
### ✅ Changes Applied
|
|
|
|
#### 1. **Fixed Crontab Schedule**
|
|
- **Before**: Auto-repair every 30 minutes + daily backup with auto-repair
|
|
- **After**:
|
|
- Daily read-only integrity check (6 AM)
|
|
- Daily backup with auto-repair **disabled** (4:15 AM)
|
|
- Manual repair intervention required
|
|
|
|
#### 2. **Disabled Auto-Repair Default**
|
|
- Changed `backup-plex.sh` default from `AUTO_REPAIR=true` to `AUTO_REPAIR=false`
|
|
- Prevents automatic repair loops that were causing corruption
|
|
|
|
#### 3. **Created Consolidated Management Tool**
|
|
- New script: `plex-db-manager.sh`
|
|
- Safe, read-only integrity checking
|
|
- Manual repair intervention (currently disabled for safety)
|
|
- Proper service management with synchronization
|
|
|
|
#### 4. **Database Status** ✅
|
|
Current check shows: **ALL DATABASES HEALTHY**
|
|
- Main database: integrity check PASSED
|
|
- Blobs database: integrity check PASSED
|
|
|
|
## 📋 Script Redundancy Analysis
|
|
|
|
### Scripts with Overlapping Functionality
|
|
|
|
1. **`plex.sh`** - Service management + basic repair
|
|
2. **`backup-plex.sh`** - Backup + auto-repair logic
|
|
3. **`plex-database-repair.sh`** - Dedicated repair functions
|
|
4. **`recover-plex-database.sh`** - Advanced recovery methods
|
|
5. **`nuclear-plex-recovery.sh`** - Nuclear recovery
|
|
6. **`restore-plex.sh`** - Backup restoration
|
|
|
|
### Consolidation Recommendations
|
|
|
|
#### Keep Active:
|
|
- **`backup-plex.sh`** - Primary backup (with auto-repair disabled)
|
|
- **`plex-db-manager.sh`** - New consolidated management tool
|
|
- **`plex.sh`** - Basic service management
|
|
- **`nuclear-plex-recovery.sh`** - Last resort recovery
|
|
|
|
#### Consider Deprecating:
|
|
- **`plex-database-repair.sh`** - Functionality moved to `plex-db-manager.sh`
|
|
- **`recover-plex-database.sh`** - Similar functionality in other scripts
|
|
- **`restore-plex.sh`** - Basic functionality covered elsewhere
|
|
|
|
## 🛡️ Prevention Measures Implemented
|
|
|
|
### 1. **Conservative Backup Schedule**
|
|
```bash
|
|
# Read-only check (daily at 6 AM)
|
|
0 6 * * * backup-plex.sh --check-integrity --disable-auto-repair
|
|
|
|
# Backup without auto-repair (daily at 4:15 AM)
|
|
15 4 * * * backup-plex.sh --non-interactive --disable-auto-repair
|
|
```
|
|
|
|
### 2. **Manual Intervention Required**
|
|
- No automatic repairs unless explicitly requested
|
|
- All repair operations require manual approval
|
|
- Comprehensive logging for audit trail
|
|
|
|
### 3. **Safe Service Management**
|
|
- Proper service stop/start synchronization
|
|
- Extended timeouts for clean shutdowns
|
|
- Race condition prevention
|
|
|
|
## 📊 Expected Improvements
|
|
|
|
1. **Stability**: Eliminated 47 daily service interruptions
|
|
2. **Reliability**: No more auto-repair corruption loops
|
|
3. **Performance**: Reduced I/O load on database files
|
|
4. **Maintainability**: Centralized database management
|
|
|
|
## 🔧 Usage Going Forward
|
|
|
|
### Regular Monitoring:
|
|
```bash
|
|
# Check database health (safe, read-only)
|
|
./plex-db-manager.sh check
|
|
```
|
|
|
|
### If Issues Detected:
|
|
```bash
|
|
# View detailed logs
|
|
tail -f /home/acedanger/shell/plex/logs/plex-backup-$(date +%Y-%m-%d).log
|
|
|
|
# Manual repair (when re-enabled)
|
|
./plex-db-manager.sh repair
|
|
```
|
|
|
|
### Emergency Recovery:
|
|
```bash
|
|
# Only if all else fails
|
|
sudo ./nuclear-plex-recovery.sh --auto
|
|
```
|
|
|
|
## ⚠️ Critical Notes
|
|
|
|
1. **Auto-repair is temporarily disabled** until stability is confirmed
|
|
2. **Manual intervention required** for any database issues
|
|
3. **Monitor logs closely** for the next week to ensure stability
|
|
4. **Backup integrity** should improve significantly
|
|
|
|
---
|
|
|
|
**Date Fixed**: June 21, 2025
|
|
**Issue**: 30-minute auto-repair cycle causing database corruption
|
|
**Resolution**: Disabled aggressive auto-repair, implemented safe backup schedule
|
|
**Status**: ✅ RESOLVED - Databases currently healthy
|