mirror of
https://github.com/acedanger/shell.git
synced 2026-03-24 20:11:49 -07:00
Compare commits
3 Commits
4ce77211b5
...
9bb99aecbf
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9bb99aecbf | ||
|
|
ddaa641668 | ||
|
|
b47f58fad7 |
@@ -1,130 +0,0 @@
|
||||
# Plex Database Corruption Resolution Summary
|
||||
|
||||
## ✅ ISSUE RESOLVED: Auto-Repair Cycle Causing Corruption
|
||||
|
||||
### Root Cause Identified
|
||||
The primary cause of your Plex database corruption was an **aggressive auto-repair schedule** running every 30 minutes via cron:
|
||||
|
||||
```bash
|
||||
# PROBLEMATIC (FIXED):
|
||||
*/30 * * * * /home/acedanger/shell/plex/backup-plex.sh --check-integrity --auto-repair
|
||||
```
|
||||
|
||||
This caused:
|
||||
- 48+ service stops/starts per day
|
||||
- WAL file manipulation conflicts
|
||||
- Repair cascading failures
|
||||
- Race conditions during service transitions
|
||||
|
||||
### ✅ Changes Applied
|
||||
|
||||
#### 1. **Fixed Crontab Schedule**
|
||||
- **Before**: Auto-repair every 30 minutes + daily backup with auto-repair
|
||||
- **After**:
|
||||
- Daily read-only integrity check (6 AM)
|
||||
- Daily backup with auto-repair **disabled** (4:15 AM)
|
||||
- Manual repair intervention required
|
||||
|
||||
#### 2. **Disabled Auto-Repair Default**
|
||||
- Changed `backup-plex.sh` default from `AUTO_REPAIR=true` to `AUTO_REPAIR=false`
|
||||
- Prevents automatic repair loops that were causing corruption
|
||||
|
||||
#### 3. **Created Consolidated Management Tool**
|
||||
- New script: `plex-db-manager.sh`
|
||||
- Safe, read-only integrity checking
|
||||
- Manual repair intervention (currently disabled for safety)
|
||||
- Proper service management with synchronization
|
||||
|
||||
#### 4. **Database Status** ✅
|
||||
Current check shows: **ALL DATABASES HEALTHY**
|
||||
- Main database: integrity check PASSED
|
||||
- Blobs database: integrity check PASSED
|
||||
|
||||
## 📋 Script Redundancy Analysis
|
||||
|
||||
### Scripts with Overlapping Functionality
|
||||
|
||||
1. **`plex.sh`** - Service management + basic repair
|
||||
2. **`backup-plex.sh`** - Backup + auto-repair logic
|
||||
3. **`plex-database-repair.sh`** - Dedicated repair functions
|
||||
4. **`recover-plex-database.sh`** - Advanced recovery methods
|
||||
5. **`nuclear-plex-recovery.sh`** - Nuclear recovery
|
||||
6. **`restore-plex.sh`** - Backup restoration
|
||||
|
||||
### Consolidation Recommendations
|
||||
|
||||
#### Keep Active:
|
||||
- **`backup-plex.sh`** - Primary backup (with auto-repair disabled)
|
||||
- **`plex-db-manager.sh`** - New consolidated management tool
|
||||
- **`plex.sh`** - Basic service management
|
||||
- **`nuclear-plex-recovery.sh`** - Last resort recovery
|
||||
|
||||
#### Consider Deprecating:
|
||||
- **`plex-database-repair.sh`** - Functionality moved to `plex-db-manager.sh`
|
||||
- **`recover-plex-database.sh`** - Similar functionality in other scripts
|
||||
- **`restore-plex.sh`** - Basic functionality covered elsewhere
|
||||
|
||||
## 🛡️ Prevention Measures Implemented
|
||||
|
||||
### 1. **Conservative Backup Schedule**
|
||||
```bash
|
||||
# Read-only check (daily at 6 AM)
|
||||
0 6 * * * backup-plex.sh --check-integrity --disable-auto-repair
|
||||
|
||||
# Backup without auto-repair (daily at 4:15 AM)
|
||||
15 4 * * * backup-plex.sh --non-interactive --disable-auto-repair
|
||||
```
|
||||
|
||||
### 2. **Manual Intervention Required**
|
||||
- No automatic repairs unless explicitly requested
|
||||
- All repair operations require manual approval
|
||||
- Comprehensive logging for audit trail
|
||||
|
||||
### 3. **Safe Service Management**
|
||||
- Proper service stop/start synchronization
|
||||
- Extended timeouts for clean shutdowns
|
||||
- Race condition prevention
|
||||
|
||||
## 📊 Expected Improvements
|
||||
|
||||
1. **Stability**: Eliminated 47 daily service interruptions
|
||||
2. **Reliability**: No more auto-repair corruption loops
|
||||
3. **Performance**: Reduced I/O load on database files
|
||||
4. **Maintainability**: Centralized database management
|
||||
|
||||
## 🔧 Usage Going Forward
|
||||
|
||||
### Regular Monitoring:
|
||||
```bash
|
||||
# Check database health (safe, read-only)
|
||||
./plex-db-manager.sh check
|
||||
```
|
||||
|
||||
### If Issues Detected:
|
||||
```bash
|
||||
# View detailed logs
|
||||
tail -f /home/acedanger/shell/plex/logs/plex-backup-$(date +%Y-%m-%d).log
|
||||
|
||||
# Manual repair (when re-enabled)
|
||||
./plex-db-manager.sh repair
|
||||
```
|
||||
|
||||
### Emergency Recovery:
|
||||
```bash
|
||||
# Only if all else fails
|
||||
sudo ./nuclear-plex-recovery.sh --auto
|
||||
```
|
||||
|
||||
## ⚠️ Critical Notes
|
||||
|
||||
1. **Auto-repair is temporarily disabled** until stability is confirmed
|
||||
2. **Manual intervention required** for any database issues
|
||||
3. **Monitor logs closely** for the next week to ensure stability
|
||||
4. **Backup integrity** should improve significantly
|
||||
|
||||
---
|
||||
|
||||
**Date Fixed**: June 21, 2025
|
||||
**Issue**: 30-minute auto-repair cycle causing database corruption
|
||||
**Resolution**: Disabled aggressive auto-repair, implemented safe backup schedule
|
||||
**Status**: ✅ RESOLVED - Databases currently healthy
|
||||
@@ -66,6 +66,7 @@ NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
PLEX_DB_DIR="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases"
|
||||
PLEX_SQLITE="/usr/lib/plexmediaserver/Plex SQLite"
|
||||
PLEX_USER="plex"
|
||||
PLEX_GROUP="plex"
|
||||
BACKUP_TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
|
||||
@@ -244,27 +245,66 @@ restore_from_backup() {
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to verify restored databases
|
||||
# Function to verify restored databases (structural + FTS)
|
||||
verify_databases() {
|
||||
print_status "$YELLOW" "Verifying restored databases..."
|
||||
|
||||
# Check main database
|
||||
if sqlite3 "${PLEX_DB_DIR}/com.plexapp.plugins.library.db" "PRAGMA integrity_check;" | grep -q "ok"; then
|
||||
print_status "$GREEN" "Main database integrity check: PASSED"
|
||||
else
|
||||
print_status "$RED" "Main database integrity check: FAILED"
|
||||
return 1
|
||||
# Use Plex's bundled SQLite for ICU compatibility; fall back to system sqlite3
|
||||
local sqlite_bin="sqlite3"
|
||||
if [[ -x "$PLEX_SQLITE" ]]; then
|
||||
sqlite_bin="$PLEX_SQLITE"
|
||||
fi
|
||||
|
||||
# Check blobs database
|
||||
if sqlite3 "${PLEX_DB_DIR}/com.plexapp.plugins.library.blobs.db" "PRAGMA integrity_check;" | grep -q "ok"; then
|
||||
print_status "$GREEN" "Blobs database integrity check: PASSED"
|
||||
local overall_ok=true
|
||||
|
||||
for db_file in \
|
||||
"${PLEX_DB_DIR}/com.plexapp.plugins.library.db" \
|
||||
"${PLEX_DB_DIR}/com.plexapp.plugins.library.blobs.db"; do
|
||||
|
||||
local db_name
|
||||
db_name=$(basename "$db_file")
|
||||
|
||||
if [[ ! -f "$db_file" ]]; then
|
||||
print_status "$RED" "$db_name: NOT FOUND"
|
||||
overall_ok=false
|
||||
continue
|
||||
fi
|
||||
|
||||
# Structural integrity
|
||||
local result
|
||||
result=$("$sqlite_bin" "$db_file" "PRAGMA integrity_check;" 2>&1)
|
||||
if [[ "$result" == "ok" ]]; then
|
||||
print_status "$GREEN" "$db_name structural integrity: PASSED"
|
||||
else
|
||||
print_status "$RED" "$db_name structural integrity: FAILED"
|
||||
overall_ok=false
|
||||
fi
|
||||
|
||||
# FTS index integrity
|
||||
local fts_tables
|
||||
fts_tables=$("$sqlite_bin" "$db_file" \
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND sql LIKE '%fts%';" 2>/dev/null) || true
|
||||
if [[ -n "$fts_tables" ]]; then
|
||||
while IFS= read -r table; do
|
||||
[[ -z "$table" ]] && continue
|
||||
local fts_result
|
||||
fts_result=$("$sqlite_bin" "$db_file" \
|
||||
"INSERT INTO ${table}(${table}) VALUES('integrity-check');" 2>&1) || true
|
||||
if [[ -n "$fts_result" ]]; then
|
||||
print_status "$RED" "$db_name FTS index '$table': DAMAGED"
|
||||
overall_ok=false
|
||||
fi
|
||||
done <<< "$fts_tables"
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ "$overall_ok" == true ]]; then
|
||||
print_status "$GREEN" "All database integrity checks passed!"
|
||||
return 0
|
||||
else
|
||||
print_status "$RED" "Blobs database integrity check: FAILED"
|
||||
print_status "$RED" "One or more database checks failed!"
|
||||
return 1
|
||||
fi
|
||||
|
||||
print_status "$GREEN" "All database integrity checks passed!"
|
||||
}
|
||||
|
||||
# Function to fix ownership issues
|
||||
|
||||
@@ -52,10 +52,26 @@ readonly MAIN_DB="$PLEX_DB_DIR/com.plexapp.plugins.library.db"
|
||||
readonly BLOBS_DB="$PLEX_DB_DIR/com.plexapp.plugins.library.blobs.db"
|
||||
readonly BACKUP_ROOT="/mnt/share/media/backups/plex"
|
||||
|
||||
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
|
||||
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
|
||||
readonly SCRIPT_DIR
|
||||
readonly LOG_FILE="$SCRIPT_DIR/logs/db-manager-$(date +%Y%m%d_%H%M%S).log"
|
||||
|
||||
# DBRepair.sh location — searched in order: script dir, database dir, /usr/local/bin
|
||||
find_dbrepair() {
|
||||
local candidates=(
|
||||
"${SCRIPT_DIR}/DBRepair.sh"
|
||||
"${PLEX_DB_DIR}/DBRepair.sh"
|
||||
"/usr/local/bin/DBRepair.sh"
|
||||
)
|
||||
for path in "${candidates[@]}"; do
|
||||
if [[ -x "$path" ]]; then
|
||||
echo "$path"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# Create log directory
|
||||
mkdir -p "$SCRIPT_DIR/logs"
|
||||
|
||||
@@ -210,26 +226,54 @@ check_database_integrity() {
|
||||
fi
|
||||
fi
|
||||
|
||||
# Run integrity check
|
||||
# Run structural integrity check
|
||||
local integrity_result
|
||||
integrity_result=$(sudo "$PLEX_SQLITE" "$db_file" "PRAGMA integrity_check;" 2>&1)
|
||||
local check_exit_code=$?
|
||||
|
||||
if [[ $check_exit_code -ne 0 ]]; then
|
||||
log_error "Failed to run integrity check on $db_name"
|
||||
if [[ $check_exit_code -ne 0 && -z "$integrity_result" ]]; then
|
||||
log_error "Failed to open database: $db_name (exit code $check_exit_code)"
|
||||
return 1
|
||||
fi
|
||||
|
||||
if echo "$integrity_result" | grep -q "^ok$"; then
|
||||
log_success "Database integrity check passed: $db_name"
|
||||
return 0
|
||||
local struct_ok=true
|
||||
if [[ "$integrity_result" == "ok" ]]; then
|
||||
log_success "Structural integrity check passed: $db_name"
|
||||
else
|
||||
log_warning "Database integrity issues detected in $db_name:"
|
||||
echo "$integrity_result" | while IFS= read -r line; do
|
||||
struct_ok=false
|
||||
log_warning "Structural integrity issues detected in $db_name:"
|
||||
echo "$integrity_result" | head -n 10 | while IFS= read -r line; do
|
||||
log_warning " $line"
|
||||
done
|
||||
return 1
|
||||
fi
|
||||
|
||||
# FTS (Full-Text Search) index integrity check
|
||||
# Standard PRAGMA integrity_check does NOT detect FTS corruption.
|
||||
local fts_ok=true
|
||||
local fts_tables
|
||||
fts_tables=$(sudo "$PLEX_SQLITE" "$db_file" \
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND sql LIKE '%fts%';" 2>/dev/null) || true
|
||||
|
||||
if [[ -n "$fts_tables" ]]; then
|
||||
log_message "Checking FTS (Full-Text Search) indexes in $db_name..."
|
||||
while IFS= read -r table; do
|
||||
[[ -z "$table" ]] && continue
|
||||
local fts_result
|
||||
fts_result=$(sudo "$PLEX_SQLITE" "$db_file" \
|
||||
"INSERT INTO ${table}(${table}) VALUES('integrity-check');" 2>&1) || true
|
||||
if [[ -n "$fts_result" ]]; then
|
||||
fts_ok=false
|
||||
log_warning "FTS index '${table}' — DAMAGED: $fts_result"
|
||||
else
|
||||
log_success "FTS index '${table}' — OK"
|
||||
fi
|
||||
done <<< "$fts_tables"
|
||||
fi
|
||||
|
||||
if [[ "$struct_ok" == true && "$fts_ok" == true ]]; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# Check all databases
|
||||
@@ -315,20 +359,73 @@ main() {
|
||||
;;
|
||||
|
||||
"repair")
|
||||
echo -e "${RED}${BOLD}⚠️ REPAIR FUNCTIONALITY TEMPORARILY DISABLED${RESET}"
|
||||
echo -e "${YELLOW}Database repairs are disabled until corruption issues are resolved.${RESET}"
|
||||
echo -e "${CYAN}Use the individual repair scripts if manual intervention is needed:${RESET}"
|
||||
echo -e " ${DIM}- plex-database-repair.sh${RESET}"
|
||||
echo -e " ${DIM}- recover-plex-database.sh${RESET}"
|
||||
echo -e " ${DIM}- nuclear-plex-recovery.sh${RESET}"
|
||||
exit 2
|
||||
print_header
|
||||
check_prerequisites
|
||||
|
||||
local dbrepair_bin
|
||||
if dbrepair_bin=$(find_dbrepair); then
|
||||
log_success "Found DBRepair.sh: $dbrepair_bin"
|
||||
log_message "Running: stop → auto (check + repair + reindex + FTS rebuild) → start → exit"
|
||||
if sudo "$dbrepair_bin" stop auto start exit; then
|
||||
log_success "DBRepair automatic repair completed successfully"
|
||||
exit 0
|
||||
else
|
||||
log_error "DBRepair automatic repair failed"
|
||||
exit 2
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}${BOLD}⚠️ DBRepair.sh NOT FOUND${RESET}"
|
||||
echo -e "${YELLOW}Install DBRepair for repair/optimize/reindex/FTS-rebuild:${RESET}"
|
||||
echo -e " ${CYAN}wget -O ${SCRIPT_DIR}/DBRepair.sh https://github.com/ChuckPa/PlexDBRepair/releases/latest/download/DBRepair.sh${RESET}"
|
||||
echo -e " ${CYAN}chmod +x ${SCRIPT_DIR}/DBRepair.sh${RESET}"
|
||||
echo -e "${YELLOW}Then re-run: $(basename "$0") repair${RESET}"
|
||||
exit 2
|
||||
fi
|
||||
;;
|
||||
|
||||
"nuclear")
|
||||
echo -e "${RED}${BOLD}⚠️ NUCLEAR RECOVERY TEMPORARILY DISABLED${RESET}"
|
||||
echo -e "${YELLOW}Nuclear recovery is disabled until corruption issues are resolved.${RESET}"
|
||||
echo -e "${CYAN}Use nuclear-plex-recovery.sh directly if absolutely necessary.${RESET}"
|
||||
exit 2
|
||||
print_header
|
||||
check_prerequisites
|
||||
|
||||
echo -e "\n${RED}${BOLD}⚠️ WARNING: NUCLEAR RECOVERY ⚠️${RESET}"
|
||||
echo -e "${RED}This replaces your Plex database with the best available PMS backup!${RESET}"
|
||||
echo -e "${YELLOW}All changes since the backup was created will be lost.${RESET}\n"
|
||||
|
||||
echo -e "${CYAN}Type 'YES' to proceed: ${RESET}"
|
||||
read -r confirmation
|
||||
if [[ "$confirmation" != "YES" ]]; then
|
||||
log_message "Nuclear recovery cancelled by user"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
local dbrepair_bin
|
||||
if dbrepair_bin=$(find_dbrepair); then
|
||||
log_success "Found DBRepair.sh: $dbrepair_bin"
|
||||
log_message "Running: stop → replace → reindex → start → exit"
|
||||
if sudo "$dbrepair_bin" stop replace reindex start exit; then
|
||||
log_success "Nuclear recovery (replace from backup) completed"
|
||||
exit 0
|
||||
else
|
||||
log_error "Nuclear recovery failed"
|
||||
exit 2
|
||||
fi
|
||||
else
|
||||
# Fallback to dedicated nuclear script
|
||||
local nuclear_script="${SCRIPT_DIR}/nuclear-plex-recovery.sh"
|
||||
if [[ -x "$nuclear_script" ]]; then
|
||||
log_message "DBRepair not found, falling back to nuclear-plex-recovery.sh"
|
||||
if sudo "$nuclear_script" --auto; then
|
||||
log_success "Nuclear recovery completed"
|
||||
exit 0
|
||||
else
|
||||
log_error "Nuclear recovery failed"
|
||||
exit 2
|
||||
fi
|
||||
else
|
||||
log_error "Neither DBRepair.sh nor nuclear-plex-recovery.sh found"
|
||||
exit 2
|
||||
fi
|
||||
fi
|
||||
;;
|
||||
|
||||
"help"|"--help"|"-h")
|
||||
|
||||
378
plex/plex.sh
378
plex/plex.sh
@@ -70,11 +70,29 @@ readonly RESET='\033[0m'
|
||||
|
||||
# 🔧 Configuration
|
||||
readonly PLEX_SERVICE="plexmediaserver"
|
||||
readonly PLEX_SQLITE="/usr/lib/plexmediaserver/Plex SQLite"
|
||||
readonly PLEX_DB_DIR="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases"
|
||||
SCRIPT_NAME="$(basename "$0")"
|
||||
readonly SCRIPT_NAME
|
||||
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
|
||||
readonly SCRIPT_DIR
|
||||
|
||||
# DBRepair.sh location — searched in order: script dir, database dir, /usr/local/bin
|
||||
find_dbrepair() {
|
||||
local candidates=(
|
||||
"${SCRIPT_DIR}/DBRepair.sh"
|
||||
"${PLEX_DB_DIR}/DBRepair.sh"
|
||||
"/usr/local/bin/DBRepair.sh"
|
||||
)
|
||||
for path in "${candidates[@]}"; do
|
||||
if [[ -x "$path" ]]; then
|
||||
echo "$path"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# 🎭 ASCII symbols for compatible output
|
||||
readonly CHECKMARK="[✓]"
|
||||
readonly CROSS="[✗]"
|
||||
@@ -114,218 +132,228 @@ show_loading() {
|
||||
printf "\r%s%s %s %s%s\n" "${CYAN}" "${HOURGLASS}" "${message}" "${CHECKMARK}" "${RESET}"
|
||||
}
|
||||
|
||||
# 🔧 Enhanced function to repair database issues
|
||||
# 🔧 Repair database using ChuckPa/DBRepair when available, else manual steps
|
||||
# DBRepair: https://github.com/ChuckPa/PlexDBRepair
|
||||
repair_database() {
|
||||
print_status "${INFO}" "Attempting to repair Plex database..." "${BLUE}"
|
||||
|
||||
local db_dir="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases"
|
||||
local main_db="$db_dir/com.plexapp.plugins.library.db"
|
||||
local backup_db="$db_dir/com.plexapp.plugins.library.db.backup.$(date +%Y%m%d_%H%M%S)"
|
||||
local corrupted_dir="$db_dir/corrupted-$(date +%Y%m%d_%H%M%S)"
|
||||
local main_db="$PLEX_DB_DIR/com.plexapp.plugins.library.db"
|
||||
|
||||
if [[ ! -f "$main_db" ]]; then
|
||||
print_status "${CROSS}" "Main database not found at: $main_db" "${RED}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# ---------- Try DBRepair.sh (preferred) ----------
|
||||
local dbrepair_bin
|
||||
if dbrepair_bin=$(find_dbrepair); then
|
||||
print_status "${CHECKMARK}" "Found DBRepair.sh: $dbrepair_bin" "${GREEN}"
|
||||
print_status "${INFO}" "Running: stop → auto (check + repair + reindex + FTS rebuild) → start → exit" "${BLUE}"
|
||||
|
||||
# DBRepair supports scripted mode — pass commands directly
|
||||
if sudo "$dbrepair_bin" stop auto start exit; then
|
||||
print_status "${CHECKMARK}" "DBRepair automatic repair completed successfully!" "${GREEN}"
|
||||
return 0
|
||||
else
|
||||
print_status "${CROSS}" "DBRepair automatic repair failed (exit code $?)" "${RED}"
|
||||
print_status "${INFO}" "Falling back to manual repair steps..." "${YELLOW}"
|
||||
fi
|
||||
else
|
||||
print_status "${INFO}" "DBRepair.sh not found — using manual repair" "${YELLOW}"
|
||||
echo -e "${DIM}${CYAN} For better repairs, install DBRepair:${RESET}"
|
||||
echo -e "${DIM}${CYAN} wget -O ${SCRIPT_DIR}/DBRepair.sh https://github.com/ChuckPa/PlexDBRepair/releases/latest/download/DBRepair.sh${RESET}"
|
||||
echo -e "${DIM}${CYAN} chmod +x ${SCRIPT_DIR}/DBRepair.sh${RESET}"
|
||||
fi
|
||||
|
||||
# ---------- Manual fallback (same approach DBRepair uses internally) ----------
|
||||
# Prefer Plex's bundled SQLite for ICU compatibility
|
||||
local sqlite_bin="sqlite3"
|
||||
if [[ -x "$PLEX_SQLITE" ]]; then
|
||||
sqlite_bin="$PLEX_SQLITE"
|
||||
fi
|
||||
|
||||
# Stop Plex service first
|
||||
print_status "${INFO}" "Stopping Plex service..." "${BLUE}"
|
||||
sudo systemctl stop "$PLEX_SERVICE" 2>/dev/null || true
|
||||
sleep 3
|
||||
|
||||
# Check if critical tables exist
|
||||
print_status "${INFO}" "Checking database structure..." "${BLUE}"
|
||||
local has_metadata_table=false
|
||||
if sudo -u plex sqlite3 "$main_db" "SELECT name FROM sqlite_master WHERE type='table' AND name='metadata_items';" 2>/dev/null | grep -q metadata_items; then
|
||||
has_metadata_table=true
|
||||
# Step 1: WAL checkpoint
|
||||
if [[ -f "${main_db}-wal" ]]; then
|
||||
print_status "${INFO}" "Checkpointing WAL journal..." "${BLUE}"
|
||||
sudo -u plex "$sqlite_bin" "$main_db" "PRAGMA wal_checkpoint(TRUNCATE);" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
if [[ "$has_metadata_table" == "false" ]]; then
|
||||
print_status "${CROSS}" "Critical table 'metadata_items' is missing! Database is severely corrupted." "${RED}"
|
||||
print_status "${INFO}" "Attempting recovery from available backups..." "${YELLOW}"
|
||||
# Step 2: Export → import (the core of DBRepair's repair/optimize)
|
||||
local ts
|
||||
ts=$(date +%Y%m%d_%H%M%S)
|
||||
local sql_dump="/tmp/plex_dump_${ts}.sql"
|
||||
local new_db="/tmp/plex_repaired_${ts}.db"
|
||||
local backup_db="$PLEX_DB_DIR/com.plexapp.plugins.library.db-BACKUP-${ts}"
|
||||
|
||||
# Find the best recovery candidate
|
||||
local recovery_db=""
|
||||
local recovery_candidates=(
|
||||
"$db_dir/com.plexapp.plugins.library.db.recovery-"*
|
||||
"$db_dir/com.plexapp.plugins.library.db.20"*
|
||||
)
|
||||
|
||||
for candidate in "${recovery_candidates[@]}"; do
|
||||
if [[ -f "$candidate" && "$candidate" != *"tmp"* && "$candidate" != *"empty"* ]]; then
|
||||
# Test if this candidate has the metadata_items table
|
||||
if sudo -u plex sqlite3 "$candidate" "SELECT name FROM sqlite_master WHERE type='table' AND name='metadata_items';" 2>/dev/null | grep -q metadata_items; then
|
||||
recovery_db="$candidate"
|
||||
break
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ -n "$recovery_db" ]]; then
|
||||
print_status "${CHECKMARK}" "Found recovery database: $(basename "$recovery_db")" "${GREEN}"
|
||||
|
||||
# Move corrupted database to backup location
|
||||
print_status "${INFO}" "Moving corrupted database to backup location..." "${BLUE}"
|
||||
sudo mkdir -p "$corrupted_dir"
|
||||
sudo mv "$main_db" "$corrupted_dir/"
|
||||
sudo mv "$main_db-shm" "$corrupted_dir/" 2>/dev/null || true
|
||||
sudo mv "$main_db-wal" "$corrupted_dir/" 2>/dev/null || true
|
||||
|
||||
# Copy recovery database as new main database
|
||||
print_status "${INFO}" "Restoring database from recovery file..." "${BLUE}"
|
||||
if sudo cp "$recovery_db" "$main_db"; then
|
||||
sudo chown plex:plex "$main_db"
|
||||
sudo chmod 644 "$main_db"
|
||||
print_status "${CHECKMARK}" "Database restored successfully!" "${GREEN}"
|
||||
|
||||
# Verify the restored database
|
||||
print_status "${INFO}" "Verifying restored database..." "${BLUE}"
|
||||
local integrity_result
|
||||
integrity_result=$(sudo -u plex sqlite3 "$main_db" "PRAGMA integrity_check;" 2>&1)
|
||||
|
||||
if echo "$integrity_result" | grep -q "ok"; then
|
||||
print_status "${CHECKMARK}" "Restored database integrity verified!" "${GREEN}"
|
||||
return 0
|
||||
elif echo "$integrity_result" | grep -q "no such collation sequence: icu"; then
|
||||
print_status "${CROSS}" "ICU collation sequence issue detected!" "${YELLOW}"
|
||||
print_status "${INFO}" "Attempting ICU-aware recovery..." "${BLUE}"
|
||||
|
||||
# Try ICU-aware recovery script
|
||||
local icu_script="${SCRIPT_DIR}/icu-aware-recovery.sh"
|
||||
if [[ -f "$icu_script" ]]; then
|
||||
if "$icu_script" --auto; then
|
||||
print_status "${CHECKMARK}" "ICU-aware recovery completed!" "${GREEN}"
|
||||
return 0
|
||||
else
|
||||
print_status "${CROSS}" "ICU-aware recovery failed!" "${RED}"
|
||||
fi
|
||||
else
|
||||
print_status "${INFO}" "ICU recovery script not found, trying manual fix..." "${YELLOW}"
|
||||
|
||||
# Try to recreate database without ICU dependencies
|
||||
local temp_db="/tmp/plex_temp_$(date +%Y%m%d_%H%M%S).db"
|
||||
print_status "${INFO}" "Attempting to dump and recreate database..." "${BLUE}"
|
||||
|
||||
if sudo -u plex sqlite3 "$recovery_db" ".dump" | grep -v "COLLATE icu_" | sudo -u plex sqlite3 "$temp_db"; then
|
||||
print_status "${INFO}" "Database dump successful, replacing main database..." "${BLUE}"
|
||||
sudo mv "$temp_db" "$main_db"
|
||||
sudo chown plex:plex "$main_db"
|
||||
sudo chmod 644 "$main_db"
|
||||
|
||||
# Verify the recreated database
|
||||
if sudo -u plex sqlite3 "$main_db" "PRAGMA integrity_check;" 2>/dev/null | grep -q "ok"; then
|
||||
print_status "${CHECKMARK}" "Database recreated successfully without ICU!" "${GREEN}"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Clean up temp file if it exists
|
||||
sudo rm -f "$temp_db" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
print_status "${CROSS}" "Failed to resolve ICU collation issues!" "${RED}"
|
||||
return 1
|
||||
else
|
||||
print_status "${CROSS}" "Restored database failed integrity check!" "${RED}"
|
||||
print_status "${INFO}" "Integrity check result:" "${YELLOW}"
|
||||
echo -e "${DIM}${YELLOW} $integrity_result${RESET}"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
print_status "${CROSS}" "Failed to restore database!" "${RED}"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
print_status "${CROSS}" "No valid recovery databases found!" "${RED}"
|
||||
print_status "${INFO}" "Available options:" "${YELLOW}"
|
||||
echo -e "${DIM}${YELLOW} 1. Check manual backups in /mnt/share/media/backups/plex/${RESET}"
|
||||
echo -e "${DIM}${YELLOW} 2. Let Plex rebuild database (will lose all metadata)${RESET}"
|
||||
echo -e "${DIM}${YELLOW} 3. Run: sudo rm '$main_db' && sudo systemctl start plexmediaserver${RESET}"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create backup of current database
|
||||
print_status "${INFO}" "Creating backup of current database..." "${BLUE}"
|
||||
if ! sudo cp "$main_db" "$backup_db"; then
|
||||
print_status "${CROSS}" "Failed to create database backup!" "${RED}"
|
||||
print_status "${INFO}" "Exporting database to SQL..." "${BLUE}"
|
||||
if sudo -u plex "$sqlite_bin" "$main_db" ".dump" | sudo -u plex tee "$sql_dump" >/dev/null 2>&1; then
|
||||
print_status "${CHECKMARK}" "SQL export completed ($(du -sh "$sql_dump" | cut -f1))" "${GREEN}"
|
||||
else
|
||||
print_status "${CROSS}" "SQL export failed — database may be too damaged for this method" "${RED}"
|
||||
rm -f "$sql_dump" 2>/dev/null
|
||||
print_status "${INFO}" "Install DBRepair.sh for more robust repair capabilities" "${YELLOW}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
print_status "${CHECKMARK}" "Database backed up to: $backup_db" "${GREEN}"
|
||||
|
||||
# Try to vacuum the database
|
||||
print_status "${INFO}" "Running VACUUM on database..." "${BLUE}"
|
||||
if sudo -u plex sqlite3 "$main_db" "VACUUM;" 2>/dev/null; then
|
||||
print_status "${CHECKMARK}" "Database VACUUM completed successfully" "${GREEN}"
|
||||
|
||||
# Test integrity again
|
||||
if sudo -u plex sqlite3 "$main_db" "PRAGMA integrity_check;" 2>/dev/null | grep -q "ok"; then
|
||||
print_status "${CHECKMARK}" "Database integrity restored!" "${GREEN}"
|
||||
print_status "${INFO}" "You can now try starting Plex again" "${BLUE}"
|
||||
return 0
|
||||
else
|
||||
print_status "${CROSS}" "Database still corrupted after VACUUM" "${RED}"
|
||||
fi
|
||||
print_status "${INFO}" "Importing into fresh database..." "${BLUE}"
|
||||
if sudo cat "$sql_dump" | sudo -u plex "$sqlite_bin" "$new_db" 2>/dev/null; then
|
||||
print_status "${CHECKMARK}" "Import into fresh database succeeded" "${GREEN}"
|
||||
else
|
||||
print_status "${CROSS}" "VACUUM operation failed" "${RED}"
|
||||
print_status "${CROSS}" "Import failed" "${RED}"
|
||||
rm -f "$sql_dump" "$new_db" 2>/dev/null
|
||||
return 1
|
||||
fi
|
||||
rm -f "$sql_dump" 2>/dev/null
|
||||
|
||||
# Step 3: Verify repaired database (structural + FTS)
|
||||
print_status "${INFO}" "Verifying repaired database..." "${BLUE}"
|
||||
local verify_out
|
||||
verify_out=$(sudo -u plex "$sqlite_bin" "$new_db" "PRAGMA integrity_check;" 2>&1)
|
||||
if [[ "$verify_out" != "ok" ]]; then
|
||||
print_status "${CROSS}" "Repaired database failed integrity check" "${RED}"
|
||||
rm -f "$new_db" 2>/dev/null
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Try reindex as last resort
|
||||
print_status "${INFO}" "Attempting REINDEX operation..." "${BLUE}"
|
||||
if sudo -u plex sqlite3 "$main_db" "REINDEX;" 2>/dev/null; then
|
||||
print_status "${CHECKMARK}" "Database REINDEX completed" "${GREEN}"
|
||||
# Step 4: Reindex (rebuilds all indexes INCLUDING FTS)
|
||||
print_status "${INFO}" "Rebuilding indexes (including FTS)..." "${BLUE}"
|
||||
sudo -u plex "$sqlite_bin" "$new_db" "REINDEX;" 2>/dev/null || true
|
||||
|
||||
# Test integrity one more time
|
||||
if sudo -u plex sqlite3 "$main_db" "PRAGMA integrity_check;" 2>/dev/null | grep -q "ok"; then
|
||||
print_status "${CHECKMARK}" "Database integrity restored after REINDEX!" "${GREEN}"
|
||||
return 0
|
||||
# Rebuild FTS index content explicitly
|
||||
local fts_tables
|
||||
fts_tables=$(sudo -u plex "$sqlite_bin" "$new_db" \
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND sql LIKE '%fts%';" 2>/dev/null) || true
|
||||
if [[ -n "$fts_tables" ]]; then
|
||||
while IFS= read -r table; do
|
||||
[[ -z "$table" ]] && continue
|
||||
print_status "${INFO}" "Rebuilding FTS index: $table" "${BLUE}"
|
||||
sudo -u plex "$sqlite_bin" "$new_db" \
|
||||
"INSERT INTO ${table}(${table}) VALUES('rebuild');" 2>/dev/null || true
|
||||
done <<< "$fts_tables"
|
||||
fi
|
||||
|
||||
# Step 5: Swap databases
|
||||
print_status "${INFO}" "Backing up old database and activating repaired copy..." "${BLUE}"
|
||||
sudo cp "$main_db" "$backup_db"
|
||||
sudo mv "$new_db" "$main_db"
|
||||
sudo chown plex:plex "$main_db"
|
||||
sudo chmod 644 "$main_db"
|
||||
# Remove stale journals for the old database
|
||||
sudo rm -f "${main_db}-shm" "${main_db}-wal" 2>/dev/null || true
|
||||
|
||||
print_status "${CHECKMARK}" "Database repaired and activated. Backup at: $(basename "$backup_db")" "${GREEN}"
|
||||
return 0
|
||||
}
|
||||
|
||||
# 🔍 Function to check FTS (Full-Text Search) index integrity
|
||||
# Standard PRAGMA integrity_check does NOT detect FTS corruption.
|
||||
# This is the exact class of damage shown in the user's screenshot.
|
||||
check_fts_integrity() {
|
||||
local db_file="$1"
|
||||
local sqlite_bin="${2:-sqlite3}" # Use Plex SQLite if available
|
||||
local issues=0
|
||||
|
||||
# Discover FTS tables dynamically from sqlite_master
|
||||
local fts_tables
|
||||
fts_tables=$(sudo -u plex "$sqlite_bin" "$db_file" \
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND sql LIKE '%fts%';" 2>/dev/null) || return 0
|
||||
|
||||
if [[ -z "$fts_tables" ]]; then
|
||||
return 0 # No FTS tables — nothing to check
|
||||
fi
|
||||
|
||||
print_status "${INFO}" "Checking FTS (Full-Text Search) indexes..." "${BLUE}"
|
||||
|
||||
while IFS= read -r table; do
|
||||
[[ -z "$table" ]] && continue
|
||||
local result
|
||||
result=$(sudo -u plex "$sqlite_bin" "$db_file" \
|
||||
"INSERT INTO ${table}(${table}) VALUES('integrity-check');" 2>&1) || true
|
||||
if [[ -n "$result" ]]; then
|
||||
print_status "${CROSS}" "FTS index '${table}' — DAMAGED" "${RED}"
|
||||
echo -e "${DIM}${YELLOW} $result${RESET}"
|
||||
((issues++))
|
||||
else
|
||||
print_status "${CHECKMARK}" "FTS index '${table}' — OK" "${GREEN}"
|
||||
fi
|
||||
done <<< "$fts_tables"
|
||||
|
||||
if (( issues > 0 )); then
|
||||
print_status "${CROSS}" "FTS integrity check complete. $issues index(es) damaged." "${RED}"
|
||||
print_status "${INFO}" "Run: ${SCRIPT_NAME} repair (uses DBRepair reindex to rebuild FTS)" "${YELLOW}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
print_status "${CROSS}" "All repair attempts failed" "${RED}"
|
||||
print_status "${INFO}" "Manual intervention required. Consider:" "${YELLOW}"
|
||||
echo -e "${DIM}${YELLOW} 1. Restore from external backup using restore-plex.sh${RESET}"
|
||||
echo -e "${DIM}${YELLOW} 2. Use nuclear recovery: ./nuclear-plex-recovery.sh${RESET}"
|
||||
echo -e "${DIM}${YELLOW} 3. Check corrupted database moved to: $corrupted_dir${RESET}"
|
||||
|
||||
return 1
|
||||
return 0
|
||||
}
|
||||
|
||||
# 🔍 Function to check database integrity
|
||||
check_database_integrity() {
|
||||
print_status "${INFO}" "Checking database integrity..." "${BLUE}"
|
||||
|
||||
local db_dir="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases"
|
||||
local main_db="$db_dir/com.plexapp.plugins.library.db"
|
||||
local repair_script="${SCRIPT_DIR}/plex-database-repair.sh"
|
||||
local main_db="$PLEX_DB_DIR/com.plexapp.plugins.library.db"
|
||||
|
||||
if [[ ! -f "$main_db" ]]; then
|
||||
print_status "${CROSS}" "Main database not found at: $main_db" "${RED}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Use shared repair script for integrity checking if available
|
||||
if [[ -f "$repair_script" ]]; then
|
||||
if "$repair_script" check "$main_db" >/dev/null 2>&1; then
|
||||
print_status "${CHECKMARK}" "Database integrity check passed" "${GREEN}"
|
||||
return 0
|
||||
else
|
||||
print_status "${CROSS}" "Database integrity check failed!" "${RED}"
|
||||
print_status "${INFO}" "Consider running database repair: plex repair" "${YELLOW}"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
# Fallback to basic sqlite3 check
|
||||
if ! sudo -u plex sqlite3 "$main_db" "PRAGMA integrity_check;" >/dev/null 2>&1; then
|
||||
print_status "${CROSS}" "Database integrity check failed!" "${RED}"
|
||||
print_status "${INFO}" "Consider running database repair or restore from backup" "${YELLOW}"
|
||||
return 1
|
||||
fi
|
||||
# Prefer Plex's bundled SQLite for ICU compatibility
|
||||
local sqlite_bin="sqlite3"
|
||||
if [[ -x "$PLEX_SQLITE" ]]; then
|
||||
sqlite_bin="$PLEX_SQLITE"
|
||||
fi
|
||||
|
||||
print_status "${CHECKMARK}" "Database integrity check passed" "${GREEN}"
|
||||
# Clean up stale WAL/SHM journals left by non-clean shutdowns.
|
||||
if [[ -f "${main_db}-wal" ]]; then
|
||||
print_status "${INFO}" "WAL journal found — attempting checkpoint before integrity check..." "${BLUE}"
|
||||
if ! sudo -u plex "$sqlite_bin" "$main_db" "PRAGMA wal_checkpoint(TRUNCATE);" 2>/dev/null; then
|
||||
print_status "${INFO}" "WAL checkpoint failed (non-critical, continuing check)" "${YELLOW}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Standard structural integrity check ---
|
||||
local integrity_output
|
||||
local sqlite_exit_code=0
|
||||
integrity_output=$(sudo -u plex "$sqlite_bin" "$main_db" "PRAGMA integrity_check;" 2>&1) || sqlite_exit_code=$?
|
||||
|
||||
local struct_ok=true
|
||||
|
||||
if [[ "$integrity_output" == "ok" ]]; then
|
||||
print_status "${CHECKMARK}" "Database structural integrity check passed" "${GREEN}"
|
||||
elif [[ $sqlite_exit_code -ne 0 && -z "$integrity_output" ]]; then
|
||||
print_status "${CROSS}" "sqlite3 failed to open the database (exit code $sqlite_exit_code)" "${RED}"
|
||||
print_status "${INFO}" "Check file permissions and ensure Plex is fully stopped" "${YELLOW}"
|
||||
return 1
|
||||
else
|
||||
struct_ok=false
|
||||
print_status "${CROSS}" "Database structural integrity check reported issues:" "${RED}"
|
||||
echo "$integrity_output" | head -n 5 | while IFS= read -r line; do
|
||||
echo -e "${DIM}${YELLOW} $line${RESET}"
|
||||
done
|
||||
local total_lines
|
||||
total_lines=$(echo "$integrity_output" | wc -l)
|
||||
if (( total_lines > 5 )); then
|
||||
echo -e "${DIM}${YELLOW} ... and $((total_lines - 5)) more issue(s)${RESET}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- FTS index integrity (the most common unreported corruption) ---
|
||||
local fts_ok=true
|
||||
if ! check_fts_integrity "$main_db" "$sqlite_bin"; then
|
||||
fts_ok=false
|
||||
fi
|
||||
|
||||
if [[ "$struct_ok" == true && "$fts_ok" == true ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
print_status "${INFO}" "Consider running database repair: ${SCRIPT_NAME} repair" "${YELLOW}"
|
||||
return 1
|
||||
}
|
||||
|
||||
# <20>🚀 Enhanced start function
|
||||
@@ -341,11 +369,11 @@ start_plex() {
|
||||
# Reset any failed state first
|
||||
sudo systemctl reset-failed "$PLEX_SERVICE" 2>/dev/null || true
|
||||
|
||||
# Check database integrity before starting
|
||||
# Check database integrity before starting (warn only — don't block startup).
|
||||
# Many "failures" are benign WAL journal leftovers that Plex resolves on its own.
|
||||
if ! check_database_integrity; then
|
||||
print_status "${CROSS}" "Database integrity issues detected. Service may fail to start." "${RED}"
|
||||
echo -e "${DIM}${YELLOW} Try: sudo systemctl stop plexmediaserver && sudo -u plex sqlite3 /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/com.plexapp.plugins.library.db 'VACUUM;'${RESET}"
|
||||
return 1
|
||||
print_status "${INFO}" "Database integrity issues detected — starting Plex anyway (it may self-repair)." "${YELLOW}"
|
||||
echo -e "${DIM}${YELLOW} If Plex fails to start, run: ${SCRIPT_NAME} repair${RESET}"
|
||||
fi
|
||||
|
||||
print_status "${INFO}" "Attempting to start service..." "${BLUE}"
|
||||
|
||||
Reference in New Issue
Block a user