mirror of
https://github.com/acedanger/shell.git
synced 2025-12-06 00:00:13 -08:00
Commit local changes before merging with remote
This commit is contained in:
Peter Wood
334
docs/production-deployment-guide.md
Normal file
334
docs/production-deployment-guide.md
Normal file
@@ -0,0 +1,334 @@
|
||||
# Plex Backup System - Production Deployment Guide
|
||||
|
||||
This guide helps you deploy the enhanced Plex backup system safely in a production environment.
|
||||
|
||||
## Pre-Deployment Checklist
|
||||
|
||||
### 1. System Requirements Verification
|
||||
|
||||
- [ ] **Operating System**: Linux (tested on Ubuntu/Debian)
|
||||
- [ ] **Shell**: Bash 4.0+ available
|
||||
- [ ] **Dependencies Installed**:
|
||||
- [ ] `jq` - JSON processing (required for performance logging)
|
||||
- [ ] `sqlite3` - Database tools (for fallback integrity checks)
|
||||
- [ ] `curl` - HTTP client (for webhook notifications)
|
||||
- [ ] `sendmail` - Email delivery (if using email notifications)
|
||||
- [ ] `tar` and `gzip` - Archive tools
|
||||
- [ ] `sudo` access to Plex files and service management
|
||||
|
||||
### 2. Environment Setup
|
||||
|
||||
- [ ] **Backup Directory**: Ensure `/mnt/share/media/backups/plex` exists and has sufficient space
|
||||
- [ ] **Log Directory**: Ensure `/mnt/share/media/backups/logs` exists and is writable
|
||||
- [ ] **Script Directory**: Place scripts in `/home/acedanger/shell` or update paths accordingly
|
||||
- [ ] **Permissions**: Verify script user can read Plex files and control Plex service
|
||||
|
||||
### 3. Configuration Verification
|
||||
|
||||
- [ ] **Plex Service Management**: Test `systemctl stop plexmediaserver` and `systemctl start plexmediaserver`
|
||||
- [ ] **File Paths**: Verify Plex database locations in script match your installation
|
||||
- [ ] **Plex SQLite Binary**: Confirm `/usr/lib/plexmediaserver/Plex SQLite` exists
|
||||
- [ ] **Disk Space**: Ensure backup location has 2x current Plex database size available
|
||||
|
||||
## Testing Phase
|
||||
|
||||
### 1. Run Unit Tests
|
||||
|
||||
```bash
|
||||
cd /home/acedanger/shell
|
||||
./test-plex-backup.sh unit
|
||||
```
|
||||
|
||||
**Expected Result**: All 9 tests should pass (100% success rate)
|
||||
|
||||
### 2. Run Integration Tests
|
||||
|
||||
```bash
|
||||
cd /home/acedanger/shell
|
||||
./integration-test-plex.sh
|
||||
```
|
||||
|
||||
**Expected Result**: All integration tests should pass
|
||||
|
||||
### 3. Test Dry Run
|
||||
|
||||
```bash
|
||||
# Test integrity check only (non-destructive)
|
||||
sudo ./backup-plex.sh --check-integrity --non-interactive
|
||||
```
|
||||
|
||||
**Expected Result**: Should complete without stopping Plex or creating backups
|
||||
|
||||
### 4. Test Notification Systems
|
||||
|
||||
#### Webhook Testing
|
||||
|
||||
```bash
|
||||
# Replace with your actual webhook URL
|
||||
sudo ./backup-plex.sh --check-integrity --webhook=https://your-webhook-endpoint.com/test
|
||||
```
|
||||
|
||||
#### Email Testing
|
||||
|
||||
```bash
|
||||
# Replace with your email address
|
||||
sudo ./backup-plex.sh --check-integrity --email=admin@yourdomain.com
|
||||
```
|
||||
|
||||
## Production Deployment Steps
|
||||
|
||||
### 1. Initial Backup Test
|
||||
|
||||
```bash
|
||||
# Create a manual backup during maintenance window
|
||||
sudo ./backup-plex.sh --non-interactive
|
||||
```
|
||||
|
||||
**Verify**:
|
||||
|
||||
- [ ] Plex service stopped and restarted properly
|
||||
- [ ] Backup files created in `/mnt/share/media/backups/plex/YYYYMMDD/`
|
||||
- [ ] Log files updated in `/mnt/share/media/backups/logs/`
|
||||
- [ ] Performance log created if enabled
|
||||
- [ ] Notifications sent if configured
|
||||
|
||||
### 2. Validate Backup Integrity
|
||||
|
||||
```bash
|
||||
# Run validation on the created backup
|
||||
./validate-plex-backups.sh --report
|
||||
```
|
||||
|
||||
### 3. Test Restore Process (Optional)
|
||||
|
||||
```bash
|
||||
# In a test environment, verify restore functionality
|
||||
./restore-plex.sh --list
|
||||
./restore-plex.sh --validate YYYYMMDD
|
||||
```
|
||||
|
||||
## Automated Scheduling
|
||||
|
||||
### 1. Cron Configuration
|
||||
|
||||
Create a cron job for automated backups:
|
||||
|
||||
```bash
|
||||
# Edit crontab for root user
|
||||
sudo crontab -e
|
||||
|
||||
# Add entry for daily backup at 2 AM
|
||||
0 2 * * * /home/acedanger/shell/backup-plex.sh --non-interactive --webhook=https://your-webhook.com/plex-backup 2>&1 | logger -t plex-backup
|
||||
```
|
||||
|
||||
### 2. Systemd Timer (Alternative)
|
||||
|
||||
Create systemd service and timer files:
|
||||
|
||||
```bash
|
||||
# Create service file
|
||||
sudo tee /etc/systemd/system/plex-backup.service > /dev/null << 'EOF'
|
||||
[Unit]
|
||||
Description=Plex Media Server Backup
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=root
|
||||
ExecStart=/home/acedanger/shell/backup-plex.sh --non-interactive
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
EOF
|
||||
|
||||
# Create timer file
|
||||
sudo tee /etc/systemd/system/plex-backup.timer > /dev/null << 'EOF'
|
||||
[Unit]
|
||||
Description=Run Plex backup daily
|
||||
Requires=plex-backup.service
|
||||
|
||||
[Timer]
|
||||
OnCalendar=daily
|
||||
Persistent=true
|
||||
RandomizedDelaySec=30m
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
EOF
|
||||
|
||||
# Enable and start timer
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable plex-backup.timer
|
||||
sudo systemctl start plex-backup.timer
|
||||
```
|
||||
|
||||
## Monitoring and Maintenance
|
||||
|
||||
### 1. Log Monitoring
|
||||
|
||||
Monitor backup logs for issues:
|
||||
|
||||
```bash
|
||||
# Check recent backup logs
|
||||
tail -f /mnt/share/media/backups/logs/plex-backup-$(date +%Y-%m-%d).log
|
||||
|
||||
# Check system logs for backup service
|
||||
sudo journalctl -u plex-backup.service -f
|
||||
```
|
||||
|
||||
### 2. Performance Log Analysis
|
||||
|
||||
```bash
|
||||
# View performance trends
|
||||
jq '.[] | select(.operation == "full_backup") | {timestamp, duration_seconds}' \
|
||||
/home/acedanger/shell/logs/plex-backup-performance.json | tail -10
|
||||
```
|
||||
|
||||
### 3. Regular Validation
|
||||
|
||||
Schedule weekly backup validation:
|
||||
|
||||
```bash
|
||||
# Add to crontab
|
||||
0 3 * * 0 /home/acedanger/shell/validate-plex-backups.sh --report --fix 2>&1 | logger -t plex-backup-validation
|
||||
```
|
||||
|
||||
## Troubleshooting Guide
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. Permission Denied Errors
|
||||
|
||||
```bash
|
||||
# Fix script permissions
|
||||
chmod +x /home/acedanger/shell/*.sh
|
||||
|
||||
# Fix backup directory permissions
|
||||
sudo chown -R $(whoami):$(whoami) /mnt/share/media/backups/
|
||||
```
|
||||
|
||||
#### 2. Plex Service Issues
|
||||
|
||||
```bash
|
||||
# Check Plex service status
|
||||
sudo systemctl status plexmediaserver
|
||||
|
||||
# Manually restart if needed
|
||||
sudo systemctl restart plexmediaserver
|
||||
```
|
||||
|
||||
#### 3. Insufficient Disk Space
|
||||
|
||||
```bash
|
||||
# Check available space
|
||||
df -h /mnt/share/media/backups/
|
||||
|
||||
# Clean old backups manually if needed
|
||||
./backup-plex.sh # Script will auto-cleanup based on retention policy
|
||||
```
|
||||
|
||||
#### 4. Database Integrity Issues
|
||||
|
||||
```bash
|
||||
# Run integrity check only
|
||||
sudo ./backup-plex.sh --check-integrity --auto-repair
|
||||
|
||||
# Manual database repair if needed
|
||||
sudo ./backup-plex.sh --auto-repair
|
||||
```
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
#### 1. Parallel Processing
|
||||
|
||||
- Enable parallel verification for faster backups (default: enabled)
|
||||
- Disable with `--no-parallel` if experiencing issues
|
||||
|
||||
#### 2. Performance Monitoring
|
||||
|
||||
- Disable with `--no-performance` if not needed
|
||||
- Monitor trends to optimize backup timing
|
||||
|
||||
#### 3. Notification Optimization
|
||||
|
||||
- Use webhooks instead of email for faster notifications
|
||||
- Configure webhook endpoints with proper error handling
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### 1. File Permissions
|
||||
|
||||
```bash
|
||||
# Secure backup files
|
||||
chmod 600 /home/acedanger/shell/logs/plex-backup*.json
|
||||
chmod 700 /mnt/share/media/backups/plex/
|
||||
```
|
||||
|
||||
### 2. Webhook Security
|
||||
|
||||
- Use HTTPS endpoints for webhooks
|
||||
- Implement webhook signature verification if possible
|
||||
- Avoid including sensitive data in webhook payloads
|
||||
|
||||
### 3. Access Control
|
||||
|
||||
- Limit script execution to authorized users
|
||||
- Consider using dedicated backup user account
|
||||
- Regularly audit file access permissions
|
||||
|
||||
## Backup Retention Strategy
|
||||
|
||||
The script automatically manages backup retention:
|
||||
|
||||
- **Default**: Keep 10 most recent backups
|
||||
- **Age-based**: Remove backups older than 30 days
|
||||
- **Configurable**: Modify `MAX_BACKUPS_TO_KEEP` and `MAX_BACKUP_AGE_DAYS` in script
|
||||
|
||||
## Recovery Planning
|
||||
|
||||
### 1. Backup Restoration
|
||||
|
||||
```bash
|
||||
# List available backups
|
||||
./restore-plex.sh --list
|
||||
|
||||
# Restore specific backup
|
||||
sudo ./restore-plex.sh --restore YYYYMMDD
|
||||
```
|
||||
|
||||
### 2. Emergency Procedures
|
||||
|
||||
1. Stop Plex service: `sudo systemctl stop plexmediaserver`
|
||||
2. Backup current data: `./restore-plex.sh --backup-current`
|
||||
3. Restore from backup: `sudo ./restore-plex.sh --restore YYYYMMDD`
|
||||
4. Start Plex service: `sudo systemctl start plexmediaserver`
|
||||
|
||||
## Success Metrics
|
||||
|
||||
Monitor these metrics to ensure backup system health:
|
||||
|
||||
- [ ] **Backup Success Rate**: >99% successful backups
|
||||
- [ ] **Backup Duration**: Consistent timing (tracked in performance logs)
|
||||
- [ ] **Storage Usage**: Within acceptable limits
|
||||
- [ ] **Service Downtime**: Minimal Plex service interruption
|
||||
- [ ] **Notification Delivery**: Reliable alert delivery
|
||||
- [ ] **Validation Results**: Regular successful backup validation
|
||||
|
||||
## Support and Updates
|
||||
|
||||
### Getting Help
|
||||
|
||||
1. Check logs for error messages
|
||||
2. Run validation tools for diagnosis
|
||||
3. Review troubleshooting guide
|
||||
4. Test with `--check-integrity` for safe debugging
|
||||
|
||||
### Script Updates
|
||||
|
||||
- Keep scripts updated with latest features
|
||||
- Test updates in non-production environment first
|
||||
- Backup current scripts before updating
|
||||
- Review changelog for breaking changes
|
||||
|
||||
---
|
||||
|
||||
**Note**: This deployment guide assumes a typical Plex Media Server installation. Adjust paths and configurations based on your specific environment.
|
||||
Reference in New Issue
Block a user