mirror of
https://github.com/acedanger/shell.git
synced 2025-12-06 04:30:13 -08:00
feat: Add enhanced backup-media script and documentation
- Introduced demo-enhanced-backup.sh to showcase new features. - Created backup-media-enhancement-summary.md for a side-by-side comparison of original and enhanced scripts. - Developed enhanced-media-backup.md detailing features, usage, configuration, and error handling of the new backup script. - Enhanced logging, error handling, and performance monitoring capabilities. - Added support for multiple media services with improved safety and maintenance features.
This commit is contained in:
Peter Wood
268
docs/enhanced-media-backup.md
Normal file
268
docs/enhanced-media-backup.md
Normal file
@@ -0,0 +1,268 @@
|
||||
# Enhanced Media Backup Script
|
||||
|
||||
## Overview
|
||||
|
||||
The enhanced `backup-media.sh` script provides robust, enterprise-grade backup functionality for Docker-based media services including Sonarr, Radarr, Prowlarr, Audiobookshelf, Tautulli, SABnzbd, and Jellyseerr.
|
||||
|
||||
## Features
|
||||
|
||||
### Core Functionality
|
||||
- **Multi-service support**: Backs up 7 different media services
|
||||
- **Parallel execution**: Run multiple backups simultaneously for faster completion
|
||||
- **Verification**: Optional integrity checking of backed up files
|
||||
- **Error handling**: Comprehensive error detection and reporting
|
||||
- **Performance monitoring**: Track backup duration and performance metrics
|
||||
|
||||
### Enhanced Logging
|
||||
- **Multiple log formats**: Plain text, JSON, and Markdown reports
|
||||
- **Detailed tracking**: File sizes, checksums, timestamps, and status
|
||||
- **Performance logs**: JSON-formatted performance data for analysis
|
||||
- **Color-coded output**: Easy-to-read terminal output with status colors
|
||||
|
||||
### Safety Features
|
||||
- **Dry run mode**: Preview operations without making changes
|
||||
- **Pre-flight checks**: Verify disk space and Docker availability
|
||||
- **Container verification**: Check if containers are running before backup
|
||||
- **Graceful error handling**: Continue with other services if one fails
|
||||
|
||||
### Maintenance
|
||||
- **Automatic cleanup**: Remove old backups based on age and count limits
|
||||
- **Configurable retention**: Customize how many backups to keep
|
||||
- **Space management**: Monitor and report disk usage
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Usage
|
||||
```bash
|
||||
# Run standard backup
|
||||
./backup-media.sh
|
||||
|
||||
# Preview what would be backed up
|
||||
./backup-media.sh --dry-run
|
||||
|
||||
# Run backups sequentially instead of parallel
|
||||
./backup-media.sh --sequential
|
||||
|
||||
# Skip verification for faster backup
|
||||
./backup-media.sh --no-verify
|
||||
|
||||
# Interactive mode with confirmations
|
||||
./backup-media.sh --interactive
|
||||
```
|
||||
|
||||
### Command Line Options
|
||||
|
||||
| Option | Description |
|
||||
| --------------- | ------------------------------------------------------ |
|
||||
| `--dry-run` | Show what would be backed up without actually doing it |
|
||||
| `--no-verify` | Skip backup verification for faster execution |
|
||||
| `--sequential` | Run backups one at a time instead of parallel |
|
||||
| `--interactive` | Ask for confirmation before each backup |
|
||||
| `--webhook URL` | Custom webhook URL for notifications |
|
||||
| `-h, --help` | Show help message |
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
The script uses several configurable parameters at the top of the file:
|
||||
|
||||
```bash
|
||||
# Retention settings
|
||||
MAX_BACKUP_AGE_DAYS=30 # Delete backups older than 30 days
|
||||
MAX_BACKUPS_TO_KEEP=10 # Keep only 10 most recent backups
|
||||
|
||||
# Directory settings
|
||||
BACKUP_ROOT="/mnt/share/media/backups"
|
||||
LOG_ROOT="/mnt/share/media/backups/logs"
|
||||
|
||||
# Feature toggles
|
||||
PARALLEL_BACKUPS=true # Enable parallel execution
|
||||
VERIFY_BACKUPS=true # Enable backup verification
|
||||
PERFORMANCE_MONITORING=true # Track performance metrics
|
||||
```
|
||||
|
||||
### Services Configuration
|
||||
The script automatically detects and backs up these services:
|
||||
|
||||
| Service | Container Path | Backup Content |
|
||||
| -------------- | --------------------------------------- | --------------------- |
|
||||
| Sonarr | `/config/Backups/scheduled` | Scheduled backups |
|
||||
| Radarr | `/config/Backups/scheduled` | Scheduled backups |
|
||||
| Prowlarr | `/config/Backups/scheduled` | Scheduled backups |
|
||||
| Audiobookshelf | `/metadata/backups` | Metadata backups |
|
||||
| Tautulli | `/config/backups` | Statistics backups |
|
||||
| SABnzbd | `/config/sabnzbd.ini` | Configuration file |
|
||||
| Jellyseerr | `/config/db/` + `/config/settings.json` | Database and settings |
|
||||
|
||||
## Output Files
|
||||
|
||||
### Log Files
|
||||
- **Text Log**: `media-backup-YYYYMMDD_HHMMSS.log` - Standard log format
|
||||
- **Markdown Report**: `media-backup-YYYYMMDD_HHMMSS.md` - Human-readable report
|
||||
- **JSON Log**: `media-backup.json` - Machine-readable backup status
|
||||
- **Performance Log**: `media-backup-performance.json` - Performance metrics
|
||||
|
||||
### Backup Structure
|
||||
```
|
||||
/mnt/share/media/backups/
|
||||
├── logs/
|
||||
│ ├── media-backup-20250525_143022.log
|
||||
│ ├── media-backup-20250525_143022.md
|
||||
│ └── media-backup.json
|
||||
├── sonarr/
|
||||
│ └── scheduled/
|
||||
├── radarr/
|
||||
│ └── scheduled/
|
||||
├── prowlarr/
|
||||
│ └── scheduled/
|
||||
├── audiobookshelf/
|
||||
│ └── backups/
|
||||
├── tautulli/
|
||||
│ └── backups/
|
||||
├── sabnzbd/
|
||||
│ ├── sabnzbd_20250525.ini
|
||||
│ └── sabnzbd_20250524.ini
|
||||
└── jellyseerr/
|
||||
├── backup_20250525/
|
||||
│ ├── db/
|
||||
│ └── settings.json
|
||||
└── backup_20250524/
|
||||
```
|
||||
|
||||
## Notifications
|
||||
|
||||
The script supports webhook notifications (compatible with ntfy.sh and similar services):
|
||||
|
||||
```bash
|
||||
# Default webhook
|
||||
WEBHOOK_URL="https://notify.peterwood.rocks/lab"
|
||||
|
||||
# Custom webhook via command line
|
||||
./backup-media.sh --webhook "https://your-notification-service.com/topic"
|
||||
```
|
||||
|
||||
Notification includes:
|
||||
- Backup status (success/failure)
|
||||
- Number of successful/failed services
|
||||
- Total execution time
|
||||
- Hostname for identification
|
||||
|
||||
## Performance Monitoring
|
||||
|
||||
When enabled, the script tracks:
|
||||
- Individual service backup duration
|
||||
- Overall script execution time
|
||||
- Timestamps for performance analysis
|
||||
- JSON format for easy parsing and graphing
|
||||
|
||||
Example performance log entry:
|
||||
```json
|
||||
{
|
||||
"timestamp": "2025-05-25T14:30:22-05:00",
|
||||
"operation": "backup_sonarr",
|
||||
"duration": 45,
|
||||
"hostname": "media-server"
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The script provides robust error handling:
|
||||
|
||||
1. **Container Health**: Checks if Docker containers are running
|
||||
2. **Disk Space**: Verifies sufficient space before starting
|
||||
3. **Docker Access**: Ensures Docker daemon is accessible
|
||||
4. **Verification**: Optional integrity checking of backups
|
||||
5. **Graceful Failures**: Continues with other services if one fails
|
||||
|
||||
## Integration
|
||||
|
||||
### Cron Job
|
||||
Add to crontab for automated daily backups:
|
||||
```bash
|
||||
# Daily at 2 AM
|
||||
0 2 * * * /home/acedanger/shell/backup-media.sh >/dev/null 2>&1
|
||||
|
||||
# Weekly full backup with verification
|
||||
0 3 * * 0 /home/acedanger/shell/backup-media.sh --verify
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
Use the JSON logs for monitoring integration:
|
||||
```bash
|
||||
# Check last backup status
|
||||
jq '.sonarr.status' /home/acedanger/shell/logs/media-backup.json
|
||||
|
||||
# Get performance metrics
|
||||
jq '.[] | select(.operation == "full_media_backup")' /home/acedanger/shell/logs/media-backup-performance.json
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Container Not Running**
|
||||
```
|
||||
WARNING: Container 'sonarr' is not running
|
||||
```
|
||||
- Verify the container is running: `docker ps`
|
||||
- Start the container: `docker start sonarr`
|
||||
|
||||
2. **Permission Denied**
|
||||
```
|
||||
ERROR: Backup failed for sonarr
|
||||
```
|
||||
- Check Docker permissions
|
||||
- Verify backup directory permissions
|
||||
- Ensure script has execute permissions
|
||||
|
||||
3. **Disk Space**
|
||||
```
|
||||
ERROR: Insufficient disk space
|
||||
```
|
||||
- Free up space in backup directory
|
||||
- Adjust `MAX_BACKUP_AGE_DAYS` for more aggressive cleanup
|
||||
- Run manual cleanup: `find /mnt/share/media/backups -mtime +7 -delete`
|
||||
|
||||
### Debug Mode
|
||||
For troubleshooting, run with verbose output:
|
||||
```bash
|
||||
# Enable debugging
|
||||
bash -x ./backup-media.sh --dry-run
|
||||
|
||||
# Check specific service
|
||||
docker exec sonarr ls -la /config/Backups/scheduled
|
||||
```
|
||||
|
||||
## Comparison with Original Script
|
||||
|
||||
| Feature | Original | Enhanced |
|
||||
| -------------- | --------------- | --------------------------------- |
|
||||
| Error Handling | Basic | Comprehensive |
|
||||
| Logging | Simple text | Multi-format (text/JSON/markdown) |
|
||||
| Performance | No tracking | Full metrics |
|
||||
| Verification | None | Optional integrity checking |
|
||||
| Execution | Sequential only | Parallel and sequential modes |
|
||||
| Notifications | Basic webhook | Enhanced with statistics |
|
||||
| Cleanup | Manual | Automatic with retention policies |
|
||||
| Safety | Limited | Dry-run, pre-flight checks |
|
||||
| Documentation | Minimal | Comprehensive help and docs |
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Script runs with user permissions (no sudo required for Docker operations)
|
||||
- Backup files inherit container security context
|
||||
- Webhook URLs should use HTTPS for secure notifications
|
||||
- Log files may contain sensitive path information
|
||||
- JSON logs are readable by script owner only
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
Potential improvements for future versions:
|
||||
- Database integrity checking for specific services
|
||||
- Compression of backup archives
|
||||
- Remote backup destinations (S3, rsync, etc.)
|
||||
- Backup restoration functionality
|
||||
- Integration with monitoring systems (Prometheus, etc.)
|
||||
- Encrypted backup storage
|
||||
- Incremental backup support
|
||||
Reference in New Issue
Block a user