mirror of
https://github.com/acedanger/shell.git
synced 2025-12-05 22:50:18 -08:00
Commit local changes before merging with remote
This commit is contained in:
Peter Wood
320
docs/env-backup-system.md
Normal file
320
docs/env-backup-system.md
Normal file
@@ -0,0 +1,320 @@
|
||||
# Environment Files Backup System
|
||||
|
||||
This document describes the secure backup system for `.env` files from Docker containers to a private Gitea repository.
|
||||
|
||||
## Overview
|
||||
|
||||
The environment files backup system provides:
|
||||
|
||||
- **Automated discovery** of all `.env` files in `~/docker/*` directories
|
||||
- **Secure version control** using private Git repository
|
||||
- **Change tracking** with timestamps and commit history
|
||||
- **Easy restoration** of backed up configurations
|
||||
- **Validation tools** to ensure backup integrity
|
||||
|
||||
## Components
|
||||
|
||||
### Scripts
|
||||
|
||||
1. **backup-env-files.sh** - Main backup script
|
||||
2. **validate-env-backups.sh** - Validation and integrity checking
|
||||
|
||||
### Repository Structure
|
||||
|
||||
```
|
||||
~/.env-backup/
|
||||
├── .git/ # Git repository
|
||||
├── .gitignore # Security-focused gitignore
|
||||
├── README.md # Repository documentation
|
||||
├── .env-backup-config # Configuration file
|
||||
└── docker-containers/ # Backed up files
|
||||
├── container1/
|
||||
│ ├── .env # Environment file
|
||||
│ └── docker-compose.yml.ref # Reference compose file
|
||||
├── container2/
|
||||
│ └── .env
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### 🔒 Critical Security Points
|
||||
|
||||
1. **Repository Privacy**: The backup repository MUST be private
|
||||
2. **Access Control**: Only you should have access to the repository
|
||||
3. **Network Security**: Use HTTPS or SSH for Git operations
|
||||
4. **Local Security**: Backup directory should have restricted permissions
|
||||
|
||||
### Best Practices
|
||||
|
||||
- Use SSH keys for Git authentication (more secure than passwords)
|
||||
- Regularly rotate any exposed credentials
|
||||
- Monitor repository access logs
|
||||
- Consider encrypting the entire backup repository
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Initial Setup
|
||||
|
||||
```bash
|
||||
# First time setup
|
||||
./backup-env-files.sh --init
|
||||
|
||||
# Follow prompts to configure:
|
||||
# - Gitea instance URL
|
||||
# - Username
|
||||
# - Repository name
|
||||
```
|
||||
|
||||
### 2. Create Repository in Gitea
|
||||
|
||||
1. Log into your Gitea instance
|
||||
2. Create a new **private** repository named `docker-env-backup`
|
||||
3. Do not initialize with README (the script handles this)
|
||||
|
||||
### 3. Configure Authentication
|
||||
|
||||
#### Option A: SSH Key (Recommended)
|
||||
|
||||
```bash
|
||||
# Generate SSH key if you don't have one
|
||||
ssh-keygen -t ed25519 -C "your_email@domain.com"
|
||||
|
||||
# Add public key to Gitea:
|
||||
# 1. Go to Settings → SSH/GPG Keys
|
||||
# 2. Add the content of ~/.ssh/id_ed25519.pub
|
||||
```
|
||||
|
||||
#### Option B: Personal Access Token
|
||||
|
||||
```bash
|
||||
# In Gitea: Settings → Applications → Generate Token
|
||||
# Configure Git to use token:
|
||||
git config --global credential.helper store
|
||||
```
|
||||
|
||||
### 4. First Backup
|
||||
|
||||
```bash
|
||||
# List all .env files that will be backed up
|
||||
./backup-env-files.sh --list
|
||||
|
||||
# Perform dry run to see what would happen
|
||||
./backup-env-files.sh --dry-run
|
||||
|
||||
# Execute actual backup
|
||||
./backup-env-files.sh
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Regular Backup
|
||||
|
||||
```bash
|
||||
# Standard backup (only backs up changed files)
|
||||
./backup-env-files.sh
|
||||
|
||||
# Force backup all files
|
||||
./backup-env-files.sh --force
|
||||
|
||||
# See what would be backed up
|
||||
./backup-env-files.sh --dry-run
|
||||
```
|
||||
|
||||
### Validation
|
||||
|
||||
```bash
|
||||
# Basic validation
|
||||
./validate-env-backups.sh
|
||||
|
||||
# Detailed validation with file differences
|
||||
./validate-env-backups.sh --diff --verbose
|
||||
|
||||
# Show only missing files
|
||||
./validate-env-backups.sh --missing-only
|
||||
```
|
||||
|
||||
### Restoration
|
||||
|
||||
```bash
|
||||
# Restore all .env files from backup
|
||||
./backup-env-files.sh --restore
|
||||
|
||||
# This will:
|
||||
# 1. Pull latest changes from remote
|
||||
# 2. Prompt before overwriting existing files
|
||||
# 3. Create directory structure as needed
|
||||
```
|
||||
|
||||
## Automation
|
||||
|
||||
### Cron Job Setup
|
||||
|
||||
Add to your crontab for automated backups:
|
||||
|
||||
```bash
|
||||
# Backup .env files daily at 2 AM
|
||||
0 2 * * * /home/yourusername/shell/backup-env-files.sh >/dev/null 2>&1
|
||||
|
||||
# Validate backups weekly on Sundays at 3 AM
|
||||
0 3 * * 0 /home/yourusername/shell/validate-env-backups.sh --summary-only
|
||||
```
|
||||
|
||||
### Integration with Existing Backup System
|
||||
|
||||
Add to your main backup script:
|
||||
|
||||
```bash
|
||||
# In your existing backup script
|
||||
echo "Backing up environment files..."
|
||||
/home/yourusername/shell/backup-env-files.sh
|
||||
|
||||
# Validate the backup
|
||||
if ! /home/yourusername/shell/validate-env-backups.sh --summary-only; then
|
||||
echo "Warning: .env backup validation failed"
|
||||
fi
|
||||
```
|
||||
|
||||
## File Discovery
|
||||
|
||||
The system automatically finds:
|
||||
|
||||
- `*.env` files (e.g., `production.env`, `staging.env`)
|
||||
- `.env*` files (e.g., `.env`, `.env.local`, `.env.production`)
|
||||
- `env.*` files (e.g., `env.development`, `env.local`)
|
||||
|
||||
### Example Structure
|
||||
|
||||
```
|
||||
~/docker/
|
||||
├── traefik/
|
||||
│ ├── .env # ✓ Backed up
|
||||
│ └── docker-compose.yml
|
||||
├── nextcloud/
|
||||
│ ├── .env.production # ✓ Backed up
|
||||
│ ├── .env.local # ✓ Backed up
|
||||
│ └── docker-compose.yml
|
||||
├── grafana/
|
||||
│ ├── env.grafana # ✓ Backed up
|
||||
│ └── docker-compose.yml
|
||||
└── plex/
|
||||
├── config.env # ✓ Backed up
|
||||
└── docker-compose.yml
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Git Push Fails**
|
||||
|
||||
```bash
|
||||
# Check remote URL
|
||||
cd ~/.env-backup && git remote -v
|
||||
|
||||
# Test connectivity
|
||||
git ls-remote origin
|
||||
```
|
||||
|
||||
2. **Missing Files**
|
||||
|
||||
```bash
|
||||
# List what would be found
|
||||
./backup-env-files.sh --list
|
||||
|
||||
# Check file permissions
|
||||
ls -la ~/docker/*/
|
||||
```
|
||||
|
||||
3. **Repository Not Found**
|
||||
- Ensure repository exists in Gitea
|
||||
- Check repository name matches configuration
|
||||
- Verify you have access permissions
|
||||
|
||||
### Recovery Scenarios
|
||||
|
||||
#### Disaster Recovery
|
||||
|
||||
If you lose your entire system:
|
||||
|
||||
```bash
|
||||
# 1. Clone your backup repository
|
||||
git clone https://git.yourdomain.com/username/docker-env-backup.git ~/.env-backup
|
||||
|
||||
# 2. Restore all files
|
||||
cd /path/to/shell
|
||||
./backup-env-files.sh --restore
|
||||
```
|
||||
|
||||
#### Selective Recovery
|
||||
|
||||
```bash
|
||||
# Restore specific file manually
|
||||
cp ~/.env-backup/docker-containers/traefik/.env ~/docker/traefik/
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Log Files
|
||||
|
||||
- **backup-env-files.sh**: `logs/env-backup.log`
|
||||
- **validate-env-backups.sh**: `logs/env-backup-validation.log`
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Weekly health check script
|
||||
#!/bin/bash
|
||||
echo "=== .env Backup Health Check ==="
|
||||
./validate-env-backups.sh --summary-only
|
||||
|
||||
# Check last backup time
|
||||
cd ~/.env-backup
|
||||
echo "Last backup: $(git log -1 --format='%ci')"
|
||||
|
||||
# Check repository status
|
||||
git status --porcelain
|
||||
```
|
||||
|
||||
## Security Enhancements
|
||||
|
||||
### Additional Security Measures
|
||||
|
||||
1. **GPG Encryption** (Optional)
|
||||
|
||||
```bash
|
||||
# Encrypt sensitive files before committing
|
||||
gpg --symmetric --cipher-algo AES256 file.env
|
||||
```
|
||||
|
||||
2. **Restricted Permissions**
|
||||
|
||||
```bash
|
||||
# Secure backup directory
|
||||
chmod 700 ~/.env-backup
|
||||
chmod 600 ~/.env-backup/.env-backup-config
|
||||
```
|
||||
|
||||
3. **Audit Trail**
|
||||
|
||||
```bash
|
||||
# Monitor repository access
|
||||
git log --oneline --graph --all
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Regular Testing**: Test restoration process monthly
|
||||
2. **Version Control**: Never force push; preserve history
|
||||
3. **Documentation**: Keep README.md updated with changes
|
||||
4. **Monitoring**: Set up alerts for failed backups
|
||||
5. **Security**: Regularly review repository access permissions
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
|
||||
1. Check the troubleshooting section
|
||||
2. Review log files for error details
|
||||
3. Validate your Gitea configuration
|
||||
4. Test Git connectivity manually
|
||||
Reference in New Issue
Block a user