Merge branch 'main' of https://github.com/acedanger/shell

2025-12-06 08:50:12 -08:00 · 2025-05-29 07:32:02 -04:00
parent ed50395490 07711a4aef
commit 3763c92b2a
13 changed files with 1511 additions and 38 deletions
--- a/docs/env-backup-integration-guide.md
+++ b/docs/env-backup-integration-guide.md
@@ -0,0 +1,146 @@
+# .env Backup Integration Guide
+
+## Quick Setup Summary
+
+Your .env backup system is now fully operational! Here's what was set up:
+
+### ✅ What's Working
+
+- **31 .env files** discovered across your Docker containers
+- **30 files backed up** successfully to `/home/acedanger/.env-backup`
+- **Private Gitea repository** configured and pushed successfully
+- **Version control** with automatic commit messages and timestamps
+- **Reference files** included (docker-compose.yml for context)
+
+### 🔧 Integration Options
+
+#### 1. Manual Backup (Current)
+
+```bash
+cd /home/acedanger/shell
+./backup-env-files.sh              # Regular backup
+./backup-env-files.sh --dry-run     # Preview changes
+./backup-env-files.sh --list        # Show all .env files
+```
+
+#### 2. Automated Daily Backup (Recommended)
+
+Add to your crontab for daily backups at 2 AM:
+
+```bash
+# Daily .env backup at 2 AM
+0 2 * * * /home/acedanger/shell/backup-env-files.sh >/dev/null 2>&1
+```
+
+#### 3. Integration with Existing Backup Scripts
+
+The backup integrates with your existing backup system through:
+
+- **Logs**: Written to `/home/acedanger/shell/logs/env-backup.log`
+- **Completion**: Tab completion available via `env-backup-completion.bash`
+- **Validation**: Use `validate-env-backups.sh` for integrity checks
+
+### 🔐 Security Features
+
+1. **Private Repository**: Only you have access
+2. **Gitignore**: Excludes temporary files and logs
+3. **SSH Authentication**: Uses your existing SSH key
+4. **Local + Remote**: Dual backup (local git + remote Gitea)
+
+### 📊 Backup Structure
+
+```
+~/.env-backup/
+├── docker-containers/
+│   ├── authentik/
+│   │   └── .env.example
+│   ├── caddy/
+│   │   ├── .env
+│   │   ├── .env.example
+│   │   └── docker-compose.yml.ref
+│   ├── database/
+│   │   ├── .env
+│   │   ├── .env.example
+│   │   └── docker-compose.yml.ref
+│   └── ... (all your containers)
+├── README.md
+└── .env-backup-config
+```
+
+### 🔄 Common Operations
+
+#### Restore Files (if needed)
+
+```bash
+./backup-env-files.sh --restore
+```
+
+#### Force Backup (ignore unchanged files)
+
+```bash
+./backup-env-files.sh --force
+```
+
+#### Check What Would Change
+
+```bash
+./backup-env-files.sh --dry-run
+```
+
+### 🚨 Emergency Recovery
+
+If you lose your filesystem:
+
+1. **Clone the backup**: `git clone https://git.ptrwd.com/peterwood/docker-env-backup.git`
+2. **Restore files**: `./backup-env-files.sh --restore`
+3. **Recreate containers**: Your docker-compose.yml reference files are included
+
+### 📈 Monitoring
+
+- **Logs**: Check `/home/acedanger/shell/logs/env-backup.log`
+- **Git History**: View changes with `git log` in backup directory
+- **Validation**: Run `validate-env-backups.sh` for integrity checks
+
+### 🔧 Maintenance
+
+#### Weekly Validation (Recommended)
+
+```bash
+# Add to crontab for weekly validation
+0 3 * * 0 /home/acedanger/shell/validate-env-backups.sh >/dev/null 2>&1
+```
+
+#### Cleanup Old Logs (Monthly)
+
+The system automatically manages logs, but you can clean them manually if needed.
+
+### 🆘 Troubleshooting
+
+#### Push Fails
+
+- Check SSH key: `ssh -T git@git.ptrwd.com`
+- Verify repository exists and is private
+- Check network connectivity
+
+#### Files Not Found
+
+- Verify Docker directory structure: `ls -la ~/docker/*/`
+- Check file permissions
+- Run with `--list` to see what's detected
+
+#### Restore Issues
+
+- Ensure target directories exist
+- Check file permissions
+- Use `--dry-run` first to preview
+
+## Integration Complete! 🎉
+
+Your .env files are now safely backed up and version controlled. The system will:
+
+1. Track all changes to your .env files
+2. Maintain a secure backup in your private Gitea
+3. Provide easy restore capabilities
+4. Integrate with your existing shell toolkit
+
+Run `./backup-env-files.sh` regularly or set up the cron job for automatic backups!
--- a/docs/env-backup-system.md
+++ b/docs/env-backup-system.md
@@ -0,0 +1,320 @@
+# Environment Files Backup System
+
+This document describes the secure backup system for `.env` files from Docker containers to a private Gitea repository.
+
+## Overview
+
+The environment files backup system provides:
+
+- **Automated discovery** of all `.env` files in `~/docker/*` directories
+- **Secure version control** using private Git repository
+- **Change tracking** with timestamps and commit history
+- **Easy restoration** of backed up configurations
+- **Validation tools** to ensure backup integrity
+
+## Components
+
+### Scripts
+
+1. **backup-env-files.sh** - Main backup script
+2. **validate-env-backups.sh** - Validation and integrity checking
+
+### Repository Structure
+
+```
+~/.env-backup/
+├── .git/                           # Git repository
+├── .gitignore                      # Security-focused gitignore
+├── README.md                       # Repository documentation
+├── .env-backup-config              # Configuration file
+└── docker-containers/              # Backed up files
+    ├── container1/
+    │   ├── .env                    # Environment file
+    │   └── docker-compose.yml.ref  # Reference compose file
+    ├── container2/
+    │   └── .env
+    └── ...
+```
+
+## Security Considerations
+
+### 🔒 Critical Security Points
+
+1. **Repository Privacy**: The backup repository MUST be private
+2. **Access Control**: Only you should have access to the repository
+3. **Network Security**: Use HTTPS or SSH for Git operations
+4. **Local Security**: Backup directory should have restricted permissions
+
+### Best Practices
+
+- Use SSH keys for Git authentication (more secure than passwords)
+- Regularly rotate any exposed credentials
+- Monitor repository access logs
+- Consider encrypting the entire backup repository
+
+## Setup Instructions
+
+### 1. Initial Setup
+
+```bash
+# First time setup
+./backup-env-files.sh --init
+
+# Follow prompts to configure:
+# - Gitea instance URL
+# - Username
+# - Repository name
+```
+
+### 2. Create Repository in Gitea
+
+1. Log into your Gitea instance
+2. Create a new **private** repository named `docker-env-backup`
+3. Do not initialize with README (the script handles this)
+
+### 3. Configure Authentication
+
+#### Option A: SSH Key (Recommended)
+
+```bash
+# Generate SSH key if you don't have one
+ssh-keygen -t ed25519 -C "your_email@domain.com"
+
+# Add public key to Gitea:
+# 1. Go to Settings → SSH/GPG Keys
+# 2. Add the content of ~/.ssh/id_ed25519.pub
+```
+
+#### Option B: Personal Access Token
+
+```bash
+# In Gitea: Settings → Applications → Generate Token
+# Configure Git to use token:
+git config --global credential.helper store
+```
+
+### 4. First Backup
+
+```bash
+# List all .env files that will be backed up
+./backup-env-files.sh --list
+
+# Perform dry run to see what would happen
+./backup-env-files.sh --dry-run
+
+# Execute actual backup
+./backup-env-files.sh
+```
+
+## Usage
+
+### Regular Backup
+
+```bash
+# Standard backup (only backs up changed files)
+./backup-env-files.sh
+
+# Force backup all files
+./backup-env-files.sh --force
+
+# See what would be backed up
+./backup-env-files.sh --dry-run
+```
+
+### Validation
+
+```bash
+# Basic validation
+./validate-env-backups.sh
+
+# Detailed validation with file differences
+./validate-env-backups.sh --diff --verbose
+
+# Show only missing files
+./validate-env-backups.sh --missing-only
+```
+
+### Restoration
+
+```bash
+# Restore all .env files from backup
+./backup-env-files.sh --restore
+
+# This will:
+# 1. Pull latest changes from remote
+# 2. Prompt before overwriting existing files
+# 3. Create directory structure as needed
+```
+
+## Automation
+
+### Cron Job Setup
+
+Add to your crontab for automated backups:
+
+```bash
+# Backup .env files daily at 2 AM
+0 2 * * * /home/yourusername/shell/backup-env-files.sh >/dev/null 2>&1
+
+# Validate backups weekly on Sundays at 3 AM
+0 3 * * 0 /home/yourusername/shell/validate-env-backups.sh --summary-only
+```
+
+### Integration with Existing Backup System
+
+Add to your main backup script:
+
+```bash
+# In your existing backup script
+echo "Backing up environment files..."
+/home/yourusername/shell/backup-env-files.sh
+
+# Validate the backup
+if ! /home/yourusername/shell/validate-env-backups.sh --summary-only; then
+    echo "Warning: .env backup validation failed"
+fi
+```
+
+## File Discovery
+
+The system automatically finds:
+
+- `*.env` files (e.g., `production.env`, `staging.env`)
+- `.env*` files (e.g., `.env`, `.env.local`, `.env.production`)
+- `env.*` files (e.g., `env.development`, `env.local`)
+
+### Example Structure
+
+```
+~/docker/
+├── traefik/
+│   ├── .env                    # ✓ Backed up
+│   └── docker-compose.yml
+├── nextcloud/
+│   ├── .env.production         # ✓ Backed up
+│   ├── .env.local             # ✓ Backed up
+│   └── docker-compose.yml
+├── grafana/
+│   ├── env.grafana            # ✓ Backed up
+│   └── docker-compose.yml
+└── plex/
+    ├── config.env             # ✓ Backed up
+    └── docker-compose.yml
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Git Push Fails**
+
+   ```bash
+   # Check remote URL
+   cd ~/.env-backup && git remote -v
+   
+   # Test connectivity
+   git ls-remote origin
+   ```
+
+2. **Missing Files**
+
+   ```bash
+   # List what would be found
+   ./backup-env-files.sh --list
+   
+   # Check file permissions
+   ls -la ~/docker/*/
+   ```
+
+3. **Repository Not Found**
+   - Ensure repository exists in Gitea
+   - Check repository name matches configuration
+   - Verify you have access permissions
+
+### Recovery Scenarios
+
+#### Disaster Recovery
+
+If you lose your entire system:
+
+```bash
+# 1. Clone your backup repository
+git clone https://git.yourdomain.com/username/docker-env-backup.git ~/.env-backup
+
+# 2. Restore all files
+cd /path/to/shell
+./backup-env-files.sh --restore
+```
+
+#### Selective Recovery
+
+```bash
+# Restore specific file manually
+cp ~/.env-backup/docker-containers/traefik/.env ~/docker/traefik/
+```
+
+## Monitoring
+
+### Log Files
+
+- **backup-env-files.sh**: `logs/env-backup.log`
+- **validate-env-backups.sh**: `logs/env-backup-validation.log`
+
+### Health Checks
+
+```bash
+# Weekly health check script
+#!/bin/bash
+echo "=== .env Backup Health Check ==="
+./validate-env-backups.sh --summary-only
+
+# Check last backup time
+cd ~/.env-backup
+echo "Last backup: $(git log -1 --format='%ci')"
+
+# Check repository status
+git status --porcelain
+```
+
+## Security Enhancements
+
+### Additional Security Measures
+
+1. **GPG Encryption** (Optional)
+
+   ```bash
+   # Encrypt sensitive files before committing
+   gpg --symmetric --cipher-algo AES256 file.env
+   ```
+
+2. **Restricted Permissions**
+
+   ```bash
+   # Secure backup directory
+   chmod 700 ~/.env-backup
+   chmod 600 ~/.env-backup/.env-backup-config
+   ```
+
+3. **Audit Trail**
+
+   ```bash
+   # Monitor repository access
+   git log --oneline --graph --all
+   ```
+
+## Best Practices
+
+1. **Regular Testing**: Test restoration process monthly
+2. **Version Control**: Never force push; preserve history
+3. **Documentation**: Keep README.md updated with changes
+4. **Monitoring**: Set up alerts for failed backups
+5. **Security**: Regularly review repository access permissions
+
+## Support
+
+For issues or questions:
+
+1. Check the troubleshooting section
+2. Review log files for error details
+3. Validate your Gitea configuration
+4. Test Git connectivity manually