shell/docs/env-backup-system.md

# Environment Files Backup System

This document describes the secure backup system for `.env` files from Docker containers to a private Gitea repository.

## Overview

The environment files backup system provides:

- **Automated discovery** of all `.env` files in `~/docker/*` directories
- **Secure version control** using private Git repository
- **Change tracking** with timestamps and commit history
- **Easy restoration** of backed up configurations
- **Validation tools** to ensure backup integrity

## Components

### Scripts

1. **backup-env-files.sh** - Main backup script
2. **validate-env-backups.sh** - Validation and integrity checking

### Repository Structure

```
~/.env-backup/
├── .git/                           # Git repository
├── .gitignore                      # Security-focused gitignore
├── README.md                       # Repository documentation
├── .env-backup-config              # Configuration file
└── docker-containers/              # Backed up files
    ├── container1/
    │   ├── .env                    # Environment file
    │   └── docker-compose.yml.ref  # Reference compose file
    ├── container2/
    │   └── .env
    └── ...
```

## Security Considerations

### 🔒 Critical Security Points

1. **Repository Privacy**: The backup repository MUST be private
2. **Access Control**: Only you should have access to the repository
3. **Network Security**: Use HTTPS or SSH for Git operations
4. **Local Security**: Backup directory should have restricted permissions

### Best Practices

- Use SSH keys for Git authentication (more secure than passwords)
- Regularly rotate any exposed credentials
- Monitor repository access logs
- Consider encrypting the entire backup repository

## Setup Instructions

### 1. Initial Setup

```bash
# First time setup
./backup-env-files.sh --init

# Follow prompts to configure:
# - Gitea instance URL
# - Username
# - Repository name
```

### 2. Create Repository in Gitea

1. Log into your Gitea instance
2. Create a new **private** repository named `docker-env-backup`
3. Do not initialize with README (the script handles this)

### 3. Configure Authentication

#### Option A: SSH Key (Recommended)

```bash
# Generate SSH key if you don't have one
ssh-keygen -t ed25519 -C "your_email@domain.com"

# Add public key to Gitea:
# 1. Go to Settings → SSH/GPG Keys
# 2. Add the content of ~/.ssh/id_ed25519.pub
```

#### Option B: Personal Access Token

```bash
# In Gitea: Settings → Applications → Generate Token
# Configure Git to use token:
git config --global credential.helper store
```

### 4. First Backup

```bash
# List all .env files that will be backed up
./backup-env-files.sh --list

# Perform dry run to see what would happen
./backup-env-files.sh --dry-run

# Execute actual backup
./backup-env-files.sh
```

## Usage

### Regular Backup

```bash
# Standard backup (only backs up changed files)
./backup-env-files.sh

# Force backup all files
./backup-env-files.sh --force

# See what would be backed up
./backup-env-files.sh --dry-run
```

### Validation

```bash
# Basic validation
./validate-env-backups.sh

# Detailed validation with file differences
./validate-env-backups.sh --diff --verbose

# Show only missing files
./validate-env-backups.sh --missing-only
```

### Restoration

```bash
# Restore all .env files from backup
./backup-env-files.sh --restore

# This will:
# 1. Pull latest changes from remote
# 2. Prompt before overwriting existing files
# 3. Create directory structure as needed
```

## Automation

### Cron Job Setup

Add to your crontab for automated backups:

```bash
# Backup .env files daily at 2 AM
0 2 * * * /home/yourusername/shell/backup-env-files.sh >/dev/null 2>&1

# Validate backups weekly on Sundays at 3 AM
0 3 * * 0 /home/yourusername/shell/validate-env-backups.sh --summary-only
```

### Integration with Existing Backup System

Add to your main backup script:

```bash
# In your existing backup script
echo "Backing up environment files..."
/home/yourusername/shell/backup-env-files.sh

# Validate the backup
if ! /home/yourusername/shell/validate-env-backups.sh --summary-only; then
    echo "Warning: .env backup validation failed"
fi
```

## File Discovery

The system automatically finds:

- `*.env` files (e.g., `production.env`, `staging.env`)
- `.env*` files (e.g., `.env`, `.env.local`, `.env.production`)
- `env.*` files (e.g., `env.development`, `env.local`)

### Example Structure

```
~/docker/
├── traefik/
│   ├── .env                    # ✓ Backed up
│   └── docker-compose.yml
├── nextcloud/
│   ├── .env.production         # ✓ Backed up
│   ├── .env.local             # ✓ Backed up
│   └── docker-compose.yml
├── grafana/
│   ├── env.grafana            # ✓ Backed up
│   └── docker-compose.yml
└── plex/
    ├── config.env             # ✓ Backed up
    └── docker-compose.yml
```

## Troubleshooting

### Common Issues

1. **Git Push Fails**

   ```bash
   # Check remote URL
   cd ~/.env-backup && git remote -v
   
   # Test connectivity
   git ls-remote origin
   ```

2. **Missing Files**

   ```bash
   # List what would be found
   ./backup-env-files.sh --list
   
   # Check file permissions
   ls -la ~/docker/*/
   ```

3. **Repository Not Found**
   - Ensure repository exists in Gitea
   - Check repository name matches configuration
   - Verify you have access permissions

### Recovery Scenarios

#### Disaster Recovery

If you lose your entire system:

```bash
# 1. Clone your backup repository
git clone https://git.yourdomain.com/username/docker-env-backup.git ~/.env-backup

# 2. Restore all files
cd /path/to/shell
./backup-env-files.sh --restore
```

#### Selective Recovery

```bash
# Restore specific file manually
cp ~/.env-backup/docker-containers/traefik/.env ~/docker/traefik/
```

## Monitoring

### Log Files

- **backup-env-files.sh**: `logs/env-backup.log`
- **validate-env-backups.sh**: `logs/env-backup-validation.log`

### Health Checks

```bash
# Weekly health check script
#!/bin/bash
echo "=== .env Backup Health Check ==="
./validate-env-backups.sh --summary-only

# Check last backup time
cd ~/.env-backup
echo "Last backup: $(git log -1 --format='%ci')"

# Check repository status
git status --porcelain
```

## Security Enhancements

### Additional Security Measures

1. **GPG Encryption** (Optional)

   ```bash
   # Encrypt sensitive files before committing
   gpg --symmetric --cipher-algo AES256 file.env
   ```

2. **Restricted Permissions**

   ```bash
   # Secure backup directory
   chmod 700 ~/.env-backup
   chmod 600 ~/.env-backup/.env-backup-config
   ```

3. **Audit Trail**

   ```bash
   # Monitor repository access
   git log --oneline --graph --all
   ```

## Best Practices

1. **Regular Testing**: Test restoration process monthly
2. **Version Control**: Never force push; preserve history
3. **Documentation**: Keep README.md updated with changes
4. **Monitoring**: Set up alerts for failed backups
5. **Security**: Regularly review repository access permissions

## Support

For issues or questions:

1. Check the troubleshooting section
2. Review log files for error details
3. Validate your Gitea configuration
4. Test Git connectivity manually