shell/.github/copilot-instructions.md

# GitHub Copilot Instructions for Shell Repository

This document provides context and guidance for GitHub Copilot when working with this shell script and dotfiles repository.

## Repository Overview

This repository contains:

1. **Shell scripts** for system administration tasks
2. **Dotfiles** for system configuration
3. **Setup scripts** for automated environment configuration
4. **Docker-based testing framework** for validating setup across environments

## Repository Structure

- **Root directory**: Contains various utility shell scripts
- **docs/**: Documentation for individual scripts and components
- **dotfiles/**: System configuration files that get symlinked to the user's home directory
- **powershell/**: PowerShell scripts for Windows environments
- **setup/**: Setup scripts and package lists for automated environment configuration
- **.github/**: GitHub-related configuration files

## Key Files Overview

### Shell Scripts

- **bootstrap.sh**: Entry point script for automated setup
- **test-setup.sh**: Testing script for validating environment setup
- **run-docker-tests.sh**: Runner for Docker-based testing
- **update.sh**: System update scripts
- **plex.sh**: Plex Media Server management

### Configuration Files

- **setup/packages.list**: List of packages to install during setup
- **dotfiles/my-aliases.zsh**: Custom ZSH aliases
- **dotfiles/tailscale-acl.json**: Tailscale ACL configuration

### Documentation

- **README.md**: Main repository documentation
- **docs/docker-bootstrap-testing-framework.md**: Detailed documentation for the Docker-based bootstrap validation framework
- **dotfiles/README.md**: Documentation for dotfiles setup and usage

### Plex Scripts

- **plex.sh**: Main Plex Media Server management script
- **backup-plex.sh**: Automated backup system for Plex databases and configurations
- **monitor-plex-backup.sh**: Monitoring and reporting for backup operations
- **restore-plex.sh**: Database and configuration restoration utilities
- **validate-plex-backups.sh**: Backup integrity verification system
- **test-plex-backup.sh**: Testing framework for backup operations
- **integration-test-plex.sh**: End-to-end integration testing for Plex services
- **plex-recent-additions.sh**: Recent media additions reporting

## Style Guidelines

When suggesting code or modifications:

1. **Shell Scripts**:
   - Use `#!/bin/bash` for most scripts
   - Include proper error handling with `set -e` where appropriate
   - For test scripts, avoid `set -e` to allow testing all components
   - Add descriptive comments for script sections
   - Use colors for terminal output when appropriate (GREEN, RED, YELLOW, etc.)
   - Use capitalized variable names (e.g., `USER_HOME=/home/user`)

2. **Docker Files**:
   - Follow Docker best practices for layer optimization
   - Use specific tags for base images rather than 'latest'
   - Include proper LABEL directives for metadata

3. **Documentation**:
   - Use proper Markdown formatting
   - Include code examples using appropriate syntax highlighting
   - Document script parameters and usage

## Testing Framework

When modifying the testing framework:

1. Make sure to test across both Ubuntu and Debian environments
2. Ensure tests continue even when individual components fail
3. Track and summarize all errors at the end of tests
4. Maintain proper error reporting and logging

### Docker Testing Enhancements

The Docker-based testing framework includes these key features:

1. **Continuous Testing**: Tests continue running even when individual package installations fail
   - Achieved by removing `set -e` from test scripts
   - Uses a counter to track errors rather than exiting immediately

2. **Package Testing**:
   - Dynamically reads packages from `setup/packages.list`
   - Tests each package individually
   - Maintains an array of missing packages for final reporting

3. **Summary Reporting**:
   - Provides a comprehensive summary of all failed tests
   - Suggests commands to fix missing packages
   - Saves detailed logs to a timestamped file

4. **Cross-Platform Testing**:
   - Tests both Ubuntu and Debian environments
   - Handles platform-specific package names (e.g., `bat` vs `batcat`)

## Key Concepts

- **Shell Environment Setup**: Focuses on ZSH with Oh My Zsh and plugins
- **Docker Testing**: Validates environment setup in isolated containers
- **Dotfiles Management**: Uses symbolic links to user's home directory
- **Package Installation**: Uses apt/nala on Debian-based systems

## Main Use Cases

1. **Setting up a new development environment**: Using bootstrap.sh
2. **Managing media services**: Using plex.sh and related scripts
3. **System maintenance**: Using update.sh and backup scripts
4. **Testing configuration changes**: Using the Docker testing framework

## Code Organization Principles

1. **Modularity**: Keep scripts focused on one task
2. **Documentation**: Document all scripts and configurations
3. **Testing**: Ensure all changes are testable
4. **Cross-platform**: Support both Ubuntu and Debian where possible

## Security Practices

When suggesting security-related code:

1. **Permissions**:
   - Avoid running scripts as root unless necessary
   - Use `sudo` for specific commands rather than entire scripts
   - Set appropriate file permissions (e.g., `chmod 600` for sensitive files)

2. **Secret Management**:
   - Never include hardcoded credentials in scripts
   - Use environment variables or external secret management
   - Add sensitive files to .gitignore

3. **Input Validation**:
   - Validate and sanitize all user inputs
   - Use quotes around variables to prevent word splitting and globbing
   - Implement proper error handling for invalid inputs

4. **Network Security**:
   - Verify URLs before downloading (`curl`/`wget`)
   - Use HTTPS instead of HTTP when possible
   - Validate checksums for downloaded packages

## Contribution Guidelines

For contributors and Copilot suggestions:

1. **Script Modifications**:
   - Test all changes using the Docker testing framework
   - Update documentation when adding new functionality
   - Maintain backward compatibility when possible

2. **New Scripts**:
   - Follow the established naming conventions
   - Include a header with description, usage, and author
   - Add appropriate documentation to the docs/ directory
   - Add any new dependencies to setup/packages.list

3. **Review Process**:
   - Run tests before submitting changes
   - Document what was changed and why
   - Consider both Ubuntu, Debian, and Fedora compatibility

## Automatic Code Validation and Critical Error Checking

When generating or modifying shell scripts, GitHub Copilot must automatically validate code and check for critical errors before suggesting changes.

### Mandatory Validation Checks

1. **Syntax Validation**:
   - Always verify proper `if`/`fi`, `for`/`done`, `while`/`done` matching
   - Check for balanced brackets: `[`, `]`, `[[`, `]]`
   - Ensure proper function definition syntax: `function_name() { ... }`
   - Validate case statement structure: `case`/`esac` matching
   - Check for missing quotes around variables in conditions

2. **Control Flow Validation**:
   - Verify `else` statements have corresponding `if` statements
   - Check that nested conditionals are properly indented and closed
   - Ensure `break` and `continue` are only used within loops
   - Validate that functions return appropriate exit codes

3. **Variable and Quote Validation**:
   - Always quote variables to prevent word splitting: `"$VARIABLE"`
   - Check for undefined variables being used
   - Validate array syntax and indexing
   - Ensure proper escaping in strings containing special characters

### Critical Logic Error Patterns to Check

Based on analysis of plex scripts, watch for these common issues:

1. **Service Management Logic**:
   ```bash
   # CORRECT: Check service status before operations
   if sudo systemctl is-active --quiet plexmediaserver; then
       sudo systemctl stop plexmediaserver
   fi

   # INCORRECT: Assuming service state without checking
   sudo systemctl stop plexmediaserver  # May fail if already stopped
   ```

2. **File Operation Safety**:
   ```bash
   # CORRECT: Check file existence and permissions
   if [[ -f "$BACKUP_FILE" && -r "$BACKUP_FILE" ]]; then
       # Process file
   else
       echo "Error: Backup file not found or not readable"
       exit 1
   fi

   # INCORRECT: Operating on files without validation
   tar -xzf "$BACKUP_FILE"  # May fail silently
   ```

3. **Directory Operations**:
   ```bash
   # CORRECT: Verify directory creation and permissions
   if ! mkdir -p "$BACKUP_DIR"; then
       echo "Error: Failed to create backup directory"
       exit 1
   fi

   # Set proper permissions
   chmod 755 "$BACKUP_DIR"

   # INCORRECT: Assuming directory operations succeed
   mkdir -p "$BACKUP_DIR"
   cd "$BACKUP_DIR"  # May fail if mkdir failed
   ```

4. **Database Operations**:
   ```bash
   # CORRECT: Validate database integrity before and after operations
   if ! sqlite3 "$DB_FILE" "PRAGMA integrity_check;" | grep -q "ok"; then
       echo "Error: Database integrity check failed"
       exit 1
   fi

   # INCORRECT: Operating on database without validation
   sqlite3 "$DB_FILE" ".backup backup.db"  # May corrupt if DB is damaged
   ```

5. **Parallel Processing Safety**:
   ```bash
   # CORRECT: Proper parallel job management with wait
   for file in "${files[@]}"; do
       process_file "$file" &
       ((++job_count))

       # Limit concurrent jobs
       if (( job_count >= MAX_JOBS )); then
           wait
           job_count=0
       fi
   done
   wait  # Wait for remaining jobs

   # INCORRECT: Uncontrolled parallel execution
   for file in "${files[@]}"; do
       process_file "$file" &  # May overwhelm system
   done
   ```

### Error Handling Patterns

1. **Function Error Handling**:
   ```bash
   # CORRECT: Proper function with error handling
   backup_database() {
       local db_file="$1"
       local backup_file="$2"

       if [[ -z "$db_file" || -z "$backup_file" ]]; then
           echo "Error: Missing required parameters"
           return 1
       fi

       if [[ ! -f "$db_file" ]]; then
           echo "Error: Database file not found: $db_file"
           return 1
       fi

       if ! sqlite3 "$db_file" ".backup $backup_file"; then
           echo "Error: Database backup failed"
           return 1
       fi

       return 0
   }
   ```

2. **Cleanup on Error**:
   ```bash
   # CORRECT: Cleanup trap for error handling
   cleanup() {
       if [[ -n "$TEMP_DIR" && -d "$TEMP_DIR" ]]; then
           rm -rf "$TEMP_DIR"
       fi

       if [[ "$SERVICE_STOPPED" == "true" ]]; then
           sudo systemctl start plexmediaserver
       fi
   }

   trap cleanup EXIT ERR
   ```

### Validation Checklist for Code Generation

Before suggesting any shell script code, verify:

- [ ] All `if` statements have matching `fi`
- [ ] All `for`/`while` loops have matching `done`
- [ ] All `case` statements have matching `esac`
- [ ] Functions are properly defined with `()` and `{}`
- [ ] Variables are quoted in conditions and expansions
- [ ] File operations check for existence and permissions
- [ ] Service operations verify current state before changes
- [ ] Database operations include integrity checks
- [ ] Parallel operations are properly managed with job limits
- [ ] Error handling includes cleanup and restoration
- [ ] Exit codes are properly set and checked
- [ ] Temporary files and directories are cleaned up

### Testing Integration

When modifying scripts:

1. **Syntax Check**: Always run `bash -n script.sh` validation
2. **Logic Testing**: Test with various input conditions
3. **Error Scenarios**: Test failure modes and recovery
4. **Integration Testing**: Verify interaction with system services
5. **Permission Testing**: Test with different user privileges

### Common Anti-Patterns to Avoid

1. **Unmatched Control Structures**:
   ```bash
   # WRONG: Missing fi
   if [[ condition ]]; then
       do_something
   # Missing fi here
   ```

2. **Unsafe Variable Expansion**:
   ```bash
   # WRONG: Unquoted variables
   if [ $VAR = "value" ]; then  # Will break if VAR contains spaces

   # CORRECT:
   if [[ "$VAR" = "value" ]]; then
   ```

3. **Ignoring Command Failures**:
   ```bash
   # WRONG: Not checking critical command results
   sudo systemctl stop plexmediaserver
   # Continue without knowing if stop succeeded

   # CORRECT:
   if ! sudo systemctl stop plexmediaserver; then
       echo "Error: Failed to stop Plex service"
       exit 1
   fi
   ```

4. **Race Conditions in Service Management**:
   ```bash
   # WRONG: Race condition
   sudo systemctl stop plexmediaserver
   sudo systemctl start plexmediaserver  # May start before stop completes

   # CORRECT:
   sudo systemctl stop plexmediaserver
   sleep 2  # Allow time for clean shutdown
   sudo systemctl start plexmediaserver
   ```