mirror of
https://github.com/acedanger/shell.git
synced 2025-12-06 06:40:13 -08:00
222 lines
6.1 KiB
Markdown
222 lines
6.1 KiB
Markdown
# Log ANSI Cleanup Utility
|
|
|
|
This utility script removes ANSI color codes from log files to improve readability and reduce file sizes when viewing logs in editors or processing them with other tools.
|
|
|
|
## Features
|
|
|
|
- **Auto-discovery**: Automatically finds log files in common locations
|
|
- **Backup creation**: Creates backup files before cleaning (recommended)
|
|
- **Pattern filtering**: Process only files matching specific patterns
|
|
- **Recursive processing**: Can process directories recursively
|
|
- **Dry run mode**: Preview changes without making them
|
|
- **Comprehensive detection**: Detects various ANSI escape sequence formats
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
# Make the script executable
|
|
chmod +x cleanup-log-ansi.sh
|
|
|
|
# Optional: Create a symbolic link for system-wide access
|
|
sudo ln -s "$(pwd)/cleanup-log-ansi.sh" /usr/local/bin/cleanup-log-ansi
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Basic Usage
|
|
|
|
```bash
|
|
# Clean a single log file with backup
|
|
./cleanup-log-ansi.sh --backup /path/to/logfile.log
|
|
|
|
# Clean all log files in a directory
|
|
./cleanup-log-ansi.sh --recursive --backup /path/to/logs/
|
|
|
|
# Auto-discover and clean all common log files
|
|
./cleanup-log-ansi.sh --auto-discover --backup
|
|
```
|
|
|
|
### Advanced Usage
|
|
|
|
```bash
|
|
# Dry run to see what would be cleaned
|
|
./cleanup-log-ansi.sh --dry-run --auto-discover
|
|
|
|
# Clean only .log files in a directory tree
|
|
./cleanup-log-ansi.sh --recursive --filter "*.log" --backup /var/log/
|
|
|
|
# Clean without creating backups (use with caution)
|
|
./cleanup-log-ansi.sh --no-backup /path/to/logfile.log
|
|
|
|
# Verbose output
|
|
./cleanup-log-ansi.sh --verbose --auto-discover --backup
|
|
```
|
|
|
|
## Options
|
|
|
|
| Option | Description |
|
|
|--------|-------------|
|
|
| `-h, --help` | Show help message |
|
|
| `-r, --recursive` | Process directories recursively |
|
|
| `-b, --backup` | Create backup files before cleaning (recommended) |
|
|
| `-n, --no-backup` | Don't create backup files |
|
|
| `-v, --verbose` | Show detailed output |
|
|
| `-d, --dry-run` | Show what would be done without making changes |
|
|
| `-f, --filter PATTERN` | Only process files matching pattern (e.g., '*.log') |
|
|
| `-a, --auto-discover` | Find common log files automatically |
|
|
|
|
## Auto-Discovery Locations
|
|
|
|
The script automatically searches for log files in these locations:
|
|
|
|
- `./logs/`
|
|
- `./crontab/logs/`
|
|
- `./plex/logs/`
|
|
- `../logs/`
|
|
- `~/.local/share/logs/`
|
|
- `/var/log/` (if readable)
|
|
|
|
## File Extensions
|
|
|
|
Auto-discovery looks for files with these extensions:
|
|
- `.log`
|
|
- `.out`
|
|
- `.err`
|
|
|
|
## ANSI Code Detection
|
|
|
|
The script detects and removes these ANSI escape sequence patterns:
|
|
- `\033[0;31m` (literal backslash sequences)
|
|
- `\x1b[0;31m` (hexadecimal escape sequences)
|
|
- `\033[0;31m` (octal escape sequences)
|
|
|
|
## Examples
|
|
|
|
### Clean Crontab Logs
|
|
|
|
```bash
|
|
# Clean all crontab management logs
|
|
./cleanup-log-ansi.sh --backup crontab/logs/
|
|
|
|
# Example output:
|
|
# INFO: Starting ANSI cleanup utility
|
|
# INFO: Backup created: crontab/logs/crontab-management.log.ansi-backup
|
|
# SUCCESS: Cleaned: crontab/logs/crontab-management.log (24K → 21K)
|
|
```
|
|
|
|
### Clean Plex Logs
|
|
|
|
```bash
|
|
# Clean specific Plex database recovery logs
|
|
./cleanup-log-ansi.sh --backup plex/logs/database-recovery-*.log
|
|
|
|
# Clean all Plex logs recursively
|
|
./cleanup-log-ansi.sh --recursive --backup plex/logs/
|
|
```
|
|
|
|
### System-wide Cleanup
|
|
|
|
```bash
|
|
# Find and clean all log files (with dry run first)
|
|
./cleanup-log-ansi.sh --dry-run --auto-discover
|
|
|
|
# If satisfied with the preview, run for real
|
|
./cleanup-log-ansi.sh --auto-discover --backup
|
|
```
|
|
|
|
## Backup Management
|
|
|
|
When using `--backup`, backup files are created with the `.ansi-backup` suffix:
|
|
|
|
```bash
|
|
# Original file: app.log
|
|
# Backup file: app.log.ansi-backup
|
|
|
|
# Remove all backup files when satisfied
|
|
rm -f *.ansi-backup **/*.ansi-backup
|
|
```
|
|
|
|
## Integration with Crontab
|
|
|
|
You can add this script to your crontab for regular cleanup:
|
|
|
|
```bash
|
|
# Clean logs weekly (Sundays at 3 AM)
|
|
0 3 * * 0 /path/to/cleanup-log-ansi.sh --auto-discover --backup --quiet 2>&1 | logger -t log-cleanup
|
|
|
|
# Clean specific directory daily
|
|
0 2 * * * /path/to/cleanup-log-ansi.sh --recursive --no-backup /var/log/myapp/ 2>&1 | logger -t log-cleanup
|
|
```
|
|
|
|
## Safety Features
|
|
|
|
1. **Backup Creation**: Always create backups by default (use `--backup`)
|
|
2. **File Validation**: Checks file existence and readability before processing
|
|
3. **Integrity Verification**: Ensures cleaned files aren't corrupted
|
|
4. **Dry Run Mode**: Preview changes before applying them
|
|
5. **Error Handling**: Graceful error handling with meaningful messages
|
|
|
|
## Common Use Cases
|
|
|
|
### After Log Rotation
|
|
```bash
|
|
# Clean old log files after rotation
|
|
./cleanup-log-ansi.sh --filter "*.log.*" --backup /var/log/
|
|
```
|
|
|
|
### CI/CD Integration
|
|
```bash
|
|
# Clean build logs in CI pipeline
|
|
./cleanup-log-ansi.sh --no-backup --recursive build/logs/
|
|
```
|
|
|
|
### Development Environment
|
|
```bash
|
|
# Clean development logs regularly
|
|
./cleanup-log-ansi.sh --auto-discover --backup --verbose
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Permission Denied
|
|
```bash
|
|
# If you get permission errors for system logs
|
|
sudo ./cleanup-log-ansi.sh --backup /var/log/
|
|
```
|
|
|
|
### No Files Found
|
|
```bash
|
|
# Check what files would be processed
|
|
./cleanup-log-ansi.sh --dry-run --verbose --auto-discover
|
|
|
|
# Specify paths manually if auto-discovery fails
|
|
./cleanup-log-ansi.sh --backup /specific/path/to/logs/
|
|
```
|
|
|
|
### Large Files
|
|
For very large log files, the script processes them efficiently in a single pass, but you may want to:
|
|
|
|
1. Use `--dry-run` first to estimate processing time
|
|
2. Process files individually for better control
|
|
3. Ensure sufficient disk space for backups
|
|
|
|
## Performance
|
|
|
|
- **Memory Efficient**: Uses streaming processing via `sed`
|
|
- **Fast**: Single-pass processing of files
|
|
- **Safe**: Creates backups before modification
|
|
- **Scalable**: Can handle hundreds of log files
|
|
|
|
## Related Scripts
|
|
|
|
- `crontab-backup-system.sh`: Now has improved logging (ANSI codes properly stripped)
|
|
- `backup-*.sh`: Various backup scripts that generate logs
|
|
- `validate-*.sh`: Validation scripts with colored output
|
|
|
|
## Version History
|
|
|
|
- **v1.0**: Initial release with basic ANSI cleanup functionality
|
|
- **v1.1**: Added auto-discovery and recursive processing
|
|
- **v1.2**: Improved pattern matching and backup handling
|
|
- **v1.3**: Added comprehensive safety features and verbose output
|