hvac-kia-content/docs/DEPLOYMENT_CHECKLIST.md

# Production Deployment Checklist

## Pre-Deployment Verification

### Environment Setup
- [ ] Ubuntu 20.04+ or compatible Linux distribution
- [ ] Python 3.9+ installed
- [ ] 2GB+ RAM available
- [ ] 10GB+ disk space available
- [ ] Display server running (for TikTok scraping)
- [ ] Network connectivity to all target sites

### Dependencies
- [ ] Install system packages:
  ```bash
  sudo apt update
  sudo apt install python3-pip python3-venv git chromium-browser
  ```
- [ ] Install Python packages:
  ```bash
  pip install -r requirements.txt
  ```
- [ ] Verify Chromium browser works:
  ```bash
  chromium-browser --version
  ```

### Configuration Files
- [ ] `.env` file created with all required variables:
  - [ ] WORDPRESS_USERNAME
  - [ ] WORDPRESS_API_KEY
  - [ ] YOUTUBE_CHANNEL_URL
  - [ ] INSTAGRAM_USERNAME
  - [ ] INSTAGRAM_PASSWORD
  - [ ] TIKTOK_TARGET
  - [ ] NAS_PATH
  - [ ] TIMEZONE (default: America/Halifax)
  - [ ] HEALTHCHECK_URL (optional)
  - [ ] ALERT_EMAIL (optional)

- [ ] `.env` file permissions set to 600:
  ```bash
  chmod 600 .env
  ```

### Directory Structure
- [ ] Create production directories:
  ```bash
  sudo mkdir -p /opt/hvac-kia-content
  sudo mkdir -p /var/log/hvac-content
  ```
- [ ] Set proper ownership:
  ```bash
  sudo chown -R $USER:$USER /opt/hvac-kia-content
  sudo chown -R $USER:$USER /var/log/hvac-content
  ```

### NAS Configuration
- [ ] NAS mount point exists and is accessible
- [ ] Write permissions verified:
  ```bash
  touch /mnt/nas/hkia/test.txt && rm /mnt/nas/hkia/test.txt
  ```
- [ ] Sufficient space available on NAS

## Deployment Steps

### 1. Code Deployment
- [ ] Clone repository to staging location:
  ```bash
  git clone https://git.tealmaker.com/ben/hvac-kia-content.git
  cd hvac-kia-content
  ```
- [ ] Checkout correct branch/tag:
  ```bash
  git checkout main  # or specific version tag
  ```

### 2. Configuration
- [ ] Copy `.env.example` to `.env`:
  ```bash
  cp .env.example .env
  ```
- [ ] Edit `.env` with production values
- [ ] Verify environment variables:
  ```bash
  python3 -c "from run_production import validate_environment; validate_environment()"
  ```

### 3. Test Individual Scrapers
- [ ] Test WordPress:
  ```bash
  python test_real_data.py --source wordpress --items 1
  ```
- [ ] Test YouTube:
  ```bash
  python test_real_data.py --source youtube --items 1
  ```
- [ ] Test Instagram (carefully):
  ```bash
  python test_real_data.py --source instagram --items 1
  ```
- [ ] Test TikTok:
  ```bash
  DISPLAY=:0 python test_real_data.py --source tiktok --items 1
  ```
- [ ] Test MailChimp RSS:
  ```bash
  python test_real_data.py --source mailchimp --items 1
  ```
- [ ] Test Podcast RSS:
  ```bash
  python test_real_data.py --source podcast --items 1
  ```

### 4. Test Production Runner
- [ ] Dry run test:
  ```bash
  python run_production.py --job regular --dry-run
  ```
- [ ] Verify output file created
- [ ] Check log files for errors
- [ ] Verify NAS sync (if enabled)

### 5. Install Systemd Services
- [ ] Run installation script:
  ```bash
  chmod +x install_production.sh
  ./install_production.sh
  ```
- [ ] Verify services installed:
  ```bash
  systemctl list-unit-files | grep hvac
  ```

### 6. Enable Services
- [ ] Enable main timer:
  ```bash
  sudo systemctl enable hkia-content-aggregator.timer
  ```
- [ ] Start timer:
  ```bash
  sudo systemctl start hkia-content-aggregator.timer
  ```
- [ ] Verify timer is active:
  ```bash
  systemctl status hkia-content-aggregator.timer
  ```

### 7. Optional: TikTok Captions
- [ ] Only if captions are required:
  ```bash
  sudo systemctl enable hvac-tiktok-captions.timer
  sudo systemctl start hvac-tiktok-captions.timer
  ```

## Post-Deployment Verification

### Immediate Checks
- [ ] Timer scheduled correctly:
  ```bash
  systemctl list-timers | grep hvac
  ```
- [ ] No errors in service status:
  ```bash
  systemctl status hkia-content-aggregator.service
  ```
- [ ] Log files being created:
  ```bash
  ls -la /var/log/hvac-content/
  ```

### First Run Verification
- [ ] Manually trigger first run:
  ```bash
  sudo systemctl start hkia-content-aggregator.service
  ```
- [ ] Monitor logs in real-time:
  ```bash
  tail -f /var/log/hvac-content/aggregator.log
  ```
- [ ] Verify all sources processed
- [ ] Check output file created
- [ ] Verify NAS sync completed
- [ ] Health check ping received (if configured)

### 24-Hour Verification
- [ ] Check timer fired at scheduled times (8 AM, 12 PM)
- [ ] Review metrics.json for performance data
- [ ] Check disk usage:
  ```bash
  df -h /opt/hvac-kia-content
  ```
- [ ] Review error logs:
  ```bash
  grep ERROR /var/log/hvac-content/*.log
  ```
- [ ] Verify incremental updates working (no duplicates)

## Monitoring Setup

### Log Monitoring
- [ ] Set up log rotation if needed:
  ```bash
  sudo nano /etc/logrotate.d/hvac-content
  ```
  ```
  /var/log/hvac-content/*.log {
      daily
      rotate 7
      compress
      missingok
      notifempty
  }
  ```

### Health Monitoring
- [ ] Configure health check service (e.g., Healthchecks.io)
- [ ] Set up email alerts for failures
- [ ] Create dashboard for metrics visualization

### Backup Configuration
- [ ] Schedule state file backups:
  ```bash
  0 2 * * * tar -czf /backup/hvac-state-$(date +\%Y\%m\%d).tar.gz /opt/hvac-kia-content/state/
  ```
- [ ] Test restore procedure

## Troubleshooting Checklist

### If Scrapers Fail
- [ ] Check environment variables are set
- [ ] Verify network connectivity
- [ ] Check API rate limits
- [ ] Review authentication credentials
- [ ] Check display server (for TikTok)

### If Timer Doesn't Fire
- [ ] Check timer is enabled
- [ ] Verify system time is correct
- [ ] Check systemd timer status
- [ ] Review journal logs:
  ```bash
  journalctl -u hkia-content-aggregator.timer
  ```

### If NAS Sync Fails
- [ ] Verify NAS is mounted
- [ ] Check write permissions
- [ ] Verify sufficient space
- [ ] Test rsync manually

## Rollback Procedure

### Quick Rollback
1. [ ] Stop services:
   ```bash
   sudo systemctl stop hkia-content-aggregator.timer
   ```
2. [ ] Restore previous version:
   ```bash
   cd /opt/hvac-kia-content
   git checkout <previous-version>
   ```
3. [ ] Restart services:
   ```bash
   sudo systemctl start hkia-content-aggregator.timer
   ```

### Full Rollback
1. [ ] Stop and disable all services
2. [ ] Restore backup of state files
3. [ ] Restore previous code version
4. [ ] Re-run installation script
5. [ ] Verify functionality
6. [ ] Re-enable services

## Sign-off

- [ ] Deployment completed successfully
- [ ] All verification steps passed
- [ ] Monitoring configured
- [ ] Documentation updated
- [ ] Team notified

**Deployed By:** _________________  
**Date:** _________________  
**Version:** _________________  
**Notes:** _________________

---

*Last Updated: 2024-12-18*