hvac-kia-content/docs/deployment_strategy.md

HVAC Know It All - Deployment Strategy

Summary

After thorough testing and implementation, the content aggregation system has been successfully built with 6 scrapers. However, deployment strategy has been revised due to technical constraints with TikTok scraping requirements.

Source Status

✅ Working Sources (5/6)

• WordPress Blog: REST API - ✅ Working
• MailChimp RSS: RSS Feed - ✅ Working
• Podcast RSS: Libsyn Feed - ✅ Working
• YouTube: yt-dlp - ✅ Working
• Instagram: instaloader with session persistence - ✅ Working

⚠️ TikTok Constraints

• TikTok: Requires headed browser with DISPLAY=:0 for bot detection avoidance
• Cannot be containerized due to GUI browser requirement
• Not suitable for Kubernetes deployment

Deployment Decision

Original Plan: Kubernetes Container

• ❌ Not viable due to TikTok headed browser requirement
• ❌ Running GUI applications in containers adds significant complexity
• ❌ Display forwarding in Kubernetes is not practical for production

Revised Plan: Direct System Service

Deploy as systemd service on control plane node:

1. Installation Location: /opt/hvac-kia-content/
2. Service Management: systemd units for scheduling
3. Environment: Direct execution on control plane with DISPLAY access
4. Scheduling: cron-like scheduling via systemd timers

Benefits of Direct Deployment

✅ Advantages

• Simple deployment - no container complexity
• Full system access - DISPLAY, browsers, sessions
• Reliable TikTok scraping - headed browser support
• Easy maintenance - direct file access and logging
• Resource efficiency - no container overhead

⚠️ Considerations

• Host dependency - requires control plane node
• Manual updates - no container image versioning
• Environment coupling - tied to specific system

Implementation Plan

Phase 1: Service Setup

1. Install Python environment at /opt/hvac-kia-content/
2. Configure environment variables and credentials
3. Set up logging directory with rotation
4. Create systemd service unit

Phase 2: Scheduling

1. Create systemd timer units for 8AM and 12PM ADT
2. Configure NAS sync via rsync
3. Set up monitoring and alerting

Phase 3: Monitoring

1. Log rotation and archival
2. Health checks and status reporting
3. Error notification system

File Structure

/opt/hvac-kia-content/
├── src/                    # Source code
├── logs/                   # Application logs
├── data/                   # Scraped content and state
├── .env                    # Environment configuration
├── requirements.txt        # Python dependencies
└── systemd/               # Service configuration
    ├── hvac-scraper.service
    ├── hvac-scraper-morning.timer
    └── hvac-scraper-afternoon.timer

NAS Integration

Sync to: /mnt/nas/hvacknowitall/

• Markdown files with timestamped archives
• Organized by source and date
• Incremental sync to minimize bandwidth

Conclusion

While the original containerized approach is not viable due to TikTok's GUI requirements, the direct deployment approach provides a robust and maintainable solution for the HVAC Know It All content aggregation system.

The system successfully aggregates content from 5 major sources with the option to include TikTok when needed, providing comprehensive coverage of the HVAC Know It All brand across digital platforms.