- Changed OnCalendar from UTC (11:00, 15:00) to local times (08:00, 12:00)
- Fixed timezone confusion that caused missed morning runs
- Services now run at proper 8:00 AM and 12:00 PM Atlantic time
- Manual test confirms YouTube and other scrapers working correctly
🤖 Generated with [Claude Code](https://claude.ai/code)
MAJOR CHANGES:
- TikTok scraper disabled in orchestrator (GUI dependency issues)
- Created new hkia-scraper systemd services replacing hvac-content-*
- Added comprehensive installation script: install-hkia-services.sh
- Updated documentation to reflect 5 active sources (WordPress, MailChimp, Podcast, YouTube, Instagram)
PRODUCTION DEPLOYMENT:
- Services installed and active: hkia-scraper.timer, hkia-scraper-nas.timer
- Schedule: 8:00 AM & 12:00 PM ADT scraping + 30min NAS sync
- All sources now run in parallel (no TikTok GUI blocking)
- Automated twice-daily content aggregation with image downloads
TECHNICAL:
- Orchestrator simplified: removed TikTok special handling
- Service files: proper naming convention (hkia-scraper vs hvac-content)
- Documentation: marked TikTok as disabled, updated deployment status
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Created SystemMonitor class for health check monitoring
- Implemented system metrics collection (CPU, memory, disk, network)
- Added application metrics monitoring (scrapers, logs, data sizes)
- Built alert system with configurable thresholds
- Developed HTML dashboard generator with real-time charts
- Added systemd services for automated monitoring (15-min intervals)
- Created responsive web dashboard with Bootstrap and Chart.js
- Implemented automatic cleanup of old metric files
- Added comprehensive documentation and troubleshooting guide
Features:
- Real-time system resource monitoring
- Scraper performance tracking and alerts
- Interactive dashboard with trend charts
- Email-ready alert notifications
- Systemd integration for production deployment
- Security hardening with minimal privileges
- Auto-refresh dashboard every 5 minutes
- 7-day metric retention with automatic cleanup
Alert conditions:
- Critical: CPU >80%, Memory >85%, Disk >90%
- Warning: Scraper inactive >24h, Log files >100MB
- Error: Monitoring failures, configuration issues
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Production Readiness Improvements:
- Fixed scheduling to match spec (8 AM & 12 PM ADT instead of 6 AM/6 PM)
- Enabled NAS synchronization in production runner with error handling
- Fixed file naming convention to spec format (hvacknowitall_combined_YYYY-MM-DD-THHMMSS.md)
- Made systemd services portable (removed hardcoded user/paths)
- Added environment variable validation on startup
- Moved DISPLAY/XAUTHORITY to .env configuration
Systemd Improvements:
- Created template service file (@.service) for any user
- Changed all paths to /opt/hvac-kia-content
- Updated installation script for portable deployment
- Fixed service dependencies and resource limits
Documentation:
- Created comprehensive PRODUCTION_TODO.md with 25 tasks
- Added PRODUCTION_GUIDE.md with deployment instructions
- Documented spec compliance gaps (65% complete)
Remaining work includes retry logic, connection pooling, media downloads,
and pytest test suite as documented in PRODUCTION_TODO.md
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
