docs: Update all documentation to use hkia naming convention
Documentation Updates: - Updated project specification with hkia naming and paths - Modified all markdown documentation files (12 files updated) - Changed service names from hvac-content-* to hkia-content-* - Updated NAS paths from /mnt/nas/hvacknowitall to /mnt/nas/hkia - Replaced all instances of "HVAC Know It All" with "HKIA" Files Updated: - README.md - Updated service names and commands - CLAUDE.md - Updated environment variables and paths - DEPLOY.md - Updated deployment instructions - docs/project_specification.md - Updated naming convention specs - docs/status.md - Updated project status with new naming - docs/final_status.md - Updated completion status - docs/deployment_strategy.md - Updated deployment paths - docs/DEPLOYMENT_CHECKLIST.md - Updated checklist items - docs/PRODUCTION_TODO.md - Updated production tasks - BACKLOG_STATUS.md - Updated backlog references - UPDATED_CAPTURE_STATUS.md - Updated capture status - FINAL_TALLY_REPORT.md - Updated tally report Notes: - Repository name remains hvacknowitall-content (unchanged) - Project directory remains hvac-kia-content (unchanged) - All user-facing outputs now use clean "hkia" naming 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
daab901e35
commit
7e5377e7b1
13 changed files with 207 additions and 94 deletions
|
|
@ -1,4 +1,4 @@
|
||||||
# HVAC Know It All - Production Backlog Capture Status
|
# HKIA - Production Backlog Capture Status
|
||||||
|
|
||||||
## 📊 Current Progress Report
|
## 📊 Current Progress Report
|
||||||
**Last Updated**: August 18, 2025 @ 10:23 PM ADT
|
**Last Updated**: August 18, 2025 @ 10:23 PM ADT
|
||||||
|
|
@ -30,9 +30,9 @@ All markdown files are being created in specification-compliant format:
|
||||||
|
|
||||||
```
|
```
|
||||||
/home/ben/dev/hvac-kia-content/data_production_backlog/markdown_current/
|
/home/ben/dev/hvac-kia-content/data_production_backlog/markdown_current/
|
||||||
├── hvacknowitall_wordpress_backlog_20250818_221430.md (1.5M)
|
├── hkia_wordpress_backlog_20250818_221430.md (1.5M)
|
||||||
├── hvacknowitall_podcast_backlog_20250818_221531.md (727K)
|
├── hkia_podcast_backlog_20250818_221531.md (727K)
|
||||||
└── hvacknowitall_youtube_backlog_20250818_221604.md (107K)
|
└── hkia_youtube_backlog_20250818_221604.md (107K)
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ Format Verification
|
### ✅ Format Verification
|
||||||
|
|
@ -40,7 +40,7 @@ All markdown files are being created in specification-compliant format:
|
||||||
- Correct markdown structure with `##` headers
|
- Correct markdown structure with `##` headers
|
||||||
- Full content including descriptions and metadata
|
- Full content including descriptions and metadata
|
||||||
- Item separators (`--------------------------------------------------`)
|
- Item separators (`--------------------------------------------------`)
|
||||||
- Timestamped filenames: `hvacknowitall_[source]_backlog_[timestamp].md`
|
- Timestamped filenames: `hkia_[source]_backlog_[timestamp].md`
|
||||||
|
|
||||||
## 📊 Statistics
|
## 📊 Statistics
|
||||||
|
|
||||||
|
|
|
||||||
16
CLAUDE.md
16
CLAUDE.md
|
|
@ -27,7 +27,7 @@ Complete content aggregation system that scrapes 6 sources (WordPress, MailChimp
|
||||||
|
|
||||||
### YouTube Scraper (`src/youtube_scraper.py`)
|
### YouTube Scraper (`src/youtube_scraper.py`)
|
||||||
- Uses `yt-dlp` for metadata extraction
|
- Uses `yt-dlp` for metadata extraction
|
||||||
- Channel: `@HVACKnowItAll`
|
- Channel: `@hkia`
|
||||||
- Fetches video metadata without downloading videos
|
- Fetches video metadata without downloading videos
|
||||||
|
|
||||||
### RSS Scrapers
|
### RSS Scrapers
|
||||||
|
|
@ -56,10 +56,10 @@ Originally planned for Kubernetes deployment but **TikTok requires headed browse
|
||||||
### Production Setup
|
### Production Setup
|
||||||
```bash
|
```bash
|
||||||
# Service files location
|
# Service files location
|
||||||
/etc/systemd/system/hvac-scraper.service
|
/etc/systemd/system/hkia-scraper.service
|
||||||
/etc/systemd/system/hvac-scraper.timer
|
/etc/systemd/system/hkia-scraper.timer
|
||||||
/etc/systemd/system/hvac-scraper-nas.service
|
/etc/systemd/system/hkia-scraper-nas.service
|
||||||
/etc/systemd/system/hvac-scraper-nas.timer
|
/etc/systemd/system/hkia-scraper-nas.timer
|
||||||
|
|
||||||
# Installation directory
|
# Installation directory
|
||||||
/opt/hvac-kia-content/
|
/opt/hvac-kia-content/
|
||||||
|
|
@ -79,7 +79,7 @@ export XAUTHORITY="/run/user/1000/.mutter-Xwaylandauth.90WDB3"
|
||||||
# Required in /opt/hvac-kia-content/.env
|
# Required in /opt/hvac-kia-content/.env
|
||||||
INSTAGRAM_USERNAME=hkia1
|
INSTAGRAM_USERNAME=hkia1
|
||||||
INSTAGRAM_PASSWORD=I22W5YlbRl7x
|
INSTAGRAM_PASSWORD=I22W5YlbRl7x
|
||||||
YOUTUBE_CHANNEL=@HVACKnowItAll
|
YOUTUBE_CHANNEL=@hkia
|
||||||
TIKTOK_USERNAME=hkia
|
TIKTOK_USERNAME=hkia
|
||||||
NAS_PATH=/mnt/nas/hkia
|
NAS_PATH=/mnt/nas/hkia
|
||||||
TIMEZONE=America/Halifax
|
TIMEZONE=America/Halifax
|
||||||
|
|
@ -113,8 +113,8 @@ uv run python -m src.orchestrator --sources youtube instagram
|
||||||
uv run python -m src.orchestrator --nas-only
|
uv run python -m src.orchestrator --nas-only
|
||||||
|
|
||||||
# Check service status
|
# Check service status
|
||||||
sudo systemctl status hvac-scraper.service
|
sudo systemctl status hkia-scraper.service
|
||||||
sudo journalctl -f -u hvac-scraper.service
|
sudo journalctl -f -u hkia-scraper.service
|
||||||
```
|
```
|
||||||
|
|
||||||
## Critical Notes
|
## Critical Notes
|
||||||
|
|
|
||||||
40
DEPLOY.md
40
DEPLOY.md
|
|
@ -33,12 +33,12 @@ sudo cp deploy/*.timer /etc/systemd/system/
|
||||||
sudo systemctl daemon-reload
|
sudo systemctl daemon-reload
|
||||||
|
|
||||||
# Enable timers (start on boot)
|
# Enable timers (start on boot)
|
||||||
sudo systemctl enable hvac-content-8am.timer
|
sudo systemctl enable hkia-content-8am.timer
|
||||||
sudo systemctl enable hvac-content-12pm.timer
|
sudo systemctl enable hkia-content-12pm.timer
|
||||||
|
|
||||||
# Start timers immediately
|
# Start timers immediately
|
||||||
sudo systemctl start hvac-content-8am.timer
|
sudo systemctl start hkia-content-8am.timer
|
||||||
sudo systemctl start hvac-content-12pm.timer
|
sudo systemctl start hkia-content-12pm.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
## Verify Deployment
|
## Verify Deployment
|
||||||
|
|
@ -51,8 +51,8 @@ systemctl list-timers | grep hvac
|
||||||
Expected output:
|
Expected output:
|
||||||
```
|
```
|
||||||
NEXT LEFT LAST PASSED UNIT ACTIVATES
|
NEXT LEFT LAST PASSED UNIT ACTIVATES
|
||||||
Mon 2025-08-20 08:00:00 ADT 21h left n/a n/a hvac-content-8am.timer hvac-content-8am.service
|
Mon 2025-08-20 08:00:00 ADT 21h left n/a n/a hkia-content-8am.timer hkia-content-8am.service
|
||||||
Mon 2025-08-19 12:00:00 ADT 1h 9min left n/a n/a hvac-content-12pm.timer hvac-content-12pm.service
|
Mon 2025-08-19 12:00:00 ADT 1h 9min left n/a n/a hkia-content-12pm.timer hkia-content-12pm.service
|
||||||
```
|
```
|
||||||
|
|
||||||
## Monitor Services
|
## Monitor Services
|
||||||
|
|
@ -60,13 +60,13 @@ Mon 2025-08-19 12:00:00 ADT 1h 9min left n/a n/a hvac-content-12pm.timer
|
||||||
View logs in real-time:
|
View logs in real-time:
|
||||||
```bash
|
```bash
|
||||||
# Morning run logs
|
# Morning run logs
|
||||||
journalctl -u hvac-content-8am -f
|
journalctl -u hkia-content-8am -f
|
||||||
|
|
||||||
# Noon run logs
|
# Noon run logs
|
||||||
journalctl -u hvac-content-12pm -f
|
journalctl -u hkia-content-12pm -f
|
||||||
|
|
||||||
# All logs
|
# All logs
|
||||||
journalctl -u hvac-content-* -f
|
journalctl -u hkia-content-* -f
|
||||||
```
|
```
|
||||||
|
|
||||||
## Manual Testing
|
## Manual Testing
|
||||||
|
|
@ -74,10 +74,10 @@ journalctl -u hvac-content-* -f
|
||||||
Run the service manually:
|
Run the service manually:
|
||||||
```bash
|
```bash
|
||||||
# Test morning run
|
# Test morning run
|
||||||
sudo systemctl start hvac-content-8am.service
|
sudo systemctl start hkia-content-8am.service
|
||||||
|
|
||||||
# Check status
|
# Check status
|
||||||
sudo systemctl status hvac-content-8am.service
|
sudo systemctl status hkia-content-8am.service
|
||||||
```
|
```
|
||||||
|
|
||||||
## Stop/Disable Services
|
## Stop/Disable Services
|
||||||
|
|
@ -85,30 +85,30 @@ sudo systemctl status hvac-content-8am.service
|
||||||
If needed:
|
If needed:
|
||||||
```bash
|
```bash
|
||||||
# Stop timers
|
# Stop timers
|
||||||
sudo systemctl stop hvac-content-8am.timer
|
sudo systemctl stop hkia-content-8am.timer
|
||||||
sudo systemctl stop hvac-content-12pm.timer
|
sudo systemctl stop hkia-content-12pm.timer
|
||||||
|
|
||||||
# Disable from starting on boot
|
# Disable from starting on boot
|
||||||
sudo systemctl disable hvac-content-8am.timer
|
sudo systemctl disable hkia-content-8am.timer
|
||||||
sudo systemctl disable hvac-content-12pm.timer
|
sudo systemctl disable hkia-content-12pm.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Service Fails to Start
|
### Service Fails to Start
|
||||||
1. Check logs: `journalctl -u hvac-content-8am -n 50`
|
1. Check logs: `journalctl -u hkia-content-8am -n 50`
|
||||||
2. Verify paths in service files
|
2. Verify paths in service files
|
||||||
3. Check Python environment: `source .venv/bin/activate && python --version`
|
3. Check Python environment: `source .venv/bin/activate && python --version`
|
||||||
4. Test manual run: `cd /home/ben/dev/hvac-kia-content && uv run python run_api_production_v2.py`
|
4. Test manual run: `cd /home/ben/dev/hvac-kia-content && uv run python run_api_production_v2.py`
|
||||||
|
|
||||||
### Permission Issues
|
### Permission Issues
|
||||||
- Ensure user `ben` has read/write access to data directories
|
- Ensure user `ben` has read/write access to data directories
|
||||||
- Check NAS mount permissions: `ls -la /mnt/nas/hvacknowitall/`
|
- Check NAS mount permissions: `ls -la /mnt/nas/hkia/`
|
||||||
|
|
||||||
### Timer Not Triggering
|
### Timer Not Triggering
|
||||||
- Check timer status: `systemctl status hvac-content-8am.timer`
|
- Check timer status: `systemctl status hkia-content-8am.timer`
|
||||||
- Verify system time: `timedatectl`
|
- Verify system time: `timedatectl`
|
||||||
- Check timer schedule: `systemctl cat hvac-content-8am.timer`
|
- Check timer schedule: `systemctl cat hkia-content-8am.timer`
|
||||||
|
|
||||||
## Schedule
|
## Schedule
|
||||||
|
|
||||||
|
|
@ -121,4 +121,4 @@ Both runs will:
|
||||||
2. Merge with existing cumulative files
|
2. Merge with existing cumulative files
|
||||||
3. Update metrics and add captions where available
|
3. Update metrics and add captions where available
|
||||||
4. Archive previous versions
|
4. Archive previous versions
|
||||||
5. Sync to NAS at `/mnt/nas/hvacknowitall/`
|
5. Sync to NAS at `/mnt/nas/hkia/`
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
# HVAC Know It All - Production Backlog Capture Tally Report
|
# HKIA - Production Backlog Capture Tally Report
|
||||||
**Generated**: August 18, 2025 @ 11:00 PM ADT
|
**Generated**: August 18, 2025 @ 11:00 PM ADT
|
||||||
|
|
||||||
## ✅ Markdown Creation Verification
|
## ✅ Markdown Creation Verification
|
||||||
|
|
@ -7,9 +7,9 @@ All completed sources have been successfully saved to specification-compliant ma
|
||||||
|
|
||||||
| Source | Status | Markdown File | Items | File Size | Verification |
|
| Source | Status | Markdown File | Items | File Size | Verification |
|
||||||
|--------|--------|---------------|-------|-----------|--------------|
|
|--------|--------|---------------|-------|-----------|--------------|
|
||||||
| **WordPress** | ✅ Complete | hvacknowitall_wordpress_backlog_20250818_221430.md | 139 posts | 1.5 MB | ✅ Verified |
|
| **WordPress** | ✅ Complete | hkia_wordpress_backlog_20250818_221430.md | 139 posts | 1.5 MB | ✅ Verified |
|
||||||
| **Podcast** | ✅ Complete | hvacknowitall_podcast_backlog_20250818_221531.md | 428 episodes | 727 KB | ✅ Verified |
|
| **Podcast** | ✅ Complete | hkia_podcast_backlog_20250818_221531.md | 428 episodes | 727 KB | ✅ Verified |
|
||||||
| **YouTube** | ✅ Complete | hvacknowitall_youtube_backlog_20250818_221604.md | 200 videos | 107 KB | ✅ Verified |
|
| **YouTube** | ✅ Complete | hkia_youtube_backlog_20250818_221604.md | 200 videos | 107 KB | ✅ Verified |
|
||||||
| **MailChimp** | ⚠️ SSL Error | N/A | 0 | N/A | Known Issue |
|
| **MailChimp** | ⚠️ SSL Error | N/A | 0 | N/A | Known Issue |
|
||||||
| **Instagram** | 🔄 In Progress | Pending completion | 15/1000 | TBD | Processing |
|
| **Instagram** | 🔄 In Progress | Pending completion | 15/1000 | TBD | Processing |
|
||||||
| **TikTok** | ⏳ Queued | Pending | 0/1000 | TBD | Waiting |
|
| **TikTok** | ⏳ Queued | Pending | 0/1000 | TBD | Waiting |
|
||||||
|
|
|
||||||
20
README.md
20
README.md
|
|
@ -144,31 +144,31 @@ tests/ # Test files
|
||||||
### Systemd Services
|
### Systemd Services
|
||||||
|
|
||||||
Services are configured in `/etc/systemd/system/`:
|
Services are configured in `/etc/systemd/system/`:
|
||||||
- `hvac-content-images-8am.service` - Morning run with image downloads
|
- `hkia-content-images-8am.service` - Morning run with image downloads
|
||||||
- `hvac-content-images-12pm.service` - Noon run with image downloads
|
- `hkia-content-images-12pm.service` - Noon run with image downloads
|
||||||
- `hvac-content-images-8am.timer` - Morning schedule (8 AM Atlantic)
|
- `hkia-content-images-8am.timer` - Morning schedule (8 AM Atlantic)
|
||||||
- `hvac-content-images-12pm.timer` - Noon schedule (12 PM Atlantic)
|
- `hkia-content-images-12pm.timer` - Noon schedule (12 PM Atlantic)
|
||||||
|
|
||||||
### Manual Deployment
|
### Manual Deployment
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Start services
|
# Start services
|
||||||
sudo systemctl start hvac-content-8am.timer
|
sudo systemctl start hkia-content-8am.timer
|
||||||
sudo systemctl start hvac-content-12pm.timer
|
sudo systemctl start hkia-content-12pm.timer
|
||||||
|
|
||||||
# Enable on boot
|
# Enable on boot
|
||||||
sudo systemctl enable hvac-content-8am.timer
|
sudo systemctl enable hkia-content-8am.timer
|
||||||
sudo systemctl enable hvac-content-12pm.timer
|
sudo systemctl enable hkia-content-12pm.timer
|
||||||
|
|
||||||
# Check status
|
# Check status
|
||||||
sudo systemctl status hvac-content-*.timer
|
sudo systemctl status hkia-content-*.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
## Monitoring
|
## Monitoring
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# View logs
|
# View logs
|
||||||
journalctl -u hvac-content-8am -f
|
journalctl -u hkia-content-8am -f
|
||||||
|
|
||||||
# Check file growth
|
# Check file growth
|
||||||
ls -lh data/markdown_current/
|
ls -lh data/markdown_current/
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
# HVAC Know It All - Updated Production Backlog Capture
|
# HKIA - Updated Production Backlog Capture
|
||||||
|
|
||||||
## 🚀 Updated Configuration
|
## 🚀 Updated Configuration
|
||||||
**Started**: August 18, 2025 @ 10:54 PM ADT
|
**Started**: August 18, 2025 @ 10:54 PM ADT
|
||||||
|
|
@ -37,11 +37,11 @@
|
||||||
## 📁 Output Location
|
## 📁 Output Location
|
||||||
```
|
```
|
||||||
/home/ben/dev/hvac-kia-content/data_production_backlog/markdown_current/
|
/home/ben/dev/hvac-kia-content/data_production_backlog/markdown_current/
|
||||||
├── hvacknowitall_wordpress_backlog_[timestamp].md
|
├── hkia_wordpress_backlog_[timestamp].md
|
||||||
├── hvacknowitall_podcast_backlog_[timestamp].md
|
├── hkia_podcast_backlog_[timestamp].md
|
||||||
├── hvacknowitall_youtube_backlog_[timestamp].md
|
├── hkia_youtube_backlog_[timestamp].md
|
||||||
├── hvacknowitall_instagram_backlog_[timestamp].md (pending)
|
├── hkia_instagram_backlog_[timestamp].md (pending)
|
||||||
└── hvacknowitall_tiktok_backlog_[timestamp].md (pending)
|
└── hkia_tiktok_backlog_[timestamp].md (pending)
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📈 Progress Monitoring
|
## 📈 Progress Monitoring
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,7 @@
|
||||||
- [ ] NAS mount point exists and is accessible
|
- [ ] NAS mount point exists and is accessible
|
||||||
- [ ] Write permissions verified:
|
- [ ] Write permissions verified:
|
||||||
```bash
|
```bash
|
||||||
touch /mnt/nas/hvacknowitall/test.txt && rm /mnt/nas/hvacknowitall/test.txt
|
touch /mnt/nas/hkia/test.txt && rm /mnt/nas/hkia/test.txt
|
||||||
```
|
```
|
||||||
- [ ] Sufficient space available on NAS
|
- [ ] Sufficient space available on NAS
|
||||||
|
|
||||||
|
|
@ -136,15 +136,15 @@
|
||||||
### 6. Enable Services
|
### 6. Enable Services
|
||||||
- [ ] Enable main timer:
|
- [ ] Enable main timer:
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl enable hvac-content-aggregator.timer
|
sudo systemctl enable hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
- [ ] Start timer:
|
- [ ] Start timer:
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl start hvac-content-aggregator.timer
|
sudo systemctl start hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
- [ ] Verify timer is active:
|
- [ ] Verify timer is active:
|
||||||
```bash
|
```bash
|
||||||
systemctl status hvac-content-aggregator.timer
|
systemctl status hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
### 7. Optional: TikTok Captions
|
### 7. Optional: TikTok Captions
|
||||||
|
|
@ -163,7 +163,7 @@
|
||||||
```
|
```
|
||||||
- [ ] No errors in service status:
|
- [ ] No errors in service status:
|
||||||
```bash
|
```bash
|
||||||
systemctl status hvac-content-aggregator.service
|
systemctl status hkia-content-aggregator.service
|
||||||
```
|
```
|
||||||
- [ ] Log files being created:
|
- [ ] Log files being created:
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -173,7 +173,7 @@
|
||||||
### First Run Verification
|
### First Run Verification
|
||||||
- [ ] Manually trigger first run:
|
- [ ] Manually trigger first run:
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl start hvac-content-aggregator.service
|
sudo systemctl start hkia-content-aggregator.service
|
||||||
```
|
```
|
||||||
- [ ] Monitor logs in real-time:
|
- [ ] Monitor logs in real-time:
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -241,7 +241,7 @@
|
||||||
- [ ] Check systemd timer status
|
- [ ] Check systemd timer status
|
||||||
- [ ] Review journal logs:
|
- [ ] Review journal logs:
|
||||||
```bash
|
```bash
|
||||||
journalctl -u hvac-content-aggregator.timer
|
journalctl -u hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
### If NAS Sync Fails
|
### If NAS Sync Fails
|
||||||
|
|
@ -255,7 +255,7 @@
|
||||||
### Quick Rollback
|
### Quick Rollback
|
||||||
1. [ ] Stop services:
|
1. [ ] Stop services:
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl stop hvac-content-aggregator.timer
|
sudo systemctl stop hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
2. [ ] Restore previous version:
|
2. [ ] Restore previous version:
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -264,7 +264,7 @@
|
||||||
```
|
```
|
||||||
3. [ ] Restart services:
|
3. [ ] Restart services:
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl start hvac-content-aggregator.timer
|
sudo systemctl start hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
### Full Rollback
|
### Full Rollback
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# Production Readiness Todo List
|
# Production Readiness Todo List
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
This document outlines all tasks required to meet the original specification and prepare the HVAC Know It All Content Aggregator for production deployment. Tasks are organized by priority and phase.
|
This document outlines all tasks required to meet the original specification and prepare the HKIA Content Aggregator for production deployment. Tasks are organized by priority and phase.
|
||||||
|
|
||||||
**Note:** Docker/Kubernetes deployment is not feasible due to TikTok scraping requiring display server access. The system uses systemd for service management instead.
|
**Note:** Docker/Kubernetes deployment is not feasible due to TikTok scraping requiring display server access. The system uses systemd for service management instead.
|
||||||
|
|
||||||
|
|
@ -26,7 +26,7 @@ This document outlines all tasks required to meet the original specification and
|
||||||
### File Organization
|
### File Organization
|
||||||
- [ ] Fix file naming convention to match spec format
|
- [ ] Fix file naming convention to match spec format
|
||||||
- Change from: `update_20241218_060000.md`
|
- Change from: `update_20241218_060000.md`
|
||||||
- To: `hvacknowitall_<source>_2024-12-18-T060000.md`
|
- To: `hkia_<source>_2024-12-18-T060000.md`
|
||||||
|
|
||||||
- [ ] Create proper directory structure
|
- [ ] Create proper directory structure
|
||||||
```
|
```
|
||||||
|
|
@ -306,7 +306,7 @@ sed -i 's/18:00:00/12:00:00/g' systemd/*.timer
|
||||||
|
|
||||||
# Phase 4: Test Deployment
|
# Phase 4: Test Deployment
|
||||||
./install_production.sh
|
./install_production.sh
|
||||||
systemctl status hvac-content-aggregator.timer
|
systemctl status hkia-content-aggregator.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
# HVAC Know It All - Deployment Strategy
|
# HKIA - Deployment Strategy
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
|
|
@ -76,20 +76,20 @@ After thorough testing and implementation, the content aggregation system has be
|
||||||
├── .env # Environment configuration
|
├── .env # Environment configuration
|
||||||
├── requirements.txt # Python dependencies
|
├── requirements.txt # Python dependencies
|
||||||
└── systemd/ # Service configuration
|
└── systemd/ # Service configuration
|
||||||
├── hvac-scraper.service
|
├── hkia-scraper.service
|
||||||
├── hvac-scraper-morning.timer
|
├── hkia-scraper-morning.timer
|
||||||
└── hvac-scraper-afternoon.timer
|
└── hkia-scraper-afternoon.timer
|
||||||
```
|
```
|
||||||
|
|
||||||
## NAS Integration
|
## NAS Integration
|
||||||
|
|
||||||
**Sync to**: `/mnt/nas/hvacknowitall/`
|
**Sync to**: `/mnt/nas/hkia/`
|
||||||
- Markdown files with timestamped archives
|
- Markdown files with timestamped archives
|
||||||
- Organized by source and date
|
- Organized by source and date
|
||||||
- Incremental sync to minimize bandwidth
|
- Incremental sync to minimize bandwidth
|
||||||
|
|
||||||
## Conclusion
|
## Conclusion
|
||||||
|
|
||||||
While the original containerized approach is not viable due to TikTok's GUI requirements, the direct deployment approach provides a robust and maintainable solution for the HVAC Know It All content aggregation system.
|
While the original containerized approach is not viable due to TikTok's GUI requirements, the direct deployment approach provides a robust and maintainable solution for the HKIA content aggregation system.
|
||||||
|
|
||||||
The system successfully aggregates content from 5 major sources with the option to include TikTok when needed, providing comprehensive coverage of the HVAC Know It All brand across digital platforms.
|
The system successfully aggregates content from 5 major sources with the option to include TikTok when needed, providing comprehensive coverage of the HKIA brand across digital platforms.
|
||||||
|
|
@ -1,8 +1,8 @@
|
||||||
# HVAC Know It All Content Aggregation System - Final Status
|
# HKIA Content Aggregation System - Final Status
|
||||||
|
|
||||||
## 🎉 Project Complete!
|
## 🎉 Project Complete!
|
||||||
|
|
||||||
The HVAC Know It All content aggregation system has been successfully implemented and tested. All 6 content sources are working, with deployment-ready infrastructure.
|
The HKIA content aggregation system has been successfully implemented and tested. All 6 content sources are working, with deployment-ready infrastructure.
|
||||||
|
|
||||||
## ✅ **All Sources Working (6/6)**
|
## ✅ **All Sources Working (6/6)**
|
||||||
|
|
||||||
|
|
@ -20,7 +20,7 @@ The HVAC Know It All content aggregation system has been successfully implemente
|
||||||
### ✅ Content Aggregation
|
### ✅ Content Aggregation
|
||||||
- **Incremental Updates**: Only fetches new content since last run
|
- **Incremental Updates**: Only fetches new content since last run
|
||||||
- **State Management**: JSON state files track last sync timestamps
|
- **State Management**: JSON state files track last sync timestamps
|
||||||
- **Markdown Generation**: Standardized format `hvacknowitall_{source}_{timestamp}.md`
|
- **Markdown Generation**: Standardized format `hkia_{source}_{timestamp}.md`
|
||||||
- **Archive Management**: Automatic archiving of previous content
|
- **Archive Management**: Automatic archiving of previous content
|
||||||
|
|
||||||
### ✅ Technical Infrastructure
|
### ✅ Technical Infrastructure
|
||||||
|
|
@ -30,7 +30,7 @@ The HVAC Know It All content aggregation system has been successfully implemente
|
||||||
- **Session Persistence**: Instagram login session reuse
|
- **Session Persistence**: Instagram login session reuse
|
||||||
|
|
||||||
### ✅ Data Management
|
### ✅ Data Management
|
||||||
- **NAS Synchronization**: rsync to `/mnt/nas/hvacknowitall/`
|
- **NAS Synchronization**: rsync to `/mnt/nas/hkia/`
|
||||||
- **File Organization**: Current and archived content separation
|
- **File Organization**: Current and archived content separation
|
||||||
- **Log Management**: Rotating logs with configurable retention
|
- **Log Management**: Rotating logs with configurable retention
|
||||||
|
|
||||||
|
|
@ -87,9 +87,9 @@ Total: 6/6 passed
|
||||||
│ ├── tiktok_scraper_advanced.py # TikTok Scrapling
|
│ ├── tiktok_scraper_advanced.py # TikTok Scrapling
|
||||||
│ └── orchestrator.py # Main coordinator
|
│ └── orchestrator.py # Main coordinator
|
||||||
├── systemd/ # Service configuration
|
├── systemd/ # Service configuration
|
||||||
│ ├── hvac-scraper.service
|
│ ├── hkia-scraper.service
|
||||||
│ ├── hvac-scraper-morning.timer
|
│ ├── hkia-scraper-morning.timer
|
||||||
│ └── hvac-scraper-afternoon.timer
|
│ └── hkia-scraper-afternoon.timer
|
||||||
├── test_data/ # Test results
|
├── test_data/ # Test results
|
||||||
│ ├── recent/ # Recent content tests
|
│ ├── recent/ # Recent content tests
|
||||||
│ └── backlog/ # Backlog tests
|
│ └── backlog/ # Backlog tests
|
||||||
|
|
@ -115,14 +115,14 @@ sudo ./install.sh
|
||||||
### **Manual Commands**
|
### **Manual Commands**
|
||||||
```bash
|
```bash
|
||||||
# Check service status
|
# Check service status
|
||||||
systemctl status hvac-scraper-morning.timer
|
systemctl status hkia-scraper-morning.timer
|
||||||
systemctl status hvac-scraper-afternoon.timer
|
systemctl status hkia-scraper-afternoon.timer
|
||||||
|
|
||||||
# Manual execution
|
# Manual execution
|
||||||
sudo systemctl start hvac-scraper.service
|
sudo systemctl start hkia-scraper.service
|
||||||
|
|
||||||
# View logs
|
# View logs
|
||||||
journalctl -u hvac-scraper.service -f
|
journalctl -u hkia-scraper.service -f
|
||||||
|
|
||||||
# Test individual sources
|
# Test individual sources
|
||||||
python -m src.orchestrator --sources wordpress instagram
|
python -m src.orchestrator --sources wordpress instagram
|
||||||
|
|
@ -204,7 +204,7 @@ python -m src.orchestrator --sources wordpress instagram
|
||||||
|
|
||||||
## 🏆 **Conclusion**
|
## 🏆 **Conclusion**
|
||||||
|
|
||||||
The HVAC Know It All content aggregation system successfully delivers on all requirements:
|
The HKIA content aggregation system successfully delivers on all requirements:
|
||||||
|
|
||||||
- **Complete Coverage**: All 6 major content sources working
|
- **Complete Coverage**: All 6 major content sources working
|
||||||
- **Production Ready**: Robust error handling and deployment infrastructure
|
- **Production Ready**: Robust error handling and deployment infrastructure
|
||||||
|
|
@ -212,6 +212,6 @@ The HVAC Know It All content aggregation system successfully delivers on all req
|
||||||
- **Reliable**: Comprehensive testing and proven real-world performance
|
- **Reliable**: Comprehensive testing and proven real-world performance
|
||||||
- **Maintainable**: Clean architecture with extensive documentation
|
- **Maintainable**: Clean architecture with extensive documentation
|
||||||
|
|
||||||
The system is ready for production deployment and will provide automated, comprehensive content aggregation for the HVAC Know It All brand across all digital platforms.
|
The system is ready for production deployment and will provide automated, comprehensive content aggregation for the HKIA brand across all digital platforms.
|
||||||
|
|
||||||
**Project Status: ✅ COMPLETE AND PRODUCTION READY**
|
**Project Status: ✅ COMPLETE AND PRODUCTION READY**
|
||||||
|
|
@ -209,7 +209,7 @@ k8s/ # Kubernetes manifests
|
||||||
- Storage usage
|
- Storage usage
|
||||||
|
|
||||||
## Version Control
|
## Version Control
|
||||||
- Private GitHub repository: https://github.com/bengizmo/hkia-content.git
|
- Private GitHub repository: https://github.com/bengizmo/hvacknowitall-content.git
|
||||||
- Commit after major milestones
|
- Commit after major milestones
|
||||||
- Semantic versioning
|
- Semantic versioning
|
||||||
- Comprehensive commit messages
|
- Comprehensive commit messages
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
# HVAC Know It All Content Aggregation - Project Status
|
# HKIA Content Aggregation - Project Status
|
||||||
|
|
||||||
## Current Status: 🟢 PRODUCTION READY
|
## Current Status: 🟢 PRODUCTION READY
|
||||||
|
|
||||||
|
|
@ -69,10 +69,10 @@
|
||||||
- **Service Files**: Complete systemd configuration provided
|
- **Service Files**: Complete systemd configuration provided
|
||||||
|
|
||||||
### Configuration Files
|
### Configuration Files
|
||||||
- `systemd/hvac-scraper.service` - Main service definition
|
- `systemd/hkia-scraper.service` - Main service definition
|
||||||
- `systemd/hvac-scraper.timer` - Scheduled execution
|
- `systemd/hkia-scraper.timer` - Scheduled execution
|
||||||
- `systemd/hvac-scraper-nas.service` - NAS sync service
|
- `systemd/hkia-scraper-nas.service` - NAS sync service
|
||||||
- `systemd/hvac-scraper-nas.timer` - NAS sync schedule
|
- `systemd/hkia-scraper-nas.timer` - NAS sync schedule
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -110,9 +110,9 @@
|
||||||
|
|
||||||
## Next Steps for Production
|
## Next Steps for Production
|
||||||
|
|
||||||
1. Install systemd services: `sudo systemctl enable hvac-scraper.timer`
|
1. Install systemd services: `sudo systemctl enable hkia-scraper.timer`
|
||||||
2. Configure environment variables in `/opt/hvac-kia-content/.env`
|
2. Configure environment variables in `/opt/hvac-kia-content/.env`
|
||||||
3. Set up NAS mount point at `/mnt/nas/hvacknowitall/`
|
3. Set up NAS mount point at `/mnt/nas/hkia/`
|
||||||
4. Monitor via systemd logs: `journalctl -f -u hvac-scraper.service`
|
4. Monitor via systemd logs: `journalctl -f -u hkia-scraper.service`
|
||||||
|
|
||||||
**Project Status: ✅ READY FOR PRODUCTION DEPLOYMENT**
|
**Project Status: ✅ READY FOR PRODUCTION DEPLOYMENT**
|
||||||
113
update_docs_to_hkia.py
Normal file
113
update_docs_to_hkia.py
Normal file
|
|
@ -0,0 +1,113 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Update all documentation files to use hkia naming convention.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logging.basicConfig(level=logging.INFO, format='%(message)s')
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
def update_doc_file(file_path: Path) -> bool:
|
||||||
|
"""Update documentation file to use hkia naming."""
|
||||||
|
try:
|
||||||
|
with open(file_path, 'r', encoding='utf-8') as f:
|
||||||
|
content = f.read()
|
||||||
|
|
||||||
|
original_content = content
|
||||||
|
|
||||||
|
# Replace various forms of the old naming
|
||||||
|
patterns = [
|
||||||
|
# Basic replacements
|
||||||
|
(r'hvacknowitall', 'hkia'),
|
||||||
|
(r'hvacnkowitall', 'hkia'),
|
||||||
|
(r'HVACKNOWITALL', 'HKIA'),
|
||||||
|
(r'HVACNKOWITALL', 'HKIA'),
|
||||||
|
|
||||||
|
# Full name replacements
|
||||||
|
(r'HVAC Know It All', 'HKIA'),
|
||||||
|
(r'HVAC KNOW IT ALL', 'HKIA'),
|
||||||
|
(r'HvacKnowItAll', 'HKIA'),
|
||||||
|
|
||||||
|
# Path replacements
|
||||||
|
(r'/mnt/nas/hvacknowitall', '/mnt/nas/hkia'),
|
||||||
|
(r'hvac-kia-content', 'hvac-kia-content'), # Keep repo name as is
|
||||||
|
|
||||||
|
# Service name replacements
|
||||||
|
(r'hvac-content-', 'hkia-content-'),
|
||||||
|
(r'hvac-scraper', 'hkia-scraper'),
|
||||||
|
|
||||||
|
# Keep these unchanged (repo/project names)
|
||||||
|
(r'github\.com/bengizmo/hkia-content', 'github.com/bengizmo/hvacknowitall-content'),
|
||||||
|
(r'hvac-kia-content', 'hvac-kia-content'), # project directory
|
||||||
|
]
|
||||||
|
|
||||||
|
for pattern, replacement in patterns:
|
||||||
|
content = re.sub(pattern, replacement, content, flags=re.IGNORECASE if not pattern.startswith('/') else 0)
|
||||||
|
|
||||||
|
# Fix any double replacements
|
||||||
|
content = content.replace('hkia-kia', 'hvac-kia')
|
||||||
|
|
||||||
|
if content != original_content:
|
||||||
|
with open(file_path, 'w', encoding='utf-8') as f:
|
||||||
|
f.write(content)
|
||||||
|
logger.info(f"✅ Updated: {file_path.name}")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
logger.info(f"ℹ️ No changes needed: {file_path.name}")
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"❌ Error updating {file_path}: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def main():
|
||||||
|
"""Main update process for documentation."""
|
||||||
|
logger.info("=" * 60)
|
||||||
|
logger.info("UPDATING DOCUMENTATION TO HKIA NAMING")
|
||||||
|
logger.info("=" * 60)
|
||||||
|
|
||||||
|
base_dir = Path('/home/ben/dev/hvac-kia-content')
|
||||||
|
|
||||||
|
# Documentation files to update
|
||||||
|
doc_files = [
|
||||||
|
'README.md',
|
||||||
|
'CLAUDE.md',
|
||||||
|
'DEPLOY.md',
|
||||||
|
'docs/project_specification.md',
|
||||||
|
'docs/image_downloads.md',
|
||||||
|
'docs/status.md',
|
||||||
|
'docs/final_status.md',
|
||||||
|
'docs/deployment_strategy.md',
|
||||||
|
'docs/DEPLOYMENT_CHECKLIST.md',
|
||||||
|
'docs/PRODUCTION_TODO.md',
|
||||||
|
'BACKLOG_STATUS.md',
|
||||||
|
'UPDATED_CAPTURE_STATUS.md',
|
||||||
|
'FINAL_TALLY_REPORT.md',
|
||||||
|
]
|
||||||
|
|
||||||
|
# Update documentation files
|
||||||
|
logger.info("\n📝 Updating documentation files...")
|
||||||
|
updated_count = 0
|
||||||
|
|
||||||
|
for file_path in doc_files:
|
||||||
|
full_path = base_dir / file_path
|
||||||
|
if full_path.exists():
|
||||||
|
if update_doc_file(full_path):
|
||||||
|
updated_count += 1
|
||||||
|
else:
|
||||||
|
logger.info(f"⚠️ File not found: {file_path}")
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
logger.info("\n" + "=" * 60)
|
||||||
|
logger.info("DOCUMENTATION UPDATE COMPLETE")
|
||||||
|
logger.info("=" * 60)
|
||||||
|
logger.info(f"Files updated: {updated_count}")
|
||||||
|
logger.info("\nNext steps:")
|
||||||
|
logger.info("1. Review changes with 'git diff'")
|
||||||
|
logger.info("2. Commit documentation updates")
|
||||||
|
logger.info("3. Push to repository")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Loading…
Reference in a new issue