hvac-kia-content/data_quick_test/.state/youtube_state.json at 299eb35910190acf14d838dc478793a6f3f50f4b - ben/hvac-kia-content - Forgejo: Beyond coding. We Forge.

ben/hvac-kia-content

Ben Reed 8b83185130 Fix HTML/XML contamination in WordPress markdown extraction

- Update base_scraper.py convert_to_markdown() to properly clean HTML
- Remove script/style blocks and their content before conversion
- Strip inline JavaScript event handlers
- Clean up br tags and excessive blank lines
- Fix malformed comparison operators that look like tags
- Add comprehensive HTML cleaning during content extraction (not after)
- Test confirms WordPress content now generates clean markdown without HTML

This ensures all future WordPress scraping produces specification-compliant
markdown without any HTML/XML contamination.

2025-08-18 23:11:08 -03:00

7 lines

No EOL

171 B

JSON

Raw Blame History

 {
   "last_update": "2025-08-18T21:58:33.774108",
   "last_item_count": 5,
   "backlog_captured": true,
   "backlog_timestamp": "20250818_215833",
   "last_id": "70hcZ1wB7RA"
 }