hvac-kia-content/data_quick_test/.state/podcast_state.json at main - ben/hvac-kia-content - Forgejo: Beyond coding. We Forge.

ben/hvac-kia-content

Ben Reed 8b83185130 Fix HTML/XML contamination in WordPress markdown extraction

- Update base_scraper.py convert_to_markdown() to properly clean HTML
- Remove script/style blocks and their content before conversion
- Strip inline JavaScript event handlers
- Clean up br tags and excessive blank lines
- Fix malformed comparison operators that look like tags
- Add comprehensive HTML cleaning during content extraction (not after)
- Test confirms WordPress content now generates clean markdown without HTML

This ensures all future WordPress scraping produces specification-compliant
markdown without any HTML/XML contamination.

2025-08-18 23:11:08 -03:00

7 lines

No EOL

196 B

JSON

Raw Permalink Blame History

 {
   "last_update": "2025-08-18T21:58:04.803753",
   "last_item_count": 5,
   "backlog_captured": true,
   "backlog_timestamp": "20250818_215804",
   "last_id": "185a21b3-66e1-4472-a0e8-65bbc66f5217"
 }