dabef8bfcb
Major Production Improvements: - Added retry logic with exponential backoff using tenacity - Implemented HTTP connection pooling via requests.Session - Added health check monitoring with metrics reporting - Implemented configuration validation for all numeric values - Fixed error isolation (verified continues on failure) Technical Changes: - BaseScraper: Added session management and make_request() method - WordPressScraper: Updated all HTTP calls to use retry logic - Production runner: Added validate_config() and health check ping - Retry config: 3 attempts, 5-60s exponential backoff System is now production-ready with robust error handling, automatic retries, and health monitoring. Remaining tasks focus on spec compliance (media downloads, markdown format) and testing/documentation. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
79 lines
1.3 KiB
Text
79 lines
1.3 KiB
Text
aiohappyeyeballs==2.6.1
|
|
aiohttp==3.12.15
|
|
aiosignal==1.4.0
|
|
anyio==4.10.0
|
|
attrs==25.3.0
|
|
beautifulsoup4==4.13.4
|
|
brotli==1.1.0
|
|
browserforge==1.2.3
|
|
camoufox==0.4.11
|
|
certifi==2025.8.3
|
|
charset-normalizer==3.4.3
|
|
click==8.2.1
|
|
coloredlogs==15.0.1
|
|
cssselect==1.3.0
|
|
defusedxml==0.7.1
|
|
feedparser==6.0.11
|
|
filelock==3.19.1
|
|
flatbuffers==25.2.10
|
|
frozenlist==1.7.0
|
|
geoip2==5.1.0
|
|
greenlet==3.2.4
|
|
h11==0.16.0
|
|
httpcore==1.0.9
|
|
httpx==0.28.1
|
|
humanfriendly==10.0
|
|
idna==3.10
|
|
iniconfig==2.1.0
|
|
instaloader==4.14.2
|
|
language-tags==1.2.0
|
|
lxml==6.0.0
|
|
magika==0.6.2
|
|
markdownify==1.2.0
|
|
markitdown==0.1.2
|
|
maxminddb==2.8.2
|
|
mpmath==1.3.0
|
|
multidict==6.6.4
|
|
numpy==2.3.2
|
|
onnxruntime==1.22.1
|
|
orjson==3.11.2
|
|
packaging==25.0
|
|
platformdirs==4.3.8
|
|
playwright==1.54.0
|
|
playwright-stealth==2.0.0
|
|
pluggy==1.6.0
|
|
propcache==0.3.2
|
|
protobuf==6.32.0
|
|
pyee==13.0.0
|
|
pygments==2.19.2
|
|
pysocks==1.7.1
|
|
pytest==8.4.1
|
|
pytest-asyncio==1.1.0
|
|
pytest-mock==3.14.1
|
|
python-dotenv==1.1.1
|
|
pytz==2025.2
|
|
pyyaml==6.0.2
|
|
rebrowser-playwright==1.52.0
|
|
requests==2.32.4
|
|
requests-file==2.1.0
|
|
tenacity==8.2.3
|
|
schedule==1.2.2
|
|
scrapling==0.2.99
|
|
screeninfo==0.8.1
|
|
sgmllib3k==1.0.0
|
|
six==1.17.0
|
|
sniffio==1.3.1
|
|
socksio==1.0.0
|
|
soupsieve==2.7
|
|
sympy==1.14.0
|
|
tiktokapi==7.1.0
|
|
tldextract==5.3.0
|
|
tqdm==4.67.1
|
|
typing-extensions==4.14.1
|
|
ua-parser==1.0.1
|
|
ua-parser-builtins==0.18.0.post1
|
|
urllib3==2.5.0
|
|
w3lib==2.3.1
|
|
yarl==1.20.1
|
|
yt-dlp==2025.8.11
|
|
zstandard==0.24.0
|