crawler

ticket-monitoring

+154

✈️ 自托管的多渠道机票价格监控：五平台明细采集 → 钉钉图文推送 → 达标四层强提醒。国内航线与国际航线（五洲目的地）同一套采集-判定-推送链路；数据留在本机，配置全在网页，开源免费。

firecrawl

+2.1k

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Scrapling

+1k

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

crawl4ai

+414

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord https://discord.gg/VuxzM8mtf

creatorhub

+179

多平台内容监控·采集·搬运 一个 Web 面板管起抖音 / 小红书 / 快手

scrapy/scrapy

+140

Scrapy, a fast high-level web crawling & scraping framework for Python.

Douyin_TikTok_Download_API

+112

🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos. Self-healing identity pool, PostgreSQL archive, one docker compose up.

JMComic-Crawler-Python

+91

Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀

Scrapegraph-ai

+181

Python scraper based on AI