WordPress scrapper
Extract the full structure and content from any public WordPress website via its built-in REST API — no admin access required. Use the result for research or to seed a chatbot knowledge base.
What the scrapper does
The WordPress scrapper connects to any public WordPress site through the standard REST API and retrieves published content — posts, pages, categories, and tags — into an organized, browsable view inside WebGPT. Because it uses the built-in REST API (enabled by default on virtually all WordPress sites), no login credentials or admin access are needed. Enter a URL and the scrapper handles the rest.
Running a scrape
- Open the scrapper — Utility Tools → WordPress Scrapper, or the WordPress Scrapper card on the dashboard.
- Enter a site URL — full URL including protocol, e.g.
https://example.com. Subdirectory installations likehttps://example.com/blogalso work. - Click Get. The URL is verified as a reachable WordPress site before the scrape starts — non-WordPress URLs and unreachable sites are flagged inline on the URL field with a specific reason (not a WordPress site, REST API blocked, connection timed out, etc.) and no usage is consumed. Each successful scrape counts against the daily usage limit.
- Wait for progress to complete. Depending on site size, this takes a few seconds up to a couple of minutes.
- Browse the results organized by tab: Posts, Pages, Categories, and Tags. Click any item title to read the full content in a modal preview.
/wp-json/. Sites with the REST API disabled (a choice some owners make for security) are flagged with a "REST API blocked" message at submission time. Most WordPress sites leave the REST API enabled by default.
Save into a chatbot's knowledge base
Scraped posts and pages can be added directly to a chatbot's knowledge base from the results view. Select items, choose the target chatbot, and the content is saved with appropriate tags — a fast way to seed a chatbot from an existing WordPress site's content. See AI chatbots.
History
The History tab records every site previously scraped, so it's possible to reopen past results without re-running the scrape. Useful when comparing how a site has evolved, revisiting content during a longer research session, or queuing up interesting sites to save into chatbots later.
Usage
Each scrape counts as one use against the daily WordPress scrapper limit on your plan. The usage indicator on the page shows current consumption and the remaining quota; the Get button disables when the limit is reached and re-enables at the next daily reset (00:00 UTC). See Plans & usage limits.