# Scrapling v0.4.15 — Release v0.4.15

- Product: Scrapling (https://whatsnew.fyi/product/scrapling)
- Vendor: Scrapling
- Date: 2026-08-23
- Version: v0.4.15
- Original notes: https://github.com/D4Vinci/Scrapling/releases/tag/v0.4.15
- Permalink: https://whatsnew.fyi/product/scrapling/releases/v0.4.15

What's New is an index, not a publisher: every entry below links to the vendor's own release notes, which are the authoritative source. Entries are labelled where they are hand-curated sample data, pre-releases, or drawn from a secondary source such as a developer blog.

Reuse: the summaries, labels and curation here are © What's New. Quote freely with attribution and a link back; wholesale republication of the corpus is not permitted — terms: https://whatsnew.fyi/terms. The vendors' own release notes remain their publishers'.

---

- **added** — Browser tabs now stay open and get reused across requests, with automatic reapplication of settings to prevent leakage between requests
- **added** — New `Response.markdown()` method to convert any page into clean, LLM-ready Markdown with option to extract main content only or specific CSS selectors
- **added** — New `SiteToMarkdownSpider` template to crawl entire websites into Markdown corpus for RAG pipelines with configurable output directory and page limits
- **added** — New `rag` extra for pip installation that includes RAG-ready functionality
- **added** — New `close_pages()` method to close all open browser tabs
- **added** — New `session_fetch` tool in MCP server for fetching through a browser session
- **added** — New `open_request_session` and `session_make_request` tools in MCP server for persistent HTTP sessions that keep cookies and browser fingerprint between requests
- **changed** — MCP server split into one-shot tools (fetch, bulk_fetch, stealthy_fetch, bulk_stealthy_fetch) and session tools for different use cases
- **changed** — MCP server `get` tool renamed to `make_request` with support for any HTTP method
- **changed** — MCP server `open_session` now holds only browser-level settings and returns the session's effective settings for the AI agent
- **changed** — MCP server HTTP transport now requires authentication by default and binds to localhost, with `--auth-token` flag or `SCRAPLING_MCP_AUTH_TOKEN` environment variable for bearer token authentication
- **changed** — Page setup functions on subsequent requests now run on the previously loaded page before navigating away, enabling automation chaining across requests
- **fixed** — Cloudflare Turnstile and Interstitial solving now works regardless of browser locale and no longer loops forever on interactive challenges in headless mode
- **fixed** — Cloudflare stealth pages no longer crash mid-solve
- **fixed** — Fixed `find` and `find_all` with `class_` silently missing multi-class elements
- **fixed** — Fixed blank `class_` values and unescaped CSS string values in element selection
- **fixed** — Fixed cached responses in `development_mode` losing the request meta on replay
- **fixed** — Fixed HTTP requests with `retries` below 1 failing without sending the request

**One of the biggest releases this year: a reworked MCP server, RAG-ready Markdown in one line, an improved Cloudflare solver, and browser tabs that stay open for automation 🚀**

> [!NOTE]
> **[Follow us on X for daily tips and tricks](https://x.com/Scrapling_dev)**

> [!WARNING]
> **This release introduces breaking changes to the MCP server. Check the [breaking changes](https://scrapling.readthedocs.io/en/latest/ai/mcp-server.html#breaking-changes) section before updating.**

##### 🚀 New Stuff and quality of life changes

- **Browser tabs now stay open and get reused across requests** (Check the [docs](https://scrapling.readthedocs.io/en/latest/fetching/dynamic.html#session-management)):
    - All browser sessions keep their tabs after a request, and the next request reuses a free tab instead of opening a new one.
    - Every request re-applies its own settings (timeouts, headers, resource blocking) to the tab it gets, so nothing leaks between requests.
    - Tabs that hit an error are closed and replaced, and the new `close_pages()` method closes every open tab.
    - The page you fetched stays loaded, so a `page_setup` function on the next request runs on it before navigating away. That's the building block for chaining automation across requests.
- **Turn any page into clean, LLM-ready Markdown with `Response.markdown()`** (Check the [docs](https://scrapling.readthedocs.io/en/latest/ai/building-rag-systems.html)):
    ```python
    from scrapling.fetchers import Fetcher

    markdown = Fetcher.get("https://example.com").markdown(main_content_only=True)
    ```
    - Scripts, styles, and hidden prompt-injection content are always stripped first, the same as the cleaning the MCP server does.
    - Pass `css_selector` to convert only the elements you need.
    - Available through the new `rag` extra (`pip install "scrapling[rag]"`), which the `ai`/`shell`/`all` extras include too.
- **New `SiteToMarkdownSpider` template to crawl a whole website into a Markdown corpus for RAG pipelines** (Check the [docs](https://scrapling.readthedocs.io/en/latest/ai/building-rag-systems.html)):
    ```python
    from scrapling.spiders import SiteToMarkdownSpider

    class DocsSpider(SiteToMarkdownSpider):
        name = "docs"
        start_urls = ["https://example.com/docs/"]
        allowed_domains = {"example.com"}
        output_dir = "docs_markdown"

    result = DocsSpider().start()
    result.items.to_jsonl("docs.jsonl")
    ```
    - Yields one item per page with `url`/`title`/`markdown`, and the optional `output_dir` writes one Markdown file per page.
    - `max_pages` caps the crawl, and since it builds on `CrawlSpider`, overriding `rules()` gives you full control over which links get followed.
- **The MCP server is reworked (breaking)** (Check the [breaking changes](https://scrapling.readthedocs.io/en/latest/ai/mcp-server.html#breaking-changes) and the [docs](https://scrapling.readthedocs.io/en/latest/ai/mcp-server.html)):
    - The 13 tools are now split into two modes: one-shot tools (`fetch`, `bulk_fetch`, `stealthy_fetch`, `bulk_stealthy_fetch`) that always launch their own browser and show their real defaults, and session tools that work through a session opened once.
    - The new `session_fetch` tool fetches through a browser session, while `open_session` now holds the browser-level settings only and returns the session's effective settings for the AI agent.
    - The `get` tool is renamed to `make_request`, and it now supports any HTTP method.
    - The new `open_request_session` and `session_make_request` tools give the AI persistent HTTP sessions that keep cookies and the browser fingerprint between requests.
    - This also ends fetches resetting the session's settings, first fixed by @Yigtwxx in [#418](https://github.com/D4Vinci/Scrapling/pull/418).
- **The MCP server's HTTP transport now requires authentication and binds to localhost by default (breaking)** by @yamantaka-

_[Truncated at 4000 characters — full notes: https://github.com/D4Vinci/Scrapling/releases/tag/v0.4.15]_