Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Cloudflare

The cloudflare Cargo feature ships a driver that bypasses Cloudflare's interactive Turnstile challenge — the "verify you are human" checkbox page that gates many sites behind a CDN. It is not a generic anti-Cloudflare solution; it specifically automates clicking the visible Turnstile checkbox iframe and waiting for the resulting clearance token.

Enable it in Cargo.toml:

[dependencies]
zendriver = { version = "0.1", features = ["cloudflare"] }

The entry point is Tab::cloudflare, which constructs a CloudflareBypass driver scoped to that tab's session. Call wait_for_clearance with a timeout to run the full detect-click-poll flow.

Usage

//! Demonstrates the P5 Cloudflare Turnstile bypass driver.
//!
//! Sequence:
//!   1. Launch a headless browser and navigate to a Cloudflare-protected
//!      URL. Set this to whatever endpoint you actually want to clear.
//!   2. Call [`Tab::cloudflare`] to construct a [`CloudflareBypass`] bound
//!      to the tab's session.
//!   3. Call [`CloudflareBypass::wait_for_clearance`] with a 30s budget.
//!      The driver runs a single CDP poll loop that, per tick, looks for
//!      the clearance token, the Turnstile challenge iframe's bounding box
//!      (shadow-DOM aware), and any challenge marker on the page. When the
//!      interactive iframe is present, the driver scrolls it into view,
//!      re-measures it, and dispatches a raw left-click at the canonical
//!      (15% x, 50% y) offset (Turnstile checkbox), retrying up to three
//!      times in case the widget swallowed the first one. Resolves the
//!      first tick a token is observed, or the challenge stops being
//!      actionable.
//!   4. Print the [`ClearanceOutcome`] enum variant + the page title.
//!
//! Which markers to look for and how to click are caller data —
//! `.selectors(TurnstileSelectors { .. })`,
//! `.click_policy(ClickPolicy { .. })` and `.on_click(..)` — so a change on
//! Cloudflare's side is a change here, not a release of the crate. This
//! example takes the defaults.
//!
//! Outcomes:
//!   - `TokenAcquired(_)` — Turnstile yielded a `cf-turnstile-response`
//!     token, either after clicking the checkbox or directly (invisible
//!     Turnstile, where the iframe never mounts).
//!   - `ChallengeGone` — the challenge stopped being actionable without a
//!     token (e.g. clearance-cookie shortcut): the clicked iframe is no
//!     longer a valid click target, or every marker seen earlier is gone.
//!     A widget that is merely hidden or zero-size is also not a valid
//!     click target, so this is not by itself proof the gate was passed.
//!   - `TimedOut { saw_challenge: false }` — the full timeout window
//!     elapsed without any challenge markers ever being observed (no
//!     container, no hidden input, no iframe). The page likely has no
//!     Cloudflare gate; not a failure.
//!   - `TimedOut { saw_challenge: true }` — 30s elapsed with markers
//!     present but neither success state observed.
//!
//! Requires the `cloudflare` cargo feature:
//! `cargo run --example cloudflare_bypass --features cloudflare`.

use std::time::Duration;

use zendriver::{Browser, ClearanceOutcome};

#[tokio::main]
#[allow(clippy::result_large_err)] // example boundary; users wrap in their own Error
async fn main() -> zendriver::Result<()> {
    tracing_subscriber::fmt::init();

    let browser = Browser::builder().headless(true).launch().await?;
    let tab = browser.main_tab();

    tab.goto("https://nopecha.com/demo/cloudflare").await?;
    tab.wait_for_load().await?;

    match tab
        .cloudflare()
        .wait_for_clearance(Duration::from_secs(30))
        .await
    {
        Ok(ClearanceOutcome::TimedOut {
            saw_challenge: false,
        }) => {
            println!("no challenge detected (page already cleared or no CF gate present)");
        }
        Ok(ClearanceOutcome::TimedOut {
            saw_challenge: true,
        }) => {
            println!("clearance timed out within 30s");
        }
        Ok(outcome) => println!("cleared: {outcome:?}"),
        Err(e) => return Err(e.into()),
    }

    let title = tab.title().await?;
    println!("title = {title:?}");

    browser.close().await?;
    Ok(())
}

The driver returns a ClearanceOutcome on success. All three are ordinary terminals, the last one included:

  • TokenAcquired(token) — the cf-turnstile-response input picked up a non-empty value. The page can now proceed; the token is also forwarded to Cloudflare server-side on the next request.
  • ChallengeGone — the challenge stopped being actionable without yielding a token, typically because Cloudflare honored a clearance cookie and short-circuited the gate. Either the iframe the driver clicked is no longer a valid click target, or every challenge marker seen on an earlier tick has vanished. The first case is weaker than it sounds: a widget that is still mounted but hidden or zero-sized is also not a valid click target, so once a click has landed this outcome can arrive on a tick where the widget simply is not clickable at that moment. That case says the driver ran out of things to click, not that the gate is behind you; confirm the page before trusting what you scrape next.
  • TimedOut { saw_challenge } — the deadline elapsed. saw_challenge separates the two cases worth telling apart: true means a real challenge sat on the page and never resolved, false means no Cloudflare marker ever appeared and the bypass was probably called on a page with no gate on it.

A deadline is not an error here, so CloudflareError is left to carry genuine faults, of which there are two:

  • Call — the underlying CDP call failed, usually a dead tab or a closed connection.
  • JsError — the in-page evaluator raised, or handed back a payload the driver could not decode.

The enum is #[non_exhaustive], so a match on it needs a wildcard arm. There is no NoChallenge or ClearanceTimeout variant — if you are looking for those, the first is TimedOut { saw_challenge: false } and the second is TimedOut { saw_challenge: true }.

How it works

The driver runs a single poll loop, and each tick costs one CDP round-trip. A shadow-DOM-aware walk of the page's main world reports three things at once: the cf-turnstile-response token if one is present, the challenge iframe's box if that iframe is a valid click target, and whether any Cloudflare marker at all (container, hidden input, or live iframe) is on the page. The tick then resolves in this order:

  1. A token wins outright. A non-empty token returns TokenAcquired immediately. This is also the invisible-Turnstile path, where no iframe mounts and Cloudflare's loader script fills the field in without anything to click.
  2. Scroll, re-measure, click. Raw mouse events carry viewport coordinates, so a widget below the fold has to be scrolled in and re-measured before a click can land on it. The click goes to (bbox.x + bbox.width * 0.15, bbox.y + bbox.height * 0.5) — by default the 15%-from-left, 50%-from-top position of the checkbox inside the iframe. No Bezier-path motion; Cloudflare wants a real click on a real checkbox. A widget that is mounted but hidden or zero-sized is never clicked, since there is no meaningful point to click.
  3. Retry, up to a cap. Cloudflare drops clicks that land while the widget is still booting, so one run spends up to three of them, each at least four poll ticks after the last (about two seconds at the default interval). A swallowed first click no longer strands the run until the deadline.
  4. Otherwise keep polling — every 500 ms by default, override via poll_interval — until a terminal fires or the deadline passes.

After ten consecutive ticks with no progress the driver logs a warning asking whether BrowserBuilder::stealth is on, which is the answer most of the time. See Pairing with stealth below. A run configured never to click (max_attempts: 0, below) is exempt — nothing is stalled there, and stealth would be the wrong thing to go change.

Every number and every selector in that description is a default, not a constant — the next section is how to change them.

Markers and clicks are yours

Cloudflare owns the class names, the hidden input's name, the iframe host and where the checkbox sits inside the widget. They change all of it on their own schedule and with no notice. A crate that writes those into its source needs a release every time they move, and everyone using it is stuck on the old literals until that release lands.

So they aren't in the source. TurnstileSelectors is what the injected evaluators are built from, and ClickPolicy decides whether and where to click:

use zendriver::{ClickPolicy, TurnstileSelectors};

tab.cloudflare()
    .selectors(TurnstileSelectors {
        // Cloudflare moved the widget behind a new host.
        iframe_src_contains: "challenges.example-cdn.com".into(),
        ..TurnstileSelectors::default()
    })
    .click_policy(ClickPolicy {
        max_attempts: 5,
        ..ClickPolicy::default()
    })
    .wait_for_clearance(Duration::from_secs(30))
    .await?;

Both are plain structs with public fields and no closures in them, which is what lets browser_solve_turnstile take them over MCP too — an agent that finds the defaults no longer matching a page can hand the solver the new markers itself.

The defaults:

FieldDefault
TurnstileSelectors::iframe_src_contains"challenges.cloudflare.com"
TurnstileSelectors::container".cf-turnstile, .turnstile, [data-sitekey]"
TurnstileSelectors::token_inputs["[name=\"cf-turnstile-response\"]", "[name=\"cf_challenge_response\"]"]
ClickPolicy::max_attempts3 (0 watches without ever clicking)
ClickPolicy::retry_ticks4
ClickPolicy::x_fraction0.15
ClickPolicy::y_fraction0.5

token_inputs is tried in order, so a page carrying both the modern and the legacy input is read the modern way. An empty value turns a marker off rather than widening it: an empty container drops the container signal, an empty iframe_src_contains matches no iframe (rather than every iframe, which is what a bare substring test would otherwise do), and an empty token_inputs leaves only the marker-vanished and deadline terminals.

Two things stay fixed, because they are the library's judgement rather than Cloudflare's markup. What counts as clickable — a widget needs non-zero size and must not be hidden by visibility, display or opacity — and how it is brought into view, a centred instant scroll.

Replacing the click entirely

The click is the part most likely to stop working, and a policy struct can only describe clicks that look like the built-in one. on_click hands the whole step over: it is called once per attempt with the session and the widget's post-scroll box, and what happens next is up to you — a human-ish pointer path, a solver service, a click through a higher level of the library.

tab.cloudflare()
    .on_click(|session, target| async move {
        my_humanized_click(&session, target.x, target.y).await
    })
    .wait_for_clearance(Duration::from_secs(30))
    .await?;

A handler counts against max_attempts exactly as the built-in click does, and an error from it fails the run rather than being retried.

It replaces the click, not the decision to click. The driver still applies its own clickability rule first, so a handler installed specifically to reach a hidden or zero-size widget never runs — that widget needs a different approach entirely.

Limitations

The driver clicks the visible interactive Turnstile checkbox, and picks up the token on the invisible path when Cloudflare's own script produces one. It does not solve:

  • Silent / invisible Turnstile. There is no UI element to click and the verdict comes from passive fingerprinting. The loop returns TokenAcquired the moment the field is populated, but nothing here makes Cloudflare populate it — that is stealth's job. Pair StealthProfile::spoofed() with a clean residential IP.
  • Cloudflare's full Pro / Enterprise managed challenge (which can escalate to image puzzles or even hCaptcha).
  • Bot Fight Mode soft blocks that issue 403s without a UI.
  • Rate-limit blocks (1015 errors) that don't expose a challenge UI at all.

If the bypass times out, switch to a real browser session, manually inspect the page, and confirm whether the gate is the interactive checkbox flow. If it's not, this driver can't help and you'll need a different strategy (better stealth, rotating residential proxies, or giving up on that target).

Pairing with stealth

Cloudflare's challenge logic checks several signals before deciding whether to show the visible checkbox or escalate to the silent flow: TLS JA3 fingerprint, User-Agent, header order, navigator.webdriver, etc. Out-of-the-box headless Chrome trips most of those, so it tends to get the harder challenge path — sometimes one this driver can't pass.

Pair the bypass with StealthProfile::spoofed() for the best results:

use zendriver::{Browser, StealthProfile};

let browser = Browser::builder()
    .stealth(StealthProfile::spoofed())  // patches navigator.webdriver etc.
    .launch()
    .await?;
let tab = browser.main_tab();
tab.goto("https://target.example.com").await?;
tab.wait_for_load().await?;

tab.cloudflare()
    .wait_for_clearance(std::time::Duration::from_secs(30))
    .await?;

spoofed patches the Navigator-prototype tells that Cloudflare also checks during the protocol-level challenge — together they pass most consumer-site Cloudflare gates. See Stealth for the profile tradeoffs.

When to call it

Call wait_for_clearance after the navigation completes but before any post-challenge code that depends on being past the gate. The typical sequence:

tab.goto(url).await?;
tab.wait_for_load().await?;

match tab.cloudflare()
    .wait_for_clearance(Duration::from_secs(30))
    .await?
{
    ClearanceOutcome::TokenAcquired(_) | ClearanceOutcome::ChallengeGone => {
        // Past the gate.
    }
    ClearanceOutcome::TimedOut { saw_challenge: false } => {
        // No Cloudflare marker ever appeared — this page has no gate.
        // Usually fine to continue.
    }
    ClearanceOutcome::TimedOut { saw_challenge: true } => {
        // A real challenge that never resolved. Worth failing the job.
        return Err("cloudflare challenge did not clear".into());
    }
}

// Now your normal scraping / interaction code.
let data = tab.find().css(".product-grid").one().await?;

The ? propagates the two real faults, Call and JsError. Everything else is a terminal you decide about, and the saw_challenge split is the one that changes what you do: "no gate on this page" and "a gate we could not pass" want opposite handling.

Tuning

  • .poll_interval(Duration::from_millis(200)) — tighter polling burns more CPU but reacts faster to clearance. Defaults to 500 ms which balances responsiveness against load.
  • .selectors(..) / .click_policy(..) / .on_click(..) — the markers and the click, covered in Markers and clicks are yours.
  • Pass a generous wait_for_clearance timeout (30-60 s) for the first challenge; subsequent navigations on the same user_data_dir are usually cookie-shortcut clears and resolve in <1 s via ChallengeGone.