Consent-Aware Browser Agents and CMPs

Consent management is not just a UX overlay. For privacy-preserving automation, scraping, testing, and RPA, it is a protocol surface with legal and technical semantics that must be handled deterministically. If your headless agents click Accept All on a dark-patterned banner by accident, you may unintentionally authorize cross-site tracking and violate user privacy expectations. If you ignore consent state, your CI/CD tests will be flaky across regions and devices.

This article provides a deeply technical playbook for building consent-aware browser agents that: detect CMPs reliably; set Global Privacy Control (GPC); choose minimal, compliant consent; persist IAB TCF v2.2 and US Privacy strings; sandbox trackers with Service Worker and CSP; and deterministically replay consent state in CI/CD. The focus is practical: concrete APIs, code snippets, and integration patterns for Playwright, Puppeteer, and Selenium.

1. Why consent-aware agents now

• Regulatory momentum: GDPR enforcement actions and California Privacy Rights Act (CPRA) drive stronger opt-out defaults and tighter definitions of consent. The Global Privacy Control (GPC) specification gives a machine-readable opt-out signal that publishers are expected to honor.
• Platform shifts: Storage partitioning, ITP, and strengthened CSP/Permissions-Policy change how trackers operate and how your tests behave. Consent signals are increasingly tied to tag behavior and server responses.
• Engineering ergonomics: CI pipelines need reproducible consent state to make analytics, paywalls, and ad-tech integration tests stable across geos. Manual banner clicking is brittle and slow; protocol-level control is faster and deterministic.

Key idea: treat consent as data, not clicks. Instrument the consent protocol surface first; only fall back to UI automation when necessary.

2. Signals and specs you must speak

A minimal consent-aware agent should support at least three surfaces:

• GPC: A browser-level do-not-sell/share preference. Sites detect via the Sec-GPC request header and the navigator.globalPrivacyControl boolean.
• IAB TCF v2.2: A standardized encoding of user choices for GDPR region. CMPs expose a __tcfapi JavaScript interface and store the TC string in cookies and localStorage.
• US Privacy (CCPA/CPRA) and IAB GPP: A compact string (commonly usprivacy) or a GPP multi-section string exposed via __uspapi or __gpp.

A short primer follows.

2.1 GPC

• Transport: HTTP request header Sec-GPC: 1, applied to all top-level and subresource requests when enabled; and a JS property navigator.globalPrivacyControl === true.
• Semantics: A user preference to opt out of sale/sharing or targeted advertising where applicable. Legal effect depends on jurisdiction.

2.2 IAB TCF v2.2 basics

• Surface: window.__tcfapi(command, version, callback[, parameter]) that CMPs must provide. Core commands include addEventListener, getTCData, and ping.
• Storage: The TC string (TCF v2.2) is commonly persisted as cookie euconsent-v2 at the eTLD+1, and mirrored to localStorage under IABTCF_* keys (e.g., IABTCF_TCString, IABTCF_PurposeConsents, IABTCF_VendorConsents). Exact storage can vary by CMP.
• Encoding: The TC string is a base64url-encoded bit field describing purposes, vendors, legitimate interest flags, and publisher restrictions. Libraries exist to parse and generate it.
• Evolution: v2.2 tightens transparency and vendor disclosure expectations. Vendors have reduced ability to rely solely on legitimate interest for certain advertising purposes and must communicate retention and data categories more clearly in CMP UIs. From an agent standpoint, the API surface (__tcfapi) and data containers remain familiar; the governance around consent options and defaults is stricter.

Always consult the latest IAB Europe documentation for canonical rules and migration notes.

2.3 US Privacy and GPP

• Legacy CCPA: window.__uspapi(command, version, callback). The usprivacy string (e.g., 1YNN) communicates opt-out. Storage often in a usprivacy cookie and/or IABUSPrivacy_String localStorage key.
• GPP: The IAB Global Privacy Platform generalizes multiple US state frameworks via window.__gpp. It returns a GPP string with separate sections (e.g., for US National). While many sites still use __uspapi, expect __gpp adoption to grow.

Your agent should detect and honor whichever of __uspapi or __gpp is present, preferring __gpp when both exist for richer statefulness.

3. Threat model: dark patterns and brittle UIs

• Dark-pattern banners: Ambiguous copy, low-contrast reject buttons, or multi-step flows that try to elicit Accept All. Your agent must not rely on arbitrary CSS selectors that break or lead to an implicit accept.
• Shadow DOM and iframes: CMPs frequently render inside sandboxed iframes, nested in shadow roots, with domain isolation. UI automation is brittle across vendors and locales.
• Re-prompts and region-dependent logic: Consent options differ by IP, Accept-Language, timezone, and geolocation. What worked yesterday in staging may fail today in Paris.

Solution: focus on protocol endpoints and canonical storages first. Use UI automation only when you must convey a choice through the CMP’s officially supported controls.

4. Architecture: a consent-aware agent pipeline

A robust agent implements the following stages:

1. Initialize privacy posture

• Enable GPC at the network edge and in JS. Normalize Accept-Language, timezone, and geolocation so the site chooses a consistent legal framework.

2. Detect CMPs programmatically

• Probe for __tcfapi, __uspapi, and __gpp. Listen for tcloaded and useractioncomplete events via __tcfapi('addEventListener', 2, cb). Use ping to confirm CMP status even when UI is hidden.

3. Choose minimal consent deterministically

• Prefer a direct protocol call or vendor-provided API to set minimal consent. If unavailable, drive the CMP UI in a vendor-specific, auditable way, always searching for Reject All or equivalent.

4. Persist consent artifacts

• Write and verify IAB strings (TCF, US Privacy, or GPP) and any vendor-specific cookies so that the agent preserves state across navigations and origins under test.

5. Enforce via sandboxing

• Deploy a Service Worker (where you control the origin) or a network proxy/extension to block, strip, or downgrade third-party trackers unless consent is present. Reinforce with CSP and Permissions-Policy where possible.

6. Deterministic replay in CI/CD

• Snapshot the consent state into fixtures, lock environment inputs (IP, UA, time), and replay the exact state in parallel jobs and across regions.

The remainder of the article details concrete implementations for each stage.

5. Enabling GPC in automation stacks

Set both the network header and the JS property.

5.1 Playwright

ts
import { chromium } from 'playwright';

(async () => {
  const browser = await chromium.launch();
  const context = await browser.newContext({
    extraHTTPHeaders: {
      'Sec-GPC': '1',
    },
    locale: 'en-US',
    timezoneId: 'Europe/Paris', // lock jurisdictional behavior if desired
  });

  // Ensure navigator.globalPrivacyControl === true
  await context.addInitScript(() => {
    try {
      Object.defineProperty(Navigator.prototype, 'globalPrivacyControl', {
        get: () => true,
        configurable: true,
      });
    } catch {}
  });

  const page = await context.newPage();
  await page.goto('https://example.com');
})();

Notes:

• Some sites inspect both Sec-GPC and navigator.globalPrivacyControl.
• If your framework allows a browser launch flag that natively enables GPC, prefer that. For now, the init script property definition is widely used in automation.

5.2 Puppeteer

js
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.setExtraHTTPHeaders({ 'Sec-GPC': '1' });
  await page.evaluateOnNewDocument(() => {
    try {
      Object.defineProperty(Navigator.prototype, 'globalPrivacyControl', {
        get: () => true,
        configurable: true,
      });
    } catch {}
  });
  await page.goto('https://example.com');
})();

5.3 Selenium (CDP bridge)

Use Chrome DevTools Protocol to set extra headers and inject a script.

java
DevTools devTools = ((HasDevTools) driver).getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
Map<String, Object> headers = new HashMap<>();
headers.put("Sec-GPC", "1");
devTools.send(Network.setExtraHTTPHeaders(new Headers(headers)));

((JavascriptExecutor) driver).executeScript(
  "Object.defineProperty(Navigator.prototype, 'globalPrivacyControl', {get: () => true, configurable: true});"
);

6. Detecting CMPs reliably

Start with protocol discovery; enhance with vendor heuristics when you need to drive a UI.

6.1 TCF discovery

• Look for a stub: CMPs inject a hidden iframe named __tcfapiLocator for cross-frame messaging. If present, you can postMessage to call __tcfapi even when the function is not on the main window.
• Use ping: __tcfapi('ping', 2, cb) returns metadata such as gdprApplies, cmpLoaded, and cmpStatus.

Playwright helper:

ts
async function waitForTCF(page) {
  // First try direct global
  const hasTcf = await page.evaluate(() => typeof window.__tcfapi === 'function');
  if (hasTcf) return true;

  // Otherwise, poll for the locator iframe
  await page.waitForFunction(() => !!document.querySelector('iframe[name="__tcfapiLocator"]'), { timeout: 10000 }).catch(() => {});

  // Check again after potential iframe stub resolution
  return await page.evaluate(() => typeof window.__tcfapi === 'function');
}

Listening for readiness and state updates:

ts
async function getTCData(page) {
  return await page.evaluate(() => new Promise((resolve) => {
    const cb = (tcData, success) => {
      if (success && (tcData.eventStatus === 'tcloaded' || tcData.eventStatus === 'useractioncomplete')) {
        resolve(tcData);
      }
    };
    // Subscribe first
    window.__tcfapi('addEventListener', 2, cb);
    // Then actively request
    window.__tcfapi('getTCData', 2, (tcData, success) => success && resolve(tcData));
  }));
}

6.2 US Privacy and GPP discovery

ts
async function detectUSP(page) {
  return await page.evaluate(() => typeof window.__uspapi === 'function');
}

async function detectGPP(page) {
  return await page.evaluate(() => typeof window.__gpp === 'function');
}

Query values if present:

ts
// US Privacy
const usp = await page.evaluate(() => new Promise((res) => {
  if (typeof window.__uspapi !== 'function') return res(null);
  window.__uspapi('getUSPData', 1, (data, success) => res(success ? data : null));
}));

// GPP
const gpp = await page.evaluate(() => new Promise((res) => {
  if (typeof window.__gpp !== 'function') return res(null);
  window.__gpp('getGPPData', null, (data, success) => res(success ? data : null));
}));

6.3 Vendor heuristics for UI fallback

When you must drive the UI, detect known CMPs to load the correct playbook. Heuristics (non-exhaustive):

• OneTrust: window.Optanon, cookies starting with OptanonConsent, DOM elements with id/cls containing onetrust- or ot-.
• Quantcast Choice: window.__tcfapi with isQuantcastChoice flag in ping response; DOM with data-quantcast attributes.
• Didomi: window.didomiExists or window.Didomi; data attributes like data-didomi-.
• TrustArc: window.truste; elements with id truste-consent-track, or CSS classes trustarc-
• Sourcepoint: window.sp or sp.loaded; iframe names containing sp consent.

Store the vendor ID in logs and metrics to track coverage and breakages over time.

7. Choosing minimal consent deterministically

There are two reliable avenues:

1. Protocol-level: Use the CMP’s API or libraries to compute and write a consent string with all purposes denied and no vendor consent. This is most deterministic, but not all CMPs expose a setter in the public API.
2. UI-level: Drive the CMP’s documented controls to express Reject All or equivalent. Ensure your automation waits for tcloaded and confirms via getTCData that consents are actually off.

A best-effort approach is hybrid: if you control the test environment or have permission, set the consent string directly; else, run the vendor-specific UI routine and verify via __tcfapi.

7.1 Programmatic TCF v2.2 minimal string

Use a reference implementation to avoid bit-level errors. One popular package is @iabtcf/core.

ts
import { TCModel, GVL, TCString } from '@iabtcf/core';

async function minimalTCFString({
  cmpId = 0, // your CMP ID if applicable, else 0 for testing
  cmpVersion = 1,
  gvlUrl = 'https://vendor-list.consensu.org/v2/vendor-list.json',
}) {
  const gvl = await new GVL(gvlUrl);
  const tcModel = new TCModel(gvl);

  // Basic metadata
  tcModel.cmpId = cmpId;
  tcModel.cmpVersion = cmpVersion;
  tcModel.policyVersion = gvl.tcfPolicyVersion;
  tcModel.isServiceSpecific = false;
  tcModel.useNonStandardStacks = false;

  // GDPR applies, but no consent
  tcModel.gdprApplies = true;

  // Explicitly set no purpose consents or legitimate interests
  gvl.getPurposes().forEach((p) => {
    tcModel.purposeConsents.unset(p.id);
    tcModel.purposeLegitimateInterests.unset(p.id);
  });

  // No vendor consents or LI
  gvl.getVendors().forEach((v) => {
    tcModel.vendorConsents.unset(v.id);
    tcModel.vendorLegitimateInterests.unset(v.id);
  });

  // Publisher restrictions none

  const tcString = TCString.encode(tcModel);
  return tcString; // base64url string
}

Once generated, set it where the site expects it:

• Cookie: euconsent-v2 at eTLD+1 with appropriate path and max-age.
• localStorage mirrors: IABTCF_TCString and optionally additional IABTCF_* keys for faster CMP boot.

Example in Playwright:

ts
async function setTCFInPage(page, tcString, domain) {
  await page.addInitScript((tc) => {
    try {
      localStorage.setItem('IABTCF_TCString', tc);
    } catch {}
  }, tcString);
  const cookie = {
    name: 'euconsent-v2',
    value: tcString,
    domain, // e.g., .example.com
    path: '/',
    httpOnly: false,
    secure: true,
    sameSite: 'Lax',
  };
  await page.context().addCookies([cookie]);
}

Caveats:

• Only set consent strings this way where you have the right to do so (e.g., your own site or permitted testing). For third-party properties, rely on CMP UI or official APIs.
• Some CMPs verify signatures or pair TC strings with additional state. Always verify via __tcfapi('getTCData', 2, ...) after setting.

7.2 Programmatic US Privacy minimal string

A common minimal string under CCPA opt-out is 1YNN, but exact semantics depend on the framework and whether the user has opted-out of sale/sharing. If the site supports GPP, prefer setting a GPP string that encodes the relevant section with opt-out flags. Where __uspapi is still used, you can persist usprivacy in a cookie:

ts
await context.addCookies([{
  name: 'usprivacy',
  value: '1YNN',
  domain: '.example.com',
  path: '/',
  secure: true,
  sameSite: 'Lax',
}]);

As with TCF, confirm via __uspapi('getUSPData', 1, ...).

7.3 Vendor-specific UI fallbacks

If you cannot set strings directly, implement selectors per vendor with careful retries and locale-agnostic logic. Two examples:

• OneTrust: Click the Reject All button with [aria-label*='Reject'], or an element with id onetrust-reject-all-handler. Then confirm via __tcfapi that purpose consents are all zeros.

ts
// OneTrust example
await page.waitForSelector('#onetrust-banner-sdk', { timeout: 10000 });
const rejectSelector = '#onetrust-reject-all-handler, button[aria-label*="Reject" i]';
if (await page.$(rejectSelector)) {
  await page.click(rejectSelector);
}

• Didomi: wait for a container with id didomi-host and click a button that has data-attribute Didomi:reject-button. Fallback to text search for Reject All in the current locale.

Always follow the click with a protocol verification step. If the banner hides without effect, treat as failure.

8. Persisting consent strings and cookies across navigations

CMPs read both cookies and localStorage to determine prior choices. To make your agent idempotent:

• Store at eTLD+1: Set euconsent-v2 and usprivacy on the top-level domain to cover subdomains.
• Mirror to localStorage: IABTCF_TCString and IABUSPrivacy_String preempt re-prompts.
• Handle Additional Consent: Some ecosystems use addtl_consent to list OOB vendors (Google AC). For strict minimal consent, set addtl_consent to '1~' (empty vendor list) if your site expects it.

Playwright routine that normalizes all common storages:

ts
async function persistConsentAll(page, { domain, tcString, usString, gppString }) {
  await page.addInitScript(({ tc, us, gpp }) => {
    try { if (tc) localStorage.setItem('IABTCF_TCString', tc); } catch {}
    try { if (us) localStorage.setItem('IABUSPrivacy_String', us); } catch {}
    try { if (gpp) localStorage.setItem('IABGPP_HDR_GppString', gpp); } catch {}
  }, { tc: tcString, us: usString, gpp: gppString });

  const cookies = [];
  if (tcString) cookies.push({ name: 'euconsent-v2', value: tcString, domain, path: '/', secure: true, sameSite: 'Lax' });
  if (usString) cookies.push({ name: 'usprivacy', value: usString, domain, path: '/', secure: true, sameSite: 'Lax' });
  if (gppString) cookies.push({ name: 'gpp', value: gppString, domain, path: '/', secure: true, sameSite: 'Lax' });
  if (cookies.length) await page.context().addCookies(cookies);
}

Validation checklist on each navigation:

• __tcfapi('ping') returns cmpLoaded true and gdprApplies consistent with your test region.
• __tcfapi('getTCData') shows purpose and vendor consents unset.
• __uspapi or __gpp returns the expected string, and site behavior matches (e.g., ad calls suppressed).

9. Sandboxing trackers with Service Worker and CSP

Even with minimal consent, misconfigured tags may still attempt to fire. Defense-in-depth: intercept and neutralize unwanted calls.

9.1 Service Worker interception (for origins you control)

Register a Service Worker at the root of your site to enforce consent locally.

js
// sw.js
self.addEventListener('fetch', (event) => {
  const url = new URL(event.request.url);

  // Block known tracker endpoints when no consent flag is present in a cookie or header
  const blockedHosts = [
    'www.google-analytics.com',
    'connect.facebook.net',
    'static.ads-twitter.com',
    'snap.licdn.com',
  ];

  const hasConsent = event.clientId ? self.clients.get(event.clientId).then(() => false) : false; // replace with real check

  if (blockedHosts.includes(url.hostname) && !hasConsent) {
    event.respondWith(new Response('', { status: 204 }));
    return;
  }

  // Strip cookies and referrers by default for third-party requests
  if (url.origin !== self.location.origin) {
    const req = new Request(event.request, {
      credentials: 'omit',
      headers: new Headers({
        'Sec-Fetch-Site': 'none',
      }),
      referrer: '',
      referrerPolicy: 'no-referrer',
    });
    event.respondWith(fetch(req));
  }
});

Register from your app:

js
if ('serviceWorker' in navigator) {
  navigator.serviceWorker.register('/sw.js', { scope: '/' });
}

9.2 CSP and Permissions-Policy (proxy injection)

If you do not control the origin code, inject headers via a local proxy in test:

js
// Very simple header-rewriting proxy (Node.js)
const httpProxy = require('http-proxy');
const proxy = httpProxy.createProxyServer({});

require('http').createServer((req, res) => {
  proxy.once('proxyRes', (proxyRes) => {
    const setHeader = (k, v) => proxyRes.headers[k.toLowerCase()] = v;
    const csp = "default-src 'self' 'unsafe-inline' 'unsafe-eval' https: data:; connect-src 'self' https:; img-src 'self' https: data:; script-src 'self' https:; frame-src 'self';";
    setHeader('content-security-policy', csp);
    setHeader('permissions-policy', 'interest-cohort=(), browsing-topics=(), geolocation=(), microphone=(), camera=()');
    setHeader('report-to', '{"group":"csp","max_age":10886400,"endpoints":[{"url":"https://example.test/csp-report"}]}');
    setHeader('reporting-endpoints', 'csp="https://example.test/csp-report"');
  });
  proxy.web(req, res, { target: req.url, changeOrigin: true });
}).listen(8080);

This is powerful for tests, but do not use it to circumvent site policies. Apply only in controlled environments you own.

9.3 Routing at the automation layer

Playwright can block or rewrite requests without a Service Worker:

ts
await context.route('**/*', (route) => {
  const url = route.request().url();
  const blocked = /google-analytics|facebook|doubleclick|hotjar|segment/.test(url);
  if (blocked) return route.fulfill({ status: 204, body: '' });
  route.continue();
});

Toggle the blocklist based on your detected consent state to emulate production behavior under different choices.

10. Deterministic replay in CI/CD

Replaying consent reliably is essential for stable integration tests.

10.1 Capture a consent fixture

• Snapshot cookies and localStorage keys relevant to consent after a successful run. In Playwright, use storageState to export cookies and localStorage for a domain.

ts
await context.storageState({ path: 'state-consent-minimal.json' });

• Also persist an explicit consent record for auditability:

json
{
  "gpc": true,
  "tcf": {
    "string": "<TCF_STRING>",
    "gdprApplies": true,
    "purposes": {}
  },
  "usPrivacy": "1YNN",
  "timestamp": "2026-05-17T10:00:00Z",
  "jurisdiction": "EU"
}

10.2 Normalize the environment

• IP geolocation: Route EU tests through an EU exit node; US tests through a US node. Many CMPs vary UI and defaults by region.
• Time and locale: Fix timezoneId and locale in your browser context.
• User agent: Pin a UA to avoid variant CMP code paths for mobile vs desktop.
• Clock: Freeze time if your site schedules consent expiry prompts.

10.3 Replay and verify

• Load the storageState at context creation. Verify via __tcfapi and __uspapi or __gpp that the state matches before proceeding to functional assertions.
• Fail fast if the CMP changes schema or breaks your detection (e.g., ping returns cmpStatus stub).

10.4 Parallelism and isolation

• Run per-worker storage directories and cookies to avoid test crosstalk.
• Use context-per-test isolation with a fresh storageState load for each.

11. Observability and compliance guardrails

• Structured logs: Emit consent events: when GPC set, when CMP detected, which vendor, the final TC string hash, and whether minimal consent was achieved.
• Screenshots when UI fallback is executed, plus DOM snapshots of the CMP container for auditability.
• Metrics: Ratio of protocol vs UI handling; time-to-consent; vendor distribution; failure modes.
• Legal boundaries: Do not forge consent or bypass CMPs to continue tracking. Your pipeline’s job is to minimize and enforce, not to trick the site. Use direct string setting only in environments where it is appropriate and lawful.

12. Edge cases and practical tips

• Cross-domain iframes: A CMP running in a third-party iframe may require postMessage invocation. The __tcfapi stub and __tcfapiLocator exist for this reason. If you cannot access the function, inject a frame-agnostic call via window.postMessage per the TCF spec.
• Safari ITP: Cookie lifetimes can be short and partitioned. Prefer localStorage mirrors and consider using Storage Access API prompts in first-party contexts if your test model requires it.
• Incognito and ephemeral sessions: Some browsers disable persistent storage. Ensure your automation framework still sets cookies before navigation; else, you will see re-prompts.
• SPAs: CMP state may be read at app boot; race conditions cause trackers to fire before consent is applied. Use addInitScript to set GPC and consent storages before any app JS runs.
• AMP or server-rendered consent: AMP has its own <amp-consent> component with different storage. Tailor your pipeline if you test AMP pages.
• Publisher restrictions: Even with minimal consent, publisher restrictions in the TC string may affect how vendors interpret purposes. Validate site behavior, not only the bitstring.
• Google Additional Consent (AC): If your stack integrates with Google ad tech, set addtl_consent appropriately (often '1~' for empty) to avoid out-of-band vendor assumptions.
• Re-prompts: Some CMPs re-prompt after n days or on new purposes. In tests, fix the clock or refresh the prior string’s lastUpdated field if the library supports it.

13. End-to-end example: a consent-aware Playwright helper

Below is a consolidated helper that sets GPC, detects a CMP, attempts a minimal programmatic TCF string for testing domains, falls back to vendor UIs, and validates the outcome.

ts
import { chromium, BrowserContext, Page } from 'playwright';
import { TCModel, GVL, TCString } from '@iabtcf/core';

async function withConsent(opts) {
  const {
    baseUrl,
    domain, // e.g., .example.com
    region = 'EU', // 'EU' or 'US'
    useProgrammaticTC = true,
  } = opts;

  const browser = await chromium.launch();
  const context = await browser.newContext({
    extraHTTPHeaders: { 'Sec-GPC': '1' },
    locale: region === 'EU' ? 'en-GB' : 'en-US',
    timezoneId: region === 'EU' ? 'Europe/Paris' : 'America/Los_Angeles',
  });
  await context.addInitScript(() => {
    try {
      Object.defineProperty(Navigator.prototype, 'globalPrivacyControl', {
        get: () => true,
        configurable: true,
      });
    } catch {}
  });

  const page = await context.newPage();

  // Optionally pre-seed a minimal TCF string
  if (region === 'EU' && useProgrammaticTC) {
    const gvl = await new GVL('https://vendor-list.consensu.org/v2/vendor-list.json');
    const tcModel = new TCModel(gvl);
    tcModel.gdprApplies = true;
    gvl.getPurposes().forEach((p) => {
      tcModel.purposeConsents.unset(p.id);
      tcModel.purposeLegitimateInterests.unset(p.id);
    });
    gvl.getVendors().forEach((v) => {
      tcModel.vendorConsents.unset(v.id);
      tcModel.vendorLegitimateInterests.unset(v.id);
    });
    const tc = TCString.encode(tcModel);
    await persistConsentAll(page, { domain, tcString: tc });
  }

  await page.goto(baseUrl, { waitUntil: 'domcontentloaded' });

  // Detect CMP and verify minimal consent
  const hasTCF = await page.evaluate(() => typeof window.__tcfapi === 'function');
  if (hasTCF) {
    const tcData = await getTCData(page);
    const allZeroPurposes = Object.values(tcData.purpose.consents || {}).every((v) => v === false);
    if (!allZeroPurposes) {
      // Fallback: try vendor UI rejects for common CMPs
      await tryRejectAllVendorUI(page);
      const check = await getTCData(page);
      const ok = Object.values(check.purpose.consents || {}).every((v) => v === false);
      if (!ok) throw new Error('Failed to achieve minimal consent via CMP');
    }
  }

  return { browser, context, page };
}

async function getTCData(page) {
  return await page.evaluate(() => new Promise((resolve) => {
    const onEvent = (tcData, success) => {
      if (success && (tcData.eventStatus === 'tcloaded' || tcData.eventStatus === 'useractioncomplete')) {
        resolve(tcData);
      }
    };
    try { window.__tcfapi('addEventListener', 2, onEvent); } catch {}
    window.__tcfapi('getTCData', 2, (tcData, success) => success && resolve(tcData));
  }));
}

async function persistConsentAll(page, { domain, tcString }) {
  await page.addInitScript((tc) => { try { localStorage.setItem('IABTCF_TCString', tc); } catch {} }, tcString);
  if (tcString) await page.context().addCookies([{ name: 'euconsent-v2', value: tcString, domain, path: '/', secure: true, sameSite: 'Lax' }]);
}

async function tryRejectAllVendorUI(page) {
  // OneTrust
  const oneTrustBanner = await page.$('#onetrust-banner-sdk');
  if (oneTrustBanner) {
    const reject = await page.$('#onetrust-reject-all-handler, button[aria-label*="Reject" i]');
    if (reject) await reject.click();
  }
  // Didomi
  const didomiHost = await page.$('#didomi-host');
  if (didomiHost) {
    const sel = 'button[data-action*="reject" i], button[id*="reject" i]';
    if (await page.$(sel)) await page.click(sel);
  }
  // Add more vendor playbooks as needed
}

This helper is a starting point: extend vendor playbooks, add US Privacy GPP handling, and wire in network-level sandboxing for strong enforcement.

14. Validating effectiveness

• Network asserts: Ensure Sec-GPC: 1 is present on all requests. Confirm that analytics and ad endpoints are absent or receive 204s when consent is off.
• DOM asserts: No trackers injected in the DOM (e.g., ga.js, gtag, fbq) when consent is off.
• CMP asserts: The TC string parses to zeros; the usprivacy or GPP strings match expected opt-out.
• Server-side: If you control the backend, log and flag any request that contains marketing cookies when consent is off.

15. References and further reading

• IAB Europe Transparency and Consent Framework: developer portal and policy docs.
• Global Privacy Control specification: secgpc.org and associated W3C Community resources.
• IAB US Privacy and GPP specifications for multi-framework consent in the US.
• Major CMP vendor docs: OneTrust, Quantcast Choice, Didomi, TrustArc, Sourcepoint.

16. Opinionated conclusions

• Prefer protocol over pixels. Your agents should speak __tcfapi, __uspapi, and __gpp before they learn any CSS selector.
• Enforce with defense-in-depth. Even perfect consent strings do not guarantee perfect tag behavior; sandbox with SW/CSP and blocklists in test.
• Treat consent as a first-class test fixture. Snapshot, version, and replay it deterministically like any other part of your test data.
• Be conservative and transparent. Minimal consent means deny by default unless a test explicitly needs an allow scenario. Log every step so auditors and teammates can reconstruct decisions.

By following these practices, you can ship browser agents that measurably reduce privacy risk, stabilize tests across jurisdictions, and stay ahead of a fast-moving compliance landscape while keeping your automation stack maintainable.