隐身浏览器自动化

结合多重反检测层的静默、不可察觉的网络自动化。

快速登录流程 (重要)

当用户要求登录任何网站时：

1. 以有界面模式打开(可见浏览器，用于手动登录)：

python scripts/stealth_session.py -u "https://target.com/login" -s sitename --headed

1. 用户在可见浏览器中手动登录

2. 登录确认后保存会话：

python scripts/stealth_session.py -u "https://target.com" -s sitename --headed --save

1. 后续使用- 加载已保存的会话 (无界面模式)：

python scripts/stealth_session.py -u "https://target.com" -s sitename --load

会话存储于：~/.clawdbot/browser-sessions/<sitename>.json

执行策略 (重要)

1. 先静默后显示

• 优先使用无界面模式静默尝试
• 如果失败或需要验证码，再切换到有界面显示模式
• 避免打扰用户操作

2. 断点续传

长任务使用task_runner.py管理状态：

from task_runner import TaskRunner
task = TaskRunner('my_task')
task.set_total(100)
for i in items:
    if task.is_completed(i):
        continue  # 跳过已完成
    # 处理...
    task.mark_completed(i)
task.finish()

3. 超时处理

• 默认单页超时: 30秒
• 长任务每50项保存一次进度
• 失败自动重试3次

4. 记录尝试

所有登录尝试记录在:~/.clawdbot/browser-sessions/attempts.json

架构

┌─────────────────────────────────────────────────────┐
│                  隐身浏览器                         │
├─────────────────────────────────────────────────────┤
│  第一层: 反检测引擎                                 │
│  - puppeteer-extra-plugin-stealth                   │
│  - 浏览器指纹伪造                                  │
│  - WebGL/Canvas/Audio 指纹掩码                     │
├─────────────────────────────────────────────────────┤
│  第二层: 验证绕过                                  │
│  - Cloudflare Turnstile/JS 验证                    │
│  - hCaptcha / reCAPTCHA 集成                       │
│  - 2Captcha / Anti-Captcha API                     │
├─────────────────────────────────────────────────────┤
│  第三层: 会话持久化                                │
│  - Cookie 存储 (JSON/SQLite)                       │
│  - localStorage 同步                               │
│  - 多配置文件管理                                  │
├─────────────────────────────────────────────────────┤
│  第四层: 代理与身份                                │
│  - 轮换住宅代理                                    │
│  - User-Agent 轮换                                 │
│  - 时区/区域设置伪造                               │
└─────────────────────────────────────────────────────┘

设置

安装核心依赖

npm install -g puppeteer-extra puppeteer-extra-plugin-stealth
npm install -g playwright
pip install undetected-chromedriver DrissionPage

可选: 验证码解决器

将API密钥存储在~/.clawdbot/secrets/captcha.json:

{
  "2captcha": "你的_2CAPTCHA_密钥",
  "anticaptcha": "你的_ANTICAPTCHA_密钥",
  "capsolver": "你的_CAPSOLVER_密钥"
}

可选: 代理配置

存储在~/.clawdbot/secrets/proxies.json:

{
  "rotating": "http://user:pass@proxy.provider.com:port",
  "residential": ["socks5://ip1:port", "socks5://ip2:port"],
  "datacenter": "http://dc-proxy:port"
}

快速开始

1. 隐身会话 (Python - 推荐)

# scripts/stealth_session.py - 用于最大兼容性
import undetected_chromedriver as uc
from DrissionPage import ChromiumPage

# 选项 A: undetected-chromedriver (基于Selenium)
driver = uc.Chrome(headless=True, use_subprocess=True)
driver.get("https://nowsecure.nl")  # 测试反检测

# 选项 B: DrissionPage (更快，原生Python)
page = ChromiumPage()
page.get("https://cloudflare-protected-site.com")

2. 隐身会话 (Node.js)

// scripts/stealth.mjs
import puppeteer from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';

puppeteer.use(StealthPlugin());

const browser = await puppeteer.launch({
  headless: 'new',
  args: [
    '--disable-blink-features=AutomationControlled',
    '--disable-dev-shm-usage',
    '--no-sandbox'
  ]
});

const page = await browser.newPage();
await page.goto('https://bot.sannysoft.com'); // 验证隐身效果

核心操作

打开隐身页面 --- # 使用带有隐身配置文件的agent-browser agent-browser --profile ~/.stealth-profile open https://target.com # 或通过脚本 python scripts/stealth_open.py --url "https://target.com" --headless --- 绕过Cloudflare --- # 使用 DrissionPage 自动绕过 Cloudflare from DrissionPage import ChromiumPage page = ChromiumPage() page.get("https://cloudflare-site.com") # DrissionPage 会自动等待 Cloudflare 挑战 # 如果需要，手动等待 page.wait.ele_displayed("main-content", timeout=30) --- 对于顽固的 Cloudflare 站点，使用 FlareSolverr： --- # 启动 FlareSolverr 容器 docker run -d --name flaresolverr -p 8191:8191 ghcr.io/flaresolverr/flaresolverr # 请求清除 curl -X POST http://localhost:8191/v1 \ -H "Content-Type: application/json" \ -d '{"cmd":"request.get","url":"https://cf-protected.com","maxTimeout":60000}' --- 解决验证码 --- # scripts/solve_captcha.py import requests import json import time def solve_recaptcha(site_key, page_url, api_key): """通过 2Captcha 解决 reCAPTCHA v2/v3""" # 提交任务 resp = requests.post("http://2captcha.com/in.php", data={ "key": api_key, "method": "userrecaptcha", "googlekey": site_key, "pageurl": page_url, "json": 1 }).json() task_id = resp["request"] # 轮询结果 for _ in range(60): time.sleep(3) result = requests.get(f"http://2captcha.com/res.php?key={api_key}&action=get&id={task_id}&json=1").json() if result["status"] == 1: return result["request"] # Token return None def solve_hcaptcha(site_key, page_url, api_key): """通过 Anti-Captcha 解决 hCaptcha""" resp = requests.post("https://api.anti-captcha.com/createTask", json={ "clientKey": api_key, "task": { "type": "HCaptchaTaskProxyless", "websiteURL": page_url, "websiteKey": site_key } }).json() task_id = resp["taskId"] for _ in range(60): time.sleep(3) result = requests.post("https://api.anti-captcha.com/getTaskResult", json={ "clientKey": api_key, "taskId": task_id }).json() if result["status"] == "ready": return result["solution"]["gRecaptchaResponse"] return None --- 持久会话 --- # scripts/session_manager.py import json import os from pathlib import Path SESSIONS_DIR = Path.home() / ".clawdbot" / "browser-sessions" SESSIONS_DIR.mkdir(parents=True, exist_ok=True) def save_cookies(driver, session_name): """将 Cookies 保存到 JSON""" cookies = driver.get_cookies() path = SESSIONS_DIR / f"{session_name}_cookies.json" path.write_text(json.dumps(cookies, indent=2)) return path def load_cookies(driver, session_name): """从保存的会话中加载 Cookies""" path = SESSIONS_DIR / f"{session_name}_cookies.json" if path.exists(): cookies = json.loads(path.read_text()) for cookie in cookies: driver.add_cookie(cookie) return True return False def save_local_storage(page, session_name): """保存 localStorage""" ls = page.evaluate("() => JSON.stringify(localStorage)") path = SESSIONS_DIR / f"{session_name}_localStorage.json" path.write_text(ls) return path def load_local_storage(page, session_name): """恢复 localStorage""" path = SESSIONS_DIR / f"{session_name}_localStorage.json" if path.exists(): data = path.read_text() page.evaluate(f"(data) => {{ Object.entries(JSON.parse(data)).forEach(([k,v]) => localStorage.setItem(k,v)) }}", data) return True return False --- 静默自动化工作流 --- # 完整的静默自动化示例 from DrissionPage import ChromiumPage, ChromiumOptions # 为隐身配置 options = ChromiumOptions() options.headless() options.set_argument('--disable-blink-features=AutomationControlled') options.set_argument('--disable-dev-shm-usage') options.set_user_agent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36') page = ChromiumPage(options) # 通过 Cloudflare 绕过进行导航 page.get("https://target-site.com") # 等待任何挑战 page.wait.doc_loaded() # 静默交互 page.ele("@id=username").input("user@email.com") page.ele("@id=password").input("password123") page.ele("@type=submit").click() # 保存会话以便重用 page.cookies.save("~/.clawdbot/browser-sessions/target-site.json") --- 代理轮换 --- # scripts/proxy_rotate.py import random import json from pathlib import Path def get_proxy(): """从代理池中获取随机代理""" config = json.loads((Path.home() / ".clawdbot/secrets/proxies.json").read_text()) proxies = config.get("residential", []) return random.choice(proxies) if proxies else config.get("rotating") # 与 DrissionPage 一起使用 options = ChromiumOptions() options.set_proxy(get_proxy()) page = ChromiumPage(options) --- 需要用户输入 --- 要完成此技能，请提供： --- 验证码 API 密钥 --- （可选但推荐）： --- 2Captcha 密钥： --- https://2captcha.com --- Anti-Captcha 密钥： --- https://anti-captcha.com --- CapSolver 密钥： --- https://capsolver.com --- 代理配置 --- （可选）： --- 住宅代理提供商凭据

# Using agent-browser with stealth profile
agent-browser --profile ~/.stealth-profile open https://target.com

# Or via script
python scripts/stealth_open.py --url "https://target.com" --headless

Bypass Cloudflare

# Automatic CF bypass with DrissionPage
from DrissionPage import ChromiumPage

page = ChromiumPage()
page.get("https://cloudflare-site.com")
# DrissionPage waits for CF challenge automatically

# Manual wait if needed
page.wait.ele_displayed("main-content", timeout=30)

For stubborn Cloudflare sites, use FlareSolverr:

# Start FlareSolverr container
docker run -d --name flaresolverr -p 8191:8191 ghcr.io/flaresolverr/flaresolverr

# Request clearance
curl -X POST http://localhost:8191/v1 \
  -H "Content-Type: application/json" \
  -d '{"cmd":"request.get","url":"https://cf-protected.com","maxTimeout":60000}'

Solve CAPTCHAs

# scripts/solve_captcha.py
import requests
import json
import time

def solve_recaptcha(site_key, page_url, api_key):
    """Solve reCAPTCHA v2/v3 via 2Captcha"""
    # Submit task
    resp = requests.post("http://2captcha.com/in.php", data={
        "key": api_key,
        "method": "userrecaptcha",
        "googlekey": site_key,
        "pageurl": page_url,
        "json": 1
    }).json()
    
    task_id = resp["request"]
    
    # Poll for result
    for _ in range(60):
        time.sleep(3)
        result = requests.get(f"http://2captcha.com/res.php?key={api_key}&action=get&id={task_id}&json=1").json()
        if result["status"] == 1:
            return result["request"]  # Token
    return None

def solve_hcaptcha(site_key, page_url, api_key):
    """Solve hCaptcha via Anti-Captcha"""
    resp = requests.post("https://api.anti-captcha.com/createTask", json={
        "clientKey": api_key,
        "task": {
            "type": "HCaptchaTaskProxyless",
            "websiteURL": page_url,
            "websiteKey": site_key
        }
    }).json()
    
    task_id = resp["taskId"]
    
    for _ in range(60):
        time.sleep(3)
        result = requests.post("https://api.anti-captcha.com/getTaskResult", json={
            "clientKey": api_key,
            "taskId": task_id
        }).json()
        if result["status"] == "ready":
            return result["solution"]["gRecaptchaResponse"]
    return None

Persistent Sessions

# scripts/session_manager.py
import json
import os
from pathlib import Path

SESSIONS_DIR = Path.home() / ".clawdbot" / "browser-sessions"
SESSIONS_DIR.mkdir(parents=True, exist_ok=True)

def save_cookies(driver, session_name):
    """Save cookies to JSON"""
    cookies = driver.get_cookies()
    path = SESSIONS_DIR / f"{session_name}_cookies.json"
    path.write_text(json.dumps(cookies, indent=2))
    return path

def load_cookies(driver, session_name):
    """Load cookies from saved session"""
    path = SESSIONS_DIR / f"{session_name}_cookies.json"
    if path.exists():
        cookies = json.loads(path.read_text())
        for cookie in cookies:
            driver.add_cookie(cookie)
        return True
    return False

def save_local_storage(page, session_name):
    """Save localStorage"""
    ls = page.evaluate("() => JSON.stringify(localStorage)")
    path = SESSIONS_DIR / f"{session_name}_localStorage.json"
    path.write_text(ls)
    return path

def load_local_storage(page, session_name):
    """Restore localStorage"""
    path = SESSIONS_DIR / f"{session_name}_localStorage.json"
    if path.exists():
        data = path.read_text()
        page.evaluate(f"(data) => {{ Object.entries(JSON.parse(data)).forEach(([k,v]) => localStorage.setItem(k,v)) }}", data)
        return True
    return False

Silent Automation Workflow

# Complete silent automation example
from DrissionPage import ChromiumPage, ChromiumOptions

# Configure for stealth
options = ChromiumOptions()
options.headless()
options.set_argument('--disable-blink-features=AutomationControlled')
options.set_argument('--disable-dev-shm-usage')
options.set_user_agent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36')

page = ChromiumPage(options)

# Navigate with CF bypass
page.get("https://target-site.com")

# Wait for any challenges
page.wait.doc_loaded()

# Interact silently
page.ele("@id=username").input("user@email.com")
page.ele("@id=password").input("password123")
page.ele("@type=submit").click()

# Save session for reuse
page.cookies.save("~/.clawdbot/browser-sessions/target-site.json")

Proxy Rotation

# scripts/proxy_rotate.py
import random
import json
from pathlib import Path

def get_proxy():
    """Get random proxy from pool"""
    config = json.loads((Path.home() / ".clawdbot/secrets/proxies.json").read_text())
    proxies = config.get("residential", [])
    return random.choice(proxies) if proxies else config.get("rotating")

# Use with DrissionPage
options = ChromiumOptions()
options.set_proxy(get_proxy())
page = ChromiumPage(options)

User Input Required

To complete this skill, provide:

1. CAPTCHA API Keys(optional but recommended):

   • 2Captcha key:https://2captcha.com
   • Anti-Captcha key:https://anti-captcha.com
   • CapSolver key:https://capsolver.com

2. Proxy Configuration(optional):

   • Residential proxy provider credentials
   • 或列出SOCKS5/HTTP代理

3. 目标网站（针对预配置会话）：

   • 哪些网站需要登录状态保持？
   • 应存储哪些凭据？

文件结构

stealth-browser/
├── SKILL.md
├── scripts/
│   ├── stealth_session.py      # 主隐身浏览器封装器
│   ├── solve_captcha.py        # 验证码破解工具
│   ├── session_manager.py      # Cookie/localStorage持久化
│   ├── proxy_rotate.py         # 代理轮换
│   └── cf_bypass.py            # Cloudflare专用绕过工具
└── references/
    ├── fingerprints.md         # 浏览器指纹详情
    └── detection-tests.md      # 反检测测试网站

测试反检测能力

# 运行以下命令验证隐身功能是否生效：
python scripts/stealth_open.py --url "https://bot.sannysoft.com"
python scripts/stealth_open.py --url "https://nowsecure.nl"
python scripts/stealth_open.py --url "https://arh.antoinevastel.com/bots/areyouheadless"
python scripts/stealth_open.py --url "https://pixelscan.net"

与agent-browser集成

对于简单任务，使用带持久化配置文件的agent-browser：

# 创建隐身配置文件（仅需一次）
agent-browser --profile ~/.stealth-profile --headed open https://login-site.com
# 手动登录后关闭

# 复用已认证会话（无头模式）
agent-browser --profile ~/.stealth-profile snapshot
agent-browser --profile ~/.stealth-profile click @e5

对于Cloudflare或验证码密集型网站，请改用Python脚本。

最佳实践

1. 始终使用无头模式：'new'而非headless: true（更不易被检测）
2. 轮换用户代理需匹配浏览器版本
3. 添加随机延迟操作间隔（100-500毫秒）
4. 使用住宅代理针对敏感目标
5. 保存会话登录成功后
6. 使用前在bot.sannysoft.com测试生产环境前