From 20109e75118dda956192242b28faff270a72a112 Mon Sep 17 00:00:00 2001 From: reacher-z Date: Mon, 27 Jul 2026 17:31:45 +0800 Subject: [PATCH 1/2] docs: reference ClawBench as an external benchmark --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index fc5d188d..c3c3d115 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,12 @@ BrowserGym includes the following benchmarks by default: - [OpenApps](https://facebookresearch.github.io/OpenApps/) - [TimeWarp](https://timewarp-web.github.io) +### External related benchmarks + +The following independent benchmark is related to BrowserGym's web-agent evaluation scope but is not integrated with or supported by BrowserGym: + +* **[ClawBench](https://github.com/reacher-z/ClawBench)** — [paper](https://arxiv.org/abs/2604.08523) and [project page](https://claw-bench.com/). It evaluates web and computer-use agents on 283 tasks across 163 live platforms with isolated browser sessions, request-level outcome checks, and recorded action, screenshot, network, and message traces. See the project's own [runner and setup instructions](https://github.com/reacher-z/ClawBench#readme); task definitions are available on [Hugging Face](https://huggingface.co/datasets/NAIL-Group/ClawBench) and mirrored at [TIGER-Lab/ClawBench](https://huggingface.co/datasets/TIGER-Lab/ClawBench). + Designing new web benchmarks with BrowserGym is easy, and simply requires to inherit the [`AbstractBrowserTask`](https://github.com/ServiceNow/BrowserGym/blob/main/browsergym/core/src/browsergym/core/task.py#L7C7-L7C26) class. ## 🛠️ Setup From f6805243a01ff6dfdb2c74f5bbc66ea12ff3a545 Mon Sep 17 00:00:00 2001 From: Yuxuan Zhang <68239571+reacher-z@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:48:15 +0800 Subject: [PATCH 2/2] docs: use canonical ClawBench repository link --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c3c3d115..0d954a33 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ BrowserGym includes the following benchmarks by default: The following independent benchmark is related to BrowserGym's web-agent evaluation scope but is not integrated with or supported by BrowserGym: -* **[ClawBench](https://github.com/reacher-z/ClawBench)** — [paper](https://arxiv.org/abs/2604.08523) and [project page](https://claw-bench.com/). It evaluates web and computer-use agents on 283 tasks across 163 live platforms with isolated browser sessions, request-level outcome checks, and recorded action, screenshot, network, and message traces. See the project's own [runner and setup instructions](https://github.com/reacher-z/ClawBench#readme); task definitions are available on [Hugging Face](https://huggingface.co/datasets/NAIL-Group/ClawBench) and mirrored at [TIGER-Lab/ClawBench](https://huggingface.co/datasets/TIGER-Lab/ClawBench). +* **[ClawBench](https://github.com/TIGER-AI-Lab/ClawBench)** — [paper](https://arxiv.org/abs/2604.08523) and [project page](https://claw-bench.com/). It evaluates web and computer-use agents on 283 tasks across 163 live platforms with isolated browser sessions, request-level outcome checks, and recorded action, screenshot, network, and message traces. See the project's own [runner and setup instructions](https://github.com/TIGER-AI-Lab/ClawBench#readme); task definitions are available on [Hugging Face](https://huggingface.co/datasets/NAIL-Group/ClawBench) and mirrored at [TIGER-Lab/ClawBench](https://huggingface.co/datasets/TIGER-Lab/ClawBench). Designing new web benchmarks with BrowserGym is easy, and simply requires to inherit the [`AbstractBrowserTask`](https://github.com/ServiceNow/BrowserGym/blob/main/browsergym/core/src/browsergym/core/task.py#L7C7-L7C26) class. @@ -310,4 +310,4 @@ Here is an example of how they can be used: We use the BrowserGym framework for our experiments \cite{workarena2024,chezelles2025browsergym}. ``` ## Traces -Traces from “The BrowserGym Ecosystem for Web Agent Research” paper are available in [Huggingface](https://huggingface.co/datasets/agentlabtraces/agentlabtraces/tree/main). +Traces from “The BrowserGym Ecosystem for Web Agent Research” paper are available in [Huggingface](https://huggingface.co/datasets/agentlabtraces/agentlabtraces/tree/main). \ No newline at end of file