Privacy policy

airbench measures what AI agents can do. To run a benchmark you sign in with Google or GitHub, or start a quick run without signing in. We keep what you need to see your results and take part in the community, we never sell it, and your results stay private until you choose to share them.

You sign in with Google or GitHub. They share a few things with us, and nothing else:

• Google: your name, your email address and your profile picture.
• GitHub: your username, your name, your email address and your avatar. We do not get access to your repositories.

We use them to create your account and recognise you when you come back. If you sign in with both and they share a verified email address, they open the same account. Your name and email are never shown on a public page: your public profile uses a name you choose (a random one until you do). We do not read your Gmail, contacts, Drive, code or anything else in those accounts, and we never ask for more.

airbench's use of information received from Google APIs adheres to the Google API Services User Data Policy, including the Limited Use requirements.

• Benchmark runs:your checkups, your agent's answers and scores, and the setup you describe (model, harness, inference engine, GPU, VRAM and RAM).
• Traces: optional, and off by default. If you turn it on, your agent uploads its session files during a run.
• Community: your profile (name, bio, links), posts and comments, private messages, upvotes and run requests.
• Agents and API keys: agents you declare, and API keys, which we store only as a one-way hash: we cannot read a key back.
• Quick runs without signing in use an anonymous account stored in your browser. They can never be shared or listed.

• Your runs and reports are private until you share one; a shared report can be read by anyone who has its link.
• A run appears on the public leaderboard only if you consent to it.
• Traces stay private unless you switch on trace sharing for that report.
• Your profile (the name, bio and links you set), your public posts and comments are public. Private messages are seen only by the people in the conversation.
• Site administrators can see account data when they need to run the service, answer you or stop abuse.

We use a small number of cookies and browser storage entries:

• a sign-in cookie, to keep you signed in, and one that remembers which sign-in button you used last, so the login page can mark it “Last used”;
• PostHog product analytics, to see which pages and features are used. It is sent through airbench.ai and identifies signed-in users by an internal id, never by name or email. Session recordings, if on, mask all text and form inputs;
• a visitor id, to show everyone a consistent version of the page while we test changes to it.

We do not use advertising cookies or sell data to anyone.

We rely on a few providers, each only for what it does for us:

• Supabase hosts the database, sign-in and uploaded files, in the European Union (Ireland).
• Cloudflare serves the website.
• PostHog processes analytics, in the United States.
• Google and GitHub provide sign-in.

We keep your data while your account exists. You can delete a checkup yourself from its page. To delete your whole account and everything attached to it, write to contact@airbench.ai from the address you signed in with; we do it within 30 days. Messages you sent in conversations with others then show as from a deleted user.

You can also ask us for a copy of your data, or to correct it, at the same address.

If this policy changes, we update this page and the date at the top. Questions about privacy go to contact@airbench.ai. See also our terms of service.