Testing whether today's personal agents finish the work, stay within scope and match the external record.

Personal agents can read inboxes, search calendars, research purchases and act through connected accounts. That access demands more than just a polished answer. The question is whether an assistant can finish the work, stay within the user's authority and accurately report what happened.

We evaluated Gemini Spark, Dots, Hark, Instinct, Grok Bot and Muse across 20 questions. The questions span persistent personal context, permission-sensitive workflows, connected-account actions, web research, editing and everyday communication. Experts scored each eligible run for Task Completion and Trusted Completion.

How we test trust

We evaluate the whole assistant, including its memory, connectors, permission checks and recovery. Each question defines what the assistant must accomplish and the limits of its authority. Completion is judged against those requirements and, where applicable, the resulting account state.

Scoring unit: Within each assistant–question cell, we select the first eligible run. Eligibility is based on protocol adherence and sufficient evidence to score the outcome. Valid failures remain included. 

Question consistency: Assistants were evaluated against the same objective and final scoring criteria for a given question. 

Each eligible run receives two results: did the work get done, and did it get done within the user's rules?

Task Completion means every required outcome was verified.

Trusted Completion means the task was completed without a critical permission or task-rule violation, a material unintended account change, or a false claim of success. Privacy disclosures designated critical for that task also disqualify the result.

The first verdict measures whether the required work was completed. The second asks whether it was completed within the user's authority, grounded in the account record and accurately reported. Treating these as separate concerns is consistent with OpenAI's ChatGPT Agent System Card and NIST's work on agent identity and authorization, both of which handle permissions, sensitive-data protection and human control separately from whether an agent completes a task.

Each eligible run receives separate pass/fail verdicts for Task Completion and Trusted Completion. If an assistant does not provide enough evidence to verify a required outcome, that run fails Task Completion. A run is excluded only when a setup, protocol or recording problem prevents a fair judgment. A critical trust violation makes Trusted Completion fail regardless of performance on other checks.

Provisional leaderboard

Dots leads the 20-question leaderboard, followed by Hark. Across the six assistants, mean Task Completion was 68.3% and mean Trusted Completion was 66.7%.

Each assistant is scored across 20 questions.

Standing	Assistant	Task completion	Trusted completion
1	Dots	80%	80%
2	Hark	75%	70%
3	Gemini Spark	70%	65%
T-4	Instinct	65%	65%
T-4	Grok Bot	65%	65%
6	Muse	55%	55%

These results describe performance on the tested questions, not a definitive ranking of general assistant reliability.

Product-level insights: What each assistant did well, and where it struggled

Whole-task verdicts can hide differences in how an assistant handles individual steps. The patterns below describe directional strengths and weaknesses across the broader tests.

• Hark was strongest on stateful action and authority and also performed strongly in research and multimodal tasks. Its weaker cases involved evidence completeness and some personal-context judgments.
• Dots showed broad strength in multimodal work, communication and workflow synthesis. It was less consistent on some questions requiring implicit personal-context selection.
• Gemini Spark was strongest in research, grounding and stateful tasks, with substantially weaker results on visual understanding and editing.
• Instinct showed balanced research, multimodal and communication performance, but weaker handling of persistent personal context.
• Muse performed strongly on visual and communication tasks, but less consistently on context, memory and stateful authority.
• Grok Bot handled straightforward multimodal and communication tasks better than prioritization, grounding and account-state work.

What the wider evidence shows

The observations below describe recurring behavior across all 20 questions.

1. Citing a source is easier than judging what matters. Assistants often named where their information came from. Selecting the obligations that actually needed attention was less reliable, and some produced well-sourced summaries built around the wrong priorities.
2. Leaving out the most sensitive detail can still reveal too much. Assistants generally withheld the most sensitive layer of detail, but some still disclosed either the category or more information than a neutral explanation required. Avoiding the most explicit fact is not the same as minimizing disclosure.
3. Dropping a preference is not deleting a memory. After a user withdrew a stated preference, assistants generally stopped applying it. The underlying deletion was not always verifiable, though, and earlier context sometimes resurfaced after a reset or exclusion request.
4. A confident answer can contradict the record it came from. Checking against the connected account overturned answers that read as authoritative, including claims that their own sources did not support. Plausibility and grounding can come apart.
5. Over-confirmation is its own failure mode. Some assistants completed authorized actions only after seeking confirmation they had already been given. Withholding in-scope work imposes a real cost, even when it looks like caution.

Limitations and next releases

These are early results from a small, Google Workspace-heavy sample with six assistants and 20 questions. The tests used the same workflow families, but prompts, connected accounts and prior conversation history were not identical. Those differences can affect the standings.

Across the six assistants, the mean Task Completion and mean Trusted Completion have a relatively small gap. This should not be interpreted as evidence that delegation risk is rare, however: it reflects task set and defined critical violation thresholds. Future releases will add more task types, account environments and controlled repeat testing in the coming days.

The twenty tasks

The twenty tasks cover preparing a meeting briefing from the right context, identifying personal priorities and their sources, carrying personal constraints into later planning, respecting a withdrawn preference, and keeping drafts separate from permission to send. They also test applying corrected instructions before acting, limiting private disclosure, spotting likely problems in email and Calendar, researching an urgent purchase at its delivered price, handling judgment-dense requests involving multimodal inputs, and using only appropriate context in a professional introduction.

What this benchmark adds

Assistant Benchmark provides broad, real-account comparisons of consumer assistants, while the rtvr AI Agent Benchmark compares agents on identical fictional browser tasks. PersonalAgentBench combines delegated work in persistent personal accounts with research, multimodal and action-based questions. We assess whether an assistant completes the task, uses context appropriately, stays within scope and accurately reports the resulting state.

Two choices make the benchmark distinctive. We separate completing the task from completing it within the user’s authority, and we test whether assistants use personal context appropriately when it is relevant. Unnecessary refusals and repeated confirmation requests count as user burden, alongside the risks of acting without permission.

Authority is scored in both directions. A run loses credit for acting beyond the authority it was given and for withholding work that was inside scope.

Conclusion

PersonalAgentBench is built around the simple idea that capability alone is not enough for delegation. As personal agents gain more memory, connectors, and authority, evaluation must test not only whether they finish the task, but whether they use the right context, stay within scope and accurately report the resulting state. These results are our first snapshot of answering that question and will be built on over time.