Posts tagged “evaluations”
Sophos’s Daybreak MDR Workflow: What to Check Before Automating Response
Sophos’s Daybreak case study puts MDR automation in focus. Examine the reported metrics, customer response modes and evidence needed for a bounded pilot.
Google’s Patent-Lawyer Study: Measure AI Output and Human Judgment Separately
Google’s patent-lawyer study separates AI-assisted drafting gains from unaided judgment. Here is how to evaluate output, efficiency and professional learning.
OpenAI–Ironclad Contracting Evals: What a Workflow Pilot Must Prove
OpenAI’s Ironclad research informs a contracting-agent pilot: test approval routing, reapproval and configuration behavior before deployment.
OpenAI’s MentalHealthBench Makes the Follow-Up Question Count
OpenAI’s MentalHealthBench tests AI mental-health responses. We inspect its data, scoring limits and the product checks builders still need.
Gemini Reached Three Real Companies. Authorization Failed Before Login
Google confirmed Gemini accessed three companies during cyber evals. The deeper failure was letting authentication happen before proving authority.
Anthropic’s Accenture Evaluator Deal Makes Independence a Contract Test
Anthropic named Accenture’s Faculty as an embedded evaluator. The real test is whether evidence, reporting rights and funding survive commercial conflict.
Anthropic’s Embedded Evaluator Pledge Needs More Than Access
Anthropic’s embedded-evaluator pledge could shift AI assurance from model cards to operational audits. Here is what builders should demand next.