NeutralIntelligence

NeutralIntelligence / Research & practice

Requirements first. Evidence next.

A practical structure for AI system evaluation and AI risk assessment.

Frame the decision

Describe the work, expected users, operating environment and the consequence of an incorrect output. Define what a successful trial would need to demonstrate before choosing tests.

Make priorities explicit

Mark requirements Critical, Important, Useful or Not Relevant. A critical requirement represents a decision constraint. It should not disappear inside an average score. Revisit priorities when scope changes.

Separate documentation from performance

A supplier document can describe an intended control without demonstrating that the control is effective in your environment. Keep documentary evidence, observed testing and reviewer judgment distinct.

Measure the gaps

The comparison view counts documented and partially documented active requirements. These are coverage counts, not performance ratings. Missing evidence is not proof of failure, and documented evidence is not proof of suitability.

Assign a decision owner

Record who accepts remaining uncertainty, which conditions apply to a pilot, and what would stop deployment. This demonstration generates questions; it cannot accept risk on behalf of an organization.

Define your evaluation