Analysis Protocol — Applied Technology Index

Scope

This protocol defines the minimum reporting standard for research papers, including how subjects are selected, how tests are executed, and how limitations are disclosed.

Definitions

• Subject: A tool, platform, or environment included in an analysis.
• Benchmark task: A standardized task executed across subjects to enable comparison.
• Production latency: The measured friction from intent to a verified, deployed result, as defined per study.

Procedure

1. Define selection criteria and exclusions.
2. Describe the testing environment, including hardware, network conditions, and tooling versions.
3. Define measurement units and any composite metrics.
4. Execute the benchmark task at least three times per subject and report the aggregation method.
5. Report results with sources, and explicitly list limitations that bound interpretation.

Units & Metrics

All metrics must include:

• unit (e.g., ms, seconds, tokens, cost)
• measurement window (e.g., per run, per 10 tasks)
• aggregation method (mean/median, variance)

Known Limitations

Protocol compliance does not guarantee external validity. Differences in configuration, model versions, and network conditions can materially affect outcomes.

Version History

• 1.0 (2025-12-21): Initial protocol definition.