Search evaluation sheet
Yenra | September 10, 2026
https://yenra.com/enterprise-search-solutions/

Use one record per query and test configuration. Write expected behavior before running the query. Include exact IDs, spelling errors, constraints, absent content, and permission-sensitive cases. Keep a baseline and reserve queries for later testing.

Collection/schema version and date:
Search configuration and source snapshot:
Query and query type:
User intent and hard constraints:
Test identity and expected permissions (no credentials):
Expected useful document/product IDs:
Relevance rule agreed by reviewers:
Rank 1 ID, relevant yes/no, reason:
Rank 2 ID, relevant yes/no, reason:
Rank 3 ID, relevant yes/no, reason:
Rank 4 ID, relevant yes/no, reason:
Rank 5 ID, relevant yes/no, reason:
Convention if fewer than five results:
Relevant results among first five / 5:
First useful result rank:
Latency (milliseconds), measurement method/workload:
Filter behavior and zero-result behavior:
Permission leakage checks (results/snippets/answers/direct links):
Update/delete/revocation freshness (elapsed time and expected limit):
Completed user task:
Failure, owner, proposed change, retest result:

Fictional example: three relevant results in the first five gives 3/5 = 0.60. This metric alone does not measure authorization, completeness, speed, or purchase success.
Metric reference: https://www.elastic.co/docs/reference/elasticsearch/rest-apis/search-rank-eval