Vendor brochures explain what a product is intended to do. A buying decision needs the next layer: the exact version, dependencies, commercial terms and evidence that it will do your work. Turn broad claims into specific questions before a demonstration.
Write requirements before comparing products
Describe a small set of real tasks and constraints. Specify the users, workload, data, integrations and operating environment. Separate requirements that determine whether a product is usable from preferences that make it more convenient. Give each requirement an owner and an acceptance condition.
For example, “export customer records” leaves important questions open. Specify which records and attachments, which relationships must survive, the format, access permissions and how the exported result will be checked. Include the point at which you may need to leave the service.
Collect dated, version-specific documents in one comparison folder: supported-platform lists, API documentation, security information, support policy, license terms, implementation statement and pricing assumptions. Record their source URLs so a reviewer can revisit the evidence.
Read different evidence for different purposes
On a narrow screen, swipe the table or focus it and use the arrow keys.
| Claim | Evidence to request | Practical test |
|---|---|---|
| Integrates with our system | Named connector or interface, versions, supported fields and failure behavior. | Run a representative transaction and reconcile both systems. |
| Handles our workload | Documented limits, test conditions and resource assumptions. | Use realistic concurrency and data volume in an agreed trial. |
| Protects our information | Access controls, logging, update policy and vulnerability reporting process. | Inspect available settings and export sample security events. |
| Easy to leave | Export formats, attachments, API limits, assistance and fees. | Export a sample and open it outside the service. |
| Includes support | Covered versions, response hours, escalation and end-of-support terms. | Walk through a real incident scenario with the service team. |
A case study can show how another organization used a product; its workload and implementation may differ from yours. A certificate or assessment report has a particular scope, period and subject. Check which entity, service and controls it covers before treating it as evidence for a product feature.
CISA's Secure by Demand guide offers specific software-purchasing questions about security features, vulnerability handling and evidence available to customers. Use it for the security part of procurement alongside your functional, operational and commercial requirements.
Turn a demonstration into an acceptance exercise
- Agree the setup. Record product edition, version, configuration and integrations. Use synthetic or appropriately approved test data.
- Run normal and exception cases. Include permissions, invalid data, interruption and recovery where relevant.
- Observe the result. Save outputs and logs, and distinguish a demonstrated feature from a future promise.
- Record gaps. Name the owner, proposed remedy, cost and due date. Retest changed behavior.
- Decide. Apply the acceptance criteria set before the trial, including any business-approved exceptions.
Connect the technical result to the agreement
Confirm that the quoted edition contains the tested capability. Ask about usage limits, required add-ons, implementation services, integration maintenance and the cost of exporting data. A feature demonstrated in a vendor's highest tier may be absent from the offered package.
Distinguish what is generally available, what is custom work and what appears only on a roadmap. If a future capability determines the purchase, record the delivery dependency and a fallback; ask the commercial and legal owners to review the proposed commitment.
Use the claims-to-evidence worksheet to compare vendors. Keep mandatory requirements visible before using a weighted score: a high total can conceal a failed requirement. Carry unresolved dependencies into the technology risk assessment, and use the business case to decide whether the demonstrated improvement is worth its full cost.
After purchase, retain the test evidence and review important claims when a major version, service policy or integration changes. The comparison becomes a useful acceptance record for the people who must operate the product.