Compare mobile networks through the services people need to complete: an ordinary call, a delivered message or a usable data transfer. Define the test conditions and success criteria first, then retain failures as well as successful attempts. A fast result on one phone at one moment is a useful observation with a limited scope.
This guide is for technical evaluators planning a small cellular service comparison. Use authorized devices, permitted workloads, consenting test recipients and accounts whose data allowances you understand. Perform tests while stationary or as a passenger; use ordinary test numbers, never emergency services.
Make the comparisons comparable
Record the device model, software, SIM or eSIM, provider, plan, selected network mode, location, time and test tool version. Note Wi-Fi Calling, VPNs and any other settings that change the traffic path. Decide whether you are comparing the complete service as a user receives it or isolating one technical variable.
For a complete-service comparison, device and plan differences may be part of the experience, but disclose them. To isolate a provider effect, hold other conditions as close as practicable and record the remaining differences. Interleave repeated tests so that one provider is not measured only at a quieter time.
ITU-T E.804 organizes mobile-service quality parameters from the user's perspective. Its E.804.1 application guide explains how to apply metrics and define observation points. These are references for designing measurements; the simplified field record here is not a claim of standards-conformant laboratory testing.
Define what each trial measures
On a narrow screen, scroll the table sideways. Keyboard: focus the table and use the arrow keys.
| Task | Start and completion events to define | Failure evidence to retain |
|---|---|---|
| Ordinary voice call | Dial action, connection, agreed conversation interval and normal end | Setup failure, dropped call and audible interruption |
| Message | Send action and confirmed receipt of the identified message | Timeout, late arrival or duplicate |
| Web task | User action and required page or application state | Error, incomplete page or login failure |
| File transfer | Agreed start and complete verified file | Timeout, incomplete content and retry |
| Service recovery | Documented loss of service and completion after return | Reconnection delay and required user action |
Specify the timeout and workload before testing. A ten-second timeout and a sixty-second timeout produce different completion counts. Record whether latency includes name lookup, connection setup, authentication and server processing. Use the same definition when comparing results.
Choose a known test endpoint and payload, and keep their behavior stable where possible. A busy server can affect the whole measured service. Retain endpoint errors instead of assigning every failure to the radio network.
Keep denominators and delays together
Use the mobile-service test record for one row per attempt. Assign a unique reference so sending and receiving observations can be matched. Prefer a single measurement clock for elapsed time; document clock synchronization when timestamps come from separate devices.
Fictional example: twenty message trials produce eighteen receipts within a predefined thirty-second deadline, one receipt at forty-five seconds and one with no receipt during the observation period. The on-time result is 18/20 = 90%, with one late and one unconfirmed. Report those categories rather than dropping the two inconvenient attempts.
Summarize the distribution of delays for the stated subset. A median describes the middle result; slower-tail measures need a declared calculation method and enough observations to be useful. Include sample size, minimum and maximum, and failures. A small pilot can reveal a recurring problem without supporting a precise population-wide reliability estimate.
Interpret scores within their method
A personal rating of how a call sounded and an instrument's objective speech-quality score come from different methods. When reporting a MOS-related result, identify the algorithm or rating procedure, version, codec conditions and test setup. Keep subjective notes labeled as such.
An app's 4G or 5G indicator describes an observed connection state; use device diagnostics when a comparison requires a specific underlying radio configuration. Preserve automatic transitions as part of an ordinary user-experience test. Only lock a mode through supported controls when the experiment specifically calls for it and the service supports it.
When a pattern appears, repeat a discriminating comparison: same task at another time, same location with another device, or the same endpoint through another path. Separate the observation from the proposed cause. If the concern is a particular meeting app, the video-conferencing guide supplies application-focused checks.
Finish with the tested places, dates, devices, plans, workloads, all attempt counts, results and unresolved explanations. That record lets a colleague reproduce the comparison and tells a decision-maker exactly how far the evidence extends.