Anatomy of a test case
What a test case contains: identifier, purpose, the requirement it proves, preconditions with PICS and PIXIT, the test body, expected observations, pass criteria, verdicts and post-conditions, and how conformance, interoperability and lab test cases differ. With one example written out.
Why the structure matters
A test case is a small contract. Anyone who runs it must reach the same verdict on the same product, and anyone who reads the verdict must know which requirement it proves. The conformance methodology of ISO/IEC 9646 gets there in steps: requirements of the standard become test purposes, test purposes become test cases, and the test cases are grouped into a test suite. The ISO 15118 conformance parts follow this path; ISO 15118-4 writes its test cases in TTCN-3, and ISO 15118-21 and -23 describe theirs in tables.
The work item for ISO 15118-22 names the parts of a test case plainly: pre-conditions, test behaviour, expected results to decide pass or fail, and post-conditions that return the system under test to a safe state. The fields below follow that order.
The fields
| Field | What it holds | Typical mistake |
|---|---|---|
| Identifier | A stable id: suite, side, topic, number (for example DC-07). Never reused. | Renumbering, so old reports point to the wrong case |
| Purpose | One sentence: what behaviour the case proves | Two behaviours in one case, so a fail does not say which |
| Requirement | The clause or requirement id of the standard it traces to | No trace, so nobody can tell whether the case is still valid after a new edition |
| System under test and tester | Which side is tested (EVCC or SECC) and what plays the other side | Testing both real products and calling it conformance |
| Preconditions | The claimed options (PICS), the parameters (PIXIT), the initial state, the equipment | An initial state that depends on the previous case |
| Test body | The stimulus steps, in order, with the values the tester sends | Steps a person reads differently, such as "wait a while" |
| Expected observations | Messages, fields, timings and electrical values that must appear | Only the final state, not the path to it |
| Pass criteria | The rule that turns observations into a verdict, with its limits | Limits in the text instead of in the parameter set |
| Verdict | Pass, fail or inconclusive, with the reason | Fail for a case that could not run |
| Post-conditions | How the product returns to a safe, known state | Leaving a DC output energised for the next case |
PICS, PIXIT and verdicts
The PICS (protocol implementation conformance statement) lists the options a product claims: AC or DC, EIM or Plug and Charge, TLS, which protocols. It selects which test cases apply. The PIXIT (protocol implementation extra information for testing) gives the values the tester needs: addresses, timeouts the product uses, maximum voltage and current, certificates. It parameterises the selected cases. A case whose option the PICS does not claim is not run, so it cannot fail. More in Conformance testing: ISO 15118-4, -5 and Testivals.
| Verdict | Meaning |
|---|---|
| Pass | The observations match the test purpose: the product did what the requirement asks |
| Fail | An observation contradicts the requirement: a wrong message, a wrong value, a limit exceeded |
| Inconclusive | Nothing contradicts the requirement, but the case could not reach the point where it decides, for example because the product stopped for an unrelated reason |
TTCN-3 also knows the values none (no verdict set yet) and error (a fault in the test system itself). The TTCN-3 core language (ETSI ES 201 873-1, ITU-T Z.140) orders the values none, pass, inconc, fail, error, and a verdict can only get worse during a run, never better.
Conformance, interoperability and a lab plan
| Conformance test case | Interoperability test case | Lab's own test plan | |
|---|---|---|---|
| Other side | A test system that plays the peer and can misbehave on purpose | A real product | Either, often a simulator |
| Source | The conformance part of the standard | Event or program plans (CharIN, ChargeX) | Customer needs, field faults, regulations |
| Faults | Injected by the tester: late answers, wrong ids, bad certificates | Mostly real use: plug, stop, unplug, pull the network | Both, plus environment: heat, cold, grid |
| Verdict | Pass, fail, inconclusive against one requirement | Works or does not, per pairing | As the plan defines |
| Blame | Always the system under test | Open: either side, or the standard | As investigated |
| Repeatable | Yes, by design | Only as far as the two products are | Depends on the plan |
The ChargeX interoperability plan shows the second kind: each case has a purpose, a setup, a procedure and pass criteria, but the scenarios are those a driver meets, such as a partly inserted connector, a broken latch, or the internet dropping during power transfer.
One test case written out
| Field | Content |
|---|---|
| Identifier | DC-07 (from DC test cases, worked) |
| Purpose | The charger raises its output to the voltage the car asks for in pre-charge, within the tolerance and before the timeout |
| Requirement | IEC 61851-23 pre-charge; the PreCharge messages and timing of ISO 15118-2 (or DC_PreCharge of ISO 15118-20) |
| System under test | SECC with its power stage (the charger) |
| Tester | EV simulator with an EVCC stack, a DC load or battery emulator and a voltage probe at the inlet |
| PICS | DC, ISO 15118-2, EIM; TLS optional |
| PIXIT | Maximum and minimum output voltage of the charger; pre-charge tolerance and timeout to judge against |
| Preconditions | Session set up to the end of CableCheck with isolation valid; battery emulator at 400 V behind open contactors |
| Test body | 1. Send PreChargeReq with EVTargetVoltage 400 V and EVTargetCurrent 2 A. 2. Repeat PreChargeReq until EVSEPresentVoltage is within tolerance of 400 V, or the timeout expires. 3. Close the EV contactors. 4. Send PowerDeliveryReq Start. |
| Expected observations | Every PreChargeRes answers with ResponseCode OK within the message timeout; EVSEPresentVoltage rises steadily; the probe at the inlet agrees with EVSEPresentVoltage |
| Pass criteria | Present voltage within tolerance of the target before the pre-charge timeout; no response later than the message timeout |
| Fail | Timeout reached with the voltage outside tolerance, a response code other than OK, or a late response |
| Inconclusive | The session ended before PreCharge for a reason outside the case, for example a CableCheck failure |
| Post-conditions | PowerDeliveryReq Stop, WeldingDetection, SessionStopReq; output below a safe voltage before the connector unlocks |
| Observed in this tool by | DC-01 (pre-charge reaches the target), TIM-01 (response times), the Curve tab (voltage over time) |
The tolerance and the timeout are parameters, not constants. The default limit profile uses 20 V and 7 s, both flagged verify: the first is common practice for IEC 61851-23 pre-charge, the second V2G_EVCC_PreCharge_Timeout from secondary sources.
Writing good test cases
- One behaviour per case. A case named "session works" fails without saying why.
- Every limit comes from the parameter set, so the same case runs against another edition or another product class.
- Write the negative case next to the positive one: pre-charge that reaches the target, and pre-charge whose output stalls.
- Record evidence: a capture of the powerline traffic, a Control Pilot log, and meter readings. A verdict without evidence cannot be reviewed.
- End every case in a safe state, and check it: no voltage on the inlet, contactors open, connector unlocked.
In this tool
Each check of the analyzer works like the pass criteria of a test case: it names its reference, reads its limits from the limit profile, and ends in pass, fail or not applicable. Not applicable plays the part of a case that the PICS excludes. The demo precharge-fail shows the negative variant of the example above, and iso20 a passing DC session. See Limit profiles: the numbers behind every check for the parameters and Session end verdicts for how the checks add up.
Sources
- ISO/IEC 9646, conformance testing methodology: test purposes, abstract test suites, PICS, PIXIT, verdicts
- ISO 15118-4:2018, network and application protocol conformance test, catalogue abstract: abstract test suite in TTCN-3 for an EVCC or SECC implementing ISO 15118-2, layer 3 and above
- ISO 15118-21:2025, common second generation conformance test plan, catalogue abstract: requirements of ISO 15118-20 common to AC, DC, ACD and WPT; descriptive tables, layers 3 to 7
- ISO/AWI 15118-22, second generation protocol security requirements conformance test plan, approved work item: simulated SECC, EVCC and PKI; Plug and Charge, XML signature, end to end security; test case attributes
- ISO/PAS 15118-23:2026, second generation conformance test plan for DC charging, catalogue abstract: DC specific requirements of ISO 15118-20; test architecture inherited from ISO 15118-21
- ChargeX EV-EVSE Interoperability Test Plan (EEITP), version 2.0, July 2025, Argonne, Idaho and NREL researchers of the ChargeX testing task force; test tree with baseline, safety and advanced tests
- ChargeX prescribed testing program at the CharIN June 2024 Testival, eight scenarios in the last 30 minutes of a 90 minute slot
- IEC 61851-23, DC charging station: cable check, pre-charge, emergency shutdown
- This tool: default limit profile, src/core/limits/iec61851-iso15118-2-default.json, values and their verify flags
- ETSI ES 201 873-1: TTCN-3 core language, verdict values none, pass, inconc, fail and error, and their overwriting rules
- ITU-T Z.140: TTCN-3 core language