Running test cases
Judge a capture against a test case: pass, fail or inconclusive, per file of a batch or from the command line, with JUnit for a test bench.
What a test case checks
The worked test cases on the Learn shelf on testing say what each run has to show. Those a capture can decide are built into the analyzer as expectations on its own analysis: the verdict (outcome, category, likely party), checks that have to pass, checks that must not fail, checks that have to fail because the test injects that fault, and the state each stage has to end in.
| Result | When |
|---|---|
| pass | Every expectation holds |
| fail | At least one does not: a check failed, a check the test expects to fail passed, the verdict or a stage is not the one expected |
| inconclusive | Nothing failed, but the capture cannot decide: a check that has to pass did not apply (no Control Pilot log, no TLS), or a stage is not in the capture |
- A test records one run. When a file holds several sessions, each has to meet the expectations: one failing session fails the test, so a retry that worked does not hide the attempt that did not. Sessions the test cannot judge count only when none can be judged.
- A negative test, such as DC-06 with an insulation fault, passes when the capture shows the fault and the reaction the standard asks for. The same capture fails the positive test DC-05.
- A pass says what the capture shows against the expectations. It is not a certification result: the test body, the setup and the equipment are yours to document.
In batch mode
- Name each capture after its test case, the way a test bench numbers its records:
DC-06_insulation_fault.pcapng,AC-03 run 2.pcapng. The id has to stand on its own, soDC-061is not DC-06. - On the Batch page, set Test cases to From the file names, or pick one test case for every file, then drop the folder.
- The table gets a column with the result of each file; open a result to read why. Export JUnit writes one test per file and test case, a failure for fail and skipped for inconclusive. Export test results writes the same as CSV.
In the analyzer
- Open the capture and the Test cases tab beside the findings. It takes the test case from the file name, or pick one; the result covers every session of the file.
- The report you export then carries a Test cases section with each result and why, in the full report and the one page summary.
- Write a test plan opens an editor: checks by id, the outcome, the category, the likely party and the stages, checked as you type. Use this plan keeps it for the tab and for batch mode; Download plan saves it as JSON for the command line.
From the command line
node dist-cli/v2g-analyze.mjs DC-06_fault.pcapng --testcase auto --junit dc-06.xml node dist-cli/v2g-analyze.mjs run.pcapng cp.csv --testcase AC-03,BE-02 node dist-cli/v2g-analyze.mjs run.pcapng --testplan lab-plan.json
--testcasetakes ids separated by commas, orautofor the ids in the file name.--testplanreads test cases of your own; without--testcaseit runs all of them.- With test cases, they decide the exit code: 0 when every one passes, 1 when one fails or is inconclusive, 2 when an id is unknown or the plan cannot be read. The JUnit file gets a Test cases suite in front of the per check suites,
--jsonatestCaseslist and--htmla Test cases section.
A test plan of your own
A test plan is a JSON file with a list of test cases, or an object with a testCases list. A test case of the plan with the id of a built-in one replaces it. Checks are named by id, including the custom checks of the active limit profile; the plan is refused when it names a check the analyzer does not have, repeats an id, or expects nothing.
{
"testCases": [
{
"id": "LAB-12",
"title": "Cable check with 90 kOhm insulation fault",
"reference": "IEC 61851-23",
"expect": {
"outcome": ["Never charged"],
"fail": ["SQ-05"],
"noFail": ["TIM-01"],
"stages": [
{ "name": "Cable check", "status": ["failed"] },
{ "name": "Energy transfer", "status": ["not-reached"] }
]
}
}
]
}| Field of expect | Meaning |
|---|---|
outcome | The verdict outcome is one of these: Completed, Stopped early, Never charged, Incomplete capture |
category | The verdict category is one of these, as the verdict names it |
likelyParty | The likely party is one of these: EV, EVSE, Either, Unknown |
pass | These checks run and pass; one that does not apply makes the test inconclusive |
noFail | These checks do not fail; one that does not apply is fine |
fail | These checks run and fail |
stages | Each stage, by the name the stage strip shows, ends in one of the states: passed, failed, not-reached, not-in-capture |
Built-in test cases
Ids match the worked test cases on the Learn shelf; a letter marks a variant that injects a fault. The test cases that need equipment beyond a capture (a firmware update, a payment terminal) stay on the Learn pages; the OCPP boot (BE-01) needs an OCPP log that starts with the BootNotification.
| Id | Test case | Expects | Passes on the demo |
|---|---|---|---|
| AC-01 | Basic charging with PWM only | pass CP-01, CP-02, CP-03, PW-01, PW-02; no failure of CP-10, PF-01, PF-02, PF-03 | ac-ok |
| AC-02 | 5 percent start for HLC | pass CP-03, CP-04, SL-01; no failure of SEC-01, SEC-02, SEC-04 | ac-ok |
| AC-03 | ISO 15118-2 AC session with EIM | outcome Completed; pass SQ-01, SQ-02, SQ-03, SQ-04, SQ-05, SQ-06, SQ-07, TIM-01, TIM-04, VT-01, VT-02; no failure of CP-06, CP-07, TIM-03, TIM-06; Energy transfer passed | ac-ok |
| AC-04 | Limit change by PWM | pass PW-01; no failure of CP-03 | |
| AC-05 | Limit change by schedule | pass SQ-08, SQ-09; no failure of PW-01 | renegotiation |
| AC-06 | Cable rating by PP | pass PP-01, PW-01; no failure of PF-04 | ac-ok |
| AC-07 | Three phase and single phase | pass PW-01; no failure of IS20-10 | |
| AC-08 | Stop by the car | pass PW-03, CP-08; Stop passed | ac-ok |
| AC-09 | Stop by the charger | pass PW-03; no failure of SQ-09 | |
| AC-10 | Pause and resume | pass SQ-11; no failure of PW-03 | pause-resume |
| AC-11 | Control Pilot short and charger fault | pass CP-02, CP-09; no failure of PW-03 | |
| AC-12 | No SLAC after 5 percent | pass CP-05; no failure of SL-07 | |
| AC-13 | Loss of communication while charging | outcome Stopped early; pass PW-03; no failure of TIM-03 | |
| AC-14 | Ventilation request (state D) | pass CP-02 | |
| AC-15 | Reactive power setpoint | pass IS20-11; no failure of IS20-10 | |
| DC-01 | SLAC under attenuation | pass SL-01, SL-02, SL-03, SL-04, SL-05, SL-06, SL-07, SL-08; no failure of SL-09 | iso20 |
| DC-02 | SECC discovery | pass SD-01, SD-02, SD-03, SD-04, SD-05, SD-06, TC-01; no failure of TC-03 | iso20 |
| DC-03 | TLS handshake | pass TL-01, TL-02, TL-03, TL-04, PK-01; no failure of TL-05, TL-06, TL-07, TL-08, PK-02, PK-04, SD-04, SEC-03 | tls-ok |
| DC-03b | TLS with an untrusted charger certificate is refused | outcome Never charged; category TLS failure; fail TL-01; TCP / TLS failed | tls-fail |
| DC-04 | Protocol handshake | pass HS-01, HS-02; no failure of HS-03 | ac-ok |
| DC-05 | Cable check, normal | pass TIM-02, SQ-05, SQ-01; Cable check passed | iso20 |
| DC-06 | Cable check with insulation fault | outcome Never charged; fail SQ-05; Cable check failed; Energy transfer not-reached | cablecheck-fail |
| DC-07 | Pre-charge within tolerance | pass DC-01, TIM-01; Pre-charge passed | iso20 |
| DC-08 | Pre-charge timeout | outcome Never charged; fail DC-01; Pre-charge failed; Energy transfer not-reached | precharge-fail |
| DC-09 | Current demand following | pass TIM-01, TIM-05, PW-05; no failure of PW-01, PW-07, IS20-02, IS20-04; Energy transfer passed | iso20 |
| DC-10 | Charger power limitation | no failure of PW-01, IS20-03 | iso20 |
| DC-11 | Renegotiation | pass IS20-09, SQ-01; no failure of SQ-09 | |
| DC-12 | Bidirectional discharge limits | pass IS20-01, IS20-02, IS20-03, IS20-04, IS20-05, IS20-07, IS20-08 | |
| DC-13 | Stop by the car | outcome Completed; pass SQ-01, TC-02; no failure of PW-03; Stop passed | iso20 |
| DC-14 | Emergency stop or stop at the charger | category EVSE stopped; no failure of SQ-09, PW-03 | emergency-stop |
| DC-15 | Welding detection | pass DC-02, SQ-01; Stop passed | emergency-stop |
| DC-16 | Pause and resume | no failure of SQ-11, IS20-06 | pause-resume |
| DC-17 | CHAdeMO basic session | pass CH-01, CH-02, CH-03, CH-04, CH-05, CH-06, CAN-01, CAN-02 | chademo |
| DC-17b | CHAdeMO charger fault | outcome Stopped early; likely party EVSE; fail CH-04 | chademo-stop |
| DC-19 | Limits of both sides held | pass LM-01, LM-02, LM-03, LM-04, LM-05 | din-ok |
| DC-18 | GB/T 27930 basic session | pass GB-01, GB-02, GB-03, GB-04, GB-05, GB-06, GB-07, GB-08, CAN-01 | gbt |
| DC-18b | GB/T 27930 stop by a BMS fault | outcome Stopped early; category Vehicle fault; likely party EV | gbt-bst |
| BE-01 | Boot | pass OC-06, OC-01; no failure of OC-02, OC-04 | |
| BE-02 | Authorization accepted | pass TIM-02, SQ-05; Setup passed | ac-ok |
| BE-03 | Authorization rejected | outcome Never charged; category Authorization not granted; pass XL-04 | ocpp-rejected |
| BE-05 | Transaction events | outcome Completed; pass PW-06, OC-01, OC-02, XL-01, XL-02, XL-05, CDR-01, CDR-02, CDR-03; no failure of OC-03, OC-05, OC-07, OC-08, OC-09, CDR-04 | ac-ok |
| BE-06 | Meter values | pass PW-04, PW-06, OC-01; no failure of SQ-10, XL-03, SM-01, SM-02, SM-03 | ac-ok |
| BE-10 | Charging profile | no failure of PW-01, SQ-08 | ac-ok |
| BE-11 | Contract certificate installation | pass TIM-01, SQ-05 | |
| BE-12 | Plug and Charge authorization | pass SQ-05, TIM-02, TL-01, PK-03, PK-05, XL-06; Setup passed | |
| BE-13 | Expired certificates | outcome Never charged; fail PK-01 | pnc-expired |