CharIN Testivals and the first public test data

What happens at a CharIN Testival, and what the first public release of Testival results (VOLTS, Long Beach, May 2023) says: the vehicles and chargers, about 400 hours of testing in 174 pairings and over 1,000 tests, and how to read aggregate interoperability data.

What a Testival is

CharIN, the association behind CCS, runs Testivals: several days where car makers, charger makers and test system vendors pair their products in fixed time slots and debug together. The format, the recent events and the ChargeX prescribed testing run at them are in Interoperability test events. Results of a single pairing normally stay with the two participants.

VOLTS 2023

The Vehicle Interoperability Testing Symposium (VOLTS) ran from 8 to 12 May 2023 in Long Beach, California: a conference and two days of testing, organised by CharIN and funded by the California Energy Commission.

As reported
Attendees at the testingOver 400, from 38 companies
Vehicles4 school buses, 4 trucks and 13 cars
Charging stations18 in the CharIN announcement, 21 in the Energy Commission summary
Test systems7
ConferenceOver 350 attendees and 34 speakers, with ISO 15118 as the focus

The first public test data

The California Energy Commission published the VOLTS results in report CEC-600-2025-012 (March 2025). CharIN called it the first time Testival results were made public. The headline numbers:

  • Almost 400 hours of interoperability testing.
  • 2 days, 7 test slots and 174 pairings of one car with one charger, each of 2 hours.
  • A summary of over 1,000 individual tests.
  • More than half of all tests used ISO 15118 communication.

The report itself could not be read for this page; the numbers above are from the CharIN announcement and the Energy Commission summary. Pass rates, the split between DIN SPEC 70121 and ISO 15118, and the failure causes the report gives are not stated here until they are read in the report.

Reading aggregate test data

  • A pairing is a slot, not a test. In two hours a pair may run many tests or spend the time on one problem, so counts of tests and counts of pairings answer different questions.
  • Products at a Testival are often prototypes or pre-release software, brought there to find faults. A low pass rate at an event says less about the market than field data does (Reliability of public charging: what field data says).
  • Published data is anonymous and aggregate. It shows which features and stages cause trouble across the industry, not which product does.
  • What makes a failed pairing useful afterwards is a record of it: a capture on the cable with a Control Pilot log, and the OCPP log where a backend was involved (Recording a charging session for analysis).

Since then

The CharIN Testival North America 2024, hosted by Lincoln Electric, was the largest so far by CharIN's count, with 415 participants. It added multi-PKI testing, prescribed testing of Plug and Charge and PKIs, end to end Plug and Charge, and products using ISO 15118-20. The prescribed testing there and at later events used procedures that became the ChargeX EEITP (The ChargeX EV-EVSE Interoperability Test Plan).

In this tool

  • Drop the captures of an event on the Batch page. Below the table, Cars against chargers sets every car (EVCCID, or the MAC when the car sends none) against every charger (EVSEID, or the modem MAC) as a pairing grid: each cell holds the sessions completed of all the sessions of that pair, and names the most common problem of the pair on hover (FR-TOOL-18).
  • A cell counts sessions, not pairing slots or tests: a pair that retried ten times in one slot shows ten sessions.
  • Export frames writes the findings into the pcapng as packet comments, so the capture of a failed pairing opens in Wireshark with the reasons beside the packets.

Sources