We’ve released dnp3c, the DNP3 conformance test harness we’ve used internally for over a decade, as open source under the GPL. It runs the DNP Users Group’s IED Certification Procedure against an outstation, either by hand or completely unattended in CI.

Where dnp3c came from

I started writing a conformance harness in 2014 to test OpenDNP3. When we wrote our Rust DNP3 library, the same harness tested that too. It was always an internal tool, and it stopped growing in 2020, once it covered what we needed: all of revision 2.7 of the certification procedure, for Subset Level 2, over TCP.

It was a heavy lift. Counting conservatively, only the days that show up in the commit history, the harness took at least 300 hours of work, spread over six years and two engineers. That bought about 12,000 lines of Scala and one revision of the certification procedure at one subset level.

Keeping it private made sense while it only tested our code. But the problem it solves isn’t ours alone.

Why conformance testing doesn’t happen in CI

DNP3 test tools automate individual tests well, but not the work around them. Many tests need something done to the device outside of DNP3: cycle its power, generate events on its inputs, change its configuration between tests. The tool stops and waits for a person to do it, then moves on.

Some tools can script those steps, but that doesn’t get you to CI. The tools themselves are desktop GUI applications, licensed to a machine. They can’t run headless on a CI runner, in a container, or on a build server, so even a scripted run needs someone at a desktop to start it and collect the results. When a test fails, you can’t read the logic behind it, and the results are saved in a format you need the same tool to open.

I’ve done a full run by hand. It takes days at the bench, and it’s brutal. Worse, it mostly happens in the wrong place. Nearly all of a device’s DNP3 behavior comes from the protocol library inside it, so that’s where most conformance problems start, and where they’re cheapest to fix. Testing the finished device still matters, because it checks the configuration and integration, but by then the protocol itself should already be right.

The library is also where testing is hardest. There’s no device, so you build a test application and then sit through hundreds of tests, prompt by prompt, for every configuration the library supports. So conformance testing happens once, before a release or a review, and not on every change. As far as we know, almost nobody runs DNP3 conformance tests in CI, because no tool has made it practical.

dnp3c addresses both problems. You implement one interface that performs the steps outside DNP3 on your device, and dnp3c itself is a headless command-line tool and a library that runs anywhere Java does. Every test that applies to your device then runs unattended, from start to finish, on every commit.

What dnp3c does

dnp3c runs the outstation tests of the DNP3 IED Certification Procedure: revision 3.1 rev 1 at Subset Levels 1 to 3, and revision 2.7 at Level 2. It talks to the device over TCP, UDP, or serial. You can use it two ways:

• As a command-line tool, for testing a device by hand. Steps it can’t perform over DNP3, like cycling power or generating events, are prompted for.
• As a library from Maven Central, for automated testing, with your implementation of that one interface performing those steps.

Each result in the report links to a log of every step: every frame sent and received, in hex, decoded at every layer and timestamped, along with every check and its outcome.

dnp3c doesn’t certify anything by itself, but it produces the evidence certification is based on. The DNP Users Group lets vendors test their own devices and declare the results. For an independent, official review, its Conformance Test Review examines a vendor’s Device Profile and test logs, and the logs dnp3c writes are the kind it asks for.

Every run produces a package anyone can read with no specialized tooling: an HTML report that opens in any browser, a log for every test, and a pcap of each test’s traffic. Pcap is better evidence than any vendor’s file format. It’s an open standard, and Wireshark decodes DNP3, so a customer or reviewer can inspect the raw traffic themselves instead of trusting the test tool’s own reading of it.

We run dnp3c on every pull request to our own DNP3 library, and it has already found real problems.

What CI caught

In 2022 we opened an issue against our outstation. When a master reads a range of points and only some of them exist, the outstation should flag the error with an internal indication bit (IIN2.2). The conformance tests of the time didn’t check this, so our outstation passed them anyway, and the issue sat. In practice, masters rarely act on those bits, and nothing forced the fix.

The 3.1 rev 1 tests do check it, along with the related case of reading a point type the device has no points of at all. The day dnp3c gained those tests, our CI flagged both. The fix landed the next day.

That’s the point of running conformance tests continuously. New tests can expose bugs that older ones missed, as they did here, and once they’re in CI, a regression shows up as a failed check on the change that caused it, not as a finding in a review months later.

Already in use

We aren’t the only ones. When I mentioned to the maintainers of py1815, an open-source DNP3 outstation written in Python, that we had conformance tooling we were thinking of releasing, they said they’d be thrilled to test against it. They were part of why it happened when it did.

They had dnp3c running in their CI about 14 hours after I sent them the link. By that afternoon they had fixes up for the first non-conformances it found. Two days later they had more fixes up for failures in the 3.1 rev 1 tests. Their reports went the other way too: three issues on how dnp3c read particular steps, each fixed in dnp3c within a day.

LLM catnip

The harness sat untouched from 2020 until this month. Moving it to a new revision of the certification procedure, or polishing it into something we’d release as open source, would each have taken hundreds of hours. We had no plans to commercialize it, and it wouldn’t have made our library noticeably more attractive to customers. For an internal tool that already did its job, the incentive just wasn’t there.

That changed when LLMs got good enough to carry it the last mile. Between October 9 and 15, 2026, with Claude Opus 5.5 writing the code and GPT-6 Astra reviewing it, I:

• moved it from Scala 2.12 to Scala 3 and a modern build;
• implemented revision 3.1 rev 1 of the certification procedure, at all three subset levels, including its new network tests;
• added UDP and serial support, including multi-drop and collision avoidance testing;
• released it as open source, with two releases on Maven Central;
• and switched our DNP3 library’s CI to run it against both revisions.

The code more than doubled, to about 27,000 lines, and the tests grew from under 900 lines to over 9,000. What took at least 300 hours to build the first time more than doubled in a week.

I made the architectural decisions and steered, but the LLM ran on autopilot about 95% of the time. I don’t think that generalizes to every codebase, and there are two reasons it worked so well here. First, the architecture was already solid, so extending it was mostly a matter of following its patterns. Second, the LLM had the full specifications and fast feedback. Every test it wrote could be run immediately against a real outstation and against simulated devices in unit tests, and checked against the text of the certification procedure and the standard. Its changes also got an independent review from a second model, which caught misreadings of the procedure before they landed.

I was also comfortable working this way because the stakes are low. dnp3c is a test tool, meant for the bench and CI, not for anything in production. If it throws an exception, the test run fails and someone looks at why. A wrong result is the real risk: a false failure wastes someone’s time, and a false pass could hide a bug. That’s why every result comes with a log and a pcap anyone can check, and why outside users reading the tests matters so much. Our DNP3 library is different. It runs inside devices on the grid, and changes to it are held to a far higher standard.

A codebase with a solid core, a spec to follow, and a fast way to check the results is LLM catnip. Conformance testing is about as spec-driven as code gets.

Testing the tests

Turning a certification procedure into code also tests it. Code forces you to pick an answer wherever the text is ambiguous.

Implementing 3.1 rev 1 turned up places where it contradicts itself. One is the same IIN handling that caught our outstation: as printed, a device without a given point type is expected to answer a read of one type with an error, and of a closely related type with a plain response that has no error bits. Another is when a device should answer a read that arrives while its startup unsolicited response is still unconfirmed: one test’s steps expect an answer within the usual response time, while the revision’s own rules for unsolicited responses, and the standard, have the device wait.

dnp3c picks one consistent reading for each of these, applied the same way everywhere it comes up, and the README lists every contradiction and how it’s resolved. Because the code and that list are public, anyone can see exactly how the tool reads each test, and argue with it.

dnp3c could also be an open place to prototype the next revision of the certification procedure: implement a draft test, run it against real devices, and find the contradictions before the revision is published, not after. We’d welcome the DNP Technical Committee, which writes and approves the procedure, using it that way. I hope that with LLM assistance, the next revisions of both DNP3 and its certification procedure can be much more consistent.

Licensing and getting involved

There’s a personal reason too. Our DNP3 library is developed in the open, but under a commercial license, and I’ve missed real open source since OpenDNP3: software anyone can use and change, built together with people outside our company. dnp3c is a chance to do that again.

dnp3c is licensed GPL-3.0. You can run it, by hand or in your own CI, without publishing any of your code. If you distribute dnp3c, or a test harness built on it, the GPL requires you to pass on the source with it. That way, anyone who receives a copy of dnp3c also gets the source to change and improve it.

If you maintain a DNP3 library, open source or commercial, build DNP3 devices, or buy them:

• Run it in CI. Catch non-conformances on the commit that introduces them, in the library, before they ship inside a device.
• Use its reports. Share them with customers, or take them to a Conformance Test Review.
• Ask for reports. If you buy DNP3 devices, ask your vendors for conformance results you can read yourself, like dnp3c’s reports and pcaps.
• Open an issue when a test reads the procedure differently than you do. Some of our best fixes so far came from users doing exactly that.

And if you’re on the DNP Technical Committee working on the next revision of the certification procedure, dnp3c is yours to prototype with.

dnp3c is the first of several internal tools we plan to release. More of our DNP3 tooling is on its way, and I’m excited to get it into people’s hands.

We built dnp3c to test our own code. Now it can test everyone’s.