Last summer I invested a huge amount of free time to implement a light-weight OIDC Provider in Rust. Being feature-complete was my first focus which made me neglect a lot of quality aspects I usually appreciate. This left my summer break with a very functional, yet barely tested program. Here I want to describe, how I achieved to cover this program under a rather complete end-to-end test suite.

Where did the tests come from?

OpenID Connect is an open standard which also allows ones software to be certified to adhere to the OIDC Standard. A big part of the certification is a conformance test suite which must be manually exercised and covers almost all parts of the OIDC specification. The tests must be done manually as there are holes in the specification (e.g. how does authentication actually take place?) which a test driver cannot generically walk through.

So all I had to do was to formulate all these test cases in some testing framework by myself and also to automate the manual parts which I know how they work for my concrete OIDC implementation.

What is there to test in OIDC?

The protocol interacts between several entities where the center is occupied by the provider, in my case tiny-auth. An example interaction is depicted in the sequence diagram below.

To automatically test the OIDC Provider I made use of several well-known programs and libraries:

• Browser: To automate browsing the visible parts of the protocol (most notably password entry and scope consent) I use the well-known Selenium Browser engine.

• User: The test driver emulates the user by providing names and passwords from the test cases via the browser.

• Client: Instead of implementing an OIDC client directly I use a combination of several technologies. If the client sends the user to some page this is done with Selenium. Once the Client directly interacts with the provider I use RestAssured to interact and verify with HTTP. And if there is some redirect from the provider to sense I redirect to a mock-server instance which easily allows both obtaining the original request as well as passing back minimal HTTP so the provider is satisified.

All this takes place inside a JUnit 5 Jupiter test engine.

One notable library I used was nimbus-jose-jwt. As OIDC is all about passing JWTs around, the library helped in verifying and extracting the tokens from the provider.

These tools in combination allowed my to mock and sense all external systems the OIDC provider interacts with. Notably all the tests run with mTLS (a.k.a. TLS with client certificates) enabled so a productive setup is simulated.

The whole test suite consists of 183 active tests (and some more which test unimplemented features) and takes about 6 minutes to run. Yes, nobody said, these fat tests are actually fast. About 2 minutes of the test are spent idling as the test oidcc-basic-certification-test-plan.oidcc-codereuse-30seconds requires the test to wait said amount of time to verify the Authorization Code really expires. Since this test occurs in several scenarios, the four executions add up quite some time. The other tests take about 2 seconds each.

Awesome JUnit 5

Using the latest JUnit version was an experiment which turned out to be very enjoyable. Both the unit under test tiny-auth as well as the mocks around it require a lot of setup code to get up and running. JUnit offeres implementing callbacks to run code before the tests are actually run (see BeforeAllCallback) and CloseableResources to be registered so the server can cleanly shut down after test execution.

It even comes with a simple dependency injection system which allows tests to declare a dependency as method parameter and JUnit will type-match the parameter to in instance of my TypeBasedParameterResolver<T> which will be called as needed.

Redundancy is evil

When looking at the different test cases, a lot of repetitive parts are immediately visible. If for example different test cases cover different aspects of the Token Endpoint, the first part of authorisation is always the same. Moreover, the different OIDC flows all end up with the client holding an Identity Token which has to pass some test criteria on its own. These have to be verified independently from the flow they were obtained with. So the obtaining strategy can be separated from the token's test criteria.

This is where, once about half the test cases were written, I realised I would drown beneath the waves of code duplication if nothing was done about it. Luckily Java has some very nice features at hand which eased elimination of the existing redundancy: Since Java 11 interfaces are allowed to carry method implementations. This feature, in combination with JUnit 5 supporting test discovery in super interfaces (not only super classes), allowed me to extract all test cases into interfaces for different scenarios. And if a scenario is applicable to a certain test suite, it suffices to implement the interface in the test suite and all the tests of that scenario are added.

For example the OIDC Refresh Token is only issued to confidential clients. So a test suite using a public client (as the suite for the Implicit Flow does) cannot exercise the test scenarios involving a refresh token. On the other hand all the different Hybrid Flows use the confidential client and can easily share the common test cases by inheritance. This leads to the funny situation that none of the actual test classes (like CodeTokenIdTokenAuthenticationTest) contain any test methods by themselves. The test cases are all split into the interfaces the test classes implement.

Where Are We Heading?

The first implementation of tiny-auth is tightly coupled to the actix-web framework. Now that there is a end-to-end test suite in place, my next plans are to exercise some TDD and rewrite it to a second version which more easily allows new features like OTP authentication, a database backend, and much more. My list of ideas is long! And thanks to the vast set of tests I can easily go about rewriting without losing any features or adding bugs.