Testing Asynchronous Behaviour in a Chat Server

Ejabberd is an instant messaging server with lots of asynchronous behaviour. Such asynchronous behaviour is awkward to test, because of its inherent non-determinism. Variable event order and timing uncertainties can easily lead test cases to fail when they ought to pass, generating “false positives” that hinder testing. Yet coping with all possible event orderings and timings can make test cases hopelessly complex. When test cases are generated, these problems become even harder.

Users had observed that certain chat messages never arrived at the recipient, but traditional unit testing couldn’t trigger this case. Load testing would indeed show that certain messages got lost, but root cause analysis from a load test was not feasible. Instead, QuickCheck was used to generate test cases, but those then had to tolerate asynchronous behaviour.

Quviq developed a new way of expressing asynchronous properties that avoids problems with false positives, and in a paper we show how we used it to develop a simple QuickCheck specification for ejabberd message delivery.