Starting with a naive MVC application my goal was to provide automated testing over headless graphics (e.g. using a standard test suite). How can we modify an application so that it can be used either by a real world user or by a robot?

I started with a concept (‘the user agent’). This is somewhat useful and helped clarify the application structure. However, it seems a better way to get this to work is may be to just write a couple of tests ‘assuming the underlying system’ is already compatible with our new requirements – in other words, design the solution outside-in.

Additionally, I’m looking into existing automations for iOS; I will shortly review a couple of attractive solutions.

The user agent

A so called user agent (I may be abusing the term) is an interface bridging the view and the controller. Why would it be useful to complicate our design in this way?

Initially, an MVC application works like this:

USER <=> {View} <=> {Controller} <=> { Model } (2)

With a testing automation, it may look like this:

{ Robot } <=> {View} <=> {Controller} <=> { Model }

Indeed certain automations work this way (great); what about testing over headless graphics?

  • The view component must not be instantiated.
  • We cannot instantiate the controller unless we hide/factor out view dependencies.

We could write tests that only exercise the model; not bad, maybe too far removed from our goal.

So the idea is to introduce an interface (‘user agent’) bridging the view and controller; then we could setup the application in either of two ways:

USER <=> View <=> { UA } <=> {Controller} <=> { Model }

{ Robot } <=> { UA } <=> {Controller} <=> { Model }

A quick refactoring

Clearly dependencies between the view and controller need to be factored out in order to realize the above. Initially the controller owns a reference to the view and may processes UI events.

One way to proceed:

  1. Migrate event handling code to the UI; inside event handling code we invoke controller methods.
  2. Provide interfaces (e.g. IUser or IUserInput, IUserOutput) that the controller uses to output to, or configure input from, the UI. The UI implements these interfaces.
Then the situation is like this:
View (Event Handlers) => Controller
Controller => View <IUser>
I have done exactly that with our application code. This refactoring is not overly time consuming.
Formally, the approach does not seem incorrect. We can write a Robot class implementing <IUser> and Robot can directly invoke controller methods. This approach, however, should be validated by answering the following questions:
Can we easily write acceptance tests?
Can we generate tests by recording sessions?
Session recording/playback
It appears that recording messages issued by the UI may not be very easy with the above solution. The idea is simple. In practice, however, the controller is often defined as an aggregation (a hierarchy of controllers):
  • RootController
    • ControllerA
    • ControllerB
Now, we can pass a Recorder to all controllers down the hierarchy and certain methods in each controller would generate messages sent to the recorder. Having to record from (and playback to) a graph instead of a single object is a complication that should be avoided. Further (contingent to the above refactoring) it is not clear which invocations should be recorded (cf. “certain methods”).
In short, it would be reasonable to introduce an interface that captures messages from the view to the controller:
View (Event Handlers) => Controller <IApplication>
Controller => View <IUser>
Writing acceptance tests
For the programmer writing acceptance tests, the controller hierarchy appears untidy. Although controller classes look ‘higher level’ once event handling code has been factored out, the methods are still designed to be called by the UI in response to user events. This may involve continuous input as well as fairly idiosyncratic sequences. As a result, introducing <IApplication> as suggested above may not be sufficient. Compared to interaction records, manually written tests should be fairly ‘stylized’.
One solution may be to write a Robot class used to capture requirements for writing acceptance tests.
Where is the UA?
A quick refactoring is insufficient. We need more kit to get this to work. The UA would then be realized by several interfaces:
  • <IUser> specifies methods invoked by the controller in order to display output or configure user input
  • <IApplication> specifies methods normally invoked by the UI in order to operate the application. Invocations of IApplication may be recorded for later playback.
  • A Robot class may define high level methods used to manually write tests.

To be continued…

My impression at this point is that I may have jumped in too quickly – starting to design based on a ‘good idea’ instead of formalizing use cases.

The UA idea is ‘kind of good’. But it took time to move from the idea to an understanding of what the UA might actually be.

Actually, the most productive step was not thinking about the UI but defining a new initializer for the session, like thus:

Session initWithContext:(id<SessionContext>)context

Where:

  1. Session represents a user session; roughly matches the model/controller.
  2. SessionContext is intended to capture anything that a user session may require to boot and run, with or without headless graphics.
    • A live session context would provide a UI.
    • A testing context would not provide a UI.
  3. Session can be instantiated in a headless graphics environment.

If a session can be defined in this way, we have done half of the work.

(tbc)