At this stage, my Script class includes about 35 statement forms. A lot of this is a little redundant. The aim was to cover many significant cases related to decision, action and interaction.
A test suite
Today I am writing a test suite for the Script class and the underlying APIs. This is necessary because testing production graphs is, as we’ve seen before, expensive.
I won’t get into unit testing using XCode/Cocoa. Instead, I’ll just write my own tests from scratch – this is because I need to invest a little time and effort into making scripts easier to test.
This will be a fairly large test suite (for me, because i’m not terribly fond of testing), however, all cases are very simple and I can take advantage of some design features for the behavior API.
- Given model/view separation, I need only setup the model, not the view.
- There is no need to setup timers to test behavior, we can just iterate the action graph as far as we need and check the data model to test the output.
Writing the tests
Macros came in really handy to keep the tests concise, but also to avoid having to create ad-hoc data structures. Each test is about 10 lines long and I have 34 tests. Here’s a sample:
// this defines a function and stores the test name
TEST(test_observe_actor,"test observe actor")
// setup a stage and actors, with B 100 units away from A
SETUP_A_B(100);
// the actual script to test
[s type:ACTOR_A];
[s observe:ACTOR_B];
[s xType];
// bind the script to the stage and iterate the system once.
BIND; STEP(1);
// some code to check the output
Vector* u=[[Vector alloc]initWithX:1 y:0 z:0];
Vector* v=a.orientation;
// this checks the specified condition and prints the result
CHECK([Vector angle:u with:v]<0.01);
// just a macro defining a closing bracket, because a dangling
// bracket would be odd.
END_TEST
Granted this is not a very clean use of macros, and won’t scale up to a really large test suite – but we’re not into ‘test-first’ development here – just running a few tests.
I hacked a function firing up the tests into the main.m file associated with my XCode project. Another time I’ll look into a nicer way of doing all that stuff. (need to move the framework to a library project and create a separate project for the test suite).
Pending, sample output looks like this:
2009-11-16 22:14:55.837 Generic2DBoardGame[5191:207] X test observe actor
(X tells us that the test failed :( )
Debugging
Unsurprisingly (given the way I drafted my code), a lot of tests are failing. I even had to quarantine 6 tests that seem to get into infinite loops. Since I have many tests, this is a good time to add logging. The first logs I want to produce are snapshots of the system showing the state of each entity. I will only display logs for failed tests.
After debugging a couple of test cases, I find that primarily, diagnosis falls into three broad categories:
- An event isn’t generated. Because events often occur given conditions related to an actor, I print the condition leading to such actor being rejected by TargetSelector
- An action isn’t being applied. I don’t expect to see this too often.
- An action isn’t effective.
These broad categories allow me to channel log output and enable/disable channels to conclude the diagnosis.
That’s about it. I’ve got 11 pass cases so I’m about 1/3 in and should be done tomorrow.