We develop small to large visualization projects for different tasks and industries and sometimes while rewriting them a couple of times in the process we hit walls because we discover that we need to add a lot of code to support new requirements. Now we have established a design process that seems to work well (at least we reduced the development time for each new project quite a bit), but we’re still left scratching our heads around this question: what exactly should we test when testing visualizations?
- If everything that we want to explore is on the screen (bounded
visualizations)? - If the data is ok – if data is valid (that’s one of the nice things about visualizations you can spot errors in your datasets)?
- Usability?
- User interaction?
- Code quality?
I can tell you for sure that a simple check of the code quality is certainly not enough! Is there a classic paper / book about how to test visualizations? Also do you happen to know about classic design patterns for visualizations (except the obvious ones like Pub-Sub)?
What are you “testing” for? Are you testing for accuracy of the visualization, in other words, whether the visualization matches the system it describes? In this case, consider a View Model: an object structure identical to the visualization. Write code that compares it to either (1) the view model to the system it describes or (2) a known correct visualization for a certain system and repeat for as many different visualizations as required.
If you want to test the visualizations themselves, specifically how they appear on the screen, this is more challenging. You can “screen scrape” (compare the bitmap representation) but this could be foiled by changes in rendering engines between OS or library upgrades among a number of things. Depending on the graphics library and operating system used, a GUI testing library may be available to help. See http://en.wikipedia.org/wiki/List_of_GUI_testing_tools for a list.
Are you testing whether the visualization is at the appropriate level of abstraction or whether it makes sense to customers? This is not really “testing”. This is more UI or visualization design and is a much bigger topic.
5
You could read through the excellent treatise The Grammar of Graphics, which elaborates on different properties of visualizations and how they convey or obstruct the flow of information to the observer. Although the book does not list a test plan for visualizations, it certainly brings to light to connection between code, model, media and perception. For me it was a most inspiring read.
In my opinion, visualizations are neither inherently right nor wrong, but there are certainly best practices. Eg., having an understanding of the perception of color will help you find good ordinal color scales.
1