Design preferences and cognitive styles: experimentation by automated website synthesis

The manner of external representation (or presentation) could affect our way of working with the internal representation (mentally) and our understanding of the information [1], e.g. in cockpit information displays for aviation [2], but few results on graphical external representation can be generalised [3].

One problem affecting all websites is that there is no reliable, general and abstract method for predicting the effect of presentation rhetorics and modality on understanding of the information. To improve knowledge communication, we should investigate how sensitive people might be to differences in the way we construct our websites. It would be useful to conduct experiments quickly to compare different models of interpretation of information of a specific domain in particular cases. There may be certain styles of presentation or navigation that are generally demanded by users and can either hinder or support users' ability to interpret the information.

Tabular and graphical representations are common in constructing visual arguments [4] and presenting relational data (especially quantitative data) [5]. Visualisation of aviation accident events generally use causal trees to represent the causal relations but there are few empirical studies on both preference and perception of causality visualisation. Specifically, we investigate users' preferences for information visualisation styles and their perception of causality as required by aviation accident reporting. As the Web is one of the main channels for publishing information of aviation accidents, it is desirable to know about how people would prefer the causal relations in accident events to be presented in a website and how they perceive this causality. The user preference data are useful in the design and re-design of websites. To elicit preferences from people, we provide multiple designs for selection and study the rationale of their design decisions. Automated website synthesis saves time and effort in building websites for such designs. Few models and theories are available to address computational website design. However, if we view websites as a form of information visualisation, we can borrow some findings from automated diagram design [6] to serve as our experiment hypotheses. Some systems for automated diagram design have incorporated text to enhance user understanding of graphical visualisation [7]. This kind of multimodal visualisation should be applicable to website design. Expressiveness and effectiveness of graphical languages as proposed by Mackinlay [8] have been influential to diagram visualisation models, including source information characteristics [9], user-defined task specification [10], and user-defined layout preferences [7] for automated diagram design.

In general, we would like to see if people would prefer different representations to display the same information. In particular, this study aims to elicit preferences of designers (and users) about visualisation patterns, particularly the preferences for tables and trees in visualising causality information. In this case, we select trees and tables as the options for selection by users. Tree representations are commonly used to graphically represent causality in printed documents. The causal relations are normally represented by arrows or lines connecting causes and effects.

If people do prefer a representation, it would be interesting to see what rationale or criteria contribute to their preferences. We categorised common rationale/criteria mentioned in website design textbooks [11, 12]:

As these seven rationales may not cover all possible rationales that are crucial to any particular preference, the experiment participants were asked (unprompted) for their rationale before seeing these seven rationales and then they were asked (prompted) to identify if any of these rationales were similar to their own rationales. They were also asked if any of their rationales was not covered by these seven rationales.

Software designs should reduce users' cognitive load [13]. We hypothesise that participants prefer one design to other designs partially because the preferred design suits their cognitive abilities. If this is true, the cognitive abilities of the participants should be related to their preferences. The relationships among preferences of trees or tables, the cognitive test results of the participants, and rationale for their preferences were studied in this experiment. As it is impossible to test numerous cognitive factors in a single experiment, the participants were only tested on cognitive styles/abilities of visualisation and analogy-making, which we guessed were related to visual representations.

The main objectives of this experiment are as follows:

This study used website synthesis to construct an experimental apparatus for Lab-on-the-Web. Using this approach we found a significant relationship among the participants' preferences, rationale, and cognitive ability test results. Most (41 out of 64) participants claimed themselves to be tree preferrers. The other participants (23 out of 64) said they preferred tables. Both table preferrers and tree preferrers found their preferred representations (tables or trees) more informative, without significant difference in rankings of this rationale. As the task given to the participants is to represent information, it is not surprising that the rationale "more informative" was one of the most important rationale. In further studies, it would be interesting to test whether the participants find the same rationale justifiable for their preferences when they are given different goals or under different conditions. This might give us more insights into how different preferrers perceive information.

To table preferrers, the rationale "easier to learn" was more important than "more informative". The rationale "easier to learn" was ranked as the most important by 13 out of 23 table preferrers, but not so important by tree preferrers. Tree preferrers ranked the rationale "more suggestive" significantly higher than the table preferrers did. To table preferrers, tables seemed to be easier to learn than trees. To tree preferrers, trees were more suggestive than tables. These discrepancies in rationale ranking indicate a perception difference between different preferrers in perceiving the given tables and trees.

Tree preferrers performed better than table preferrers in some tests of cognitive factors, particularly the paper folding visualisation test and visual analogy test. There was no significant difference between table preferrers and tree preferrers in their performance in other cognitive tests including the surface development visualisation test and the verbal analogy test. It is plausible that the interpretation of tree representations requires specific cognitive capabilities such as visualisation and visual analogy-making; thus, those who do not feel comfortable with these tasks would prefer tables and highly rank the rationale "easier to learn" for their table preference.

Among all participants, there is a statistically significant low-to-moderate rank correlation between the rankings of the rationale "easier to learn" and the results of visual analogy test and paper folding visualisation test. The low-to-moderate rank correlation indicates that the participants who performed better in such two cognitive tests ranked the rationale "easier to learn" to be less important. At similar correlation strength, the participants who performed better in visual analogy test ranked the rationale "more suggestive" to be more important. It seems that the visualisation and visual analogy-making abilities of the participants might play a role in their preferences for tables and trees. Possibly (although we cannot prove this), table preferrers lack sufficient cognitive capability to interpret graphical representations like trees; thus, they prefer tables as they are easier to learn. Tree preferrers would feel more comfortable in making visual analogy and find graphical representations like trees suggestive. As indicated by the low to moderate strengths of correlation in these results, it is probable that other factors (and other cognitive factors) may be also relevant to the participants' preferences. Further studies are required to delineate these relationships.

This experiment also demonstrated the technological significance of computational synthesis in enabling scientific experiments. This experiment re-used a website synthesiser to generate websites on the fly so that the designers/users can explore the design space. Without using a website synthesiser, this experiment would not be possible. Other than computational synthesis, the most relevant tool is Web content management system but it does not meet the requirement of this experiment. Apart from its high cost, there is no Web content management system so flexible as our our website synthesiser in accepting information mappings. This makes Web content management systems inapplicable to our experiment. Thus, computational synthesis is the only available solution for this experiment although computational synthesis following definite patterns would limit the variability of graphical presentation. To enrich the variability and functionality of graphical presentation, we would consider using web design patterns available in Web 2.0 and HTML5 to enhance further studies.

The experiment reported in this article found significant relationships between the participants' design preference, rationale, and cognitive test performance. This work also exemplifies the use of a website synthesiser as an essential instrument enabling the participants to explore different possible designs, which were generated on the fly, before they selected their preferred designs. In the tested sample, more people prefer graphical to tabular presentation. Despite the high preference for graphical presentation, the given tabular presentation was generally rated to be easier than graphical presentation to interpret, especially for those who score below average in the visualisation and analogy-making tests. This piece of evidence helps generate a hypothesis that design preferences are related to specific cognitive abilities. Without the use of computational synthesis, the experiment setup and scientific results would be impractical to obtain.