Report: From Abstract Data Mapping to 3D Photorealism

1. Background
The meaning of the term ‘visualisation’ varies widely between different disciplines according to the specific visualisation practices and techniques employed, and the research questions traditionally posed by that discipline. Interoperability and strategic approaches to tools development can be limited by research culture and focus. In addition, in both Science and Engineering and Arts and Humanities, visualisation can vary from multidimensional abstract datasets (including text visualisation and sensor data) to three-dimensional virtual reconstruction of natural and built environments. Providing introductions to, and overviews of, different areas of visualisation to a cross-domain audience is therefore quite a challenging task.

1.1 The challenge of terminology: development of a working index
Terminology is a significant barrier in any cross-domain, or multidisciplinary, interdisciplinary or transdisciplinary¹ activity, and this event was no exception. During the preparation of this report, it became necessary to develop a working index of terms. This working index is composed of terms used in presenters’ papers or from their slides and transcripts, in the cases where there was no paper submitted.

The index of more than 5000 terms enables partial identification of shared or differing terminology among presenters. It has been prepared to facilitate backwards analysis of the event and to identify potential overlaps between presenters’ conclusions and other presenters’ domains.

2. Format and methodology
For the workshop, From Abstract Data Mapping to 3D Photorealism: Understanding Emerging Intersections in Visualisation Practices and Techniques², visualisation experts in both the Sciences and Engineering and the Arts and Humanities were asked to prepare one large slide containing diagrams and visual materials appropriate to an orientation in their area of visualisation. In the morning, they were given five to seven minutes to present, somewhat akin to an exercise in speed diagramming. Cross-domain groups met in the afternoon.

While it is impossible by definition to cover all the possibilities in different visualisation areas in a short space of time, annotated diagrammed examples can provide an excellent starting point for cross-domain activity. By juxtaposing diagrams authored from the viewpoints of different domains, identification of where the same term is used differently or where different terms are traditionally employed in different domains for the same object or process can be accelerated. This visual format is also designed to challenge researchers to extend their own research questions both into the language of other domains, and to envisage ways of appropriating the practices and techniques of other domains within their own research culture and focus, thus establishing a genuine cross-domain dialogue. With transcripts of their presentations provided, experts authored short papers based on the structure and flow of their slide, on its delivery and on the discussion arising in the cross-domain groups.

2.1 Cross-domain Diagramming
The diagram below shows the overall experience of the event, principally from a presenter’s perspective: they prepare a single slide, optionally using the spatial layout of the slide to communicate aspects of the structure or content of the area. They present their slide. They view the presentations and listen to the terminologies of other presenters. They participate in two different group sessions with different combinations of 4-8 other participants. After the event, they receive a transcript of their presentation. They prepare a cross-domain paper of minimum 1000 words (with no maximum limit). This can be quite a challenging task given the slide is an overview of an extensive area of visualisation.

Click on image for full resolution.

In the diagram above, a working index of terms is introduced. As the red arrows indicate, this working index is composed of terms used in presenters’ papers, or from their slides and transcripts, in the cases where there was no paper submitted. The index (of ~5500 terms) enables (partial) identification of shared or differing terminology among presenters. It has been prepared to facilitate backwards analysis of the event and to identify potential overlaps between presenters’ conclusions and other presenters’ domains.

3. Instructions for diagramming: global and local information

Spatial configuration was presented as an opportunity to present structural information about an area of visualisation and to juxtapose diverse presentations of the same topic, for example from Science and Engineering and the Arts and Humanities.

3.1 Large single overview slide
Slides were constructed from visual material contained in Ken Brodlie’s vizNET 2007 presentations, A Rough Guide to Data Presentation Parts I & II, (Brodlie 2007³). Overall structure and collections of examples associated with different types of data visualisation could be viewed by fitting the slides to the screen. Details of the slides could be viewed by zooming in.

This process was a mock up in PowerPoint of a tool for browsing between local and global views, the suggested format for the diagrammatic presentations. While PowerPoint was at times slow to respond, an environment under development, Seadragon⁴, in which one can locally or globally interact with vast amounts of visual data, may soon offer the option to present visualisation research and supporting orientation materials in such a manner.

Slide 1 (below) was sent to all participants of the workshop prior to the event as an optional template for a diagrammatic presentation on the day. In addition, participants who were not presenting were invited to compile their own slide for presentation during the event. Meurig Beynon and Stephen Boyd Davis accepted this invitation and their contributions (see 4.3 and 4.7 below) proved to be of significant value to the workshop aims and objectives.

Data and Scientific Visualization Part I
(Julie Tolmie for Ken Brodlie) : Slide 1 employs a diagonal spatial structure starting from what is data visualisation, through visual design principles and simple graphs in the top left half of the slide, to scientific visualisation of 2D data using isolines, 3D data using isosurfaces and 3D volume rendering using colour and opacity in the bottom right half of the slide.

Data and Scientific Visualization Part II
(Julie Tolmie for Ken Brodlie): Slide 2 employs a left-right spatial narrative of three rows of diagram blocks starting from exploratory data visualisation (finding relationships in tables of information and multivariate datasets) through visualising structures of information or information hierarchies, to interacting with a visualisation including the principles of focus and context, and visualising time. (back )

3.2 Animated embedded details respecting visual and iconic structure of map
Tolmie demonstrated an annotated map diagram⁵ whose elements linked to a further twenty animated diagrams, and whose visual structure detailed the construction of visual objects made from mathematical primitives, for example a binary tree made circular and embedded in a torus, as shown.

This annotated map diagram employed spatial configuration to communicate structure or process using iconic visual elements such as the recognisable (orange) binary tree. Animated details were available as links enabling the viewer to toggle between the global and local view.

4. The slides: juxtaposition of notes on spatial structure with presenters’ conclusions/terminology
There were sixteen presentations in all, of which the majority chose to utilise the spatial structure of the slides in some way, with one or two presenters converting their sequence of PowerPoint slides to a single slide during the event. Three presenters directly addressed the topic of visualisation and reality/realism, two of which used video and VRML, respectively, rather than slides. Two further traditional sequences of slides in PowerPoint addressed the opportunities for cross-domain collaboration within the visualisation communities.

The visual structure of each slide or presentation form is outlined below. In numerous cases the slide’s structure is referenced and continued in the papers. The majority of presenters employed a left-right evolution or juxtaposition across the slide or a quadrant/centre structure. Other forms employed included circular diagrams or paths, linked thumbnails, and loosely grouped related images. An extract from each presenter’s conclusion is juxtaposed below with the slide thumbnail. Where conclusions are not given, or where they are specific to a lengthy prior discussion not related to the slide, a related extract that can stand alone or that refers to the slide is chosen. Cross-domain terms are bolded.

4.1 Visualisation Flows and Outcomes

Martin Turner
The slide employs a left –right spatial structure to juxtapose two abstract dataflow reference models used within scientific and information visualisation: the Haber and McNabb Dataflow Reference Model for Scientific Visualization and Spence’s Navigation Framework for Information Visualization. The latter developed as computers evolved and emphasis shifted from data flow to playing interactively with data. Flow visualisation images included on the slide represent different choices for the same data set.

[…] It is proposed for further discussion that this separation of the data flow stages allows repeatability and extends itself to many other non-scientific themes.

Visualization Flow and Outcomes⁶

4.2 3D Modelling in the Arts and Humanities

Carl Smith
The slide employs a left-right spatial narrative of three rows mapping the distinct phases of visualisation from the conceptual stage through capturing the source data, base level 2D drawings, 3D modelling operations, construction and positioning of multiple objects, validation and testing, addition of elements to augment senses of scale and immersion, attaching the units and methods of construction to aid intellectual transparency, and presenting the same information in multiple ways and multiple media for flexibility of use and enquiry.

The fundamental question arises how are these visualisation methods and techniques relevant in areas other than the humanities? This project encapsulates a number of different types of visualisation and interestingly the distinction between arts, humanities and science made in the group session does not seem to apply here – the visualisation methods are used for both aesthetic reasons as well as for representing information and testing hypotheses. Also the requirement of the dynamic creation of visualisations by the user resulted in an implicit combining of process and product orientated modelling which is common to applications in art, science and the humanities.

User Generated Content Mapping: The Evolution of Form in the Cistercians in Yorkshire Project⁷

4.3 Visualisation using Empirical Modelling principles and tools

Meurig Beynon
The slide employs a quadrant structure showcasing the way in which Empirical Modelling (EM) principles and tools have been deployed for visualisation. In the top left, visualising affect, an example of modelling a Schubert song. Top right, visualisation in support of cognition for a single user including cognitive processes involved in playing noughts and crosses; bottom right, a multiple user case, a reconstruction of multiple actors’ views of a railway accident. Bottom left, mathematical visualisation, including a colour based sudoku modelling environment.

[In the group session] Denard placed Empirical Modelling in the intersection between Science and the Humanities […] EM is deemed to involve making a model, and thereby having some referent in mind. It is apparent that such a model can take a precise mathematical form in relation to a scientific application where there is an underlying theory, but it can also fulfill the role of modeling in humanities computing as identified by Willard McCarty⁸, where the interpretation is always evolving, and reflects the tension between objective rationalization and subjective intuition.
[…] EM has an even more ambitious objective: that of supporting a holistic approach that can draw upon the epistemological stances of [arts, humanities and science].

Visualisation using Empirical Modelling principles and tools⁹

4.4 Visualisation, fidelity and the serious game

Sara de Freitas
The visual presentation centred on two video clips from serious games in development. The first was a code-driven scene of the death of a person who had sustained a head injury. Medical scanners were used to scan the human body, and medical experts were used to verify the realism of the actual sequence. The second clip showed an emergency scene where an explosion had occurred in a busy area. The player is required to sort through the casualties and to decide on the order of urgency of their medical care.

The match between understanding human processes and our uses of physical, abstract and virtual space provides excellent scope for developing bridges between different technologies, but merits a more participatory approach to design based upon learner profiling and modeling. […]If we are to develop useful technologies it will also be important to integrate existing functionality.[]the closer we can match the human processes and activities the more effective our systems will ultimately become.

Building bridges between computer modelling and simulation and games and virtual worlds communities¹⁰

4.5 Game Engines

Nijad Al-Najdawi
The slide employs an overall left-right timeline structure to present the evolution of Game Engines from planar worlds through the rise of the 3D, enhanced 3D, towards photorealism with a view to the future. Images above the timeline illustrate graphics capabilities enhancements, for example for the Doom Engine and the Unreal Engine. Images below include game hardware, core components of game engines, collision detection and basic shapes for 3D modelling.

[…] from the user point of view game engines have been widely accepted to replace the VR-engines when designing VR presentations and environment for the reason that they are have more visual effects and an easier workflow than the VR-engines, moreover the game engines costs are generally much lower than VR-engines, the quality and usability of game engines exceeds that of the best Virtual Reality tools. Finally the rapid development of game engines exceeds the virtual reality development.

Introduction to Visualization using Game Engines¹¹

4.6 Visualising and Reality

Gregory Sporton
The presentation of a VRML reconstruction of the house of Constantine Melnikov shown in juxtaposition to photographs taken on a visit to (the outside of) the house revealed what could and what could not be visualised in each case. Relationships between developing a model of reality and the reality of a model were discussed.

[…] what we really have is the reality of the model and the model making process in front of us. The visual arts, with its rich experience of understanding the impressions made by visuality, has much to offer the complex practices of 'visualisation', in terms of helping to apply a critical eye to the representations of the world around us as they manifest in the powerful and persuasive technologies of visualisation.

Visualising and Reality¹²

4.7 A word about the weather: depiction and visualisation

Stephen Boyd Davis
The slide employs a left-right spatial narrative of two rows of juxtaposed images. In the top left, a virtual environment military training image intended to be highly realistic, alongside an image referencing Descartes and the ‘perfect picture’, alongside a photograph in the context of its own subject. In the top right, images relating to geometry and perspective, one in particular with the curved perspective of Van Gogh. In the bottom left, images illustrating drawing techniques that may have no relation to realism. In the bottom right, tonal rendering vs. technical illustration is shown.

In many cases the informational benefits of a photorealist depiction will be negligible, and such pictures may bring more problems than they solve, yet the hunger for realism (however contentious its definition) will often mean that, for reasons which have nothing to do with information, we choose to use such images […] Depiction should be subject to the same decisive, goal-oriented considerations as any visualisation: it is all a matter of design.

A word about the weather: depiction and visualisation¹³

4.8 3D Modelling, 3D Documentation

Drew Baker
The slide contains a circular text-based diagram with visualisation and its interpretation in the centre, surrounded by its sources of data: maps & plans, anecdotal evidence, source texts, critical texts, survey data, data capture, artefacts, visual resources, site photos. A right hand panel includes selection rationale, alternatives, cognitive modelling, rendering, embedded data set, and feedback framework.
A second slide juxtaposes two diagrams related to a Data Information Knowledge Wisdom (DIKW) Model, top left and bottom right, with a diagram outlining the concept of Paradata at its centre.

[…] it will provide scholars with a new way of debating and using data artifacts within visualisations and will reduce the use of computer graphics for their own sake that may be misleading in the interpretation. By understanding and marking the journey the researcher takes, the evidence which is used and discarded, the paths that are taken and where they lead to in the construction of the research narrative, the visualisation that is created becomes more than the pretty picture: picture: it becomes a research artefact in its own right that provides the stimulus for debate and rhetoric, and it becomes a living growing data corpus.

Towards Transparency in Visualisation Based Research¹⁴

4.9 Humanities, e-Science and Visualisation

Stuart Dunn
A sequence of PowerPoint slides was presented with a view to collaboration with Science and Engineering visualisation communities. Issues and research questions in the emerging area of visualisation and federated distributed data in the Humanities were outlined. Conveying information in ways that two different communities can recognise was identified as critical for visualisations across domains.

Humanities e-science is – or at least should be - acting as an agent driving the evolution of the concept of the image for these disciplines. Extensive future interdisciplinary collaboration between humanities e-science ‘early adopters’ and professionals in all branches of the existing field of visualisation will be essential if the great possibilities inherent in this are to be realized.

Humanities, e-Science and Visualisation¹⁵

4.10 Visualization and the Grid

Nick Avis
The slide employs a quadrant structure with a central section illustrating the four quadrants as infrastructure in the support of problem solving environments. In the top left, ‘what is meant by the grid’ and the UK’s worldwide standing in this area. In the top right, grid visualisation and its potential uses. In the bottom left, research activities in grid based visualisation. In the bottom right, bringing together these components together to form a National Visualization Service.

Future visualization systems will have to both offer a means of seamlessly identifying, harnessing and orchestrating a variety of different resources, be it data, computational devices, scientific instruments and humans and provide means for allowing non-visualization experts to explore their data and make compelling presentation visualizations.

Visualization and the Grid¹⁶

4.11 Mathematical Visualisation in the Arts

Julie Tolmie
The top slide employs a horizontal split in time with historical and recent past examples (including western musical notation and early abstract film) above and current examples below. Of the current examples, Patch-based visual programming environments are located on the left, text-based environments on the right with the more mathematical examples of each located centrally.

The bottom slide employs left-right spatial juxtaposition of acquired data driven works vs. simulation. Its top row showcases real-time performance environments while its bottom row addresses appropriation of features of mappings or environments as compositional elements. An audio rhythm creature created in Pd was demonstrated.

While mapping a system as a predictive model is clearly different in terms of intention and rigour, investigating behaviours by playing with aspects of mathematical structures, data, representations, or environments as compositional elements, may not be so different in terms of process. […] This common ground offers strong potential for collaboration between the arts and the sciences and engineering. A possible constraint, or barrier for re-negotiation, lies in the arts practice of not generally making such mappings available to an audience¹⁷; focussing on the end result rather than the process. […] The relationship between (some) computer-based abstraction in the arts, and information visualisation and scientific visualisation in the sciences and engineering, […] could be remapped and documented in a rigorous way. We would, however, need to re-negotiate the language.

Mathematical Visualisation in the Visual, Sonic and Performing Arts¹⁸

4.12 Real-time Data Acquisition

Jonathan Green
Slides incorporate grouped overlayed images, for example various VRU projects that have involved real-time data acquisition processes, and various sensors used as input nodes. Also included is a text-based diagram showing integration of Web 2.0 into CODA.

We have found that most things have already been done. Therefore, we have seen it as our task not to reprogramme what already exists, but to try to find innovate uses for software and Internet services that already exist. We are, after all, artists and not programmers. Candidates that the VRU are currently working to exploit include Google Images and Google Video, YouTube, PhotoBucket, NASA images, Internet radio stations, atomic clocks, celestial movements and Digg.

Real-time Data Acquisition¹⁹

4.13 3D Visualisation within Internet Art: Current Technologies and Practice

Mike Magruder
The presentation employed a web-based interface linked to artworks selected to demonstrate the creative potentials of 2D and 3D web-based visualisation technologies. Each composition showcased a particular mainstream software platform, summarising its aesthetic and conceptual potentials within the artistic process: the 2D web technologies, Flash and Processing; the 3D web technologies, Shockwave, Processing (3D) and VRML; the 3D multiuser environment Second Life.

The Web 2.0 paradigm refers to the evolution of the Internet into a platform that enables a user and provider to exchange content via an application-based environment. […] Web 2.0 production […] is characterised by the transition of standard web functionality to complex systems of exchange that embody traditional software facilities and services [for example] wikis, online media-sharing repositories and social networking sites. […] artists are only now beginning to implement these frameworks in their creative productions. [Extract: no conclusion given]

Visualisation within Internet Art: A Sampling of Current Technologies and Practice²⁰

4.14 Website Accessibility and Usability

Nijad Al-Najdawi
The slide contains grouped screenshots. In the top left are websites that follow usability and accessibility guidelines, and websites that do not. Along the bottom left, five screenshots show webpages with exactly the same content, but which employ using cascading stylesheets (CSS). To the right, are screenshots of websites containing links to usability and accessibility guidelines.

It has shown in a limited sense that by making a site accessible to users with specific disabilities it almost inevitably becomes more usable to people without such disabilities.

Web-based visualization - Accessibility and Usability²¹

4.15 3D Modelling and Display Technologies

Roy Kalawsky
The slide consists of six sections linked in a spiral path.: the creation of 3D entities showing the phases, design/definition, 3D modelling and the database for the 3D entities; text boxes outlining the application computing, culling and drawing; a diagram showing the runtime environment or scene graph system; the presentation then moves to display technologies organised as whole body devices, visual display, haptic display, and auditory display; visual displays are then detailed in the sub-categories projection, direct view, and virtual; finally photographs showing use of an immersive desk, a vision dome, a force reflective system and an auditory display are shown.

(Paper not available)

4.16 Visualization using Commodity Clusters

Andrew Kaye
A sequence of PowerPoint slides explained distributed visualisation using commodity clusters and different layers in digital curation. In particular, computational steering, where one can interact with the models running on the large clusters, and the possibility of interfacing one’s top layer of software down to their systems were discussed with a view to collaboration in visualisation communities.

(Paper not available)

5. Post-diagramming discussion: objectives and issues

A combined diagramming session was scheduled to take place after the presentations. Instead the time was used to discuss the objectives, both immediate and longer term, of the diagrammatic orientation materials and the collaborative authoring process.

5.1 Objectives
In the context of the completed presentations, three objectives were outlined:
• Overall objective: create draft orientation materials / maps with annotated diagrams in different areas of visualisation destined for researchers in the Arts and Humanities, but authored in collaboration with the Sciences and Engineering.

• Clarify terms used during group discussions, as you go, in order to develop some common language. The diagrams presented in the slides in the morning may assist in identification the same terms employed differently by diverse members of the group, or different terms utilised for the same concept or process. The conversations in the groups are an inherent part of the process of clarification.

• Start to think in diagrams and visual juxtaposition in order to
o Use visual design to present structural information about different areas of visualisation
o Use visual design to present diverse views simultaneously. This involves finding the commonality and showing the relationships between these views, but also encoding their differences in these simultaneous diagrams.

5.2 Issues
A number of issues and discussion points were raised:
• Orientation materials may date quickly in a field where technologies are changing rapidly.
• There are issues of copyright clearance for the eventual materials.
• There is a need to clarify the objectives of the superdiagram, (a diagram obtained from links between all the groups’ diagrams). Is it subject map? A visual wiki? A mechanism for resource sharing?
• Such a superdiagram should be digitally available and modifiable as it is potentially a useful tool for others to think with.
• One of the main strengths of the approach is trying to establish the links between the different disciplines.
• This process could potentially open up very interesting research collaborations.
• How far can we get today? (This last point proved to be a significant issue. Discussion had taken longer than anticipated and the combined diagramming session prior to the group activity could not take place.)

6. Group sessions
For the first part of the afternoon session, 21 participants split into three groups:
Data / Scientific / Mathematical Visualisation (7), Web-based Visualisation (5), and 3D Modelling (9). These sessions continued for one hour twenty minutes rather than the scheduled hour as participants in at least two groups did not wish to stop their debate and change groups. Responses to these group sessions have manifested both directly and indirectly in presenters’ papers.

In the original concept for the event, it had been intended to have only one breakout session involving four to five groups lasting two hours. However with the number of participants capped, in order to cover a reasonable number of topics, two group breakout sessions of one hour were scheduled. In hindsight, the original concept would have been more manageable for a single day event as a large proportion of participants had to leave early to catch trains before 4.30pm, cutting short the real time available for active participation. However a second group session allowed for a permutation of participants and greater networking opportunities.

For the second part of the afternoon session, 22 participants split into three groups:
Distributed and Grid Visualisation (6), Data Acquisition (8) and Display Technologies (8).
These sessions had only forty minutes available. Again in at least two groups, participants were reluctant to stop their discussion.

6.1 Group session 1 (1hr 20 min)

Data / Scientific / Mathematical Visualisation Group
Diagrammatic notes of the group session were recorded by Turner. These notes elaborate three conclusions: whether or not visualization is the correct term to use; that an audience can understand different things and experience different levels; and that visualization in the arts and visualization in science and engineering are similar to each other. The first conclusion concerns the fact that sonification was considered along with visualization. For this reason the term visualization no longer seemed inappropriate. The second conclusion is related to whether the audience (listening or watching) has an understanding of the underlying model(s). There was unresolved debate about whether in an artwork, they should. There was also concern that the art community appears to focus on the “end result” and not necessarily worry about the process by which it was achieved²². There was a long debate on the difference between information visualization and scientific visualization, and a discussion relating to the possible discovery of new patterns through collaboration. Other members of this group were Avis, Kaye, Fowler, Tolmie, Green, Boyd Davis, and Craft. Avis, Turner and Tolmie have perceivable responses to this group session in their papers.

Click on image for full resolution.

Web-based Visualisation Group
Part of the session was used for a further presentation on internet artworks, given by Magruder, with related discussion centring around their online viewing. There was also discussion, initiated by Robertson, on the use of web visualisation to provide better proceedings of academic conferences, especially for art and design events and exhibitions, and, in the marketing of products for e-commerce. The provision of higher resolution dynamic graphics, and more interactive options was highlighted including haptic web-interfaces so a phone can be felt, as well as seen and heard. Other members of this group were Al-Najdawi, Dunn, and Reimer.

3D Modelling Group
Much of this session was used for a discussion on the differences between the epistemological stances of the arts, humanities and the sciences, initiated by Denard. Some members of the group insisted that as disciplinary perspectives are necessarily different, there would never be a common diagram. Others were in disagreement with this view. Discussions have continued by email beyond the 19 June event. It has been suggested that further debate on this topic could form part of the programme at the upcoming vizNET 2008 Annual Event. An extensive discussion of some of the points raised during the group session is found in Beynon²³. Other members of the group were Kalawsky, de Freitas, Sporton, Baker, Almiladi, Alio, Rowland, and Smith who has also included a direct response to this group session in his conclusion.

6.2 Group session 2 (40 min)

Data Acquisition Group
Chris Rowland of the University of Dundee whose research involves the 3D visualisation of historic shipwrecks gave a brief account of his work. Discussion on data acquisition issues thus started with multi-beam sonar. There were also discussions on the processes of capturing data, on observational drawing, and on data sources related to any of the senses.

Issues with scanning mechanisms raised were fragility of source, location of source, difficulty of acquisition (due to cultural or other factors), position of source in time, and that data capture with scanning mechanisms is inherently noisy. Human intervention and judgement are involved in the process of collecting data and thus reducing known noise. How should we acknowledge when data is not ‘pure data’?

Debate then centred on the neutrality of the data. Are the data inherently neutral? Is the data merely a record? What is the role of the data collector in the identity of the data? How likely is it that the people who are collecting/representing data are imposing their own aesthetic judgement on the visualization? Group members were: Rowland, Tolmie, Green, Baker, Craft, Robertson, Smith.

Distributed and Grid Visualisation Group
(no information currently available)
Group members: Avis, Sporton, Denard, Dunn, Reimer, Priddy.

Display Technologies Group
(no information currently available)
Group members: Kalawsky, Al-Najdawi, Almiladi, Alio, de Freitas, Turner, Boyd Davis, Kaye.

7. Evaluation on the day

Participants hoped to gain increased insight into cross-disciplinary understanding, into cross-disciplinary visualization methods, and into visualization as it is used in the arts and humanities. Meeting others and learning about the state of the art was also listed. The event was rated highly from a networking perspective, with one participant commenting ‘ I expect to be able to use ideas generated from discussions in developing new resources for visualization and representing my research ideas’.

Participants’ criticisms and recommendations included more demonstration ingredients in order to give time for understanding to develop, and a tighter structure and briefing for the group sessions, considered ‘good for discussion and debate but not sure about practical value’. As we have seen above, the (reported) group sessions were each very different from one another. An assignment of roles within groups proved difficult to enforce. It was suggested by one participant that the event needed to include an overnight stay to enable group members to work together over more than one day. Language and terminology remained a significant issue. This point is further discussed in section 8. below.

From an organisational and technical perspective, time constraints and technology improvisation posed significant limitations at times. It would have useful to have the slides ahead of time in order to print copies in colour for each participant, and to upload all files to one computer for later manipulation. However because of the non-traditional format the decision was taken to let presenters have until the day to prepare their slide(s). Some presenters converted their traditional PowerPoint sequence to a single slide after watching other presenters use the format.

The one slide format had met with some scepticism before the event however by the first coffee break it had succeeded in opening the way for spontaneous and lengthy discussions. One participant noted that it was very useful to be able to view all of a presenter’s visual material simultaneously; that it made it possible to (mentally) overlay one’s own perspective on another presenter’s structure or viewpoint. Thus while PowerPoint is not the optimum tool for a global and local presentation, it proved adequate as a mock-up, in part due to its ubiquity.

8. Workshop outcomes

Time constraints on the day, and in some cases difficulty in obtaining group agreement on outcomes, have forced a backwards analysis of the event. The papers have enabled a significant post-processing of the event that would have otherwise been lost, in particular concerning the pairing of language used to the examples shown. Along with the visual material presented, the working index of ~5500 terms has enabled navigation of the cumulative conceptual space of visualisation mutually defined by these participants. Notably, this conceptual space includes both their intersections and their differences.

8. 1 Workshop outcomes - the papers

The diagram (in 2.1) showing the experience of the event from a presenter’s perspective demonstrates the cross-domain challenge faced by presenters: to author their paper so as to be accessible to a diverse range of participants. In total, the body of work submitted in the papers contains of the order of 45,000 words. An examination of the conclusions presented in 4.1 – 4.14 shows a majority of presenters to address some cross-domain aspect of the event or process. The effort and thought evident in the preparation of papers demonstrates a significant commitment to furthering this dialogue.

The format and length of the papers vary considerably in keeping with the diverse nature of the presentations. The minimum length of 1000 words allowed for an annotated visual format close to the diagrammatic presentation itself, however papers containing extensive reflection on discussion at the event were also submitted. No maximum limit facilitated the inclusion of detailed examples in the text.

8. 2 Workshop outcomes - mapping the intersections in a cross-domain path

The ordering of slides presented above and summarised in the diagram below provides a cross-domain path in which overlapping ideas or topics link each slide to the next. This path is designed to communicate the span of the cross-domain visualisation topics and issues to readers who did not attend the event. It has been prepared using the working index in compliment to the slides in order to identify overlaps between presenters’ conclusions and other presenters’ domains.

The following path interlinks the sequence of the slides 4.1 – 4.14 above. An explanation of the summary path is presented below. This is followed by a discussion about terminology which takes as its starting point the conclusions of the first slide.

Click on image for full resolution.

Notes on elements linking slides in the summary path:

Turner’s slide contains two diagrams showing separation of data flow stages for two different data flow models. The second, Spence’s Navigation Framework model involves the user. Turner concludes that separation of data flow stages extends itself to many other non-scientific themes.

Smith’s slide deconstructs a project into the distinct phases of visualisation, his final phase allowing users to manipulate and alter the models. Smith concludes that requiring dynamic visualisations by the user resulted in a combination of two types of modelling, and is possibly why the differences between the arts, humanities and the sciences made explicit in a group session may not apply to his project.

Beynon’s Empirical Modelling was placed by Denard at the intersection of the humanities and the sciences during the group session. Beynon’s discussion acknowledges those aspects of EM that can fulfil the requirements of a scientific or a humanities computing model. He further suggests it supports a holistic approach that draws upon the epistemological stances of arts, humanities and the sciences.

De Freitas foregrounded the understanding of human processes and the uses of physical, abstract and virtual space, showing videos of highly realistic serious game environments requiring integration of code-driven simulations of human physical states. She concludes the closer we can match human processes the more effective our systems will become.

Al-Najdawi’s slide shows the evolution of game engines from planar worlds to photorealism. He concludes that the rapid development of game engines exceeds that of virtual reality development.

Sporton showed a VRML (Virtual Reality Modelling Language) reconstruction of the house of Constantine Melnikov discussing the reality of the model and model making process as what we see. He offered the experience of the visual arts in understanding the impressions made by visuality.

Boyd Davis’ slide juxtaposed groups of static images, many from the visual arts. He discussed depiction, in particular the mistake of photorealistic depiction, suggesting the demand for realism results in use of images having nothing to do with information.

Baker critiques the use of computer graphics for their own sake as possibly misleading in their interpretation. He offers data artefacts within visualisations, marking the paths taken in the construction of the research narrative. He defines the concept paradata, as distinct from metadata.

Dunn explains that humanists have long used visualisation techniques to represent data structures. With e-science adding new meaning to concept of image for the humanities, he concludes that interdisciplinary collaboration with experts in visualisation is essential to the development of the field.

Avis’ slide presents the grid, and grid visualisation. While in his view problem solving environments must have an aspect of collaboration, he concludes that future visualisation systems will also need to allow non-visualisation experts to explore their data and create their own presentations.

Tolmie’s slide juxtaposes examples from computer-based abstraction in the arts in order to raise the possibility of collaboration with experts in information visualisation and scientific visualisation. She discusses both acquired data and simulation driven works in different compositional environments.

Green’s slide(s) show artworks involving real-time data acquisition processes and photographs of various sensors that have been used as input nodes. He discusses integration of Web 2.0 into a collaborative online digital arts environment, CODA.

Magruder’s links to internet art demonstrate the creative potential of 2D and 3D web-based visualisation technologies. He concludes artists are just beginning to implement Web 2.0 frameworks in their creative productions.

Al-Najdawi’s slides contain accessibility and usability guidelines and examples for web-based visualisation. He concludes making sites accessible to users with disabilities ultimately makes them more usable for everyone.

8.3. Workshop outcomes - tracking terminology across domains

Terminology across domains can now be investigated by following a path from the conclusions of any slide. The working index can also locate terminology from the slide’s conclusions in other presenters’ papers. The example applies this combined methodology to the first slide.

Turner (4.1) proposed for further discussion that this separation of the data flow stages allows repeatability and extends itself to many other non-scientific themes.

The terms data flow and stages are separately tracked. Data flow reveals connections already discussed in the Data / Scientific / Mathematical Visualisation group notes and as such they are not pursued in depth here. Stages reveals connections to two of the other presenters, including the author of the next slide in the mapped sequence. A comparison of terms used by these presenters is then performed. Interesting differences in their usage of selected terms are found and investigated.

Data flow

Data flow models have relevance to the slides of Tolmie, Green and Magruder. Dataflow environments in the arts are discussed in Tolmie (Pd and vvvv) and Green (Max/MSP Jitter).

Click on image for full resolution.

From Stages to Interpretation and User/Interaction

The other presenters referring to stages of a visualisation in their papers are Smith and Baker. Similarities and differences between the terminology of Turner, Smith and Baker are examined and used as a starting point for a discussion of usage of terminology across domains.

A comparison of terms used by Turner, Baker and Smith reveals Turner and Smith to use user, interactive or interactivity and interface, none of which are used by Baker. Turner and Baker use interpretation, a term not used by Smith.

All presenters with the exception of Beynon and Baker employ the term user. Interaction or interactivity is also employed by Beynon, Al-Najdawi, Dunn, Avis, Tolmie and Magruder. Other users of the term interpretation are Beynon, Boyd Davis and Sporton.

The following analysis examines interpretation as it is used by Turner and Baker, respectively. Extracts are then provided for the reader to investigate the term user, as it is employed by Turner and Smith. Interaction/interactivity are introduced diagrammatically but not analysed in detail here.

8.3.1 Interpretation

Interpretation is one of the stages in Spence’s Navigation Framework, outlined by Turner²⁴. The others are content, internal model and browsing, with navigation defined as the creation and interpretation of an internal mental model. Turner also uses the phrases:
• interpretation of that model²⁵
• based on the interpretation, the user formulates a new browsing strategy²⁶.
• A decision as to how – as well as whether – to proceed will depend upon an interpretation of these sources of information²⁷.
• externalization of data and how it fits into the navigation model with regards to interpretation²⁸.
• The Interpret part in Spence’s model is equivalent to Rendering in Haber and McNabb with the Displayable Object forming via the Interpretation (Spence)²⁹.

Turner's last phrase demonstrates a more formalised usage of the terms interpret and interpretation, with interpret indicated as equivalent to rendering in a specified sense, and a displayable object forming via the interpretation.

Interpretation is located in the centre of Baker’s slide around visualisation. Baker uses the phrases:
• interpretation of a data object³⁰.
• metadata records the documenter’s interpretation of a data object³¹.
• paradata records the documenter’s process of interpretation, to allow it to be subjected to scrutiny and evaluation³².
• interpretation of both evidential and visualisation objects and their associated data artefacts³³.
• process of analysis and interpretation of the object and its data artefacts³⁴.
• for paradata, terms such as interpretation, knowledge, understanding, synthesis etc. are key concepts that metadata does not capture³⁵.
• use of computer graphic images for their own sake that may be misleading in their interpretation³⁶.

Baker's last phrase contains the term computer graphic images but does not contain any of his more formalised terms such as data artefact or data object. Investigating further, we see that Baker uses the following definitions³⁷:
• Data Artefact - Data recorded and attributed to a data object as a product of human conception or agency.
• Data Object - A ‘thing’, perceptible by one or more of the senses, about which data is held.

Baker’s definition of data object as a thing perceptible by one or more of the senses, appears to exclude abstract data sets, by definition. If this is the case, Baker’s terms visual research³⁸, and visual research outcomes³⁹, also exclude, by definition, any visualisation that is based on an abstract data set.

One can argue that context should provide sufficient cues for these restrictions to be understood. However context may not be apparent in a cross-domain setting. Are there cases above where Turner and Baker are not referring to the same activity when they use the term interpretation? If in a group discussion Turner and Baker both use the term interpretation, how can they, and the other group members, know when they are they talking about the same concept? At what point in the use of shared terminology is it of consequence that one person’s underlying definition excludes the objects or processes on which the other works? In a cross-domain discussion it is probable their shared usage of the term interpretation would sometimes be indicative of a shared understanding of a generalised concept, however at other times their very specific and very different formalisations would have to be clarified within that discussion.

A similar situation applies in the usage of the term transformation. This term is used by all of Turner, Smith and Baker, and by the other presenters, Beynon, Dunn, Boyd Davis, Avis and Tolmie. In some of these cases, transformation is used in a mathematical sense; in other cases the sense is closer to that of attachment of an insight or a deduction to an object or a piece of evidence. The former could possess a well-defined inverse transformation⁴⁰ for example converting from RGB (Red Green Blue) colour to HSB (Hue Saturation Brightness). The latter could not.

8.3.2 User
The same type of question can be asked of Smith and Turner when they refer to the activities of the user. In this case, the inclusion of the user may make the usage of these terms more generic. It is left to the reader to juxtapose the presenters’ statements with the content and structure of their slides:

Smith uses the phrases:
• user generated content⁴¹
• models actively re-examined by users⁴²
• process is reactivated by the user but in reverse⁴³
• allow the user to generate their own visualisations⁴⁴
• users are able to ask their own questions⁴⁵
• users to have an active hand in the construction of their own ‘take on things’⁴⁶
• user is able to query the data at many stages in its development⁴⁷
• user is able to search and transform these mouldings at specific points⁴⁸
• visualisation methods that allow the audience to interrogate dataset⁴⁹
• users can drill down through the final presentation of the model⁵⁰
• users select or take cross sections through specific objects and ‘save them’⁵¹
• objects […] can be manipulated, measured and reconfigured according to users unique research query⁵²

Turner uses the phrases:
• the possible choices a user has choices in constructing a dataflow model⁵³
• should not be imposed on the user to see just one view – but to investigate⁵⁴
• involves modifying both the rendering stage and also the data flow model itself as the user requires⁵⁵
• user can refine their browsing strategy that will add information to the model⁵⁶
• execution of any task involving information visualization will be motivated by user’s intention⁵⁷
• analysing the user’s mental model and environment to understand how the visualization may progress through interactivity⁵⁸

It is interesting to note that Smith (4.2) concludes This project encapsulates a number of different types of visualisation and interestingly the distinction between arts, humanities and science made in the group session does not seem to apply here – the visualisation methods are used for both aesthetic reasons as well as for representing information and testing hypotheses. Also the requirement of the dynamic creation of visualisations by the user resulted in an implicit combining of process and product orientated modelling which is common to applications in art, science and the humanities⁵⁹.

In the next slide in the sequence, Beynon, who does not use the term user, but who does employ the term interaction, notes […]The distinctive feature of the more informal and primitive representational stance that is found in art and experimental science is that the meaning of the artefact can only be mediated through live interaction. What potentially bring objectivity to the interpretation of the artefact are the reliable patterns of interaction and interpretation that develop around it⁶⁰.

Are Smith and Beynon offering the same insight in these statements? Interestingly, both are discussing processes associated with modelling, and different types of, and contexts for, modelling.

It is of significance that even a brief investigation starting from the conclusions in the first slide of the sequence (Turner) has revealed non-trivial intersections with material in the two following slides (Smith, Beynon). These intersections warrant investigation and clarification.

The diagram on the following page summarises use of the terms interpretation and interaction / interactivity at this event. (For diagrammatic brevity only the term interaction is used on the actual diagram.) We see that Turner and Beynon are the only presenters who use both interpretation and interaction explicitly. It could be argued however that these terms are implicit in the discussion of some of the other presenters, for example de Freitas, who uses neither. In the context of the above discussion a question that could be asked of this diagram is whether interpretation is a term used more frequently with models or static images in the arts and humanities not destined to be modifiable by the user, and/or whether in models destined to be modifiable by the user in the arts and humanities the term interaction then assumes precedence?

Click on image for full resolution.

These brief examples have served to highlight issues that can be raised when the same terms are employed differently by diverse disciplines in a cross-domain setting. Many such queries could be performed to investigate areas in which shared terminology or content, and its possible misunderstandings, are anticipated. The examples offer clarification strategies that could be employed to identify when this occurs.

8.4 Workshop outcomes - mapping key topics and discussion areas

By removing the individuals’ slides from the path diagram we obtain a new path composed of intersections in sequential presenters’ slides, rather than the slides themselves. This will enable a generalisation to broader groupings to be performed.

Click on image for full resolution.

Because of the sequential nature of the intersections, a number of key topics and broader areas of discussion can be mapped by grouping along the path as shown below. The linearity of the path will, by definition, obstruct some related intersections however it provides a well structured starting point.

Click on image for full resolution.

Only the cumulative space of visualisation defined by the presenters’ slides and papers can be mapped using this methodology. This process captured areas of visualisation that were represented at the event or referred to in presenters’ papers. Further events can be used to augment the initial mapping or to produce alternative mappings for comparison with emerging intersections in other visualisation groupings.

The diagram encapsulates the title of the event From Abstract Data Mapping to 3D Photorealism: understanding emerging intersections of visualisation practices and techniques, with abstract data mapping located towards the left and realism located towards the right of the page. Eight broader areas are identified:
• User generated (visualisations)
• Visualisation ==> shifting disciplinary boundaries
• (Real-time) game environments
• Web-based environments
• Realism
• Visual communication and depiction
• Grid-based datasets - grid visualisation - collaboration
• Abstract data mapping

Of these groupings, emerging intersections across the arts and humanities and the sciences and engineering can be seen most strikingly within the grid-based datasets - grid visualisation - collaboration juxtaposition of the slides of Dunn and Avis, and within the user generated (visualisations) juxtaposition of the slides of Turner and Smith.

Centre and right in the diagram above, that the emergence of increasingly sophisticated visualisations, and the associated real-time interaction with and authorship of those visualisations, are having a controversial impact on disciplinary boundaries, forcing a rethink of realism, visual communication and depiction, is perhaps not surprising. The same types of discussions are taking place in the abstract data mapping area between the arts and the sciences (on the left above), however territorial claims are not as much in evidence therein with identification of differences performed to facilitate collaboration rather than to preserve those differences. The emerging humanities use of data mash-ups based on diverse grid-based resources involving both realism and abstract data structures (Dunn, bottom left above), provides evidence of an emerging bridge between these two groups in the arts and humanities.

9. Conclusion

In conclusion, this event has enabled significant further development of the cross-domain discussions initiated at vizNET 2007. Many important issues have been identified that are worthy of further debate enabling these views and perspectives to be structured into future cross-domain events. This is already the case in the programming of vizNET 2008⁶¹ which will take place 7-9 May 2008.

The development of shared language and terminology was identified as an objective during the post-diagramming discussion on the day (5.1 above). Many individuals contributed a great deal of their time and considerable thought to the preparation of materials, in particular papers, for a cross-domain audience. This material in turn has been synthesised into a working format showing it to be a viable tool for navigating the cumulative conceptual space of visualisation, as defined by the research activity and visualisation support offered by participants. It is hoped to use this initial working format as a basis for a systematic synthesis that can provide a resource for cross-domain collaboration in visualisation.

In addition this workshop provided an invaluable opportunity for networking and for first hand insight into visualisation across domains. The vision of the AHRC ICT Methods Network in recognising and supporting a non-traditional format for this event has enabled its scope to be inclusive of diverse disciplinary discourses, while simultaneously trying to mitigate the barriers of discipline specific terminology. As a result, both personal and professional interaction between participants who may not have understood each others’ perspectives through terminology alone has been enabled.

10. Acknowledgements

I would like to thank Prof Roy Kalawsky, Prof Vince Gaffney and Dr Gregory Sporton for early cross-domain discussions that contributed to the proposal of this event, and the Visualisation Research Unit at Birmingham Institute of Art and Design, Dr Gregory Sporton, Mike Priddy and Jonathan Green, for the planning insights and discussions developed around the venue and set up, including audio capture by Jonathan Green and photographs by Mike Priddy. I would also like to thank Dr Nijad Al-Najdawi for many productive discussions about the format of the event. In addition, although unable to attend on the day, Prof Ken Brodlie kindly allowed his slides to be re-presented in a one slide diagrammatic format as a template for presenters. In fact the presenters and participants of vizNET 2007⁶² event contributed significantly to the design of the workshop, albeit indirectly.

The day would not have been possible without the experts themselves: the presenters and participants, the teams from vizNET and 3DVisA and the AHRC ICT Methods Network, of whom Torsten Reimer represented both the physical and virtual support they offer. A discussion group for this event has been set up by Reimer at http://www.arts-humanities.net/ .

Finally I would like to acknowledge the generous support of the JISC in funding vizNET⁶³, the UK Visualisation Support Network, and 3DVisA⁶⁴, the 3D Visualisation in the Arts Network, and for the JISC’s further support of cross-domain vizNET / 3DVisA activities, further information for which can be accessed at http://www.viznet.ac.uk/intersections.html

Julie Tolmie
London 2008

For further information, or to announce your cross-domain events to these pages,
please contact Dr Julie Tolmie (julie.tolmie@kcl.ac.uk).

============================

Appendix: Participant and group session lists

Participants
Jamil Alio, DeMontfort University
Dr Abdul Almiladi, DeMontfort University
Dr Nijad Al-Najdawi, Loughborough University
Prof Nick Avis, University of Cardiff
Dr Meurig Beynon, University of Warwick
Drew Baker, King's College London
Dr Stephen Boyd Davis, Middlesex University
Dr Hugh Denard, King's College London
Dr Stuart Dunn, King's College London
Dr Ron Fowler, Science and Technology Facilities Council
Dr Sara de Freitas, Serious Games Institute, University of Coventry
Jonathan Green, Birmingham Institute of Art and Design
Prof Roy Kalawsky, Loughborough University
Dr Andrew Kaye, Science and Technology Facilities Council
Michael Takeo Magruder, King's College London
Mike Priddy, Birmingham Institute of Art and Design
Dr Torsten Reimer, King's College London
Alec Robertson, DeMontfort University
Chris Rowland, University of Dundee
Carl Smith, London Metropolitan University
Dr Gregory Sporton, Birmingham Institute of Art and Design
Dr Julie Tolmie, King's College London

============================

Group Sessions

Data / Scientific / Mathematical Visualisation
Prof Nick Avis, University of Cardiff
Dr Andrew Kaye, Science and Technology Facilities Council
Dr Ron Fowler, Science and Technology Facilities Council
Dr Julie Tolmie, King's College London
Jonathan Green, Birmingham Institute of Art and Design
Dr Stephen Boyd Davis, Middlesex University
Dr Brock Craft, London Knowledge Lab

Web-based Visualisation
Dr Nijad Al-Najdawi, Loughborough University
Alec Robertson, DeMontfort University
Dr Stuart Dunn, King's College London
Michael Takeo Magruder, King's College London
Dr Torsten Reimer, King's College London

3D Modelling
Prof Roy Kalawsky, Loughborough University
Dr Sara de Freitas, Serious Games Institute, University of Coventry
Dr Gregory Sporton, Birmingham Institute of Art and Design
Dr Hugh Denard, King's College London
Drew Baker, King's College London
Dr Abdul Almiladi, DeMontfort University
Jamil Alio, DeMontfort University
Chris Rowland, University of Dundee
Dr Meurig Beynon, University of Warwick
Carl Smith, London Metropolitan University

Distributed and Grid Visualisation
Prof Nick Avis, University of Cardiff
Dr Gregory Sporton, Birmingham Institute of Art and Design
Dr Hugh Denard, King's College London
Dr Stuart Dunn, King's College London
Dr Torsten Reimer, King's College London
Mike Priddy, Birmingham Institute of Art and Design

Data Acquisition
Chris Rowland, University of Dundee
Dr Julie Tolmie, King's College London
Jonathan Green, Birmingham Institute of Art and Design
Drew Baker, King's College London
Dr Meurig Beynon, University of Warwick
Dr Brock Craft, London Knowledge Lab
Alec Robertson, DeMontfort University
Carl Smith, London Metropolitan University

Display Technologies
Prof Roy Kalawsky, Loughborough University
Dr Nijad Al-Najdawi, Loughborough University
Dr Abdul Almiladi, DeMontfort University
Jamil Alio, DeMontfort University
Dr Sara de Freitas, Serious Games Institute, University of Coventry
Dr Martin Turner, University of Manchester
Dr Stephen Boyd Davis, Middlesex University
Dr Andrew Kaye, Science and Technology Facilities Council

==========================================================

vizNET

Resources

Activities

Related Links