Beryl Plimmer welcomed me to Auckland with a one-day workshop, coinciding with a visit by Gem Stapleton from Brighton. Participants from Beryl's included new post-doc (previously PhD student) Rachel Blagoevich, with current PhD student Jamie Diprose and summer research assistant Ryan, as well as department lecturer Robert Sheehan. All are interested in notation design, with Robert being more oriented toward end-user programming, and the others theoretical and usability principles (especially future plans to use tangibles and multitouch alongside the group's existing expertise in sketch recognition).
This audience provided an excuse for my first attempt at an overview so far - I try to reproduce that summary below.
Motivation
This project is mainly inspired by my work with various collaborators in recent years, creating end-user language/notations for use by artists working with digital media. In the context of design research, collaboration with artists offers a valuable counterpoint to work with engineers, arguably a necessary element of exploring the full range of requirements for professional design tools. Working with artists also draws attention to particular parts of the attention investment / user experience equation, where end-users engage in construction activity for its intrinsic rewards (including, for example, flow experiences).
Observations of artist collaborators has also drawn attention to two interesting technical requirements. One is that many professional artists find the keyboard an obstacle to creative practice (thinking here of observations of Random Dance, and discussions with Bruce Gernand). This revives the long-standing challenge of the "purely visual" language, of which Dave Smith's Pygmalion, George Furnas's BitPict, Clayton Lewis's NoPumpG, and Wayne Citrin's VIPR have been stimulating if rather impractical examples. Although often mooted, such languages have seldom seemed compelling on machines that do, after all, have keyboards. In the age of tablets and touch interaction, keyboards have suddenly become a real inconvenience and hindrance in routine interaction, so this seems a better time than any to explore text-free notations. Everything that I have done so far, in principle, will be completely usable on a tablet (assuming I ever manage to port the graphical operations to Android Java).
Features
This language allows users to combine source images (photographs, simple shapes or ink) and treatments of those images.
Sources and treatments are layered on top of one another, in the manner of a palimpsest (a potential name for the language!).
All layers can be broken up into fragments, with these fragments being combined in a visual collage (another potential name).
The behaviour of treatments and fragments can be modified by adjusting values. Values themselves are also represented as fragments or layers - these can define visual parameters such as proportion/intensity, vectors, colours and so on.
Interaction with the system involves creating new layers, and superimposing them on layers already created. The resulting stack of source, values and treatments provides both a visual palimpsest, and a historical record of the process by which it was achieved.
The other element of the visual interface, apart from the main image manipulation area, is a visualisation of the stack of layers. The usability implications of the layer stack can be considered in terms of conjoining two standard features of Photoshop: the layer window and the history window. Some disadvantages of doing this can be understood by analogy to the (notoriously hard to understand) Photoshop history brush. The workshop audience expressed concern about this aspect of the user experience (they could have, but did not, express it in terms of premature commitment and viscosity). However, from my perspective, this mingling of process and product is also characteristic of artistic practice - our attempts to regularise it in the projects with Bruce Gernand and Random Dance provided them with history management mechanisms, but not mechanisms that they integrated fluently into their creative activity. We'll see how this enforced intermingling plays out.
Computational properties
The primary data structure facility at present is a layer whose fragments represent references to other layers. These fragments can be used either to modify or refer to particular image sources, treatments or values from other layers, or to preserve a whole stack as an ordered collection of fragments within a single layer.
The workshop audience was sceptical, that this data structure presents sufficient complexity to be described as a programming language. The behaviour of the system at present is rather reminiscent of Hypercard, but without any of the Hypercard script - from a traditional programming language perspective, it seems as though I have removed the only linguistic element of Hypercard (ignoring the notational analysis perspective, that the card representations and interaction environment provide a powerful notational system). I made some analogies to early programming languages - fragments as being like dictionary words in Forth, or stack sequences and references as like primitive LISP - with all symbols replaced by images. However, although these might support behavioural interpretation, they are not behaviours in themselves. A final poor analogy is that, as with the lambda calculus, these arrangements of images support a computational interpretation, in principle including sufficient expressive power to implement a Turing machine. (This interpretation won't help much in Auckland - it seems that the lambda calculus is not taught at undergrad level).
Nevertheless, the advice given by Robert and Beryl was that, if I wished to have this recognised as a programming language, it should have some behaviours that look like program execution. They recommended iteration, or evaluation of conditions. I've taken this on board, and that's where I'm going next ...
No comments:
Post a Comment