Wednesday 23 May 2012

More computational toys for artists


Luke had developed the habit of calling his early experiments "computational toys" - image-based interactive systems that exhibit some kind of computational behaviour, thereby providing opportunities for creative experiences of flow and emergence. I think he recently referred to Palimpsest as a computational toy too, though it's rather more complex than anything of the kind we've built in the past. (Though to be honest, Michelle's positive experience was more by nature of a toy - the further complexities may yet turn out to be of minimal additional advantage).

Other blog readers have had their eye out for projects that they see as being related. All of them exhibit this same kind of behaviour, suggesting that the general category is recognisable, even to people without the precise theoretical fixations that Luke and I had. Two have come up recently:


  • Sam Aaron pointed out the Recursive Drawing system built by Toby Schachman for his project Alternative Programming Interfaces for Alternative Programmers (I created the illustration above using this).
  • Beryl Plimmer noted a presentation at CHI of the Vignette system from Igarashi's Design Interface Project, which uses gestures to replicate and fill with texture elements that can be dynamically created and modified while being used as drawing elements.

Both of these recent projects might be considered within the general class of procedural drawing systems - they are oriented toward artists rather than programmers, though Recursive Drawing is especially computational in its feel - as indicated by the project title. Vignette is sufficiently close to real professional design requirements that similar systems have been created fairly regularly in the past - from Sketchpad, to Paul Richens' Piranesi system for architectural rendering with its similar perspective and texture tools.

The challenging question for Palimpsest is, to what extent can these interactive procedural drawing systems be extended with more sophisticated computational abstractions, before they lose their appeal as computational toys? The answer relies on finding a particular balance between flow and attention investment, as well as sufficient quality of design. Recursive Drawing is rather similar to Palimpsest in its visual layout, including a side bar that resembles the Palimpsest layer stack. However it is far more elegant! Perhaps because its computational properties are quite specific and limited, but perhaps also because it is an art school product informed by computing expertise, rather than the other way round.

Wednesday 16 May 2012

From naive physics to naive computation



Having done another couple of demos to interested observers, I've realised that the "translate" operation is not that useful. Translation is currently defined in terms of a vector, but the use of translation is most often to move something to the place you want it - the vector required to get it there is a byproduct, not the main point of interest. In fact, getting an object to a specified place is a real pain at present. The steps are:

  • Create a point representation layer. The default behaviour, when a point representation is created over an image, is to use the centroid of the image as the default value, so this works OK.
  • Create another point representation. The default behaviour, when another point is under this one, is to create a new randomised value. This is also OK, because an arbitrary translation can be used as the basis from which the user explores alternative values.
  • Create a vector calculator. This will make a vector derived from the two points, which is also OK.
  • Create a translation layer, which will use the original image, and the calculated vector, to move the image to the required point.

So there are four layers required to achieve one effect, which is arguably the most natural way to specify a translation anyway. Today I created a simpler variant of translation, which simply takes a point as a parameter and moves to there.

However, this does seem to point to a more general issue. My geometric operators are all nicely based on properly defined mathematical transformations. As a side-effect, this made them really easy to implement - each one simply corresponds to one of the Java Graphics2D AffineTransform operations. But perhaps we should be suspicious when the abstraction needed to implement a system function is too convenient - it's a sign that we might be imposing the programmer's mental model on the user.

In fact, every one of the geometric transforms has turned out to be not so closely related to the user applications that I've found interesting for those transforms. In a "naive geometry" approach, they could be described as follows:

  • Translate -> "move it to here"
  • Rotate -> "make it spin round" (usually as an animation that doesn't stop)
  • Scale -> "stretch or squash" (not uniformly, but in various ways that drag handles can produce)

I suspect that I should discard the proper mathematical versions, replacing them with a move layer (which is what I had originally before making the more elegant mathematical transforms), a spin layer, and a layer that can reproduce any number of adjustments using the direct manipulation handles. This last will also overcome another problem. Although it was possible to generate a mathematical transform layer initialised according to handle manipulation, only the last manipulation was included - mainly because it would be so surprising to the user to see multiple layers of transform appear in response to a single button press.

Finally, a little reflection on "naive geometry". This takes me back to my MSc thesis, when I formulated a naive physics-style "qualitative trigonometry" that could be used for robust spatial reasoning by robots. Along with the rest of the naive physics / qualitative spatial reasoning movement in the mid 1980s, this could well have represented a user-orientation within the AI community, as we tried to create knowledge representations that were better aligned with common sense. At that time, the motivation was to replicate human problem-solving performance, rather than making computer "reasoning" easier for humans to understand, but the latter was  undoubtedly a side effect.

From a more HCI orientation, the Natural Programming project of Brad Myers and his students could be considered as an approach to defining a "naive computation" where program behaviour is described in common sense terms. I should probably have spotted this earlier, because I've been recommending it to Jamie Diprose, a student of Beryl's who is creating a visual programming language for healthcare robotics. I've encouraged Jamie to take an approach derived from John Pane's natural programming work, interviewing healthcare professionals to establish a vocabulary of domain concepts for use in his language. The healthcare robotics domain is sufficiently unlike the general purpose mechanical assembly domain of my own earlier work that I hadn't noticed the analogy to qualitative trigonometry, but now that I've noticed it, I could regard my project as creating a domain-specific language for exploratory image manipulation.

Wednesday 2 May 2012

Some precedents


During James Noble's stay, he made a number of useful observations regarding precedents for some of the features in Palimpsest.

He thinks the overall feel is similar to that of early graphical constraint systems like Borning's ThingLab, or Smith's Alternate Reality Kit. The representation of constraints as "parameters" controlling the appearance of specific layer effects reminded him of the pointer constraints used in Myers' Garnet. Finally, the way that these are assigned default values from the context in which they are created (by type inference / extension from layers further down the stack) is like the implicit parameters in the Scala language.

References

Alan Borning. 1981. The Programming Language Aspects of ThingLab, a Constraint-Oriented Simulation Laboratory. ACM Trans. Program. Lang. Syst. 3, 4 (October 1981), 353-387.

Randall B. Smith. 1986. Experiences with the alternate reality kit: an example of the tension between literalism and magic. In Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface (CHI '87), John M. Carroll and Peter P. Tanner (Eds.). ACM, New York, NY, USA, 61-67.


Brad Vander Zanden, Brad A. Myers, Dario A. Giuse, and Pedro Szekely. 1994. Integrating pointer variables into one-way constraint models. ACM Trans. Comput.-Hum. Interact. 1, 2 (June 1994), 161-213.