Introduction to text linguistics. Учебное пособие

In my prototype implementation the object system is simply XML based. But to re-emphasise: A radical principle of VIGO: Interaction is always targeted objects rather than a system or an application. State cannot change without changing objects. This means that changes seen on the screen are always reflected in changes in objects. Views are very similar to views in MVC, however they provide no interaction. They just map objects to the screen.

Instruments are a central construct. All user interaction is mediated by instruments. Instruments are inspired by the real world for instance a stick to enhance ones reach, or a pen to write on a piece of paper. Instruments could be anything like text instruments, instruments for manipulating graphics, drawing instruments, selection instruments, copy past instruments etc. Instruments can basically be understood as event processors. They translate user actions on input devices to manipulations of objects. Instruments require objects to have certain properties. For instance a move instrument would require an object to have a position in some coordinate system, but not a specific type. Instruments can also provide feedback to the user, however it does that by having associated objects that it manipulates and then are rendered to the user through views. For instance a drawing instrument could have a cursor changing color according to the color the user has selected.

These instruments and objects is however not enough for all kinds of computer uses.

Take an example of a board game like Othello – Othello is a simple board game of placing and flipping black and white pieces. If we were to implement Othello in VIGO, the objects would be the board and pieces. They could just be represented as a checkered square and a number of black and white circles. Then we would have some instrument to move the pieces. Whether with controller with a mouse, keyboard or voice-commands would not matter much. But when we play a game on a computer we want the computer to enforce the rules and handle complicated consequences of actions, and not having to do it manually. Especially in Othello where there are many mistakes to be made when manually having to keep the board in a correct state.

However doing it in instruments is problematic. It would either create very specific instruments or require instruments to know every rule of

91

every game. Keeping rules in objects is also problematic because we would like to keep them lightweight to support distribution. Instead we introduce the concept of Governors. Governors embody the rules and consequences for manipulating objects.

Governors are associated to objects at the level of individual properties. And only cares if what aspect of an object it governs are changed. In Othello this means that the pieces will have a governor associated to their position, and if the user changes the position of a piece it as to adhere to the rules of the governor. When the position is changed the governor reacts by flipping pieces on the board accordingly. However if the user does something else, like drawing on the board with a drawing instrument, the governors do not react. Governors are stateless pieces of code, all state is in objects. This means that they can be distributed easily, and executed easily anywhere in a system.

But to show you what this actually means for interaction design, let me show a video.

I have created a small prototype system to experiment with VIGO on. In the video I will show you a system where the user does graphical editing mediated by instruments, and how pick-and-drop is implemented to facilitate multi-device interaction.

Here we see the setup: We have a smartboard like this one behind me, and on the table is a Nokia Internet Tablet. I have a canvas object on the smartboard.

Here we have a canvas on a smartboard which we draw on using objects and colors picked on the internet tablet.

Instruments are selected with gestures on the smartboard

Finally the canvas is moved to the internet tablet and interaction is continued there.

In the video you saw how I manipulated graphics with simple instruments. I showed that the move instruments were applicable on both the canvas and the graphical objects. The move instrument also provide feedback by a line object. This will be more apparent in the next video.

In the video you furthermore saw that there were two distributed instrument. A pick-and-drop instrument and a color-picker instrument. The instrumentsshareeventsoverthenetworkbetweentheuserstwodevices,and this distributed aspect is completely decoupled from the objects worked on.

92

On the smartboard we furthermore saw how a meta instrument, a gesture instrument, were used to activate other instruments.

Now in the first video everything was handled in instruments. Things get a bit more interesting when we add governors.

Here we see an implementation of Othello on the internet tablet.

We can move pieces and the governors react. The board can now be picked and dropped on the smartboard. We now have two synchronized boards creating primitive multiplayer. In this video the governors are executed on a server, but they could in principle be anywhere on the network. But here we see that the VIGO architecture actually generalises pick-and-drop and makes it capable of picking and dropping what resembles live applications.

To sum up:

Complex artifact ecologies challenge the concept of applications and the way we traditionally build interactive software.

Ubiquitous Instrumental Interaction is proposed as an alternative interaction paradigm.

VIGO is proposed as an architectural model to realise the new interaction paradigm.

For the remained of the presentation I will elaborate on some issues that were raised by my opponents after having read my dissertation, and I believe that could also be of general interest.

Firstly:

What are the use and future of VIGO.

The utopian goal would be to create a VIGO based operating system, so everything was VIGO based following the ubiquitous instrumental interaction paradigm. However as my opponents commented, there has already been invested millions of millions of dollars in what we have now. But the VIGO architecture could also have potentials for less holistic projects.

In my dissertation work I did a study with a group group of experimental physicists on creating a laboratory notebook environment on a VIGO architecture. Here what they would like to be able to do was to integrate documentation, analysis and control in one, and be able to dynamically access the laboratory notebook from multiple different interactive artifacts. Both something that are quite natural to theVIGO architecture.

93

To generalise a bit on the laboratory notebook study, it would be interesting to use VIGO to create a multi-device document editing environment, where it was possible to do anything from a shared family scrapbook to e.g. laboratory notebooks. Another thing that would be interesting to explore is to create a VIGO based framework for building interactive software to run in ubiquitous environments. In a week I am heading to Paris to start on my post doc position in Michel laboratory where they have an installation in a room with a giant display, with various exotic input capabilities and an interactive table.And they would like to be able to control stuff on the screen through the table, or through handheld devices. Here it could be obvious to apply VIGO.

Another issue raised was how services should be understood in VIGO. One way to think services into VIGO is to think of them in the same manner I describe external objects such as for instance a servo. A servo could be represented by an object in VIGO through a governor. Hence it would be possible to control the servo and read the state of the servo by writing in the object. Likewise something external to VIGO like a compiler could be represented in a similar manner. Having an object that

gave both control and inspection of the compiler service.

Opponents voiced concerns that users would be unable to comprehend the use of instruments were they completely detached from objects. The point with VIGO is that decoupling is possible, given that the coupling between instruments and objects is discretisised from tool->filetype to tool->property. This does not mean that there will not be specialised instruments, in fact I believe they are unavoidable.

Like in the physical world some instruments wont make sense outside their designated activity, or together with the objects they are intended to be used on.

A final thing I would like to sort out, is to emphasise that there are limitations with the theoretical approach taken in the development of the human-artifact model.

Activity theory is very work oriented, and has little vocabulary to deal with experience, emotions or aesthetics.Therefore I also in my dissertation state that the work should be seen as a supplement to the recent work on these aspects of HCI. For instance the newly revitalized pragmatist or symbolic interactionalist approaches.

94

Finally, given activity theoryʼs rich vocabulary, there are pitfalls of using it for mindless categorization. The same pitfall is there with the Human-Artifact Model, however this is not the way it is intended to be used. It is intended to be used as a microscope where filters can be changed, and we emphasise this alot in our paper.

Presented two contributions for coping with Human-Computer interaction in ComplexArtifact Ecologies

One theoretical: The Human-Artifact Model One Technological: The VIGO architectural model

Clemens Nylandsted Klokmose, PhD

Department of Digital Design and Information Studies

University ofAarhus

Helsingforsgade 14

DK-8200Aarhus N

Denmark1

1 http://www.klokmose.net/clemens/files/Manuscript.pdf.

95