skip to content

Transparent Computing - Human Computer Interaction in the pervasive environment

Contact: Nick (Sheep) Dalton (N.Dalton 'at' open.ac.uk)

The transparent computing research theme is concerned with how you might use a thousand computers at once. How can this be done in a way which provides better service rather than more distractions (an area known as calm computing). This is the area where a good deal of new computing is emerging, for example Microsoft have introduced `the surface', a pervasive computer in a table intended to be used by many collocated users simultaneously. Much of the future computing that needs to be studied hasn’t been built yet and so this theme is strongly related to the production of such new types of future computing platform.

The concept of ubiquitous computing emerged from Mark Weiser, who coined the phrase around 1988, during his tenure as Chief Technologist of the Xerox Palo Alto Research Center (PARC). At its core ubiquitous computing realises the historical trend from a single computer serving many people, through the rise of the mini computer, to the personal computer: the era where you may be using many computers simultaneously (your computer viewing the web page, the router computers for the internet, the computer serving the internet page to you, the mobile phone computer in your pocket, the ipod computer, the computer in your heating system, fridge, washing machine, video camera). Pervasive computing looks to the time when you have access to hundreds or thousands of computers. 

What does this mean? The transparent computing research theme is concerned with how you might use a thousand computers at once. How can this be done in a way which provides better service rather than more distractions (an area known as calm computing). This is the area were a good deal of new computing is emerging, for example Microsoft have introduced the surface, a pervasive computer in a table intended to be used by many collocated users simultaneously. Much of the future computing that needs to be studied hasn’t been built yet and so this theme is strongly related to the production of such new types of future computing platform.  

The transparent computer theme looks to build and study a range of devices trying to investigate how computing will change over the next ten years. Transparency has two opposing threads which make up the theme, how to make interaction with a device transparent - were the way you use a computer becomes so transparent you don’t have to think about how to use the computer but on how to achieve your goal. Currently this is based on creating touch and gesture based interaction systems. The second thread of the theme is how to visualise elements of the computer and network which are largely beyond our perception. We live in an age when the computer is a black box and when it goes wrong we can only just switch it on then off again. The second theme is to make what is going on inside the computer more transparent to the user. For example most people are not aware of the private information about them that is being reported from their web browser, to central authorities. While we could inform/train/alarm people about this, the theme of transparency suggests would we could also provide an ambient visualisation such as the Weiser 'Dangling thread'. Equally the programmer does not perceive how inefficient their code is or how hard their garbage collector is working. The transparent computer might display this but in a way that is not intrusive or interrupts the programmers train of thought. For more on this vision, check out this video.

This is a list of current speculations for projects: be quick if you want to do one, the field is moving rapidly forward. Proposals will be considered if they are strongly related to these topics and are sufficiently innovative. Initially the work is biased towards the production of new types of prototype transplant computing systems with a call for stronger evaluation based work to follow on.

Image processing: body trackerFollowing on from an initial trial this project is about constructing software which can track the movement of people from a camera suspended above them. Using the supplied previously recorded video video as a data set, the project  is to construct a real time image processing system (possibly based on processing or OpenCV). The software should be able to track and label any person sized disturbance and emit the tracked locations using OpenSoundControl protocol similar to TUIO or TUIO2 which will stand in for the host application. Once initial motion tracking has been established the system should also provide some mechanism for ‘body click’ a way of indicating ‘clicking’ by a body movement. Outputs will be working software and reports on the accuracy of the system.

• Image processing: tracking hands. Currently OpenCV has the ability to track faces and bodies in profile. There are mechanisms in place to permit training the system to recognise other objects. In this project the objective is to train openCV to track hands and recognise a number of gestures (pointing, OK, stop). This kind of hand gesture recognition could be used to drive a large scale display ( projector behind a window) or control a home entertainment system in a simplified ‘Minority report‘ system.  Outputs will be working software and reports on the accuracy of the system.

• Image processing: tracking fingers. Currently table top computing relies on a large complex projection system and is unable to identify separate users. This project is to create the laptop of table computing, a mobile table based computer able to be used on any table. Core to this is to produce a system capable of tracking hands/arms and identify the finger position on these arms. The project is to take a data set of pre-recorded video  and produce real time software capable of tracking the finger tips. The software should be able to track and label any finger and emit the tracked locations using OpenSoundControl protocol similar to TUIO or TUIO2 which will stand in for the host application. If time permits a squeezing interface may be produced.  Outputs will be working software and reports on the accuracy of the system.

• Tabletop interaction. Many tabletop applications now use  TUIO and TUIO2 as the standard way to drive a tabletop application. This project is to  create and prototype an application which might have some serious use in an non educational context. Using a TUIO multi-touch simulator  is possible to prototype the application on a desktop/laptop. This must go beyond the standard sorting pictures, mapping and concept mapping applications but be relevant to some application area where a number of co-located users work together to the same goal. Suggestions include a shared audio mixing desk, a newspaper editors layout decision making tool.

• Ambient displays. If you work in an office building this may be the one for you. Persuasive computing has a branch looking at supporting and encouraging people to become physically fitter by using computing to reward positive behaviour. This currently assumes that people have got to the stage were they have made the active decision to engage in some formal exercise. This experiment is asking the question what does it take to get people to make that decision or be aware of the issue. The experiment consists of observing how people use the lifts or stair cases using some real time software and using that to drive a visualisation which feeds back to the building the number of people using the stairs and lifts each day. Does such a system changed peoples behaviour?This is a list of current speculations for projects: be quick if you want to do one, the field is moving rapidly forward. Proposals will be considered if they are strongly related to these topics and are sufficiently innovative. Initially the work is biased towards the production of new types of prototype transplant computing systems with a call for stronger evaluation based work to follow on.

• Image processing: body tracker. Following on from an initial trial this project is about constructing software which can track the movement of people from a camera suspended above them. Using the supplied previously recorded video video as a data set, the project  is to construct a real time image processing system (possibly based on processing or OpenCV). The software should be able to track and label any person sized disturbance and emit the tracked locations using OpenSoundControl protocol similar to TUIO or TUIO2 which will stand in for the host application. Once initial motion tracking has been established the system should also provide some mechanism for ‘body click’ a way of indicating ‘clicking’ by a body movement. Outputs will be working software and reports on the accuracy of the system.

• Image processing: tracking hands. Currently OpenCV has the ability to track faces and bodies in profile. There are mechanisms in place to permit training the system to recognise other objects. In this project the objective is to train openCV to track hands and recognise a number of gestures (pointing, OK, stop). This kind of hand gesture recognition could be used to drive a large scale display ( projector behind a window) or control a home entertainment system in a simplified ‘Minority report‘ system.  Outputs will be working software and reports on the accuracy of the system.

• Image processing: tracking fingers. Currently table top computing relies on a large complex projection system and is unable to identify separate users. This project is to create the laptop of table computing, a mobile table based computer able to be used on any table. Core to this is to produce a system capable of tracking hands/arms and identify the finger position on these arms. The project is to take a data set of pre-recorded video  and produce real time software capable of tracking the finger tips. The software should be able to track and label any finger and emit the tracked locations using OpenSoundControl protocol similar to TUIO or TUIO2 which will stand in for the host application. If time permits a squeezing interface may be produced.  Outputs will be working software and reports on the accuracy of the system.

• Tabletop interaction. Many tabletop applications now use  TUIO and TUIO2 as the standard way to drive a tabletop application. This project is to  create and prototype an application which might have some serious use in an non educational context. Using a TUIO multi-touch simulator  is possible to prototype the application on a desktop/laptop. This must go beyond the standard sorting pictures, mapping and concept mapping applications but be relevant to some application area where a number of co-located users work together to the same goal. Suggestions include a shared audio mixing desk, a newspaper editors layout decision making tool.

• Ambient displays. If you work in an office building this may be the one for you. Persuasive computing has a branch looking at supporting and encouraging people to become physically fitter by using computing to reward positive behaviour. This currently assumes that people have got to the stage were they have made the active decision to engage in some formal exercise. This experiment is asking the question what does it take to get people to make that decision or be aware of the issue. The experiment consists of observing how people use the lifts or stair cases using some real time software and using that to drive a visualisation which feeds back to the building the number of people using the stairs and lifts each day. Does such a system changed peoples behaviour?