- CSP Research & Development - http://rd.csp.it -

Multi-touch Table

Multi-touch displays are the natural evolution of the old touch screens as they are capable of recognising multiple touches at the same time. This way of dealing with the hardware (and applications) opens up a host of new opportunities for human-computer interaction, enabling the objects on a screen to be directly manipulated by the user. For this reason these techniques are also known as natural user interfaces.

There are different ways to create multitouch screens, depending on the context of use and the size required for the display. The techniques that allow the realization of  multi-touch hardware, are based on different solutions such as capacitive, resistive, motion, orientation, pressure and so on. The one we chose to implement our table is a computer vision based technique and is called Laser Light Plane (LLP). This vision based solutions are the most known and used when you need to interact with large screens because the scalability, the low cost and the easy to build feature; in addition to this there are other four major (computer vision based) techniques that allow  the creation of a stable multitouch hardware system: Frustrated Total Internal Reflaction (FTIR), Rear Diffused Illumination (Rear DI), LED-Light Plane (LED-LP) and Diffused Surface Illumination (DSI).

How LLP works.

Each vision based techniques consist of a camera, an infrared light source and a display. The infrared light source, in the LLP solution, is generated by IR-laser and this infrared light is shined just above the surface to create a laser plane of light that is about 1mm thick and is positioned just above the surface. When the finger touches the display, the laser plane hits the tip of the finger and the system registers an IR blob so infrared lasers and LLP method are an easy and usually inexpensive way to create a MT setup. Most setups go with 2-4 lasers, positioned on the corners of the touch surface. The laser wattage power rating (mW) is related to the brightness of the laser, so the more powerful the laser, the greater the brightness of the IR plane. The common light wavelengths used are 780nm and 940nm. Laser modules need to have a line lense generator on them to transform the beam into a light plane; the 89 and 120 degree line lens are most commonly used to reduce the number of lasers necessary to cover the entire touch surface. The advantages of this technique are that no compliant surface is needed, so you can use any transparent material for the surface like glass and not only acrylic; no LED frame and no closed box needed and could be cheaper than other solutions. The disadvantages are that traditional objects and fiducials are not trackable and is not pressure sensitive (is not able to detect the force with which someone presses on the surface but only the contact).

Laser Light Plane [1]

Laser Light Plane

 

For more detailed information and reference, please visit to this NUI Group Wiki page about LLP technology [2].


CSP Mutlti-touch table.

In order to test multitouch applications at the development stage, CSP equipped itself with a dedicated hardware solution. When it comes to large screens (non-portable devices), there are various commercial platforms available – e.g. Microsoft Surface. However these are closed solutions that do not integrate with other software programmes and do not allow modifications to the configuration of the hardware, as well as representing a considerable expense. For these reasons CSP’s applied research has focused on creating and integrating an “open source” hardware. For multitouch applications a development framework based on Adobe Flash was used, chosen for its versatility and compatibility with the vast majority of operating systems. The applications developed are also based on interaction patterns typical of multitouch environments, such as pinch to zoom, tap to open and drag to move. In its research work CSP has developed a series of applications designed to respond to the needs of a range of different settings and sectors, from library services and public information, to teaching tools for learners with special needs, from the aviation industry to tourism, to entertainment, with multitouch games. Our prototype (built in 2009) is a table with a surface composed by a retro projection film sandwiched between two tempered glasses. On each corner of the surface there is an infrared laser with a line lens generator. A short throw projector, placed under the table, display the images and a modified infrared camera captures the users generated blobs. A mirror assists the projector in redirecting the image on the multitouch surface.

The surface has been produced in the form of a horizontal table, but the versatility of LLP technology means it can be used to construct vertical interactive surfaces in larger sizes, while keeping costs low. Indeed 2010 saw the creation of an interactive wall measuring 100” (around 2 metres wide and 1.5 metres high), with costs comparable to those of producing a table, despite being four times larger.

Multitouch table hardware specs:

[3]

CSP Multi-Touch Table

 

Lasers Safety

Safety when using lasers of any power is important. Infrared lasers are used to achieve the LLP effect, and these lasers carry some inherent risk. Infrared lasers are imperceptible to the human eye and therefore activate no link response, allowing for greater damage to the retina. A Class IIIB laser is hazardous if the eye is exposed directly but a line lens is used to expand a laser’s line into a plane. This line lens reduces the intensity of the laser from IIIB to Class I. Exercise anyway common sense and be mindful of where the laser beams are travelling. If extreme caution is exercised, LLP setups are quite safe.

Software

To implement multitouch applications, you can choose between different programming languages. Each technology has to know when and where the fingers are on the screen, so a protocol is needed in order to translate blob information into touch point messages. We used  the TUIO (http://www.tuio.org [4])  protocol as a middleware between the multitouch hardware and the user application. TUIO is an open framework that defines a protocol (and API) to standardize an interface that makes it possible to exchange touch events between a detecting device and an application or a user interface. TUIO is built on top of OSC (Open Sound Control) that is a protocol, defined in 1983, used to interconnect PC to multimedia devices. It represents an Internet based alternative to the MIDI interface. As OSC, the TUIO protocol is based on the exchange of UDP messages between the device and the client  application.  The TUIO protocol encodes control data from a tracker application (e.g. based on computer vision) and sends it to any client application that is capable of decoding the protocol.

The TUIO protocol defines two main classes of messages: set messages and alive messages. Set messages are used to communicate information about an object’s state such as position, orientation, and other recognized states. Alive messages indicate the current set of objects present on the surface using a list of unique session IDs. Since TUIO is implemented using OSC, it follows its general syntax:

/tuio/[profileName] [messageType] [Parameters]

The profileName is used to indicate the sensing capabilities of the devices:

The “obj” profile is used to recognize objects, the “cur” profile only to obtain cursor information.

The following tables describes the basic messages used by the TUIO protocol when using a 2D Interactive Surface.

messageType and parameters Description
source name Identify the message source
set id x y X Y m States the update for an object/finger providing the numeric identification, the x,y position, the X,Y motion vector and the acceleration (m)
alive id1 id2 … idn This is an alive message sent to indicate what are the
objects/cursors that still are on the surface
set id x y X Y m States the update for an object/finger providing the numeric identification, the x,y position, the X,Y motion vector and the acceleration (m)
fseq num This message sends a periodic sequence number. This is provided because actually UDP does not guarantee that the messages are received in the same order in which are sent.

The TUIO implementations are available for most common programming languages and media environment:

 

[5] [6] [7] [8]