The development of hybrid videogames involves the integration
of physical, social and virtual aspects, thus
introducing a new dimension of complexity
to the entertainment applications of the
pre-pervasive computing era.
Hence, it is crucial to be able to quickly prototype
and tweak game rules as well as the involved
user interfaces and interacction devices.
Ullmer, B., and Ishii, H. (2000) Emerging Frameworks for Tangible User Interfaces.
IBM Systems Journal, 39(3), pp. 915-931

V.2.0 (January 2013)

ToyVision. Toolkit for prototiping tangible computer board-games from Javier Marco on Vimeo.

Recent expansion of tabletop devices gives rise to a new generation of physical and social videogames which mix traditional board games with the new possibilities of digitally augmenting the area of interaction with computer image and audio. Most of the tabletop games are based on multitouch interaction in which playing pieces are virtually projected on the surface and players manipulate them dragging their fingers on the table.

Several tabletop devices are not only capable of detecting user fingers and hands, but also have hardware sensors capable of identifying conventional objects placed on the tabletop surface and track users’ manipulations. Thanks to this functionality, hybrid tabletop games based on Tangible Interaction are also feasible. By keeping playing pieces in the player’s physical environment emotional impact of videogame is reinforced and digital technology becomes accessible to other user profiles such as very young children, users with disabilities and the seniors. We have created NIKVision, a tabletop device expecially designed for very young children and children with disabilities, taking benefith of object manipulation with these users.

Design hybrid games based on Tangible Interaction brings new ways of interaction with the digital system by taking benefit of the physical affordances of conventional objects (playing-pieces). However, the building of a functional prototype implies to tackle many technological issues.

Although the tabletop hardware can be used to identify and track objects position on the table surface, other kinds of passive and active interactions may require advanced technical skills such as electronics and hardware programming. This situation brings a breach between the design process and the corresponding implementation tasks, (i.e., between designers and developers) which is hindering designers from freely explore all the Tangible Interaction feasibilities of the physical playing pieces.

To tackle the problem, we have created ToyVision, a software framework aimed to provide an easier development environment for prototyping Hybrid board-games for visual based tabletop devices.

ToyVision is based in the Reactivision framework, adding new functionalities to support passive and active manipulations of playing pieces.

  • Passive manipulations: ToyVision identifies and tracks not only users placing and dragging playing pieces on the table surface, but also more complex manipulations such us combining two or more playing pieces, or manipulate mobile parts of a playing piece.
  • Active manipulations: ToyVision is able to act upon the playing pieces, carrying out physical changes on them that address the senses of the user, such us color changes, sound, or autonomous movements during the game. ToyVision supports the Arduino platform to comunicate embedded electronics with the tabletop system.

Any developement environment can be used to code tabletop games based on the ToyVision framework. You just need to implement an standard TCP-IP socket. Messages coming from the ToyVision framework and coded in XML-compliant text, so yoy just need to parse the XML to extract updated status of any playing piece manipulated on the tabletop surface.

ToyVision binaries are supported only for Microsoft Windows systems; althought we provide the source code, so it should be possible to compile it in other systems such as MAC/OS and Linux. Go to the Download section to get and install ToyVision, and go to the Tutorials section to learn to use ToyVision and start creating board-games for tabletops.

ToyVision has been created by Dr. Javier Marco, and the Computer Engineering students:

  • Elisa Ubide
  • Guillermo Navarro
  • Alejandro Enjuanes

ToyVision is aimed to easy the prototyping og computer augmented board-games. It provides with tools to upgrade any conventional playing piece in a control in an hybrid computer game. Toyvision supports passive playing pieces (manipulated by players) as well as active playing pieces (manipulated by the computer system) through the Arduino board (

Tabletop devices based on difusse illumination (, are able to detect and track objects on their surface; thus, it is tecnically possible to create computer board-games that use physical playing pieces. However, at this moment, there are not many proposals of this kind of hybrid videogames. The reason relies in the intrinsic difficulty of developing this games, cause complex algorimts and electronics have to be implemented in order to detect and track playing pieces’ manipulations.

ToyVision lowers the threshold of designing and developing hybrid computer board-games, by offering high processed data of event straight related with playing pieces status during the game.



TOYVision tools and the games developed with it require Adobe Air player instaled in your computer.

TOYVision binaries are compiled to MS Windows 32 bits. However, using the source files and projects, it should be possible to compile the TOYVision tools for other O.S. (such us Apple I/OS)

ToyVision is based on the Reactivision frameworl (, adding new functionalities for blob recognition and Arduino support.  Previous users of the Reactivision toolkit will not find any difference of use in TOYVision framework. Indeed, they just have to launch the framework application, as framework parameters will be automatically configured to detect each playing piece modeled using the TOYVision GUI.

3. TOYVision Widget Layer: This is a new Adobe Action Script 3 class that gives access to all the data needed to recover updated status of each playing piece during the game activity. Provided with this Widget Layer, developers just have to worry about game feedback on the table surface, as they will receive high abstract events of any change in the playing pieces on the table. Games created with TOYVision Widget layer must be created as Adobe AIR projects under any Action Script 3 development environment. The TOYVision Complete package you have downloaded  includes an Adobe Flash CS5 and CS5.5 project ready to be used to create new games. Also, there are some game samples with complete source code that you can use and modify to learn how to create your games using the TOYVision Widget Layer.

Second and last thing you have to learn, is that TOYVision clasifies every playing piece in four diferent categories, and you need to take this into account when designing your own games:


chips Simple Tokens are the most common playing pieces in board games: checkers, marbles, chips… Players arrange a limited amount of playing pieces on the board according to game rules (e.g. Checkers, Ludo, Stairs and Ladders, Roulette…) In general, Simple Tokens can be physically described as small flat cylindrical pieces, all identical with the exception of the colour used to distinguish the piece of each player, or to give different values (money, points…). Usually, most board games need a lot of Simple Tokens to be played, but grouped in few categories (Checkers only uses black and white pieces, Ludo uses four colors…).

This video shows you how to use Simple Tokens in TOYVision:


cardsOther kind of playing pieces get from game rules a very specific role and unique name, which are perceived by the player through their physical appearance. A classic example is the Chess game: the Tower piece has a different appearance and rules than the Pawn. It is possible to have more than one instance of a kind of Named Token in a game (Chess has four Towers and sixteen Pawns), or to have unique instances of each playing piece (in card games, each card is unique in the game, and has a unique name such as Ace of Diamond, Three of Spade …). A Named Token physically identifies the player it belongs to (usually by colour) and the role it has in the game.

This video shows you how to use Named Tokens in TOYVision:


trivialA Constraint Token can be described as a playing piece that acts as a physical constraint of a set of smaller Simple Tokens. The bigger piece can be moved in the game board as any other playing piece, but with the manipulation of the Simple Tokens associated with it, the Constraint Token gets new meanings. For example, the playing piece of the Trivial Pursuit game is composed of six triangular small pieces (Simple Tokens) which can be placed inside a bigger circular piece (Constraint Token) which represents the progress of each player during the game. In this particular case, handling of the playing piece is based on associating (placing/removing) Simple Tokens inside a Constraint Token. Other playing pieces require more complex manipulations, for example, in the Roulette game, players spin a little marble (Simple Token) inside a circular plate (Constraint Token); in this case the playing piece is based on manipulating (moving/rotating) a Simple Token inside the Constraint Token.

This video shows you how to use Associative Constraint Tokens in TOYVision:

And this video shows how to use Manipulative Constraint Tokens in TOYVision:



Finally, there are other playing pieces which do not have a constant shape, as they change with the manipulations of the player, as they are made of malleable materials, such us clay, cardboard, cloth… In a table, players can use these materials for creating new shapes and by combining them, new structures are created.

This video shows you how to use Deformable Tokens in TOYVision:

  • affectivelab
  • diis_unizar
  • RSS