Abstract |
Advancements in Mixed Reality (MR) technologies and Tangible User Interfaces
(TUIs) have led to the development of dynamic and immersive systems, offering
unique and innovative user experiences. Within the domain of education,
these technologies have proven to be extremely beneficial in enhancing student
motivation and engagement, promoting collaborative learning and creating more
accessible educational environments. Despite the individual attention received in
previous research, the fusion of Mixed Reality with tangible physical object interaction
is an area of study with considerable potential, and it remains an active
domain of exploration.
This work aims to address this gap and explore the potential of such integration,
combining MR technologies with physical objects’ interaction. By leveraging
advanced technologies that allow the computer and sensors to hide within the
environment itself, it aims to cover a wide range of application domains, from educational
application to professional settings. In this context, the AugmenTable
was designed as a projection-based system that integrates tangible interaction with
physical objects. A key component of the system is a touch-sensitive tabletop surface
that allows intuitive touch-based interactions either with the surface itself or
with printed material. The system is able to track physical objects featuring fiducial
markers, currently tested with both paper cards and 3D objects. The paper
cards are enriched with printed information, and the system seamlessly integrates
multimedia content, encompassing images, videos and 3D models, onto the printed
materials and their surrounding area. The 3D printed objects are designed to resemble
everyday objects familiar to the users, such as arrows or flashlights. These
objects function as interactive tools within the system, emulating their real-world
counterparts’ functionality. For instance, the flashlight object emits virtual light,
illuminating otherwise concealed digital elements, in keeping with its real-world
counterpart.
The tabletop surface is surrounded with LED light strips that change color
based on the projected content, providing an immersive user experience. Additionally,
the system seamlessly integrates with a Mixed Reality Head Mounted
Display (HMD), offering users a greater level of immersion. Through the HMD,
users can experience the dynamic content in a 3D virtual world, allowing them to
feel fully engaged in the AugmenTable experience. Projected elements are transformed
into 3D objects that appear to stand on, float over or emerge from the table
surface, right before the users’ eyes. The 3D digital content remains closely associated
with the digitally projected and physical elements, responding dynamically
to users’ interactions with both the system and physical objects.
This thesis presents a software infrastructure that facilitates AugmenTable’s
capabilities. The software infrastructure is designed based on the microservices
architecture pattern. The software is built on two fundamental groups of components:
the AugmenTable Core and the AugmenTable Player. The AugmenTable
Core is responsible for initializing and orchestrating a set of core components responsible
for backbone services. These services encompass critical functions such
as object detection, communication between components, the distribution of multimedia
content and the manipulation of the artifact (e.g., LED lights). The AugmenTable
Player comprises various services responsible for the dynamic creation
of digital elements, the user interaction with the system and the physical objects,
the handling of the physical objects and the seamless translation of the digital content.
Moreover, the AugmenTable MR Companion operates in parallel with these
components, offering supplementary features, enhancing the user’s immersion and
engagement when needed. To illustrate AugmenTable Player’s capabilities and
applications, a diverse range of scenarios is presented, showcasing its versatility
across various domains and system functionalities.
AugmenTable fosters collaborative learning as well as personal exploration and
provides an intuitive and interactive learning experience, turning it into a versatile
tool and a valuable asset that can be employed in various contexts, such as formal
and informal educational applications, cultural heritage applications, or even
professional presentations.
|