| Abstract |
In last decade there has been a lot of research on immersive audio capturing and reproduction. However, a complete communication system that utilizes immersive audio in real-time does not exist. In this work, we realize ImmACS, a complete communication system that allows the capturing and reproduction of high-quality immersive audio in real-time. ImmACS is based in a computationally efficient yet robust technique, that utilizes a circular microphone array for audio capturing, and headphones or loudspeakers for audio reproduction. Circular arrays overcome the inherent ambiguities of linear arrays and provide estimations in the full [0°, 360°) range in the 2D space. The communication model supports multiple, concurrently active clients, that can simultaneously transmit and receive immersive audio. The model requires the mediation of a server for the distribution of the streams. The immersive audio stream consists of one audio channel accompanied by spatial meta¬data, which we explicitly include in every packet as side information. Optional video streaming is also supported.
We implement Immacssip, the client of ImmACS, by modifying Baresip, an open-source VoIP client. We realize all the immersive audio functions in Libim-macs, a library we designed for optimal real-time performance and flexibility. We incorporate Libimmacs into Baresip and extend its configuration interface accord¬ingly. Furthermore, we modify Baresip so that it is able to receive multiple streams per call, so as to allow the adjustment of the stream of each peer separately. Lastly, we utilize multiple threads for the decoding of the streams, so as to increase the throughput of the system in multi-core systems.
Bareserver is the server of ImmACS and is also based on Baresip. Each client connects to Bareserver directly, exploiting the fact that Baresip is by default a VoIP client. However, we have disabled the transmission of the local streams at Bareserver; it only relays the packets between clients connected to the same SIP account. The packets are relayed completely untouched, with no transcoding or down mix taking place, making the server's presence transparent to the overall communication. In addition, we utilize several worker threads that perform the relaying of the packets, so as to take advantage of the multi-core systems.
To facilitate the manipulation of the audio streams at the client, we provide Immacs Control, a flexible graphical interface. Immacs Control is an autonomous application that uses network sockets to communicate with the client. It allows the monitoring and adjustment, in real-time, of the direction and volume of the audio sources. All signal processing is performed at the client, while the interface only sends meta-data describing the audio filters.
Finally, to accommodate for system testing, we provide the ability to capture and reproduce audio from and to files at disk in real-time. In this way, a fully controlled input and output environment can be set easily, without the need for external hardware. In the same basis, we provide a special "echo" mode of function, where the server transmits the packets back to their senders.
|
Collections
