Project Overview
-
Features
This technology uses multiple speakers to erase sound waves, which normally propagete in all directions, so that unnecessary sounds are inaudible. This way, sound can be transmitted only to specific areas (local sound synthesis). Not only it can transmit sound to just one area, but it can also transmit different sounds into multiple areas (multiple sound spot synthesis). Because it does not use ultrasonic waves, there are no health risks, and the sound quality degradation is kept to a minimum.
-
SEEDs
This technology can be used to play an announcement that is audible only in a specific place, make multiple people hear different sounds, and more. In addition, by combining it with the real-time speech translation technology developed by NICT, it is possible to synthesize real-time translated synthetic voices in different languages. It can also be applied to entertainment using sound.
-
Goals
This technology developed by NICT can change the world once it is placed in actual locations. We are looking for partners who will use multiple sound spot synthesis technology in society and help create new sound spaces together.
Video Explanations
▲Click on the video to play it
▲Click on the video to play it
About Multiple Sound Spot Synthesis Technology
Sound field control technology that flexibly creates audible and inaudible zones
Sound normally propagates in all directions because of vibrations in the air. Multiple sound spot synthesis technology erases some of the sound waves so they are inaudible, making it possible to implement local sound synthesis, which can synthesize sound only in a specific direction, or multiple spot sound synthesis, which can syntyesize different sounds in multiple directions.
The basic mechanism is to use multiple loudspeakers to mutually interfere with the sound emitted by each loudspeaker. By calculating and reproducing the playback signal of each loudspeaker based on the wave equation, which represents the propagation phenomenon of sound waves, sound waves can reach only "audible areas" and cancel out sound waves that spread to areas defined as "inaudible areas".
The reason it is called "multiple spot" is that you can set up multiple audible locations. For example, suppose that normal loudspeakers are placed facing four different directions, and each loudspeaker plays back a different sound. Since the loudspeakers spread the sound not only forward but also in other directions, the sounds will mix with each other and become difficult to hear.
Using multiple loudspeakers arranged in a circle (16 channels for the prototypes) and multiple sound spot synthesis technology, different sounds are transmitted to four areas, and sound that travels outside the desired area are erased. This way, it is possible to make only certain sounds audible in certain areas. For example, you can make an area where only sound A is audible and an area where only sound B is audible.
By arranging the devices and control system, it is possible to transmit sound to not four areas, but to one area or many areas. We are also developing proximity area sound synthesis technology that can not only manage areas, but also make it so that sounds can be heard when a listener is close to a loudspeaker and become inaudible when the listener moves away.
Features
Can transmit sound to a specific target area with high sound quality and precision. Because it does not use ultrasonic waves, there is no risk of health problems.
Before, there have been directional speakers that can also transmit sound in a specific direction by using ultrasonic waves. They can transmit sound using the properties of ultrasonic waves, which have high directionality (the property of propagating in a specific direction).
Ultrasonic arrays often have sound distortions and skipping, and they emit loud sounds of over 150 db that cannot be heard by humans. There are concerns that they can harm human bodies and affect animals that have a wide range of hearing.
Multiple sound spot synthesis technology does not reduce sound quality because it plays sound sources through speakers normally. In addition, because it's not loud, it doesn't harm humans' health or affect animals.
We have made a prototype that is as small as about 14 cm. We offer demonstration devices that can fit inside a suitcase. Please contact us to try them.
Multiple sound spot synthesis technology is used with multiple loudspeakers arranged in a circle or straight line. We are working to make loudspeaker arrays as small as possible so that they can be easily installed in even more locations.
The photo shows a prototype of a small circular loudspeaker array that was developed in cooperation with Kitanihon Onkyo, Co., Ltd. It has a diameter of about 14 cm and consists of 16 loudspeakers with a diameter of 2.5 cm arranged in a circle. It can fit inside a suitcase along with an amplifier, stand, and other equipment, so you can hold a multiple sound spot synthesis demonstration anywhere.
The photo shows a linear array of 64 loudspeakers. Whereas a circular array can partition sound synthesis areas in a circular manner, a linear array can partition them horizontally.
We have prototypes that demonstrate our ability to make devices this small. Moving forward, we will work with our business partners to make them even smaller and further improve sound quality.
It can be used with multilingual voice translation technology used by VoiceTra and be managed in real time using a tablet. It can be used for an even wider range of situations.
It can be used with multilingual voice translation technology or other technologies and be managed using a tablet, allowing an even wider range of applications.
VoiceTra is an app that uses AI to translate speech into multiple languages and use speech synthesis to play it back in each language. The speech synthesis technology was developed by Okamoto in NICT, who also developed the multiple sound spot synthesis technology, and uses cutting-edge neural network technology to synthesize speech that sounds almost completely natural in real time.
As an example of a use case, an announcement or speech can be translated into multiple languages using multilingual voice translation, then converted into synthetic speech that is played back locally in many areas by using multiple sound spot synthesis. This way, listeners can hear the announcement or speech in different languages depending on where they are standing.
- VoiceTra is available for demonstration purposes. To use it for commercial use, please ask us for a separate consultation. We also offer consumer products and services that use VoiceTra.
It is also possible to manage synthesized sounds and the areas they are played in using a tablet or smartphone in real time. This makes it possible to move areas even if the listeners do not move.
Value Provided
Completely contactless without earphones or headphones,
contributing to the new normal
Usually, having specific people hear certain sounds required them to wear earphones or headphones. Multiple sound spot synthesis technology makes it possible to do that without touching any equipment.
Not only is it suitable for today's new normal lifestyle, but it can also be used in situations where it would be difficult to put on a device (e.g., situations with many unspecified people or listeners that are not humans).
The same sound quality as normal loudspeakers,
creating the future of entertainment
Multiple sound spot synthesis is no different from normal loudspeaker playback except that sound spreading out of range is eliminated, and there is no degradation of sound quality due to processing for localized sound spot synthesis. It can be used in situations where high sound quality is required, such as entertainment of songs and tunes.
[The technology we want] Sounds do not leak outside the range where you want them to be heard. This technology contributes to the digital transformation of society
Thanks to this technology, loudspeakers can play back sound that is inaudible outside a specified area. Sounds can be transmitted to a limited area or to several areas without worrying about its effect on the surroundings.
With this, messages can be translated into many different languages at the same time and played back to people of different nationalities. Areas can be split up, and different information can be spread to each one through sound. That way, it can contribute to digitally transforming jobs that normally require people.
SEEDs
At large exhibition booths
Announcements and sounds can be played in booths and sales floors without being heard anywhere else. You can play promotional announcements without worrying about it being heard anywhere else or blending with other sounds and becoming hard to discern.
At arcades and recreational facilities
Even if there are loud sounds coming from game or elsewhere, the announcements will be audible only in certain areas and will not blend with other sounds, so people can concentrate on what they are doing. Announcements, such as those looking for a lost child, will no longer be hard to hear because of loud games.
It is also possible to use it as a gimmick for games by controlling it with a tablet.
With autonomous driving
The use of autonomous driving is expected to spread in the future. By playing different sounds for different seats, it will be possible to adapt to new features and mobility services.
For example, it is possible to play navigation directions to a driver while playing music or movies to others in the car.
In factories, construction sites, and other dangerous places
In factories, construction sites, and other areas that can be dangerous, messages can be played to specific people, preventing accidents and injuries.
Another advantage is that it can let people with poor eyesight identify dangerous areas.
Goals with Partners
Universal Communication Research Institute
Advanced Speech Translation Research and Development Promotion Center
Senior Researcher in the Advanced Speech Technology Laboratory
By using multiple sound spot synthesis technology, it is possible to shut out sounds spreading outside the area where you want to hear them, as if the area is separated by an invisible partition. We invite you to experience and enjoy this mysterious sound space, where sound can be heard and not heard simply by moving in and out of the area.
We are researching this technology because we think it is "interesting" and we would like to convey the fun of it to others. If you also find this technology "interesting," we would like to work together with you to find ways to utilize it. We need the help of our partners to implement the technologies created in our laboratory in various scenes of society and to provide benefits to as many people as possible.
It is a simple technology that allows people to hear where they want to be heard and not hear where they do not want to be heard, but because of this, it has a wide range of applications. From public use to business and entertainment applications, the possibilities are endless. It is also possible to contribute to society through social welfare initiatives that utilize "sound" to help the blind and visually impaired, and through animal welfare initiatives that prevent animals sensitive to sound from hearing announcements at zoos and aquariums.
We are looking forward to hearing from companies that are willing to work with us to create a "new sound space" by using this technology.
About the Institute
The National Institute of Information and Communications Technology (NICT) is Japan's sole public research institute specializing in the field of information and communications. It promotes research and development in information and communications technology with a comprehensive point of view, from the foundations to practical use. It cooperates with universities, industry, municipal governments, other research institutes in Japan and abroad, and other organizations with the goal of spurring innovation and using the results of its research and development to widely contribute to society.