Culture

Translate light and shadow into rich sounds

2025-06-10   

I am a researcher in the field of computer science, and our team has been deeply involved in the field of information accessibility for a long time. The experience of walking into the Chinese Braille Library completely changed my research direction. That day coincided with the screening of an accessible movie, and in the darkness, the screen began to display images. The "broadcasting tone" playing in a corner of the theater outlined the outline of the story for the audience in advance. The image is "translated" into sound, and visually impaired viewers immerse themselves in the world of light and shadow just like ordinary people. This experience cannot be replaced by listening to recordings or reading text. After the screening, the exhausted yet satisfied expression of the production team touched me. They need hundreds of hours to handcraft an accessible movie - how I wish this heavy love could have strong wings. On the return journey, team members engaged in heated discussions in the bumpy carriage, with keyboard sounds and sparks of inspiration bursting forth together. At that moment, we were determined to accelerate the transmission of love through artificial intelligence. The core functionality of our EagleMovie intelligent production system is derived from the precise collaboration of three AI engines. The first to appear is the "Gap Catcher", which combines speech recognition and text recognition technology to accurately locate silent intervals in movie soundtracks for inserting voiceovers. Then the 'visual commentator' began working, which is a core module based on a multimodal large model that can understand the speeding cars, falling cherry blossoms in the picture, and even interpret the tearful smiles of characters. What makes me most proud is that it can describe the scene with literary language like 'the wind rolls up her bright red scarf, like a flame that refuses to extinguish'. After the intelligently generated text passes strict verification, the 'voice magician' begins to sing. The speech synthesis system we trained can adjust the speed and emotional concentration, seamlessly integrating the commentary voice with the movie soundtrack. The work that used to require professionals to spend a week can now be compressed to a few hours with the assistance of AI. When the first batch of accessible movies were delivered to visually impaired friends through the Braille library, the voice feedback we received was choked up: "So the sound of Superman's cloak fluttering is like this!" At Zhejiang Special Education Vocational College, visually impaired students refreshed my understanding. A boy touched his Braille notebook and said, "Teacher, do you know why I always sit in the front row of the classroom? When I 'listen' to movies, I need to engrave every character's footsteps in my heart. What they crave is not only the story, but also the right to resonate with social emotions through images. This desire has turned into a heavy trust, weighing on the shoulders of each of us developers. The current AI's understanding of complex scenes in TV dramas is still insufficient, and real-time commentary in live streaming scenes is a huge challenge. In a certain test, the system mistakenly described the jade pendant in the ancient costume drama as a mobile phone, which made us realize that the general visual understanding ability still needs to evolve. More importantly, how to ensure that visually impaired elderly people in Shandong and blind children in Shanghai can receive explanations that are in line with their own language habits? The challenge of personalized adaptation is driving us to explore more sophisticated algorithms. When more volunteers start recording sound for accessible movies in their hometown dialect, and when video platforms open AI commentary plugin interfaces, the temperature of technology will eventually melt the ice. This road has no end, but every time I receive voice thanks from visually impaired viewers, I am convinced that the goals we are pursuing are gradually becoming a reality. (New Society)

Edit:Momo Responsible editor:Chen zhaozhao

Source:

Special statement: if the pictures and texts reproduced or quoted on this site infringe your legitimate rights and interests, please contact this site, and this site will correct and delete them in time. For copyright issues and website cooperation, please contact through outlook new era email:lwxsd@liaowanghn.com

Recommended Reading Change it

Links