System extracts spoken language from video recording, converts it to searchable text

A brand new strategy to looking by video content material has been developed by a staff in South Korea. The system, described within the Worldwide Journal of Computational Imaginative and prescient and Robotics, extracts spoken phrase from a video recording, converts it to textual content, after which makes that textual content searchable. Importantly, the system thus doesn’t depend on embedded key phrases nor curated tags or hashtags to be related to the video content material.

The strategy clearly depends on the dialogue or spoken commentary of an merchandise being related to the scenes within the video that customers would possibly want to search. It’s, in fact, superfluous if the video has subtitles already baked in. However, it will likely be a boon for customers wishing to look the hundreds of thousands of hours of video accessible in databases, on streaming providers, and elsewhere on the web and may very well be used to assist catalogue movies.

Kitae Hwang, In Hwan Jung, and Jae Moon Lee of the Faculty of Laptop Engineering at Hansung University in Seoul, have developed an Android app to be used with acceptable smartphones. It’s price noting, nevertheless, that there’s a minimum of one different app with the identical title, so ought to this app be made accessible within the Google Play Retailer for Android apps, it’s prone to require a change of title.

The brand new app works by extracting audio from movies utilizing the FFmpeg code and changing it into textual content in 10-second increments. This, the staff explains, creates a searchable timeline for the video. Superior speech recognition know-how then generates a transcription of these audio segments, that are listed on the video timeline.

For a 20-minute video, the method is full in simply two to a few minutes and runs within the background whereas the video performs. The staff factors out that customers can then seek for particular phrases and discover all mentions within the video.

The app may have purposes in training, information evaluation, and different information-dense video the place fast entry to particular info is required. As an illustration, college students reviewing lecture recordings or journalists looking for particular statements in interviews may make use of this app. There are lots of extra eventualities the place it will be helpful to have the ability to search video on this method.