As more tech companies angle themselves to better combat the flow of fake news and hate speech, Facebook seeks to position itself as a leader in the fight. The social media giant has announced a machine learning tool dubbed Rosetta that aims to understand text written in pictures and videos.
The technology isn’t perfect according to Facebook, but it’s improving on the previously flawed methods of automatically comprehending the wrongful text spread through memes on social media. Over a billion public images are sourced by Rosetta each day, in multiple languages, and the software runs them through a text recognition model to accurately grasp the context of each post. The next step is to train Rosetta to automatically filter through videos frame by frame, though typical text extraction “is not scalable … and would only lead to wasted computational resources.” Instead, the team is experimenting with 3D convolutions to select key frames.
Read more details on Facebook’s code site.
Facebook recently revealed a quartet of gender choices for its forthcoming dating app.