Modern Machine Learning operations have made it possible to label and manage vast datasets, train models with billions of parameters, and monitor model performance with ease. Despite that, extensively testing models and carefully identifying their operational envelopes is still more of an academic, rather than an engineering activity.
Efermerai wants to bring the same rigor and confidence to AI as testing brought to the field of software engineering. Building machine learning models that can reliably and safely process unseen edge cases is one of the biggest challenges that every single machine learning team is currently facing. The issue is even more acute in the robotics domain where working with videos taken in the wild is the norm.
“At Efemarai, we are building a machine learning testing platform, or CI for ML, which can automatically stress test models, identify regions of poor performance and help machine learning teams fix numerous issues before they affect their users,” says CEO Svet Penkov.
“As part of the ELISE 2nd open call we are focusing on developing the capabilities of testing models that work with entire video streams instead of individual frames. Working with videos brings not only its engineering challenges, but also touches upon core research problems such as designing and developing temporarily consistent video perturbations.”
Although their platform is currently targeted towards ML engineers and researchers, Efermerai has wider relevance related to AI safety and reliability. “Therefore,” says Penkov, “we are excited to explore how our platform can support entire organizations in their journey of building and shipping safe and reliable models.”
Find out more about Efemerai and their work here and follow them on LinkedIn here!
More about the ELISE 2nd Open call and awarded companies is available here.
