Lingjiao Chen

Bio: Lingjiao Chen is a PhD candidate in the computer sciences department at Stanford University. He is broadly interested in machine learning, data management and optimization. Working with Matei Zaharia and James Zou, he is currently exploring the fast-growing marketplaces of artificial intelligence and data. His work has been published at premier conferences and journals such as ICML, NeurIPS, SIGMOD and PVLDB, and partially supported by a Google fellowship.

Talk Title: Understanding and Exploiting Machine Learning Prediction APIs

Talk Abstract: Machine Learning (ML) prediction APIs are a fast-growing industry and an important part of ML as a service. For example, one could use Google prediction API to classify an image for $0.0015 or to classify the sentiment of a text passage for $0.00025. While many such services are available, the heterogeneity in their price and performance makes it challenging for users to decide which API or combination of APIs to use for their own data.

In this talk, I will present FrugalML, a principled framework that jointly learns the strength and weakness of each API on different data, and performs an efficient optimization to automatically identify the best sequential strategy to adaptively use the available APIs within a budget constraint. Our theoretical analysis shows that natural sparsity in the formulation can be leveraged to make FrugalML efficient. We conduct systematic experiments using ML APIs from Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including facial emotion recognition, sentiment analysis and speech recognition. Across various tasks, FrugalML can achieve up to 90% cost reduction while matching the accuracy of the best single API, or up to 5% better accuracy while matching the best API’s cost. If time permits, I will also discuss recent follow-up studies on API performance shifts and multi-label APIs.