Statisticians like to categorize errors into Type 1 (False Positives) and Type 2 (False Negatives). But experienced data scientists know that the most important errors in AI are "Type 3" errors. The term was coined in 1948 by statistician Fred Mosteller, the founding chairman of Harvard’s statistics department. Mosteller defined a Type 3 error as “correctly rejecting the null hypothesis for the wrong reason." In colloquial terms, a Type 3 error means giving the right answer to the wrong question.
With the commercial adoption of AI, much time, money, and effort has been spent building mechanisms to fit bigger and bigger models using larger and larger data centers. For a certain class of problems this approach is the right answer. For many others, it is a Type 3 error. The problems that demand massive data infrastructure tend to be those where the goal is either highly personalized service, or the need to classify items from a large catalog. For many more traditional data problems there are diminishing returns to huge models. Despite the tremendous resources devoted to AI infrastructure, the main problem with AI and data science today isn't that the models are not good enough. It’s that the models are not getting used.
A 2019 MIT-Sloan/Boston Consulting Group survey found that 65% of companies reported that they are seeing little or no value from the AI investments they’ve made in recent years. Nearly 40% of organizations that have made “significant investments” in AI say they have not seen any business gains. The reality is that many companies are spending enormous amounts of time and money on AI and seeing little or no payoff. AI models that take many months to develop often wind up never seeing action in production. Without business adoption, they have produced no value.
Aible takes a different approach. Aible provides a forum where data scientists and business users collaborate as a team. Business users are an integral part of the AI process; they have a say in setting assumptions, providing the business expectations, and giving feedback about the models as circumstances change. This alleviates the problems data scientists commonly have in trying to explain the results of the modeling and have the business people understand it.
Frontline data scientists have to deal with modeling complexities that business users never see and wouldn’t understand if they did – choosing which variables to include and which to leave out, testing the model to make sure it’s valid, making sure the model doesn’t violate any regulatory restrictions. A common reaction to that process by business people is: Why is it taking so long?
Aible removes that friction by automating much of the rote part of the model fitting process, by simplifying the administrative work around “owning” a model, and by translating model metrics into language business people understand – dollars and cents. What drives business adoption of AI is the promise of more revenue and lower costs, not a high F1 score. Aible makes it easy to go straight from the modeling step to deploying the model and having it used in production. It also automates the monitoring process to signal when models need to be adjusted.
Aible optimizes models for business impact, with the incredibly useful byproduct that the impact gets measured in the first place. This is obviously useful for business leaders looking to justify investments in their data science teams, and it is a veritable gold mine for data scientists.
"Most AI companies are built to get data scientists predictions. Aible is built to get them promoted."
Ironically, the people who are among the most quantitative in their organization often struggle to put a dollar value on their work. This can put data scientists at a disadvantage during performance reviews. When raises are determined by contributions to the bottom line, how should that improved F1 score be judged against a sales or marketing initiative that brought in an easily understood $X million dollars? Aible automatically quantifies a data scientist’s contributions in the financial terms best understood by their business stakeholders.
Conventional AI struggles with adoption because of a fundamental Type 3 error. Too much time has been spent precisely answering questions about how to make better models, while ignoring the high probability that the models won’t be used in the first place. Aible is built from the ground up to aim at the right target, to get models adopted, and to deliver value to the business.