July 4th Product Release
Ever since we launched Aible Sense on Valentine’s Day this year (our “features you will love” release), customers loved the automated features in Sense such as automated data cleansing, automated feature creation, automated data evaluation, etc. They also loved the fact that all of this happened by bringing the code to the data that remained in their own private cloud, via Python code in Jupyter notebooks that they could access and reuse. They loved the security model with built-in k-anonymity that meant Aible users could not access the raw data that they evaluated, analyzed, or created models based on.
But, customers asked: “why can’t we do manual data engineering the same way you are doing automated data engineering?” For our Fourth of July release (“features that set you free”) we decided to give them that ability, while still leveraging Aible’s augmentation, data security, and end-to-end auditability capabilities.
These features are automatically available to Aible Explore and Optimize users today at no additional charge. Existing Sense-only users would need to upgrade to a Sense+ license to get these augmented data engineering capabilities.
Augmented Data Engineering
Even if you remove the race or gender variable from the data set, the AI will search for proxies for those variables. If women had a lower approval rate for loans, for example, then high school teachers will have a lower rate, because high school teachers are disproportionately female. If men had a higher rate of loan approval, then construction workers will have a higher rate, because construction workers are disproportionately male.
Augmented data engineering features are now available with Aible Sense that allow you to combine, transform and clean your data. Leverage transformations like join datasets, deduplicate rows, filter rows, append rows, create derived fields, conditional if-then-else statements, pivot and unpivot, standard column updates (drop, rename, change type), etc. All of the transforms are maintained as code templates and only applied to the data when requested. All transforms are done using serverless techniques at the lowest possible cost, or for really large datasets, done on servers which are run only for the duration of the transformation code execution and then shut down immediately.
Of course, all of the fully automated Aible capabilities like automated data cleansing, automated feature engineering, etc. remains available in Sense. 80% of the time, the automated approaches will be sufficient, but now you are free to use manual approaches for the remaining 20%.
Transformation Stores
Once you create a complex data transformation you never have to do it again. Sense allows you to leverage transformations from existing datasets or transformation templates and apply to other datasets. Feature stores have become common in the market, and have always been a part of Aible. But, now we can templatize and reuse complex transformations such as pivot / unpivot.
Evaluate multiple datasets
Instead of evaluating datasets one at a time, Aible helps evaluate multiple tables at once. Use Sense to figure out which dataset offers a stronger AI signal for your projects. For example, you may have hundreds of possible datasets available in your Snowflake instance. Want to quickly figure out which ones you should focus on to generate useful business insights or machine learning models? Just choose multiple tables/datasets and let Sense evaluate them all. Because Sense runs serverless (wherever possible) it can evaluate many datasets in parallel, and at extremely low cost. A typical evaluation takes 3-5 minutes and costs less than $0.05. So, instead of wasting hours and maybe days of your life on evaluating a dozen datasets, you can choose them all and let Sense tell you where to focus.
The Power of Aible Explore, now in Tableau
Continuing with the “features that set you free” theme, all of the power of Aible Explore is now available inside AIble Extension for Tableau. Click on a visualization in Tableau and see what the dozens of models used by AIble think of the visualization: is it useful or misleading? What are the related visualizations Aible recommends that you look at? You can of course also just search for insights related to any variable or value.
Hundreds of Other Enhancements
While these are some of the major new features in the Fourth of July release, it includes hundreds of other feature enhancements and bug fixes. Aible customers, please contact your Customer Success team for a full list of enhancements.
