Skip to content

Duet Competitive Advantage

  • Editable Schema: In machine learning, it is challenging to come up with a labeling schema (e.g. your customer support categories) without having seen enough examples of the documents in your data you want to analyze. The classical way of discovering the schema is to look at some samples of the data and decide one such initial schema, write some annotation guidelines based on what you have seen in the sample data and send it to some human annotators to label the data. The human annotators usually discover new patterns in the data that they cannot label in accordance with the annotation guidelines. The human annotators also get confused and introduce noisy labels. In this situation, the annotators go back to the product manager asking for guidance who in turn changes the schema based on the newly discovered patterns. The annotators have to start labeling from scratch against the new data schema. This process is quite lengthy, erroneous and costly. Duet presents a unique solution to this problem by enabling users to build their schema incrementally as they explore the data. Duet enables users to define their custom hierarchical schemas where they can add, delete, rename, or move nodes without losing their previous labeling work. This unique capability applies throughout the lifecycle of the model even after the model is deployed. Duet users can leverage the feedback loop in the system to update their models after deployment based on the real traffic received by the model. This enables models to adapt to changes in data distribution.
  • Unbiased Selection of Documents: Duet assists the user in building a custom model by automatically choosing documents from the unlabeled dataset and asking the user to label. We call this process "automatic sampling". Duet has a unique technology to select the most informative document while respecting the original data distribution to avoid selection bias. Duet uses the most updated model to select the next document for the user to label, which optimizes the productivity of human effort by choosing the most informative example that the model is confused about.
  • Feature Suggestion: Coming up with the most suitable set of machine learning features is usually a very critical component to build a high quality machine learning model. The number of machine learning features added to the model prescribes the number of labels needed. Since Duet optimizes for human productivity while building AI models, the machine teaching loop has a unique mechanism to help the user control the model capacity and decide on the optimal set of features needed for the model. The machine teaching loop has a feature suggestion component that offers a set of suggested concepts in association with categories in the user-defined schema. So multiple pairs of a concept and a category in the schema are suggested to the user to choose from. Features are only suggested in case of conflicts where the user has given a document a certain label while the system predicts a different category. Feature suggestion correlates concepts with conflicts on the training set and vets them to the user in the order of relevance to the errors where the top of the suggestion list has the highest correlation with the conflicts. Duet suggests concepts that if the user adds to the model would help the model discriminate between categories. The suggested concepts are obtained from two sources. The first source is some phrases in the training set expanded with contextual synonyms obtained from large corpora of public data. The second source is a set of curated feature repositories built by Duet to address relevant use cases for Duet customers. Having the user rely on the feature suggestion mechanism during the iterative process of machine teaching assists the user in creating high quality machine learning features with guarantees of controlled capacity.
  • CI-CD of AI: Duet has continuous integration, continuous delivery and continuous deployment capabilities for AI models in a similar manner to software development. Duet enables the user to create one or more version(s) of their models. The user can choose to do continuous integration by always publishing the new version to the same model endpoint in which case the new model updates reflect in the user system where the model is used instantaneously. The user also has the option to publish the new model version to a new endpoint in which case the user can do A/B testing or flighting to test the latest model updates before rolling them out to their production environments. MLOps in Duet is made much easier through a few button clicks.
Back to top