Fix Conflicts by Looking at Suggested Features and Add Them to the Model as well as Reviewing the Conflicts¶
As you are updating the models by editing the schema or adding labels, conflicts between the label and the system’s prediction will arise.
The system will suggest features to remedy these conflicts. To resolve specific system conflicts in a label, click on the conflict count to the right.
Feature suggestions are shown in the document to show the suggested feature in context. Suggested features are also shown at the bottom right of the page. The feature suggestion pane is closed when there are no suggestions. Feature suggestions are associated with specific categories. Duet suggests the features to help the system discriminate between categories and resolve the conflicts but it needs help from the user to verify the suggestion. Hovering over a suggested feature will show the category for which it is suggested. You need to review the content of the feature before adding it.
Clicking a suggestion will open a popup where you can add it as a new feature or add the suggested phrases to an existing feature. Click to read more about adding dictionary features and context features.
Pay attention to fixing conflicts before pursuing labeling. As you label, keep your eye on the quality metric next to each category in the schema on the right. Resolve conflicts as they arise and assess features that are suggested.
The two forms of feature suggestion are:
- Dictionary phrases, which can represent concepts like "methods" for the payment method category.
- Context features, which enables users to form linguistic patterns by composing features. For example, the pattern of "issue" precedes "pay" by few words can be captured by a context feature that composes the dictionary of the concept "issue" and the dictionary of the concept "pay" separated by few tokens (i.e. words). Such context features will capture something like "I have an issue when trying to pay for my online order". The teaching loop is fully interactive where the model gets updated with every schema edit, labeling a new document, or adding a new feature suggested by the system when there are conflicts.
In addition to acting upon feature suggestions, you can highlight words to initiate a new feature when labeling a new document or reviewing the label of a previously-labeled document.
You can check your progress by looking at the quality metric with every model update. When you obtain the quality that you are looking for, you should stop teaching and move onto testing.
See machine learning features to learn about resolving conflicts in more detail.