Skip to content

Continue Teaching to Get to the Level of Accuracy You Aspire To

Teaching happens interactively. Continue retrieving more documents to label by clicking the next sample button or searching for keywords. Read the document and choose which leaf category on the right best fits the document category. Select the appropriate category from the right hand side schema hierarchy. You can only choose one category for each document. Please note that if you'd like to watch an instructional video on labeling you can access it here.

Once you move onto the next document, the top right of the page will update to reflect the total number of labels you added so far. The counters next to each category on the right will also update after the model automatically trains with the new label you just submitted. With every model update, the system also updates the quality indicator of each category to give you a sense of the model quality so long as that schema category has at least five labels. It is strongly recommended that you provide each schema category with five labels prior to publishing. If you try to move to publishing before this, the system will provide a warning. Additionally, prior to ten documents being labeled, each document is shown without its system prediction. When ten documents have been labeled, the system shows the prediction of the model on new documents (top predicted category and confidence score) both at the top of the document view and with a blue highlight in the schema. If the prediction is correct, select the radio button of the indicated category. If the prediction is not correct, choose the alternative category. In both cases, the system will retrain the model after the new label is added and the prediction will be updated.

You will see the automatic update occur as the label count rises and the prediction score changes to match the consistency of your labeling. There are two ways to raise accuracy: schema edits and resolving conflicts via adding suggested features. Building your schema through adding, renaming, deleting and moving schema labels ought to be done as you label and add features to fix conflicts. The continuously updated model will show you conflicts as they emerge. In Duet, you address the conflicts via reviewing and adding suggested features, which are then reinforced with further labeling. This is called the iterative teaching process.

If you skip any documents by pressing "Next Sample" without labeling, you can access them by pressing the gear button in the top right.

Upon doing so, you will get a list of the skipped documents along with their predicted schema category and confidence score. Click a document to retrieve it and have the opportunity to assign a label.

Back to top