User permissions required: ‘View Sources’ AND ‘Review and label’
For the last phase of training, 'Teach' mode helps you refine your model to improve the precision and recall of each label. It’s used to find examples of labels where the model is confused as to which labels apply or not, using 'Teach' will show you verbatims with labels predicted with close to 50% confidence (or close to 66% for datasets with sentiment enabled).
By applying labels to verbatims where the model has a low confidence, you provide stronger training signals to the model than if you label a verbatim where it’s 99% confident that the label applies.
Teach mode is found within the Explore page, and you can access it simply by using the dropdown menu in the top right-hand corner of the page.
Dropdown menu to access Teach
You may find that if you’ve labelled lots of similar verbatims within Discover for a label, it’s useful to go straight to Teach (after the model has had some time to retrain) to give it some examples where the model is less certain.
Teach + unreviewed
After selecting Teach from the dropdown menu, select a label from the left-hand menu that requires some more training, and then select ‘Unreviewed’.
Teach mode will then present you with 20 unreviewed verbatims where Re:infer thinks that the selected label could apply but is unsure. You can then either confirm the label by clicking on it or delete it and provide a correct alternative using the ‘+’ button.
Teach by label on unreviewed verbatims
Using this method is a really effective way of training, as by confirming a label where Re:infer is really unsure, you send a much stronger training signal to Re:infer than if you were to confirm a label on a verbatim where Re:infer is 95% confident that it applies.
If you were to select ‘Reviewed’ and then a label, Re:infer would show you (up to 20) verbatims where it is unsure why you have given those verbatims that label, or why that label has not been applied.
Teach by Label on Reviewed Verbatims
This can be a useful way of checking that you’ve been consistent in your labelling and understanding labels for which Re:infer may be confused.
If you find you’ve made a mistake or that you’ve been inconsistent in your labelling, you can easily correct it and apply the correct label.
- Move, rename and merge labels to ensure you’re happy with the hierarchical structure — to do this use the label edit modal in Explore.
- If for a label there are multiple different ways of saying the same thing (e.g. A, B or C), make sure that you give Re:infer training examples for each way of saying it. If you give it 30 examples of A, and only a few of B and C, the model will struggle to pick up future examples of B or C for that label.
- Adding a new label to mature taxonomy may mean it’s not been applied to previously reviewed verbatims. This then requires going back and teaching the model on new labels, using the Teach function – see here for how
Previous: Using Explore