User permissions required: ‘View Sources’ AND ‘Review and label’
Introduction to using 'Teach Label'
'Teach' is the second step in the Explore phase and its purpose is to show predictions for a label where the model is most confused if it applies or not. Like previous steps, we need to confirm if the prediction is correct or incorrect, and by doing so provide the model strong training signals. It is the most important label-specific training mode.
- Select Teach from the top-left dropdown menu as shown
- Select the label you wish to train - the default selection in Teach mode is to show unreviewed verbatims
- You will be presented with a selection of Verbatims where the model is most confused as to whether the selected label applied or not - review the predictions and apply the label if they are correct, or apply other labels if they are incorrect
- Predictions will range outwards from ~50% for data with no sentiment and 66% for data with sentiment enabled
- Remember to apply all other labels that apply as well as the specific label you are focusing on
You should use this training mode as required to boost the number of training examples for each label to above 25, whereby the platform can then accurately estimate the performance of the label.
The number of examples required for each label to perform well will depend on a number of factors. In the 'Refine' phase we cover how to understand and improve the performance of each label.
The platform will regularly recommend using 'Teach Label' as a means of improving the performance of specific labels by providing more varied training examples that it can use to identify other instances in your dataset where the label should apply.
However, it's only useful to review predictions that are not high-confidence (90%+). This is because when the model is very confident (i.e. above 90%), then by confirming the prediction you are not telling the model any new information, it's already confident that the label applies. Look for less confident examples further down the page if needed. Although, if predictions have high confidences and are wrong, then it's important to apply the correct label(s), thereby rejecting the incorrect prediction(s).
- If for a label there are multiple different ways of saying the same thing (e.g. A, B or C), make sure that you give the platform training examples for each way of saying it. If you give it 30 examples of A, and only a few of B and C, the model will struggle to pick up future examples of B or C for that label.
- Adding a new label to a mature taxonomy may mean it’s not been applied to previously reviewed verbatims. This then requires going back and teaching the model on new labels, using the 'Missed label' function – see here for how