Advanced prediction filters represent a significant step forward in how the label filters work in both the Explore and Reports pages in Re:infer.
We've significantly expanded Re:infer's capability to support intelligent filter combinations on label predictions.
Here's some of the things you can now do when filtering by label predictions:
- Apply multiple label filters at once, in both Explore and Reports
- Filter to verbatims that have one of a number of selected labels predicted (i.e., Label X OR Label Y OR ...)
- Filter to verbatims that have multiple different labels predicted (i.e. Label X AND Label Y AND ...)
- Filter to verbatims that do not have certain label(s) predicted (i.e. NOT Label Y)
- Search for verbatims containing specific search terms, whilst having label filters applied
Essentially, the new filters let you apply any combination of 'AND', 'OR' and 'NOT' when applying more than one label filter. These filters can give you much greater flexibility when training and interpreting your data, and can provide much deeper insights on what's happening in your communication channels.
Impact on model training and filtering charts:
As a result of these changes, we've also updated the way you select a label to sort by when training in Explore, and how you select labels to appear in the charts in Reports.
When you filter to a label now in Explore, this no longer determines the training mode or the sort order of the current training mode. Instead this is determined by a taxonomy dropdown menu that appears when you select a training mode that is specific to a certain label (e.g. 'Check label' or 'Missed label').
This is to enable you to be sort by a certain label for a training mode and have an additional label filter applied.
How this works in practice is discussed in more detail further down this article.
Similarly, when you filter to labels in Reports this no longer determines which labels appear in charts. Instead, this also acts exclusively as a filter and the dropdown 'Charted labels' menu determine which labels are shown in the charts. How this works in practice is also discussed in more detail at the end of this article.
Applying advanced prediction filters
There are now two ways to apply label filters, and they can be used in combination with each other to create the right type of query.
The taxonomy of labels functions as a normal filter bar, and allows you to select multiple labels at once with a single click for each.
Selecting multiple labels from the taxonomy creates an 'OR' type query.
If you selected Label A, Label B and Label C in the taxonomy, this creates a 'Show me verbatims with Label A, Label B, or Label C predicted' query.
This example below would return verbatims that contained at least one of the three selected labels:
Combined label filter returning verbatims containing any one of the three select labels
Add Label Filter
The second filter option is the '+ Add Label Filter' button above the taxonomy bar.
This enables a dropdown taxonomy that allows you to select more complex filters, including excluding certain labels from consideration.
'+ Add Label Filter' dropdown menu
From this dropdown, you can select multiple labels to include or exclude by clicking the plus or minus buttons.
The result would be something like the example below, which would return verbatims predicted to have the label 'Margin Call > Confirmation', but that also do not have the label 'Margin Call > Interest Accrual' predicted:
Example label filter combination
You can click '+ Add Label Filter', multiple times to add additional layers to your query. Two separate label filters create an 'AND' type query, whilst multiple labels selected in the same label filter create an 'OR' type query.
In the example below, multiple label filters have been applied individually. This creates a filter that will return verbatims predicted to have any of the four labels in the first filter, but that also have the 'Complaint' label predicted, and do not have the 'Query' label predicted.
An example of complex label query combining 'OR', 'AND' and 'NOT' label filters
A helpful tip is that by clicking the '&' sign in an individual filter containing multiple labels, you can automatically split them out into individual filters. This would change the query from 'OR' (i.e. any of these labels predicted) to 'AND' (i.e. all of these labels predicted).
Combining taxonomy bar filters and added label filters
It's possible to combine filters from both the taxonomy bar, and individually added label filters. Filters applied in the taxonomy bar are treated as an 'AND' query with any individually applied label filters.
For example, in the image below, this combined query would return any verbatims that had the 'Margin Call > Partial Agreement' label predicted AND either of 'Margin Call > Confirmation' OR 'Margin Call > Dispute' predicted.
Combined label filter using taxonomy bar and individually added label filters
Combining label filters and sorting by label for training
What these new filters also mean, is that you can now apply label filters and sort by a specific label for a training mode.
In example below, you can see verbatims where 'Margin Call > Full Agreement' may have been applied incorrectly, but that also do not have the label 'Margin Call > Dispute' predicted:
Explore page showing 'Check label' mode for a specific label, with an additional label exclusion filter applied
Sorting by label for training modes in Explore
To enable the advanced prediction filters, a number of other updates have been made that do change the way you select labels to train in Explore. Overall, these changes make Re:infer more consistent across the platform, as well as providing greater capabilities to run detailed queries on your data.
The top bar in Explore now shows the selected training mode and sort order in the top left, with the search bar on the top right (see below).
Importantly, when selecting a specific label to train you no longer use the filter bar on the left, but rather you select the training mode first and a taxonomy dropdown then prompts you to select the label you wish to sort by.
The example below shows a user having selected 'Check Label' and then being prompted to select which label to sort by:
Sort order label dropdown showing pinned label counts and label warnings
You can also see that the pinned label counts, red dial indicators, and any performance warnings for a label are also displayed in the taxonomy dropdown, giving you all the information you need to select the right label to sort by.
Once a label is selected, it will be shown next to the training mode alongside the number of pinned examples and any performance warning indicators:
Explore page showing a selected training mode and label to sort by
Lastly, there is now a specific mode in Explore, 'Sentiment', to sort by positive or negative sentiment only (see below). If you want to sort by label(s) and sentiment, you can still do that using the 'Label' sort order and adding a sentiment filter:
Explore page showing 'Sentiment' sort mode
Selecting labels to appear in charts in Reports
Similarly to the changes in Explore, adding a label filter via the taxonomy in Reports no longer adds those labels to the label specific charts. Instead, the label filters act exclusively as filters to update the verbatims that are in consideration (much like any other user property filter) by including or removing verbatims that have or don't have certain labels predicted.
To select which labels actually appear in the charts in the Reports page, there is now a 'Charted labels' dropdown menu in the top bar.
Simply select whichever combination of labels you want to appear and these will be plotted on the charts which distinguish between different labels, as in the example below.
Remember, this does not change the verbatims that are in consideration by Re:infer, merely which labels are plotted on the label specific charts. An easy way to remind yourself of this is to note that the total number of verbatims shown at the top of the page does not change when you select any labels from the 'Charted labels' dropdown.
Updated Reports page with 'Charted labels' selected
All of the labels that you select will appear highlighted at the top of the page as a visual reminder, and can easily be removed by clicking the 'x' icon next to each label name.