User permissions required: ‘View Sources’ AND ‘Review and label’ 

 

Assigning labels with sentiment enabled

 

Assigning labels with sentiment is very similar to assigning labels without sentiment – see Steps 1, 2 and 3 below, which demonstrate labelling a verbatim from a dataset of customer hotel reviews.


The main difference is in Step 2, where after typing the label name, you must always select either positive or negative sentiment, denoted by the green or red face icons (this step has been repeated for both the ‘Price’ and the ‘Room > Size’ labels).



One important thing to remember when applying labels with sentiment, is that it's important to create a taxonomy with neutral label names (where possible). For example, 'Price' has been used above rather than 'Expensive', as 'Price' is neutral whereas 'Expensive' in inherently negative.


The selection of negative sentiment for a label with a neutral name would capture the instances where the verbatim is expressing a negative perception of the label.


Choosing which sentiment to apply

 

Much of the time it will be obvious which sentiment you should choose when you apply a label, based on the inherent positivity or negativity of the language (e.g. the 'Price' and 'Room > Size' examples above).


For certain labels, the concept may not lend itself to a neutral name and will be inherently negative or positive, and thus always always be applied with only one sentiment. For example, 'Error' related labels will typically all be applied with negative sentiment. This is fine, but should be applied consistently.


Sometimes, however, it can be quite unclear. If the language in a verbatim is very neutral in tone, we need to think more carefully about which sentiment to apply. Here there are two main things to consider:


Verbatim metadata


The first is to look at the verbatim's metadata. For customer feedback related verbatims (the most common type of data in a sentiment-enabled dataset), there will often be some kind of score or rating associated with a verbatim (e.g. NPS score). You can often use these scores to gauge whether a comment that appears neutral in tone, is more positive or negative in sentiment - i.e. a customer rarely leaves an NPS score of 10 if they're unhappy. 

If you consistently apply label sentiment for verbatims that are neutral in tone, based on a 'score' metadata field, Re:infer can learn to pick up on this and predict the sentiment accordingly.


Consistency of application


The second is to be consistent in how you apply the sentiment for a label when it's quite neutral in tone, and there's no other differentiator (e.g. a 'score' related metadata field). 


If it's more common for feedback to be positive for a certain label, assume it's positive unless the verbatim is explicitly negative, and vice versa. If you're not consistent, however, the model will struggle to predict the sentiment.


Applying multiple sentiments

 

Another important thing to consider when using sentiment analysis is that the model applies each label (root and leaf) independently, so you can have two leaf labels from the same parent label that have different sentiments.


In these instances you must then judge what the overall sentiment for the parent label is. In this example below, the parent label ‘Room’ is positive overall.


If both leaf labels have the same sentiment, then the model will infer that the parent label also has a negative sentiment and only the leaf labels will be shown as pinned labels (though that implies the parent label is also applied).



Previous: Training tips and common pitfalls     |     Next: Training support & resources