Custom negation

[JimDunlop]

Hi, i am trying to build a german sentiment analysis with spacy and sentiws.
The Sentence: "Das ist kein Problem" gets a negative score, because the token "kein" has a dep_ attribute of "nk" (i am filtering for dep_ == 'ng' to find negation and recalculate the sentiment score)
Is this matter of training, or would i have to use custom list of negation words to check against?

[polm]

It looks like this is the package you're using?

https://github.com/Liebeck/spacy-sentiws

If so, it looks like it's dictionary-based, not statistical. Since the problem you have is with negative modifiers it's going to be very situational ("nicht gut" and "nicht schlecht" are different), so using the dependency parse to invert scores seems like a reasonable approach. You can't train it because it's dictionary based, and you can't handle modifiers just by changing the dictionary, so the dep parse is better than that.

If you need to handle negations of arbitrary concepts I think you might find things get too complicated; that's just one of the limitations of dictionary based sentiment analysis. Depending on your dataset negation might not be a problem - it's pretty typical for negations to be infrequent enough that in aggregate you can ignore them and still get ok results.

You might want to look at negspaCy for a package that does some handling of negation.

https://github.com/jenojp/negspacy

[JimDunlop]

Thanks a lot for your reply! Yes this is exactly the package i am using.
So far i have worked around the problem using an aditional modifiers list with words that diminish or strengthen a given sentiment score. So a "huge Problem" has a bigger Sentiment Score then just "a problem"
When i find a word with a sentiment-score i use the dependency to find all the "word-neighbors" and check them against my dictionary of modifier-words. In my example the word "kein" has a modifier-score of -1.0 and negates the score of "Problem"

[polm]

words that diminish or strengthen a given sentiment score

These are traditionally called "intensifiers" in sentiment analysis. Not sure how many off the shelf libraries support them, but it might be a useful term if you want to search through academic papers.