Detecting future tense using morphological features

[jbuzza]

Feature description

I am trying to use morphological features to determine tense of text like "is rising", "has risen", "will rise".

Some examples would be:

1. 'is rising'

token.text	token.tag_	token.dep_	nlp.vocab.morphology.tag_map[token.tag_])
is	VBZ	aux	{74: 99, 'VerbForm': 'fin', 'Tense': 'pres', 'Number': 'sing', 'Person': 3}
rising	VBG	ROOT	{74: 99, 'VerbForm': 'part', 'Tense': 'pres', 'Aspect': 'prog'}

2. 'has risen'

token.text	token.tag_	token.dep_	nlp.vocab.morphology.tag_map[token.tag_])
has	VBZ	aux	{74: 99, 'VerbForm': 'fin', 'Tense': 'pres', 'Number': 'sing', 'Person': 3}
risen	VBN	ROOT	{74: 99, 'VerbForm': 'part', 'Tense': 'past', 'Aspect': 'perf'}

3. 'will rise'

token.text	token.tag_	token.dep_	nlp.vocab.morphology.tag_map[token.tag_])
will	MD	aux	{74: 99, 'VerbType': 'mod'}
rise	VB	ROOT	{74: 99, 'VerbForm': 'inf'}

As seen above, 'Tense' is populated for the root with 'pres' and 'past' but not 'fut'.

It looks like tag_map.py is used to assign the morphological features based on POS tag. There are also some exception rules defined in morph_rules.py.

However, I do not see 'Tense' value of 'fut' included for any of the POS tags in those files.

I am right in assuming that setting Tense to 'fut' is not implemented ? Is this a feature that could be added or am I doing something wrong ?

Your Environment

• spaCy version: 2.0.12
• Platform: Windows-10-10.0.17134-SP0
• Python version: 3.6.6
• Models: en_core_web_lg

Could the feature be a custom component or spaCy plugin?

Unknown

[DuyguA]

You're not doing anything wrong, neither there is anything wrong with the SpaCy output here.

The verb will is a modal auxiliary, and it does not have a tense. Indeed, English has 2 morphological tenses (you can read more here: https://en.wikipedia.org/wiki/Grammatical_tense#English), the present and the past. In general, the tense concept is about the sentence, not about the individual words. ( A great resource on this subject: https://plato.stanford.edu/entries/tense-aspect/ )
As you see, no such thing as 'future tense verb' or 'future tense modal verb' exists. Future tense is formed by some conventions: use a modal will/shall followed by an infinitive verb.

Summary: if you're interested in sentence tense, mix both POS tags and the words themselves in general. For your case

Future tense: will/shall + a infinitive verb

Then look for the pattern

word1 word2

where
word1.dep_ == 'aux' and word1.text in ['will', 'shall'] and word2.tag_ == 'VB'

By the way, Proto-Germanic had no future tense. Future tense is a new invention compared to the past and present tenses. That's why people had to invent a way of expressing future from existing words/forms.

[lordrisborik]

@DuyguA , Thank you. I think this is an excellent advice from you.

[jbuzza]

@DuyguA Thanks for this information which is very useful. I will experiment further with this but am trying to avoid the need to have a list of 'future words' as there could be other scenarios e.g. 'should rise', 'could rise', 'to rise' etc. There are also forms like 'is expected to rise' which seem more complex to handle .

[DuyguA]

/That's correct. Detecting the fututre tense can be tricky in the absence of will.
One thing I do is to get help from time expressions/adverbs. I daily process customer service corpus in my work, tomorrow/in 2 days/next week etc time expressions occur a lot. What sort of corpus are you processing?

[jbuzza]

It relates to determining if facts are checkable. We could check "Inflation rose by 0.2% in 2016" but not "Inflation will rise by 0.2% in 2020".

[DuyguA]

Looks tricky. Then one has to know synonyms of "expected to..", "known to", "foreseen to"...
If you want to benefit from time expressions here's how I do with CFG: https://duygua.github.io/blog/2018/03/28/chatbot-nlu-series-datetimeparser/

I can ask our in-house analytical linguist meanwhile and return a more concrete answer 😉

[ghost]

@DuyguA Thanks for this information which is very useful. I will experiment further with this but am trying to avoid the need to have a list of 'future words' as there could be other scenarios e.g. 'should rise', 'could rise', 'to rise' etc. There are also forms like 'is expected to rise' which seem more complex to handle .

Have you found any way to detect future tenses?

[jbuzza]

@DuyguA Thanks for this information which is very useful. I will experiment further with this but am trying to avoid the need to have a list of 'future words' as there could be other scenarios e.g. 'should rise', 'could rise', 'to rise' etc. There are also forms like 'is expected to rise' which seem more complex to handle .

Have you found any way to detect future tenses?

Checking tense was not 'past' or 'pres' seemed to work OK in practice but I would imagine not in all cases.

[lordrisborik]

@jbuzza , I am very interested to look at your code if you do not mind. I have all these variets in my headline collection: 'should rise', 'could rise', 'to rise' , 'is exptected to rise' etc.

[migrod09]

Hello,

I have a problem with the import:
from ...symbols import POS, PUNCT, SYM, ADJ, CCONJ, NUM, DET, ADV, ADP, X, VERB
from ...symbols import NOUN, PROPN, PART, INTJ, SPACE, PRON, AUX

I dont know how to solve it: (see image)
Image

[DuyguA]

Can you bring the cursor onto from and paste the error message here?

[migrod09]

This is the message: Attempted relative import beyond top-level packagepylint(relative-beyond-top-level)

Thanks for your help.

[DuyguA]

This is not related to spaCy or any other package, it's a Python notion. As far as I think you use VS code for IDE, right?

First of all you might like to read about notion of package and relative import in Python:

https://realpython.com/absolute-vs-relative-python-imports/
https://stackoverflow.com/questions/6323860/sibling-package-imports
https://stackoverflow.com/questions/14132789/relative-imports-for-the-billionth-time/14132912#14132912

It's not so difficult to fix it in VS code:

https://github.com/MicrosoftDocs/azure-docs/issues/30588
https://stackoverflow.com/questions/55617696/how-to-fix-an-attempted-relative-import-beyond-top-level-package-error-on-this
https://stackoverflow.com/questions/48973742/proper-relative-imports-unable-to-import-module
https://superuser.com/questions/1405038/visual-studio-code-relative-import-with-no-known-parent-package-file
https://stackoverflow.com/questions/58709973/vscode-1-39-x-python-3-7-x-importerror-attempted-relative-import-with-no-kn

You can stackoverflow a bit for how to fix it in VS code, still having trouble you can drop me an email 😉

[migrod09]

Thanks for your help.

I solved it.