parser doesn't produce amr-unknown

[PolKul]

I was able to train the parser as per your instructions. But when testing the trained model I found that it didn't produce amr-unknown node. For example:

Text: Which architect of Marine Corps Air Station Kaneohe Bay was also tenant of New Sanno hotel?
# ::node	1	person	1-2
# ::node	2	architect-01	1-2
# ::node	3	facility	3-9
# ::node	5	also	10-11
# ::node	6	reside-01	11-12
# ::node	7	company	13-16
# ::node	10	name	3-9
# ::node	11	"Marine"	3-9
# ::node	12	"Corps"	3-9
# ::node	13	"Air"	3-9
# ::node	14	"Station"	3-9
# ::node	15	"Kaneohe"	3-9
# ::node	16	"Bay"	3-9
# ::node	18	name	13-16
# ::node	19	"New"	13-16
# ::node	20	"Sanno"	13-16
# ::node	21	"Hotel"	13-16
# ::root	6	reside-01
# ::edge	person	ARG0-of	architect-01	1	2	
# ::edge	architect-01	ARG1	facility	2	3	
# ::edge	reside-01	mod	also	6	5	
# ::edge	reside-01	ARG0	person	6	1	
# ::edge	reside-01	ARG1	company	6	7	
# ::edge	facility	name	name	3	10	
# ::edge	name	op1	"Marine"	10	11	
# ::edge	name	op2	"Corps"	10	12	
# ::edge	name	op3	"Air"	10	13	
# ::edge	name	op4	"Station"	10	14	
# ::edge	name	op5	"Kaneohe"	10	15	
# ::edge	name	op6	"Bay"	10	16	
# ::edge	company	name	name	7	18	
# ::edge	name	op1	"New"	18	19	
# ::edge	name	op2	"Sanno"	18	20	
# ::edge	name	op3	"Hotel"	18	21	
# ::short	{1: 'p', 2: 'a', 3: 'f', 5: 'a2', 6: 'r', 7: 'c', 10: 'n', 11: 'x0', 12: 'x1', 13: 'x2', 14: 'x3', 15: 'x4', 16: 'x5', 18: 'n2', 19: 'x6', 20: 'x7', 21: 'x8'}	
(r / reside-01
      :ARG0 (p / person
            :ARG0-of (a / architect-01
                  :ARG1 (f / facility
                        :name (n / name
                              :op1 "Marine"
                              :op2 "Corps"
                              :op3 "Air"
                              :op4 "Station"
                              :op5 "Kaneohe"
                              :op6 "Bay"))))
      :ARG1 (c / company
            :name (n2 / name
                  :op1 "New"
                  :op2 "Sanno"
                  :op3 "Hotel"))
      :mod (a2 / also))

[PolKul]

parsing the same sentence with amrlib parser, for example, gives me this result with amr-unknown:

# ::snt Which architect of Marine Corps Air Station Kaneohe Bay was also tenant of New Sanno hotel?
(t / tenant-01
      :ARG0 (a / amr-unknown
            :ARG0-of (a2 / architect-01
                  :ARG1 (f / facility
                        :name (n / name
                              :op1 "Marine"
                              :op2 "Corps"
                              :op3 "Air"
                              :op4 "Station"
                              :op5 "Kaneohe"
                              :op6 "Bay"))))
      :ARG1 (h / hotel
            :name (n2 / name
                  :op1 "New"
                  :op2 "Sanno"))
      :mod (a3 / also))

[ramon-astudillo]

It should produce amr-unknown, we use this often for question parsing.

What did you trained it with? I just checked on a v0.4.2 deploy and it parses correctly. Also, do you tokenize?

[PolKul]

hi @ramon-astudillo, well, I was trying to follow your setup instructions from here for setup and training (the default action-pointer network config bash run/run_experiment.sh configs/amr2.0-action-pointer.sh ). This is the code for inference:

from transition_amr_parser.parse import AMRParser
amr_parser_checkpoint = "/DATA/AMR2.0/models/exp_cofill_o8.3_act-states_RoBERTa-large-top24/_act-pos-grh_vmask1_shiftpos1_ptr-lay6-h1_grh-lay123-h2-allprev_1in1out_cam-layall-h2-abuf/ep120-seed42/checkpoint_best.pt"
parser = AMRParser.from_checkpoint(amr_parser_checkpoint)
words = [word.strip(string.punctuation) for word in text.split()]
annotations = parser.parse_sentences([words])

[PolKul]

would mind sharing your trained checkpoint to see if it makes any difference?

[ramon-astudillo]

would mind sharing your trained checkpoint to see if it makes any difference?

I am certain it should. We are looking into sharing pre-trained models but I can not say anything at this point.

Also FYI we will update to v0.5.1 soon (post EMNLP preprint submission deadline). This new model (Structured-BART) is new SoTA for AMR2.0 and will be published at EMNLP2021, a non updated prerprint is here https://openreview.net/forum?id=qjDQCHLXCNj

From experience in parsing questions, I can say silver-data fine-tuning works well. You can parse some text corpus with questions, filter it with a couple of rules*, and the use it as additional training data. The training scheme silver+gold pre-training with gold fine-tuning seems to work best, see e.g. https://aclanthology.org/2020.findings-emnlp.288/

(*) For example ignore all parses having :rel (which indicates a detached subgraph) or with missing amr-unknown (if you are certain it should have one).