How to distinguish between terminal nodes and non terminal nodes

[liu1234567yi]

Dear authors, Thanks for your outstanding work. I have a question for you. When you want to predict a node, you don't know whether it is a terminal node or a non terminal node in advance，and this two kinds of nodes are predicted in different ways(described in the article as two methods：Predicting AST Nodes and Predicting Subtokens). So, how to distinguish these two nodes in order to use different prediction methods in code implementation？

[urialon]

Hi @liu1234567yi ,
Thank you for your interest in our work!

We have this script that as a preprocessing step, goes through the training data and finds all the terminal and nonterminal node types.

Than, at test time when we want to predict the child node of a given node a: if a is a nonterminal - we predict from the nodes vocabulary; if a is a terminal - we predict a subtoken from the subtoken vocabulary.

I hope it helps,
Best,
Uri