Tokenization of case markers?

[geetsgithub]

Nepali allows flexible word order, and its case markers are written as orthographic attachments to noun phrases. In this context, should these case markers be treated as separate syntactic tokens through multiword‑expression tokenization, or should they be analyzed instead as inflectional morphology? Also, what are the criteria used to determine whether or not an item should be considered a separate syntactic token?

[nschneid]

Good question. Does the case marker ever attach to part of the noun phrase that is not the head? E.g. 's in "The king of France's head". If so, it is arguably a clitic that should be considered a separate syntactic word.

[geetsgithub]

Thank you. They don't. That means, they should be treated just like the inflection markers, right?

[nschneid]

Haspelmath (2023) uses nonselectivity as a criterion for clitics. That is, if a marker only ever occurs on nouns, it is an affix; but if it can occur on modifiers, for example, it is a clitic.

That said, tokenization questions are often tricky. There may be disagreements regarding a language or group of related languages due to different grammatical traditions.

[bulbulistan]

Haspelmath also believes that word classes only consist of verbs, nouns and adjectives (the same paper, but also Haspelmath 2024) and in any case, he has repeatedly asserted that his work does not really apply to what he calls "p-linguistics", i.e. the description of individual languages.
More to the point @geetsgithub raises: I am not that familiar with Nepali, but I think the markers in question are quite similar to the Hindi/Urdu ones, such as -ko (with a homophonous cognate in Nepali) or -men (Nepali -mā). If that is the case, maybe it's a good idea to look at how HDTB deals with those.

[dan-zeman]

The difference is that in Hindi and Urdu, postpositions are separate orthographic words (when applied to nouns; it is somewhat different from pronouns). If they are written as suffixes in Nepali, I would not cut them off just because in Hindi they are written separately - we can use morphological features to annotate the case on nouns. Segmentation of orthographic tokens to multiple syntactic words comes at a price; if not needed, I would not do it.

[Stormur]

If these case markers never appear separated from the nouns heading the noun phrases (including possible other bound markers in between), they represent an affixal inflection (as opposed to clitical) and should be treated as morphological features.

This is in accord to that Haspelmath's paper, which aims to be applicable to languages in general, especially in a typological context like in UD.

Like @dan-zeman says, there is no worth in splitting them if they never show variation in their position, that is, if they never occur "naturally split" from the head.