Tiny questions

[hydroperx]

I'm wondering if with Langium I'm able to do the following:

• Implement three-dimensional/namespaced names (e.g. (prefix, localName))
• Parse different input goals (InputElementDiv, InputElementRegExp, InputElementXMLTag, InputElementXMLContent), or at least handwrite parsing of certain constructs but also being able to scan tokens manually within the handwritten parser (e.g. the Unicode characters).
• Like Java, be able to resolve source files lazily based on source path and fully qualified packages.

[msujew]

Hey @hydroperx, thanks for the question!

Implement three-dimensional/namespaced names (e.g. (prefix, localName))

Langium supports fully qualified names, if that's what you mean? Even google doesn't know what three-dimensional names are supposed to be :/

Parse different input goals (InputElementDiv, InputElementRegExp, InputElementXMLTag, InputElementXMLContent), or at least handwrite parsing of certain constructs but also being able to scan tokens manually within the handwritten parser (e.g. the Unicode characters).

You can exchange the tokenizer in Langium, but not the parser. With the way how we build the AST during the parsing phase, changes to the parser are not possible. Is that enough for your use case? I don't understand what your input goals are supposed to be.

Like Java, be able to resolve source files lazily based on source path and fully qualified packages.

Yes, Langium allows to lazily resolve files on the file system. By default, it will attempt to load the currently open workspace (in your IDE at least), but that behavior can be adjusted for your particular language.

[hydroperx]

Thanks for the quick reply, @msujew! Sorry, I'll clarify my questioned terms.

---

Three-dimensional names are like this: a variable x may be of a namespace http://www.fun.com/2005; and we could have a namespace alias namespace fun = "http://www.fun.com/2005" so we could refer to that variable as fun::x. Namespaces are not just URI; there is also public and internal (tied to a package or activation) and private and protected.

Fully qualified names? Well, these three-dimensional names still have a parent, like a package, class or scope.

---

As to lexer goals, well, I'll place some ECMAScript 4 examples:

var person:XML = (
    <person>
        <name>John</name>
        <age>{10 + 20}</age>
    </person>
);

Here, starting from the <person> opening tag we get inside InputElementXMLTag, and content is scanned as InputElementXMLContent. However, you can see there is 10 + 20: that is scanned as either InputElementDiv or InputElementRegExp.

InputElementDiv contains lexer rules containing the language stuff, like punctuators, comments, whitespace, literals... InputElementRegExp too does the same, but does not include the punctuators / and /=, but instead the regex literal (e.g. /(?:)/, /(?:)/gi).

InputElementXMLTag contains lexer rules for things that appear inside a tag, like XML names, attribute names, and some punctuators like =, /> and >.

InputElementXMLContent contains lexer rules for things that appear inside a tag, like plain text, <, markup like <![CDATA[, <!-- and <?.

---

I understand, so basically we can't override the parser, right?

In any case, if I can't do my project in Langium, if you know about another tooling for this, please let me know. :)

[msujew]

Three-dimensional names are like this: a variable x may be of a namespace http://www.fun.com/2005; and we could have a namespace alias namespace fun = "http://www.fun.com/2005" so we could refer to that variable as fun::x. Namespaces are not just URI; there is also public and internal (tied to a package or activation) and private and protected.

Usually, we call those fully qualified names (FQN), and yes, they're supported. We have a whole guide on them.

As to lexer goals, well, I'll place some ECMAScript 4 examples:

Ah, you mean lexer states? Yes, Langium support those as well, but you need to work a bit with the TokenBuilder service to enable them. We have a WIP guide on that actually. Once the lexer state is in place, parsing the embedded XML shouldn't be too difficult.

In any case, if I can't do my project in Langium, if you know about another tooling for this, please let me know. :)

There is very little tooling available for that, beyond some parsing libraries for JavaScript. Except for our parser, everything in Langium is built from scratch :)

[hydroperx]

Usually, we call those fully qualified names (FQN), and yes, they're supported. We have a whole guide on them.

@msujew Sorry, I've not clarified correctly FQN correctly.

In ActionScript 3, let's say, a class com.fun.fragments.Fragment may have an instance variable x. In that case, x is still a three-dimensional name independently of the qualifying com.fun.fragments.Fragment class:

package com.fun.fragments {
    import com.fun.fun_internals;

    public class Fragment {
        fun_internals var x:Number;
    }
}

In that case we have:

• Fragment: a class whose namespace is the public namespace of com.fun.fragments.*, and whose parent is com.fun.fragments.*.
• x: an instance property whose namespace is fun_internals (this is an alias, so the actual namespace might be an URI), and whose parent is that Fragment class.

I'm sorry, it looked like a FQN at first ^_^' If we had a static constant then it could look like com.fun.fragments.Fragment.fun_internals::x

Ah, you mean lexer states? Yes, Langium support those as well, but you need to work a bit with the TokenBuilder service to enable them. We have a WIP guide on that actually. Once the lexer state is in place, parsing the embedded XML shouldn't be too difficult.

Nice, so it's possible, hehe

[msujew]

In ActionScript 3, let's say, a class com.fun.fragments.Fragment may have an instance variable x. In that case, x is still a three-dimensional name independently of the qualifying com.fun.fragments.Fragment class:

I see, yeah that's essentially just a bit of scoping logic that needs to be implemented for your language. See our domainmodel example for some pointers:

langium/examples/domainmodel/example/qualified-names.dmodel

Lines 5 to 14 in 910e8a0

	package foo.bar {
	datatype Complex
	entity E2 extends E1 {
	next: E2
	other: baz.E3
	nested: baz.nested.E5
	time: big.Int
	}
	}