Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TypeScript] Runtime Lexer missing "channel" setter and getter. #4719

Open
kaby76 opened this issue Oct 21, 2024 · 0 comments
Open

[TypeScript] Runtime Lexer missing "channel" setter and getter. #4719

kaby76 opened this issue Oct 21, 2024 · 0 comments

Comments

@kaby76
Copy link
Contributor

kaby76 commented Oct 21, 2024

According to the Definitive ANTLR 4 Reference: An ANTLR lexer creates a Token object after matching a lexical rule. Each request for a token starts in Lexer.nextToken(), which calls emit() once it has identified a token. emit() collects information from the current state of the lexer to build the token. It accesses fields _type, _text, _channel, _tokenStartCharIndex, _tokenStartLine, and _tokenStartCharPositionInLine. You can set the state of these with the various setter methods such as setType(). For example, the following rule turns enum into an identifier if enumIsKeyword is false: ENUM : 'enum' {if (!enumIsKeyword) setType(Identifier);} ;

Unfortunately, for the TypeScript target, some of the setters and getters are missing. We really need these.

  • channel

Note, some of the fields are directly accessible rather than through getters and setters, some with leading underscore, some not.

If I want to override emit(), I'll need a token factory. Otherwise, I will be making an assumption that the token implementation is always CommonToken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@kaby76 and others