Internal Error Lexer
Sign in Pricing Blog Support Search GitHub This repository Watch 188 Star 1,722 Fork 530 antlr/antlr4 Code Issues 221 Pull requests 37 Projects 0 Wiki Pulse Graphs New issue NullPtr and bad tree parse upon parser rule ref in lexer rule in combined grammar #661 Closed parrt opened this Issue Jul 11, 2014 · 0 comments Projects None yet Labels atn-analysis comp:tool grammars type:bug Milestone ANTLR 4.4 Assignees No one assigned 1 participant Antlr Project member parrt commented Jul 11, 2014 Grammar test: grammar U; A : a ; Yields: ~/IdeaProjects/testplugin $ a4 U.g4 GrammarTreeVisitor.g: node from line 3:4 required (...)+ loop did not match anything at input 'a' GrammarTreeVisitor.g: node from line 3:4 required (...)+ loop did not match anything at input 'a' GrammarTreeVisitor.g: node from line 3:4 required (...)+ loop did not match anything at input 'a' GrammarTreeVisitor.g: node from line 3:4 required (...)+ loop did not match anything at input 'a' error(20): U.g4:3:4: internal error: Rule a undefined error(20): internal error: element list has first|last == null Exception in thread "main" java.lang.NullPointerException at org.antlr.v4.automata.ParserATNFactory.elemList(ParserATNFactory.java:504) at org.antlr.v4.automata.ParserATNFactory.alt(ParserATNFactory.java:473) at org.antlr.v4.parse.ATNBuilder.alternative(ATNBuilder.java:556) at org.antlr.v4.parse.ATNBuilder.ruleBlock(ATNBuilder.java:290) at org.antlr.v4.automata.ParserATNFactory._createATN(ParserATNFactory.java:184) at org.antlr.v4.automata.LexerATNFactory.createATN(LexerATNFactory.java:137) at org.antlr.v4.Tool.processNonCombinedGrammar(Tool.java:414) at org.antlr.v4.Tool.process(Tool.java:379) at org.antlr.v4.Tool.processGrammarsOnCommandLine(Tool.java:346) at org.antlr.v4.Tool.main(Tool.java:193) parrt added this to the ANTLR 4.4 milestone Jul 11, 2014 parrt added atn-analysis grammars comp:tool type:bug:1 labels Jul 11, 2014 parrt referenced this issue in antlr/intellij-plugin-v4 Jul 11, 2014 Closed Exception on frammar file #115 parrt added a commit that closed this issue
resources Windows Server 2012 resources Programs MSDN subscriptions Overview Benefits Administrators Students Microsoft Imagine Microsoft Student Partners ISV Startups TechRewards Events Community Magazine Forums Blogs Channel 9 Documentation APIs and reference Dev centers Samples Retired content We’re sorry. The content you requested has been removed. You’ll be auto redirected in 1 second. Visual C++ Programmer's Guide Build Errors NMAKE Errors U1000 through U4011 NMAKE Errors U1000 through U4011 NMAKE Fatal Error U1038 NMAKE Fatal Error U1038 NMAKE Fatal Error U1038 NMAKE Fatal Error U1000 NMAKE Fatal Error U1001 NMAKE Fatal Error U1002 NMAKE Fatal Error U1003 NMAKE Fatal Error U1004 NMAKE Fatal Error U1005 NMAKE Fatal Error U1006 NMAKE Fatal https://github.com/antlr/antlr4/issues/661 Error U1007 NMAKE Fatal Error U1017 NMAKE Fatal Error U1018 NMAKE Fatal Error U1019 NMAKE Fatal Error U1020 NMAKE Fatal Error U1021 NMAKE Fatal Error U1022 NMAKE Fatal Error U1023 NMAKE Fatal Error U1024 NMAKE Fatal Error U1025 NMAKE Fatal Error U1031 NMAKE Fatal Error U1033 NMAKE Fatal Error U1034 NMAKE Fatal Error U1035 NMAKE Fatal Error U1036 NMAKE Fatal Error U1037 NMAKE Fatal Error U1038 NMAKE Fatal Error U1039 https://msdn.microsoft.com/en-us/library/aa268150(v=vs.60).aspx NMAKE Fatal Error U1040 NMAKE Fatal Error U1041 NMAKE Fatal Error U1042 NMAKE Fatal Error U1043 NMAKE Fatal Error U1045 NMAKE Fatal Error U1047 NMAKE Fatal Error U1048 NMAKE Fatal Error U1049 NMAKE Fatal Error U1050 NMAKE Fatal Error U1051 NMAKE Fatal Error U1052 NMAKE Fatal Error U1053 NMAKE Fatal Error U1054 NMAKE Fatal Error U1055 NMAKE Fatal Error U1056 NMAKE Fatal Error U1057 NMAKE Fatal Error U1058 NMAKE Fatal Error U1059 NMAKE Fatal Error U1060 NMAKE Fatal Error U1061 NMAKE Fatal Error U1062 NMAKE Fatal Error U1063 NMAKE Fatal Error U1064 NMAKE Fatal Error U1065 NMAKE Fatal Error U1069 NMAKE Fatal Error U1070 NMAKE Fatal Error U1071 NMAKE Fatal Error U1072 NMAKE Fatal Error U1073 NMAKE Fatal Error U1076 NMAKE Fatal Error U1077 NMAKE Fatal Error U1078 NMAKE Fatal Error U1079 NMAKE Fatal Error U1080 NMAKE Fatal Error U1081 NMAKE Fatal Error U1082 NMAKE Fatal Error U1083 NMAKE Fatal Error U1084 NMAKE Fatal Error U1085 NMAKE Fatal Error U1086 NMAKE Fatal Error U1087 NMAKE Fatal Error U1088 NMAKE Fatal Error U1089 NMAKE Fatal Error U1090 NMAKE Fatal Error U1092 NMAKE Fatal Error U1093 NMAKE Fatal Error U1094 NMAKE Fatal Error U1095 NMAKE Fatal Error U1096 NMAKE Fatal Error U1097 NMAKE Fatal Error U1098 NMAKE Fatal Error U1099 NMAKE Fa
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more about Stack Overflow the company Business Learn http://stackoverflow.com/questions/23621165/responsibilities-of-the-lexer-and-the-parser more about hiring developers or posting ads with us Stack Overflow Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of 4.7 million programmers, just like you, helping each other. Join them; it only takes a minute: Sign up Responsibilities of the Lexer and the Parser up vote 2 down vote favorite I'm currently implementing a lexer for a simple programming language. So far, I internal error can tokenize identifiers, assignment symbols, and integer literals correctly; in general, whitespace is insignificant. For the input foo = 42, three tokens are recognized: foo (identifier) = (symbol) 42 (integer literal) So far, so good. However, consider the input foo = 42bar, which is invalid due to the (significant) missing space between 42 and bar. My lexer incorrectly recognizes the following tokens: foo (identifier) = (symbol) 42 (integer literal) bar (identifier) Once the lexer sees the internal error lexer digit 4, it keeps reading until it encounters a non-digit. It therefore consumes the 2 and stores 42 as an integer literal token. Because whitespace is insignificant, the lexer discards any whitespace (if there is any) and starts reading the next token: It finds the identifier bar. Now, here's my question: Is it still the lexer's responsibility to recognize that an identifier is not allowed at that position? Or does that check belong to the responsibilities of the parser? parsing compiler-construction tokenize lexical-analysis share|improve this question asked May 13 '14 at 0:47 Marius Schulz 8,66953477 add a comment| 4 Answers 4 active oldest votes up vote 1 down vote accepted I don't think there is any consensus to the question of whether 42foo should be recognised as an invalid number or as two tokens. It's a question of style and both usages are common in well-known languages. For example: $ python -c 'print 42and False' False $ lua -e 'print(42and false)' lua: (command line):1: malformed number near '42a' $ perl -le 'print 42and 0' 42 # Not an idiosyncracy of tcc; it's defined by the standard $ tcc -D"and=&&" -run - <<<"main(){return 42and 0;}" stdin:1: error: invalid number # gcc has better error messages $ gcc -D"and=&&" -x c - <<<"main(){return 42and 0;}" && ./a.out