Comments on: A simple lexer in Python http://www.gooli.org/blog/a-simple-lexer-in-python/ on software development and related issues Tue, 29 Nov 2011 07:10:01 -0600 http://wordpress.org/?v=2.8.2 hourly 1 By: lastindex http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-18619 lastindex Mon, 22 Mar 2010 12:18:52 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-18619 [...] Ramblings Jasper Potts's Blog is proudly powered by WordPress. Entries (RSS) and Comments (RSS) ...gooli.org A simple lexer in PythonI'm taking a course on building compilers at the Israeli Open University and just learned ... to my [...] [...] Ramblings Jasper Potts's Blog is proudly powered by WordPress. Entries (RSS) and Comments (RSS) …gooli.org A simple lexer in PythonI'm taking a course on building compilers at the Israeli Open University and just learned … to my [...]

]]> By: Marcin http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-16653 Marcin Fri, 08 Jan 2010 20:58:44 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-16653 I like your trick of creating this one humongous ORed regexp that combines all of the other expressions. I don't think it generalizes very well for very complicated regexps (i.e. regexps that contain ORs in them), but still it's pretty neat. I like your trick of creating this one humongous ORed regexp that combines all of the other expressions. I don’t think it generalizes very well for very complicated regexps (i.e. regexps that contain ORs in them), but still it’s pretty neat.

]]> By: William http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-7687 William Wed, 28 Jan 2009 06:53:43 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-7687 Nice idea, well presented. A good example also of using the "yield" statement. Nice idea, well presented. A good example also of using the “yield” statement.

]]> By: Evan http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-7480 Evan Mon, 05 Jan 2009 08:34:53 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-7480 Wow, very neat. I'll be using what you've shown here in future projects. Maybe a template parser. You structured this article very well, by the way. Wow, very neat. I’ll be using what you’ve shown here in future projects. Maybe a template parser.

You structured this article very well, by the way.

]]> By: Fırat KÜÇÜK http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-2143 Fırat KÜÇÜK Tue, 22 Jan 2008 21:04:35 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-2143 good design. congratulations. good design. congratulations.

]]> By: gooli http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1813 gooli Mon, 03 Dec 2007 12:45:47 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1813 I am, but apparently it is only suitable for lexing languages that are similar to shell scripts. I am, but apparently it is only suitable for lexing languages that are similar to shell scripts.

]]> By: Norbert Klamann http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1812 Norbert Klamann Mon, 03 Dec 2007 12:08:01 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1812 Are you aware of the 'shlex' module ? Are you aware of the ’shlex’ module ?

]]> By: gooli http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1562 gooli Wed, 24 Oct 2007 21:14:34 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1562 I don't think lexing gets that much more complex, at least as long as you aren't concerned with performance too much. There might be some issues with precedence and you could add support for different start conditions to better support things like C multiline comments. Semantical parsing on the other hand, is a much more complicated matter. That aperiot module looks really cool. Thanks for the link. I don’t think lexing gets that much more complex, at least as long as you aren’t concerned with performance too much. There might be some issues with precedence and you could add support for different start conditions to better support things like C multiline comments. Semantical parsing on the other hand, is a much more complicated matter.

That aperiot module looks really cool. Thanks for the link.

]]> By: Kumar McMillan http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1561 Kumar McMillan Wed, 24 Oct 2007 21:07:52 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1561 unfortunately, lexical parsing doesn't remain "simple" for very long! Last year at Pycon I saw an excellent presentation on aperiot, an easy to use lexical parsing module: http://moncs.cs.mcgill.ca/people/eposse/projects/aperiot/atglance.dtml I've never had a good reason to try it out though. unfortunately, lexical parsing doesn’t remain “simple” for very long! Last year at Pycon I saw an excellent presentation on aperiot, an easy to use lexical parsing module:
http://moncs.cs.mcgill.ca/people/eposse/projects/aperiot/atglance.dtml
I’ve never had a good reason to try it out though.

]]> By: gooli http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1549 gooli Mon, 22 Oct 2007 07:19:52 +0000 http://www.gooli.org/blog/a-simple-lexer-in-python/#comment-1549 @James I did look at the tokenize module, but it can only tokenize Python code. That's great it all you need to do is syntax highlight Python code. But it's not a generic syntax highlighter. The implementation seems to be similar to what I've done although much more complete of course. @James
I did look at the tokenize module, but it can only tokenize Python code. That’s great it all you need to do is syntax highlight Python code. But it’s not a generic syntax highlighter. The implementation seems to be similar to what I’ve done although much more complete of course.

]]>