Showing posts with label parsing. Show all posts
Showing posts with label parsing. Show all posts

Saturday, December 12, 2015

CSS Parser v0.1

This morning, after putting in some extra hours I got the first iteration of my CSS parser working. Originally I had made seven example selectors that manually placed into my cssSheet API to test out how my "trees of selectors" would work out.  After thoroughly debugging these methods, I went ahead and spent some hours on how the parser's actions behaves. Originally I thought I was in hot water because I would only be passing around one selector, but thanks to the way the grammar behaves, it actually worked, and parsed a test css sheet that contained these same examples! It was very exciting.

Also the fact that this worked with multiple selectors floating around within the grammar gives me hope for implementing things like the wildcard selector and giving multiple selectors the same set of properties in one declaration. Honestly, because ANTLR and grammars are so powerful, these things could possibly work already-- but that's providing my code plays nice. Anyways, the next step will be implementing the parser into the engine, and then I imagine I'll be maintaining it both through our enhancement and in the future. But I don't mind. I think ANTLR is cool, and I like working with it. I really felt the pressure from the deadlines I've been working under, but I'd like to make parsers a regular part of my programming diet. There's this magical moment where you're mentally considering only so many cases, and your completed parser readily handles so many more. It makes you want to attribute some intelligence to the design that isn't there at all-- it's just a beautiful structure.

Sorry for being sappy.

Adding Actions To Your ANTLR Grammar

We established last time that getting antlr to generate code is a huge pain. Even worse, is the fact that it's hard to find good examples of the antlr workflow, i.e. generating code with actions, adding arguments, and etc. Thankfully, in my quest to build a CSS parser in C++, I've managed to track down two invaluable Stack Overflow posts that proved invaluable.
The first is a simple ANTLR3 example. I found this to be a very helpful workflow example for antlr3. While it's aimed towards a Java target, it should be very simple to translate the code actions to any other language. While working on this example I also highly recommend learning what ANTLRWorks has to offer, especially the syntax diagrams and built-in interpreter. These tools should help you better understand your grammar, including what it can handle and what it can't. It's also not a bad idea to follow along in a debugger when running the parsing routine to get a hold of how the grammar actions behave. If you find any tweaks that need to be made, just remember to apply them to the grammar, or they will be wiped away if you ever generate your parser again.

By the end of that example you should have built it all; a Exp parser and lexer from ANTLR3, and added target code to the grammar that works. At this point you should be ready to work with a larger and more sophisticated grammar. However, you may find using the limited rules in the Exp example that you can't accomplish everything you'd like with only returns. Thankfully this other post shows how to pass in arguments.

There are a couple of other things to consider as you fiddle with your grammar. Code actions must go under a rule. Also, don't expect ANTLRworks to know what the hell your code is supposed to. Its job is to understand your grammar. This is both a blessing and a curse. It's a blessing in that you don't really have to show ANTLR the classes and structs you're working with, but at the same time, you can sabotage yourself and write some funked up code-- but your IDE will let you know.


Thursday, December 3, 2015

ANTLR quickstart

He's a resource I wrote for those of you wanted to get started with ANTLR, but cut through the BS red tape and get straight to writing grammars! Here it is:

I spent a lot of time trying to get ANTLR in usable shape, and I found the documentation dreadful. So allow me to lay out a much easier alternative to using ANTLR right out of the box, that doesn't require finnicking with system variables that never work.

This may not be the most "flexible" route, but is certainly the quickest one in my experience. The instructions are usable in both UNIX and Windows.



  1. Download the java jdk.After you have it installed you *should* be able to invoke the "java" command from your command prompt. test it using "java -version". If you get some kind of output from that, then great!
  2. Download the ANTLR complete jar.There are two antlr sites in an attempt to separate ANTLR3 from ANTLR4.
    If you want ANTLR4, go to antlr.org
    If you want ANTLR3, go to antlr3.org
    In either case, download the complete jar to a location you wouldn't mind using it.
  3. Test installationIn your command prompt move to the directory that contains the complete jar.
    try the command

    java -cp antlr-x.y.z-complete.jar org.antlr.Tool

    in ANTLR3, or in ANTLR4,

    java -cp antlr-x.y.z-complete.jar org.antlr.v4.Tool

    This command should work, and return options for ANTLR.
    The cp argument means you can you specify CLASSPATH within the command, removing
    the additional step of fooling around with system variables.

  4. Test output
    Use the included "Exp.g" grammar taken from a stackoverflow question and run the following command in ANTLR3

    java -cp antlr-x.y.z-complete.jar org.antlr.Tool Exp.g

    or in ANTLR4

    java -cp antlr-x.y.z-complete.jar org.antlr.v4.Tool Exp.g

    If you're using your own grammar, make sure that the filename matches the first line, e.g. grammar css = css.g

  5. Test different target.In your grammar, add the line

    options{language = C;}

    where "C" is whatever target language you'd want. I'm typically concerned with C or C++
    output, so I use "C" or "Cpp", though ANTLR4 doesn't target C or CPP.
    Run the same command again, and check that it ran correctly and output matches the desired filetype.

  6. Design your grammar
    Congratulations. If the step before worked, than you have everything you need to start doing *actual work* with ANTLR! I hope this was helpful in getting you started, and not hung up on installing stuff. 

Edit:
I found myself also need to check out antlrworks to see how my grammar is structured. As of 1.5.2, try

java -jar antlrworks-1.5.2-complete.jar org.antlr.Tool <grammar.g>


"Installing things is the hardest part of programming."