If you decide to implement your own programming language whether it's a compiled language or an interpreter
language you're bound to require some sophisticated tools in order to build your actual language.
So what I wanted to do in this particular lesson is I want to give you an example of a real life project
that I have for designing a programming language.
Now before we look at what kind of frameworks I'm actually using Let me actually execute the demo so
that you get to see what it's all about.
So essentially I have a programming language which doesn't compile into native code.
What it does instead is it gets translated into C++ So essentially I'm building my own toy language.
And as I type things here you can see that on the right I get valid C plus plus being updated for me.
So this kind of construct is made possible because I'm using a library in order to help me actually
build the language and the library is called boost spirit.
So here you can see a large number of includes But the ones we are mainly interested in are the boost
spirit includes And what they let you do is they let you build the parser using some predefined constructs.
So let me first of all show you the definition of the abstract syntax tree that I have for the different
parts of the language.
So as you can see I'm running a two part or double dispatch visiter which you can learn more about in
the visitors section of the course and I have different elements all inheriting from nasty element.
But the interesting thing about this is that all of these elements are subsequently adapted using Boost
fusion.
Now boost fusion is yet another library that's actually being used by Spirit.
So here you can see some of the adaptations Unfortunately the ID isn't smart enough to figure out what's
going on here it's highlighting everything in red.
But trust me this code is perfectly fine and it basically does the fusion adaptation.
And the reason why it does this is so that when we actually build the parsing process it can leverage
all these structures.
So if we now go into the parser itself let me just jump in the parser I'll show you how the definitions
appear.
So essentially we have a few predefined structures from boost spirit and these pre-defined structures
are for example the grammar base class.
So we're defining a grammar for a particular kind of file in this case it's a file.
In the learned programming language that's what the language is called.
And here I'm defining certain rules now in terms of the grammar that's being used to define the grammar
that is being recursive here.
The grammar is kind of like the Baccus now or for me if you're not familiar with the Baccus now a form
it's kind of like regular expressions but for languages as opposed to text.
So here for example what I'm saying is that the identifier can be one or more.
And that's what the plus indicates one or more of the following things so it can be an alphanumeric
character or it can be one of these special characters that I also allow in my language.
And then every single construct in the language is defined in terms of these rules so for example the
minus here would allow you to dedicate something to be optional and there a special special kind of
symbols as well.
For example the percent what does the percent mean.
So essentially what we have when we say identify a rule percent comma means that you can have any number
of identifiers which are comma separated.
And what happens as you adapt your t to boost fusion.
If you have a vector of strings this set of identifiers separated by commas will actually be stored
in that vector of strings which is very useful.
I mean if we look at the A C for a second you're going to see that I'm using plenty of types from the
SDL like vectors.
I'm also using Boost Varians to actually define certain class members so for example a class member
can be a function body or a property or a function signature.
So having variants and having subsequent leave visitors which allow you to visit the Antibes is very
useful for actually printing the language because remember I'm translating it from my own language into
C++.
So here you see more of the kind of markup that we have so the stream input the operators just allow
you to sequence the tokens one after another so you have some expectations that we're going to.
Well let's say we're going to have the colon literal followed by the type specification rule.
Here for example we have the operators so essentially in this particular case we can have either a construct
which looks like this or a construct which looks like this.
And in both of these cases we also have special rules.
So in addition to being able to just define what you're parsing You can also define for example that
here we're going to set a particular attribute to false.
And here we're going to set this particular attribute to true.
So depending on the kind of operator that the user has entered you can actually change the status of
the objects as they are parsed which is also very useful.
But then ultimately the definition of the entire language is just a set of these rules you define functions
you define properties whatever it is that you want you can define it all and eventually you have all
of these rules here and these rules define the different parts of your language.
Like for example how to determine a basic type how to write an identifier and so on and so forth.
I also have debug instructions because it allows you to figure out what's actually going on and sometimes
when you design your own language you're wondering why can't I parse this.
Why can't I get this to work.
And certainly having some sort of debugging functionality is very very useful.
So having constructed all of this what I then do is I use some of the code from spirit once again in
order to parse a string.
So I have an input file I parsed the entire file and I tried to get it into this Object-Oriented format
and subsequently printed into C++.
So let's take a look at the way that the C++ printer works.
So essentially a printer is just a visitor that does double dispatch it inherits from a printer base
class which by the way we can also take a look at.
So here is the printer base class.
We have some helpers here for printing things like name spaces for example and then coming back to the
CPV printer you can see that I have special rules so for example in my language you can define types
like ideate and you wait kind of like they are defined in the rust language and they get translated
to the appropriate C++ structures for example.
So then I have lots and lots of essentially visitors so I can visit for example a blog I can visit a
function by the end and all of these cases what's happening is I'm actually printing.
I'm basically appending a very large string which is the C++ representation.
Notice that I sometimes have boost variant.
And so I have to use these visitors.
So I have to call apply visiter and actually provide a visitor for the kind of thing that I'm interested
in.
So here is a static visitor so if I have a variant which can take let's say a string or something else
I have these visages for actually printing those things sometimes I have variants which actually cover
several different options so here you can have an assignment statement or a parameter declaration or
a function signature and I can sort of visit either of these.
So all of this put together allows me to very quickly specify my language so essentially it goes like
this.
So if you want to specify your language you define the object oriented structures so you defined the
abstract syntax tree.
You say for example here is a block and a block can contain a number of statements.
Or here is the assignment statement and it can contain a number of names I'm using.
Sort of if you say a b c that's a vector of strings and then the value that's being assigned and so
on and so forth and you build this very large tree and then you build a visitor for this tree so you
have a structure which implements the visitor pattern with double dispatch You just has a visit for
every single element and then when it comes to building those visitors that's what you can make a printer
for example so that's where you would make a static analyze that which goes through your code and analyzes
it.
The boost fusion adapter part is necessary in order for this to be able to be used by boost spirit.
And of course what the parser does in spirit it allows you to define these rules and these rules whenever
the Whenever spirit actually parses something like it parses an identifier.
It knows exactly where to put the identifier because if you look at the assignment statement row if
I just jump to its definition you can see that in its template definition it specifies the type that
is going to map the data onto.
And if we go to the type then here is our type.
So we have a vector of names here and that means that as we kind of go and actually process these names
if we look out sort of here these names commas separate that are going to be put into that vector.
So this is essentially an illustration of how using a framework can let you make a parser more convenient.
Now one thing you'll notice and it's also very important to note is that there is no explicit Lexar
here there is no explicit lexical part.
At no point in time M-I keeping lists of tokens this might be done behind the scenes but I don't really
care because of all I'm interested in is basically taking a large chunk of text like the kind of text
that you see on the left turning it into some object oriented structure and then traversing that structure
using a visitor to actually print something meaningful on the right hand side.
So this is more industry grade approach I guess.