Dev Streams

Dev Streams

Postby A-Man » July 1st, 2017, 7:33 pm

I'll start doing live dev streams if anybody is interested in seeing how I do things. So a lot of this time, I've been working on the new A-SMUL (A-Script Markup Lanauge) API. Currently, I'm working on a tokenizer for it. It's a very important component of the A-Engine, but might be boring to watch.

If you have any questions, you can throw them to the chat, and I'll answer them as soon as I see them. I'll start streaming in a while.
Edit:

Links:
Dev Stream 1: https://www.youtube.com/watch?v=c7sQ8YKGm_c
Dev Stream 2: https://www.youtube.com/watch?v=unEWQuvyg1w
Image
User avatar
A-Man
Infamous Pirate
 
Posts: 401
Joined: March 31st, 2015, 11:40 am

Re: Dev Streams

Postby Shiya » July 2nd, 2017, 6:08 pm

It's difficult to watch without your commentary. But I'm glad that you still find it interesting to work on this project. Is this done just only to highlight syntax?
Shiya
Registered User
 
Posts: 4
Joined: April 8th, 2015, 7:30 am

Re: Dev Streams

Postby A-Man » July 2nd, 2017, 11:32 pm

I figured since I don't speak when I work much, it wouldn't be very useful. But I agree it might have helped explaining the things that went wrong easier.

To address your question, the A::Tokenizer class can be used for syntax highlighting, but it's not limited to that. The tokenizer takes some text such as
Code: [Select all] [Expand/Collapse]
[F=103] tag = 1 [/F]
GeSHi © Codebox Plus

and instead of giving me characters "[", "F", "=", "1", "0", "3" ..etc, it will give me meaningful tokens that I can use directly, like: {ID_COMPONENT_OPENING}, {ID_COMPONENT_ID, "F"}, {ID_COMPONENT_ID_EQUAL}, {ID_COMPONENT_ID_VALUE, "103"}, {ID_COMPONENT_ID_CLOSING}, {TAG, "tag"}..etc.

This can be quite complicated, because there are many scenarios where the identity of the token is context sensitive. But then will make writing the scripting engine a breeze (what has been done before parsed markup syntax fine, but failed miserably when I attempted to extend it for scripts).

I'm glad somebody is watching though :P

Anyway, I'll be online again in a couple of minutes. Hopefully this time I'll do commentary, but my voice is loud and it's late. So I'll have to speak softly, and my voice isn't good at that as it tends to oscillate up and down in the entire spectrum of audible frequencies :P

Link: https://www.youtube.com/watch?v=auv2qInhNuQ

Edit:
Sorry for the terrible audio today. But yes, even though we didn't test it, it seems we're done with everything markup. Parsing the rvalues (values that comes to the right of a definition, like "5+4" in "a = 5+4") was definitely the hardest, as that's where most of the ambiguity comes up.

Next up, we'll extend the tokenizer so that it can parse through script (respective to it is the markup). We will assume that the last nest level is "SCRIPT_MODULE" or something until we figure out good opening and closing tags. Since we have (almost) evreything for rvalues already, all is left, all is left is to do declarations, loops and other similar constructs. Which should be quite easy. I will discuss the language's final "specification" early on in the video, so it's easier to follow up.

After that, we'll write the compiler which is pretty fun. That sounds complicated, but since we'll have a very powerful tokenizer, it should be very easy to get something quickly.
Image
User avatar
A-Man
Infamous Pirate
 
Posts: 401
Joined: March 31st, 2015, 11:40 am

Re: Dev Streams

Postby Shiya » July 8th, 2017, 1:41 pm

I get it. And I watched most of the last video and understood good amount how it is supposed to work even that I don't use C++. I just wonder how can you not test if it works correctly every some time, are you that confident in your code? Anyway, it's gonna be awesome and I can't wait to finally be able to use your engine!
Shiya
Registered User
 
Posts: 4
Joined: April 8th, 2015, 7:30 am

Re: Dev Streams

Postby A-Man » July 9th, 2017, 10:01 am

Thank you =). That's very nice to hear.
I just wonder how can you not test if it works correctly every some time, are you that confident in your code?

That's the joke. I'm very guilty for doing that. It's not that I'm confident in my code, but that I consider coming up with the "scheme" to be most of the work. Moreover, testing can be off-putting, so I'd rather continue writing down what's on my mind rather than testing while I'm focused. Then after I've written enough, I would dedicate a few hours to attempting to compile and to testing.

So I've finished writing the tokenizer class (generally speaking), so next up, we'll begin working on the scripting language's compiler.
Image
User avatar
A-Man
Infamous Pirate
 
Posts: 401
Joined: March 31st, 2015, 11:40 am

Re: Dev Streams

Postby Dragon5 » July 13th, 2017, 2:02 am

Oh wow. It's been a while. Time to catch up on the streams.
Just stay calm for now. We will do it when the time comes~
User avatar
Dragon5
Pirate Subordinate
 
Posts: 46
Joined: April 5th, 2015, 1:50 am

Re: Dev Streams

Postby A-Man » July 15th, 2017, 8:39 pm

Yes! I hope you're enjoying them.

Streaming again: https://www.youtube.com/watch?v=unEWQuvyg1w

Edit: Summary of what we did:
We started working on the symantics of the scripting component of A-SMUL. We created a "Value" class that holds objects in the scripting language, however, it relies on the new type safe union std::variant introduced in C++17. And it turned out that for some reason the C++ compilers I have on my computer don't support that still, even though the official specification checklist states that clang 4.0 should (which is what I tried). I'll try to look more into this; it could be that I don't have the latest version, and that std::variant was supported in a minor version 4.0.1 or something. Otherwise we'll have to either rely on a member of type void pointer, or create different members for the different types.

And then we wrote a "MemoryBlock" class which would contain allocations. The idea is that we will have a global "Memory" object that contains different "MemoryBlock" objects. And then instantiating different symantic contexts (will be represented by "SymanticContext" objects) will create different memory blocks in the memory, and they'll have access to them. In addition, the symantic contexts would have a map from variable names to addresses in our memory. Adding two contexts together then would just result in a context that has a map (offsetted) and an array of memory blocks of both contexts. This will make it easier to throw scopes around, and should make it easy to even do function closures. Once we have closures, the scripting language could be easily be considered an "object-oriented" language, and a "class" keyword could easily be introduced. But I'm not really interested in doing that for now, although it's good to know it's doable. Perhaps something to improve on later in the future.

So next time, we'll implement what's explained above. Then we'll begin working on a tiny "byte code" (calling it "A-code" :P) engine. A-code will be a very primitive language that a script would compile to. For instance, this program:
Code: [Select all] [Expand/Collapse]
let x = 1 # suppose address of x is 1245134, and temp address of 1 is 51381729847
x = x + 2 #let temporary address of 2 be 47817284198
GeSHi © Codebox Plus

would compile to something like this
Code: [Select all] [Expand/Collapse]
000 BEGIN_PROGRAM
001 DECLARATION
002 1245134
003 51381729847
004 ASSIGN
005 1245134
006 ADD
007 1245134
008 47817284198
009 END_PROGRAM
GeSHi © Codebox Plus

the latter should be very fast to execute.
Image
User avatar
A-Man
Infamous Pirate
 
Posts: 401
Joined: March 31st, 2015, 11:40 am

Re: Dev Streams

Postby A-Man » July 19th, 2017, 11:25 am

Streaming tonight~
Image
User avatar
A-Man
Infamous Pirate
 
Posts: 401
Joined: March 31st, 2015, 11:40 am

Re: Dev Streams

Postby A-Man » September 3rd, 2017, 10:04 pm

So I have been working on this still. I just kinda haven't been in a mode to stream. Sorry :P.

Either way, I got around polishing (fixed lots of bugs, remedied all warnings, added proper exceptions ..etc) the tokenizer part of the source code yesterday, and pushed it into a repository. I only pushed parts that are already in a working condition. More will follow soon hopefully.
https://github.com/Student-A/A-SMUL-Lib
Image
User avatar
A-Man
Infamous Pirate
 
Posts: 401
Joined: March 31st, 2015, 11:40 am


Return to A-Engineer's Zone

Who is online

Users browsing this forum: No registered users and 1 guest