It's pretty wild what I *don't* yet have in this language: * Basic math operations * Conditionals * Loops * User-defined variables The easiest one to rectify, I think, will be some basic math operations. With that, I can at least have a decent RPN calculator. Before I do that, I'd like to now start separating the language primitives I used in the creation of the interpreter versus everyhing else. Putting everything else in another file and including it turns out to be super easy with NASM: %include 'stdlib.asm' So far, stdlib.asm contains just these: ps (print stack) inspect inspect_all all (all names) So now I'll...YUCK, nevermind! NASM doesn't report the line number of an error in an included file - it really pretends the included content is actually in the file that did the including. It was probably premature to split this up anyway. So everything will stay in meow5.asm as before. Okay, so the math operations are easy because I'm just popping the arguments, calling the CPU instructions, and pushing the answer: $ mr 20 8 - ps 12 4 * ps 48 3 / ps 0 16 "4 divided by 3 is $ remainder $\n" print 4 divided by 3 is 16 remainder 0 bin 1011 0100 + ps 1111 1111 hex ps f f 0beef ps f f beef + ps f befe + ps bf0d Goodbye. Exit status: 0 Increment and decrement! Bam! Easy! 45 inc ps 46 dec dec dec ps 43 I love easy stuff! I'll need to figure out some more substantial stuff next. Variables, maybe. Or conditionals. Next week: I chose to implement variables first. Then I took some time off to rest a bit. Now I've got a first stab at 'var'. (I originally planned to have variables put their VALUES on the stack when called, but after I started implementing them, I've decided to follow yet another Forth convention: variables will put their ADDRESSES on the stack. It's just so much simpler and more flexible that way.) So far, 'var' borrows from three existing words: 1. From 'colon', it takes the idea of getting the next token from the input stream as a name. 2. From 'quote', it takes the compiling of the machine code to push the immediate value containing the address of the variable's memory. 3. From 'semicolon' - actually, it doesn't borrow, it flat out *includes* semicolon. Does it work? var foo hex foo "Address of foo: $\n" print Address of foo: 804c16c var bar hex bar "Address of bar: $\n" print Address of bar: 804c189 Seems plausible! So I guess I need new words to get values to/from that address and the stack. I think I'll call them "get" and "set" rather than the ! ("store") and @ ("fetch") terminology from Forth. They're easy to write in assembly. Just a handful of instructions. Here goes: var foo 42 foo set : foo? foo "Foo contains $\n" print ; foo? ./build.sh: line 34: 1465 Segmentation fault ./$F Exit status: 139 Oops, I didn't even write that correctly. I forgot the 'get' in the definition of 'foo?'. But it should still have printed the address of foo. So something has gone awry. Now I know what I'm doing tomorrow night. Next night: Okay, let's see where this crashes using GDB. First, Can I set and get the variable? Starting program: /home/dave/meow5/meow5 var foo 55 foo set foo get ps 134525204 55 Yes! Okay, I've got some extra garbage on the stack (looks like an address that I'm not cleaing up at some point). I'll deal with that in good time. But 55 on the stack totally means 'foo get' worked. Neat! And can I print it from a string in immediate mode? "foo is $\n" print foo is 55 No problemo. Now in a compiled word where it crashed last night. (This time I remembered to do a 'get' as well): : foo? foo get "foo=$\n" ; foo? Program received signal SIGSEGV, Segmentation fault. 0x0804b114 in token_buffer () Yeah, so there we are. Apparently it crashes while...what? Trying to execute code in token_buffer? Well, I think the real problem is probably something fundamental. Like, I haven't really thought through how something works in a compiled word versus immediate mode. Next night: Okay, LOL, so first of all, I never implemented numeric literals for compile mode. So I can't even test strings like this: $ mr : x 5 "five is $" print ; TODO: a new compile number word?Exit status: 0 And second of all, I don't think I ever tested number interpolation (via '$' placeholders) in compiled words either. Because that crashes: $ mr : x "stack has $" print ; 5 x ./build.sh: line 34: 1390 Segmentation fault ./$F Exit status: 139 So I should probably get those working. I'll start with the second problem and work my way back to variable printing: [ ] Get $ placeholders working in compiled words [ ] Implement compiled number literals [ ] Implement compiled variable get/set I'm not sure yet if that third item is even needed. Okay, so I'm reviewing my 'quote' definition and the thing that immediately stands out is that since I call 'quote' from the interpreter in immediate mode, it should be "baking" in the placeholder value when the string is put in memory. So, for example: 12 : foo "I have $ coins." ; 6 foo Should be printing "I have 12 coins." instead of "I have 6 coins." Instead, it's crashing. But before I fix that, I think I should re-think how this works. Because I'm pretty sure as a programmer, I would expect to have 'foo' output "I have 6 coins." I mean, both ways have advantages and disadvantages. But doing "late binding" on the placeholder seems much more sensible and useful. I want the string to reflect the value on the stack when I *run* 'foo', not when I define it. So I think what I need is for 'print' to contain the placeholder stuff, not 'quote'! Yuck, that's a pretty significant chnage. But I don't really see a way around it if I want the language to make any sense. So, the new TODO is gonna have to be: [ ] Make 'quote' just store the string with '$' [ ] Make 'print' evaluate '$' placeholders in strings (at "run time") I guess this will also require a new output buffer so I can build up strings to output. Or maybe I store them in my general "free" memory as temporary storage? Next night: Okay, let's get to this. Step one is to not have 'quote' replace placeholder '$' with numbers anymore. I'll just comment them out for now. Here's the simple example that crashed above: : x "stack has $" print ; 5 x stack has $ Easy. Next, 'print' needs to print a number from the stack when it sees a '$' in the string. There are two ways I could do this: 1. Make yet *another* copy of this string for temporary printing purposes that has the number string incorporated and print that. 2. Print the string up to the place holder, then print the number, then print the next bit of string after the placeholder, etc. Hmmm... I'm not *loving* either of those options, The disadvantage of #1 is that I need to find another place to store the string - ideally temporary. The disadvantage of #2 is that it's more complicated. I feel like #1 can't be made "nicer". But maybe with some thoughtful design, #2 could be broken into palatable chunks of functionality. Next night: Okay, after some painful stack and register management, I've got it! var foo : foo? foo get "foo=$\n" print$ ; 55 foo set foo? foo=55 Okay, so I stuck with the plan to put the '$' placeholder functionality in the printing mechanism rather than the string building mechanism ('quote'). Rather than turn 'print' into an abomination, I simply added a 'print$' word that does number interpolation. Here you can see 'print' vs 'print$' for comparison: 42 "--$--" print --$-- "--$--" print$ --42-- And 'print$' really wasn't too bad. I mean, it's still quite compact. It was no fun to write. I like how it cleaned up 'quote' a bit as well. That word was way too long. Okay, I need to add a *bunch* of stuff to my test script next to double-check that what I have so far is rock-solid. I already know that something (semicolon?) is leaving an address on the stack that it shouldn't. So I'll be fixing that as well. [ ] Fix whatever is leaving an addr on stack Wait, I almost forgot the fun part: I was going to add a new printing convenience word to wrap up 'print$' plus a newline: 'say'. I love convenience! 1 2 + "I have $ coins." say I have 3 coins. Ah! I love it so much. Yes! What a difference one little handy word makes. I'm going update my hello world examples with 'say' now. Done. And now a confession: I've not been happy with my Expect-based tests. Expect waits for the input you asked for (until a timeout is reached) and it is perfectly okay with getting things you *didn't* ask for. Which makes plenty of sense for interacting with some applications. But that does fit what I want to see, which is *exact* output. And I want *immediate* failure when something other than that is provided. No doubt I can force Expect to do that, but I'm gonna give shell scripts a shot. First of all, I can't remember if I've tried piping input into my interpreter yet, so here goes: $ echo '"Hello" say' | ./meow5 Hello Goodbye. And we can make sure that I get my exact output with a grep search: $ echo '"Hello" say' | ./meow5 | grep '^Hello$' Hello $ echo $? 0 grep's 0 exit status means match was found. $ echo '"Hello" say' | ./meow5 | grep '^foobar$' $ echo $? 1 And 1 exit status is not found. So scripting this is no problem. Here's my shell function: function try { # -q tells grep to not echo, just return match status if echo "$1" | ./meow5 | grep -q "^$2\$" then printf '.' else echo echo "Error!" echo "Wanted:" echo "-------------------------------------------" echo $2 echo "-------------------------------------------" echo echo "But got:" echo "-------------------------------------------" echo "$1" | ./meow5 echo "-------------------------------------------" echo fi } And some simple tests: try '"Hello" say' 'Hello' try '"Hello" say' 'beans' $ ./test.sh . Error! Wanted: ------------------------------------------- beans ------------------------------------------- But got: ------------------------------------------- Hello Goodbye. ------------------------------------------- Cool, then I can re-write the Expect script with this and verify the functionality so far. Oh, and implement compiled numeric literals. That's still an open TODO. Next night: Cool as a consequence of the new test script, I've fixed the address being left on the stack (I had guessed 'semicolon', but it was actually 'colon'. I was pushing the source address to copy the token string for the new word/function name, but 'gettoken' was already doing that, so it was redundant.) I also implemented numeric literals in compile mode. I'm getting a LOT of mileage out of that x86 opcode to push an immediate 32 bit value onto the stack! Now my tests look like: # Input Expected Result # ------------------- ------------------------ try 'ps' '' try '5 ps' '5 ' # note space try '5 5 5 + ps' '5 10 ' try '9 2 * ps' '18 ' try '18 5 / ps' '3 3 ' try '5 2 - ps' '3 ' try '"Hello$\\\$\n" print' 'Hello$\\$' try '"Hello" say' 'Hello' try ': five 5 ; five ps' '5 ' try ': m "M." print ; m '' say' 'M.' try ': m "M." print ; : m5 m m m m m ; m5 '' say' 'M.M.M.M.M.' And they all pass: $ ./test.sh ........... Passed! However, it looks like 'var' is pushing one to many addresses because there's still one on my precious stack that fouls up this test: $ ./test.sh ........... Error! Wanted: ------------------------------------------- var x 4 x set x get x ps 4 ------------------------------------------- But got: ------------------------------------------- var x 4 x set x get x ps 134525172 4 134529332 Goodbye. ------------------------------------------- But I'm getting there! Feeling good about having these tests to check against regressions as i add features. I'll debug the stack issue with a bunch of DEBUG statements to narrow it down like I did for 'colon'. Next night: Yup, in fact it was the exact same bug I had in 'colon'. Shame. By the way, I found both of these by just peppering the offending area my DEBUG macros and printing the value of the esp register to see where something was being left on the stack that shouldn't have been. It looked like this once I fixed it: var x var start: ffafb9c0 var copys: ffafb9c0 var push: ffafb9bc var end: ffafb9c0 4 x set x get ps 4 x ps 4 134529376 get ps 4 4 Now I've fixed my test: try 'var x 4 x set x get ps' '4 ' And all is well: $ mt ............ Passed! And to finish it off, I need to test using (not creating...yet) variables within compiled words. The test: var x 4 x set : x? x get "x=$" say ; x? Expected result: x=4 Crossing fingers: $ mt ............. Passed! Yay! So it looks like that checks everything off for this log: [x] Get $ placeholders working in compiled words [x] Implement compiled number literals [x] Implement compiled variable get/set [x] Make 'quote' just store the string with '$' [x] Make 'print' evaluate '$' placeholders in strings (at "run time") [x] Fix whatever is leaving an addr on stack (twice) See you in log10.txt!