The LATEST Word (I'm missing something)
Mystery Solved!
I asked on the Forth subreddit here and got an answer right away along with an excellent explanation that steered me straight:
Incredibly, I did not fully grok that a FORTH variable leaves its address on the stack, not its value. Which makes complete sense because to change a variable’s value, we need its address for storing (with !).
By the way, this demonstrates that I’ve spent all this time writing a FORTH interpreter, but I still barely know how FORTH works as a language.
Amazing.
Anyway, on to my misunderstanding…
The Mystery
I’m missing something and I can’t find it.
When I was working on the assembly portion (now complete) of "nasmjf" my JonesFORTH port, I ran into an issue while defining the word : (COLON): I can’t figure out where or how the LATEST word is defined.
For reference, here are my copies of the original JonesFORTH source files mentioned below:
jonesforth.S (assembly)
jonesforth.f (FORTH)
The LATEST variable
Here’s a comment in jonesforth.S describing the variable LATEST:
LATEST Points to the latest (most recently defined) word in the dictionary.
And here’s how it’s defined:
defvar "LATEST",6,,LATEST,name_SYSCALL0
Clearly LATEST contains the address of the word SYSCALL0, right?
The LATEST word
Ah, but a comment above the built-in variables section says:
LATEST is also a FORTH word which pushes the address of
LATEST (the variable) on to the stack, so you can read
or write it using @ and ! operators. For example, to print
the current value of LATEST (and this can apply to any FORTH
variable) you would do:
LATEST @ . CR
And that is how the word LATEST is used in the COLON (:) definition:
defword ":",1,,COLON ... .int LATEST, FETCH, HIDDEN // Make the word hidden
and SEMICOLON (;):
defword ";",1,F_IMMED,SEMICOLON ... .int LATEST, FETCH, HIDDEN // Toggle hidden flag
(Note that FETCH is the internal name for @.)
What’s the problem?
The only problem is that I don’t see where the LATEST word is being defined. Here’s every mention of LATEST in the assembly. There is no defword or defcode to be found:
dave@phobos~/proj/nasmjf$ ag LATEST jonesforth/jonesforth.S 191: A FORTH variable called LATEST contains a pointer to the most recently defined word, in 210: LATEST 213: the dictionary (just walk along the dictionary entries starting at LATEST and matching 215: and add a word to the dictionary (create a new definition, set its LINK to LATEST, and set 216: LATEST to point to the new word). We'll see precisely these functions implemented in 1100: have discussed so far was LATEST, which points to the last (most recently defined) word in the 1101: FORTH dictionary. LATEST is also a FORTH word which pushes the address of LATEST (the variable) 1103: the current value of LATEST (and this can apply to any FORTH variable) you would do: 1105: LATEST @ . CR 1125: LATEST Points to the latest (most recently defined) word in the dictionary. 1133: defvar "LATEST",6,,LATEST,name_SYSCALL0 // SYSCALL0 must be last in built-in dictionary 1520: mov var_LATEST,%edx // LATEST points to name header of the latest word in the dictionary 1645: LATEST points here points to codeword of DUP 1672: pointer to previous word (from LATEST) +-- Afterwards, HERE points here, where 1680: (3) Set LATEST to point to the newly defined word, ... 1786: movl var_LATEST,%eax // Get link pointer 1799: // Update LATEST and HERE. 1801: movl %eax,var_LATEST 1875: .int LATEST, FETCH, HIDDEN // Make the word hidden (see below for definition). 1885: .int LATEST, FETCH, HIDDEN // Toggle hidden flag -- unhide the word (see below for definition). 1918: movl var_LATEST,%edi // LATEST word. 1927: LATEST @ HIDDEN 2198: initialise the LATEST variable to point to it. This means that if you want to extend the assembler 2199: part, you must put new words before SYSCALL0, or else change how LATEST is initialised.
And I missed it while porting, too, because to get my NASM port to work, I simply removed the FETCH after LATEST in the COLON and SEMICOLON definitions. It worked just fine!
DEFWORD ":",1,,COLON ... dd LATEST, HIDDEN ; Make the word hidden while it's being compiled.
and
DEFWORD ";",1,F_IMMED,SEMICOLON ... dd LATEST, HIDDEN ; Unhide word now that it's been compiled.
(By the way, I only discovered the problem while stepping through the execution of my port with GDB and trying to understand why my words weren’t being set to hidden while being compiled!)
It seemed extremely unlikely that there was a bug in JonesFORTH, which has had so many eyeballs on it (and obviously works!), but I eventually just shrugged and continued on.
But now I’m at the phase where I’m slowly stepping through every word definition in the FORTH half of JonesFORTH and I ran into the problem again while trying to understand why RECURSE wasn’t working. Sure enough, it was the LATEST @ problem again. RECURSE was fetching the address of the previous word (because the current word starts with a pointer to the previous in a linked list fashion).
Here’s the whole definition:
: RECURSE IMMEDIATE
LATEST @ \ LATEST points to the word being compiled at the moment
>CFA \ get the codeword
, \ compile it
;
I’m sure removing the @ will "fix" it, but I don’t want to do that. I’m okay with making changes to how the assembly portion runs (I already have), but I want to keep the original FORTH source unaltered.
And LATEST is always used like the word definition described above in the FORTH source. Here’s every mention. It always has the @ fetch:
dave@phobos~/proj/nasmjf$ ag LATEST jonesforth/jonesforth.f 110:\ access to the word which we are currently compiling through the LATEST pointer so we 113: LATEST @ \ LATEST points to the word being compiled at the moment 739: For example: LATEST @ ID. would print the name of the last word that was defined. 777: The implementation simply iterates backwards from LATEST using the link pointers. 780: LATEST @ ( start at LATEST dictionary entry ) 804: and definitions will overwrite memory starting at the word. We also need to set LATEST to 815: DUP @ LATEST ! ( set LATEST to point to the previous word ) 829: LATEST @ 128 DUMP 991: LATEST @ ( start at LATEST dictionary entry ) 1028: LATEST @ ( word last curr ) 1653: LATEST @ DUP
So clearly I’ve somehow missed where LATEST becomes both a FORTH variable and a FORTH word in the assembly source.