"Back & Forth" - Local variables

[Hans Bezemer]

This is gonna get me lose a lot of subscribers, but it has been a hot
topic on YT. So I had to address it sooner or later.

Still, there is a little known implementation of Locals that looks like
Chuck Moore could have invented it - if he had wanted to.

Anyways, I expanded it into a fully Forth 2012 compliant 4tH
implementation, which essentially means you can now port your locals
infested programs to 4tH without ever changing a line.

I hope you are happy now :)

Hans Bezemer

[a***@spenarnc.xs4all.nl]

Post by Hans Bezemer
This is gonna get me lose a lot of subscribers, but it has been a hot
topic on YT. So I had to address it sooner or later.
Still, there is a little known implementation of Locals that looks like
Chuck Moore could have invented it - if he had wanted to.
Anyways, I expanded it into a fully Forth 2012 compliant 4tH
implementation, which essentially means you can now port your locals
infested programs to 4tH without ever changing a line.
I hope you are happy now :)
http://youtu.be/Y7cax2fDS84

You are optimistic that these programs now are interpreted
by 4th.
If it came from Marcel Hendrix you would be obliged
to get rid of '--' comment signals and that may bit be the end of it.

I took a different approach.
The lisp implemented for gforth was infested with locals.

I just converted by using some text transformation, by a regular
expression:

----------- for illustration purpose only -----------------------
#!/home/albert/PROJECT2/refilter -s
\ Filter away those nasty locals.
WANT 2>R COMPARE

: GLOBAL "." ETYPE GLOBAL ;
: ONCE "." ETYPE ONCE ;

: one-value
"(:.*){(.*) +([a-zA-Z0-9]+ *)--}" "0 VALUE \3\n\1 TO \3{ \2 --}" GLOBAL ;

1 ARG[] GET-FILE
"--[^{}]*}" "--}" GLOBAL
BEGIN 2DUP 2>R one-value 2DUP 2R> COMPARE 0= UNTIL
"{ *--}" "" GLOBAL
2 ARG[] PUT-FILE
--------------------------------
Usage:
localfilter in.frt out.frt
This transform LOCALs to VALUEs.


I wrote the regular expression a long time ago.
Application were the handling of my chapter of thinking forth,
and the musical score file of manx, that changes syntax a couple
of times. Recently I converted a bunch of c-files from old fashioned
prototype (1980) to modern ones.

Note that refilter inherits its scripting options from ciforth.
The -s option works on a ciforth but also on turnkeys.
Probably I should publish regexp.frt + refilter.frt
somewhere.

Post by Hans Bezemer
Hans Bezemer

[Hans Bezemer]

Post by a***@spenarnc.xs4all.nl
You are optimistic that these programs now are interpreted
by 4th.
If it came from Marcel Hendrix you would be obliged
to get rid of '--' comment signals and that may bit be the end of it.

From "Forth 2012":
"All text between `--' and `:}' is ignored. This eases documentation by
allowing a complete stack comment in the locals definition."

So whether you do "-- :}" or just ":}" it doesn't really matter. It can
handle that.

Post by a***@spenarnc.xs4all.nl
I took a different approach.
The lisp implemented for gforth was infested with locals.
This transform LOCALs to VALUEs.

I'm not sure replacing local variables with global variables is exactly
what I mean. I mean "using the stack". I got myself an integer BASIC
interpreter where the data stack is actually used as a calculator stack
by the interpreter:

: arith \ map operators to words
case
[char] < of < ;endof \ less-than operands
[char] > of > ;endof \ greater-than operands
[char] = of = ;endof \ compare operands
[char] # of <> ;endof \ not equal operands
[char] - of - ;endof \ subtract operands
[char] + of + ;endof \ add operands
[char] * of * ;endof \ multiply operands
[char] / of / ;endof \ divide operands
[char] % of mod ;endof \ modula operands
[char] ^ of 0 max ** ;endof \ exponent operands
endcase
;

And no, they're not secretly accumulated in another data structure and
then dumped right before the "moment suprème".

Hans Bezemer

[dxf]

This is gonna get me lose a lot of subscribers, but it has been a hot topic on YT.
So I had to address it sooner or later.
...
http://youtu.be/Y7cax2fDS84

"Write Forth the way it's supposed to be written."

Oh my goodness. It's at least half of Forth users that you'll be losing ;-)

[a***@spenarnc.xs4all.nl]

Post by Hans Bezemer
This is gonna get me lose a lot of subscribers, but it has been a hot
topic on YT. So I had to address it sooner or later.
Still, there is a little known implementation of Locals that looks like
Chuck Moore could have invented it - if he had wanted to.
Anyways, I expanded it into a fully Forth 2012 compliant 4tH
implementation, which essentially means you can now port your locals
infested programs to 4tH without ever changing a line.
I hope you are happy now :)
http://youtu.be/Y7cax2fDS84

I was impressed with the Behringer solution.
(I didn't care about the politically correct solution.)

====================================
: local{ R> SWAP DUP >R @ >R >R ;
: }global R> R> R> ! >R ;
=================

But I can do you one better.
Remember the word ;: from colorforth. That is actually a coroutine call.
I call it CO. (Present in ciforth since the year 00)

An example of its use is the following:
:NONAME .S ; ' LIST decorated
decorated is not the point. The noname decorates LIST with
the anonymous function, so that the stack is printed, before LIST.
Now we go one step further:
:NONAME .S CO ." AFTER " .S ; ' LIST decorated
The noname decorates LIST with the anonymous function, so that the
stack is printed, before, but now noname is suspended, LIST is
executed as a coroutine, and afterword the stack is printed once more.

With CO the example become
---------------------------------------
: LOCAL R> SWAP DUP >R @ >R >R CO R> R> ! ;

VARIABLE A
VARIABLE B

: divide
A LOCAL
B LOCAL
B ! A ! A @ B @ /
. CR
;

15 3 divide
---------------------------------------

This saves a definition and a word-of-code, and a line for every
LOCAL used. Now that is closer to what Chuck Moore would have used.
Remember for Moore CO aka ;: is a standard word.

CO is not standard but it should be, and it is elementary as hell.

******************************************************
* The code for CO is one third of the code of R>. *
******************************************************
Look for yourself e.g. https://github.com/albertvanderhorst/ciforth

Post by Hans Bezemer
Hans Bezemer

Groetjes Albert

P.S. This doesn't convince me to use local values.

[Hans Bezemer]

Post by a***@spenarnc.xs4all.nl
I was impressed with the Behringer solution.
(I didn't care about the politically correct solution.)
====================================
: }global R> R> R> ! >R ;
=================
But I can do you one better.
Remember the word ;: from colorforth. That is actually a coroutine call.
I call it CO. (Present in ciforth since the year 00)

<snipped>

Post by a***@spenarnc.xs4all.nl
With CO the example become
---------------------------------------
VARIABLE A
VARIABLE B
: divide
A LOCAL
B LOCAL
. CR
;
15 3 divide
---------------------------------------
This saves a definition and a word-of-code, and a line for every
LOCAL used. Now that is closer to what Chuck Moore would have used.
Remember for Moore CO aka ;: is a standard word.
CO is not standard but it should be, and it is elementary as hell.

Couldn't find the source for either CO or ;: but I got some primitive,
high level form of co-routine in 4tH:

====================================
: yield r> r> swap >r >r ; \ remember that ; compiles EXIT!
aka rdrop grab \ so add a [FORCE] when needed.
====================================

Can't say how they measure up. But I guess co-routines is something
Chuck would like - since it's something you can implement quite easily.
So yes, I agree Chuck wouldn't waste that line ;-)

Hans Bezemer

[a***@spenarnc.xs4all.nl]

Post by Hans Bezemer

Post by a***@spenarnc.xs4all.nl
I was impressed with the Behringer solution.
(I didn't care about the politically correct solution.)
====================================
: }global R> R> R> ! >R ;
=================
But I can do you one better.
Remember the word ;: from colorforth. That is actually a coroutine call.
I call it CO. (Present in ciforth since the year 00)

<snipped>

Post by a***@spenarnc.xs4all.nl
With CO the example become
---------------------------------------
VARIABLE A
VARIABLE B
: divide
A LOCAL
B LOCAL
. CR
;
15 3 divide
---------------------------------------
This saves a definition and a word-of-code, and a line for every
LOCAL used. Now that is closer to what Chuck Moore would have used.
Remember for Moore CO aka ;: is a standard word.
CO is not standard but it should be, and it is elementary as hell.

Couldn't find the source for either CO or ;: but I got some primitive,
====================================
: yield r> r> swap >r >r ; \ remember that ; compiles EXIT!
aka rdrop grab \ so add a [FORCE] when needed.
====================================

That is the equivalent in high level code.

In assembler, assuming ESI is the interpreter pointer and EBP is the return
stack pointer:

CODE CO
XCHG ESI,[EBP]
NEXT,
END-CODE

In assembler the return stack is uncluttered.

Post by Hans Bezemer
Can't say how they measure up. But I guess co-routines is something
Chuck would like - since it's something you can implement quite easily.
So yes, I agree Chuck wouldn't waste that line ;-)
Hans Bezemer

[mhx]

This specific version of XCHG causes an implicit LOCK on the bus and
will
execute slower than (probably) expected. (For those readers that were
getting fancy ideas).

[a***@spenarnc.xs4all.nl]

Post by mhx
This specific version of XCHG causes an implicit LOCK on the bus and
will
execute slower than (probably) expected. (For those readers that were
getting fancy ideas).

That is interesting. Slower than pushing two things on the stack than popping
two?

Groetjes Albert

[Hans Bezemer]

On 08-01-2025 17:27, ***@spenarnc.xs4all.nl wrote:

( my CO variant, using the return address)

: LOCAL R> SWAP DUP >R @ >R EXECUTE R> R> ! ;

VARIABLE A
VARIABLE B

\ I'm paranoid :)

8 a !
7 b !

: divide
A LOCAL
B LOCAL
B ! A ! A @ B @ /
. CR
;

15 3 divide a ? b ?
\ it doesn't mean they're not out to get you

Wow! This works! Can't say how solid it is.. but still!

Hans Bezemer

[dxf]

Post by Hans Bezemer
( my CO variant, using the return address)
VARIABLE A
VARIABLE B
\ I'm paranoid :)
8 a !
7 b !
: divide
    A LOCAL
    B LOCAL
    . CR
;
15 3 divide a ? b ?
\ it doesn't mean they're not out to get you
Wow! This works! Can't say how solid it is.. but still!

Alas not portable.

If I'm playing this game (e.g. a routine needs to be re-entrant) then
I spell out what I'm doing. Nobody gets confused and no surprises.

8 VALUE A
7 VALUE B

: divide
A B 2>R TO B TO A
A B / . CR
2R> TO B TO A
;

15 3 divide a . b .

[dxf]

Post by dxf

Post by Hans Bezemer
( my CO variant, using the return address)
VARIABLE A
VARIABLE B
\ I'm paranoid :)
8 a !
7 b !
: divide
    A LOCAL
    B LOCAL
    . CR
;
15 3 divide a ? b ?
\ it doesn't mean they're not out to get you
Wow! This works! Can't say how solid it is.. but still!

Alas not portable.
...

More portable

: (lx) >R ;

: LOCAL R> SWAP DUP >R @ >R (lx) R> R> ! ;

VARIABLE A
VARIABLE B

8 A !
7 B !

: divide ( a b -- )
A LOCAL
B LOCAL
B ! A ! A @ B @ /
. CR
;

15 3 divide A ? B ?

[a***@spenarnc.xs4all.nl]

Post by dxf

Post by Hans Bezemer
( my CO variant, using the return address)
VARIABLE A
VARIABLE B
\ I'm paranoid :)
8 a !
7 b !
: divide
    A LOCAL
    B LOCAL
    . CR
;
15 3 divide a ? b ?
\ it doesn't mean they're not out to get you
Wow! This works! Can't say how solid it is.. but still!

Alas not portable.
...

Definitely not portable.

R EXECUTE doesn't work for ciforth.

There is no guarantee that a saved interpreter pointer on the
stack is an execution token.

More portable
: (lx) >R ;
VARIABLE A
VARIABLE B
8 A !
7 B !
: divide ( a b -- )
A LOCAL
B LOCAL
. CR
;
15 3 divide A ? B ?

Good catch!
This is an implementation for CO and it works e.g. on ciforth.
But why a different name, CO or ;: as per Chuck Moore.
: CO >R ;
: ;: >R ;
If I use the word (lx) for a decorator, the name makes absolutely no sense.

It is less efficient than a machine code implementation, but it doesn't
rely on information on register use in the interpret machine. This
implementation has a good chance to work an a majority of implementations.

Groetjes Albert

[dxf]

Post by a***@spenarnc.xs4all.nl

Post by dxf

Post by Hans Bezemer
( my CO variant, using the return address)
VARIABLE A
VARIABLE B
\ I'm paranoid :)
8 a !
7 b !
: divide
    A LOCAL
    B LOCAL
    . CR
;
15 3 divide a ? b ?
\ it doesn't mean they're not out to get you
Wow! This works! Can't say how solid it is.. but still!

Alas not portable.
...

Definitely not portable.

R EXECUTE doesn't work for ciforth.

There is no guarantee that a saved interpreter pointer on the
stack is an execution token.

More portable
: (lx) >R ;
VARIABLE A
VARIABLE B
8 A !
7 B !
: divide ( a b -- )
A LOCAL
B LOCAL
. CR
;
15 3 divide A ? B ?

Good catch!
This is an implementation for CO and it works e.g. on ciforth.
But why a different name, CO or ;: as per Chuck Moore.
: CO >R ;
: ;: >R ;
If I use the word (lx) for a decorator, the name makes absolutely no sense.

I didn't realize I was implementing CO. Now I know what it does.
;: seems a good name. It was also the name of DOES> predecessor but I could
never find any details.

Post by a***@spenarnc.xs4all.nl
It is less efficient than a machine code implementation, but it doesn't
rely on information on register use in the interpret machine. This
implementation has a good chance to work an a majority of implementations.
Groetjes Albert

[Hans Bezemer]

Post by a***@spenarnc.xs4all.nl
There is no guarantee that a saved interpreter pointer on the
stack is an execution token.

Nope - in ANS-Forth it is listed as:

nest-sys; definition calls; implementation dependent

So - that's obvious. But in 4tH it works out. And defining it as >R
works out as well. BTW, I've tested the thing - and it holds up.

I got my work cut out for a next episode! On co-routines! ;-)

Hans Bezemer

BTW, I've heard there are implementations where nest-sys aren't even on
the return stack. The standard seems to confirm this:

return stack: A stack that _MAY_BE_ used for program execution nesting,
do-loop execution, temporary storage, and other purposes.

.. and sorry to spoil the fun, but what we're doing here is illegal anyways:

"A program shall _NOT_ access values on the return stack (using R@, R>,
2R@ or 2R>) that it _DID_NOT_ place there using >R or 2>R;"

In other words: your mileage may (be) very, very illegal.

Hans Bezemer

[dxf]

Post by Hans Bezemer

Post by a***@spenarnc.xs4all.nl
There is no guarantee that a saved interpreter pointer on the
stack is an execution token.

Albert said that. He's referring to the use of EXECUTE. What surprised me
is half the forths on my desktop worked (though not DX-Forth).

Post by Hans Bezemer
nest-sys; definition calls; implementation dependent
So - that's obvious. But in 4tH it works out. And defining it as >R works out as well. BTW, I've tested the thing - and it holds up.
I got my work cut out for a next episode! On co-routines! ;-)

Great. While I've seen co-routines mentioned, examples were rare so I
tended to ignore it.

Post by Hans Bezemer
Hans Bezemer
return stack:  A stack that _MAY_BE_ used for program execution nesting, do-loop execution, temporary storage, and other purposes.
In other words: your mileage may (be) very, very illegal.

The >R dependency is on what was (and perhaps still is) 'common practice'.
I was unsure how it would fare but all the popular forths seem to work.
If ANS-FORTH has issued one too many 'Thou shalt not's and in doing so has
excluded itself then that's too bad.

[Hans Bezemer]

Post by dxf
The >R dependency is on what was (and perhaps still is) 'common practice'.
I was unsure how it would fare but all the popular forths seem to work.
If ANS-FORTH has issued one too many 'Thou shalt not's and in doing so has
excluded itself then that's too bad.

Oh, you won't find me on the side of "the standard". As a matter of
fact, 4tH allows a lot of things that the standard doesn't allow and
vice versa. But I think it is a useful tool when deciding what SHOULD be
portable or SHOULDN'T be portable.

In general, I'm all for adopting "common practice" (I'm not going to
list all my exceptions). In this case - why not?

1. It's very hard to enforce (unless you flag all RS items - ugly - or
separate the call stack);
2. It's useful to create e.g. co-routines;
3. I don't recommend the practice, though. It's hard to wrap your head
around and in 4tH it may clash with the tail call optimizer. But since
4tH is completely sandboxed, you can't do much damage. If you (attempt
to) jump outside the box, the program is halted.

Hans Bezemer

[sjack]

Post by dxf
Great. While I've seen co-routines mentioned, examples were rare so I
tended to ignore it.

-- A closed paren defined as an immediate co-routine can be used at
-- compile time to divide a word into two parts.
-- At run-time the word's first part executes then performs co-routine
-- and somewhere down the input ')' performs co-routine to return control
-- back to the the word's second part.
-- : foo( first part ) second part ;
-- Example:

: foo( ." --> " s0 @ sp! ) begin depth while 5 * . repeat ; OK

foo( 3 2 1 ) --> 5 10 15 OK

I use 'hi( foo bar baz )' to highlight the output of a sequence of
Forth words.

[dxf]

Post by sjack

Post by dxf
Great. While I've seen co-routines mentioned, examples were rare so I
tended to ignore it.

-- A closed paren defined as an immediate co-routine can be used at
-- compile time to divide a word into two parts.
-- At run-time the word's first part executes then performs co-routine
-- and somewhere down the input ')' performs co-routine to return control
-- back to the the word's second part.
-- : foo( first part ) second part ;
foo( 3 2 1 ) --> 5 10 15 OK
I use 'hi( foo bar baz )' to highlight the output of a sequence of
Forth words.

We'll see. I'd like to be able to say 'yes, that's unequivocally
superior to any other solution I might use'. I had that sense with
... R> DROP ... to discard a nest-sys. I may not use it every day in
programming but it's present in applications I use nearly every day.

[a***@spenarnc.xs4all.nl]

Post by Hans Bezemer

Post by a***@spenarnc.xs4all.nl
There is no guarantee that a saved interpreter pointer on the
stack is an execution token.

nest-sys; definition calls; implementation dependent
So - that's obvious. But in 4tH it works out. And defining it as >R
works out as well. BTW, I've tested the thing - and it holds up.
I got my work cut out for a next episode! On co-routines! ;-)
Hans Bezemer
BTW, I've heard there are implementations where nest-sys aren't even on
return stack: A stack that _MAY_BE_ used for program execution nesting,
do-loop execution, temporary storage, and other purposes.
In other words: your mileage may (be) very, very illegal.

The use of CO / ;: can't be illegal. Its implementation may not be
portable, that is another matter.
There is no reference to the return stack in the glossary of CO.

========================
CO

STACKEFFECT:

DESCRIPTION: []

Return to the caller, suspending interpretation of the current
definition, such that when the caller exits, this definition is
resumed. The return stack must not be engaged, such as between >R and
R> , or DO and LOOP .

GLOSSARY INDEX

SEE ALSO: CONTROL EXIT
========================
Merely a warning that the ciforth implementation doesn't work well with
certainly manipulations of the return stack.

The strict forbidden of nested definitions is non-sensical,
I allow nested definitions in ciforth with an extension.
: aap { : add + ; ] 1 2 add 3 4 add * ;
The program is non-standard, but if this extension is loaded the Forth
is still conforming.

Post by Hans Bezemer
Hans Bezemer

Groetjes Albert

[Anton Ertl]

Post by a***@spenarnc.xs4all.nl

R EXECUTE doesn't work for ciforth.

There is no guarantee that a saved interpreter pointer on the
stack is an execution token.

In development Gforth, if you use the xt of a colon definition as a
return address, you get the same effect as EXECUTE. E.g.

: colon-execute >r ;

1 ' . colon-execute \ prints "1"

I have recently had a puzzling bug syndrome in the following word
(buggy version shown):

: reverse-sections-execute ( xt -- )
j i @ section-execute
loop ;

[: current-section @ cr h. ;] reverse-sections-execute

The output is:

$7FF5F428F000
$7FF5F3F1A000
$7FF5F3F3B000
$7FF5E5BFD000
$7FF5E5DFE040
$7FF5F4298000
$7FF5E5DFE040 ok

The last line should not be output. What happens is that the xt
passed to REVERSE-SECTIONS-EXECUTE is pushed on the return stack, and
I then forgot to drop it. But instead of that resulting in a crash
(as you would get in Gforth up to around 2020), with the new xt
implementation the ; at the end COLON-EXECUTEs the xt. And because
the xt is a colon definition, this calls the xt again.

xt=pfa has it's advantages, but here a disadvantage showed up.

- anton

[sjack]

Post by dxf
More portable
: (lx) >R ;

BacForth

: ENTER >R ; \ call word whose address is on the data stack

[a***@spenarnc.xs4all.nl]

<SNIP>

Post by a***@spenarnc.xs4all.nl

Post by Hans Bezemer
http://youtu.be/Y7cax2fDS84

I was impressed with the Behringer solution.
(I didn't care about the politically correct solution.)
====================================
: }global R> R> R> ! >R ;
=================
But I can do you one better.
Remember the word ;: from colorforth. That is actually a coroutine call.
I call it CO. (Present in ciforth since the year 00)
:NONAME .S ; ' LIST decorated
decorated is not the point. The noname decorates LIST with
the anonymous function, so that the stack is printed, before LIST.
:NONAME .S CO ." AFTER " .S ; ' LIST decorated

Or
:NONAME .S ;: ." AFTER " .S ; ' LIST decorated

Post by a***@spenarnc.xs4all.nl
The noname decorates LIST with the anonymous function, so that the
stack is printed, before, but now noname is suspended, LIST is
executed as a coroutine, and afterword the stack is printed once more.
With CO the example become
---------------------------------------
VARIABLE A
VARIABLE B
: divide
A LOCAL
B LOCAL
. CR
;
15 3 divide
---------------------------------------

Suppose we adopt Chuck Moore's name

: LOCAL R> SWAP DUP >R @ >R >R ;: R> R> ! ;

This looks even better.

Post by a***@spenarnc.xs4all.nl
Groetjes Albert

[Hans Bezemer]

Post by a***@spenarnc.xs4all.nl
Suppose we adopt Chuck Moore's name
This looks even better.

I chose >YIELD, but I think it as elegant as ;THEN - it signifies "The
definition is ending here, but continues after this". It's a LOT better
than CO, if I may be so frank.

I'll fix it today.

Hans Bezemer