"Back & Forth" is back!

[Hans Bezemer]

After explaining why Forth is so hard, I'm explaining what reasons there
could be to use it. With a little personal journey put in as a side note.

Hans Bezemer

[dxf]

After explaining why Forth is so hard, I'm explaining what reasons there could be to use it. With a little personal journey put in as a side note.
http://youtu.be/MXKZPGzlx14
Hans Bezemer

Who is Forth for? Based on that video it's for folks who enjoy building the
functions they'll need :)

The 'test string for a float' function got my attention. Wondering how I'd
do that I decided on a stripped down >FLOAT .

https://pastebin.com/UgpE14pc

[Hans Bezemer]

Post by dxf

After explaining why Forth is so hard, I'm explaining what reasons there could be to use it. With a little personal journey put in as a side note.
http://youtu.be/MXKZPGzlx14
Hans Bezemer

Who is Forth for? Based on that video it's for folks who enjoy building the
functions they'll need :)

If you perceive it like that, I can hardly deny it. I do remember
clearly though adding a few of my professional experiences as well.
I could have included another one, where some "name=value" format was
encountered. So, yes, that one got converted as well. A colleague of
mine wanted the source, I printed it for him and to my surprise he
opened the printer tray. "What are you doing?" I asked him - and he
answered "I thought there'd be more.."

But I thought it was too similar to the XML example - and although it
actually happened (I was there) - a bit over the top.

Post by dxf
The 'test string for a float' function got my attention. Wondering how I'd
do that I decided on a stripped down >FLOAT .
https://pastebin.com/UgpE14pc
Gimme points for originality ;-)

I did a quick hack once of a full (ZenFloat) >FLOAT. It works on a
subset of floats, but it's not as nice as yours.

max-n 10 / constant (limit) \ cell boundary

: (convert) ( a1 n1 n2 -- a2 n3 n4 n5)
0 >r >r \ setup exponent, save accu
begin
over over \ is there any string left?
while \ if so, get digit and compare
c@ [char] 0 - dup 0< over 9 > or 0= r@ (limit) < and
while \ don't cross the cell boundary
r> 10 * + r> 1+ >r >r chop \ shift left, increment exponent
repeat drop r> r> \ drop value, get accu and exponent
;
\ returns an exponent
: (+exp) ( a1 n1 -- a2 n2 n3)
0 >r begin dup >r 0 (convert) nip over r> < while r> + >r repeat drop r>
; \ loop until string no longer changes

: (sign!) if negate then ; ( n bool -- -n|n)
: (sign?) over c@ [char] - = dup >r if chop then r> ;
: (frac) chop rot >r rot (convert) negate r> + 2swap (+exp) drop ;
: (exp) chop (sign?) >r 0 (convert) drop r> (sign!) >r rot r> + -rot ;

: >float ( a n -- f bool|-bool)
-trailing dup if (sign?) -rot else swap true exit then
dup if 0 (convert) >zero 2swap else 2drop drop false ;then
dup if (+exp) swap >r swap >r + r> r> then
dup if over c@ [char] . = if (frac) then then
dup if over c@ bl or [char] e = if (exp) then then
nip if 2drop drop false ;then \ if string left, no floating point

Post by dxf
r swap (sign!) r> true \ apply sign, signal ok

;

Hans Bezemer

[Buzz McCool]

Post by Hans Bezemer
After explaining why Forth is so hard, I'm explaining what reasons there
could be to use it. With a little personal journey put in as a side note.
http://youtu.be/MXKZPGzlx14

I looked through a few of these videos and found them interesting, thank
you Hans for going to the trouble of making them.

I found Hans' recommendation on one of the videos (if I'm paraphrasing
it correctly) to avoid using the stack for more than two or three values
as treating the stack as an array makes for incomprehensible code,
enlightening.

I am trying to follow this recommendation, but am running into trouble
when trying to pass parameters into a loop. I'm trying to avoid using
the stack as a large array but what I came up by injecting a parameter
with a variable doesn't seem right.

Does anyone have suggestions on a better approach when you have several
parameters and loop counts to deal with?

(Trivial Example)

: AreaOfCir 2.0e f** pi f* ; \ w/ radius on stack,
\ compute area (radius^2 * pi)

: VolOfCyl AreaOfCir f* ; \ w/ height & radius on stack,
\ compute vol (height * area)
1.0e AreaOfCir fe.
3.1416E0

2.0e 1.0e VolOfCyl fe.
6.2832E0


fvariable radius \ create a floating point variable
1.0e radius f! \ store 1.0 into radius
radius f@ fe. \ fetch and print radius
1.0000E0

: CylVolLoop
cr ." Radius " radius f@ fe. \ print a new line and then fetch and print
radius
1 \ start counter from a cyl height of 1
begin dup 20 <= \ duplicate counter to see if counter <= end value
while \ while true (i.e. counter is <= to 20)
dup \ duplicate counter
s>f \ convert counter (height) to floating point
fdup \ duplicate height to print and use to compute vol
cr ." Height " fe. \ print a new line and then print height
radius f@ \ fetch radius
VolOfCyl \ compute volume
." Volume " fe. \ print volume
1 + \ add one to counter
repeat \ repeat the test at the "begin" word
drop ; \ remove the leftover loop counter value

CylVolLoop \ Execute CylVolLoop word
Radius 1.0000E0
Height 1.0000E0 Volume 3.1416E0
Height 2.0000E0 Volume 6.2832E0
...
Height 19.000E0 Volume 59.690E0
Height 20.000E0 Volume 62.832E0

[minforth]

Two classic answers:
use DO..LOOPs to hide away loop indices
use locals if you have too many parameters
(some technical/physical formulas are difficult
or impossible to factorise into smaller words
which would otherwise be the classic Forth mantra)

[BuzzMcCool]

Post by minforth
use DO..LOOPs to hide away loop indices
use locals if you have too many parameters

I hadn't thought about using locals. Thanks for the suggestion.

[Buzz McCool]

Post by minforth
use locals if you have too many parameters

I like this quite a bit. Tell me if I like it too much.

: CylVolLoop {: W: StartHeight W: FinalHeight F: Radius -- Tabular Output :}
cr ." Radius " Radius fe.
StartHeight
begin dup FinalHeight <=
while
dup
s>f
fdup
cr ." Height " fe.
Radius
VolOfCyl
." Volume " fe.
1 +
repeat
drop
cr ;

17 20 1.0e CylVolLoop

Radius 1.0000E0
Height 17.000E0 Volume 53.407E0
Height 18.000E0 Volume 56.549E0
Height 19.000E0 Volume 59.690E0
Height 20.000E0 Volume 62.832E0

[Buzz McCool]

...

\ Without locals...
: CylVolLoop ( StartHeight FinalHeight Radius -- )
cr ." Radius " fdup fe.
swap ( FinalHeight Height)
begin 2dup >= while
dup s>f fdup cr ." Height " fe.
fover ( Height Radius) VolOfCyl ." Volume " fe.
1+
repeat 2drop fdrop
cr ;
see CylVolLoop
...
( 148 bytes, 27 instructions )

Nice. I will study your technique.

[dxf]

Post by Buzz McCool
...

\ Without locals...
: CylVolLoop ( StartHeight FinalHeight Radius -- )
   cr ." Radius " fdup fe.
   swap ( FinalHeight Height)
   begin 2dup >= while
     dup s>f  fdup cr ." Height " fe.
     fover ( Height Radius)  VolOfCyl ." Volume " fe.
     1+
   repeat 2drop fdrop
   cr ;
see CylVolLoop
...
( 148 bytes, 27 instructions )

Nice. I will study your technique.

Efficient use of the stack is Moore's technique :)

[dxf]

Post by Buzz McCool

Post by minforth
use locals if you have too many parameters

I like this quite a bit. Tell me if I like it too much.
: CylVolLoop {: W: StartHeight W: FinalHeight F: Radius -- Tabular Output :}
cr ." Radius " Radius fe.
StartHeight
begin dup FinalHeight <=
while
dup
s>f
fdup
cr ." Height " fe.
Radius
VolOfCyl
." Volume " fe.
1 +
repeat
drop
cr ;

Under VFX Forth:

see CylVolLoop
...
( 193 bytes, 39 instructions )

\ Without locals...

: CylVolLoop ( StartHeight FinalHeight Radius -- )
cr ." Radius " fdup fe.
swap ( FinalHeight Height)
begin 2dup >= while
dup s>f fdup cr ." Height " fe.
fover ( Height Radius) VolOfCyl ." Volume " fe.
1+
repeat 2drop fdrop
cr ;

see CylVolLoop
...
( 148 bytes, 27 instructions )

[Hans Bezemer]

Post by minforth
use DO..LOOPs to hide away loop indices
use locals if you have too many parameters
(some technical/physical formulas are difficult
or impossible to factorise into smaller words
which would otherwise be the classic Forth mantra)

Tips:
- Use multiple Return Stack registers (R@, R'@, R"@);
- If parameters come in duplets or triplets, use corresponding stack
operators (3DUP, 3OVER, 3DROP);
- Reorganize parameters at the *very start* of the program in a more
palatable order. It saves stack juggling later on;
- Maybe a strange one, but codify stack patterns!
E.g. SPIN ( a b c -- c b a)
STOW ( a b -- a a b)
RISE ( a b c -- b a c)

It helps you to THINK in these patterns and more easily recognize them.
It depends highly on your coding habits, so it helps to analyze your
legacy code to see if they often occur.

Hans Bezemer

[minforth]

Post by Hans Bezemer
- If parameters come in duplets or triplets, use corresponding stack
operators (3DUP, 3OVER, 3DROP);
- Reorganize parameters at the *very start* of the program in a more
palatable order. It saves stack juggling later on;
- Maybe a strange one, but codify stack patterns!
E.g. SPIN ( a b c -- c b a)
STOW ( a b -- a a b)
RISE ( a b c -- b a c)
It helps you to THINK in these patterns and more easily recognize them.
It depends highly on your coding habits, so it helps to analyze your
legacy code to see if they often occur.

Good advice if you can access the return stack directly.

Otherwise, for non-trivial words, it is preferable to let the compiler
recognise patterns and save your precious human time. If the compiled
code is too bad, profile and optimise it afterwards.

[Hans Bezemer]

Post by minforth

Post by Hans Bezemer
- If parameters come in duplets or triplets, use corresponding stack
operators (3DUP, 3OVER, 3DROP);
- Reorganize parameters at the *very start* of the program in a more
palatable order. It saves stack juggling later on;
- Maybe a strange one, but codify stack patterns!
   E.g. SPIN ( a b c -- c b a)
        STOW ( a b -- a a b)
        RISE ( a b c -- b a c)
It helps you to THINK in these patterns and more easily recognize them.
It depends highly on your coding habits, so it helps to analyze your
legacy code to see if they often occur.

Good advice if you can access the return stack directly.
Otherwise, for non-trivial words, it is preferable to let the compiler
recognise patterns and save your precious human time. If the compiled
code is too bad, profile and optimise it afterwards.

You know - in my experience these kinds of problems mostly manifest
themselves when making my library routines - the stuff you rarely touch
afterwards (and even more rarely in a fundamental way).

Putting the application components to work doesn't affect the stack in
the same way. I think there is where the "10x savings" actually are.

Again - just a hunch of mine..

Hans Bezemer

[dxf]

Post by minforth
use DO..LOOPs to hide away loop indices
use locals if you have too many parameters
(some technical/physical formulas are difficult
or impossible to factorise into smaller words
which would otherwise be the classic Forth mantra)

- If parameters come in duplets or triplets, use corresponding stack operators (3DUP, 3OVER, 3DROP);
- Reorganize parameters at the *very start* of the program in a more palatable order. It saves stack juggling later on;
- Maybe a strange one, but codify stack patterns!
  E.g. SPIN ( a b c -- c b a)
       STOW ( a b -- a a b)
       RISE ( a b c -- b a c)
It helps you to THINK in these patterns and more easily recognize them. It depends highly on your coding habits, so it helps to analyze your legacy code to see if they often occur.

swap rot 0
over swap 0
rot swap 1

[dxf]

...
Does anyone have suggestions on a better approach when you have several parameters and loop counts to deal with?

I see little wrong with your example other than cosmetics - excess comments
that don't add value and missing stack parameter comment in colon definitions.

[BuzzMcCool]

Post by dxf

...
Does anyone have suggestions on a better approach when you have several parameters and loop counts to deal with?

I see little wrong with your example other than cosmetics - excess comments
that don't add value and missing stack parameter comment in colon definitions.

Thanks for the feedback. Yes I do need to work on my stack parameter
comments.

[Hans Bezemer]

Post by BuzzMcCool

Post by dxf

Post by Buzz McCool
...
Does anyone have suggestions on a better approach when you have
several parameters and loop counts to deal with?

I see little wrong with your example other than cosmetics - excess comments
that don't add value and missing stack parameter comment in colon definitions.

Thanks for the feedback. Yes I do need to work on my stack parameter
comments.

Given that the area of the circle doesn't change - why recalculate that
every time? Ok, I changed VolOfCirc a bit, but it saves me both time and
complexity. Note this only works if there is a separate FP stack. Which
is the standard nowadays.

Alternatives:
1. Change the order of parameters (float last);
2. Change the order of parameters (carnal knowledge of the size of a float);
3. Specify the radius as an integer.

: AreaOfCir fdup pi f* f* ;
: VolOfCyl f* ;

: CylVolLoop
cr ." Radius " fdup fe.
AreaOfCir 1+ swap ?do
i s>f fdup cr ." Height " fe.
fover VolOfCyl ." Volume " fe.
loop fdrop
;

Hans Bezemer

[Hans Bezemer]

Post by Hans Bezemer

Post by BuzzMcCool

Post by dxf

Post by Buzz McCool
...
Does anyone have suggestions on a better approach when you have
several parameters and loop counts to deal with?

I see little wrong with your example other than cosmetics - excess comments
that don't add value and missing stack parameter comment in colon definitions.

Thanks for the feedback. Yes I do need to work on my stack parameter
comments.

Given that the area of the circle doesn't change - why recalculate that
every time? Ok, I changed VolOfCirc a bit, but it saves me both time and
complexity. Note this only works if there is a separate FP stack. Which
is the standard nowadays.
1. Change the order of parameters (float last);
2. Change the order of parameters (carnal knowledge of the size of a float);
3. Specify the radius as an integer.

This is the same routine with a shared stack. Note I used option 3. here
- it retains the same possibilities as the original. Note this is in
4tH. F% is followed by an FP number:

include lib/fp2.4th
include lib/zenconst.4th
include 4pp/lib/float.4pp

: AreaOfCir fdup pi f* f* ;
aka f* VolOfCyl ( 4tH alias)

: CylVolLoop ( radius start end --)

Post by Hans Bezemer
r >r cr ." Radius " fdup fe.

AreaOfCir r> r> 1+ swap ?do
i s>f fdup cr ." Height " fe.
fover VolOfCyl ." Volume " fe.
loop fdrop cr
;

f% 1.2 1 20 CylVolLoop

Radius 1.E0
Height 1.E0 Volume 3.141592653589793238E0
Height 2.E0 Volume 6.283185307179586476E0
Height 3.E0 Volume 9.42477796076937971E0
...
Height 19.E0 Volume 59.69026041820607152E0
Height 20.E0 Volume 62.83185307179586476E0

[Hans Bezemer]

Post by Hans Bezemer
f% 1.2 1 20 CylVolLoop
Radius 1.E0

Yeah, I copied the last test with the output of the fist test. My bad..
Sorry ;-)

Should have been: f% 1 1 20 CylVolLoop

Hans Bezemer

[Buzz McCool]

Post by Hans Bezemer
Given that the area of the circle doesn't change - why recalculate that
every time?

Excellent observation.

Would you have any videos talking about Forth locals? You and dxf are
far more adept at stack manipulations than I. I'm thinking I can get a
word up and working with locals and then convert to manual stack
manipulations afterwards if necessary.

When is it necessary? dxf showed a word w/o locals to have ~%30 fewer
instructions than a word with locals. Is that a common occurrence?

[Hans Bezemer]

Post by Buzz McCool

Post by Hans Bezemer
Given that the area of the circle doesn't change - why recalculate
that every time?

Excellent observation.
Would you have any videos talking about Forth locals? You and dxf are
far more adept at stack manipulations than I. I'm thinking I can get a
word up and working with locals and then convert to manual stack
manipulations afterwards if necessary.

Oh, I talk a lot about locals: don't use them. The point is: you have
random access to locals. So I doubt very much it will help you to
uncover a smart way to do it without them. Basically any non-Forth
Algol-like language will do the job.

And that's in essence you I am opposed to them. It takes out what makes
Forth unique - and the way thinking of Forth unique.

Post by Buzz McCool
When is it necessary? dxf showed a word w/o locals to have ~%30 fewer
instructions than a word with locals. Is that a common occurrence?

I can't really tell. In 4tH (my own implementation) the use of locals
requires an external library - so it always consumes more instructions.
It also heavily depends on the style and the skill of the programmer. If
you're a newbie doing a lot of stack acrobatics, I doubt it.

What bothers me most technologically is that parameters flow through the
stack undisturbed. You break that paradigm when using locals. With
locals you *HAVE TO* create some kind of stack frame that you have to
destroy when you exit.

Needless to say this copying, releasing and stuff takes time. Even when
you don't use locals. In all honesty I must state that this overhead is
not always translated to a diminished performance - at least not in the
tests I did.

****
TL;DR my objections are mostly based on pure architectural arguments,
rather than practicality. I also don't like Python, PHP and Perl for
those very same reasons - one because I think its paradigms are
fundamentally flawed, the second and third because of their "have we
thrown in the kitchen sink yet" mentality.

I don't think there will ever be a "Back&Forth" episode on locals -
frankly, because - apart from some demonstrations - there is only one
single, ported program that uses locals in my repository. How can you
teach if you never used them yourself?
****

Note that 4tH features R@, R'@ and R"@ which can server very
conveniently as "local variables" - provided you leave the Return Stack
alone. I learned that trick from the programmer of the FIG editor.

See:
https://sourceforge.net/p/forth-4th/code/HEAD/tree/trunk/4th.src/lib/gcircle.4th
for a nice example of that one.

Hans Bezemer

[Paul Rubin]

Post by Hans Bezemer
What bothers me most technologically is that parameters flow through
the stack undisturbed. You break that paradigm when using locals. With
locals you *HAVE TO* create some kind of stack frame that you have to
destroy when you exit.

Forth programs very frequently end up juggling parameters and other data
to and from the return stack, instead of using locals. Simple
implementations of locals put them in the return stack too.
"Destroying" the stack frame just means adjusting RP when the function
exits. Usually a single instruction.

Post by Hans Bezemer
Needless to say this copying, releasing and stuff takes time.

Similar to DUP (copy) or DROP (release).

Post by Hans Bezemer
In all honesty I must state that this overhead is not always
translated to a diminished performance

Right, I don't think one can assert a performance hit without
measurements supporting the idea.

Post by Hans Bezemer
TL;DR my objections are mostly based on pure architectural arguments,
rather than practicality.

Sure, that's reasonable, it's a matter of what you prefer. That's
harder to take issue with than claims about performance.

Post by Hans Bezemer
I also don't like Python, PHP and Perl for those very same reasons -

Those are at a totally different level than Forth, in terms of layers of
implementation and runtime libraries, overhead, etc. It's better to
compare to something like C, or a hypothetical cleaned up version of C,
or even to Forth with locals ;).

[dxf]

Post by Paul Rubin

Post by Hans Bezemer
What bothers me most technologically is that parameters flow through
the stack undisturbed. You break that paradigm when using locals. With
locals you *HAVE TO* create some kind of stack frame that you have to
destroy when you exit.

Forth programs very frequently end up juggling parameters and other data
to and from the return stack, instead of using locals. Simple
implementations of locals put them in the return stack too.
"Destroying" the stack frame just means adjusting RP when the function
exits. Usually a single instruction.
...

In forth the programmer uses the return stack as a temporary holder. Not
so locals which spill all input to the return stack and then shuffle these
to/from the parameter stack. The latter is akin to a novice programmer who
uses too many variables.

[dxf]

Post by Paul Rubin

Post by Hans Bezemer
What bothers me most technologically is that parameters flow through
the stack undisturbed. You break that paradigm when using locals. With
locals you *HAVE TO* create some kind of stack frame that you have to
destroy when you exit.

Forth programs very frequently end up juggling parameters and other data
to and from the return stack, instead of using locals.

Looking at an application with 154 colon definitions, only 2 were found
to use the return stack for temporary storage. Even I was surprised :)

[dxf]

Post by dxf

Post by Paul Rubin

Post by Hans Bezemer
What bothers me most technologically is that parameters flow through
the stack undisturbed. You break that paradigm when using locals. With
locals you *HAVE TO* create some kind of stack frame that you have to
destroy when you exit.

Forth programs very frequently end up juggling parameters and other data
to and from the return stack, instead of using locals.

Looking at an application with 154 colon definitions, only 2 were found
to use the return stack for temporary storage. Even I was surprised :)

From the same app:

dup 54
drop 29
swap 22
over 16
2drop 9
rot 8
2dup 3

Post by dxf
r 2

r> 2
2swap 1
2nip 1
locals 0

The easiest stack operations (DUP DROP) account for most. SWAP averaged
1 in 7 definitions. OVER 1 in 9. Is 'stack juggling' a problem in forth?
It doesn't appear to be.

[Paul Rubin]

Post by dxf

Looking at an application with 154 colon definitions...

The easiest stack operations (DUP DROP) account for most.

Is the code for this app available?

Post by dxf
SWAP averaged 1 in 7 definitions. OVER 1 in 9. Is 'stack juggling' a
problem in forth? It doesn't appear to be.

The 100+ occurrences of DUP, DROP, and SWAP are either an abstraction
inversion (with a smart compiler, the data ends up in registers that
could be named by locals) or they are stack traffic whose cost has to be
compared with the cost of indexed references to locals in the return
stack. I'd agree that they aren't necessary "juggling" which evokes
permuting stuff in the stack outside the usual FIFO order. That does
happpen a little bit though, with OVER, ROT, etc.

[dxf]

Post by Paul Rubin

Post by dxf

Looking at an application with 154 colon definitions...

The easiest stack operations (DUP DROP) account for most.

Is the code for this app available?

Previously posted. You may have seen it.

https://pastebin.com/2xcRSbQW

Post by Paul Rubin

Post by dxf
SWAP averaged 1 in 7 definitions. OVER 1 in 9. Is 'stack juggling' a
problem in forth? It doesn't appear to be.

The 100+ occurrences of DUP, DROP, and SWAP are either an abstraction
inversion (with a smart compiler, the data ends up in registers that
could be named by locals) or they are stack traffic whose cost has to be
compared with the cost of indexed references to locals in the return
stack. I'd agree that they aren't necessary "juggling" which evokes
permuting stuff in the stack outside the usual FIFO order. That does
happpen a little bit though, with OVER, ROT, etc.

If a cost, it's one the programmer can keep to minimum. With locals there's
an upfront cost that can't be avoided. Using registers is appealing until
one realizes a call to an external function necessitates placing it back on
the stack. Costs multiply in the face of many small functions. Moore touches
on this in one of his speeches:

"I keep asking that question. What is Forth? Forth is highly factored code.
I don't know anything else to say except that Forth is definitions. If you
have a lot of small definitions you are writing Forth. In order to write a
lot of small definitions you have to have a stack. Stacks are not popular.
Its strange to me that they are not. There is a just lot of pressure from
vested interests that don't like stacks, they like registers. Stacks are not
a solve all problems concept but they are very very useful, especially for
information hiding and you have to have two of them." - Chuck Moore 1999

[minforth]

Post by dxf
If a cost, it's one the programmer can keep to minimum. With locals there's
an upfront cost that can't be avoided. Using registers is appealing until
one realizes a call to an external function necessitates placing it back on
the stack. Costs multiply in the face of many small functions.

This is history (or your archaic compiler). Modern compilers try to pass
most parameters through registers.

https://langdev.stackexchange.com/questions/2584/are-modern-compilers-passing-parameters-in-registers-instead-of-on-the-stack

[mhx]

Post by minforth
This is history (or your archaic compiler). Modern compilers try to pass
most parameters through registers.

The rules are very complicated, though. One has to account for there
being
too many parameters, for different architectures with different register
assignments, for integer and floating-point type parameters, and under
some
circumstances both the registers *and* the stack must be used, where
some
extra 'working space' may, or may not, be needed.

I was very happy when it finally worked on all of our target OSes.

-marcel

[minforth]

I can well imagine that. Some wheels are particularly difficult
to reinvent. For desktop systems, it can therefore make sense
to use an IR (e.g. LLVM or WASM, or simply C) and use the
optimisation functions of proven compilers for this IR.

Sometimes a much simpler solution: use code inlining.

[Anton Ertl]

Post by dxf
Using registers is appealing until
one realizes a call to an external function necessitates placing it back on
the stack.

Not if the stack item does not live across the call. And even if it
lives across the call and cannot be placed in a callee-saved register,
the save before and restore after the call is amortized typically
across more than one register access on each side of the call.

Register allocation is one of the most effective optimizations in
compilers. That's also true of Forth.

Post by dxf
Costs multiply in the face of many small functions.

Register allocation is also effective for small functions.

- anton

[minforth]

Post by Anton Ertl
Register allocation is one of the most effective optimizations in
compilers. That's also true of Forth.

Post by dxf
Costs multiply in the face of many small functions.

Register allocation is also effective for small functions.

Moore talked about registers. It's worth repeating for those who may be
new
to forth.
"But such registers raises the question of local variables. There is a
lot of
discussion about local variables. That is another aspect of your
application
where you can save 100% of the code. I remain adamant that local
variables
are not only useless, they are harmful. If you are writing code that
needs
them you are writing, non-optimal code" - Chuck Moore 1999

The only thing that can be deduced from this is that back in 1999
this was Moore's opinion in the specific context of his work.

Besides, the world has changed a wee bit since then...

[dxf]

Post by minforth

Post by Anton Ertl
Register allocation is one of the most effective optimizations in
compilers.  That's also true of Forth.

Post by dxf
Costs multiply in the face of many small functions.

Register allocation is also effective for small functions.

Moore talked about registers.  It's worth repeating for those who may be
new
to forth.
"But such registers raises the question of local variables.  There is a
lot of
 discussion about local variables.  That is another aspect of your
application
 where you can save 100% of the code.  I remain adamant that local
variables
 are not only useless, they are harmful.  If you are writing code that
needs
 them you are writing, non-optimal code" - Chuck Moore 1999

The only thing that can be deduced from this is that back in 1999
this was Moore's opinion in the specific context of his work.
Besides, the world has changed a wee bit since then...

Claims made in respect of locals in forth - ease of use, better performance
through less 'stack juggling', better readability/maintainability - were all
made in the 1980's. What has changed? Forthers today are more willing to
believe, to accept the word of authority, lack the interest to discover the
truth for themselves? If so, that would be a pity.

[Paul Rubin]

Post by dxf

"I remain adamant that local variables  are not only useless, they
are harmful. If you are writing code that needs  them you are
writing, non-optimal code" - Chuck Moore 1999 ...

Claims made in respect of locals in forth - ease of use, better
performance through less 'stack juggling', better
readability/maintainability - were all made in the 1980's. What has
changed? Forthers today are more willing to believe, to accept the
word of authority, lack the interest to discover the truth for
themselves?

Is avoiding locals because of the Chuck Moore quote not an example of
accepting the word of authority? And how often do even you care whether
your code is optimal? It's likely difficult to get any interpreted
Forth code to run at better than 1/5th the speed of assembly code. So
if optimization is your main concern, why use Forth to begin with?

I would say that the claim of better performance from locals depends on
the implementation and in any case has to be scrutinized if it matters,
but even if there's a performance loss, that might be an acceptable
trade if the programmer finds offsetting gains in the other areas.

My main programming language for random hacking is Python, which is
possibly 10x slower than interpreted Forth or 50x slower than compiled
Forth or C. Yet it usually doesn't matter unless I'm trying to do
something unusually compute intensive. Once the program is fast enough
to not be annoying to use, I don't need to optimize it more.

[Jan Coombs]

On Fri, 13 Sep 2024 03:38:51 -0700

Post by Paul Rubin
I would say that the claim of better performance from locals depends
on the implementation[...]

Absolutely. As Chucks prime target of interest (hardware) uses LIFO
registers for stacks, only the top top one, or so, R stack items could
be used for restricted local storage (which is also common practice).

I accept that locals are useful, and would like to see hardware stack
engine implementations that support this better while retaining the
performance advantage of a stack cache implemented as LIFO registers
rather than in RAM.

Jan Coombs
--

[Anton Ertl]

Post by Jan Coombs
Absolutely. As Chucks prime target of interest (hardware) uses LIFO
registers for stacks, only the top top one, or so, R stack items could
be used for restricted local storage (which is also common practice).
I accept that locals are useful, and would like to see hardware stack
engine implementations that support this better while retaining the
performance advantage of a stack cache implemented as LIFO registers
rather than in RAM.

AFAIK Chuck Moore implements the stack as SRAM indexed with his stack
pointer; maybe the stack pointer is a rotating shift register with
only one bit set, don't remember.

He also uses an A register in addition to R and the data TOS last I
looked. So much for Chuck Moore denouncing registers. When he
introduced A, some people played with the idea to add A and possibly
more registers to Forth.

- anton

[dxf]

Post by Paul Rubin

Post by dxf

"I remain adamant that local variables  are not only useless, they
are harmful. If you are writing code that needs  them you are
writing, non-optimal code" - Chuck Moore 1999 ...

Claims made in respect of locals in forth - ease of use, better
performance through less 'stack juggling', better
readability/maintainability - were all made in the 1980's. What has
changed? Forthers today are more willing to believe, to accept the
word of authority, lack the interest to discover the truth for
themselves?

Is avoiding locals because of the Chuck Moore quote not an example of
accepting the word of authority?

Or I've yet to hear a convincing argument from the locals authorities :)

You have the source to my app. Perhaps you can nominate where locals
could have been used to better effect.

[Paul Rubin]

Post by dxf
You have the source to my app. Perhaps you can nominate where locals
could have been used to better effect.

: EMITS ( n char -- ) swap 0 ?do dup emit loop drop ;

could be written:

: EMITS {: n char -- :} n 0 ?do char emit loop ;

[dxf]

Post by Paul Rubin

Post by dxf
You have the source to my app. Perhaps you can nominate where locals
could have been used to better effect.

: EMITS ( n char -- ) swap 0 ?do dup emit loop drop ;
: EMITS {: n char -- :} n 0 ?do char emit loop ;

Compiling under DX-Forth resulted in a code size of 23 and 26 bytes
respectively. Under VFX ...

( 71 bytes, 18 instructions )

( 102 bytes, 28 instructions )

Not only were you able to read forth code, the result was more efficient.
Perhaps locals in forth were meant to be clever? That would explain the
interest however it's high price to pay.

[Paul Rubin]

Post by dxf
Compiling under DX-Forth resulted in a code size of 23 and 26 bytes
respectively. Under VFX ...

I can't help it if those compilers generate worse code for the locals
version. Can you conveniently try lxf?

Post by dxf
Not only were you able to read forth code, the result was more
efficient.

Sometimes it isn't too hard to read, sometimes it takes head scratching,
and sometimes I can't make any sense of it. The function Anton posted
was an example that didn't make sense. I remember thinking I might sit
down and try to figure it out to rewrite it, but it doesn't seem worth
the effort.

Anyway, if efficiency was important for that example, I'd use CODE.

[a***@spenarnc.xs4all.nl]

Post by dxf

Post by minforth

Post by Anton Ertl
Register allocation is one of the most effective optimizations in
compilers.  That's also true of Forth.

Post by dxf
Costs multiply in the face of many small functions.

Register allocation is also effective for small functions.

Moore talked about registers.  It's worth repeating for those who may be
new
to forth.
"But such registers raises the question of local variables.  There is a
lot of
 discussion about local variables.  That is another aspect of your
application
 where you can save 100% of the code.  I remain adamant that local
variables
 are not only useless, they are harmful.  If you are writing code that
needs
 them you are writing, non-optimal code" - Chuck Moore 1999

The only thing that can be deduced from this is that back in 1999
this was Moore's opinion in the specific context of his work.
Besides, the world has changed a wee bit since then...

Claims made in respect of locals in forth - ease of use, better performance
through less 'stack juggling', better readability/maintainability - were all
made in the 1980's. What has changed? Forthers today are more willing to
believe, to accept the word of authority, lack the interest to discover the
truth for themselves? If so, that would be a pity.

I object to locals because it introduce a superfluous extra concept.
It is foreign to a stack oriented language.
Also there are numerous conflicting notations, and giving a name to a
single cell, isn't sufficient. You need not local doubles, floats and
structures.
There are people fond of their information hiding aspect, that can
easily be done with normal data and an addition like marking
some words private.
The remaining argument is re-entrancy, an overrated argument.

I am also fond of Algol68/go. A different end of the spectrum,
but it has a common feature that Forth has: consistency.
Local variables break that.

I don't take Moore's word for gospel, but I pay attention, because
he is an accomplished individual.

Groetjes Albert

[Anton Ertl]

Post by dxf
Claims made in respect of locals in forth - ease of use, better performance
through less 'stack juggling', better readability/maintainability - were all
made in the 1980's.

Where can I find claims about better performance? All I have read is
claims about worse performance.

Post by dxf
What has changed? Forthers today are more willing to
believe, to accept the word of authority

Is that why you cite Chuck Moore on locals rather than arguing from
facts?

- anton

[dxf]

Post by Anton Ertl

Post by dxf
Claims made in respect of locals in forth - ease of use, better performance
through less 'stack juggling', better readability/maintainability - were all
made in the 1980's.

Where can I find claims about better performance? All I have read is
claims about worse performance.

'Eliminate stack juggling' sounds like an argument for better performance.
It's a catch cry that's become synonymous with locals. Identify something
wrong with forth and introduce a solution is the gameplay.

Post by Anton Ertl

Post by dxf
What has changed? Forthers today are more willing to
believe, to accept the word of authority

Is that why you cite Chuck Moore on locals rather than arguing from
facts?

The facts AFAICT is locals are an appeal to prejudice. If locals were a bona-
fide extension it ought to be crystal clear when to apply them and when not.
Vague statements about readability and maintainability don't cut it. The fact
is locals challenge and contradict forth - why I'm vitally interested in getting
at the truth of it. The best way I knew of doing that is see whether I needed
locals in practice. When the result is good forth coding can stand on its own,
why shouldn't I quote Moore.

[minforth]

Post by dxf
The facts AFAICT is locals are an appeal to prejudice.

This is one of the best sentences ever uttered on this forum! :-)

[Anton Ertl]

Post by dxf

Post by Anton Ertl
Where can I find claims about better performance? All I have read is
claims about worse performance.

'Eliminate stack juggling' sounds like an argument for better performance.

Not to me. To me it sounds like a statement about the ease of writing
and reading the code.

The performance of locals vs. stack juggling depends on the
implementation. I know no implementation that performs register
allocation of locals or stack items (except the TOS) to registers
across basic block boundaries. This seems to hurt code with locals
more than code that keeps everything on the stacks. Here's the data
from an earlier posting <***@mips.complang.tuwien.ac.at>,
now including data from iForth:

locals stack
401 336 gforth-fast (AMD64)
179 132 lxf 1.6-982-823 (IA-32)
182 119 VFX FX Forth for Linux IA32 Version: 4.72 (IA-32)
241 159 VFX Forth 64 5.43 (AMD64)
163 175 iforth-5.1 mini (AMD64)

The data from iForth is the outlier here, let's look at the code:

Source code:
defer dummy
: z" [char] " parse 2drop postpone dummy ; immediate
defer zformat
defer z+
defer >name
defer error

: VICHECK1 {: pindex paddr -- pindex' paddr :} \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
pindex 0 paddr @ WITHIN IF \ Index is valid
pindex paddr
ELSE \ Index is invalid
Z" Invalid index " pindex ZFORMAT Z+
Z" for " Z+ paddr >NAME 1+ Z+ \ >NAME does not work for separated data
Z" length " Z+ paddr @ ZFORMAT Z+
ERROR
0 paddr \ Use zeroth index
THEN ;

: VICHECK2 ( pindex paddr -- pindex' paddr ) \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
over 0 2 pick @ WITHIN 0= IF \ Index is invalid
Z" Invalid index " 2 PICK ZFORMAT Z+
Z" for " Z+ OVER CELL- @ Z+ \ Add NFA from extra cell
Z" length " Z+ OVER @ ZFORMAT Z+
ERROR
NIP 0 SWAP \ Use zeroth index
THEN ;

One difference is that VICHECK2 does not just replace the locals with
stack stuff and eliminate the first branch of the IF, but also
replaces ">NAME 1+" with "CELL- @".

Disassembled code:
VICHECK1 VICHECK2
pop rbx pop rbx
lea rsi, [rsi #-16 +] qword mov rdi, [rsp] qword
mov [esi] dword, rbx push rbx
pop rbx push rdi
lea rsi, [rsi #-16 +] qword push 0 b#
mov [esi] dword, rbx mov rbx, [rsp #16 +] qword
mov rbx, [rsi #16 +] qword pop rdi
mov rbx, [rbx] qword mov rax, rdi
mov rdi, [rsi] qword sub rax, [rbx] qword
cmp rbx, rdi neg rax
jbe $10227337 offset NEAR pop rbx
push [rsi] qword sub rbx, rdi
push [rsi #16 +] qword cmp rax, rbx
jmp $10227395 offset NEAR seta bl
call $10226600 qword-offset movzx rbx, bl
push [rsi] qword neg rbx
call $10226E90 qword-offset cmp rbx, 0 b#
call $10226EB0 qword-offset jne $10227465 offset NEAR
call $10226600 qword-offset call $10226600 qword-offset
call $10226EB0 qword-offset mov rbx, [rsp #16 +] qword
push [rsi #16 +] qword push rbx
call $10226ED0 qword-offset call $10226E90 qword-offset
pop rbx call $10226EB0 qword-offset
lea rbx, [rbx 1 +] qword call $10226600 qword-offset
push rbx call $10226EB0 qword-offset
call $10226EB0 qword-offset pop rbx
call $10226600 qword-offset mov rdi, [rsp] qword
call $10226EB0 qword-offset push rbx
mov rbx, [rsi #16 +] qword push [rdi -8 +] qword
push [rbx] qword call $10226EB0 qword-offset
call $10226E90 qword-offset call $10226600 qword-offset
call $10226EB0 qword-offset call $10226EB0 qword-offset
call $10226EF0 qword-offset pop rbx
push 0 b# mov rdi, [rsp] qword
push [rsi #16 +] qword push rbx
add rsi, #32 b# push [rdi] qword
; call $10226E90 qword-offset
call $10226EB0 qword-offset
call $10226EF0 qword-offset
pop rbx
pop rdi
mov rdi, 0 d#
mov rcx, rdi
push rcx
push rbx
;

iForth 5.1-mini does not even keep the TOS in a register on basic
block boundaries, which results in pops and pushes at all the
boundaries, especially for the stack-only code. However, in the
actual application (where Z", ZFORMAT etc. don't compile as deferred
words) it would probably inline many of these words which might result
in better code for the stack variant. It does not keep locals in
stack items, either, but accesses them in memory through a separate
stack pointer.

The code at the start of VICHECK2 does not suffer from basic block
boundaries, yet makes less use of registers than I expected. By
contrast, in VICHECK1 iforth discovers that "0 paddr @ within" is
equivalent to "paddr @ u<", while for "0 2 pick @ within" it fails to
make the equivalent discovery.

- anton

[dxf]

Post by Anton Ertl

Post by dxf

Post by Anton Ertl
Where can I find claims about better performance? All I have read is
claims about worse performance.

'Eliminate stack juggling' sounds like an argument for better performance.

Not to me. To me it sounds like a statement about the ease of writing
and reading the code.
The performance of locals vs. stack juggling depends on the
implementation.
...

Surely you mean locals vs. forth. The easiest way to achieve performance
in forth is making your stack operations efficient. 'Stack juggling' is
a visual cue that it's not. I'm sorry that you feel forth isn't readable.

[Anton Ertl]

Post by dxf
https://pastebin.com/2xcRSbQW

Post by dxf
SWAP averaged 1 in 7 definitions. OVER 1 in 9. Is 'stack juggling' a
problem in forth? It doesn't appear to be.

: ARG ( n -- adr len -1 | 0 )

Post by dxf
r 0 0 cmdtail r> 0 ?do

2nip
bl skip 2dup bl scan
rot over - -rot
loop 2drop
dup if -1 end and ;

The heavy use of global variables in this program also does not
support the idea that proper usage of the stacks makes locals
unnecessary.

- anton

[Ahmed]

Hi,
In fuzzy logic, a triangular membership function mf(x;a,b,c) is defined
as:

mf(x;a,b,c) = (x-a)/(b-a) for a <= x < b,
(c-x)/(c-b) for b <= x < c,
0e elsewere.

defining it with locals:

: tri_mf() { f: x f: a f: b f: c } ( f: x a b c -- mv)
x a f>= x b f< and if x a f- b a f- f/ exit then
x b f>= x c f< and if c x f- c b f- f/ exit then
0e
;

But defining it without locals ????!!!!!

: tri_mf() ( f: x a b c -- mv) ....

How?

Ahmed

[Anton Ertl]

Post by Ahmed
Hi,
In fuzzy logic, a triangular membership function mf(x;a,b,c) is defined
mf(x;a,b,c) = (x-a)/(b-a) for a <= x < b,
(c-x)/(c-b) for b <= x < c,
0e elsewere.
: tri_mf() { f: x f: a f: b f: c } ( f: x a b c -- mv)
x a f>= x b f< and if x a f- b a f- f/ exit then
x b f>= x c f< and if c x f- c b f- f/ exit then
0e
;
But defining it without locals ????!!!!!
: tri_mf() ( f: x a b c -- mv) ....
How?

I wonder if the notation "mf(x;a,b,c)" indicates that a,b,c is a tuble
that tends to get passed around without changing it. In that case
defining it as a structure in memory and accessing its members there
might be a solution.

But OTOH, unless you see programming in Forth as a religious exercise,
why worry, as long as your solution works.

- anton

[Ahmed]

Post by Anton Ertl
I wonder if the notation "mf(x;a,b,c)" indicates that a,b,c is a tuble
that tends to get passed around without changing it. In that case
defining it as a structure in memory and accessing its members there
might be a solution.

a, b and are the parameters of the membership function.
Yes, we can use structures, arrays ...

Post by Anton Ertl
But OTOH, unless you see programming in Forth as a religious exercise,
why worry, as long as your solution works.

I did it without locals as an exercise. Here it is:


Without locals:

: tri_mf: ( f: a b c )
create frot f, fswap f, f,
does> ( ad_a) ( f: x)
dup fdup ( ad_a ad_a) ( f: x x)
f@ ( ad_a) ( f: x x a)
f>= ( ad_a -1|0) ( f: x)
over float+ ( ad_a -1|0 ad_b) ( f: x)
fdup f@ ( ad_a -1|0) ( f: x x b)
f< and if ( ad_a) ( f: x)
dup f@ f- ( ad_a) ( f: x-a)
dup f@ ( ad_a) ( f: x-a a)
float+ ( ad_b) ( f: x-a a)
f@ fswap f- ( f: x-a b-a)
f/ ( f: [x-a]/[b-a])
exit
then
float+ ( ad_b) ( f: x)
dup fdup ( ad_b ad_b) ( f: x x)
f@ ( ad_b) ( f: x x b)
f>= ( ad_b -1|0) ( f: x)
over float+ ( ad_b -1|0 ad_c) ( f: x)
fdup f@ ( ad_b -1|0) ( f: x x c)
f< and if ( ad_b) ( f: x)
dup float+ f@ ( ad_b) ( f: x c)
f- ( ad_b) ( f: x-c)
dup float+ ( ad_b ad_c) ( f: x-c)
swap f@ f@ f- ( f: x-c b-c)
f/ ( f: [x-c]/[b-c])
exit
then
drop fdrop
0e
;


-1e309 -1e 0e tri_mf: neg_big
-1e 0e 1e tri_mf: zero
0e 1e 1e309 tri_mf: pos_big

: fuzzify ( f: x)
fdup neg_big cr f.
fdup zero cr f.
pos_big cr f.
;

Examples: for x in {-10e, -1e, -0.8e, -0.5e, -0.3e, 0e, 0.2e, 0.5e,
0.7e, 1e, 20e}
-10e fuzzify and so on.

\ ---------------

With locals:
: tri_mf() { f: x f: a f: b f: c } ( f: x a b c -- mv)
x a f>= x b f< and if x a f- b a f- f/ exit then
x b f>= x c f< and if c x f- c b f- f/ exit then
0e
;

: neg_big -1e309 -1e 0e tri_mf() ;
: zero -1e 0e 1e tri_mf() ;
: pos_big 0e 1e 1e309 tri_mf() ;


: fuzzify { f: x }
x neg_big cr f.
x zero cr f.
x pos_big cr f.
;


Examples: for x in {-10e, -1e, -0.8e, -0.5e, -0.3e, 0e, 0.2e, 0.5e,
0.7e, 1e, 20e}
-10e fuzzify and so on.

I notice a great difference in readibality and simplicity when using
locals.

Using gforth under WSL (Windows Subsystem for Linux):

utime 0.1e neg_big utime d- dnegate d.
with locals: about 19 ms
without locals: about 18 ms

Ahmed

[Ahmed]

Oops.
Please read micro seconds (us) instead of milli seconds (ms).

Without locals: about 18 us
with locals: about 19 us

Ahmed

[Ahmed]

Post by Ahmed
utime 0.1e neg_big utime d- dnegate d.
with locals: about 19 ms
without locals: about 18 ms
Ahmed

Oops.

Please read micro seconds (us) instead of milli seconds (ms).

with locals: about 19 us
without locals: about 18 us

Ahmed

[mhx]

Post by Ahmed

Post by Ahmed
utime 0.1e neg_big utime d- dnegate d.
with locals: about 19 ms
without locals: about 18 ms
Ahmed

Oops.
Please read micro seconds (us) instead of milli seconds (ms).
with locals: about 19 us
without locals: about 18 us

That can't be correct.

In iForth I used dfloats instead of floats
( 4.9ns instead of 7.3ns).
Using structs is not a great idea in this case.

anew -testlocals

: tri_mf: ( f: a b c )
create frot df, fswap df, df,
does> ( F: x -- y )
( ad_a) ( f: x)
dup fdup ( ad_a ad_a) ( f: x x)
df@ ( ad_a) ( f: x x a)
f>= ( ad_a -1|0) ( f: x)
over dfloat+ ( ad_a -1|0 ad_b) ( f: x)
fdup df@ ( ad_a -1|0) ( f: x x b)
f< and if ( ad_a) ( f: x)
dup df@ f- ( ad_a) ( f: x-a)
dup df@ ( ad_a) ( f: x-a a)
dfloat+ ( ad_b) ( f: x-a a)
f@ fswap f- ( f: x-a b-a)
f/ ( f: [x-a]/[b-a])
exit
then
dfloat+ ( ad_b) ( f: x)
dup fdup ( ad_b ad_b) ( f: x x)
df@ ( ad_b) ( f: x x b)
f>= ( ad_b -1|0) ( f: x)
over dfloat+ ( ad_b -1|0 ad_c) ( f: x)
fdup df@ ( ad_b -1|0) ( f: x x c)
f< and if ( ad_b) ( f: x)
dup dfloat+ df@ ( ad_b) ( f: x c)
f- ( ad_b) ( f: x-c)
dup dfloat+ ( ad_b ad_c) ( f: x-c)
swap df@ df@ f- ( f: x-c b-c)
f/ ( f: [x-c]/[b-c])
exit
then
drop fdrop
0e
;

-1e309 -1e 0e tri_mf: nol_neg_big

: (tri_mf) ( f: x a b c -- mv)
FLOCALS| c b a x |
x a f>= x b f< and if x a f- b a f- f/ exit then
x b f>= x c f< and if c x f- c b f- f/ exit then
0e ;

: loc_neg_big -1e309 -1e 0e (tri_mf) ;
: .timing MS? S>F 1e-3 F* 1e7 F/ F.N2 ." s/call." ;

: tnb CR ." \ no locals: " TIMER-RESET #10000000 ( 1e7 times )
0 DO -10e nol_neg_big FDROP LOOP .timing
CR ." \ locals: " TIMER-RESET #10000000 ( 1e7 times )
0 DO -10e loc_neg_big FDROP LOOP .timing ;

FORTH> tnb
\ no locals: 4.9ns/call.
\ locals: 21.3ns/call. ok

-marcel

[Ahmed]

Post by mhx
That can't be correct.

You are right.
I find with gforth:

: go 0 do -0.1e neg_big fdrop loop ;

without locals:
utime 100000000 go utime d>f d>f f- 1e-8 f* f. 0.06762074 us ok for 1e8
times: (67.62 ns)

and with locals:
utime 100000000 go utime d>f d>f f- 1e-8 f* f. 0.09961387 us ok for
1e8 times: (99.61 ns)

I missused the timing in the previous post.
Thanks for the correction.

Post by mhx
FORTH> tnb
\ no locals: 4.9ns/call.
\ locals: 21.3ns/call. ok
-marcel

Ahmed

[Anton Ertl]

Post by Paul Rubin
The 100+ occurrences of DUP, DROP, and SWAP are either an abstraction
inversion (with a smart compiler, the data ends up in registers that
could be named by locals)

I don't see an inversion here. The programmer-visible stack abstracts
(ideally) the registers in one way, the programmer-visible locals
abstracts them in a different way.

And if we look at the VICHECK example from Nick Nelson's Better Values
<http://www.euroforth.org/ef22/papers/nelson-values-slides.pdf> the
version with locals, followed by the version that eliminates the
locals:

: VICHECK {: pindex paddr -- pindex' paddr :} \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
pindex 0 paddr @ WITHIN IF \ Index is valid
pindex paddr
ELSE \ Index is invalid
Z" Invalid index " pindex ZFORMAT Z+
Z" for " Z+ paddr >NAME 1+ Z+ \ >NAME does not work for separated data
Z" length " Z+ paddr @ ZFORMAT Z+
ERROR
0 paddr \ Use zeroth index
THEN ;

: VICHECK ( pindex paddr -- pindex' paddr ) \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
over 0 2 pick @ WITHIN 0= IF \ Index is invalid
Z" Invalid index " 2 PICK ZFORMAT Z+
Z" for " Z+ OVER CELL- @ Z+ \ Add NFA from extra cell
Z" length " Z+ OVER @ ZFORMAT Z+
ERROR
NIP 0 SWAP \ Use zeroth index
THEN ;

So by keeping the values on the stack you not just eliminate their
repeated mention, but also eliminate one branch of the IF. With a
more capable Forth system a synthesis of the two approaches is
possible:

: VICHECK ( pindex paddr -- pindex' paddr ) \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
over 0 2 pick @ WITHIN 0= IF \ Index is invalid
{: pindex paddr :}
Z" Invalid index " pindex ZFORMAT Z+
Z" for " Z+ paddr >NAME 1+ Z+ \ >NAME does not work for separated data
Z" length " Z+ paddr @ ZFORMAT Z+
ERROR
0 paddr \ Use zeroth index
THEN ;

Or one could factor out the code between IF and THEN and stay within
the confines of VFX:

: VIERROR {: pindex paddr -- 0 paddr :}
Z" Invalid index " pindex ZFORMAT Z+
Z" for " Z+ paddr >NAME 1+ Z+ \ >NAME does not work for separated data
Z" length " Z+ paddr @ ZFORMAT Z+
ERROR
0 paddr \ Use zeroth index
;

: VICHECK ( pindex paddr -- pindex' paddr ) \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
over 0 2 pick @ WITHIN 0= IF \ Index is invalid
VIERROR
THEN ;

The check can be simplified, which also simplifies the stack handling:

: VICHECK ( pindex paddr -- pindex' paddr ) \ Checks for valid index
\ paddr is the address of the data, the first cell of which contains
\ the array size
2dup @ u>= IF \ Index is invalid
VIERROR
THEN ;

Post by Paul Rubin
or they are stack traffic whose cost has to be
compared with the cost of indexed references to locals in the return
stack.

That check often results in the code without locals winning, but that
is, for a large part, due to suboptimal implementations of locals.
Ideally a perfect compiler will produce the same code for code using
locals and for equivalent code using stack manipulation words, because
the data flow is the same. This actually works out in the case of lxf
processing various implementations of 3DUP, including a locals-based
one; see <***@mips.complang.tuwien.ac.at>. However, in
general Forth systems do not produce perfect results.

I have now looked at what happens for the first two variants of
VICHECK; I have defined the non-standard words as follows to make it
possible to compile the code:

defer dummy
: z" [char] " parse 2drop postpone dummy ; immediate
defer zformat
defer z+
defer >name
defer error

I looked at 3 systems: Gforth (because I work on it); lxf (because it
produces the best results in the 3DUP case); VFX (because it's the
system Nick Nelson uses). The numbers below are the number of bytes
of native code:

locals stack
401 336 gforth-fast (AMD64)
179 132 lxf 1.6-982-823 (IA-32)
182 119 VFX FX Forth for Linux IA32 Version: 4.72 (IA-32)
241 159 VFX Forth 64 5.43 (AMD64)

Post by Paul Rubin
I'd agree that they aren't necessary "juggling" which evokes
permuting stuff in the stack outside the usual FIFO order. That does
happpen a little bit though, with OVER, ROT, etc.

In particular, in Starting Forth ROT is illustrated with a juggler
(you see the juggling balls right beside her), and the swap dragon
comments: "I hate jugglers".

Loading Image...

- anton

[Hans Bezemer]

Post by Paul Rubin

Post by Hans Bezemer
What bothers me most technologically is that parameters flow through
the stack undisturbed. You break that paradigm when using locals. With
locals you *HAVE TO* create some kind of stack frame that you have to
destroy when you exit.

Forth programs very frequently end up juggling parameters and other data
to and from the return stack, instead of using locals. Simple
implementations of locals put them in the return stack too.
"Destroying" the stack frame just means adjusting RP when the function
exits. Usually a single instruction.

Post by Hans Bezemer
Needless to say this copying, releasing and stuff takes time.

Similar to DUP (copy) or DROP (release).

Post by Hans Bezemer
In all honesty I must state that this overhead is not always
translated to a diminished performance

Right, I don't think one can assert a performance hit without
measurements supporting the idea.

Post by Hans Bezemer
TL;DR my objections are mostly based on pure architectural arguments,
rather than practicality.

Sure, that's reasonable, it's a matter of what you prefer. That's
harder to take issue with than claims about performance.

Post by Hans Bezemer
I also don't like Python, PHP and Perl for those very same reasons -

Those are at a totally different level than Forth, in terms of layers of
implementation and runtime libraries, overhead, etc. It's better to
compare to something like C, or a hypothetical cleaned up version of C,
or even to Forth with locals ;).

A lot depends on how solid you want to make your implementation. I got
locals in uBasic/4tH.

: exec_local ( --)
[: get_exp 0 max 27 frame dup @ - + min negate cells frame + dup local <
if E.MANYLOC throw else frame @ over ! to frame then ;]
exec_function \ execution semantics for LOCALS()
;

This one reserves room for locals. You may use up to 26 locals per
function since there are 26 letters in the alphabet (duh!).

: exec_param ( --)
frame exec_local frame \ allocate locals, save pointers
begin over over > while cell+ (pop) over ! repeat drop drop
;

If the reserved room has to be initialized by the stack, it calls
EXEC_LOCAL and then copies the values there.

: exec_return ( --)
get_token paren? putback if ['] get_push exec_function then
gpop prog ! frame dup local #local 1- cells + >
if E.NOSCOPE throw ;then @ to frame
;

This one looks whether RETURN returns a value - and if it does, it
pushes this value on the stack. Then it sets the return address. It
checks for the sanity of the stack frame and if okay THEN it finally
updates the stack pointer.

You comfortable left out the initialization of the stack frame. Agreed,
if ALL values are transferred to the return stack the overhead is
minimal. But how often happens that?

Post by Paul Rubin
Those are at a totally different level than Forth, in terms of layers of
implementation and runtime libraries, overhead, etc. It's better to
compare to something like C, or a hypothetical cleaned up version of C,
or even to Forth with locals ;).

True - but that's not the level of abstraction I'm considering. I think
a language should have a well designed core, surrounded by a
constellation of extensions. Like C with its standard library and Forth
with its word sets. For comparison - C got a few dozen keywords. PHP got
at least two different ways to extend binary extensions alone. A full
Python installation is scattered all over the filesystem, so you got a
hell of a job to extract a single, transferable application. Not to
mention the awkward syntax (although they fixed some of it in v3). In
Perl you always have to wonder which prefix is fashionable today.

Now, I won't say Forth doesn't have its issues. I think IN ESSENCE
recognizers are a beautiful idea. Extend it to strings and you could
eradicate "parsing words" and have something like:

"lib/mylib.4th" include

"Square" : "the square is:" print dup * cr ;

But okay, we'll do with what we have ;-) And BTW, TURNKEY should be
standard. Clean up the dictionary, pump out an executable.

Hans Bezemer

[a***@spenarnc.xs4all.nl]

In article <nnd$545e2daa$***@548f76d6156a46d8>,
Hans Bezemer <***@gmail.com> wrote:
<SNIP>

Post by Hans Bezemer
Now, I won't say Forth doesn't have its issues. I think IN ESSENCE
recognizers are a beautiful idea. Extend it to strings and you could
"lib/mylib.4th" include
"Square" : "the square is:" print dup * cr ;

You have that backward, it must be:

{ "the square is:" print dup * cr } : Square

If there is one thing to preserve in Forth that is the
convention that defining words can parse new names in
the dictionary by forward scanning, without those considered strings.
Here { introduces a denotation, without being a PREFIX (" recognizer"),
such as 0x in 0xDEADBEEF is. It is the same within a definition like
numbers and nowadays strings.

{ "the square is:" print dup * cr } CONSTANT orang_utan
orang_utan DUP : Square : quadrate

Post by Hans Bezemer
But okay, we'll do with what we have ;-) And BTW, TURNKEY should be
standard. Clean up the dictionary, pump out an executable.

I have create a language on that principle, e.g. meta
accepts 2 xt's a build and a run one. meta is the mother of
all defining words:
{ , } { @ } meta CONSTANT
{ CELL ALLOT } { } meta VARIABLE
{ 2 CELLS ALLOT } { } meta 2VARIABLE
{ } { EXECUTE } meta :
{ } { } meta DATA \ My favorite.

CREATE DOES> is the right idea, an object with an allocation
part and a behavior, but the syntax is akward beyond despair.

I have a backlog, busy with preserving projects dating from the
80's, so don't expect a publication soon.

Post by Hans Bezemer
Hans Bezemer

[Paul Rubin]

Post by Hans Bezemer
You comfortable left out the initialization of the stack
frame. Agreed, if ALL values are transferred to the return stack the
overhead is minimal. But how often happens that?

I don't understand this. {: a b c :} transfers 3 elements from the
parameter stack to the return stack. That has some cost, but it is
offset by avoiding some DUP and similar operations. Is it relevant at
all anyway? Old fashioned Forth interpreters are pretty fast, and if
you're worrying about avoiding a stack transfer here or there, you need
an optimizing compiler.

Adding safety checks has a cost, but once the program appears debugged,
I think Forth philosophy is to turn off the checks.

Post by Hans Bezemer
True - but that's not the level of abstraction I'm considering. I
think a language should have a well designed core, surrounded by a
constellation of extensions. Like C with its standard library and
Forth with its word sets.

You might like Lua or Scheme for simple higher level languages with that
style of design. C has some warts but its complexity in terms of
keywords doesn't seem much worse than Forth's core words.

[Stephen Pelc]

Post by Buzz McCool
Would you have any videos talking about Forth locals? You and dxf are
far more adept at stack manipulations than I. I'm thinking I can get a
word up and working with locals and then convert to manual stack
manipulations afterwards if necessary.

Don't. You will only become dependent on locals. Use of locals should
be a considered decision.

Post by Buzz McCool
When is it necessary? dxf showed a word w/o locals to have ~%30 fewer
instructions than a word with locals. Is that a common occurrence?

We (MPE) converted much of our TCP/IP stack not to use locals. This
was mostly on ARM7 devices, but the figures for other 32 bit CPUs of
the period (say 15 years ago) were similar. Code density improved by
about 25% and performance by about 50%.

Stephen

[minforth]

Post by Stephen Pelc

Post by Buzz McCool
Would you have any videos talking about Forth locals? You and dxf are
far more adept at stack manipulations than I. I'm thinking I can get a
word up and working with locals and then convert to manual stack
manipulations afterwards if necessary.

Don't. You will only become dependent on locals. Use of locals should
be a considered decision.

Post by Buzz McCool
When is it necessary? dxf showed a word w/o locals to have ~%30 fewer
instructions than a word with locals. Is that a common occurrence?

We (MPE) converted much of our TCP/IP stack not to use locals. This
was mostly on ARM7 devices, but the figures for other 32 bit CPUs of
the period (say 15 years ago) were similar. Code density improved by
about 25% and performance by about 50%.

These are good examples of "it depends". And also that one should never
start optimising without profiling. I have had similar experiences in
the
other direction (i.e. with locals) with vector maths.

Another observation is that many Forthers do not seem to put much
emphasis
on programming time and code maintainability or readability, which is
easier to achieve by using locals. The code conversion for your TCP/IP
stack must have taken a lot of programming time, but it must have been
worth it because it paid off on another level.

But when to use or avoid locals is an old argument that has long since
been put to rest. It all depends...

[Hans Bezemer]

Post by minforth
Another observation is that many Forthers do not seem to put much
emphasis
on programming time and code maintainability or readability, which is
easier to achieve by using locals.

I won't dispute that using the "locals" shortcut *may* save some
programming time - but to me, the moment you decide to put the whole
shebang in locals, you enter another mindset. Because at that moment you
cease to consider the algorithm itself, but start banging out code.

You no longer consider "do I need that, do I need that now, do I need
that here", you just start creating more local variables. Somehow that
kills my train of mind..

I do dispute that "no locals" Forth kills maintainability - or
readability. I'm always happy to see a whole bunch of one-liners.
Doesn't happen to me every day, but often enough. And then you can
functionally comment your code. I usually comment it from column 40 on
and at the top of a word.

I've maintained non-trivial programs for *DECADES* without any trouble.
I've plugged in a garbage collection module in my uBasic/4tH interpreter
- and radically changed it later. My rule is: if you can't figure it
out, rewrite it until you do. It happens, but not frequently.

Hans Bezemer

[minforth]

Post by Hans Bezemer
I won't dispute that using the "locals" shortcut *may* save some
programming time - but to me, the moment you decide to put the whole
shebang in locals, you enter another mindset. Because at that moment you
cease to consider the algorithm itself, but start banging out code.
You no longer consider "do I need that, do I need that now, do I need
that here", you just start creating more local variables. Somehow that
kills my train of mind..

The thing is that your train of mind is focused on optimising the
parameter flow via the stack. you are doing stupid work that an
intelligent compiler does automatically today. it makes much more sense
to focus your brainware on the algorithms or automation tasks to be
solved.

Since such algorithms/tasks are mostly formulated mathematically or
logically, an almost 1:1 translation of such formulations by using
locals
is straightforward and less error prone. Use descriptive names and the
code
becomes quasi commented simultaneously.

[dxf]

Post by minforth
...
Since such algorithms/tasks are mostly formulated mathematically or
logically, an almost 1:1 translation of such formulations by using
locals
is straightforward and less error prone. Use descriptive names and the
code
becomes quasi commented simultaneously.

Mathematical formulations are typically expressed algebraically. Forth
is stack-based and uses RPN. It's a different world. To use the latter
effectively requires a different mindset. Do you really formulate or
sketch out tasks algebraically? For me it ended when I stopped using
BASIC.

[a***@spenarnc.xs4all.nl]

Post by Hans Bezemer

Post by minforth
Another observation is that many Forthers do not seem to put much
emphasis
on programming time and code maintainability or readability, which is
easier to achieve by using locals.

I won't dispute that using the "locals" shortcut *may* save some
programming time - but to me, the moment you decide to put the whole
shebang in locals, you enter another mindset. Because at that moment you
cease to consider the algorithm itself, but start banging out code.
You no longer consider "do I need that, do I need that now, do I need
that here", you just start creating more local variables. Somehow that
kills my train of mind..
I do dispute that "no locals" Forth kills maintainability - or
readability. I'm always happy to see a whole bunch of one-liners.
Doesn't happen to me every day, but often enough. And then you can
functionally comment your code. I usually comment it from column 40 on
and at the top of a word.
I've maintained non-trivial programs for *DECADES* without any trouble.
I've plugged in a garbage collection module in my uBasic/4tH interpreter
- and radically changed it later. My rule is: if you can't figure it
out, rewrite it until you do. It happens, but not frequently.

I'm cleaning up the editor that I use all the time. It sports dozens of
global variables and it is hard to see why it could dispense with them.

LOCAL is an expensive feature, because they are re-entrant.
Forthers may know where and why an expensive feature is used.

Post by Hans Bezemer
Hans Bezemer

[Anton Ertl]

Post by Stephen Pelc
Don't. You will only become dependent on locals. Use of locals should
be a considered decision.

Post by Buzz McCool
When is it necessary? dxf showed a word w/o locals to have ~%30 fewer
instructions than a word with locals. Is that a common occurrence?

We (MPE) converted much of our TCP/IP stack not to use locals. This
was mostly on ARM7 devices, but the figures for other 32 bit CPUs of
the period (say 15 years ago) were similar. Code density improved by
about 25% and performance by about 50%.

So MPE (and Forth, Inc.) discourage the use of locals because they
implement locals inefficiently, and they implement locals
inefficiently because there are so few uses of locals around. A
chicken-and-egg problem.

Concerning the conversion of the TCP/IP stack: Have you considered the
alternative of spending MPE's time on making the locals implementation
more efficient?

See also:

@InProceedings{ertl22-locals,
author = {M. Anton Ertl},
title = {Are Locals Inevitably Slow?},
crossref = {euroforth22},
pages = {48--49},
url = {http://www.euroforth.org/ef22/papers/ertl-locals.pdf},
url-slides = {http://www.euroforth.org/ef22/papers/ertl-locals-slides.pdf},
video =

OPTnote = {presentation slides},
abstract = {Code quality of locals on two code examples on
various systems}
}

An update on the table for the example:

: 3dup.3 {: a b c :} a b c a b c ;

instr. bytes system
31 117 Gforth AMD64
16 44 iforth 5.0.27 (plus 20 bytes entry and return code)
7 19 lxf 1.6-982-823 32-bit
32 127 SwiftForth 4.0.0-RC89 (calls LSPACE)
26 92 VFX Forth 64 5.11 RC2

- anton

[Anton Ertl]

Post by Anton Ertl
@InProceedings{ertl22-locals,
author = {M. Anton Ertl},
title = {Are Locals Inevitably Slow?},
crossref = {euroforth22},
pages = {48--49},
url = {http://www.euroforth.org/ef22/papers/ertl-locals.pdf},
url-slides = {http://www.euroforth.org/ef22/papers/ertl-locals-slides.pdf},
video = http://youtu.be/tPjSKetEJn0
OPTnote = {presentation slides},
abstract = {Code quality of locals on two code examples on
various systems}
}
: 3dup.3 {: a b c :} a b c a b c ;
instr. bytes system
31 117 Gforth AMD64
16 44 iforth 5.0.27 (plus 20 bytes entry and return code)
7 19 lxf 1.6-982-823 32-bit
32 127 SwiftForth 4.0.0-RC89 (calls LSPACE)
26 92 VFX Forth 64 5.11 RC2

And here's another update. A recent change in Gforth resulted in more
code, and we now have reverted that change:

instr. bytes system
28 103 Gforth AMD64
16 44 iforth 5.0.27 (plus 20 bytes entry and return code)
7 19 lxf 1.6-982-823 32-bit
32 127 SwiftForth 4.0.0-RC89 (calls LSPACE)
26 92 VFX Forth 64 5.11 RC2

- anton