Test cases for quotations

[Ruvim]

The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".

The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;" (at least for the resulting stack
effect), but the system should correctly work regardless whether the
current definition exists (i.e., a definition that is being compiled).
It means, if a definition is being compiled, its compilation shall be
correctly suspended by "[:", and resumed after ";]" under the hood.

Bellow are the test cases for the standardized compilation semantics,
and for the expected interpretation semantics.

Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.


My implementation passes all these test cases. Its basic factors are as
follows.

The updated "current definition" term:
*current definition*: The definition whose compilation has been started
most recently but not yet ended.

CONCEIVE ( -- ) ( C: -- def-sys )
Start compilation of the new definition.
This word shall correctly work regardless whether the current definition
exists. It does not change STATE.

BIRTH ( -- xt ) ( C: def-sys -- )
End compilation of the current definition and return its xt.

GERM ( -- xt|0 )
If the current definition exists return its xt, otherwise return 0.




===== start of "quotation.test.fth"

\ Test cases for quotations "[: ... ;]"



\ Testing the compilation semantics

\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t

: lit, postpone literal ;
: ([:) postpone [: ;
: (;]) postpone ;] ;

\ t12
t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t
\ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t



\ Testing the interpretation semantics
\ (the expected behavior)

\ t21
t{ depth [: ;] depth 1- = ?dup nip -> 0 }t
\ t22
t{ [: 123 ;] execute -> 123 }t
\ t23
t{ [: 1 [: 2 ;] 3 ;] execute swap execute -> 1 3 2 }t



\ Testing the interpretation semantics
\ doing compilation of another definition
\ (the expected behavior)

\ t31
t{ [: [ depth [: ;] depth 1- = ?dup nip ] literal ;] execute -> 0 }t
\ t32
t{ [: 1 [ [: 2 ;] ] literal 3 ;] execute swap execute -> 1 3 2 }t


===== end of "quotation.test.fth"




[1] http://www.forth200x.org/quotations-v4.txt

--
Ruvim

[Ruvim]

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;" (at least for the resulting stack
effect), but the system should correctly work regardless whether the
current definition exists (i.e., a definition that is being compiled).
It means, if a definition is being compiled, its compilation shall be
correctly suspended by "[:", and resumed after ";]" under the hood.
Bellow are the test cases for the standardized compilation semantics,
and for the expected interpretation semantics.
Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.

[...]

Post by Ruvim
===== start of "quotation.test.fth"
\ Test cases for quotations "[: ... ;]"
\ Testing the compilation semantics
\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t
: lit, postpone literal ;
: ([:) postpone  [: ;
: (;]) postpone  ;] ;
\ t12
t{ :noname  [ ([:) (;]) ]  ;  0<>  ->  -1  }t > \ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t

t12 roughly tests that the compilation semantics for the words "[:" and
";]" can be correctly performed programmatically.

This test fails in Gforth, VfxForth, minForth.

The problem is that the system enters compilation state after executing
"([:)". But it shall not. This action cannot be a part of compilation
semantics at all (it's not possible to correctly specify this action,
and it is not actually specified for the "[:" compilation semantics).

The reason of the problem is that "postpone" is implemented not quite
correctly in these systems. After including a polyfill for "postpone"
[2], the tests t12 and t13 are passed in Gforth.

VfxForth (5.20 Alpha 1 [build 4065]) and minForth (V3.4.8) provide
neither "find-name" nor correct "find", so "postpone" cannot be
correctly defined by the polyfill.


[2] About POSTPONE semantics in edge cases / Solution
<https://github.com/ForthHub/discussion/discussions/103#solution>


--
Ruvim

[Anton Ertl]

Post by Ruvim

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;" (at least for the resulting stack
effect), but the system should correctly work regardless whether the
current definition exists (i.e., a definition that is being compiled).
It means, if a definition is being compiled, its compilation shall be
correctly suspended by "[:", and resumed after ";]" under the hood.
Bellow are the test cases for the standardized compilation semantics,
and for the expected interpretation semantics.
Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.

[...]

Post by Ruvim
===== start of "quotation.test.fth"
\ Test cases for quotations "[: ... ;]"
\ Testing the compilation semantics
\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t
: lit, postpone literal ;
: ([:) postpone [: ;
: (;]) postpone ;] ;
\ t12
t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t > \ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t

t12 roughly tests that the compilation semantics for the words "[:" and
";]" can be correctly performed programmatically.
This test fails in Gforth, VfxForth, minForth.

Is there any Forth system where this test succeeds? I.e., is this
supposed failure actually common practice?

Post by Ruvim
The problem is that the system enters compilation state after executing
"([:)". But it shall not. This action cannot be a part of compilation
semantics at all (it's not possible to correctly specify this action,
and it is not actually specified for the "[:" compilation semantics).

Let's see. <http://www.forth200x.org/documents/forth19-1.pdf> says:

|Suspends compiling to the current definition, starts a new nested
|definition with execution token xt, and compilation continues with
|this nested definition.

The CfV on which it is based <http://www.forth200x.org/quotations.txt>
uses the same wording.

One could argue that "compilation continues" allows switching to
compilation state, so it's not clear that this test should succeed.
But anyway, it is better to improve the wording to make this switch
explicit.

And probably similarly for ";]". We would have to check whether all
systems switch to compilation state after the compilation semantics of
;], or whether some restore the state before "[:".

- anton

[Ruvim]

Post by Anton Ertl

Post by Ruvim

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;" (at least for the resulting stack
effect), but the system should correctly work regardless whether the
current definition exists (i.e., a definition that is being compiled).
It means, if a definition is being compiled, its compilation shall be
correctly suspended by "[:", and resumed after ";]" under the hood.
Bellow are the test cases for the standardized compilation semantics,
and for the expected interpretation semantics.
Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.

[...]

Post by Ruvim
===== start of "quotation.test.fth"
\ Test cases for quotations "[: ... ;]"
\ Testing the compilation semantics
\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t
: lit, postpone literal ;
: ([:) postpone [: ;
: (;]) postpone ;] ;
\ t12
t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t
\ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t

t12 roughly tests that the compilation semantics for the words "[:" and
";]" can be correctly performed programmatically.
This test fails in Gforth, VfxForth, minForth.

Is there any Forth system where this test succeeds? I.e., is this
supposed failure actually common practice?

For example, this test succeeds in SwiftForth. Also, as I mentioned, all
these tests succeed in my implementation (I'll publish it later when I
find time).

Anyway, failure of this test shows a defect in POSTPONE in an edge case.
Although it's possible to change the implementation for "[:" in these
systems to workaround this defect.

Post by Anton Ertl

Post by Ruvim
The problem is that the system enters compilation state after executing
"([:)". But it shall not. This action cannot be a part of compilation
semantics at all (it's not possible to correctly specify this action,
and it is not actually specified for the "[:" compilation semantics).

|Suspends compiling to the current definition, starts a new nested
|definition with execution token xt, and compilation continues with
|this nested definition.
The CfV on which it is based <http://www.forth200x.org/quotations.txt>
uses the same wording.
One could argue that "compilation continues" allows switching to
compilation state,

No.

1. State switching is always specified separately and explicitly.
2. If some behavior/effect is allowed and can be detected by a standard
program — it must be specified. So, if some behavior/effect is not
specified — it is not allowed.
3. The specified use case for "[:" supposes that the system is already
in compilation state (switching cannot be detected). Moreover,
compilation semantics always implies compilation state (of course, when
this fact can be detected by a standard program).

Post by Anton Ertl
so it's not clear that this test should succeed.

By my above arguments — it's clear ))

Post by Anton Ertl
But anyway, it is better to improve the wording to make this switch
explicit.

It shall not be specified, since in a standard system this fact must be
impossible to detect by a standard program.

Post by Anton Ertl
And probably similarly for ";]". We would have to check whether all
systems switch to compilation state after the compilation semantics of
;], or whether some restore the state before "[:".

Entering interpretation state after ";]" occurs only when a quotation is
used in interpretation state. But this use case is not covered by the
specification yet, so switching should not be specified.


To add this use case we should specify interpretation semantics for "[:".

To formally distinguish and specify this two use cases we can introduce
subtypes:

quotation-sys-comp => quotation-sys
quotation-sys-int => quotation-sys

And then:

"[:"
Compilation: ( C: –– quotation-sys-comp colon-sys )
Interpretation: ( C: –– quotation-sys-int colon-sys )

":]"
Compilation: ( C: quotation-sys colon-sys -- ) ( -- xt | )
If quotation-sys is quotation-sys-comp [ do some actions ]
If quotation-sys is quotation-sys-int [ do other actions, and enter
interpretation state ]


--
Ruvim

[Ruvim]

[...]

Post by Ruvim

And probably similarly for ";]".  We would have to check whether all
systems switch to compilation state after the compilation semantics of
;], or whether some restore the state before "[:".

Entering interpretation state after ";]" occurs only when a quotation is
used in interpretation state. But this use case is not covered by the
specification yet, so switching should not be specified.
To add this use case we should specify interpretation semantics for "[:".
To formally distinguish and specify this two use cases we can introduce
  quotation-sys-comp => quotation-sys
  quotation-sys-int  => quotation-sys
"[:"
Compilation: ( C: –– quotation-sys-comp colon-sys )
Interpretation: ( C: –– quotation-sys-int colon-sys )
":]"
Compilation: ( C: quotation-sys colon-sys -- ) ( -- xt | )
If quotation-sys is quotation-sys-comp [ do some actions ]
If quotation-sys is quotation-sys-int [ do other actions, and enter
interpretation state ]

An idea. A better way is to specify two sections of compilation semantic
description in the glossary entry, with different stack diagrams.
Similar to pattern matching. It allows to make description easier.


5.6.2.xxxx ":]"

Compilation: ( C: quotation-sys-comp colon-sys -- )
[ do some actions ]

Compilation: ( C: quotation-sys-int colon-sys -- ) ( S: -- xt )
[ do other actions, and enter interpretation state ]




For example, applying this approach to the "SET-ORDER" glossary entry:

16.6.1.2197 SET-ORDER

Execution: ( -1 -- )
Set the search order to the implementation-defined minimum search order.
The minimum search order shall include the words FORTH-WORDLIST and
SET-ORDER.

Execution: ( 0 -- )
Empty the search order.

Execution: ( widn ... wid1 +n -- )
Set the search order to the word lists identified by widn ... wid1.
Subsequently, word list wid1 will be searched first, and word list widn
searched last.


--
Ruvim

[a***@spenarnc.xs4all.nl]

Post by Anton Ertl

Post by Ruvim

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;" (at least for the resulting stack
effect), but the system should correctly work regardless whether the
current definition exists (i.e., a definition that is being compiled).
It means, if a definition is being compiled, its compilation shall be
correctly suspended by "[:", and resumed after ";]" under the hood.
Bellow are the test cases for the standardized compilation semantics,
and for the expected interpretation semantics.
Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.

[...]

Post by Ruvim
===== start of "quotation.test.fth"
\ Test cases for quotations "[: ... ;]"
\ Testing the compilation semantics
\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t
: lit, postpone literal ;
: ([:) postpone [: ;
: (;]) postpone ;] ;
\ t12
t{ :noname [ ([:) (;]) ] ; 0<> -> -1 }t > \ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t

t12 roughly tests that the compilation semantics for the words "[:" and
";]" can be correctly performed programmatically.
This test fails in Gforth, VfxForth, minForth.

Is there any Forth system where this test succeeds? I.e., is this
supposed failure actually common practice?

Post by Ruvim
The problem is that the system enters compilation state after executing
"([:)". But it shall not. This action cannot be a part of compilation
semantics at all (it's not possible to correctly specify this action,
and it is not actually specified for the "[:" compilation semantics).

|Suspends compiling to the current definition, starts a new nested
|definition with execution token xt, and compilation continues with
|this nested definition.
The CfV on which it is based <http://www.forth200x.org/quotations.txt>
uses the same wording.
One could argue that "compilation continues" allows switching to
compilation state, so it's not clear that this test should succeed.
But anyway, it is better to improve the wording to make this switch
explicit.
And probably similarly for ";]". We would have to check whether all
systems switch to compilation state after the compilation semantics of
;], or whether some restore the state before "[:".
- anton

I replace both [: ;] and :NONAME ; with { } that leaves an xt, but
} restores the STATE that was present with {.
Note that ;] and ; do the same, as it happens that ;] restores the
compilation state present with [: .
Also ; restore the interpretation state present with :NONAME
Now [ ] and { } can be nested width impunity.

: double [ VARIABLE A ] A ! A @ { DUP + } EXECUTE ;
2 double .
4 OK
I avoid the pathological cases in the above test by restricting
words that parse ahead.

I have reworked shani's lisp using these { } constructs, and
use it also for mal (Make Another Lisp) github project.
So count me a fan of simple quotations.

Groetjes Albert


{ } and [ ] can be constructed with the four nestings.

Let's assume our Forth has the following mechanism available:

({) (}) brackets a piece of code and leaves an execution token, not dea.
Can be used outside a definition instead of :NONAME
As lightweight as possible.
Inside: compilation mode.

(( )) make sure dictionary space can be used between them by compiling an
AHEAD, or equivalent.

([) (]) New context for definitions, maybe in the middle of a word.
- tucks away onto the return stack, any data that could be spoiled by
compiling something, then recovers it again
In particular w.r.t. current wordlist, remove the information
what word is being compiled, then restore it again.
Inside: interpretation mode.

(s s) save and restore STATE.

[Ruvim]

Post by Ruvim

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".

[...]

Post by Ruvim

Post by Ruvim
===== start of "quotation.test.fth"
\ Test cases for quotations "[: ... ;]"
\ Testing the compilation semantics
\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t
: lit, postpone literal ;
: ([:) postpone  [: ;
: (;]) postpone  ;] ;
\ t12
t{ :noname  [ ([:) (;]) ]  ;  0<>  ->  -1  }t  > \ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t

t12 roughly tests that the compilation semantics for the words "[:" and
";]" can be correctly performed programmatically.
This test fails in Gforth, VfxForth, minForth.
The problem is that the system enters compilation state after executing
"([:)". But it shall not. This action cannot be a part of compilation
semantics at all (it's not possible to correctly specify this action,
and it is not actually specified for the "[:" compilation semantics).
The reason of the problem is that "postpone" is implemented not quite
correctly in these systems. After including a polyfill for "postpone"
[2], the tests t12 and t13 are passed in Gforth.
VfxForth (5.20 Alpha 1 [build 4065]) and minForth (V3.4.8) provide
neither "find-name" nor correct "find", so "postpone" cannot be
correctly defined by the polyfill.

In minForth, "[:" leaves some data on the return stack, and ";]"
consumes this data from the return stack (contrary to the
specification). So, it fails in the following test case:

t{ : [t14] ( -- ) 123 >r ([:) r> lit, (;]) ; immediate -> }t
t{ :noname [t14] ; execute execute -> 123 }t


--
Ruvim

[minforth]

Post by Ruvim
In minForth, "[:" leaves some data on the return stack, and ";]"
consumes this data from the return stack (contrary to the
t{ : [t14] ( -- ) 123 >r ([:) r> lit, (;]) ; immediate -> }t
t{ :noname [t14] ; execute execute -> 123 }t

What specification are you talking about? Your own?

I can't find [t14] in the standard compliance tests.

[Ruvim]

Post by minforth

Post by Ruvim
In minForth, "[:" leaves some data on the return stack, and ";]"
consumes this data from the return stack (contrary to the
t{ : [t14] ( -- ) 123 >r ([:) r> lit, (;]) ; immediate -> }t
t{ :noname [t14] ; execute execute -> 123 }t

What specification are you talking about? Your own?

Of course, by default I mean the specification from the latest standard
[3,4]. Otherwise I would indicate which specification I mean.

If a word affects the return stack, it's indicated in its glossary
entry, and if it's not indicated — a word is not allowed to affect the
return stack.

Compare:

"[:" Compilation: ( C: –– quotation-sys colon-sys )

">R" Execution: ( x –– ) ( R: –– x )
(NB: Interpretation semantics for ">R" are undefined)

Post by minforth
I can't find [t14] in the standard compliance tests.

"[t14]" is just a name for an auxiliary word in my test.

Anyway, it's a standard compliant program, which is called "test case"
just because its purpose to show presence or absence of a particular bug
in a Forth system. A standard Forth system providing all relevant words
shall pass this test.


[3] http://www.forth200x.org/quotations.txt
[4] http://www.forth200x.org/documents/forth19-1.pdf

--
Ruvim

[minforth]

Thanks for pointing out this bug in the standard specification for quotations.

Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

This is the most expected way because analogously

123 >r :noname r> lit, ;

is also not guaranteed to work in ISO Forth.

[Anton Ertl]

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

If you think so, make a proposal. In the rest of the standard, the
compilation semantics of control-flow words communicate through the
control-flow stack, which usually is the data stack. I see no reason
to make an exception for [: ... ;].

Post by minforth
This is the most expected way because analogously
123 >r :noname r> lit, ;
is also not guaranteed to work in ISO Forth.

However,

: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123

is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

- anton

[minforth]

Post by Anton Ertl

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

If you think so, make a proposal. In the rest of the standard, the
compilation semantics of control-flow words communicate through the
control-flow stack, which usually is the data stack. I see no reason
to make an exception for [: ... ;].

Some systems might use the return stack for local frames. But well, then
follows probably 'implementation-defined' or 'ambiguity' ...

But if it is better for clarification and portability, Ruvim is right and
some more test cases should be added to the reference tests.

Post by Anton Ertl

Post by minforth
This is the most expected way because analogously
123 >r :noname r> lit, ;
is also not guaranteed to work in ISO Forth.

However,
: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123
is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

There is no control flow here.

[Ruvim]

Post by minforth

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

If you think so, make a proposal.  In the rest of the standard, the
compilation semantics of control-flow words communicate through the
control-flow stack, which usually is the data stack.  I see no reason
to make an exception for [: ... ;].

Some systems might use the return stack for local frames.

Of course, it's allowed!

Post by minforth
But well, then follows probably 'implementation-defined' or 'ambiguity' ...
But if it is better for clarification and portability, Ruvim is right and
some more test cases should be added to the reference tests.

Post by minforth
This is the most expected way because analogously
 123 >r :noname r> lit, ;
is also not guaranteed to work in ISO Forth.

However,
: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123
is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

There is no control flow here.

My test case is similar to this, but tests "[:". It can be rewritten as
follows:

: foo ( -- xt1 )
:noname ( -- xt2 ) \ xt2 ( -- 123 )
123 >r postpone [: r> postpone literal postpone ;]
postpone ;
;

foo execute execute . \ it shall print "123"


--
Ruvim

[Anton Ertl]

Post by minforth

Post by Anton Ertl

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

If you think so, make a proposal. In the rest of the standard, the
compilation semantics of control-flow words communicate through the
control-flow stack, which usually is the data stack. I see no reason
to make an exception for [: ... ;].

Some systems might use the return stack for local frames. But well, then
follows probably 'implementation-defined' or 'ambiguity' ...

How locals are stored is indeed implementation-defined, but there are
limits. E.g., a standard system must not store them on the data
stack, while the return stack is explicitly mentioned ("he storage
resource may be the return stack"), and other restrictions on standard
programs that use locals also make it easy for standard systems to use
the return stack for locals.

In any case, note that this is about where the locals reside at
run-time. When a standard system compiles locals, it must leave the
locals stack alone. Look at
<https://forth-standard.org/standard/usage#subsection.3.1.5>: There
are system-compilation types on the control-flow stack (which may be
the data stack), and system-execution types on the return stack.

Post by minforth

Post by Anton Ertl
However,
: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123
is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

There is no control flow here.

I consider calling and returning to be control-flow. In any case,
:NONAME has the stack effect ( C: -- colon-sys ) ( S: -- xt ), i.e.,
it pushes a colon-sys on the control-flow stack.

- anton

[minforth]

Post by Anton Ertl
Look at
<https://forth-standard.org/standard/usage#subsection.3.1.5>: There
are system-compilation types on the control-flow stack (which may be
the data stack), and system-execution types on the return stack.

There in 3.2.3.3 Return Stack the standard stipulates that
"A system may use the return stack in an implementation-dependent manner ..
for storing run-time nesting information."

This is exactly the case in MinForth's definitions for nestable [: ;]
and why Ruvim's [T14] failed.

BTW many CATCH/THROW implementations use the return stack in a similar way,
i.e. they don't use an exception stack.

[Ruvim]

Post by minforth

Post by Anton Ertl
Look at
<https://forth-standard.org/standard/usage#subsection.3.1.5>: There
are system-compilation types on the control-flow stack (which may be
the data stack), and system-execution types on the return stack.

There in 3.2.3.3 Return Stack the standard stipulates that
"A system may use the return stack in an implementation-dependent manner ..
for storing run-time nesting information."
This is exactly the case in MinForth's definitions for nestable [: ;]
and why Ruvim's [T14] failed.

No. It's not a case of storing run-time nesting information. In my test
data is passed on the same level of nesting.

Currently, minForth behaves something like this:

"[:" Compilation: ( -- colon-sys ) ( R: -- quotation-sys )
";]" Compilation: ( colon-sys -- ) ( R: quotation-sys -- )

These words are not allowed to leave/consume something on the return stack.

Post by minforth
BTW many CATCH/THROW implementations use the return stack in a similar way,
i.e. they don't use an exception stack.

CATCH ( i*x xt -- j*x 0 | i*x n )

The state of the return stack after performing CATCH is the same as
before performing.


--
Ruvim

[minforth]

Post by Ruvim

Post by minforth
BTW many CATCH/THROW implementations use the return stack in a similar way,
i.e. they don't use an exception stack.

CATCH ( i*x xt -- j*x 0 | i*x n )
The state of the return stack after performing CATCH is the same as
before performing.

Performing CATCH ends with its inner EXECUTE. The part after EXECUTE is performed
by some THROW. Before that, nesting information is pushed onto the return stack:

VARIABLE HANDLER 0 HANDLER ! \ last exception handler

: CATCH ( xt -- exception# | 0 \ return addr on stack
SP@ >R ( xt ) \ save data stack pointer
HANDLER @ >R ( xt ) \ and previous handler
RP@ HANDLER ! ( xt ) \ set current handler
EXECUTE ( ) \ execute returns if no THROW
R> HANDLER ! ( ) \ restore previous handler
R> DROP ( ) \ discard saved stack ptr
0 ( 0 ) \ normal completion
;

[Ruvim]

Post by minforth

Post by Ruvim

Post by minforth
BTW many CATCH/THROW implementations use the return stack in a similar way,
i.e. they don't use an exception stack.

| 9.3.3 Exception stack
| A stack used for the nesting of exception frames
| by CATCH and THROW. It may be, but need not be,
| implemented using the return stack.

Formally they do use the exception stack, which is usually located on
the return stack.

Post by minforth

Post by Ruvim
CATCH ( i*x xt -- j*x 0 | i*x n )
The state of the return stack after performing CATCH is the same as
before performing.

Performing CATCH ends with its inner EXECUTE.

I don't understand you objection.
Please, don't appeal to internal details. Show me a test case that shows
us the different state of the return stack before execution of CATCH and
after this execution. You can also use "['] CATCH EXECUTE".

Post by minforth
The part after EXECUTE is performed by some THROW.

1. If no exceptions occur, the part after "EXECUTE" is performed in a
normal way.
2. It does not matter how the part after "EXECUTE" is performed. A
standard program cannot detect this and cannot depend on this.

Post by minforth
Before that, nesting information is pushed onto the
VARIABLE HANDLER 0 HANDLER ! \ last exception handler
: CATCH ( xt -- exception# | 0 \ return addr on stack
  EXECUTE               ( )          \ execute returns if no THROW
  R> HANDLER !          ( )          \ restore previous handler
  R> DROP               ( )          \ discard saved stack ptr
   0                 ( 0 )        \ normal completion
;

--
Ruvim

[minforth]

This is my last reply to this fruitless language lawywering:

The function to be executed within CATCH receives nesting information
via the return stack. CATCH passes control over to that function, which
does with that information on the return stack what has to be done:
use it in an exception case or jump back for cleanup.

The point is, the standard is rather fuzzy in such points. The IMO
overzealous interpretation that return stack usage is forbidden unless
it is explicitly allowed, thereby ignoring 3.2.3.3 first paragraph, and
building test cases on top of this overzealous interpretation, is moot.

--
Now back to my new 3-wheeler motorscooter for a happy sunny ride! :-)

[a***@spenarnc.xs4all.nl]

Post by Anton Ertl

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

If you think so, make a proposal. In the rest of the standard, the
compilation semantics of control-flow words communicate through the
control-flow stack, which usually is the data stack. I see no reason
to make an exception for [: ... ;].

Post by minforth
This is the most expected way because analogously
123 >r :noname r> lit, ;
is also not guaranteed to work in ISO Forth.

However,
: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123
is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

I thought that
"
123 >r : noname r> postpone literal postpone ;
noname . \ prints 123
"
is not guaranteed to work in Forth-94 (ISO Forth).
Why is this different?

Post by Anton Ertl
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html

Groetjes Albert

[Anton Ertl]

Post by Anton Ertl

Post by Anton Ertl
However,
: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123
is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

I thought that
"
123 >r : noname r> postpone literal postpone ;
noname . \ prints 123
"
is not guaranteed to work in Forth-94 (ISO Forth).
Why is this different?

The latter is non-standard because interpretation semantics for >R are
undefined. Also, the definition of NONAME is not finished, and if it
was finished, then calling NONAME results in NONAME taking a value
from the return stack that it did not put there.

Why are you asking that question? You should know that.

- anton

[a***@spenarnc.xs4all.nl]

Post by Anton Ertl

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

If you think so, make a proposal. In the rest of the standard, the
compilation semantics of control-flow words communicate through the
control-flow stack, which usually is the data stack. I see no reason
to make an exception for [: ... ;].

Post by minforth
This is the most expected way because analogously
123 >r :noname r> lit, ;
is also not guaranteed to work in ISO Forth.

However,
: foo 123 >r :noname r> postpone literal postpone ; ;
foo execute . \ prints 123
is guaranteed to work in Forth-94 (ISO Forth) and Forth-2012.

I thought that
"
123 >r : noname r> postpone literal ;
noname . \ prints 123
"
is not guaranteed to work in Forth-94 (ISO Forth).
Why is this different?
This is assuming >R works in interpret mode, such as gforth.

Ignore the previous answer, I forget to delete a postpone.

Post by Anton Ertl
- anton

Groetjes Albert

[Ruvim]

Post by minforth
Thanks for pointing out this bug in the standard specification for quotations.

It's not a bug at all — that the word "[:" (the compilation semantics
of) is not allowed to change the return stack. It's an completely
consistent limitation.

Post by minforth
Although I find it obvious and not worth the hassle, the specification
should be amended so that "a program shall not use the return stack to
pass data between outer and inner functions, i.e. from an enclosing word
to its inner quotation(s) of any nesting depth, or from a quotation to
its outer enclosing word regardless of its nesting depth".

1. In my test case data is not passed between outer and inner functions.

2. It's already specified in "3.2.3.3 Return stack":

| A program shall not access values on the return stack
| (using R@, R>, 2R@, 2R> or NR>) that it did not place
| there using >R, 2>R or N>R;

| All values placed on the return stack within a definition
| shall be removed before the definition is terminated or
| before EXIT is executed.

Also, due to the Initiation semantics: ( R: -- nest-sys )
for colon definitions, the items placed on the return stack in a caller
cannot be access in the callee.

A real bug (an omission) is that this initiation semantics is missed in
the glossary entry for "[:".

Post by minforth
This is the most expected way because analogously
 123 >r :noname r> lit, ;

Probably you mean:

123 >r :noname [ r> lit, ] ;

Post by minforth
is also not guaranteed to work in ISO Forth.

I specially noted that interpretation semantics are not defined for
">r". And you give an example based on the interpretation semantics for
">r". Of course, it's not guaranteed.

Within a definition using r-stack is guaranteed to work, as Anton also
pointed out.


--
Ruvim

[minforth]

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are ...

Expected by whom?

I'd rather prefer some error message on a stray [: or ;]
(principle of least surprise)

[Ruvim]

Post by minforth

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are ...

Expected by whom?

By me and many other people.

Some Forth systems are actually implement these interpretation semantics
(some of them for the case when the current definition is absent only).

Post by minforth
I'd rather prefer some error message on a stray [: or ;] (principle of
least surprise)

According to the principle of least surprising, interpreting a
definition name should be equivalent to interpreting of the definition body.

And if they are not equivalent — it should be for some clear and
convincing reason. For example, `exit` can be used only in a definition
body.


Let's take the definition:
: foo [: 123 . ;] execute ;

There is no a convincing reason why the following lines should not be
observationally equivalent:
foo
[: 123 . ;] execute



--
Ruvim

[minforth]

Post by Ruvim
: foo [: 123 . ;] execute ;
There is no a convincing reason why the following lines should not be
foo
[: 123 . ;] execute

Then this should also be equivalent?

: BLA ." 1 " ;

BLA
" 1 "

[Ruvim]

Post by minforth

   : foo [: 123 . ;] execute ;
There is no a convincing reason why the following lines should not be
   foo
   [: 123 . ;] execute

Then this should also be equivalent?
: BLA ." 1 " ;
BLA
" 1 "

Yes, `BLA` and `." 1 "`.


Yes,
BLA
and
." 1 "


It's the most expected interpretation semantics for the word `."`

Forth-2012 allows such semantics, but a portable program cannot depend
on that.


--
Ruvim

[minforth]

Post by Ruvim

Post by minforth

   : foo [: 123 . ;] execute ;
There is no a convincing reason why the following lines should not be
   foo
   [: 123 . ;] execute

Then this should also be equivalent?
: BLA ." 1 " ;
BLA
" 1 "

Yes, `BLA` and `." 1 "`.
Yes,
BLA
and
." 1 "

They are not, they do different things: BLA does not parse the input stream.

Simmilarly: [: replacing :noname is not a quotation, it is just some kind
of alias.

Anyhow IMO quotations are crippled as long as they can't access upvalues.
E.g.
: TTT { a } [: a . ;] dup execute 2 to a execute ;
1 TTT -> 1 2

But then interpreting
[: a . ;]
makes no sense.

[Ruvim]

Post by minforth

Post by minforth

   : foo [: 123 . ;] execute ;
There is no a convincing reason why the following lines should not
   foo
   [: 123 . ;] execute

Then this should also be equivalent?
: BLA ." 1 " ;
BLA
" 1 "

Yes,
   BLA
and
   ." 1 "

They are not, they do different things: BLA does not parse the input stream.

It does not matter, because the effect of the whole phrase is the same.

Post by minforth
Simmilarly: [: replacing :noname is not a quotation, it is just some kind
of alias.

It depends on terminology. Let's assume:
*quotation*: source code for a nameless Forth definition in a special
form that is allowed to be placed inside another definition body as well
as outside of any definition body.

The run-time semantics of a quotation are to place the execution token
of the corresponding Forth definition on the stack. If the start of a
quotation is encountered by the Forth text interpreter in compilation
state, then the run-time semantics of this quotation are appended to the
current definition, otherwise these run-time semantics are performed
when the quotation end is encountered.


So, "[: ... ;]" — is a particular form for quotations. NB: another form
can be introduced via synonyms.

"[: 1 2 3 ;]" — is a quotation, regardless whether it's placed inside
another definition body, or outside.

":noname 1 2 3 ;" is not a quotation, as far as this form cannot be used
inside another definition body.

Post by minforth
Anyhow IMO quotations are crippled as long as they can't access upvalues.
E.g.
: TTT  { a } [: a . ;] dup execute 2 to a execute ;
1 TTT -> 1 2

If they could access local variables of a containing definition — they
should be called "closures", not "quotations".

And since they cannot — they are called differently (i.e. "quotations").

Post by minforth
But then interpreting
[: a . ;]
makes no sense.

If you replace a definition's name by its body, you have to use the
whole body (in this case, including the local variables declaration).
But local variables can only be used inside a definition. And this is a
clear and convincing reason for non-equivalence.



--
Ruvim

[minforth]

Post by Ruvim

Post by minforth
Anyhow IMO quotations are crippled as long as they can't access upvalues.
E.g.
: TTT  { a } [: a . ;] dup execute 2 to a execute ;
1 TTT -> 1 2

If they could access local variables of a containing definition — they
should be called "closures", not "quotations".

Closures are a completely different matter. Closures own their own individual
copy/reference of their lexical environment at the time of their creation.
TTT does not have this copy.

[Ruvim]

Post by minforth

Post by Ruvim

Post by minforth
Anyhow IMO quotations are crippled as long as they can't access upvalues.
E.g.
: TTT  { a } [: a . ;] dup execute 2 to a execute ;
1 TTT -> 1 2

If they could access local variables of a containing definition — they
should be called "closures", not "quotations".

Closures are a completely different matter. Closures own their own
individual copy/reference of their lexical environment at the time
of their creation.
TTT does not have this copy.

Do you mean that the following will not work in your model?

: TTT2 { a -- xt } [: a dup 1+ to a . ;] dup execute dup execute ;
1 TTT2 execute -> 1 2 3

If this does not work in full, this access to outer locals is confusing
and useless.


--
Ruvim

[minforth]

My quotation "model" with access to upvalues works here.
Useful now and then, but they cannot pass the man-or-boy test.

As I said, pure closures are a different thing altogether.
Read it up, if you are interested.

IIRC a while ago Anton Ertl and/or Bernd Paysan have implemented
closures for gforth:

: A {: w^ k x1 x2 x3 xt: x4 xt: x5 | w^ B :} recursive
k @ 0<= IF x4 x5 f+ ELSE
B k x1 x2 x3 action-of x4 [{: B k x1 x2 x3 x4 :}L
-1 k +!
k @ B @ x1 x2 x3 x4 A ;] dup B !
execute THEN ;

10 [: 1e ;] [: -1e ;] 2dup swap [: 0e ;] A f.

They introduced a special closure syntax, so quotations with
access to locals of the enclosing function must not be confused
with closures.

Additionally for your pleasure they also introduced interpreted [: ;] ;-)

[Ruvim]

Post by minforth
My quotation "model" with access to upvalues works here.
Useful now and then, but they cannot pass the man-or-boy test.

I still cannot see how your model works in edge cases. If you have an
implementation, could you make it available for testing?

Post by minforth
As I said, pure closures are a different thing altogether.
Read it up, if you are interested.

Thank you, I'm very familiar with the concept of closures and it's
implement options ))

Post by minforth
IIRC a while ago Anton Ertl and/or Bernd Paysan have implemented
: A {: w^ k x1 x2 x3 xt: x4 xt: x5 | w^ B :} recursive
   B k x1 x2 x3 action-of x4 [{: B k x1 x2 x3 x4 :}L
     -1 k +!
     execute  THEN ;
10 [: 1e ;] [: -1e ;] 2dup swap [: 0e ;] A f.
They introduced a special closure syntax, so quotations with
access to locals of the enclosing function must not be confused
with closures.

I disagree to call this mechanism "closures". Since it cannot lexically
capture the local variables of the containing (surrounding) definition,
and it cannot create an own persistent instance of mutable environment.

Let's define the following words:

synonym e execute
: partial1 ( x xt1 -- xt2 )
2>r :noname r> r> lit, compile, postpone ;
;

Now the following lines are observationally equivalent:

:noname 2 [{: x :}d x . x 1+ to x x . ;] ; e dup e e
\ output "2 3 2 3"

:noname 2 [: {: x :} x . x 1+ to x x . ;] partial1 ; e dup e e
\ output "2 3 2 3"


So, what is in Gforth is a kind of partial application that is joint
with local variables and some means to control the live time of a
dynamically created definition.

Post by minforth
Additionally for your pleasure they also introduced interpreted [: ;] ;-)

Yes, I know. If you missed it, I pointed out a bug in Gforth, VfxForth
and minForth in POSTPONE when it's applied to "[:".

Gforth and VfxForth succeed in the tests t11, t21 and t22 (but not in
t12, t13, and t3*) from my first message in this thread. minForth — only
in t11. SwiftForth — in t1* tests.


--
Ruvim

[Anton Ertl]

Post by minforth
My quotation "model" with access to upvalues works here.
Useful now and then, but they cannot pass the man-or-boy test.

Why not?

What use did you find for them?

I tried to find uses for the Gforth closures for the paper that could
not be replaced by code without this feature, but failed to find a
concise example where the benefit was convincing (as in: much easier
than the alternative).

So I asked Niklaus Wirth, who has implemented this feature in Pascal,
Modula, Modula-2, Oberon, and Oberon-2; I thought that, with keeping
this feature for so many decades, he must have a good reason. At
first I had trouble reaching him, but he came to Vienna in January
2020, and I asked him in person. He told me that he had removed this
feature from Oberon-07 (based on Oberon) in 2013, and later also
answered in writing.

Anyway, in the meantime we have found a good use for closures: Gforth
implements a variant of the actor model
<https://gforth.org/manual/Message-queues.html>. The messages are
actually xts that are executed by the target task.

Originally, in order to parameterize the passed xts, there was also a
mechanism for passing values that would be pushed on the stack of the
target task, and finally one would pass a word that would consume
these stack items.

Thanks to closures the protocol could be simplified such that only xts
are passed. If you want to parameterize the message, you pass a
closure that includes the parameters. For extra convenience, there
are one-shot heap-allocated closures (defined with :}h1) that FREE
themselves when they run.

Post by minforth
IIRC a while ago Anton Ertl and/or Bernd Paysan have implemented

...

Post by minforth
They introduced a special closure syntax, so quotations with
access to locals of the enclosing function must not be confused
with closures.

For Gforth's closures, just say "Gforth closures". There are lots of
other equally valid uses of the word "closure"; Gforth closures are
based on the concept of flat closures [dybvig87,keep+12]. Gforth
closures leave flat-closure conversion to the programmer, and that
turns out to often be more convenient to program than defining outer
locals and capturing them implicitly.

@PhdThesis{dybvig87,
author = "R. Kent Dybvig",
school = "University of North Carolina at Chapel Hill",
title = "Three Implementation Models for Scheme",
year = "1987",
url = "http://agl.cs.unm.edu/~williams/cs491/three-imp.pdf",
brokenURL = "ftp://ftp.cs.indiana.edu/pub/scheme-repository/txt/3imp.ps.Z",
month = apr,
OPTannote = "Introduces flat closures under the name \emph{display
closures} in Section 4.4.2"
}

@InProceedings{keep+12,
title = "Optimizing Closures in {O}(0) Time",
author = "Andrew W. Keep and Alex Hearn and R. Kent Dybvig",
bibdate = "2015-05-09",
bibsource = "DBLP,
http://dblp.uni-trier.de/db/conf/icfp/scheme2012.html#KeepHD12",
booktitle = "Proceedings of the 2012 Annual Workshop on Scheme and
Functional Programming, Scheme 2012, Copenhagen,
Denmark, September 9-15, 2012",
publisher = "ACM",
year = "2012",
xbooktitle = "***@ICFP",
editor = "Olivier Danvy",
ISBN = "978-1-4503-1895-2",
pages = "30--35",
URL = "http://doi.acm.org/10.1145/2661103",
urlwithoutbibliography = "https://www.cs.indiana.edu/~dyb/pubs/closureopt.pdf",
OPTannote = "Describes a number of optimizations for a
flat-closure implementation of Scheme."
}

- anton

[minforth]

Post by Anton Ertl

Post by minforth
My quotation "model" with access to upvalues works here.
Useful now and then, but they cannot pass the man-or-boy test.

Why not?

It falls into the funarg problem category because I use a locals
stack. Reading/writing locals of the enclosing function from within
an enclosed quotation is as easy as reaching deeper down the stack.

So there is no recursion-proof activation record per xt as was
needed for proper closures.

Post by Anton Ertl
What use did you find for them?

Calling embedded functions (quotations) without having to pass parameters
or having to use glocbal variables can result in concise readable notation.

Here it is sometimes handy to "not" pass longish matrices around, just fetch
them from the caller's array locals one level up.

OTOH I have absolutely no uses case for closures. Also my gut feeling
tells me they don't fulfill the KISS principle.

[a***@spenarnc.xs4all.nl]

Post by Anton Ertl
So I asked Niklaus Wirth, who has implemented this feature in Pascal,
Modula, Modula-2, Oberon, and Oberon-2; I thought that, with keeping
this feature for so many decades, he must have a good reason. At
first I had trouble reaching him, but he came to Vienna in January
2020, and I asked him in person. He told me that he had removed this
feature from Oberon-07 (based on Oberon) in 2013, and later also
answered in writing.

I find this extremely interesting. The Pascal specification is
clear. I'd not thought that - for me - an obscure feature as
closures is present in Pascal, much less that you could remove
this feature.

I always thought that closures are invented by lispers, because
in lisp you cannot write normal programs.

Post by Anton Ertl
- anton

[Paul Rubin]

Post by a***@spenarnc.xs4all.nl
I always thought that closures are invented by lispers, because
in lisp you cannot write normal programs.

Lisp didn't get them until fairly late in its evolution, I think. Maybe
old time Lispers here would know. But I think Scheme was the first
dialect that really made use of them. Algol 60 its own version much
earlier, in the form of call-by-name parameters.

[Anton Ertl]

Post by Paul Rubin

Post by a***@spenarnc.xs4all.nl
I always thought that closures are invented by lispers, because
in lisp you cannot write normal programs.

Lisp didn't get them until fairly late in its evolution, I think. Maybe
old time Lispers here would know. But I think Scheme was the first
dialect that really made use of them.

Yes, Scheme was the first implemented Lisp with lexical scoping and it
allows treating closures as first-class values. However, static
scoping was the original intention of McCarthy when he designed Lisp,
and he considered the dynamically scoped implementation to be a bug.
However, Hyrum's law struck, and mainstream Lisp kept dynamic scoping,
with static scoping being added in Common Lisp.

Post by Paul Rubin
Algol 60 its own version much
earlier, in the form of call-by-name parameters.

That, too, but Algol 60 also has nested functions/procedures with
static scoping. Algol 60 does not have first-class closures, though,
and some people reservere the name "closure" to first-class closures.

- anton

[a***@spenarnc.xs4all.nl]

Post by Paul Rubin

Post by a***@spenarnc.xs4all.nl
I always thought that closures are invented by lispers, because
in lisp you cannot write normal programs.

Lisp didn't get them until fairly late in its evolution, I think. Maybe
old time Lispers here would know. But I think Scheme was the first
dialect that really made use of them. Algol 60 its own version much
earlier, in the form of call-by-name parameters.

Jenkin's device, call-by-name parameters. That was an obscure feature
got in algol60 their by accident.
That was a form of closure? Never had thought that.
Algol68 replaced it by allowing reference variables, kind of type clean
pointers, and passing references as parameters. That was understandable.

Groetjes Albert

[Paul Rubin]

Post by a***@spenarnc.xs4all.nl
Jenkin's device, call-by-name parameters. That was an obscure feature
got in algol60 their by accident.

I think you mean

https://en.wikipedia.org/wiki/Jensen's_device

which I was unfamiliar with. But I thought call-by-name was not at all
accidental.

Post by a***@spenarnc.xs4all.nl
That was a form of closure?

I would say that it is, but in Algol-60 I guess it can be stack
allocated, unlike Scheme closures which have to be on the heap. You
could imagine an Algol-60 extension that allowed passing procedures as
values. That in turn might require heap allocating closures.

[Anton Ertl]

Post by Paul Rubin
But I thought call-by-name was not at all
accidental.

What I read about Algol 60 (and I read several articles, so I cannot
point to where I read the following; the first thing I would look at
is the HOPL paper) says that they just wanted an elegant specification
that (I think) supports in-out semantics. What they wrote down was
call-by-name, but they were not aware of all the consequences when
they wrote it. This only was recognized later, and was finally widely
publicized with Jensen's device.

The claim that it was unintended is supported by the fact that neither
Algol W nor Algol 68 (two direct successors of Algol 60) have call by
name.

[Is a call-by-name parameter a closure]

Post by Paul Rubin
I would say that it is,

It certainly accesses variables visible in the caller, like more
typical closures. If people want to emulate closures, they usually
pass something like an xt and execute it in the callee, just like more
typical closures.

Post by Paul Rubin
but in Algol-60 I guess it can be stack
allocated, unlike Scheme closures which have to be on the heap.

Sophisticated Scheme compilers can determine when they can reside on
the stack.

- anton

[Paul Rubin]

Post by Anton Ertl
is the HOPL paper) says that they just wanted an elegant specification
that (I think) supports in-out semantics. What they wrote down was
call-by-name, but they were not aware of all the consequences when
they wrote it.

I don't remember Algol syntax but I had thought using call-by-name as a
cheap inline function was idiomatic in it. E.g. to add up the first n
squares, you could say

a = sum(i, 1, n, i*i)

Post by Anton Ertl

but in Algol-60 I guess it can be stack allocated, unlike Scheme
closures which have to be on the heap.

Sophisticated Scheme compilers can determine when they can reside on
the stack.

Sure, but Algol-60 didn't create the possibility of having to heap
allocate anything. So it avoided needing GC, which would have been a
big minus in that era. Lisp existed then but idk if it was actually
used for anything outside of research.

[Anton Ertl]

Post by Paul Rubin

Post by Anton Ertl
is the HOPL paper) says that they just wanted an elegant specification
that (I think) supports in-out semantics. What they wrote down was
call-by-name, but they were not aware of all the consequences when
they wrote it.

I don't remember Algol syntax but I had thought using call-by-name as a
cheap inline function was idiomatic in it. E.g. to add up the first n
squares, you could say
a = sum(i, 1, n, i*i)

It may have become idiomatic after Jensen's device became well-known.
That does not mean that it was intended.

However, I don't think that it became idiomatic, because if it had
become idiomatic, the successor languages of Algol 60 would have
supported call by name, maybe as default, or maybe as a special option
for passing parameters (syntactically similar to the VAR parameters in
Pascal). None of that happened.

If you want to see what happens if something becomes idiomatic, look
at Lisp: The intention for the language was lexical scoping, but the
implementation used dynamic scoping. By the time this was recognized
as a bug, enough programs had been written that relied on dynamic
scoping and enough programmers had become accustomed to this behaviour
that they could not just fix it, but instead used a workaround (the
FUNARG device) when they wanted to have lexical-scoping semantics.
Eventually Common Lisp (started 1981, released 1984) added a separate
syntax for lexical scoping to mainstream Lisp, but that was more than
two decades after dynamically scoped Lisp had been implemented and
become idiomatic.

Another case is the story of S-expressions vs. (Algol- or ML-like)
M-expressions in Lisp.

Post by Paul Rubin
Sure, but Algol-60 didn't create the possibility of having to heap
allocate anything. So it avoided needing GC, which would have been a
big minus in that era. Lisp existed then but idk if it was actually
used for anything outside of research.

And yet, Lisp had so much existing code by the time the scoping
implementation was discovered as being buggy that they could not fix
it. Algol-60 has been described as a publication language, so maybe
there was actually more running Lisp code around than Algol-60 code.
Sure, Burroughs used Algol-60 for their large systems, but they and
their customers did not like Jensen's device themselves, or they did
not participate in the development of other programming languages that
received any scrutiny in language design discussions. In any case,
call-by-name does not appear in any later languages that I have ever
heard of.

- anton

[Paul Rubin]

Post by Anton Ertl
However, I don't think that it became idiomatic, because if it had
become idiomatic, the successor languages of Algol 60 would have
supported call by name,

I don't see this implication. It could be idiomatic and simultaneously
been considered a bad idea. Maybe there is an Algol textbook online
that could say.

Post by Anton Ertl
The intention for [Lisp] was lexical scoping, but the implementation
used dynamic scoping. ... Eventually Common Lisp (started 1981,
released 1984) added a separate syntax for lexical scoping to
mainstream Lisp, but that was more than two decades after dynamically
scoped Lisp had been implemented and become idiomatic.

Scheme had lexical scope in the late 1970s and I believe it appeared in
some Lisps earlier than Common Lisp, but that was before my time. I
might ask on the Lisp group. There is also the matter that dynamic
scope is very easy to implement, so that might have affected what people
did.

Post by Anton Ertl
Another case is the story of S-expressions vs. (Algol- or ML-like)
M-expressions in Lisp.

M-expressions never caught on because Lispers liked S-expressions.

Post by Anton Ertl
And yet, Lisp had so much existing code by the time the scoping
implementation was discovered as being buggy that they could not fix
it.

I don't know about this, there are some implementation techniques
(naming conventions) you can use to (somehwat) prevent dynamic scope
from going awry. GNU Emacs uses that approach extensively since it
exclusively used dynamic scope for a long time (it has lexical scope
now).

Post by Anton Ertl
In any case, call-by-name does not appear in any later languages that
I have ever heard of.

https://www.geeksforgeeks.org/scala-functions-call-by-name/

[Anton Ertl]

Post by Paul Rubin

Post by Anton Ertl
However, I don't think that it became idiomatic, because if it had
become idiomatic, the successor languages of Algol 60 would have
supported call by name,

I don't see this implication. It could be idiomatic and simultaneously
been considered a bad idea.

Programmers tend to love their idioms, even if others consider them to
be a bad idea. And if programmers had loved call-by-name, we would
have seen more programming languages that support it.

What I see in the Algol 60, Algol W, Algol 68, and Pascal entries of
<https://rosettacode.org/wiki/Jensen's_Device> is a steady progression
away from call-by-name (and towards call-by-value):

In Algol 60, call-by-name is the default and call-by-value has to be
declared separately.

In Algol W, every parameter has a declared mode: i has mode %name%, lo
and hi have mode value, and term have mode procedure. The %name% mode
is probably the full-blown thing which also allows assignment to it,
while assigning to a procedure parameter is probably not allowed or
has a different semantics.

In Algol 68, i is passed by-REFerence, lo and hi are passed by-value
(the default in Algol 68), and term is passed by-PROCedure (probably
like the procedure mode of Algol 60).

In Pascal (successor of Algol W), i is passed as a VAR parameter
(i.e., by-reference), lo and hi are passed by-value (the default), and
term is passed as a function parameter. The function has to be
written separately, and in the call only its name is given. The
rosettacode entry has a term function that gets i as a parameter, but
I think that it is also possible to write a parameterless function
that accesses i through lexical scoping. In either case, passing
something in this by-name emulation in Pascal is much more cumbersome
than in the three Algols, which shows that Wirth did not consider
Jensen's device to be particularly important. And given the success
of Pascal, this sentiment was shared by many.

Post by Paul Rubin

Post by Anton Ertl
The intention for [Lisp] was lexical scoping, but the implementation
used dynamic scoping. ... Eventually Common Lisp (started 1981,
released 1984) added a separate syntax for lexical scoping to
mainstream Lisp, but that was more than two decades after dynamically
scoped Lisp had been implemented and become idiomatic.

Scheme had lexical scope in the late 1970s and I believe it appeared in
some Lisps earlier than Common Lisp, but that was before my time.

Scheme was first in 1975, but it did not become the Lisp mainstream.
1975 was 15 years after Lisp was introduced in 1960 and probably
almost as many years after the difference between the intention and
the implementation was discovered.

Post by Paul Rubin
There is also the matter that dynamic
scope is very easy to implement, so that might have affected what people
did.

Common Lisp requires implementing both dynamic scoping (using the '
syntax) and static scoping (IIRC using the #' syntax). Dynamic
scoping does not make the implementation of Common Lisp easier. It's
there because programs were written to work with dynamic scoping. So
many programs that eliminating it and switching to Scheme was
impractical.

Post by Paul Rubin

Post by Anton Ertl
Another case is the story of S-expressions vs. (Algol- or ML-like)
M-expressions in Lisp.

M-expressions never caught on because Lispers liked S-expressions.

Exactly. The original idea was that S-expressions are just a stopgap
until M-expressions are implemented, but they found out that
programmers preferred S-expressions, so Lisp uses S-expressions to
this day. Compare this to what happened to call-by-value.

Post by Paul Rubin

Post by Anton Ertl
In any case, call-by-name does not appear in any later languages that
I have ever heard of.

https://www.geeksforgeeks.org/scala-functions-call-by-name/

Interesting. Looking at
<https://rosettacode.org/wiki/Jensen%27s_Device#Scala>, we see that
the call to the term is somewhat Algol-like, but the supposed
call-by-name is actually restricted like the procedure/PROC modes of
Algol W and Algol 68: It cannot be used for passing i, and therefore
some extra work was done to pass i in a way that is modifyable.

- anton

[Paul Rubin]

Post by Anton Ertl
What I see in the Algol 60, Algol W, Algol 68, and Pascal entries of
<https://rosettacode.org/wiki/Jensen's_Device> is a steady progression

It sounds like Algol 68 and Pascal (don't know about Algol W) had a way
to pass procedures as parameters. That was missing from Algol 60 so the
only way to get that effect was call by name. E.g. if you want to write
a root finder that finds a zero of some function, how do you pass the
function?

It's weird though. Function parameters were straightforward and
important in FORTRAN, which predated Algol 60, so I'd expect the Algol
60 designers to have known better. It does sound from Naur's report
that not all of the Algol 60 committee knew what it was getting into
with call by name.

Post by Anton Ertl
In Algol 68, i is passed by-REFerence, lo and hi are passed by-value
(the default in Algol 68), and term is passed by-PROCedure (probably
like the procedure mode of Algol 60).

Algol 60 had a procedure mode? Hmm, maybe I misunderstood Naur. I'll
try to look at Rosetta Code.

Post by Anton Ertl
Common Lisp requires implementing both dynamic scoping (using the '
syntax) and static scoping (IIRC using the #' syntax).

No ' just quotes a sexp and #' quotes a function. Scheme also has
dynamic scoping using fluid-let.

Post by Anton Ertl
Dynamic scoping does not make the implementation of Common Lisp
easier. It's there because programs were written to work with dynamic
scoping. So many programs that eliminating it and switching to Scheme
was impractical.

That might be, it was before my time. But I wrote a purely dynamically
scoped Lisp back in the day, and I studied the internals of Emacs Lisp,
and dynamic scope is very easy to implement using shallow binding.
Lexical scope is somewhat harder, though maybe not enough to matter in a
compiler.

Post by Anton Ertl
[Scala] the supposed call-by-name is actually restricted like the
procedure/PROC modes of Algol W and Algol 68: It cannot be used for
passing i, and therefore some extra work was done to pass i in a way
that is modifyable.

It's maybe similar in Haskell, whose lazy evaluation is supposed to be
semantically equivalent to call-by-name, but which is memoized, doable
because mutation is not allowed.

[Anton Ertl]

Post by Paul Rubin

Post by Anton Ertl
What I see in the Algol 60, Algol W, Algol 68, and Pascal entries of
<https://rosettacode.org/wiki/Jensen's_Device> is a steady progression

It sounds like Algol 68 and Pascal (don't know about Algol W) had a way
to pass procedures as parameters. That was missing from Algol 60 so the
only way to get that effect was call by name. E.g. if you want to write
a root finder that finds a zero of some function, how do you pass the
function?
It's weird though. Function parameters were straightforward and
important in FORTRAN, which predated Algol 60, so I'd expect the Algol
60 designers to have known better. It does sound from Naur's report
that not all of the Algol 60 committee knew what it was getting into
with call by name.

Post by Anton Ertl
In Algol 68, i is passed by-REFerence, lo and hi are passed by-value
(the default in Algol 68), and term is passed by-PROCedure (probably
like the procedure mode of Algol 60).

Algol 60 had a procedure mode?

No only by-name and by-value. I meant the procedure mode of Algol W.

Post by Paul Rubin

Post by Anton Ertl
[Scala] the supposed call-by-name is actually restricted like the
procedure/PROC modes of Algol W and Algol 68: It cannot be used for
passing i, and therefore some extra work was done to pass i in a way
that is modifyable.

It's maybe similar in Haskell, whose lazy evaluation is supposed to be
semantically equivalent to call-by-name, but which is memoized, doable
because mutation is not allowed.

The lazy evaluation of Haskell is certainly not equivalent to
call-by-name, because you cannot implement Jensen's device with it.
It's not even like the PROC/procedure mechanism, because it evaluates
the argument at most once, while a PROC/procedure parameter is
evaluated repeatedly.

<https://rosettacode.org/wiki/Jensen%27s_Device#Haskell> does
something with monads that I don't understand, but it's noticable that
sum has no parameter i and the code for the actual term parameter does
not reference i.

- anton

[Paul Rubin]

Post by Anton Ertl
The lazy evaluation of Haskell is certainly not equivalent to
call-by-name, because you cannot implement Jensen's device with it.

Jensen's device depends on having mutable variables. If they are not
allowed, Haskell's evaluation is equivalent to call-by-name.

Post by Anton Ertl
<https://rosettacode.org/wiki/Jensen%27s_Device#Haskell> does
something with monads that I don't understand

It creates a mutable memory cell (STRef) that is read and written as if
it's an i/o device (readSTRef/writeSTRef). Then it makes a closure that
reads from the cell and returns the reciprocal of the contents, and
sums over the result of that closure when it writes k=1,2...n into the
cell. Ugh!!!

[a***@spenarnc.xs4all.nl]

Post by Paul Rubin

Post by Anton Ertl
The lazy evaluation of Haskell is certainly not equivalent to
call-by-name, because you cannot implement Jensen's device with it.

Jensen's device depends on having mutable variables. If they are not
allowed, Haskell's evaluation is equivalent to call-by-name.

Post by Anton Ertl
<https://rosettacode.org/wiki/Jensen%27s_Device#Haskell> does
something with monads that I don't understand

It creates a mutable memory cell (STRef) that is read and written as if
it's an i/o device (readSTRef/writeSTRef). Then it makes a closure that
reads from the cell and returns the reciprocal of the contents, and
sums over the result of that closure when it writes k=1,2...n into the
cell. Ugh!!!

It serves in my opinion that Jensen's is serependitious technique that
is only used as there are no better techniques available.
There is no discretion at Rosetta, otherwise this example would thrown
out for being not appropriate for this forum to show techniques.

Groetjes Albert

[minforth]

Post by a***@spenarnc.xs4all.nl

Post by Paul Rubin

Post by Anton Ertl
The lazy evaluation of Haskell is certainly not equivalent to
call-by-name, because you cannot implement Jensen's device with it.

Jensen's device depends on having mutable variables. If they are not
allowed, Haskell's evaluation is equivalent to call-by-name.

Post by Anton Ertl
<https://rosettacode.org/wiki/Jensen%27s_Device#Haskell> does
something with monads that I don't understand

It creates a mutable memory cell (STRef) that is read and written as if
it's an i/o device (readSTRef/writeSTRef). Then it makes a closure that
reads from the cell and returns the reciprocal of the contents, and
sums over the result of that closure when it writes k=1,2...n into the
cell. Ugh!!!

It serves in my opinion that Jensen's is serependitious technique that
is only used as there are no better techniques available.
There is no discretion at Rosetta, otherwise this example would thrown
out for being not appropriate for this forum to show techniques.

My notion as well. If you look at closures eg in Javascript, it seems that
the main benefit of closures is the easy handling of private methods and
variables without opening a big barrel like OOP.

The other way round: if you already have OO in your language, you don't
need closures.

Jensen's device or Knuth's man-or-boy are just programming playgrounds.

[Anton Ertl]

Post by minforth
My notion as well. If you look at closures eg in Javascript, it seems that
the main benefit of closures is the easy handling of private methods and
variables without opening a big barrel like OOP.

The main benefit of closures is that you can pass data to a callback
that is not provided for by the interface of tha callback. OOP does
not provide this capability, so "opening a big barrel like OOP" would
not help.

As an example, take a look at one of the Jensen's device variants:

: sum ( i-xt lo hi term-xt -- r )
\ stack effects: i-xt ( -- addr ); term-xt ( -- r1 )
0e swap 1+ rot ?do ( r1 xt1 xt2 )
i 2 pick execute ! dup execute f+
loop 2drop ;

No data is passed by SUM to I-XT nor TERM-XT, so the data has to be
passed in some other way. In

variable i1 \ avoid conflict with Forth word I
' i1 1 100 :noname 1e i1 @ s>f f/ ; sum f.

a global variable I1 is used for passing the additional data. In

: main ( -- )
0 {: w^ i1 :} i1 [n:l ;] 1 100 i1 [n:l @ s>f 1/f ;] sum f. ;

a variable-flavoured local variable I1 is used, and in

i1 [n:l ;]

an xt (of a closure) is created that pushes I1 when executed.
Likewise

i1 [n:l @ s>f 1/f ;]

creates the xt of a closure that computes "i1 @ s>f 1/f" when
executed.

Post by minforth
The other way round: if you already have OO in your language, you don't
need closures.

You cannot do with objects what closures do in the example above.
Gforth has had objects (in several flavours) for decades, but we added
closures in 2018 because they add capabilities that objects do not
provide.

Post by minforth
Jensen's device or Knuth's man-or-boy are just programming playgrounds.

Sour grapes?

- anton

[minforth]

Post by Anton Ertl

Post by minforth
My notion as well. If you look at closures eg in Javascript, it seems that
the main benefit of closures is the easy handling of private methods and
variables without opening a big barrel like OOP.

The main benefit of closures is that you can pass data to a callback
that is not provided for by the interface of tha callback.

Yes, I should have mentioned callbacks too. ISTM only that closures are used
more often for encapsulation. But sure, this depends on the application domains.

Post by Anton Ertl
OOP does not provide this capability, so "opening a big barrel like OOP" would
not help.

Object methods can receive and return function pointers / execution tokens as
required for callbacks.

[Paul Rubin]

Post by a***@spenarnc.xs4all.nl
It serves in my opinion that Jensen's is serependitious technique that
is only used as there are no better techniques available.

It's pretty clear from Naur's article that Jensen's trick was
intentional rather than serependitious. One can say retrospectively
that it was a mistake because there were better things they could have
done instead. The committee was split on how to do it, and the wrong
side won. Despite this error, Algol 60 is still highly regarded, so we
can say for a language designed by a committee, it did a very good job
even if one can take issue with some specifics.

[Anton Ertl]

Post by Paul Rubin

Post by Anton Ertl
The lazy evaluation of Haskell is certainly not equivalent to
call-by-name, because you cannot implement Jensen's device with it.

Jensen's device depends on having mutable variables. If they are not
allowed, Haskell's evaluation is equivalent to call-by-name.

And if it's required to use every argument, then eager evaluation
(call-by-value) is equivalent to Haskell's lazy evaluation.

But Algol 60 has mutable variables, and, as you point out, Jensen's
device depends on that, and therefore Haskell's way of argument
passing is not equivalent to Algol 60's call-by-name.

Post by Paul Rubin

Post by Anton Ertl
<https://rosettacode.org/wiki/Jensen%27s_Device#Haskell> does
something with monads that I don't understand

It creates a mutable memory cell (STRef) that is read and written as if
it's an i/o device (readSTRef/writeSTRef). Then it makes a closure that
reads from the cell and returns the reciprocal of the contents, and
sums over the result of that closure when it writes k=1,2...n into the
cell. Ugh!!!

Yes, something in this direction is what I understood. What is
unclear to me is how this monad stuff interacts with the lazy
evaluation.

The way I have heard about Haskell programming up to now is that one
tries to have a pure functional part, and then use monads at the
fringes for things like I/O where pure functional code does not cut
it. I don't see this kind of separation in the Haskell code above. I
wonder if there is a more idiomatic way of writing this stuff in
Haskell.

- anton

[Paul Rubin]

Post by Anton Ertl
And if it's required to use every argument, then eager evaluation
(call-by-value) is equivalent to Haskell's lazy evaluation.

"required to use every argument" and "strict evaluation" are sort of the
same thing. The difference between strict and non-strict is what
happens if some of the args are not used, or only partially consumed.
With strict evaluation, passing 1/0 as an argument should raise an
error, and trying to create infinite list will run forever or exhaust
memory. With non-strict, both of those are ok, and it's only an error
if you try to actually (e.g.) print the value.

CBN and lazy evaluation are two different strategies for implementing
non-strict evaluation. If I'm using the jargon right, they have the
same denotational semantics, but different operational semantics.

Post by Anton Ertl
The way I have heard about Haskell programming up to now is that one
tries to have a pure functional part, and then use monads at the
fringes for things like I/O where pure functional code does not cut
it. I don't see this kind of separation in the Haskell code above.

I'll try to think of better explanation for this later, but basically
monads are just a notational trick, and I think that the Jensen's
example could have been done with the State monad instead of ST. State
is purely functional and works by threading a value through a chain of
function evaluations. ST (State Transformer) does sort of the same
thing as State, but it lets you use mutable memory cells, which can be
more efficient.

Post by Anton Ertl
I wonder if there is a more idiomatic way of writing this stuff in
Haskell.

I don't see one offhand, except maybe with macros or some silly thing
like that.

[Paul Rubin]

Post by Anton Ertl
It may have become idiomatic after Jensen's device became well-known.
That does not mean that it was intended.

Peter Naur's retrospective on Algol-60 history
<https://dl.acm.org/doi/10.1145/800025.1198353> doesn't discuss this
aspect in detail, but (page 38 of the pdf) Fritz Bauer commented:

A particularly sad experience is described with the events after the
conference~ had ended. Samelson and others had unsuccessfully tried
to have procedures as parameters included in ALGOL 60 in a
straightforward manner (This seemingly is not recorded by Peter
Naur, showing the arbitrariness of working along the written notes
only). Alan Perlis then with proposal 48 of Jan. 20 admitted that he
understood the "call by name" to be a hidden introduction of
procedure parameters (later to be known as Jensen's trick). The
remark "seemingly was only understood in a coherent manner by one
member" is therefore cynical and should be omitted. It was under
great pressure that the conference had ended and a majority had
voted down a minority in the controversial point 45. Straightforward
introduction of procedure parameters (and of the lambda calculus),
as it was advocated by the minority, would have outflanked some of
the ambiguities that made the Rome revision necessary.

Naur's article is interesting and shows there was more dissension in the
Algol-60 development process than I had realized.

The ACM journals of the 1960s-70s and maybe later were full of
algorithms written in Algol-60. I wonder if Jensen's device appeared
much in them. It caught my attention that some members were apparently
calling for explicit lambda calculus to be included, rather than
call-by-name.

[a***@spenarnc.xs4all.nl]

Post by Anton Ertl

Post by Paul Rubin

Post by Anton Ertl
is the HOPL paper) says that they just wanted an elegant specification
that (I think) supports in-out semantics. What they wrote down was
call-by-name, but they were not aware of all the consequences when
they wrote it.

I don't remember Algol syntax but I had thought using call-by-name as a
cheap inline function was idiomatic in it. E.g. to add up the first n
squares, you could say
a = sum(i, 1, n, i*i)

It may have become idiomatic after Jensen's device became well-known.
That does not mean that it was intended.
However, I don't think that it became idiomatic, because if it had
become idiomatic, the successor languages of Algol 60 would have
supported call by name, maybe as default, or maybe as a special option
for passing parameters (syntactically similar to the VAR parameters in
Pascal). None of that happened.
If you want to see what happens if something becomes idiomatic, look
at Lisp: The intention for the language was lexical scoping, but the
implementation used dynamic scoping. By the time this was recognized
as a bug, enough programs had been written that relied on dynamic
scoping and enough programmers had become accustomed to this behaviour
that they could not just fix it, but instead used a workaround (the
FUNARG device) when they wanted to have lexical-scoping semantics.
Eventually Common Lisp (started 1981, released 1984) added a separate
syntax for lexical scoping to mainstream Lisp, but that was more than
two decades after dynamically scoped Lisp had been implemented and
become idiomatic.
Another case is the story of S-expressions vs. (Algol- or ML-like)
M-expressions in Lisp.

Post by Paul Rubin
Sure, but Algol-60 didn't create the possibility of having to heap
allocate anything. So it avoided needing GC, which would have been a
big minus in that era. Lisp existed then but idk if it was actually
used for anything outside of research.

And yet, Lisp had so much existing code by the time the scoping
implementation was discovered as being buggy that they could not fix
it. Algol-60 has been described as a publication language, so maybe
there was actually more running Lisp code around than Algol-60 code.
Sure, Burroughs used Algol-60 for their large systems, but they and
their customers did not like Jensen's device themselves, or they did
not participate in the development of other programming languages that
received any scrutiny in language design discussions. In any case,
call-by-name does not appear in any later languages that I have ever
heard of.

In algol68 the unclean Jensen's device was replaced by references
once it was realized what it was. This permitted the same code,
without the mystification.

Post by Anton Ertl
- anton

[Anton Ertl]

Post by a***@spenarnc.xs4all.nl
In algol68 the unclean Jensen's device was replaced by references
once it was realized what it was. This permitted the same code,
without the mystification.

Whatever you may mean by "unclean" and "mystification", looking at
<https://rosettacode.org/wiki/Jensen%27s_Device#ALGOL_68> shows the
header of sum to be

PROC sum = (REF INT i, INT lo, hi, PROC REAL term)REAL:

which is then called with

sum (i, 1, 100, REAL: 1/i)

So i is passed by REFerence, lo and hi are passed by value (the
default in Algol 68, while call-by-name is the default in Algol 60),
and 1/i is passed as PROC (the implementation mechanism behind
call-by-name). Try using "REF" instead of "PROC", and see that the
program does not work as intended (if it compiles at all).

- anton

[a***@spenarnc.xs4all.nl]

Post by Anton Ertl

Post by a***@spenarnc.xs4all.nl
In algol68 the unclean Jensen's device was replaced by references
once it was realized what it was. This permitted the same code,
without the mystification.

Whatever you may mean by "unclean" and "mystification", looking at
<https://rosettacode.org/wiki/Jensen%27s_Device#ALGOL_68> shows the
header of sum to be
which is then called with
sum (i, 1, 100, REAL: 1/i)
So i is passed by REFerence, lo and hi are passed by value (the

Incorrect. `` REF INT i '' is passed by value.
Only values are passed to parameters.
That is the whole crux of algol 68.

Post by Anton Ertl
default in Algol 68, while call-by-name is the default in Algol 60),
and 1/i is passed as PROC (the implementation mechanism behind
call-by-name). Try using "REF" instead of "PROC", and see that the
program does not work as intended (if it compiles at all).
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2023: https://euro.theforth.net/2023

[minforth]

Post by a***@spenarnc.xs4all.nl
I find this extremely interesting. The Pascal specification is
clear. I'd not thought that - for me - an obscure feature as
closures is present in Pascal, much less that you could remove
this feature.

IIRC Pascal had nested functions from the beginning, unlike C
where they still don't exist. Closures are especially interesting
for functional programming. Since Forth is typeless and treats
execution tokens, addresses and numbers the same, functional
programming is perhaps not as interesting as in other languages.

[a***@spenarnc.xs4all.nl]

Post by minforth

Post by a***@spenarnc.xs4all.nl
I find this extremely interesting. The Pascal specification is
clear. I'd not thought that - for me - an obscure feature as
closures is present in Pascal, much less that you could remove
this feature.

IIRC Pascal had nested functions from the beginning, unlike C
where they still don't exist. Closures are especially interesting
for functional programming. Since Forth is typeless and treats
execution tokens, addresses and numbers the same, functional
programming is perhaps not as interesting as in other languages.

I find nested functions, and the ability to access the variables
of the encompassing function quite palatable. Has this something
to do with closures?
Anton Ertl talks about local variables from an expression that yields
an xt. Is that the same thing?

Contrasting Pascal with FORTRAN the main difference is namespaces,
i.e. "vocabularies". I feel that leaving out discussing namespaces,
in this thread is missing.

Groetjes Albert

[Anton Ertl]

Post by minforth
Since Forth is typeless

That's not the case. See
<https://forth-standard.org/standard/usage#section.3.1>.

Post by minforth
and treats
execution tokens, addresses and numbers the same, functional
programming is perhaps not as interesting as in other languages.

Why should that be the case?

Lisp has also been called typeless, and it's one of the oldest
functional programming languages.

- anton

[Ruvim]

Post by Ruvim

Post by minforth

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are ...

Expected by whom?

By me and many other people.
Some Forth systems are actually implement these interpretation semantics
(some of them for the case when the current definition is absent only).

Post by minforth
I'd rather prefer some error message on a stray [: or ;] (principle of
least surprise)

According to the principle of least surprising, interpreting a
definition name should be equivalent to interpreting of the definition body.
And if they are not equivalent — it should be for some clear and
convincing reason.  For example, `exit` can be used only in a definition
body.
  : foo [: 123 . ;] execute ;
There is no a convincing reason why the following lines should not be
  foo
  [: 123 . ;] execute

BTW, a fundamental property of a pure concatenative language is that in
the same lexical environment, *any* call to a definition can always be
replaced by the definition's body.

In Forth, it's not always possible, especially when a definition is used
interpretively (e.g., if it's name is encountered by the Forth text
interpreter in interpretation state) — due to some special words like
control-flow words, r-stack operations, parsing words, local variables,
etc, which impose corresponding restrictions on their use.

If it is not too difficult to implement a word or construct without such
restrictions, then it is worth implementing. Because it makes learning
and testing code much easier.

So, it's worth implementing interpretation semantics for such words as
`[']`, `[char]`, `c"`, `."`, `abort"`, `postpone`, "[: ... ;]", "if ...
else ... then", `>r`, `r>`, etc.

NB: R-stack operations shall affect a separate stack in their
interpretation semantics to avoid problems in edge cases in a
user-defined text interpreter [1].


[1] https://forth-standard.org/standard/tools/SYNONYM#reply-703

--
Ruvim

[Hans Bezemer]

Post by minforth

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are ...

Expected by whom?
I'd rather prefer some error message on a stray [: or ;] (principle of
least surprise)

In 4tH, you will. Because they are put along with a reference on the
control stack - and balanced. So:

: bla if bladibla ; then

Will trigger an error. BTW, some other Forths will too, but I'm not sure
if the mechanism is the same.

Hans Bezemer

[Stephen Pelc]

Post by Ruvim
The accepted proposal for quotations [1] specifies only compilation
semantics for the words "[:" and ";]".
The expected interpretation semantics for "[: ... ;]" are that this
construct behaves like ":noname ... ;"

...

Post by Ruvim
Interestingly, some Forth systems fail the t12 and t13 tests for the
standardized compilation semantics.

VFX implements [: and friends as NDCS words - words that have
Non-Default Compilation Semantics. Where the words behave correctly
and assumptions are then made about POSTPONE for non NDCS words,
MPE is not going assign priority to a fix.

In the time since we introduced NDCS words, there have been *no*
technical support issues about NDCS from application programmers.

Post by Ruvim
\ t11
t{ :noname [: 123 ;] ; execute execute -> 123 }t
: lit, postpone literal ;
: ([:) postpone [: ;
: (;]) postpone ;] ;
\ t12
t{ :noname [ ([:) (;]) ] ; 0<-> -1 }t
\ t13
t{ :noname 1 [ ([:) 2 lit, (;]) ] 3 ; execute swap execute -> 1 3 2 }t
\ Testing the interpretation semantics
\ (the expected behavior)
\ t21
t{ depth [: ;] depth 1- = ?dup nip -> 0 }t
\ t22
t{ [: 123 ;] execute -> 123 }t
\ t23
t{ [: 1 [: 2 ;] 3 ;] execute swap execute -> 1 3 2 }t
\ Testing the interpretation semantics
\ doing compilation of another definition
\ (the expected behavior)
\ t31
t{ [: [ depth [: ;] depth 1- = ?dup nip ] literal ;] execute -> 0 }t
\ t32
t{ [: 1 [ [: 2 ;] ] literal 3 ;] execute swap execute -> 1 3 2 }t
===== end of "quotation.test.fth"
[1] http://www.forth200x.org/quotations-v4.txt
--
Ruvim

Stephen