The IMMEDIATE mess

[J Thomas]

I don't have a proposal thought out, only some beginning ideas. Anybody
is welcome to build on them for whatever purpose, or to criticise them.

Usually what we need is not IMMEDIATE words or state-smart words.
IMMEDIATE was a quick simple hack that people have been working around
for 30 years. It's usually adequate, but not good.

Any word that includes POSTPONE or executes COMPILE, (or [COMPILE] ?)
is compile-only, and recursively any word that executes a word that
includes POSTPONE etc it compile-only.

The complications of state-smart etc don't belong in the words
themselves. The issue doesn't come up after you execute the words. The
issue comes up in the interpreter/compiler, when you FIND words and
decide what to do with them.

What if FIND (or its string-and-count replacement) were to give more
information? What if it were to give

-3 normal compilation behavior, do not interpret.
-2 discard the xt always, do nothing.
-1 execute when interpreting, compile when compiling.
0 not found
1 immediate
2 do not interpret, execute while compiling
3. execute while compiling, do not execute while interpreting, but a
second xt is available to execute while interpreting or fetch with '
['] etc. The other xt can be found with a special command.

Wouldn't a lot of the endless complications go away if this extra
information was available to standard programs?

Whenever you POSTPONE a word or compile COMPILE, the current word
becomes type -3 unless it already has compiled : or :NONAME or
postponed DOES> , and if you try to make it immediate it becomes type
2. The rules for that might turn out to be too complicated to use, but
some of them might give useful warning messages without getting too
much in the way.

Wordlists don't clean up the IMMEDIATE mess, they make more messes
attempting to solve that. IMMEDIATE affects FIND . State-smart words
don't work because they try to fix something that ought to happen while
parsing with something that happens at execution time -- whenever
execution happens. An improved FIND and more sophisticated behaviors at
that point might do it.

[Mark W. Humphries]

J Thomas wrote:
[snip]
> What if FIND (or its string-and-count replacement) were to give more
> information?
[snip]

I use a variant of FIND that accepts a mask telling it which behavior
I'm looking for:

" wordX" interpretation lookup
" wordX" compilation lookup

Lookup returns an lfa which I can interrogate further with IMMEDIATE?
for example.

Cheers,
Mark Humphries
Manila, Philippines

[Andrew Haley]

J Thomas <***@gmail.com> wrote:
> I don't have a proposal thought out, only some beginning ideas. Anybody
> is welcome to build on them for whatever purpose, or to criticise them.

> Usually what we need is not IMMEDIATE words or state-smart words.
> IMMEDIATE was a quick simple hack that people have been working
> around for 30 years. It's usually adequate, but not good.

> Any word that includes POSTPONE or executes COMPILE, (or [COMPILE]
> ?) is compile-only, and recursively any word that executes a word
> that includes POSTPONE etc it compile-only.

> The complications of state-smart etc don't belong in the words
> themselves.

Semi-seriously: The complications of state-smart don't belong
anywhere. Give it up; you'll feel much better. :-)

> The issue doesn't come up after you execute the words. The issue
> comes up in the interpreter/compiler, when you FIND words and decide
> what to do with them.

> What if FIND (or its string-and-count replacement) were to give more
> information? What if it were to give

> -3 normal compilation behavior, do not interpret.
> -2 discard the xt always, do nothing.
> -1 execute when interpreting, compile when compiling.
> 0 not found
> 1 immediate
> 2 do not interpret, execute while compiling
> 3. execute while compiling, do not execute while interpreting, but a
> second xt is available to execute while interpreting or fetch with '
> ['] etc. The other xt can be found with a special command.

> Wouldn't a lot of the endless complications go away if this extra
> information was available to standard programs?

I don't see how making state even more complicated is likely to
simplify anything.

> Wordlists don't clean up the IMMEDIATE mess, they make more messes
> attempting to solve that. IMMEDIATE affects FIND . State-smart words
> don't work because they try to fix something that ought to happen
> while parsing with something that happens at execution time --
> whenever execution happens.

I don't really understand this. Immediate words are precisely those
words that always execute while parsing, whether in compile or
interpret state. I wonder: are you trying to do something especially
exotic with a complicated syntax?

Andrew.

[J Thomas]

Andrew Haley wrote:

> > The complications of state-smart etc don't belong in the words
> > themselves.

> Semi-seriously: The complications of state-smart don't belong
> anywhere. Give it up; you'll feel much better. :-)

I agree with you about state-smart words. Having two behaviors for one
name is almost as bad as using two names for one behavior. :-)

> I don't see how making state even more complicated is likely to
> simplify anything.

Well, I wanted to include the common practice and add something new so
it wouldn't have to break existing code. Doing that almost inevitably
means adding complication -- you can't get rid of the existing garbage
but can only add something new and (hopefully) better.

> > Wordlists don't clean up the IMMEDIATE mess, they make more messes
> > attempting to solve that. IMMEDIATE affects FIND . State-smart words
> > don't work because they try to fix something that ought to happen
> > while parsing with something that happens at execution time --
> > whenever execution happens.

> I don't really understand this. Immediate words are precisely those
> words that always execute while parsing, whether in compile or
> interpret state. I wonder: are you trying to do something especially
> exotic with a complicated syntax?

Once you get rid of state-smart words, the only value you get from
immediate words are a few anomalies like ( . A small handful of words
that you want to do exactly the same thing whether you are compiling or
interpreting. You do better to treat almost every immediate word as
compile-only, as an addition to the compiler.

Immediate doesn't do the job right, but it's adequately usable by
people who just want to get their work done. Use it wrong and it might
bite you, and then you'll notice you used it wrong and fix the problem.
If you try to write portable code you might not notice immediate
mistakes until you port to a standard system which does things
differently from your system. At that point you can notice the problem
and fix it.

The real problem here is for people who want to build portable tools.
Say I make something that looks like a nice simple tool, that ought to
be portable to most standard Forths. I put it up here and somebody says
"What about this weird trick here, somebody can do this peculiar
programming trick and your tool will break." So then I jump through
more hoops, and the weirder the programming tricks we're willing to
consider the higher and smaller the flaming hoops get. At some point it
makes sense to just declare an environmental dependency on systems that
do what I need them to, and trust that programmers whose tricks aren't
too weird might have a use for the tools.

But I want to find a way to handle this particular problem. It looked
to me like we'd get a lot of freedom for standard programs with
POSTPONE . You could extend the compiler any way you wanted, portably.
State-smart words are potentially confusing, but there was the chance
to redefine any word as state-smart, and give it whatever extra
behavior you wanted and if you did it carefully, all the extra things
would be invisible to standard programs unless they knew just how to
look. But it turned out to be kind of a mirage. The STATE/IMMEDIATE
system didn't allow quite that much flexibility, when it also had to
fit in with the flexibility of EXECUTE and COMPILE, as well as
[COMPILE] and POSTPONE . By the time you add in the whereases and
yesbuts needed to allow various legacy systems to be standard, it turns
into what John Doty would call language lawyering. It's so much better
to have simple tools that do the same thing all the time, than simple
tools that work right up until you do something unusual and then they
break in some unusual way. And that's better than complicated tools
that have lots of complicated alternative behaviors for special
circumstances, that are likely to break in unusual ways when you least
expect them to.

My idea of having FIND (or the next word) give you more information may
seem like it's too complex. I don't see any way to make it real simple
and still keep the old STATE/IMMEDIATE stuff too. I'll keep looking.

[Jean-François Michaud]

J Thomas wrote:
> Andrew Haley wrote:
>
> > > The complications of state-smart etc don't belong in the words
> > > themselves.
>
> > Semi-seriously: The complications of state-smart don't belong
> > anywhere. Give it up; you'll feel much better. :-)
>
> I agree with you about state-smart words. Having two behaviors for one
> name is almost as bad as using two names for one behavior. :-)
>
> > I don't see how making state even more complicated is likely to
> > simplify anything.
>
> Well, I wanted to include the common practice and add something new so
> it wouldn't have to break existing code. Doing that almost inevitably
> means adding complication -- you can't get rid of the existing garbage
> but can only add something new and (hopefully) better.
>
> > > Wordlists don't clean up the IMMEDIATE mess, they make more messes
> > > attempting to solve that. IMMEDIATE affects FIND . State-smart words
> > > don't work because they try to fix something that ought to happen
> > > while parsing with something that happens at execution time --
> > > whenever execution happens.
>
> > I don't really understand this. Immediate words are precisely those
> > words that always execute while parsing, whether in compile or
> > interpret state. I wonder: are you trying to do something especially
> > exotic with a complicated syntax?
>
> Once you get rid of state-smart words, the only value you get from
> immediate words are a few anomalies like ( . A small handful of words
> that you want to do exactly the same thing whether you are compiling or
> interpreting. You do better to treat almost every immediate word as
> compile-only, as an addition to the compiler.
>
> Immediate doesn't do the job right, but it's adequately usable by
> people who just want to get their work done. Use it wrong and it might
> bite you, and then you'll notice you used it wrong and fix the problem.
> If you try to write portable code you might not notice immediate
> mistakes until you port to a standard system which does things
> differently from your system. At that point you can notice the problem
> and fix it.
>
> The real problem here is for people who want to build portable tools.
> Say I make something that looks like a nice simple tool, that ought to
> be portable to most standard Forths. I put it up here and somebody says
> "What about this weird trick here, somebody can do this peculiar
> programming trick and your tool will break." So then I jump through
> more hoops, and the weirder the programming tricks we're willing to
> consider the higher and smaller the flaming hoops get. At some point it
> makes sense to just declare an environmental dependency on systems that
> do what I need them to, and trust that programmers whose tricks aren't
> too weird might have a use for the tools.

Portability is a non problem simply because absolute portability in
general is an illusion unless everybody works with the same common
platform implemented in exactly the same way (which is in general very
difficult if not impossible. This is also why standards exist). Now
some implementations of the standard might have collateral behaviors
that are not the same in another system also implementing the standard.
This is because the standard that is followed is too "high level". The
implementation is left to the system programmers. As long as the high
level functionalities are available as per the standard. The is the
root cause of the problem you expose which is not resolvable to the
extent to which you would like to see.

The easy solution is to redefine the words that cause problem so that
they match the imlementation on which it runs originally, but only for
the particular program that you are running. This creates other issues.
You might as well just use the original compiler in which the
application was written.

An interresting solution to explore might simply be to create a
compiler executer. A program that makes sure that a given program runs
in the language of its own native compiler. Everybody would be free to
do whatever program they want, as long as some information follows the
program telling the compiler executer in which native compiler the
program is written (those compilers would have to respect certain
limitation so that the execution of compilers is disjoint as much as
possible to avoid collateral effects). But because there are commercial
Forths out there, this wouldn't work. This is why open Forths are more
interresting in this respect.

Otherwise, we are simply talking about porting a program to another
compiler.

> But I want to find a way to handle this particular problem. It looked
> to me like we'd get a lot of freedom for standard programs with
> POSTPONE . You could extend the compiler any way you wanted, portably.
> State-smart words are potentially confusing, but there was the chance
> to redefine any word as state-smart, and give it whatever extra
> behavior you wanted and if you did it carefully, all the extra things
> would be invisible to standard programs unless they knew just how to
> look. But it turned out to be kind of a mirage. The STATE/IMMEDIATE
> system didn't allow quite that much flexibility, when it also had to
> fit in with the flexibility of EXECUTE and COMPILE, as well as
> [COMPILE] and POSTPONE . By the time you add in the whereases and
> yesbuts needed to allow various legacy systems to be standard, it turns
> into what John Doty would call language lawyering. It's so much better
> to have simple tools that do the same thing all the time, than simple
> tools that work right up until you do something unusual and then they
> break in some unusual way. And that's better than complicated tools
> that have lots of complicated alternative behaviors for special
> circumstances, that are likely to break in unusual ways when you least
> expect them to.
>
> My idea of having FIND (or the next word) give you more information may
> seem like it's too complex. I don't see any way to make it real simple
> and still keep the old STATE/IMMEDIATE stuff too. I'll keep looking.

[Andrew Haley]

J Thomas <***@gmail.com> wrote:

> Andrew Haley wrote:

>> > The complications of state-smart etc don't belong in the words
>> > themselves.

>> Semi-seriously: The complications of state-smart don't belong
>> anywhere. Give it up; you'll feel much better. :-)

> I agree with you about state-smart words. Having two behaviors for one
> name is almost as bad as using two names for one behavior. :-)

OK, so we are on the same page. Good.

>> I don't see how making state even more complicated is likely to
>> simplify anything.

> Well, I wanted to include the common practice and add something new so
> it wouldn't have to break existing code. Doing that almost inevitably
> means adding complication -- you can't get rid of the existing garbage
> but can only add something new and (hopefully) better.

I'm still baffled by what it is you want to be rid of.

>> > Wordlists don't clean up the IMMEDIATE mess, they make more messes
>> > attempting to solve that. IMMEDIATE affects FIND . State-smart words
>> > don't work because they try to fix something that ought to happen
>> > while parsing with something that happens at execution time --
>> > whenever execution happens.

>> I don't really understand this. Immediate words are precisely those
>> words that always execute while parsing, whether in compile or
>> interpret state. I wonder: are you trying to do something especially
>> exotic with a complicated syntax?

> Once you get rid of state-smart words, the only value you get from
> immediate words are a few anomalies like ( . A small handful of words
> that you want to do exactly the same thing whether you are compiling or
> interpreting. You do better to treat almost every immediate word as
> compile-only, as an addition to the compiler.

OK. It seems to me that IMMEDIATE does this job quite well already:
if you have a word that only makes sense in compile state (e.g. IF,
WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
one job to do.

> But I want to find a way to handle this particular problem. It
> looked to me like we'd get a lot of freedom for standard programs
> with POSTPONE . You could extend the compiler any way you wanted,
> portably. State-smart words are potentially confusing, but there
> was the chance to redefine any word as state-smart, and give it
> whatever extra behavior you wanted and if you did it carefully, all
> the extra things would be invisible to standard programs unless they
> knew just how to look. But it turned out to be kind of a mirage. The
> STATE/IMMEDIATE system didn't allow quite that much flexibility,
> when it also had to fit in with the flexibility of EXECUTE and
> COMPILE, as well as [COMPILE] and POSTPONE . By the time you add in
> the whereases and yesbuts needed to allow various legacy systems to
> be standard, it turns into what John Doty would call language
> lawyering. It's so much better to have simple tools that do the same
> thing all the time, than simple tools that work right up until you
> do something unusual and then they break in some unusual way. And
> that's better than complicated tools that have lots of complicated
> alternative behaviors for special circumstances, that are likely to
> break in unusual ways when you least expect them to.

I'm sorry, this is all too abstract for me. I guess I don't
understand what the problems are you're trying to eliminate; in the
absence of any examples I have no way to know. I know what I think
should go: STATE. At the time of ANS, Forth, Inc already had
well-established practice for a STATE-free Forth, but STATE went into
the standard anyway.

Andrew.

[J Thomas]

Andrew Haley wrote:
> J Thomas <***@gmail.com> wrote:

> I'm still baffled by what it is you want to be rid of.

I'll try again.

> > Once you get rid of state-smart words, the only value you get from
> > immediate words are a few anomalies like ( . A small handful of words
> > that you want to do exactly the same thing whether you are compiling or
> > interpreting. You do better to treat almost every immediate word as
> > compile-only, as an addition to the compiler.

> OK. It seems to me that IMMEDIATE does this job quite well already:
> if you have a word that only makes sense in compile state (e.g. IF,
> WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
> one job to do.

IMMEDIATE is adequate to a lot of tasks. If you want a word that only
makes sense in compile state, then you can make it with IMMEDIATE . And
you can use your innate good sense not to use it except in compile
state.

If you want the same name to show two different behaviors, one in
interpret state and one in compile state, pretty much your only
portable choice is to write a state-smart word with all the
disadvantages that come with that.

There are two kinds of disadvantages to state-smart words. One kind
comes from the way you have to implement it. The word decides for
itself at run-time what to do, and for some purposes that's the wrong
time to choose. The other kind of disadvantage comes from having one
name with two behaviors. I say that sometimes it's worth it to deal
with the second set of disadvantages. You deal with those problems
every time you put two words with the same name in two different
wordlists. But then you aren't choosing at runtime, you're chosing when
you lookup the word.

If you never want one name for two behaviors, then there's no problem
here. If you might sometime want one name with two behaviors, different
for compiling versus interpreting, then you're more likely to want to
choose when you lookup the word and not at runtime. IMHO. But there's
no standard way to do that. Lots of people have implementations similar
to Mark Humphries' approach, and they work. Practically everybody has
ways to extend their interpreter. But there's no way for a standard
program to do either of those.

> I'm sorry, this is all too abstract for me. I guess I don't
> understand what the problems are you're trying to eliminate; in the
> absence of any examples I have no way to know. I know what I think
> should go: STATE. At the time of ANS, Forth, Inc already had
> well-established practice for a STATE-free Forth, but STATE went into
> the standard anyway.

Get rid of STATE and IMMEDIATE and probably my concerns will come out
in the wash. But it's hard to get rid of things that have been standard
for a long time. Get rid of IMMEDIATE and you break every line of
standard code that has an IMMEDIATE in it. (Unless the new way lets you
define IMMEDIATE portably.)

So I want to find a way to add the features I want instead of get rid
of existing features. Then as the body of code that uses the old
features ages and code that uses the new features builds up, future
standards can consider those old features obsolescent and finally get
rid of them completely.

What I want is increased ability to choose what to do with a word, at
lookup time.

Known methods that can get this include:

Wordlists. Let the search order change when compiling. This kind of
breaks the standard requirement that the latest defined word is found
first. It doubles the number of wordlists needed.

Compile-only and interpret-only words. This works provided that when
you find a compile-only word while interpreting and vice versa, you
keep looking.

Dual-xt's. This works, but requires an extra field in each word.

Interpreter hooks. This is valuable for a lot of other things. It's
wasteful for adding behaviors to individual words since at least a
compare must be done for every word parsed that isn't the one you want.

Additional choices at FIND besides -1|0|1. Breaks no existing code.
Possibly easy to convert legacy Forths, provided sufficient bits are
available in headers. Maybe nothing is needed beyond compile-only and
interpret-only words, plus the default and immediate.

And many more exotic solutions are possible.


On the other side there's the question whether these capabilities are
worth having. In each case, it could be argued that they give
programmers too much power. If you give Forth programmers power,
they'll use it.

They'll extend the language in new and tricky ways. A standard does
more than code portability, it also provides programmer portability,
and the more tricky code that's hard to understand, the less
interchangeable the programmers.

When there's only one way to solve a problem, programmers know what to
do. When there are two ways they're never sure. It could be argued that
a Forth standard should give programmers firm limits so they'll just
get their work done and not be tempted to find brilliant tricky
solutions that give the language a bad reputation. Don't let them
extend the compiler. Standard code should all be written the same way
and it should all be application code.

It's possible I want things I shouldn't want.

[Andrew Haley]

J Thomas <***@gmail.com> wrote:

> Andrew Haley wrote:
>> J Thomas <***@gmail.com> wrote:

>> I'm still baffled by what it is you want to be rid of.

> I'll try again.

>> > Once you get rid of state-smart words, the only value you get from
>> > immediate words are a few anomalies like ( . A small handful of words
>> > that you want to do exactly the same thing whether you are compiling or
>> > interpreting. You do better to treat almost every immediate word as
>> > compile-only, as an addition to the compiler.

>> OK. It seems to me that IMMEDIATE does this job quite well already:
>> if you have a word that only makes sense in compile state (e.g. IF,
>> WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
>> one job to do.

> IMMEDIATE is adequate to a lot of tasks. If you want a word that only
> makes sense in compile state, then you can make it with IMMEDIATE . And
> you can use your innate good sense not to use it except in compile
> state.

> If you want the same name to show two different behaviors, one in
> interpret state and one in compile state, pretty much your only
> portable choice is to write a state-smart word with all the
> disadvantages that come with that.

Right. So, if you simply don't use STATE, then you can't do that, and
the problem goes away.

> There are two kinds of disadvantages to state-smart words. One kind
> comes from the way you have to implement it. The word decides for
> itself at run-time what to do, and for some purposes that's the wrong
> time to choose. The other kind of disadvantage comes from having one
> name with two behaviors. I say that sometimes it's worth it to deal
> with the second set of disadvantages. You deal with those problems
> every time you put two words with the same name in two different
> wordlists. But then you aren't choosing at runtime, you're chosing when
> you lookup the word.

> If you never want one name for two behaviors, then there's no
> problem here.

Right; problem solved.

> If you might sometime want one name with two behaviors, different
> for compiling versus interpreting, then you're more likely to want
> to choose when you lookup the word and not at runtime. IMHO. But
> there's no standard way to do that. Lots of people have
> implementations similar to Mark Humphries' approach, and they
> work. Practically everybody has ways to extend their
> interpreter. But there's no way for a standard program to do either
> of those.

>> I'm sorry, this is all too abstract for me. I guess I don't
>> understand what the problems are you're trying to eliminate; in the
>> absence of any examples I have no way to know. I know what I think
>> should go: STATE. At the time of ANS, Forth, Inc already had
>> well-established practice for a STATE-free Forth, but STATE went into
>> the standard anyway.

> Get rid of STATE and IMMEDIATE and probably my concerns will come
> out in the wash. But it's hard to get rid of things that have been
> standard for a long time. Get rid of IMMEDIATE and you break every
> line of standard code that has an IMMEDIATE in it. (Unless the new
> way lets you define IMMEDIATE portably.)

No, just get rid of STATE. The great thing about programming in a
STATE-free Forth (or, at least, one in which STATE is invisible) is
that all your code can still be ANS standard Forth programs. All you
have to do is never use STATE.

I'd like to emphasize that the problem is not STATE per se, but having
words that -- as you put it -- show two different behaviors, one in
interpret state and one in compile state.

FORTH, Inc used their STATEless Forth for many years, with no
application problems of which I am aware. STATE had to go back in to
their products order to be ANS compatible.

> When there's only one way to solve a problem, programmers know what
> to do. When there are two ways they're never sure. It could be
> argued that a Forth standard should give programmers firm limits so
> they'll just get their work done and not be tempted to find
> brilliant tricky solutions that give the language a bad
> reputation. Don't let them extend the compiler. Standard code should
> all be written the same way and it should all be application code.

No, extending the compiler is good. You don't need STATE to have an
extensible compiler.

Andrew.

[Elizabeth D Rather]

J Thomas wrote:
> Andrew Haley wrote:
>> J Thomas <***@gmail.com> wrote:
>
...

Andrew is doing a fine job with this topic, I just want to add a bit.

> IMMEDIATE is adequate to a lot of tasks. If you want a word that only
> makes sense in compile state, then you can make it with IMMEDIATE . And
> you can use your innate good sense not to use it except in compile
> state.
>
> If you want the same name to show two different behaviors, one in
> interpret state and one in compile state, pretty much your only
> portable choice is to write a state-smart word with all the
> disadvantages that come with that.

Agree entirely. In 35 years I've yet to have a problem with IMMEDIATE.
In fact, I've been at a loss to understand what "mess" it is you're
worried about.

> There are two kinds of disadvantages to state-smart words. One kind

> ...
> So I want to find a way to add the features I want instead of get rid
> of existing features. Then as the body of code that uses the old
> features ages and code that uses the new features builds up, future
> standards can consider those old features obsolescent and finally get
> rid of them completely.
>
> What I want is increased ability to choose what to do with a word, at
> lookup time.
>
> Known methods that can get this include:
>
> [list snipped]
>
> On the other side there's the question whether these capabilities are
> worth having. In each case, it could be argued that they give
> programmers too much power. If you give Forth programmers power,
> they'll use it.
>
> They'll extend the language in new and tricky ways. A standard does
> more than code portability, it also provides programmer portability,
> and the more tricky code that's hard to understand, the less
> interchangeable the programmers.

You're talking about Tom Cruise-type hot-shots who have to show off.
They probably exist in every language, and are mostly anathema to
serious folks just trying to get work done. I object both to adding
features that cater to the hot shots, and equally to efforts to thwart
them that have the side-effect of hampering the ability of the serious
professionals to get their work done.

> ...
>
> It's possible I want things I shouldn't want.

Entirely possible. It seems to me that you're being excessively
theoretical in imagining problems where none exist on this issue.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. [phone]
5155 W. Rosecrans Ave. #1018 Fax: [phone]
Hawthorne, CA 90250
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Alex McDonald]

Elizabeth D Rather wrote:
> J Thomas wrote:
> > Andrew Haley wrote:
> >> J Thomas <***@gmail.com> wrote:
> >
> ...
>
> Andrew is doing a fine job with this topic, I just want to add a bit.
>
> > IMMEDIATE is adequate to a lot of tasks. If you want a word that only
> > makes sense in compile state, then you can make it with IMMEDIATE . And
> > you can use your innate good sense not to use it except in compile
> > state.
> >
> > If you want the same name to show two different behaviors, one in
> > interpret state and one in compile state, pretty much your only
> > portable choice is to write a state-smart word with all the
> > disadvantages that come with that.
>
> Agree entirely. In 35 years I've yet to have a problem with IMMEDIATE.
> In fact, I've been at a loss to understand what "mess" it is you're
> worried about.
>
> > There are two kinds of disadvantages to state-smart words. One kind
>
> > ...
> > So I want to find a way to add the features I want instead of get rid
> > of existing features. Then as the body of code that uses the old
> > features ages and code that uses the new features builds up, future
> > standards can consider those old features obsolescent and finally get
> > rid of them completely.
> >
> > What I want is increased ability to choose what to do with a word, at
> > lookup time.
> >
> > Known methods that can get this include:
> >
> > [list snipped]
> >
> > On the other side there's the question whether these capabilities are
> > worth having. In each case, it could be argued that they give
> > programmers too much power. If you give Forth programmers power,
> > they'll use it.
> >
> > They'll extend the language in new and tricky ways. A standard does
> > more than code portability, it also provides programmer portability,
> > and the more tricky code that's hard to understand, the less
> > interchangeable the programmers.
>
> You're talking about Tom Cruise-type hot-shots who have to show off.
> They probably exist in every language, and are mostly anathema to
> serious folks just trying to get work done. I object both to adding
> features that cater to the hot shots, and equally to efforts to thwart
> them that have the side-effect of hampering the ability of the serious
> professionals to get their work done.
>
> > ...
> >
> > It's possible I want things I shouldn't want.
>
> Entirely possible. It seems to me that you're being excessively
> theoretical in imagining problems where none exist on this issue.

I can't see why you claim "Tom Cruise-type hot-shots" are the only ones
interested in getting this to work. Documenting immediacy was something
that the ANS Forth committee went to great lengths to eliminate
(section A.6.1.2033 POSTPONE);

<quote>
COMPILE was designed to be applied to non-immediate words and [COMPILE]
to immediate words. This burdens the programmer with needing to know
which words in a system are immediate. Consequently, Forth standards
have had to specify the immediacy or non-immediacy of all words covered
by the Standard. This unnecessarily constrains implementors.
</quote>

What makes state-smart words different? If my implementation of word X
can't be POSTPONEd correctly because it's state-smart, but your X can,
that seems to me a fundamental burden on the programmer much worse than
the immediacy problem the standard fixed, as the list of words can very
from system to system -- and they can all still claim ANS compliance
without documenting them. I for one see it to be a problem worthy of
discussion. If serious professionals think otherwise, I'm baffled as to
why they might consider program portability and correctness an optional
feature. Enlightenment, please.

--
Regards
Alex McDonald

[Elizabeth D Rather]

Alex McDonald wrote:
> Elizabeth D Rather wrote:
>> J Thomas wrote:
...
>>> ...
>>>
>>> It's possible I want things I shouldn't want.
>> Entirely possible. It seems to me that you're being excessively
>> theoretical in imagining problems where none exist on this issue.
>
> I can't see why you claim "Tom Cruise-type hot-shots" are the only ones
> interested in getting this to work. Documenting immediacy was something
> that the ANS Forth committee went to great lengths to eliminate
> (section A.6.1.2033 POSTPONE);
>
> <quote>
> COMPILE was designed to be applied to non-immediate words and [COMPILE]
> to immediate words. This burdens the programmer with needing to know
> which words in a system are immediate. Consequently, Forth standards
> have had to specify the immediacy or non-immediacy of all words covered
> by the Standard. This unnecessarily constrains implementors.
> </quote>
>
> What makes state-smart words different? If my implementation of word X
> can't be POSTPONEd correctly because it's state-smart, but your X can,
> that seems to me a fundamental burden on the programmer much worse than
> the immediacy problem the standard fixed, as the list of words can very
> from system to system -- and they can all still claim ANS compliance
> without documenting them. I for one see it to be a problem worthy of
> discussion. If serious professionals think otherwise, I'm baffled as to
> why they might consider program portability and correctness an optional
> feature. Enlightenment, please.

Well, JET, Andrew, and I have all agreed that state-smart words are best
avoided. The few that survive in the standard that may cause trouble
(e.g., TO) have warnings against POSTPONEing them. IMO if you have
implemented some standard word in such a way that there are extra
restrictions on it (e.g. it isn't POSTPONEable) you should document
those restrictions, and your system is standard with those limitations.

I'm all in favor of program portability and correctness, but I really
don't see a problem here unless one is trying to do something really
bizarre to prove a point (such as JET's DUP example).

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. [phone]
5155 W. Rosecrans Ave. #1018 Fax: [phone]
Hawthorne, CA 90250
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[J Thomas]

Elizabeth D Rather wrote:
> J Thomas wrote:

> > IMMEDIATE is adequate to a lot of tasks. If you want a word that only
> > makes sense in compile state, then you can make it with IMMEDIATE . And
> > you can use your innate good sense not to use it except in compile
> > state.

> Agree entirely. In 35 years I've yet to have a problem with IMMEDIATE.
> In fact, I've been at a loss to understand what "mess" it is you're
> worried about.

That's understandable. You've had 35 years to learn to work within its
limitations. You've learned not to even imagine doing things outside
those limitations.

> > On the other side there's the question whether these capabilities are
> > worth having. In each case, it could be argued that they give
> > programmers too much power. If you give Forth programmers power,
> > they'll use it.

> You're talking about Tom Cruise-type hot-shots who have to show off.
> They probably exist in every language, and are mostly anathema to
> serious folks just trying to get work done. I object both to adding
> features that cater to the hot shots, and equally to efforts to thwart
> them that have the side-effect of hampering the ability of the serious
> professionals to get their work done.

I dunno. Allowed the tools to make the tools, we might wind up with
better tools. On the other hand it's better not to do hot-shot stuff
that isn't worth doing.

And then it might all be moot. The standard provided methods to create
whatever control structure you might want. With BEGIN AGAIN UNTIL IF
AHEAD THEN CS-PICK CS-ROLL we can make any control structure
whatsoever. And since 1994 I haven't noticed hotshot programmers
mis-use those tools, and I haven't noticed any valuable new control
structures get published. Maybe people will speak up about valuable
uses they've seen.

> > It's possible I want things I shouldn't want.

> Entirely possible. It seems to me that you're being excessively
> theoretical in imagining problems where none exist on this issue.

Clearly, it's possible to work around IMMEDIATE enough to get adequate
results. So I can see that you'd figure it isn't a problem. I haven't
taken a firm stand on the matter, I'm still exploring. It looks to me
like we could get even more useful results and also simpler, with one
of several other approaches. On the other hand IMMEDIATE is in common
use and it's been flash-gilded by the standard. Add the better methods
instead of using them to replace IMMEDIATE and the result may be more
complicated. I'm not sure what's best.

Can you see the possibility that there may be benefits here worth
having? Not that the old way is impossible to use, but that there
*might* be enough added value in an alternative to justify exploring
it?

[Elizabeth D Rather]

J Thomas wrote:
> Elizabeth D Rather wrote:
>> J Thomas wrote:

> ...
>> You're talking about Tom Cruise-type hot-shots who have to show off.
>> They probably exist in every language, and are mostly anathema to
>> serious folks just trying to get work done. I object both to adding
>> features that cater to the hot shots, and equally to efforts to thwart
>> them that have the side-effect of hampering the ability of the serious
>> professionals to get their work done.
>
> I dunno. Allowed the tools to make the tools, we might wind up with
> better tools. On the other hand it's better not to do hot-shot stuff
> that isn't worth doing.
>
> And then it might all be moot. The standard provided methods to create
> whatever control structure you might want. With BEGIN AGAIN UNTIL IF
> AHEAD THEN CS-PICK CS-ROLL we can make any control structure
> whatsoever. And since 1994 I haven't noticed hotshot programmers
> mis-use those tools, and I haven't noticed any valuable new control
> structures get published. Maybe people will speak up about valuable
> uses they've seen.

My point exactly.

> ...
> Can you see the possibility that there may be benefits here worth
> having? Not that the old way is impossible to use, but that there
> *might* be enough added value in an alternative to justify exploring
> it?

The history of development of Forth in general, and our products in
particular, is that we see a need for something and find a way to
provide it. I recall many years ago, Mike LaManna was working for us on
a project in which he was required to exactly implement some really
nasty, unstructured flow charts. He developed a structure which he called:
BEGIN ... WHILE ... MAYBE ... REPEAT
Needs such as this were in our minds when the TC made many of the
structure words "mix and match", as you describe above.

So, we're always looking for ways to simplify the programmer's life.
Where we see needs, we find ways to address them. Where folks have
found common needs, they've found similar solutions which, after a time,
become candidates for standardization. DEFER is a good example, as is
\S". What I've been trying to say is, we haven't found a need to "fix"
IMMEDIATE. And I don't like trying to invent problems that haven't
cropped up in real life.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. [phone]
5155 W. Rosecrans Ave. #1018 Fax: [phone]
Hawthorne, CA 90250
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[J Thomas]

Elizabeth D Rather wrote:
> J Thomas wrote:

> > Can you see the possibility that there may be benefits here worth
> > having? Not that the old way is impossible to use, but that there
> > *might* be enough added value in an alternative to justify exploring
> > it?

> The history of development of Forth in general, and our products in
> particular, is that we see a need for something and find a way to
> provide it. I recall many years ago, Mike LaManna was working for us on
> a project in which he was required to exactly implement some really
> nasty, unstructured flow charts. He developed a structure which he called:
> BEGIN ... WHILE ... MAYBE ... REPEAT
> Needs such as this were in our minds when the TC made many of the
> structure words "mix and match", as you describe above.

Sure. And the existing control-flow words are good enough that I
haven't heard of any variants getting into common use.

Even though the tools are there in any system which provides the Tools
Extension wordset.

I think Standard Forth is better off to have the capability even though
it hasn't been used. We haven't had a significant problem with hotshot
programmers misusing it, we haven't used it well, we're still better
off with it than without it.

> So, we're always looking for ways to simplify the programmer's life.
> Where we see needs, we find ways to address them.

It occurs to me that you are somebody who sells Forth systems, and you
benefit when your Forth systems are the best. I am somebody who wants
to write portable code, to write it both for multiple systems and on
multiple systems. Our needs and interests might not be perfectly
aligned.

[David N. Williams]

J Thomas wrote:
> Elizabeth D Rather wrote:
> [...]
>> The history of development of Forth in general, and our products in
>> particular, is that we see a need for something and find a way to
>> provide it. I recall many years ago, Mike LaManna was working for us on
>> a project in which he was required to exactly implement some really
>> nasty, unstructured flow charts. He developed a structure which he
called:
>> BEGIN ... WHILE ... MAYBE ... REPEAT
>> Needs such as this were in our minds when the TC made many of the
>> structure words "mix and match", as you describe above.
>
> Sure. And the existing control-flow words are good enough that I
> haven't heard of any variants getting into common use.
>
> Even though the tools are there in any system which provides the Tools
> Extension wordset.

What about Wil Baden's COND ... THENS, et al.? As far as I
remember, they're not portably implementable because there's no
CS-DEPTH, or some such.

-- David

[Neal Bridges]

"David N. Williams" <***@umich.edu> wrote in message
news:R8jJg.4016$***@newssvr21.news.prodigy.com...
> J Thomas wrote:
[snip]
> > Even though the tools are there in any system which provides the Tools
> > Extension wordset.
>
> What about Wil Baden's COND ... THENS, et al.? As far as I
> remember, they're not portably implementable because there's no
> CS-DEPTH, or some such.

COND/THENS doesn't CS-DEPTH. They assume the data stack is the
control-flow stack, and COND leaves a sentinel value on it that THENS
terminates on.

--
Neal Bridges
http://quartus.net Home of Quartus Forth for the Palm OS!

[David N. Williams]

Neal Bridges wrote:
> "David N. Williams" <***@umich.edu> wrote in message
> news:R8jJg.4016$***@newssvr21.news.prodigy.com...
>> J Thomas wrote:
> [snip]
>> > Even though the tools are there in any system which provides the Tools
>> > Extension wordset.
>>
>> What about Wil Baden's COND ... THENS, et al.? As far as I
>> remember, they're not portably implementable because there's no
>> CS-DEPTH, or some such.
>
> COND/THENS doesn't CS-DEPTH. They assume the data stack is the
> control-flow stack, and COND leaves a sentinel value on it that THENS
> terminates on.
>

Okay, so it's "some such". :-) (Or maybe et al. ...)

-- David

[J Thomas]

Neal Bridges wrote:
> "David N. Williams" <***@umich.edu> wrote in message
> news:R8jJg.4016$***@newssvr21.news.prodigy.com...
> > J Thomas wrote:
> [snip]
> > > Even though the tools are there in any system which provides the Tools
> > > Extension wordset.
>
> > What about Wil Baden's COND ... THENS, et al.? As far as I
> > remember, they're not portably implementable because there's no
> > CS-DEPTH, or some such.
>
> COND/THENS doesn't CS-DEPTH. They assume the data stack is the
> control-flow stack, and COND leaves a sentinel value on it that THENS
> terminates on.

THENS also assumes that nothing else leaves anything on the data stack
while the construction is going on.

Normally you can have that.

.... IF [ 3 ] ... IF ... THEN .... [ drop ] THEN ....

But you can't do that with THENS .

You could get almost as good a result with a variable.

variable #conds
: COND 0 #conds ! ; immediate
: IF 1 #conds +! postpone IF ; immediate

: THENS #conds @ 0 do postpone THEN loop ; immediate

This isn't as flexible because you can't nest THENS . I tend to think
that a word with nested THENS is too big, but still that's something
you can do with Wil's THENS that you can't do with mine. Mine ought to
be portable, though.

You could make Wil's version more portable with a little change.

: COND 0 ; immediate
: IF postpone IF 48813 ; immediate
: THEN drop postpone THEN ; immediate
: THENS
begin postpone THEN dup 48813 = while repeat drop ; immediate

I think this ought to work whether or not the controlflow stack uses
the data stack, but it's more complicated. It will give an undefined
result if you leave an extra 48813 on the data stack inside the THENS
code during compilation. And if you leave anything else it will
terminate early.

I expect Wil didn't make it more portable because he didn't care about
Forths that didn't use the data stack for the controlflow stack.

[Neal Bridges]

"J Thomas" <***@gmail.com> wrote in message
news:***@m79g2000cwm.googlegroups.com...
>
> Neal Bridges wrote:
[snip]
> > COND/THENS doesn't CS-DEPTH. They assume the data stack is the
> > control-flow stack, and COND leaves a sentinel value on it that THENS
> > terminates on.
>
> THENS also assumes that nothing else leaves anything on the data stack
> while the construction is going on.
>
> Normally you can have that.
>
> .... IF [ 3 ] ... IF ... THEN .... [ drop ] THEN ....
>
> But you can't do that with THENS .

Sure you can.

... cond [ 3 ] ... [ drop ] thens ...

Why you would want to something like that, I can't imagine, but you can.
--
Neal Bridges
http://quartus.net Home of Quartus Forth for the Palm OS!

[J Thomas]

Neal Bridges wrote:
> "J Thomas" <***@gmail.com> wrote in message

> > Normally you can have that.

> > .... IF [ 3 ] ... IF ... THEN .... [ drop ] THEN ....

> > But you can't do that with THENS .

> Sure you can.

> ... cond [ 3 ] ... [ drop ] thens ...

> Why you would want to something like that, I can't imagine, but you can.

Oops! You're right. As long as the extra items are gone by the time you
reach THENS then it works. And if they aren't gone for THENS they
wouldn't be gone for some individual THEN either and it wouldn't work
without THENS .

Sorry, I don't know what I was thinking.

As for why you'd want to do something like that, it would probably be
something more like

.... cond [ 0 ] ... [ figure-offset + ] ... [ another-offset + ]
literal ... thens ....

You might want to compute a literal at compile time. And as long as you
didn't try to use it while an orig or dest or do-sys was sitting on it,
and as long as you got rid of it before you use any orig or dest or
do-sys that's under it, that's all fine.

Again, I don't know why I was thinking there could be something to get
in the way of THENS that wouldn't get in the way of one particular THEN
. Some sort of bad thinking.

[Elizabeth D Rather]

J Thomas wrote:
> Elizabeth D Rather wrote:
...
> It occurs to me that you are somebody who sells Forth systems, and you
> benefit when your Forth systems are the best. I am somebody who wants
> to write portable code, to write it both for multiple systems and on
> multiple systems. Our needs and interests might not be perfectly
> aligned.

We are also frequently called in to work on applications written for
someone else's Forth, and using our own tools to write demanding
applications which often need to be ported to other platforms. Even
within our own product family, we have one SwiftX target (8051) which
doesn't have a conventional return stack (for performance reasons). We
have a very keen appreciation of portability issues.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. [phone]
5155 W. Rosecrans Ave. #1018 Fax: [phone]
Hawthorne, CA 90250
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Brad Eckert]

Elizabeth D Rather wrote:
> Even
> within our own product family, we have one SwiftX target (8051) which
> doesn't have a conventional return stack (for performance reasons). We
> have a very keen appreciation of portability issues.
>
Does this (or any of your SwiftX Forths) have a return stack that:
a) isn't one cell wide or
b) doesn't hold return addresses

In other words, something that would break on
: FOO error? if r> drop then ; \ exit from caller

Brad

[Elizabeth D Rather]

Brad Eckert wrote:
> Elizabeth D Rather wrote:
>> Even
>> within our own product family, we have one SwiftX target (8051) which
>> doesn't have a conventional return stack (for performance reasons). We
>> have a very keen appreciation of portability issues.
>>
> Does this (or any of your SwiftX Forths) have a return stack that:
> a) isn't one cell wide or
> b) doesn't hold return addresses
>
> In other words, something that would break on
> : FOO error? if r> drop then ; \ exit from caller

From the SwiftX/8051 manual:

"The 8051’s subroutine stack, instead of Forth’s return stack, is used
by target primitives to contain return addresses. If you depend on the
return stack to be separate from the subroutine stack, your code will
not be portable to systems which combine these stacks. Do not use the
return stack except under the specific rules given in Section 3.1.3."

In other words, they are separate stacks. The rules that are referenced
are the same cautions included in the Standard, the first of which is,
"A program shall not access values on the return stack (using R@, R>,
2R@, or 2R>) that it did not place there using >R or 2>R." Your FOO
word violates this, and will not work.

The Standard has come in for some criticism for imposing these
restrictions, but in this instance (as well as others) they've enabled
an implementation to run significantly faster than a more conventional
implementation would have, on a fairly slow processor family where
performance is important. I think that's a worthwhile tradeoff.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. [phone]
5155 W. Rosecrans Ave. #1018 Fax: [phone]
Hawthorne, CA 90250
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Stephen Pelc]

On 31 Aug 2006 07:04:43 -0700, "Brad Eckert"
<***@tinyboot.com> wrote:
>Does this (or any of your SwiftX Forths) have a return stack that:
>a) isn't one cell wide or
>b) doesn't hold return addresses
>
>In other words, something that would break on
>: FOO error? if r> drop then ; \ exit from caller

MPE's 8051 Forths and the X8632 system derived from work
by Michael Gassanenko both break. The second system is
probably obsolete as far as new developments are concerned,
but there are no signs (regrettably) of the 8051 going
away.

Stephen

--
Stephen Pelc, ***@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: [phone], fax: [phone]
web: http://www.mpeforth.com - free VFX Forth downloads

[Brad Eckert]

Stephen Pelc wrote:
> but there are no signs (regrettably) of the 8051 going
> away.
>

"Any place is within walking distance if you have enough time."
-- Steven Wright

Brad

[werty]

Your Forth ideas are fine for an Intel 8080 .
But we have more than 64 KB of RAM today .

The ARM cpu needs not a STACK nor STATE nor compile time
anything ...
A modern Primative uses special areas of RAM , cause it knows
exactly
what sizes it needs ( this saves time and makes code reentrant ).
Forcing all Prim's to use same stack shows lack of imagination .

Cpu's today are so fast , there's no need to "compile" nor be in a
compile STATE .

With todays ARM , programming will be much easier , no
need for so many people in that sector , sorry Eliz' ,
maybe you could do some other type of "sales" .

I do free programming , cause i'm wealthy , need a hobby .
__________________________________________________________


Elizabeth D Rather wrote:
> J Thomas wrote:
> > Andrew Haley wrote:
> >> J Thomas <***@gmail.com> wrote:
> >
> ...
>
> Andrew is doing a fine job with this topic, I just want to add a bit.
>
> > IMMEDIATE is adequate to a lot of tasks. If you want a word that only
> > makes sense in compile state, then you can make it with IMMEDIATE . And
> > you can use your innate good sense not to use it except in compile
> > state.
> >
> > If you want the same name to show two different behaviors, one in
> > interpret state and one in compile state, pretty much your only
> > portable choice is to write a state-smart word with all the
> > disadvantages that come with that.
>
> Agree entirely. In 35 years I've yet to have a problem with IMMEDIATE.
> In fact, I've been at a loss to understand what "mess" it is you're
> worried about.
>
> > There are two kinds of disadvantages to state-smart words. One kind
>
> > ...
> > So I want to find a way to add the features I want instead of get rid
> > of existing features. Then as the body of code that uses the old
> > features ages and code that uses the new features builds up, future
> > standards can consider those old features obsolescent and finally get
> > rid of them completely.
> >
> > What I want is increased ability to choose what to do with a word, at
> > lookup time.
> >
> > Known methods that can get this include:
> >
> > [list snipped]
> >
> > On the other side there's the question whether these capabilities are
> > worth having. In each case, it could be argued that they give
> > programmers too much power. If you give Forth programmers power,
> > they'll use it.
> >
> > They'll extend the language in new and tricky ways. A standard does
> > more than code portability, it also provides programmer portability,
> > and the more tricky code that's hard to understand, the less
> > interchangeable the programmers.
>
> You're talking about Tom Cruise-type hot-shots who have to show off.
> They probably exist in every language, and are mostly anathema to
> serious folks just trying to get work done. I object both to adding
> features that cater to the hot shots, and equally to efforts to thwart
> them that have the side-effect of hampering the ability of the serious
> professionals to get their work done.
>
> > ...
> >
> > It's possible I want things I shouldn't want.
>
> Entirely possible. It seems to me that you're being excessively
> theoretical in imagining problems where none exist on this issue.
>
> Cheers,
> Elizabeth
>
> --
> ==================================================
> Elizabeth D. Rather (US & Canada) 800-55-FORTH
> FORTH Inc. [phone]
> 5155 W. Rosecrans Ave. #1018 Fax: [phone]
> Hawthorne, CA 90250
> http://www.forth.com
>
> "Forth-based products and Services for real-time
> applications since 1973."
> ==================================================

[Albert van der Horst]

In article <***@news.supernews.com>,
Andrew Haley <***@littlepinkcloud.invalid> wrote:
<SNIP>
>
>OK. It seems to me that IMMEDIATE does this job quite well already:
>if you have a word that only makes sense in compile state (e.g. IF,
>WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
>one job to do.

IF and other control words working in interpreted state results in
this beautiful one-screener:
------------------8<----------------------------------
( CRC-MORE CRC ) CF: ?32 \ AvdH
REQUIRE BOUNDS REQUIRE NEW-IF HEX
\ Well the polynomial
EDB8,8320 CONSTANT CRC32_POLYNOMIAL

\ Auxiliary table with values for single bytes.
CREATE CRCTable
100 0 DO I 8 0 DO
DUP >R 1 RSHIFT R> 1 AND IF CRC32_POLYNOMIAL XOR THEN
LOOP , LOOP
\ For initial CRC and BUFFER COUNT pair, leave the updated CRC
: CRC-MORE BOUNDS DO DUP I C@ XOR 0FF AND CELLS CRCTable + @
SWAP 8 RSHIFT XOR LOOP ;
\ For BUFFER COUNT pair, leave the CRC .
: CRC -1 ROT ROT CRC-MORE INVERT ;
DECIMAL
------------------8<----------------------------------

This does a full zip-compatible crc as per
S" monkey.frt" GET-FILE ( alias SLURP ) CRC HEX .

Languages like c have to run a special program to generate
tables, then include those huge tables in the actual
program.
Forth generates the table during compilation.

<SNIP>

>I'm sorry, this is all too abstract for me. I guess I don't
>understand what the problems are you're trying to eliminate; in the
>absence of any examples I have no way to know. I know what I think
>should go: STATE. At the time of ANS, Forth, Inc already had
>well-established practice for a STATE-free Forth, but STATE went into
>the standard anyway.

The following is not to say that I'm font of the word STATE per se.

I hate people talking about state-free Forth, and not realising
that numbers are state smart, indicating that you can't get rid
of some form of state altogether.
The only one to get it right (from the purist point of view,
at least) is Chuck Moore.
He attaches state to each instance of usage of a word, not to the
word itself.
He has different colors for numbers whether they must be compiled or
put on the stack and different colors for words when they are to be
executed, or compiled, or defined.

>
>Andrew.


--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- like all pyramid schemes -- ultimately falters.
***@spenarnc.xs4all.nl http://home.hccnet.nl/a.w.m.van.der.horst

[John Passaniti]

Albert van der Horst wrote:
> Languages like c have to run a special program to generate
> tables, then include those huge tables in the actual
> program.
> Forth generates the table during compilation.

You're absolutely right in a general sense-- the inability of the C
compiler to execute code at compile time usually does result in using
external programs to generate tables (and code).

But you're wrong in this specific case. I looked at your Forth code and
saw that the inner loop wasn't doing anything terribly exotic--
certainly nothing that any decent C compiler couldn't evaluate fully at
compile time by relying on constant folding and replacing the IF/THEN
with equivalent arithmetic:


#define POLYNOMIAL 0xEDB88320uL

#define STEP(x) (((x) >> 1) ^ \
(POLYNOMIAL & (0xffffffffuL * ((x) & 1))))
#define DO_1(x) STEP(STEP(STEP(STEP(STEP(STEP(STEP(STEP(x))))))))
#define DO_2(x) DO_1(x), DO_1((x)+1)
#define DO_4(x) DO_2(x), DO_2((x)+2)
#define DO_8(x) DO_4(x), DO_4((x)+4)
#define DO_16(x) DO_8(x), DO_8((x)+8)
#define DO_32(x) DO_16(x), DO_16((x)+16)
#define DO_64(x) DO_32(x), DO_32((x)+32)
#define DO_128(x) DO_64(x), DO_64((x)+64)
#define DO_256(x) DO_128(x), DO_128((x)+128)
#define CRCTABLE DO_256(0)

unsigned int crcTable[256] = { CRCTABLE };

This compiles cleanly under GCC and generates the same table as your
Forth code. I didn't bother to write the equivalent CRC-MORE routine.

Is this as elegant as Forth? Clearly no. Is it as readable? Nope.
Does it do what you said couldn't be done? Yep.

Enjoy.

[Andreas Kochenburger]

On Wed, 30 Aug 2006 00:38:46 GMT, John Passaniti
<***@JapanIsShinto.com> wrote:
>> Languages like c have to run a special program to generate
>> tables, then include those huge tables in the actual
>> program.
>> Forth generates the table during compilation.

>You're absolutely right in a general sense-- the inability of the C
>compiler to execute code at compile time usually does result in using
>external programs to generate tables (and code).

This criticism applies to the CPP preprocessor whose macro
capabilities are _very_ limited. But you are free to use another macro
processor like f.ex. m4, which can generate table definitions during
compile time.


Andreas
-------
1 + 1 = 3, for large values of 1.

[John Passaniti]

Andreas Kochenburger wrote:
> This criticism applies to the CPP preprocessor whose macro
> capabilities are _very_ limited. But you are free to use another macro
> processor like f.ex. m4, which can generate table definitions during
> compile time.

Well, yeah. Although when you do that, you're no longer programming in
C, but in a mix of languages. And while I don't have a problem mixing
multiple languages in a single project, there is for me at least a
certain appeal in seeing just how far one can push the C preprocessor
and compiler optimizations such as constant folding.

[J Thomas]

John Passaniti wrote:

> Well, yeah. Although when you do that, you're no longer programming in
> C, but in a mix of languages. And while I don't have a problem mixing
> multiple languages in a single project, there is for me at least a
> certain appeal in seeing just how far one can push the C preprocessor
> and compiler optimizations such as constant folding.

This is something I've noticed before with you.

I respect your abilities. Your desire to think out all the implications
and see how much you can get done with a particular set of tools is
admirable.

And yet, that same combination of mindset and skills may leave you as
not the best one to decide what tools are best for other people. When
you push things as far as they can go, you might wind up writing code
that lesser programmers have trouble maintaining.

I expect that in practice you're careful about that. There's a skill to
writing simply, even sometimes writing a lot more code that will be
slower to read but easier to understand.

Still, given your special background and skills, what's easy and
convenient for you may be less easy and convenient for others, and you
may not always know what tools would be used most enthusiastically by
others.

[John Passaniti]

J Thomas wrote:
> This is something I've noticed before with you.

Oh, this should be good. After all, you know me *so* well. How many
years have we worked together? Oh, that's right. Zero.

> And yet, that same combination of mindset and skills may leave you as
> not the best one to decide what tools are best for other people. When
> you push things as far as they can go, you might wind up writing code
> that lesser programmers have trouble maintaining.

Please support your statement. Point to *any* message I have *ever*
written *anywhere* where I suggest I consider myself the best to decide
what tools are best for others. If there is one consistent message in
everything I've written in comp.lang.forth over the years is that one
should pick the right tool for the job. I know only what is best for me
and the projects I work on. I have *never* presumed to know what is
best for others.

> I expect that in practice you're careful about that. There's a skill to
> writing simply, even sometimes writing a lot more code that will be
> slower to read but easier to understand.

Let's cut through your nonsense and break it down:

1. Albert posts code, makes a claim, and provides an example.

2. I agree with his general claim, but find his example is poor.

3. I spend an enjoyable ten minutes solving a silly puzzle, testing it
on three different compilers, and posting my results, taking care to
qualify what I did as not as elegant and not as readable. I amuse
myself with the perversity of creating a macro that expands to 4.8
megabytes of code from around 640 bytes of source code.

And now the best part:

4. You read my message, ignore my qualifications on the results, and
come up with a grandiose statement about the quality of the code I pass
on to "lesser" programmers. Such amazing insight you have-- kind of
reminds me of Senator Bill Frist diagnosing Terri Schiavo not by
actually interacting with her, but by watching a videotape.

> Still, given your special background and skills, what's easy and
> convenient for you may be less easy and convenient for others, and you
> may not always know what tools would be used most enthusiastically by
> others.

I find it ironic that you direct your sage advice to me-- someone who
relentlessly seeks to find the best tool for the job-- in a newsgroup
where the prevailing assumption is that Forth is the only tool anyone needs.

[J Thomas]

John Passaniti wrote:

> > And yet, that same combination of mindset and skills may leave you as
> > not the best one to decide what tools are best for other people. When
> > you push things as far as they can go, you might wind up writing code
> > that lesser programmers have trouble maintaining.

> Please support your statement. Point to *any* message I have *ever*
> written *anywhere* where I suggest I consider myself the best to decide
> what tools are best for others.

I didn't claim that you claimed you were the best to decide what tools
others should use.

I claimed that you maynot be the best to decide what tools other people
should use.

> 4. You read my message, ignore my qualifications on the results, and
> come up with a grandiose statement about the quality of the code I pass
> on to "lesser" programmers. Such amazing insight you have-- kind of
> reminds me of Senator Bill Frist diagnosing Terri Schiavo not by
> actually interacting with her, but by watching a videotape.

I didn't say that you write code that others can't maintain. I wanted
to give you the benefit of the doubt and assume that you're careful
about that.

> > Still, given your special background and skills, what's easy and
> > convenient for you may be less easy and convenient for others, and you
> > may not always know what tools would be used most enthusiastically by
> > others.

> I find it ironic that you direct your sage advice to me-- someone who
> relentlessly seeks to find the best tool for the job-- in a newsgroup
> where the prevailing assumption is that Forth is the only tool anyone needs.

I doubt that the majority of people here have said that Forth is the
only tool anyone needs. But if somebody here did say that, it might
possibly be that Forth is usually the best tool for him personally.

Anyway, I got value from your demonstration. Albert implied he was
doing something that couldn't be done in C. You showed that a C guru
could do it with a little bit of C code.

The lesson I take from this -- in the current context -- is that for
things like standards the important thing isn't what the most expert
people can figure out ways to do. What's real important is what
signicant numbers of people can do fairly easily.

And that's likely to be like something else you discussed, you were
talking about unavoidable complexity that can be pushed from one place
to another but not eliminated. If we work out ways so it's easy to do
everything, it will be very easy to mess up. The minimum complexity
can't just go away, and anything complex leads to errors.

So if we're going to get important improvements to Forth, they might be
things that let people test easier -- errors you find quickly aren't as
bad as persistent errors. And also there might be tools that
successfully hide complexity. Where the complex stuff can be hidden
inside and it doesn't affect anything you need to know about. These
days things like that need to be portable or mostly nobody will notice
them. I can tell you about the wonderful things I do on my Forth
system, and if it depends on some unusual nonstandard thing my system
does then it might as well be Werty talking.

So other things equal, I'm better off when standard behaviors have a
lot of orthoganality, when they don't affect each other in complex
ways. So they can accept changes without needing a lot of other things
to change too.

[John Passaniti]

J Thomas wrote:
> I didn't claim that you claimed you were the best to decide what tools
> others should use.
>
> I claimed that you maynot be the best to decide what tools other people
> should use.

Question: When you play these kind of dumb word-games, is it because
you think you're being clever, or is it because you are trying to
retroactively reduce the pompousness of your message to me?

I find it fascinating that you would offer as a concern my deciding for
others what tools they should use. I have no idea where that concern
comes from-- it can't be from anything I've written over the years in
comp.lang.forth, and you've never worked with me, so it can't come from
experience of how I work with others.

> Anyway, I got value from your demonstration. Albert implied he was
> doing something that couldn't be done in C. You showed that a C guru
> could do it with a little bit of C code.

I don't consider myself a "C guru." The code I offered was a
straight-forward translation of the same algorithm in Forth. If there
is anyone here who does understand the original Forth code but doesn't
understand my C translation, then they don't understand the C language.

[J Thomas]

John Passaniti wrote:
> J Thomas wrote:
> > I didn't claim that you claimed you were the best to decide what tools
> > others should use.
> >
> > I claimed that you maynot be the best to decide what tools other people
> > should use.
>
> Question: When you play these kind of dumb word-games, is it because
> you think you're being clever, or is it because you are trying to
> retroactively reduce the pompousness of your message to me?

I'm not playing word games. You assumed meanings I didn't intend as as
far as I can see that I didn't say.

Here's my main point. People who are particularly good at getting
results from bad systems, tend not to notice how bad those systems are.
It might take days to find a workaround for a particular problem, but
once they become experts who can do the workaround almost by instinct
they tend to forget about it. In a sense they get absorbed by the
system, they no longer notice that what the system does isn't quite
what they would have wanted. They may not be the best people to
describe the flaws in the system.

http://www.generalsystemantics.com/Systemantics.htm

Zhuangzi said that when people judge pigs they pinch the shins where
the fat is thin, not the belly where it's thick. They find out more
that way. If you're looking for improvements, ask the people who have
complaints more than the people who're satisfied.

You personally say you make a point of learning multiple languages.
That gives you a sense of one language's weaknesses compared to
another's. That gives you some protection from this flaw. I noticed
that when we were discussing CVS, Elizabeth Rather said that she's used
CVS repeatedly and there are no problems with it, while you pointed out
some of the ways that Subversion fixed CVS's many problems. Would you
have listed CVS's flaws before Subversion was here to do better? Maybe
so. Having the better alternative already present makes it easier.

> > Anyway, I got value from your demonstration. Albert implied he was
> > doing something that couldn't be done in C. You showed that a C guru
> > could do it with a little bit of C code.
>
> I don't consider myself a "C guru."

Say this problem was assigned to a bunch of random C programmers.
There's a tradition that C programmers often write compiled code to do
initialisation. How many of them would notice that they could do this
with elegant constant-folding, instead? 1 in 10? 1 in 100? I consider
you a C guru. No implication intended that you're only a C guru.

[Jean-François Michaud]

John Passaniti wrote:
> J Thomas wrote:
> > I didn't claim that you claimed you were the best to decide what tools
> > others should use.
> >
> > I claimed that you maynot be the best to decide what tools other people
> > should use.
>
> Question: When you play these kind of dumb word-games, is it because
> you think you're being clever, or is it because you are trying to
> retroactively reduce the pompousness of your message to me?
>
> I find it fascinating that you would offer as a concern my deciding for
> others what tools they should use. I have no idea where that concern
> comes from-- it can't be from anything I've written over the years in
> comp.lang.forth, and you've never worked with me, so it can't come from
> experience of how I work with others.
>
> > Anyway, I got value from your demonstration. Albert implied he was
> > doing something that couldn't be done in C. You showed that a C guru
> > could do it with a little bit of C code.
>
> I don't consider myself a "C guru." The code I offered was a
> straight-forward translation of the same algorithm in Forth. If there
> is anyone here who does understand the original Forth code but doesn't
> understand my C translation, then they don't understand the C language.

Knock it off already. Both of you are being asslike.

Regards
Jean-Francois Michaud

[Albert van der Horst]

In article <***@h48g2000cwc.googlegroups.com>,
J Thomas <***@gmail.com> wrote:
>
>John Passaniti wrote:
<SNIP>
>
>> > Anyway, I got value from your demonstration. Albert implied he was
>> > doing something that couldn't be done in C. You showed that a C guru
>> > could do it with a little bit of C code.
>>
>> I don't consider myself a "C guru."
>
>Say this problem was assigned to a bunch of random C programmers.
>There's a tradition that C programmers often write compiled code to do
>initialisation. How many of them would notice that they could do this
>with elegant constant-folding, instead? 1 in 10? 1 in 100? I consider
>you a C guru. No implication intended that you're only a C guru.

It was not John's idea to not use a compiled program.
It was the challenge presented by my statement that such was needed
in C.
It takes a dan-grade C-programmer to take up that challenge
and produce that kind of code instantly for sure.

Groetjes Albert.
--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- like all pyramid schemes -- ultimately falters.
***@spenarnc.xs4all.nl http://home.hccnet.nl/a.w.m.van.der.horst

[Dave Thompson]

On Wed, 30 Aug 2006 00:38:46 GMT, John Passaniti
<***@JapanIsShinto.com> wrote:

<OT: C>

> You're absolutely right in a general sense-- the inability of the C
> compiler to execute code at compile time usually does result in using
> external programs to generate tables (and code).
>
> But you're wrong in this specific case. I looked at your Forth code and
> saw that the inner loop wasn't doing anything terribly exotic--
> certainly nothing that any decent C compiler couldn't evaluate fully at
> compile time by relying on constant folding and replacing the IF/THEN
> with equivalent arithmetic:
>
>
> #define POLYNOMIAL 0xEDB88320uL
>
> #define STEP(x) (((x) >> 1) ^ \
> (POLYNOMIAL & (0xffffffffuL * ((x) & 1))))

Or a bit simpler (and IMO clearer)
#define STEP(x) ((x) >> 1) ^ ( POLYNOMIAL * ((x) & 1) )
Or closer to the original
#define STEP(x) ((x) >> 1) ^ ( (x) & 1 ? POLYNOMIAL : 0 )

> #define DO_1(x) STEP(STEP(STEP(STEP(STEP(STEP(STEP(STEP(x))))))))
> #define DO_2(x) DO_1(x), DO_1((x)+1)
> #define DO_4(x) DO_2(x), DO_2((x)+2)
> #define DO_8(x) DO_4(x), DO_4((x)+4)
> #define DO_16(x) DO_8(x), DO_8((x)+8)
> #define DO_32(x) DO_16(x), DO_16((x)+16)
> #define DO_64(x) DO_32(x), DO_32((x)+32)
> #define DO_128(x) DO_64(x), DO_64((x)+64)
> #define DO_256(x) DO_128(x), DO_128((x)+128)
> #define CRCTABLE DO_256(0)
>
> unsigned int crcTable[256] = { CRCTABLE };
>
unsigned long. C allows int to be only 16 bits (but GCC doesn't).

> This compiles cleanly under GCC and generates the same table as your
> Forth code. I didn't bother to write the equivalent CRC-MORE routine.
>
And I haven't bothered to figure out whether the macro-expanded source
fits under the minimum implementation limits and thus is guaranteed
portable even to non-decent C compilers. (Hmm - indecent? <G>)
At the least you can expect it will either work or give a compile-time
error which you can recognize as meaning 'upgrade your compiler'.

> Is this as elegant as Forth? Clearly no. Is it as readable? Nope.
> Does it do what you said couldn't be done? Yep.
>
> Enjoy.

I have actually used this seriously for parity and popcount tables. I
don't recall doing so for CRC but it's the same in principle, I just
never happened to be the one writing that (piece of) code.


- David.Thompson1 at worldnet.att.net

[Mark W. Humphries]

Albert van der Horst wrote:
> In article <***@news.supernews.com>,
> Andrew Haley <***@littlepinkcloud.invalid> wrote:
> I hate people talking about state-free Forth, and not realising
> that numbers are state smart, indicating that you can't get rid
> of some form of state altogether.
[snip]

Often what is meant by 'state-free Forth' is 'STATE-free forth'. The
implication being that you can structure a Forth interpreter/compiler
so that there's no need for STATE-smart words that access a traditional
STATE variable. I assume STATE-free Forths are increasingly common
nowadays.

Chuck Moore's approach seems to address various Forth states, not just
interpretation/compilation by using what Jeff Raskin refers to as
quasi-modes, which are sort of a fleeting state change, similar in
concept to holding down a shift key while typing to get uppercase
instead of using the caps-lock key.

Cheers,
Mark Humphries
Manila, Philippines

[Elizabeth D Rather]

Mark W. Humphries wrote:
> Albert van der Horst wrote:
>> In article <***@news.supernews.com>,
>> Andrew Haley <***@littlepinkcloud.invalid> wrote:
>> I hate people talking about state-free Forth, and not realising
>> that numbers are state smart, indicating that you can't get rid
>> of some form of state altogether.
> [snip]
>
> Often what is meant by 'state-free Forth' is 'STATE-free forth'. The
> implication being that you can structure a Forth interpreter/compiler
> so that there's no need for STATE-smart words that access a traditional
> STATE variable. I assume STATE-free Forths are increasingly common
> nowadays.

Yes, it appears our conversations are case-sensitive ;-) I'm sure
Andrew was, indeed, meaning STATE-free. Certainly polyFORTH, which
Andrew was referring to, had both interpret and compile states, even
though it didn't require or use a variable called STATE.

However, since passage of Forth-94, any STATE-free Forths are
non-standard. STATE was mandated to protect common usage, as part of a
compromise to remove most state-smartness from the standard while
preserving the ability of programmers to make STATE-smart words if they
wish.

> Chuck Moore's approach seems to address various Forth states, not just
> interpretation/compilation by using what Jeff Raskin refers to as
> quasi-modes, which are sort of a fleeting state change, similar in
> concept to holding down a shift key while typing to get uppercase
> instead of using the caps-lock key.

Indeed.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. [phone]
5155 W. Rosecrans Ave. #1018 Fax: [phone]
Hawthorne, CA 90250
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Andrew Haley]

Albert van der Horst <***@spenarnc.xs4all.nl> wrote:
> In article <***@news.supernews.com>,
> Andrew Haley <***@littlepinkcloud.invalid> wrote:
> <SNIP>
>>
>>OK. It seems to me that IMMEDIATE does this job quite well already:
>>if you have a word that only makes sense in compile state (e.g. IF,
>>WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
>>one job to do.

> IF and other control words working in interpreted state results in
> this beautiful one-screener:

Yeah, but you don't need STATE-smart control words to generate a CRC
table. It makes almost zero practical difference whether you have
them or not.

> Languages like c have to run a special program to generate
> tables, then include those huge tables in the actual
> program.
> Forth generates the table during compilation.

And you can still do that. You just have to put it in a colon
definition. And how much of a problem is that if you're writing an
application? Zero. (Epsilon? :-)

> <SNIP>

>>I'm sorry, this is all too abstract for me. I guess I don't
>>understand what the problems are you're trying to eliminate; in the
>>absence of any examples I have no way to know. I know what I think
>>should go: STATE. At the time of ANS, Forth, Inc already had
>>well-established practice for a STATE-free Forth, but STATE went
>>into the standard anyway.

> The following is not to say that I'm font of the word STATE per se.

> I hate people talking about state-free Forth, and not realising
> that numbers are state smart, indicating that you can't get rid
> of some form of state altogether.

Numbers aren't STATE-smart. (Note capitals.)

Andrew.

[Albert van der Horst]

In article <***@news.supernews.com>,
Andrew Haley <***@littlepinkcloud.invalid> wrote:
>Albert van der Horst <***@spenarnc.xs4all.nl> wrote:
>> In article <***@news.supernews.com>,
>> Andrew Haley <***@littlepinkcloud.invalid> wrote:
>> <SNIP>
>>>
>>>OK. It seems to me that IMMEDIATE does this job quite well already:
>>>if you have a word that only makes sense in compile state (e.g. IF,
>>>WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
>>>one job to do.
>
>> IF and other control words working in interpreted state results in
>> this beautiful one-screener:
>
>Yeah, but you don't need STATE-smart control words to generate a CRC
>table. It makes almost zero practical difference whether you have
>them or not.
>
>> Languages like c have to run a special program to generate
>> tables, then include those huge tables in the actual
>> program.
>> Forth generates the table during compilation.
>
>And you can still do that. You just have to put it in a colon
>definition. And how much of a problem is that if you're writing an
>application? Zero. (Epsilon? :-)

Note that I do Forth for practice and elegance.
I can't stand that a word remains in de glossary that will
never be used.

>
>> <SNIP>
>
>>>I'm sorry, this is all too abstract for me. I guess I don't
>>>understand what the problems are you're trying to eliminate; in the
>>>absence of any examples I have no way to know. I know what I think
>>>should go: STATE. At the time of ANS, Forth, Inc already had
>>>well-established practice for a STATE-free Forth, but STATE went
>>>into the standard anyway.
>
>> The following is not to say that I'm font of the word STATE per se.
>
>> I hate people talking about state-free Forth, and not realising
>> that numbers are state smart, indicating that you can't get rid
>> of some form of state altogether.
>
>Numbers aren't STATE-smart. (Note capitals.)

Okay. But if and when state is handled by STATE, state-smart numbers
become STATE-SMART.
Your statement does seem to serve no other purpose then block
some insight, particular in what Chuck Moore is doing.
I find the analogy between using shift keys (shift control alt)
for each letter, versus caps lock (that remains applicable)
particularly illuminating.

In ciforth numbers are handled by the interpreter, this brings to the
fore what is really going on. PREFIX means that 3 can be part of a
word, and still be found in the dictionary. It is then used to
interpret the remainder of the word. >IN is backed up one place, such
that (NUMBER) can interpret the 3 that is there. In this way the same
code can be used for 4 5 6 etc.

SEE 3

: 3
-1 IN +! (NUMBER) POSTPONE SDLITERAL
; IMMEDIATE PREFIX
OK
\ (NUMBER) ommitted.

SEE SDLITERAL

: SDLITERAL
DPL @
0BRANCH [ C , ] ( between ? DROP ) POSTPONE DLITERAL
BRANCH [ 8 , ] ( between LITERAL (;) ) DROP POSTPONE LITERAL
; IMMEDIATE
\ SDLITERAL is just two LITERAL 's.
OK
CRACK LITERAL

: LITERAL
STATE @
0BRANCH [ 10 , ] ( between , (;) ) 'LIT , ,
; IMMEDIATE

\ : LITERAL STATE @ IF 'LIT , , THEN ; IMMEDIATE

So in the end LITERAL is the only state smart wordt so to say.
There are more numbers: &A ^C 'DROP "AAP"
They all wind up using LITERAL that is STATE smart.

This practically confines STATE to generalized numbers that are in the
minimum search order which I think is neat and clean.

>
>Andrew.

--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- like all pyramid schemes -- ultimately falters.
***@spenarnc.xs4all.nl http://home.hccnet.nl/a.w.m.van.der.horst

[Andrew Haley]

Albert van der Horst <***@spenarnc.xs4all.nl> wrote:
> In article <***@news.supernews.com>,
> Andrew Haley <***@littlepinkcloud.invalid> wrote:
>>Albert van der Horst <***@spenarnc.xs4all.nl> wrote:
>>> In article <***@news.supernews.com>,
>>> Andrew Haley <***@littlepinkcloud.invalid> wrote:
>>>>I'm sorry, this is all too abstract for me. I guess I don't
>>>>understand what the problems are you're trying to eliminate; in the
>>>>absence of any examples I have no way to know. I know what I think
>>>>should go: STATE. At the time of ANS, Forth, Inc already had
>>>>well-established practice for a STATE-free Forth, but STATE went
>>>>into the standard anyway.
>>
>>> The following is not to say that I'm font of the word STATE per se.
>>
>>> I hate people talking about state-free Forth, and not realising
>>> that numbers are state smart, indicating that you can't get rid
>>> of some form of state altogether.
>>
>>Numbers aren't STATE-smart. (Note capitals.)

> Okay. But if and when state is handled by STATE, state-smart numbers
> become STATE-SMART.

I've read this several times, and I don't understand it at all. You
have claimed that numbers are state smart. Unfortunately, this phrase
is somewhat ambiguous, but as I understand common Forth parlance a
state-smart word is an immediate word that checks the current state
(whether interpreting or compiling) and performs different actions
accordingly. In Forth, numbers are not state-smart.

> Your statement does seem to serve no other purpose then block
> some insight, particular in what Chuck Moore is doing.

I don't understand this comment either.

> So in the end LITERAL is the only state smart wordt so to say.

I disagree. STATE-smartness is like a virus that infects everythng it
touches. Every word that calls a STATE-smart word (via POSTPONE) is
STATE-smart.

> There are more numbers: &A ^C 'DROP "AAP"
> They all wind up using LITERAL that is STATE smart.

> This practically confines STATE to generalized numbers that are in the
> minimum search order which I think is neat and clean.

Well, it's very cute. It's also rather confusing to the reader,
because they see a word, go to look in the source for its definition,
and it's not there! TCL also does stuff like this, generating
identifiers on the fly, and I hate it.

Is ^C really so much better than [ CTRL C ] LITERAL ? I know which
I'd rather debug, for sure.

Andrew.

[Julian V. Noble]

Albert van der Horst wrote:
> In article <***@news.supernews.com>,
> Andrew Haley <***@littlepinkcloud.invalid> wrote:
> <SNIP>
>> OK. It seems to me that IMMEDIATE does this job quite well already:
>> if you have a word that only makes sense in compile state (e.g. IF,
>> WHILE, etc) then what is wrong with IMMEDIATE ? IMMEDIATE has just
>> one job to do.
>
> IF and other control words working in interpreted state results in
> this beautiful one-screener:
> ------------------8<----------------------------------
> ( CRC-MORE CRC ) CF: ?32 \ AvdH
> REQUIRE BOUNDS REQUIRE NEW-IF HEX
> \ Well the polynomial
> EDB8,8320 CONSTANT CRC32_POLYNOMIAL
>
> \ Auxiliary table with values for single bytes.
> CREATE CRCTable
> 100 0 DO I 8 0 DO
> DUP >R 1 RSHIFT R> 1 AND IF CRC32_POLYNOMIAL XOR THEN
> LOOP , LOOP
> \ For initial CRC and BUFFER COUNT pair, leave the updated CRC
> : CRC-MORE BOUNDS DO DUP I C@ XOR 0FF AND CELLS CRCTable + @
> SWAP 8 RSHIFT XOR LOOP ;
> \ For BUFFER COUNT pair, leave the CRC .
> : CRC -1 ROT ROT CRC-MORE INVERT ;
> DECIMAL
> ------------------8<----------------------------------
>
> This does a full zip-compatible crc as per
> S" monkey.frt" GET-FILE ( alias SLURP ) CRC HEX .
>
> Languages like c have to run a special program to generate
> tables, then include those huge tables in the actual
> program.
> Forth generates the table during compilation.
>
> <SNIP>
>
[ DELETED ]
>
> The following is not to say that I'm font of the word STATE per se.
>
> I hate people talking about state-free Forth, and not realising
> that numbers are state smart, indicating that you can't get rid
> of some form of state altogether.
> The only one to get it right (from the purist point of view,
> at least) is Chuck Moore.
> He attaches state to each instance of usage of a word, not to the
> word itself.
> He has different colors for numbers whether they must be compiled or
> put on the stack and different colors for words when they are to be
> executed, or compiled, or defined.
>

I'm sorry, I don't understand your example entirely. I have
created tables at compile-time (sometimes rather large tables)
without having interpretable DO's or IF's. I simply give
the executable a name. In your case I would have said something
like

: CRC_entry ( x -- x') \ make CRC table entry
8 0 DO
DUP >R 1 RSHIFT R> 1 AND
IF CRC32_POLYNOMIAL XOR THEN
LOOP
;

: fill_CRC ( adr --)
100 0 DO I ( -- adr x )
CRC_entry ( -- adr x' ) OVER ,
CELL+ ( -- adr' = adr + CELL)
LOOP DROP
;

CREATE CRCTable CRCTable fill_CRC

This even has the advantage that if you made CRC32_POLYNOMIAL
a VALUE you could make more than one table (assuming there is
more than one possible CRC32_POLYNOMIAL).

It's not quite the gem your code would be, but it complies
with ANS94, requires no state-smartness, and is factored and
commented.

BTW, I do agree that STATE and the possibility of state-smartness
are both useful and should be retained; however I also agree they
should be used sparingly and with warnings.

--
Julian V. Noble
Professor Emeritus of Physics
University of Virginia

[J Thomas]

Julian V. Noble wrote:
> Albert van der Horst wrote:

> > This does a full zip-compatible crc as per
> > S" monkey.frt" GET-FILE ( alias SLURP ) CRC HEX .
>
> I'm sorry, I don't understand your example entirely.

It looked to me like you understood it well enough to rewrite it.

> I have
> created tables at compile-time (sometimes rather large tables)
> without having interpretable DO's or IF's.

....

> It's not quite the gem your code would be, but it complies
> with ANS94, requires no state-smartness, and is factored and
> commented.

Sure. He was writing in a block, and presumably his comments would have
been in a shadow block. He doesn't require state-smartness, he only
requires that DO LOOPs somehow work outside definitions. A standard
system is allowed to provide that, but code which depends on it is not
particularly portable.

Certainly we can get by without what he's doing. For that matter we
could get by without interpret state altogether. Good Forth programmers
could get their work done just fine entirely with batch programming.
But it's a convenience to have an interpreter. Maybe it would be
convenient to have Albert's intepreter.

> BTW, I do agree that STATE and the possibility of state-smartness
> are both useful and should be retained; however I also agree they
> should be used sparingly and with warnings.

The only reason one might do STATE @ is to do something different
depending on the result. If you don't do anything STATE-smart then you
have no need for STATE whatsoever.

The standard provides no access at all to the Forth interpreter. The
interpreter is a black box, you give it standard inputs and you can
expect standard outputs, and that's it. So the only STATE-smartness you
can do is run-time STATE-smart. Some (but not all) of the problems with
STATE-smart come because people try to use run-time state-smart when it
doesn't work.

Extending the interpreter's FIND would make various things easier. If
SYSTEM-FIND was deferred that would clear up more. You could do various
sorts of preprocessing.

: MY-FIND
do-stuff-before-lookup FIND do-stuff-after-lookup ;
' MY-FIND is SYSTEM-FIND

Of course we don't need to have any access to the interpreter. We don't
even need to have an interpreter. It's just for Forth developer
convenience.

[Stephen Pelc]

On 30 Aug 2006 07:27:37 -0700, "J Thomas" <***@gmail.com> wrote:

IMMEDIATE is not in itself the problem. It's what you choose
to do with such words. In particular, delaying execution of
words which inspect STATE can lead to trouble when STATE is not
what was intended.

Anton is quite right that STATE-smart words are potential trouble.
Despite that, they are useful for supplying syntactic sugar to
non-guru programmers. In our world, that's the majority of
programmers.

>Of course we don't need to have any access to the interpreter. We don't
>even need to have an interpreter. It's just for Forth developer
>convenience.

Not in my world! It's used by MPE in an XML parser, server-side
scripting, and for run-time configuration.

--
Stephen Pelc, ***@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: [phone], fax: [phone]
web: http://www.mpeforth.com - free VFX Forth downloads

[Andrew Haley]

Stephen Pelc <***@mpeforth.com> wrote:
> On 30 Aug 2006 07:27:37 -0700, "J Thomas" <***@gmail.com> wrote:

> IMMEDIATE is not in itself the problem. It's what you choose
> to do with such words. In particular, delaying execution of
> words which inspect STATE can lead to trouble when STATE is not
> what was intended.

> Anton is quite right that STATE-smart words are potential trouble.
> Despite that, they are useful for supplying syntactic sugar to
> non-guru programmers. In our world, that's the majority of
> programmers.

Ah, it's the old "what about the ordinary programmers?" refrain. The
trouble with this kind of argument is that it doesn't take very long
before people want to start composing new words from the set of words
they already have, and with STATE-smart words this can get very hard,
very quickly. It's not just guru programmers!

We'd all be better off without it, gurus and mortals alike.

Andrew.

[J Thomas]

Andrew Haley wrote:
> Stephen Pelc <***@mpeforth.com> wrote:

> > IMMEDIATE is not in itself the problem. It's what you choose
> > to do with such words. In particular, delaying execution of
> > words which inspect STATE can lead to trouble when STATE is not
> > what was intended.

....

> Ah, it's the old "what about the ordinary programmers?" refrain. The
> trouble with this kind of argument is that it doesn't take very long
> before people want to start composing new words from the set of words
> they already have, and with STATE-smart words this can get very hard,
> very quickly. It's not just guru programmers!

I want to point out once more that Stephen's problem is due to an
implementation flaw.

When the word inspects STATE at runtime to decide what to do, it can
get compiled into something and then inspect STATE at the wrong time.
Any word you compile a STATE-smart word into becomes STATE-smart
itself, and if you don't know what's going on you can be unpleasantly
surprised.

But when you choose among behaviors when you find the word in the input
stream, this time-bomb is defused.

The idea that we mustn't provide tools to programmers because they
might misuse them is rather new to Forth. Usually Forth programmers
will try an approach and if they see it isn't working then they'll try
something else. And that's why we get so many people giving the advice
to avoid STATE-smart words -- they've tried it and gotten burned.
IMMEDIATE was a quick hack that works adequately if you use it
carefully. COMPILE-ONLY (and INTERPRET-ONLY for some special cases)
comes closer to doing what's usually needed.

Some other portable tools might be occasionally extremely useful. But
there is a rule that I've never seen explicitly stated, that goes
"Never add anything to a Forth standard until you can prove you can't
do without it." This rule has not always been followed. ;)

[Stephen Pelc]

On 31 Aug 2006 06:44:33 -0700, "J Thomas" <***@gmail.com> wrote:

>I want to point out once more that Stephen's problem is due to an
>implementation flaw.

No it's not - it's a usage problem that I was pointing out. As
always in Forth, programmers are responsible for the consequences
of their actions.

Stephen


--
Stephen Pelc, ***@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: [phone], fax: [phone]
web: http://www.mpeforth.com - free VFX Forth downloads

[J Thomas]

Stephen Pelc wrote:
> On 31 Aug 2006 06:44:33 -0700, "J Thomas" <***@gmail.com> wrote:
>
> >I want to point out once more that Stephen's problem is due to an
> >implementation flaw.
>
> No it's not - it's a usage problem that I was pointing out. As
> always in Forth, programmers are responsible for the consequences
> of their actions.

While you are completely correct about that, still there is no portable
method to do that correctly. The only available method is this broken
method. It mostly works. Be careful not to use it in the ways it's
broken and you'll do OK.

Provide a method that actually works correctly and people are less
likely to use the methods that have that risk of mis-use.

Every word that gets labeled IMMEDIATE which is really compile-only is
an invitation to execute while interpreting. sometimes it might work
that way, sometimes it won't. Make it really compile-only and the
temptation vanishes.

[Coos Haak]

Op 31 Aug 2006 16:31:38 -0700 schreef J Thomas:

> Stephen Pelc wrote:
>> On 31 Aug 2006 06:44:33 -0700, "J Thomas" <***@gmail.com> wrote:
>>
>>>I want to point out once more that Stephen's problem is due to an
>>>implementation flaw.
>>
>> No it's not - it's a usage problem that I was pointing out. As
>> always in Forth, programmers are responsible for the consequences
>> of their actions.
>
> While you are completely correct about that, still there is no portable
> method to do that correctly. The only available method is this broken
> method. It mostly works. Be careful not to use it in the ways it's
> broken and you'll do OK.
>
> Provide a method that actually works correctly and people are less
> likely to use the methods that have that risk of mis-use.
>
> Every word that gets labeled IMMEDIATE which is really compile-only is
> an invitation to execute while interpreting. sometimes it might work
> that way, sometimes it won't. Make it really compile-only and the
> temptation vanishes.

What about [IF] [ELSE] [THEN]
They are IMMEDIATE and in one implementation of myself also EXECUTE-ONLY,
not compilable nor POSTPONEable.
My interpreter for this system THROWs if it sees the COMPILE-ONLY or
EXECUTE-ONLY flag when it is encountered with a non-matching STATE.

--
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html

[J Thomas]

Coos Haak wrote:
> Op 31 Aug 2006 16:31:38 -0700 schreef J Thomas:

> > Provide a method that actually works correctly and people are less
> > likely to use the methods that have that risk of mis-use.
> >
> > Every word that gets labeled IMMEDIATE which is really compile-only is
> > an invitation to execute while interpreting. sometimes it might work
> > that way, sometimes it won't. Make it really compile-only and the
> > temptation vanishes.
>
> What about [IF] [ELSE] [THEN]
> They are IMMEDIATE and in one implementation of myself also EXECUTE-ONLY,
> not compilable nor POSTPONEable.
> My interpreter for this system THROWs if it sees the COMPILE-ONLY or
> EXECUTE-ONLY flag when it is encountered with a non-matching STATE.

So code which uses your system doesn't suffer the problems that
state-smart words on Stephen's systems do.

But the only method that standard programs can use is the buggy one,
there's no standard way to do what you do -- even in an extension
wordset. And the response I'm hearing is "Don't try to get that
functionality, because the only method we give you to to achieve it has
bad side-effects."

[Stephen Pelc]

On 31 Aug 2006 17:24:27 -0700, "J Thomas" <***@gmail.com> wrote:

>But the only method that standard programs can use is the buggy one,
>there's no standard way to do what you do -- even in an extension
>wordset. And the response I'm hearing is "Don't try to get that
>functionality, because the only method we give you to to achieve it has
>bad side-effects."

There's the nub. From a standards point of view we can:
1) encapsulate separate definitions of compilation and
interpretation behaviour,
2) encapsulate existing practice with ambiguous conditions
leading to "here there be dragons" warnings,
3) Go for it completely.

The problem with 1) and 3) above is the lack of common practice.
Elizabeth's cross-compiler proposal and EXECUTE-ONLY and friends
take very different approaches.

The problem with 2) is that it allows common bad practice to
continue.

Given that the we wish to continue to encourage exploration of
implementation techniques, I really do not want a future standard
to mandate behaviour that restricts implementations. At present,
the best you can probably achieve is to propose obsolete behaviour,
which will require some careful wordsmithing.

Having recently come across a system which internally has three
"xts", I believe it is too early to standardise. I'm still trying
to find a compromise that works for both cross compilers and
hosted systems. And that needs much more spare time than I've got.

Stephen

--
Stephen Pelc, ***@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: [phone], fax: [phone]
web: http://www.mpeforth.com - free VFX Forth downloads

[J Thomas]

Stephen Pelc wrote:

> From a standards point of view we can:
> 1) encapsulate separate definitions of compilation and
> interpretation behaviour,
> 2) encapsulate existing practice with ambiguous conditions
> leading to "here there be dragons" warnings,
> 3) Go for it completely.

> The problem with 1) and 3) above is the lack of common practice.
> Elizabeth's cross-compiler proposal and EXECUTE-ONLY and friends
> take very different approaches.

When I look at it carefully, I don't see that these approaches are so
different. But the cross-compiler proposal provides a whole lot of
clarity. What if we apply it to regular Forth systems instead of just
cross-compilers?

They suggest four scopes, HOST INTERPRETER COMPILER and TARGET . But if
it isn't a cross-compiler, the target is the host. So we can merge the
host and target scopes.

So when interpreting, a host/target word is interpreted and when
compiling it is compiled. That takes care of most cases.

INTERPRETER words are defining words. You can execute them when you
interpret, but you can only compile them into other interpreter words
-- if you aren't defining an interpreter word they aren't found. That's
like INTERPRET-ONLY but a little better, you can use old intepret-only
words to make new interpret-only words.

COMPILER words are compile-only, they execute while compiling and
aren't found while interpreting. I believe that analogous to
INTERPRETER words it should be possible to compile them into other
compiler words. But if the cross-compiler standard doesn't need that,
maybe I don't either.

It would be possible to implement this with vocabularies. But it should
also be possible to implement it with header bits or whatever, like
IMMEDIATE . 4 states, 2 bits. Get rid of IMMEDIATE and a
non-cross-compiler would only have 3 states.


There's a minor implementation issue. COMPILE-ONLY like IMMEDIATE is
usually defined to operate on the latest-defined word, after that
word's definition is completed. INTERPRETER and COMPILER are defined to
change state and operate on all following words until the state is
changed again. The obvious compromise is to define words that switch to
COMPILER or INTERPRETER for just the next word. That should be easy
enough to implement on existing systems that use COMPILE-ONLY and easy
to implement on systems that use the cross-compiler too. In fact, you
could implement COMPILER and INTERPRETER too. A system that does the
post-operation like COMPILE-ONLY can be retrofitted to do the
pre-operation like COMPILER . But I don't see how to do it the other
way around without looking at system details. So it looks more natural
to use the prefix operation.

> The problem with 2) is that it allows common bad practice to
> continue.

It allows no standard alternative to bad practice. If you need the good
results you have to use the buggy method. This is not the purpose of
standards.

> Given that the we wish to continue to encourage exploration of
> implementation techniques, I really do not want a future standard
> to mandate behaviour that restricts implementations. At present,
> the best you can probably achieve is to propose obsolete behaviour,
> which will require some careful wordsmithing.

I don't see that this would restrict implementation techniques much. If
we recognise that defining words are different from regular words, we
have 3 classes of words. Defining words are interpret-only and you can
only compile them into other defining words. It might be convenient to
make something else interpret-only too besides defining words.
Compiling words are compile-only, they execute while compiling. And
regular words, which in a cross-compiler can be divided between HOST
and TARGET . (There's the minor glitch that the cross-compiler standard
puts all the host compiling words and defining words into HOST .)
Defining words, compiling words, and regular words are truly different
in standard Forth, what do we lose by recognising that difference? Why
would we want to FIND compiling words while interpreting? It's simply a
fact that a word that executes a defining word is itself a defining
word.

I can imagine some systems that wouldn't fit this, but everything I can
imagine is so different that it's unlikely to fit into the same
standard as anything that *can* fit Forth 94.


As an aside, I'd like to see another header bit (or whatever) used. If
there was an easy way to signal whether a word is a parsing word. That
would cover almost all the remaining weirdities except the return stack
ones. Any word that executes a parsing word is also a parsing word. But
that's just the whipped cream with the cherry on top. It isn't
necessary.

[Stephen Pelc]

On 1 Sep 2006 04:44:32 -0700, "J Thomas" <***@gmail.com> wrote:

>It would be possible to implement this with vocabularies. But it should
>also be possible to implement it with header bits or whatever, like
>IMMEDIATE . 4 states, 2 bits. Get rid of IMMEDIATE and a
>non-cross-compiler would only have 3 states.

A vocabulary model is, in hindsight, a mistake. It reflects the
implementation techniques of two cross compilers designed 20+
years ago. There are new techniques now.

>> The problem with 2) is that it allows common bad practice to
>> continue.
>
>It allows no standard alternative to bad practice. If you need the good
>results you have to use the buggy method. This is not the purpose of
>standards.

No. It indicates that you need non-standard code to do it right.
There is absolutely no requirement that standard words must be
implementable using standard code.

Stephen

--
Stephen Pelc, ***@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: [phone], fax: [phone]
web: http://www.mpeforth.com - free VFX Forth downloads

[J Thomas]

Stephen Pelc wrote:

> >> The problem with 2) is that it allows common bad practice to
> >> continue.
> >
> >It allows no standard alternative to bad practice. If you need the good
> >results you have to use the buggy method. This is not the purpose of
> >standards.
>
> No. It indicates that you need non-standard code to do it right.
> There is absolutely no requirement that standard words must be
> implementable using standard code.

I agree that it isn't necessary to implement standard words using
standard code.

However --

If you think you need a new number output format the standard gives you
<# # #S HOLD #> . All the tools you need. Build it once, use it
anywhere.

If you think you need a new number input format the standard gives you
>NUMBER . All the tools you need. Build it once, use it anywhere.

If you think you need a new control structure, the standard gives you
IFTHEN AHEAD BEGIN UNTIL AGAIN CS-PICK CS-ROLL All the tools you need.
Build it once, use it anywhere.

If you think you need a new data structure, the standard gives you
CREATE DOES> . All the tools you need. Build it once, use it anywhere.

If you think you need a new defining word, the standard gives you :
:NONAME CREATE POSTPONE etc. All the tools you need. Build it once,
use it anywhere.

If you think you need a new compiler word, the standard gives you
IMMEDIATE . Clunk.

We can get along without a standard method. Hell, we can get along
without a standrd. But if would be better to provide a portable way to
do this just as we have portable ways to build your own number formats,
control structures, data structures, and defining words.

[GerryJ]

Stephen Pelc wrote:

>
> A vocabulary model is, in hindsight, a mistake. It reflects the
> implementation techniques of two cross compilers designed 20+
> years ago. There are new techniques now.
>

Would you care to expand on these 'new techniques'. I'm interested
because my system compiles itself using the
Host/Interpreter/Compiler/Target model implemented using wordlists and
it seems to work very well.

Gerry

[jacko]

J Thomas wrote:
> Stephen Pelc wrote:
>
> > From a standards point of view we can:
> > 1) encapsulate separate definitions of compilation and
> > interpretation behaviour,

two seperate words would be appropriate, and who gets confused? seems
like all words have compilation behaviour need, or why have words, and
the interpreter thing is all because IDE development WAS simplistic.

> > 2) encapsulate existing practice with ambiguous conditions
> > leading to "here there be dragons" warnings,

oh where's me lose my pointer arithmetic wrapper ;-) and ship me an RPN
translator for that infix notation too, along with a sink ;-)

> > 3) Go for it completely.

maybe??

> > The problem with 1) and 3) above is the lack of common practice.
> > Elizabeth's cross-compiler proposal and EXECUTE-ONLY and friends
> > take very different approaches.

and why not?

> When I look at it carefully, I don't see that these approaches are so
> different. But the cross-compiler proposal provides a whole lot of
> clarity. What if we apply it to regular Forth systems instead of just
> cross-compilers?

multiple dictionary contexts is always reasonable, given that fully
custom dictionary execution small code size is also a concern.

> They suggest four scopes, HOST INTERPRETER COMPILER and TARGET . But if
> it isn't a cross-compiler, the target is the host. So we can merge the
> host and target scopes.

why would you want to dump functionality that you are't using? ;-) and
what if i needed a definer scope? DEFINITIONS ....

> So when interpreting, a host/target word is interpreted and when
> compiling it is compiled. That takes care of most cases.

temporary compile of screen to dict for exec effect of interp?? why
would state ever need to be interp? Oh yeah, i forgot that a execution
and a compile/postpone scope would be different.

> INTERPRETER words are defining words. You can execute them when you
> interpret, but you can only compile them into other interpreter words
> -- if you aren't defining an interpreter word they aren't found. That's
> like INTERPRET-ONLY but a little better, you can use old intepret-only
> words to make new interpret-only words.

it possible anyhow if you stick with clever words. i.e no implicit
export of name without INDEX

> COMPILER words are compile-only, they execute while compiling and
> aren't found while interpreting. I believe that analogous to
> INTERPRETER words it should be possible to compile them into other
> compiler words. But if the cross-compiler standard doesn't need that,
> maybe I don't either.

these are called IMMEDIATE words are they not?

> It would be possible to implement this with vocabularies. But it should
> also be possible to implement it with header bits or whatever, like
> IMMEDIATE . 4 states, 2 bits. Get rid of IMMEDIATE and a
> non-cross-compiler would only have 3 states.
>
why not have a whole cell and set a good full standard??
>
> There's a minor implementation issue. COMPILE-ONLY like IMMEDIATE is
> usually defined to operate on the latest-defined word, after that
> word's definition is completed. INTERPRETER and COMPILER are defined to
> change state and operate on all following words until the state is
> changed again. The obvious compromise is to define words that switch to
> COMPILER or INTERPRETER for just the next word. That should be easy
> enough to implement on existing systems that use COMPILE-ONLY and easy
> to implement on systems that use the cross-compiler too. In fact, you
> could implement COMPILER and INTERPRETER too. A system that does the
> post-operation like COMPILE-ONLY can be retrofitted to do the
> pre-operation like COMPILER . But I don't see how to do it the other
> way around without looking at system details. So it looks more natural
> to use the prefix operation.

bring me my megabytes of prefix notation conversion code, bring me my
arrows of small code , ... da de da??

> > The problem with 2) is that it allows common bad practice to
> > continue.

seems like saving memory and writing copy protected code is deemed
sinnister nowadays??

> It allows no standard alternative to bad practice. If you need the good
> results you have to use the buggy method. This is not the purpose of
> standards.

all roads lead to rome via aeroplanes... ;-) and sinks...

> > Given that the we wish to continue to encourage exploration of
> > implementation techniques, I really do not want a future standard
> > to mandate behaviour that restricts implementations. At present,
> > the best you can probably achieve is to propose obsolete behaviour,
> > which will require some careful wordsmithing.

lets rewrite that sucker at great expense.. ?? ;-)

> I don't see that this would restrict implementation techniques much. If
> we recognise that defining words are different from regular words, we
> have 3 classes of words. Defining words are interpret-only and you can
> only compile them into other defining words. It might be convenient to
> make something else interpret-only too besides defining words.
> Compiling words are compile-only, they execute while compiling. And
> regular words, which in a cross-compiler can be divided between HOST
> and TARGET . (There's the minor glitch that the cross-compiler standard
> puts all the host compiling words and defining words into HOST .)
> Defining words, compiling words, and regular words are truly different
> in standard Forth, what do we lose by recognising that difference? Why
> would we want to FIND compiling words while interpreting? It's simply a
> fact that a word that executes a defining word is itself a defining
> word.

wheres the code?? reply: i don't know, i just twisted it some and the
cells got the right stuff in!! ;-)

> I can imagine some systems that wouldn't fit this, but everything I can
> imagine is so different that it's unlikely to fit into the same
> standard as anything that *can* fit Forth 94.

and what is this new languaged called anyhow??

> As an aside, I'd like to see another header bit (or whatever) used. If
> there was an easy way to signal whether a word is a parsing word. That
> would cover almost all the remaining weirdities except the return stack
> ones. Any word that executes a parsing word is also a parsing word. But
> that's just the whipped cream with the cherry on top. It isn't
> necessary.

well i say long live 83, and especially the 32 bit word versions ;-)

cheers

jacko

[Anton Ertl]

***@mpeforth.com (Stephen Pelc) writes:
>Having recently come across a system which internally has three
>"xts",

Can you elaborate on that?

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2006: http://www.complang.tuwien.ac.at/anton/euroforth2006/

[werty]

Were programmers , not salesmen , politicians , poets ,

do you know where you are ?

Your very first words " .....old ..." bogus , an attempt to lend
credability by long term acceptance ......

Better just switch jobs , learn programming so you can win
our respect .... ... or else gain repute as flamer !


On to productive .... The sttment below explains why we have so
much trouble today with software .


" " " " " Anton is quite right that STATE-smart words are potential
trouble.
> > Despite that, they are useful for supplying syntactic sugar to
> > non-guru programmers. In our world, that's the majority of
> > programmers.
" " " " " " " " "


I have tried to say this in my NewForth OpSys . Any delay is trouble
and today with high speed CPU's , it is exposing Luddites !
Anyone supporting STATE and COMPILE time words and other
delays is a Luddite , into Job security , bloating Software ..
Did i say ANSI ( as in ANS Forth ) was a Luddite ? Yes ....


Next rule . When kernels are dumb as in Preemptive Multi tasking
ya gotta expect trouble ! There is no reason why a modern Kernel
can't know dynamically what needs to run and when AND the
consequences if it DOES NOT RUN !
Linus is just lazy , he dont want to fix Linux .


NewForth dont do this Chineese fire drill ....


BTW www.littlechips.com has a ARM 920T equiv Eval board
for 1/3 the price Samsung had !
It lacks touch screen but shows a 2.5" HDD attached !!
64 MB SDRAM , USBh USBd RJ-45 Ethernet LCD controller
VGA max ... $300 ......

WOW !! NewForths home is close at hand ..






Andrew Haley wrote:
> Stephen Pelc <***@mpeforth.com> wrote:
> > On 30 Aug 2006 07:27:37 -0700, "J Thomas" <***@gmail.com> wrote:
>
> > IMMEDIATE is not in itself the problem. It's what you choose
> > to do with such words. In particular, delaying execution of
> > words which inspect STATE can lead to trouble when STATE is not
> > what was intended.
>
> > Anton is quite right that STATE-smart words are potential trouble.
> > Despite that, they are useful for supplying syntactic sugar to
> > non-guru programmers. In our world, that's the majority of
> > programmers.
>
> Ah, it's the old "what about the ordinary programmers?" refrain. The
> trouble with this kind of argument is that it doesn't take very long
> before people want to start composing new words from the set of words
> they already have, and with STATE-smart words this can get very hard,
> very quickly. It's not just guru programmers!
>
> We'd all be better off without it, gurus and mortals alike.
>
> Andrew.

[wizzywiz]

werty wrote:
> Were programmers , not salesmen , politicians , poets ,
>
> do you know where you are ?
>
> Your very first words " .....old ..." bogus , an attempt to lend
> credability by long term acceptance ......
>
> Better just switch jobs , learn programming so you can win
> our respect .... ... or else gain repute as flamer !
>
>
> On to productive .... The sttment below explains why we have so
> much trouble today with software .
>
>
> " " " " " Anton is quite right that STATE-smart words are potential
> trouble.
> > > Despite that, they are useful for supplying syntactic sugar to
> > > non-guru programmers. In our world, that's the majority of
> > > programmers.
> " " " " " " " " "
>
>
> I have tried to say this in my NewForth OpSys . Any delay is trouble
> and today with high speed CPU's , it is exposing Luddites !
> Anyone supporting STATE and COMPILE time words and other
> delays is a Luddite , into Job security , bloating Software ..
> Did i say ANSI ( as in ANS Forth ) was a Luddite ? Yes ....
>
>
> Next rule . When kernels are dumb as in Preemptive Multi tasking
> ya gotta expect trouble ! There is no reason why a modern Kernel
> can't know dynamically what needs to run and when AND the
> consequences if it DOES NOT RUN !
> Linus is just lazy , he dont want to fix Linux .
>
>
> NewForth dont do this Chineese fire drill ....
>
>
> BTW www.littlechips.com has a ARM 920T equiv Eval board
> for 1/3 the price Samsung had !
> It lacks touch screen but shows a 2.5" HDD attached !!
> 64 MB SDRAM , USBh USBd RJ-45 Ethernet LCD controller
> VGA max ... $300 ......
>
> WOW !! NewForths home is close at hand ..


You are a complete idiot.

[Jean-François Michaud]

J Thomas wrote:
> I don't have a proposal thought out, only some beginning ideas. Anybody
> is welcome to build on them for whatever purpose, or to criticise them.
>
> Usually what we need is not IMMEDIATE words or state-smart words.
> IMMEDIATE was a quick simple hack that people have been working around
> for 30 years. It's usually adequate, but not good.
>
> Any word that includes POSTPONE or executes COMPILE, (or [COMPILE] ?)
> is compile-only, and recursively any word that executes a word that
> includes POSTPONE etc it compile-only.
>
> The complications of state-smart etc don't belong in the words
> themselves. The issue doesn't come up after you execute the words. The
> issue comes up in the interpreter/compiler, when you FIND words and
> decide what to do with them.
>
> What if FIND (or its string-and-count replacement) were to give more
> information? What if it were to give
>
> -3 normal compilation behavior, do not interpret.
> -2 discard the xt always, do nothing.
> -1 execute when interpreting, compile when compiling.
> 0 not found
> 1 immediate
> 2 do not interpret, execute while compiling
> 3. execute while compiling, do not execute while interpreting, but a
> second xt is available to execute while interpreting or fetch with '
> ['] etc. The other xt can be found with a special command.
>
> Wouldn't a lot of the endless complications go away if this extra
> information was available to standard programs?
>
> Whenever you POSTPONE a word or compile COMPILE, the current word
> becomes type -3 unless it already has compiled : or :NONAME or
> postponed DOES> , and if you try to make it immediate it becomes type
> 2. The rules for that might turn out to be too complicated to use, but
> some of them might give useful warning messages without getting too
> much in the way.
>
> Wordlists don't clean up the IMMEDIATE mess, they make more messes
> attempting to solve that. IMMEDIATE affects FIND . State-smart words
> don't work because they try to fix something that ought to happen while
> parsing with something that happens at execution time -- whenever
> execution happens. An improved FIND and more sophisticated behaviors at
> that point might do it.

Hmmm. My version of FIND doesn't differentiate between IMMEDIATE words
and non IMMEDIATE words, it simply finds the word I'm asking for.
Weather the word is immediate or not has to be dealt with by the
interpreter during compilation. I don't believe it should be up to
FIND to determine weather a word is IMMEDIATE or not. FIND simply finds
a word in the dictionary and returns a reference to the word that is
found or 0 if nothing is found. My take on this is that another word
should be used, after the word is found by FIND, to THEN determine if
the word in question is IMMEDIATE or not.

The behavior that you propose seems overly complex to me.

In my system, the word I use for this is IMMEDIATE?

Regards
Jean-Francois Michaud

[J Thomas]

Jean-François Michaud wrote:

> Hmmm. My version of FIND doesn't differentiate between IMMEDIATE words
> and non IMMEDIATE words, it simply finds the word I'm asking for.

That doesn't fit the standard, but it might easily fit the new
standard. Why shouldn't FIND be factored?

> The behavior that you propose seems overly complex to me.

Maybe. What I want to do doesn't seem complex to me. I want to be able
to redefine any word and invisibly give it new behaviors, and have it
still keep its old behaviors visible. Is that too much to ask?

The 94 standard went a long way in that direction -- it didn't say
which words had nondefault compile behavior unless it was words that
had to have it. So you could make DUP immediate and hope to get all the
right results as well as whatever new results you wanted. But immediate
state-smart DUP did not work.

> In my system, the word I use for this is IMMEDIATE?

I like that factoring. So the word I would want to provide more
information would be that one instead of FIND .

[Jean-François Michaud]

J Thomas wrote:
> Jean-François Michaud wrote:
>
> > Hmmm. My version of FIND doesn't differentiate between IMMEDIATE words
> > and non IMMEDIATE words, it simply finds the word I'm asking for.
>
> That doesn't fit the standard, but it might easily fit the new
> standard. Why shouldn't FIND be factored?
>
> > The behavior that you propose seems overly complex to me.
>
> Maybe. What I want to do doesn't seem complex to me. I want to be able
> to redefine any word and invisibly give it new behaviors, and have it
> still keep its old behaviors visible. Is that too much to ask?

Hmmm. This completely depends on your definition of invisible. To whom?
To the users? I'm not certain I understand clearly what you are trying
to do:

1- You want to be able to redefine any word (which is fine; just create
a new definition with the same name and new semantics or extended
semantics).

2- You want to invisibly add new behavior (this depends on what you
mean by invisibly). This sounds to me like you want to extend the
semantics of a word without modifying the original behavior and by
keeping the same name for the extended definition.

3- You want the behavior of the redefined word to remain visible
(uncertain what you mean by that)

> The 94 standard went a long way in that direction -- it didn't say
> which words had nondefault compile behavior unless it was words that
> had to have it. So you could make DUP immediate and hope to get all the
> right results as well as whatever new results you wanted. But immediate
> state-smart DUP did not work.

Hmmm. I'm not following you too much here. But using the same name for
a definition that has different semantics than an existing definition
seems like a dangerous practice. For all intents and purposes, I think
the new semantics should be named differently so that no confusion can
occur between the old behavior and the new one. The confusion is akin
to using the same word in english to mean two different things, or the
same thing but extended. Confusion can occur.

> > In my system, the word I use for this is IMMEDIATE?
>
> I like that factoring. So the word I would want to provide more
> information would be that one instead of FIND .

Yes. To me, FIND has a very clear behavior (or should have it if
doesn't). FIND simply finds a word in the dictionary. If it finds it,
it outputs the reference to the word that was found, if it doesn't it
outputs 0.

Creating a new definition (IMMEDIATE?) that has the very specific
behavior of finding out if a particular definition is IMMEDIATE or not
seems more appropriate than having FIND output a jungle of different
magical values.

FIND first and then are we IMMEDIATE?

Regards
Jean-Francois Michaud

[J Thomas]

Jean-François Michaud wrote:
> J Thomas wrote:
> > Jean-François Michaud wrote:

> > > The behavior that you propose seems overly complex to me.

> > Maybe. What I want to do doesn't seem complex to me. I want to be able
> > to redefine any word and invisibly give it new behaviors, and have it
> > still keep its old behaviors visible. Is that too much to ask?

> Hmmm. This completely depends on your definition of invisible. To whom?
> To the users? I'm not certain I understand clearly what you are trying
> to do:

I mean, invisible to standard programs that don't already know about
its extra function.

If you were to redefine DUP so that it still did everything DUP is
required to do, and it doesn't do anything else that shows (It uses
nothing extra from the data stack or return stack and leaves nothing
extra on either stack, it doesn't change HERE . It doesn't change the
input buffer. It doesn't affect anything in allotted data space that an
unrelated standard program would know about, and it doesn't change
anything in allocated space that an unrelated standard program would
know about, it doesn't compile new code, it doesn't change STATE or the
<# buffer or PAD , it doesn't change any block that unrelated programs
use.It doesn't make the latest defined word immediate. There might be a
few more things it shouldn't do.

Here's one trivial example -- we want to count how many times a given
routine actually executes DUP .

marker duppy
variable calldup 0 calldup !

: dup ( S: x -- x x )
1 calldup +! dup ;

This will degrade performance some. And it can overflow. But it won't
affect the result of any standard program that doesn't access calldup,
unless that standard program was 1 cell from a data stack overflow, or
1 cell from a return stack overflow.

Here's another trivial example -- we want to count how many times DUP
gets compiled or interpreted from the source code. Not how many times
it gets executed, how many times it gets found by FIND or equivalent
and then interpreted or compiled.

marker duppy
variable #dups 0 #dups !

: dup ( S: x -- x x ) ( C: -- )
1 #dups +!
state @ if
postpone dup else
dup
then ; immediate

This will work fine for each DUP that gets interpreted.
It will work fine for each DUP that gets compiled.

: TEST ( S: -- )
100000 0 DO
['] DUP EXECUTE DROP
LOOP ;

This time we execute the new DUP every time instead of the old DUP and
we fail.

Of course, I didn't think out ahead of time how many times I wanted to
count each ' DUP or ['] DUP . Maybe once? Then there's a workaround.

SYNONYM OLDDUP DUP

: '
>IN @ BL WORD COUNT S" DUP" COMPARE 0= IF
1 #DUPS +! DROP ['] OLDDUP ELSE
>IN ! '
THEN ;

: [']
>IN @ BL WORD COUNT S" DUP" COMPARE 0= IF
1 #DUPS +! DROP ['] OLDDUP POSTPONE LITERAL ELSE
>IN ! POSTPONE [']
THEN ; IMMEDIATE

But this may not have exhausted the traps waiting for our little DUP .
It only got one that's supposed to always be there.

I expect we're all agreed that the statesmart DUP is not a very good
tool. But how would this trivial task be done well? I guess one way
would be to build a new compiler.

: MY-INTERPRETER
BEGIN
SKIP SOURCE NIP >IN - > WHILE
BL WORD DUP C" DUP" COMPARE 0= IF
1 #DUPS +!
THEN
FIND COMPILE-OR-EXECUTE-OR-NUMBER
REPEAT ;

This will handle one line of input, and so put another loop around it
to refresh the input buffer, and a catch around that, and it ought to
be ready. It doesn't count ' DUP but maybe it doesn't need to. It
doesn't handle EVALUATE but that doesn't look hard. Similarly INCLUDED
and LOAD .

Actually, if three words inside COMPILE-OR-EXECUTE-OR-NUMBER were
deferred, that would do a whole lot. But there's no standard for that
either.

> > > In my system, the word I use for this is IMMEDIATE?

> > I like that factoring. So the word I would want to provide more
> > information would be that one instead of FIND .

> Yes. To me, FIND has a very clear behavior (or should have it if
> doesn't). FIND simply finds a word in the dictionary. If it finds it,
> it outputs the reference to the word that was found, if it doesn't it
> outputs 0.

Good factoring. That thinking would allow SEARCH-WORDLIST to come
closer to a balanced stack output. As it is, it returns either false or
else an xt and an immediate flag. Your way, it could return either
false or an xt . We'd need the restriction that 0 is never a valid xt,
which is not a bad restriction to have.

[Jean-François Michaud]

J Thomas wrote:
> Jean-François Michaud wrote:
> > J Thomas wrote:
> > > Jean-François Michaud wrote:
>
> > > > The behavior that you propose seems overly complex to me.
>
> > > Maybe. What I want to do doesn't seem complex to me. I want to be able
> > > to redefine any word and invisibly give it new behaviors, and have it
> > > still keep its old behaviors visible. Is that too much to ask?
>
> > Hmmm. This completely depends on your definition of invisible. To whom?
> > To the users? I'm not certain I understand clearly what you are trying
> > to do:
>
> I mean, invisible to standard programs that don't already know about
> its extra function.
>
> If you were to redefine DUP so that it still did everything DUP is
> required to do, and it doesn't do anything else that shows (It uses
> nothing extra from the data stack or return stack and leaves nothing
> extra on either stack, it doesn't change HERE . It doesn't change the
> input buffer. It doesn't affect anything in allotted data space that an
> unrelated standard program would know about, and it doesn't change
> anything in allocated space that an unrelated standard program would
> know about, it doesn't compile new code, it doesn't change STATE or the
> <# buffer or PAD , it doesn't change any block that unrelated programs
> use.It doesn't make the latest defined word immediate. There might be a
> few more things it shouldn't do.

Hehehe, if it doesn't do any of these things, then it is DUP as we know
it!!!!! ;o).

> Here's one trivial example -- we want to count how many times a given
> routine actually executes DUP .
>
> marker duppy
> variable calldup 0 calldup !
>
> : dup ( S: x -- x x )
> 1 calldup +! dup ;
>
> This will degrade performance some. And it can overflow. But it won't
> affect the result of any standard program that doesn't access calldup,
> unless that standard program was 1 cell from a data stack overflow, or
> 1 cell from a return stack overflow.

But I don't think there is any way to consistantly know this! By
standard program do you mean basic functionalities that any
implementation of the standard must supply or do you mean a program
that is written using a compiler that adheres to the standard? Either
way, it is absolutely impractical to test against all this to know if
the conditions are at least met.

> Here's another trivial example -- we want to count how many times DUP
> gets compiled or interpreted from the source code. Not how many times
> it gets executed, how many times it gets found by FIND or equivalent
> and then interpreted or compiled.
>
> marker duppy
> variable #dups 0 #dups !
>
> : dup ( S: x -- x x ) ( C: -- )
> 1 #dups +!
> state @ if
> postpone dup else
> dup
> then ; immediate

Why do you insist on calling it something that already exists though?
It seems that the sensible solution would be to simply name it
something different. You are not defining dup here, you are defining a
meta behavior that dup isn't, and I believe shouldn't be, "aware" of.

> This will work fine for each DUP that gets interpreted.
> It will work fine for each DUP that gets compiled.
>
> : TEST ( S: -- )
> 100000 0 DO
> ['] DUP EXECUTE DROP
> LOOP ;
>
> This time we execute the new DUP every time instead of the old DUP and
> we fail.
>
> Of course, I didn't think out ahead of time how many times I wanted to
> count each ' DUP or ['] DUP . Maybe once? Then there's a workaround.
>
> SYNONYM OLDDUP DUP
>
> : '
> >IN @ BL WORD COUNT S" DUP" COMPARE 0= IF
> 1 #DUPS +! DROP ['] OLDDUP ELSE
> >IN ! '
> THEN ;
>
> : [']
> >IN @ BL WORD COUNT S" DUP" COMPARE 0= IF
> 1 #DUPS +! DROP ['] OLDDUP POSTPONE LITERAL ELSE
> >IN ! POSTPONE [']
> THEN ; IMMEDIATE
>
> But this may not have exhausted the traps waiting for our little DUP .
> It only got one that's supposed to always be there.

Oh goodness, it just got out of control ;o).

> I expect we're all agreed that the statesmart DUP is not a very good
> tool. But how would this trivial task be done well? I guess one way
> would be to build a new compiler.
>
> : MY-INTERPRETER
> BEGIN
> SKIP SOURCE NIP >IN - > WHILE
> BL WORD DUP C" DUP" COMPARE 0= IF
> 1 #DUPS +!
> THEN
> FIND COMPILE-OR-EXECUTE-OR-NUMBER
> REPEAT ;

Already much better I think. The text interpreter is usually pretty
simple. Adding a single word that counts instances of words is not too
dramatic. Instead of parsing words yourself, why not just reuse the
code of the interpreter exactly as is and add a word that counts words?

> This will handle one line of input, and so put another loop around it
> to refresh the input buffer, and a catch around that, and it ought to
> be ready.

???

It doesn't count ' DUP but maybe it doesn't need to. It
> doesn't handle EVALUATE but that doesn't look hard. Similarly INCLUDED
> and LOAD .
>
> Actually, if three words inside COMPILE-OR-EXECUTE-OR-NUMBER were
> deferred, that would do a whole lot. But there's no standard for that
> either.

Who says you have to be standard? Use whatever works ;o).

> > > > In my system, the word I use for this is IMMEDIATE?
>
> > > I like that factoring. So the word I would want to provide more
> > > information would be that one instead of FIND .
>
> > Yes. To me, FIND has a very clear behavior (or should have it if
> > doesn't). FIND simply finds a word in the dictionary. If it finds it,
> > it outputs the reference to the word that was found, if it doesn't it
> > outputs 0.
>
> Good factoring. That thinking would allow SEARCH-WORDLIST to come
> closer to a balanced stack output. As it is, it returns either false or
> else an xt and an immediate flag.

Yeah, I didn't like that way of doing things; it didn't make logical
sense to me, so I changed it.

Your way, it could return either
> false or an xt . We'd need the restriction that 0 is never a valid xt,
> which is not a bad restriction to have.

I personally think its better than outputting an immediate flag ;o).

Regards
Jean-Francois Michaud

[J Thomas]

Jean-François Michaud wrote:

> > If you were to redefine DUP so that it still did everything DUP is
> > required to do, and it doesn't do anything else that shows (It uses
> > nothing extra from the data stack or return stack and leaves nothing
> > extra on either stack, it doesn't change HERE . It doesn't change the
> > input buffer. It doesn't affect anything in allotted data space that an
> > unrelated standard program would know about, and it doesn't change
> > anything in allocated space that an unrelated standard program would
> > know about, it doesn't compile new code, it doesn't change STATE or the
> > <# buffer or PAD , it doesn't change any block that unrelated programs
> > use.It doesn't make the latest defined word immediate. There might be a
> > few more things it shouldn't do.

> Hehehe, if it doesn't do any of these things, then it is DUP as we know
> it!!!!! ;o).

Well, but as I showed it can affect cells in data space that standard
code won't know about unless for some reason the system gives that
standard code the addresses. It could open a file, change the contents,
and close the file again. It can do anything that doesn't show.

> > Here's one trivial example -- we want to count how many times a given
> > routine actually executes DUP .
> >
> > marker duppy
> > variable calldup 0 calldup !
> >
> > : dup ( S: x -- x x )
> > 1 calldup +! dup ;
> >
> > This will degrade performance some. And it can overflow. But it won't
> > affect the result of any standard program that doesn't access calldup,
> > unless that standard program was 1 cell from a data stack overflow, or
> > 1 cell from a return stack overflow.

> But I don't think there is any way to consistantly know this! By
> standard program do you mean basic functionalities that any
> implementation of the standard must supply or do you mean a program
> that is written using a compiler that adheres to the standard?

The standard is about the relationship between standard systems and
standard code. A standard system is required to supply the defined
behaviors of standard words, plus a little more. Standard code is
required to use those words only in standard ways. If a Forth system
implements DUP or +! or a variable or a literal in ways that visibly
affect any of the things on the list -- if executing DUP affects the
input buffer or the contents of PAD or the location of HERE or the
contents of the return stack etc in a way that standard code can
notice, then that Forth system has a nonstandard behavior.

> Either
> way, it is absolutely impractical to test against all this to know if
> the conditions are at least met.

The things we can't reliably test for ahead of time are stack
overflows. There are some rough-and-ready ways to check the actual
stack high-water marks, but and if you knew the maximum stack depth for
each primitive in the system then in theory you could calculate it for
every other word that didn't loop or recurse or use the input buffer,
but it mostly just isn't testable and isn't reliable in theory, though
it often is in practice.

> > Here's another trivial example -- we want to count how many times DUP
> > gets compiled or interpreted from the source code. Not how many times
> > it gets executed, how many times it gets found by FIND or equivalent
> > and then interpreted or compiled.
> >
> > marker duppy
> > variable #dups 0 #dups !
> >
> > : dup ( S: x -- x x ) ( C: -- )
> > 1 #dups +!
> > state @ if
> > postpone dup else
> > dup
> > then ; immediate
>
> Why do you insist on calling it something that already exists though?
> It seems that the sensible solution would be to simply name it
> something different. You are not defining dup here, you are defining a
> meta behavior that dup isn't, and I believe shouldn't be, "aware" of.

Because the idea is to first load this code and then load an
application, and all the DUPs in the application will count themselves.
It's trivial to do this in some Forth systems. While the standard was
being developed there was talk that it would be trivial to do such
things using standard code. But there were some glitches and it didn't
work out.

....

> > But this may not have exhausted the traps waiting for our little DUP .
> > It only got one that's supposed to always be there.
>
> Oh goodness, it just got out of control ;o).

Yes, complications piled on complications for something that ought to
be simple. There was a possibility that you could redefine any
nonimmediate word to be an immediate word and still have it do special
behaviors while compiling that include the default compilation
behavior, and -- depending on what it did, still have a standard system
that runs unrelated standard code in a completely standard way. There's
nothing in the standard that says whether DUP is immediate or not, it
only says what the visible interpret behavior is and what the visible
compile behavior is. But it doesn't work. With an extended IMMEDIATE?
it could work.

> > I expect we're all agreed that the statesmart DUP is not a very good
> > tool. But how would this trivial task be done well? I guess one way
> > would be to build a new compiler.
> >
> > : MY-INTERPRETER
> > BEGIN
> > SKIP SOURCE NIP >IN - > WHILE
> > BL WORD DUP C" DUP" COMPARE 0= IF
> > 1 #DUPS +!
> > THEN
> > FIND COMPILE-OR-EXECUTE-OR-NUMBER
> > REPEAT ;
>
> Already much better I think. The text interpreter is usually pretty
> simple. Adding a single word that counts instances of words is not too
> dramatic. Instead of parsing words yourself, why not just reuse the
> code of the interpreter exactly as is and add a word that counts words?

Well, but to do it you have to build your own text interpreter. That's
fine, but when you do that you immediately lose all the secret things
that the *last* text interpreter was doing. Text interpreters can nest
but they don't, ah, accrete. (Accrete? I can't think of the right
word.) Say you write an interpreter that counts DUPs. If the file you
load compiles its own text interpreter to do something else, then your
code just doesn't execute. You only get one active interpreter at a
time, all the interpreters that were running before the current one
started are just waiting for it to stop.

It would be much more useful to have deferred words in the system
interpreter. One of them just before or just after you parse a word. So
you could do something involving parsing yourself. One just before you
execute a word. One just before you compile. One just before you check
for a number.

Then you could append behaviors. You get the current contents of the
deferred word and you define a new behavior:

: MY-EXECUTE ( x*i xt -- x*j )
my-behaviors \ x*i xt
[ defer@ system-execute ] literal execute ;
' my-execute is system-execute

All the other special extensions that have been running invisibly keep
running. They don't just quit because you defined a new interpreter.

> > Actually, if three words inside COMPILE-OR-EXECUTE-OR-NUMBER were
> > deferred, that would do a whole lot. But there's no standard for that
> > either.

> Who says you have to be standard? Use whatever works ;o).

It's fine when I'm writing just for my own Forth system. I'd like to
write some portable tools that can be used on many Forth systems. While
it doesn't look plausible that many legacy Forth systems would be
extended to allow wonderful flexibility, still it just might be
possible that some method shows up that can be implemented easily and
efficiently on many systems, and get a standard way to access it.

It may be inefficient to allow those hooks into an interpreter. It
might slow down compilation speed. OK, we could have a word to call
that loads the modifiable interpreter when you want it. You don't have
to sacrifice efficiency when you don't need the flexibility.

[Mark W. Humphries]

J Thomas wrote:
[snip]
> : dup ( S: x -- x x ) ( C: -- )
> 1 #dups +!
> state @ if
> postpone dup else
> dup
> then ; immediate

Here's how I would do this in my Forth:
: directive ( -- ) immediate compile-only ;
: tally ( -- ) 1 #dups +! ;
: dup ( x -- xx ) tally dup ; interpret-only
: dup ( x -- xx ) tally postpone dup ; directive

Cheers,
Mark Humphries
Manila, Philippines

[J Thomas]

Mark W. Humphries wrote:
> J Thomas wrote:
> [snip]
> > : dup ( S: x -- x x ) ( C: -- )
> > 1 #dups +!
> > state @ if
> > postpone dup else
> > dup
> > then ; immediate
>
> Here's how I would do this in my Forth:
> : directive ( -- ) immediate compile-only ;
> : tally ( -- ) 1 #dups +! ;
> : dup ( x -- xx ) tally dup ; interpret-only
> : dup ( x -- xx ) tally postpone dup ; directive

That looks like a good solution. I'm curious, though, when you postpone
DUP in the directive, does it find the original DUP or the
interpret-only DUP ? I'm guessing that interpret-only means it isn't
even found while you're compiling? That looks just perfect to me.

[Mark W. Humphries]

J Thomas wrote:
> Mark W. Humphries wrote:
> > J Thomas wrote:
> > [snip]
> > > : dup ( S: x -- x x ) ( C: -- )
> > > 1 #dups +!
> > > state @ if
> > > postpone dup else
> > > dup
> > > then ; immediate
> >
> > Here's how I would do this in my Forth:
> > : directive ( -- ) immediate compile-only ;
> > : tally ( -- ) 1 #dups +! ;
> > : dup ( x -- xx ) tally dup ; interpret-only
> > : dup ( x -- xx ) tally postpone dup ; directive
>
> That looks like a good solution. I'm curious, though, when you postpone
> DUP in the directive, does it find the original DUP or the
> interpret-only DUP ? I'm guessing that interpret-only means it isn't
> even found while you're compiling? That looks just perfect to me.

Your guess is correct, the compiler only searches for words that have
their compilation bit set by doing the equivalent of:
" dup" compilation lookup

The interpreter on the other hand only looks for words that have their
interpretation bit set.
Normal words have both their interpretation and compilation bits set
and therefore are found by both the compiler and interpreter.

Cheers,
Mark Humphries
Manila, Philippines

[Julian V. Noble]

J Thomas wrote:
> I don't have a proposal thought out, only some beginning ideas. Anybody
> is welcome to build on them for whatever purpose, or to criticise them.
>
> Usually what we need is not IMMEDIATE words or state-smart words.
> IMMEDIATE was a quick simple hack that people have been working around
> for 30 years. It's usually adequate, but not good.
>
> Any word that includes POSTPONE or executes COMPILE, (or [COMPILE] ?)
> is compile-only, and recursively any word that executes a word that
> includes POSTPONE etc it compile-only.
>
> The complications of state-smart etc don't belong in the words
> themselves. The issue doesn't come up after you execute the words. The
> issue comes up in the interpreter/compiler, when you FIND words and
> decide what to do with them.
>
> What if FIND (or its string-and-count replacement) were to give more
> information? What if it were to give
>
> -3 normal compilation behavior, do not interpret.
> -2 discard the xt always, do nothing.
> -1 execute when interpreting, compile when compiling.
> 0 not found
> 1 immediate
> 2 do not interpret, execute while compiling
> 3. execute while compiling, do not execute while interpreting, but a
> second xt is available to execute while interpreting or fetch with '
> ['] etc. The other xt can be found with a special command.
>
> Wouldn't a lot of the endless complications go away if this extra
> information was available to standard programs?
>
> Whenever you POSTPONE a word or compile COMPILE, the current word
> becomes type -3 unless it already has compiled : or :NONAME or
> postponed DOES> , and if you try to make it immediate it becomes type
> 2. The rules for that might turn out to be too complicated to use, but
> some of them might give useful warning messages without getting too
> much in the way.
>
> Wordlists don't clean up the IMMEDIATE mess, they make more messes
> attempting to solve that. IMMEDIATE affects FIND . State-smart words
> don't work because they try to fix something that ought to happen while
> parsing with something that happens at execution time -- whenever
> execution happens. An improved FIND and more sophisticated behaviors at
> that point might do it.
>

I have nothing against a FIND that supplies more information, but
I fail to see why IMMEDIATE causes a mess. If you get rid of
IMMEDIATE how is ; going to work? Not to mention IF ELSE THEN BEGIN
WHILE UNTIL etc.

I am not sure that defining a DUP that tells you how often it
is compiled or executed justifies the proposed changes---after
all, you can always sort a program by words and count the
instances of any word.

--
Julian V. Noble
Professor Emeritus of Physics
University of Virginia

[J Thomas]

Julian V. Noble wrote:

> I have nothing against a FIND that supplies more information, but
> I fail to see why IMMEDIATE causes a mess. If you get rid of
> IMMEDIATE how is ; going to work? Not to mention IF ELSE THEN BEGIN
> WHILE UNTIL etc.

IMMEDIATE works. What it does usually isn't quite what you want but you
can make it work. I don't think we can get rid of IMMEDIATE any time
soon -- it would break a lot of existing code. We'll have to keep
IMMEDIATE for a good long time. I want some sharper tools in the
toolbox to go along with IMMEDIATE .

> I am not sure that defining a DUP that tells you how often it
> is compiled or executed justifies the proposed changes---after
> all, you can always sort a program by words and count the
> instances of any word.

That might not get the number you want, but my point was that this is a
trivial example that turns much harder than it ought to be.

There ought to be standard ways to extend the compiler. At present
those are very limited. The traditional way to extend the compiler is
by writing new words that do things at compile-time. That works. If you
want to extend the compiler to do something special -- anything special
-- at compile time when the compiler finds a standard word, then you
can make a traditional state-smart word (which has problems) or you can
write your own interpreter (which has other problems).

[Alex McDonald]

Julian V. Noble wrote:
> J Thomas wrote:
> > I don't have a proposal thought out, only some beginning ideas. Anybody
> > is welcome to build on them for whatever purpose, or to criticise them.
> >
> > Usually what we need is not IMMEDIATE words or state-smart words.
> > IMMEDIATE was a quick simple hack that people have been working around
> > for 30 years. It's usually adequate, but not good.
> >
> > Any word that includes POSTPONE or executes COMPILE, (or [COMPILE] ?)
> > is compile-only, and recursively any word that executes a word that
> > includes POSTPONE etc it compile-only.
> >
> > The complications of state-smart etc don't belong in the words
> > themselves. The issue doesn't come up after you execute the words. The
> > issue comes up in the interpreter/compiler, when you FIND words and
> > decide what to do with them.
> >
> > What if FIND (or its string-and-count replacement) were to give more
> > information? What if it were to give
> >
> > -3 normal compilation behavior, do not interpret.
> > -2 discard the xt always, do nothing.
> > -1 execute when interpreting, compile when compiling.
> > 0 not found
> > 1 immediate
> > 2 do not interpret, execute while compiling
> > 3. execute while compiling, do not execute while interpreting, but a
> > second xt is available to execute while interpreting or fetch with '
> > ['] etc. The other xt can be found with a special command.
> >
> > Wouldn't a lot of the endless complications go away if this extra
> > information was available to standard programs?
> >
> > Whenever you POSTPONE a word or compile COMPILE, the current word
> > becomes type -3 unless it already has compiled : or :NONAME or
> > postponed DOES> , and if you try to make it immediate it becomes type
> > 2. The rules for that might turn out to be too complicated to use, but
> > some of them might give useful warning messages without getting too
> > much in the way.
> >
> > Wordlists don't clean up the IMMEDIATE mess, they make more messes
> > attempting to solve that. IMMEDIATE affects FIND . State-smart words
> > don't work because they try to fix something that ought to happen while
> > parsing with something that happens at execution time -- whenever
> > execution happens. An improved FIND and more sophisticated behaviors at
> > that point might do it.
> >
>
> I have nothing against a FIND that supplies more information, but
> I fail to see why IMMEDIATE causes a mess. If you get rid of
> IMMEDIATE how is ; going to work? Not to mention IF ELSE THEN BEGIN
> WHILE UNTIL etc.
>
> I am not sure that defining a DUP that tells you how often it
> is compiled or executed justifies the proposed changes---after
> all, you can always sort a program by words and count the
> instances of any word.
>
> --
> Julian V. Noble
> Professor Emeritus of Physics
> University of Virginia

In a dual xt system, something like IF would work as follows;

: IF -14 throw compilation> ( xt of IF -- ) ... ; \ do IF when
compiling stuff

The part before compilation> is interpret time (and is the xt returned
by FIND), the part after is compile time. It's not immediate (it has
non-standard compilation semantics) and it's not state-smart either.
The outer interpreter is state smart, and has enough carnal knowledge
of the dual XT to know which half to execute. In this system, you can
count interpret, compilation and execution uses with (tested on an
experimental STC version of W32F that supports this)

variable dupexec 0 dupexec !
variable dupcomp 0 dupcomp !
variable dupinterp 0 dupinterp !

: dupexec+1 ( -- ) \ increment execution count
1 dupexec +! ;

: dup ( n -- n n ) \ with side effects
dup 1 dupinterp +! \ increment interpret count
compilation> drop
postpone dup
1 dupcomp +! \ increment compilation count
postpone dupexec+1 \ increment execution count
;

[The drop after compilation> is for the xt of the word being defined;
in this case, it's not needed.] In this system, an IMMEDIATE word is
simply one where the interpret and execution semantics are identical;
this

: x stuff ; immediate

is the same as this;

: x stuff compilation> drop stuff ;

The xt returned by FIND can be COMPILE,d, and it does the right thing
when EXECUTEd, as the decision is made in the interpreter, not by the
words themselves. The immediacy flag can therefore be ignored; it has
no use beyond identifying that the word has identical interpret and
compilation semantics.

--
Regards
Alex McDonald

[Alex McDonald]

Alex McDonald wrote:

> : x stuff ; immediate
>
> is the same as this;
>
> : x stuff compilation> drop stuff ;
>

Oops; meant

: x stuff compilation> drop postpone stuff ;

--
Regards
Alex McDonald

[GerryJ]

Alex McDonald wrote:

<snip>
> In a dual xt system, something like IF would work as follows;
>
> : IF -14 throw compilation> ( xt of IF -- ) ... ; \ do IF when
> compiling stuff
>
> The part before compilation> is interpret time (and is the xt returned
> by FIND), the part after is compile time. It's not immediate (it has
> non-standard compilation semantics) and it's not state-smart either.
> The outer interpreter is state smart, and has enough carnal knowledge
> of the dual XT to know which half to execute. In this system, you can
> count interpret, compilation and execution uses with (tested on an
> experimental STC version of W32F that supports this)
>
<snip>

That's more or less what my system does. Every word has a pointer to a
semantics class, which contains, amongst other things, two methods
called interpret() and
compile(). Each of these is defined with a :noname definition.
Therefore if::interpret() throws an exception, if::compile() does what
it should. The text interpreter calls interpret() or compile()
depending on the value of STATE or immediacy of the found word.

Perhaps I should make the semantics base class visible to the user, at
present it's hidden.

Gerry

[Andreas Kochenburger]

On 29 Aug 2006 10:11:14 -0700, "GerryJ"
<***@jackson9000.fsnet.co.uk> wrote:
>> In a dual xt system, something like IF would work as follows;
>> : IF -14 throw compilation> ( xt of IF -- ) ... ; \ do IF when
>> compiling stuff
>That's more or less what my system does. Every word has a pointer to a
>semantics class, which contains, amongst other things, two methods
>called interpret() and >compile()

There are only few words with dual semantics in standard Forth. How
many are in your system? It seems to me a waste of space to let nearly
every word "carry a useless xt around".


Andreas
-------
1 + 1 = 3, for large values of 1.

[J Thomas]

Andreas Kochenburger wrote:
> On 29 Aug 2006 10:11:14 -0700, "GerryJ"
> <***@jackson9000.fsnet.co.uk> wrote:
> >> In a dual xt system, something like IF would work as follows;
> >> : IF -14 throw compilation> ( xt of IF -- ) ... ; \ do IF when
> >> compiling stuff
> >That's more or less what my system does. Every word has a pointer to a
> >semantics class, which contains, amongst other things, two methods
> >called interpret() and >compile()
>
> There are only few words with dual semantics in standard Forth. How
> many are in your system? It seems to me a waste of space to let nearly
> every word "carry a useless xt around".

If you choose to implement it that way, you're using an extra cell for
each word. How many words do you use? 8000? That might be as many as
32K extra memory. Trivial in a large system. If you want to save that
space you can probably work out a way to point to one method, and then
if at any time you want to add a new method you instead point to the
thing that chooses among multiple methods. That saves space and time at
the expense of an extra call when you parse a word that has multiple
methods.

So far, I particularly like the approach that Mark Humphries uses. As I
reconstruct it, he has two flags with four meanings.

00 execute when interpreting, do not find when compiling.
01 execute when interpreting, compile when compiling.
10 do not find when interpreting, execute when compiling.
11 execute when interpreting, execute when compiling.

This is a simple addition to the current core standard. It doesn't have
to be done with changes to search order, it can be implemented just by
letting FIND or equivalent discard some matches some times (and
continue searching for another match), and add COMPILE-ONLY and
INTERPRET-ONLY to set the flags.

What would we get if COMPILE-ONLY and INTERPRET-ONLY were standard?
What I find exciting and disturbing is that it would allow more and
easier portable compiler extensions. So for example, Mitch Bradley
needed interpreted control structures. Many of his users were engineers
who needed to use Forth with just a few hours instruction. They didn't
have time to learn why things they tested while interpreting didn't
work compiled. So provide them a dummy return stack to use while
interpreting, and interpreted loops etc, and words that appear to
behave the same interpreting and compiling like ' and CHAR and S" , and
they happily debug their code.

Using STATE-smart words to do this is good for them in the short run
but then it causes a lot of trouble later. Some other implementation is
better, if it's worth doing at all.

Interpreted control structures are a convenience. You can do things
like initialise arrays with no fuss. You can test code. They aren't
necessary.

2000 2000 array foo
marker cleanup
: initialise-foo .... ;
cleanup

You can always find workarounds. And it's only a convenience to test
code at the keyboard, if you write it well enough you'll know exactly
what it will do. Everybody who sticks with Forth can do without those
conveniences easily. If we needed interpreted control structures or
easy debugging we'd have given up on Forth right away. But it might be
nice to have.

And if we had INTERPRET-ONLY those could be written portably, and added
to any Forth that had the core wordset. Load them or not, your choice.
(You can already do that, but for one Forth at a time. Forth blurs the
distinction between system programmer and application programmer, but
not portably.)



Now I'll look ahead a bit. Say we have an interpreter that worked like
that. Convenient. And we avoid the STATE-smart problems. We don't make
choices at run-time that ought to be made at find-time. Is it still
anything confusing? Of course!

: >R ... ; COMPILE-ONLY
: >R .... ; INTERPRET-ONLY

: FOO ' >R .... ;

' is supposed to pick up the interpret-only version. Execute it at the
wrong time and you'll be putting things on the wrong stack. You could
resolve that by using the fake return stack all the time, but that
might have a performance penalty. Get two different systems in parallel
and there's bound to room to mix them up. But one thing that helps is
that any word that executes a compile-only word is also compile-only,
while any word that somehow executes an interpret-only word is also
interpret-only. A lot of the time that could be arranged automatically.

[GerryJ]

J Thomas wrote:

> Andreas Kochenburger wrote:
> > On 29 Aug 2006 10:11:14 -0700, "GerryJ"
> > <***@jackson9000.fsnet.co.uk> wrote:
> > >> In a dual xt system, something like IF would work as follows;
> > >> : IF -14 throw compilation> ( xt of IF -- ) ... ; \ do IF when
> > >> compiling stuff
> > >That's more or less what my system does. Every word has a pointer to a
> > >semantics class, which contains, amongst other things, two methods
> > >called interpret() and >compile()
> >
> > There are only few words with dual semantics in standard Forth. How
> > many are in your system? It seems to me a waste of space to let nearly
> > every word "carry a useless xt around".
>
> If you choose to implement it that way, you're using an extra cell for
> each word. How many words do you use? 8000? That might be as many as
> 32K extra memory. >

No it doesn't use an extra cell - see my reply to Andreas Kochenburger

Gerry

[J Thomas]

GerryJ wrote:

> No it doesn't use an extra cell - see my reply to Andreas Kochenburger

Yes, you solved that problem.

The way I imagined you solving it was to provide one execution token
for normal words, and when you make a word special you redirect that,
you provide an execution token that points to two others and let the
interpreter choose which of them to use.

Your solution was to have each word point to a class. The normal words
would presumably all point to a class that did:

When interpreting, calculate the xt based on the return address the
normal word left, and execute it.
When compiling, calculate the xt based on the return address and
compile it.

Since they all point to the same class they take no extra space.

I think there are various efficient ways to implement this. The
question is whether it's worth doing. If it's worth doing it might be
added to the search-order extension wordset, or some other extension
wordset so there would be a standard way to do it, for whichever
implementors want to.

[GerryJ]

J Thomas wrote:
> GerryJ wrote:
>
> > No it doesn't use an extra cell - see my reply to Andreas Kochenburger
>
> Yes, you solved that problem.
>
> The way I imagined you solving it was to provide one execution token
> for normal words, and when you make a word special you redirect that,
> you provide an execution token that points to two others and let the
> interpreter choose which of them to use.
>
> Your solution was to have each word point to a class. The normal words
> would presumably all point to a class that did:
>
> When interpreting, calculate the xt based on the return address the
> normal word left, and execute it.
> When compiling, calculate the xt based on the return address and
> compile it.
>

It's not done that way, the xt (in ANS terms) is the address of the
semantics pointer i.e. the address of the pointer to the semantics
class. The text interpreter passes this xt on the stack to the
interpret() or compile() method so they don't have to calculate it.
This xt provides access to the body of the word. I didn't show the xt
in my example to avoid confusing the issue but the stack signatures
should be ( xt -- ), that is a DROP is required in the example :noname
definitions. In reality they would be consumed usefully.

If by 'normal words' you mean colon definitions, yes they all point to
the same class. Other normal words like variables all point to a
different class as different actions are required i.e. push a dataspace
address instead of executing Forth code.

Gerry

[pablo reda]

I have another aproach to this "problem"
I not sure if this is rigth but I try anyway

I remove the compiler for the languaje...and then not have IMMEDIATE
for make this I fix 5 words like MACROS ";" "(" ")" ")(" "[" "]"
and you can't make macros and does> not exist.

I not found problems for now but some tricks is not posible.

[Andreas Kochenburger]

On 30 Aug 2006 10:42:14 -0700, "pablo reda" <***@gmail.com>
wrote:
>I remove the compiler for the languaje.

Mi amigo, the coexistence of interpreter and compiler in one system is
what makes Forth singular. Remove the compiler and you'd probably be
more productive with another interpreter, eg. Python.


Andreas
-------
1 + 1 = 3, for large values of 1.

[pablo reda]

Andreas Kochenburger ha escrito:

> Mi amigo, the coexistence of interpreter and compiler in one system is
> what makes Forth singular. Remove the compiler and you'd probably be
> more productive with another interpreter, eg. Python.

yes, another guy tell me the same. but i'ts my 2 cent.
(Monty Python Flying Circus is the best serie of humor but the lang
don't like :)

Why the control of compiler is necesary ?
for optimization, but optimize its not programing !!

[Mark W. Humphries]

pablo reda wrote:
[snip]
> Why the control of compiler is necesary ?
> for optimization, but optimize its not programing !!

For increased flexibility and expressive power.

Cheers,
Mark Humphries
Manila, Philippines

[pablo reda]

Mark W. Humphries ha escrito:
>
> For increased flexibility and expressive power.
>

I not found any example that confirm this afirmation,
tell me an example when be neccesary the compilers words and not be
possible whitout this compilers words, and not be a optimization trick
!!

[J Thomas]

pablo reda wrote:
> Mark W. Humphries ha escrito:
> >
> > For increased flexibility and expressive power.
> >
>
> I not found any example that confirm this afirmation,
> tell me an example when be neccesary the compilers words and not be
> possible whitout this compilers words, and not be a optimization trick
> !!

: array
create dup , 1+ * chars allot
does> dup >r @ * + chars r> + ;

8 8 array chessboard

knight 1 5 chessboard c!

What would you do with no compiler? Would you write

( x y addr ) dup >r @ * + chars r> +

every time you compute an array address?

Well, but you can make named macros that get interpreted when the name
is invoked. That gives you all the advantages of compilation except
optimisation.

One bug (or feature), if you redefine a name then won't old
previously-defined words that call it all start getting the new
version?

[pablo reda]

>
> : array
> create dup , 1+ * chars allot
> does> dup >r @ * + chars r> + ;
>
> 8 8 array chessboard
>

#chessboard )( 64

> knight 1 5 chessboard c!
>

knight 1 8 * 5 + 'chessboard c!

the compiler "outside" the lang can optinize this or not...

> What would you do with no compiler? Would you write
>
> ( x y addr ) dup >r @ * + chars r> +
>
> every time you compute an array address?

yes,but sometimes is better make the cursor's (point to memory) in 1D,
don't use 2D
the better algoritm for line drawing is with 1D pointer and not the 2D
brassem... bla

another point is, when one coord are variable then you need compute
anyway..

[Marcel Hendrix]

"pablo reda" <***@gmail.com> writes Re: The IMMEDIATE mess

[ by "J Thomas" <***@gmail.com> ? ]

>> : array
>> create dup , 1+ * chars allot
>> does> dup >r @ * + chars r> + ;

>> 8 8 array chessboard

> #chessboard )( 64

>> knight 1 5 chessboard c!

> knight 1 8 * 5 + 'chessboard c!

> the compiler "outside" the lang can optinize this or not...

Do you have any Forth knowledge at all, or are you just able to write
a `Forth?'

-marcel

[pablo reda]

Marcel Hendrix ha escrito:

> "pablo reda" <***@gmail.com> writes Re: The IMMEDIATE mess
>
> [ by "J Thomas" <***@gmail.com> ? ]
>
> >> : array
> >> create dup , 1+ * chars allot
> >> does> dup >r @ * + chars r> + ;
>
> >> 8 8 array chessboard
>
> > #chessboard )( 64
>
> >> knight 1 5 chessboard c!
>
> > knight 1 8 * 5 + 'chessboard c!
>
> > the compiler "outside" the lang can optinize this or not...
>
> Do you have any Forth knowledge at all, or are you just able to write
> a `Forth?'
>

yes, http:/www.geocities.com/redaforth

[Mark W. Humphries]

pablo reda wrote:
> Mark W. Humphries ha escrito:
> >
> > For increased flexibility and expressive power.
> >
>
> I not found any example that confirm this afirmation,
> tell me an example when be neccesary the compilers words and not be
> possible whitout this compilers words, and not be a optimization trick
> !!

An end-user application that includes a built-in macro compiler.

Cheers,
Mark Humphries
Manila, Philippines

[GerryJ]

Andreas Kochenburger wrote:

> On 29 Aug 2006 10:11:14 -0700, "GerryJ"
> <***@jackson9000.fsnet.co.uk> wrote:
> >> In a dual xt system, something like IF would work as follows;
> >> : IF -14 throw compilation> ( xt of IF -- ) ... ; \ do IF when
> >> compiling stuff
>>
> >That's more or less what my system does. Every word has a pointer to a
> >semantics class, which contains, amongst other things, two methods
> >called interpret() and >compile()
>
> There are only few words with dual semantics in standard Forth. How
> many are in your system? It seems to me a waste of space to let nearly
> every word "carry a useless xt around".
>

It depends on what you mean by dual semantics, I think this includes
all words that have special compilation semantics and there are many of
those as it includes all control words ; s" literal does> etc. (you can
count them as easy as I can). The scheme I have implemented has a
semantics class for each of these as it gives a way to get straight to
the compiling code without having to test STATE (that is done by the
text interpreter).

Every word does not carry a "useless xt" around. The cell in every word
that points to the semantics class for that word is equivalent to the
CFA (I believe that's what it's called) in a traditional Forth
implementation, and so takes up no more space than a CFA. It is not
useless as the system uses it every time a word is interpreted,
compiled, postponed or executed. For example this cell ( the semantics
pointer) in all colon definitions point to the same semantics class
that carries out the necessary interpretation and compilation actions
for colon definitions; that for all variables points to a variable
semantics class and so on. The actual xt is the address of semantics
pointer. The semantics pointer adds an extra level of indirection and
so interpreting, executing and compiling are slightly slower, but this
has no detrimental effect on the compiled code (except for EXECUTE).

Using this scheme for control words, which are compile-only means that
they do not have to test STATE to see whether an exception should be
thrown. The exception is thrown in the interpret() method. This code
only occurs once in a compile-only semantics class which is inherited
by all compile-only words. Also these words do not have to be immediate
as the compile() method is automatically executed when compiling.

Getting back to J Thomas's original point which was, I believe, that it
would be nice if we could define words that had different behaviours
when interpreting and compiling without having to use IMMEDIATE and
without having to test STATE (correct me if I'm wrong).

With my implementation I believe that it would be very easy to provide
what he is asking for, but in practice not portable and therefore not a
lot of use as the following is totally non-standard. In essence a base
semantics class and its methods would be available for the programmer
to use as a parent class and he would write something like (I'm using
Bernd Paysan's Mini-OOF notation here which should be self-evident):

Semantics class
end-class FooSemantics \ declares a new class

:noname
." Foo is being interpreted"
; FooSemantics defines interpret()

:noname
." Foo is compiling"
; FooSemantics defines compile()

FooSemantics define-new foo

Note define-new magically creates a new word whose semantics pointer
points to FooSemantics (and doesn't exist yet but would be easy).

Then typing:
foo <cr> Foo is being interpreted ok
: x foo ; <cr> Foo is compiling ok

Different behaviours achieved without use of STATE or IMMEDIATE

I believe it would be fairly easy to add different run-time behaviour
if desired but of course using CREATE ... DOES> ... in the methods
could do that.

Personally I'm not sure how useful all this is and so will probably
never implement it, I've simply described a method by which J Thomas's
wishes could be achieved.

Gerry

[Marcel Hendrix]

"Julian V. Noble" <***@virginia.edu> writes Re: The IMMEDIATE mess
[..]
> I am not sure that defining a DUP that tells you how often it
> is compiled or executed justifies the proposed changes---after
> all, you can always sort a program by words and count the
> instances of any word.

Try it.

-- -------------------
[UNDEFINED] DF@+ [IF] : DF@+ ( addr -- addr' ) ( F: -- r ) DUP DF@ DFLOAT+ ; [THEN]
[UNDEFINED] DF!+ [IF] : DF!+ ( addr -- addr' ) ( F: r -- ) DUP DF! DFLOAT+ ; [THEN]
[UNDEFINED] DF+!+ [IF] : DF+!+ ( addr -- addr' ) ( F: r -- ) DUP DF@ F+ DF!+ ; [THEN]
[UNDEFINED] DF+! [IF] : DF+! ( addr -- ) ( F: r -- ) DUP DF@ F+ DF! ; [THEN]

\ DUP is .... DUP and DROP DUP ...

S" foo.bar" INCLUDED
-- -------------------

-marcel

[a***@netcourrier.com]

> Wordlists don't clean up the IMMEDIATE mess, they make more messes
> attempting to solve that. IMMEDIATE affects FIND . State-smart words
> don't work because they try to fix something that ought to happen while
> parsing with something that happens at execution time -- whenever
> execution happens. An improved FIND and more sophisticated behaviors at
> that point might do it.

This might be a bit OT cause all subject seems to be linked to some
standardization process, but Moore keeps using a separate wordlist
(althought not exaclty in the usual meaning of the word) in his
colorForth -- and he is not the kind of guy to let live any mess in his
Forth. I for one use the same approach, and I'm quite happy with it.
One point is that I don't remember having used immediacy in application
code. I'll probably use some kind of FOR-EACH in a near future and I
don't expect problems, even if I had to write an interpreted version,
which I don't actually need. I see I can do this, create new
control-flow structures when it's really worth it, I see I can use it
to expand inline some code for speed when it's worth it ( sort of a
macros, which is BTW Chuck names immediate words now), but that's about
the end of it for me.
Perhaps I don't write enough Forth code. Maybe I miss something (funny
I wrote 'nothing' by mistake at first - ok, let me call my lawyer ;) .
It would be interesting to me - and who knows, maybe it would clarify
the problem a bit - if one writes down the use cases or sort them into
categories, if any, of the feature.

Amicalement,
Astrobe

[Anton Ertl]

"J Thomas" <***@gmail.com> writes:
>The complications of state-smart etc don't belong in the words
>themselves. The issue doesn't come up after you execute the words. The
>issue comes up in the interpreter/compiler, when you FIND words and
>decide what to do with them.
>
>What if FIND (or its string-and-count replacement) were to give more
>information? What if it were to give
>
>-3 normal compilation behavior, do not interpret.
>-2 discard the xt always, do nothing.
>-1 execute when interpreting, compile when compiling.
> 0 not found
> 1 immediate
> 2 do not interpret, execute while compiling
> 3. execute while compiling, do not execute while interpreting, but a
>second xt is available to execute while interpreting or fetch with '
>['] etc. The other xt can be found with a special command.
>
>Wouldn't a lot of the endless complications go away if this extra
>information was available to standard programs?

First of all, you cannot change FIND in this way, because that might
break standard programs (if there are any that can use FIND, hardly
defined as it is). In the standard, the same functionality could be
expressed as, e.g.,

find dup if find dup if
1 = if -1 <> if
<code a> <code a>
else else
<code b> <code b>
then then
else else
<code c> <code c>
then then

and you can imagine other variations. These two pieces of code would
behave the same way with a standard FIND, but not with your changed
FIND; in the -3, -2, 2, and 3 cases they would behave differently, and
probably both variants would behave differently in some cases than
intended.

Second, FIND sucks in many respects: it takes a counted string, and
its results are not well specified.

So, what should actually happen is that FIND is replaced by something
better. And if we do that, we can do better than make a FIND-like
word that's even uglier than FIND.

Gforth has:

|`find-name' c-addr u - nt | 0 gforth ``find-name''
| Find the name c-addr u in the current search order. Return its nt,
|if found, otherwise 0.
|
|`name>int' nt - xt gforth ``name>int''
| xt represents the interpretation semantics of the word nt. If nt has
|no interpretation semantics (i.e. is `compile-only'), xt is the
|execution token for `ticking-compile-only-error', which performs `-2048
|throw'.
|
|`name?int' nt - xt gforth ``name?int''
| Like `name>int', but perform `-2048 throw' if nt has no
|interpretation semantics.
|
|`name>comp' nt - w xt gforth ``name>comp''
| w xt is the compilation token for the word nt.

While this is relatively clean, it would not be a good fit for systems
like that of Mark Humphries, or for compilation-wordlist systems. So
if we want to support them, we need something else, e.g., something
like:

find-interpretation ( c-addr u -- xt | 0 )

find-compilation ( c-addr u -- w xt | 0 0 )

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2006: http://www.complang.tuwien.ac.at/anton/euroforth2006/

[J Thomas]

Anton Ertl wrote:
> "J Thomas" <***@gmail.com> writes:
> >The complications of state-smart etc don't belong in the words
> >themselves. The issue doesn't come up after you execute the words. The
> >issue comes up in the interpreter/compiler, when you FIND words and
> >decide what to do with them.

....

> >Wouldn't a lot of the endless complications go away if this extra
> >information was available to standard programs?

[Snip excellent explanation why FIND can't be modified as I suggested
without breaking existing code.]

> Second, FIND sucks in many respects: it takes a counted string, and
> its results are not well specified.

> So, what should actually happen is that FIND is replaced by something
> better. And if we do that, we can do better than make a FIND-like
> word that's even uglier than FIND.

Yes, I was thinking of replacing FIND by a better word that accepts
string-and-count. Something even better than I was thinking sounds
great.

> Gforth has:
>
> |`find-name' c-addr u - nt | 0 gforth ``find-name''
> | Find the name c-addr u in the current search order. Return its nt,
> |if found, otherwise 0.

Good, you've factored out the finding from choosing the behavior.

> |`name>int' nt - xt gforth ``name>int''
> | xt represents the interpretation semantics of the word nt. If nt has
> |no interpretation semantics (i.e. is `compile-only'), xt is the
> |execution token for `ticking-compile-only-error', which performs `-2048
> |throw'.
> |
> |`name?int' nt - xt gforth ``name?int''
> | Like `name>int', but perform `-2048 throw' if nt has no
> |interpretation semantics.
> |
> |`name>comp' nt - w xt gforth ``name>comp''
> | w xt is the compilation token for the word nt.

Are you saying the compilation token needs more than a cell? What do
you do with the second cell?

> While this is relatively clean, it would not be a good fit for systems
> like that of Mark Humphries, or for compilation-wordlist systems. So
> if we want to support them, we need something else, e.g., something
> like:
>
> find-interpretation ( c-addr u -- xt | 0 )
>
> find-compilation ( c-addr u -- w xt | 0 0 )

OK! That looks just fine to me, apart from the unexplained w. Does the
w say whether to compile versus execute the xt during compilation? That
makes sense.

You are leaving the state out of these words. You look for the
interpretation xt (whether or not you're interpreting at the time), or
look for the compilation xt (whether or not you're compiling). So all
the state-smartness can be confined to the interpreter itself, that's
the only place the decision must be made to do different things
compiling versus interpreting.

I like the idea of finding the name with one word and finding its
interpret or compile behavior with a second word, but that may not make
sense in context. Some implementations might return a name field
address. If you look for an interpret behavior for that word and there
isn't one, ideally you should search for an earlier-defined word with
the same name that does have an interpret behavior, and return false
only if you don't find one. So we haven't exactly separated the
find-name from the find-behavior. The system might be combining them
behind your back, and it only looks like they're factored out.

[Anton Ertl]

"J Thomas" <***@gmail.com> writes:
>
>Anton Ertl wrote:
>> |`name>comp' nt - w xt gforth ``name>comp''
>> | w xt is the compilation token for the word nt.
>
>Are you saying the compilation token needs more than a cell?

Yes, that's more practical.

> What do
>you do with the second cell?

In Gforth, currently such a compilation token consists of:

- for words with default compilation semantics: w is the xt
representing the execution semantics, xt is the xt of COMPILE,.

- for words with non-default compilation semantics: w is the xt of a
word that performs the compilation semantics of the word we are
interested in; xt is the xt of EXECUTE.

E.g., for + the compilation token would be ' + ' COMPILE,. For S" the
compilation token would be ' S"-comp ' EXECUTE, where S"-comp performs
the compilation semantics of S".

One could represent the compilation semantics with a single xt, burt
that would require producing an additional xt for all words with
default compilation semantics, like this:

: ct-to-xt ( w xt1 -- xt2 )
dup ['] execute = if
drop
else
2>r :noname 2r> swap postpone literal compile, postpone ;
then ;

To avoid that, I prefer to deal with a two-cell compilation token.

>OK! That looks just fine to me, apart from the unexplained w. Does the
>w say whether to compile versus execute the xt during compilation? That
>makes sense.

Actually the xt says that.

>I like the idea of finding the name with one word and finding its
>interpret or compile behavior with a second word, but that may not make
>sense in context. Some implementations might return a name field
>address.

It's a good match for systems that have only one header even for words
like S" and TO, like Gforth. Then FIND-NAME returns the NFA, and the
other words access various parts of that header. But it is not a good
match for systems that use several headers in this case.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2006: http://www.complang.tuwien.ac.at/anton/euroforth2006/

[J Thomas]

Anton Ertl wrote:

> In Gforth, currently such a compilation token consists of:
>
> - for words with default compilation semantics: w is the xt
> representing the execution semantics, xt is the xt of COMPILE,.
>
> - for words with non-default compilation semantics: w is the xt of a
> word that performs the compilation semantics of the word we are
> interested in; xt is the xt of EXECUTE.
>
> E.g., for + the compilation token would be ' + ' COMPILE,. For S" the
> compilation token would be ' S"-comp ' EXECUTE, where S"-comp performs
> the compilation semantics of S".

That looks supremely flexible to me. If you aren't limited to EXECUTE
and COMPILE, you might substitute whatever you want for the xt. So for
my trivial example, you could make the compiler xt for DUP be something
that increments a variable and then does COMPILE, . It won't count the
' DUPs but it will have a well-defined behavior, not like a state-smart
DUP .

I like it.

Now, what should a standard say about this? I want to review:

When interpreting, you find execution tokens and execute them. That's
it. If there's something you don't want executed while interpreting
then ideally its execution token shouldn't be found.

When compiling you get an execution token and then you get a second
execution token which does something to the first. The defaults are
EXECUTE and COMPILE, but the compiling behavior could be anything.

How could that be made standard?

Interpreters that hardcode the execute/compile choice wouldn't work.
But it's easy to build standard interpreters. You can even do it with
standard code.

It should be possible to retrofit this functionality onto existing
Forths pretty easily. One possible way:

Have a header bit that says whether you get nondefault compilation
behavior. If not, just execute while interpreting and compile while
compiling, and nothing new is needed.

If it's nondefault compilation behavior, execute it and it gets two
execution tokens and executes the top one.

You'd have a word to give an existing word new compilation behavior. It
could go something like:

' behavior COMPILES name

: COMPILES
>IN @ >R BL FIND \ included while the dust settles around the FIND
replacement
DROP R> >IN ! SWAP CREATE , , SPECIAL-WORD COMPILE-ONLY
DOES> 2@ EXECUTE ;

This doesn't put the new word in the same wordlist as the old word, and
so on. SPECIAL-WORD would set a header bit to say it wasn't default
compiler behavior. So you execute it and it does whatever it's supposed
to do. With the new behavior compile-only, when you're interpreting you
find the old name instead and execute that. There's no standard way to
suppress the redefine warning.

But you could mostly retrofit old Forths pretty easily, I think.
Provided they have an extra header bit available.

Say you want to redefine the compile behavior twice. Do you get the
first redefine's execution token the second time? No, you FIND while
interpreting so you pick up the original xt. There are less wasteful
ways to do it than make a new CREATE DOES> word but that way ought to
work. Oh, except you have to start with a word that can be found while
interpreting. OK, you probably can't implement it with mostly-standard
code and have it run everywhere, but still on most systems it shouldn't
be hard to get it running with a few implementation-dependent tricks.

What words would need to be defined to make it work?

Would you want one word to make an existing word special, and one word
to change the compilation behavior of an existing special word?

[Albert van der Horst]

In article <***@mips.complang.tuwien.ac.at>,
Anton Ertl <***@mips.complang.tuwien.ac.at> wrote:
<SNIP>
>
>Second, FIND sucks in many respects: it takes a counted string, and
>its results are not well specified.
>
>So, what should actually happen is that FIND is replaced by something
>better. And if we do that, we can do better than make a FIND-like
>word that's even uglier than FIND.
>
>Gforth has:
>
>|`find-name' c-addr u - nt | 0 gforth ``find-name''
>| Find the name c-addr u in the current search order. Return its nt,
>|if found, otherwise 0.

I have this same word in ciforth.
(But I call nt a 'dictionary entry address', introduced in tforth, and
probably also in use in iforth's documentation. For a simple
ITC system a dea can double as an xt.)

Because this very fundamental word deserves a concise name,
I came up with FOUND.
This makes its stack effect easy to remember.
because it can be used as
S" DROP" FOUND IF ... THEN

Because if you want to look up something, you want to be able
to retrieve all properties, get-name / FOUND is superior
to FIND and more basic.

>|
>|`name>int' nt - xt gforth ``name>int''
>| xt represents the interpretation semantics of the word nt. If nt has
>|no interpretation semantics (i.e. is `compile-only'), xt is the
>|execution token for `ticking-compile-only-error', which performs `-2048
>|throw'.
>|
>|`name?int' nt - xt gforth ``name?int''
>| Like `name>int', but perform `-2048 throw' if nt has no
>|interpretation semantics.
>|
>|`name>comp' nt - w xt gforth ``name>comp''
>| w xt is the compilation token for the word nt.
>
>While this is relatively clean, it would not be a good fit for systems
>like that of Mark Humphries, or for compilation-wordlist systems. So
>if we want to support them, we need something else, e.g., something
>like:
>
>find-interpretation ( c-addr u -- xt | 0 )
>
>find-compilation ( c-addr u -- w xt | 0 0 )

This may not be good fit for other systems. Things will always be
somewhat cumbersome on some systems or another.

Let's agree on a few things:
- nt / dea is a fundamental concept that is present in every
Forth in some way
- get-name / FOUND is underlying FIND and is a cleaner way
to specify lookup

The difference between a compilation-bit in some flag-field of
a dea and presence in a compilation-wordlist is a matter of
implementation, and should not unduely influence language
design. I can imagine using the interpretation xt as an
nt, with a field pointing to the compilation xt.

>- anton
>--

Groetjes Albert
--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- like all pyramid schemes -- ultimately falters.
***@spenarnc.xs4all.nl http://home.hccnet.nl/a.w.m.van.der.horst

[Mark W. Humphries]

Anton Ertl wrote:
[snip]
> While this is relatively clean, it would not be a good fit for systems
> like that of Mark Humphries, or for compilation-wordlist systems. So
> if we want to support them, we need something else, e.g., something
> like:
>
> find-interpretation ( c-addr u -- xt | 0 )
>
> find-compilation ( c-addr u -- w xt | 0 0 )

This is what I currently use:

The word IMMEDIATE sets the latest header's precedence bit.
The word COMPILE-ONLY zeroes the latest header's compilation bit.
The word INTERPRET-ONLY zeroes the latest header's interpretation bit.

LOOKUP ( a # mask -- {lfa -1}|0 )
Looks up a word name (a #) and mask in the currently
active search order. If found returns it's lfa (i.e. name token) and
-1,
otherwise returns 0. A match is found if any bit in mask is set in
the header.

The interpreter starts out with:
: INTERPRETER ( -- ??? ) tibcount >upper interpretation lookup
........

The compiler starts out with:
: COMPILER ( -- ??? ) tibcount >upper compilation lookup ..........
immediate? .......

I treat a name's findability and what should be done with it during
interpretation or compilation to be a properties of the name (i.e. the
header, symbol table entry), not of the cfa (i.e. xt) which represents
the code to which it should be done.

Cheers,
Mark Humphries
Manila, Philippines

[Mark W. Humphries]

Mark W. Humphries wrote:
[snip]
> The word COMPILE-ONLY zeroes the latest header's compilation bit.
> The word INTERPRET-ONLY zeroes the latest header's interpretation bit.

Excuse the typo, the above should have been:
The word COMPILE-ONLY zeroes the latest header's interpretation bit.
The word INTERPRET-ONLY zeroes the latest header's compilation bit.

Cheers,
Mark Humphries
Manila, Philippines