Post by RuvimPost by RuvimPost by RuvimBut if you hate parsing words in principle (just because they do
parsing), why not hate such long parsing words like `[if]`, `\`, the
construct "]]...[[", etc? What is an alternative for them?
...
Post by RuvimI meant the word `[IF]` by itself, without connection with `WANT`.
Not necessarily a parsing word. Could also be treated as something
like another state (i.e., the text interpreter does the parsing, but
does something different with the words than interpretation state or
compile state.
[...]
\ parses, but apart from interactions like above it looks fine to me.
Post by RuvimThe word `]]` is also a parsing word (in a standard-compliant
implementation).
You mean that implementing ]] as a standard program requires parsing.
That's true, but the usual implementation in systems is as another
state-like thing.
[...]
Post by RuvimPost by RuvimHow to implement such functionality without active parsing the input stream?
How does :NONAME or ] implement its functionality? Do you also
consider it a parsing word? Note that in some Forth, Inc. Forth
systems ] parses on its own rather than using a state of the ordinary
text interpreter.
Yes, I mean that in a standard program the only approach available to
implement such functionality is the active parsing approach (at the moment).
The Recognizer API allows to replace short parsing words with
syntactically recognized forms (limited by one lexeme). Actually, it is
a generalization of numeric literals.
But to implement string literals or string templates (for string
interpolation), we still need active parsing. And we need it even if we
implement the beginning of such a literal as a recognizable form.
s" lorem ipsum dolor"
"lorem ipsum dolor"
If a recognizer has no side effects and returns a token translator on
success, then, for a string literal, the returned translator *parses*
the input buffer (or the input stream) till `"` [1,2].
So, in general, parsing is *inevitable*. Whether it is a parsing word
or a parsing translator — it does not matter.
One difference between parsing words and a recognizable syntactic forms
is visual — the latter does not require a space before the enclosed
content (this makes sense when whitespaces matter, like in string literals).
Another difference is related to API: `[']` and `postpone` are not
applicable to the latter (but are applicable to the former).
NB: applying `postpone` to `s"` is not the same as applying `postpone`
to a particular string literal.
So, a parsing word can be used by itself to reuse its functionality. But
to reuse functionality of some recognizable syntactic form, additional
words should be provided. It's notable that the Recognizers API
encourages (or even forces) us to provide such additional words.
One of features that Recognizer API provides is the *ability* to reuse
the system's Forth text interpreter loop without nesting.
An example:
\ Common data types that are related to the Recognizer API:
\ DataType: recognizer ⇒ xt
\ DataType: tt ⇒ xt
\ DataType: token ⇒ ( S: i*x F: j*k )
\ DataType: qt ⇒ ( token tt | 0 )
\ Functional data types qualifications:
\ DataType: recognizer = ( sd.lexeme -- qt )
\ DataType: tt = ( i*x token -- j*x )
wordlist constant foo-wid
here constant foo-magic
\ DataType: foo-sys ⇒ ( recognizer.prev x.foo-magic )
: end-foo ( foo-sys -- )
foo-magic <> -22 and throw \ "control structure mismatch"
set-perceptor \ restore the perceptor state
;
: recognize-foo ( sd.lexeme -- tt | 0 )
2dup "}foo" equals if 2drop ['] end-foo exit then
foo-wid search-wordlist if exit then 0
;
: begin-foo ( -- foo-sys )
\ save the perceptor state
perceptor foo-magic ( recognizer.prev x.foo-magic ) ( foo-sys )
\ set the system to use our recognizer
['] recognize-foo set-perceptor
;
: foo{ ( -- foo-sys )
begin-foo
; immediate
\ NB: "}foo" is not a word, but just a terminator.
\ create some test words
get-current foo-wid set-current
: test ." (test passed)" ;
: n1 1 ; : n2 2 ; : + + ; : . . ;
set-current
\ run some tests
foo{ test }foo \ should print "(test passed)"
foo{ n1 n2 + . }foo \ should print "3"
t{ foo{ n1 n2 + }foo -> 3 }t
Voilà! The third test fails with error -22. Because of foo-sys, we
cannot freely consume or produce stack parameters from/to outside of the
foo{ }foo structure.
NB: this example is not intended to show how to implement this dummy
functionality, but to show how the Recognizer API is used.
This functionality can be also implemented as a parsing word, using the
same Recognizer API, as follows:
: recognize-foo ( sd.lexeme -- tt | 0 )
foo-wid search-wordlist if exit then 0
;
: foo{
[: ( sd.lexeme -- qt )
2dup "}foo" equals 0= if recognize-foo exit then
2drop ['] unnest-translation
;] translate-input-with
; immediate
Where the following common factors are used:
: extract-lexeme ( -- sd.lexeme | 0 0 )
begin parse-name dup if exit then 2drop refill 0= until 0 0
;
: unnest-translation ( -- ⊥ )
true abort" unnest-translation is not handled"
;
: translate-input-till-unnest ( i*x -- j*x )
begin extract-lexeme
dup 0= -39 and throw \ "unexpected end of file"
perceive dup 0= -13 and throw \ "unrecognized"
dup ['] unnest-translation <> while execute
repeat drop
;
: translate-input-with ( i*x recognizer -- j*x )
perceptor >r set-perceptor
['] translate-input-till-unnest catch
r> set-perceptor throw
;
This implementation for "foo{" does not leave foo-sys on the stack, so
the following works as expected:
t{ foo{ n1 n2 + }foo -> 3 }t
Also, in this implementation the perceptor state is restored on error.
Thus, with Recognizer API, in some cases we have the choice of using the
non-parsing approach or the parsing approach.
The advantages of the parsing approach are:
- no additional items on the data stack;
- restoring the system's state on error;
- the API for reuse the implemented functionality does not impose a
particular terminator (and the provided recognizer is not polluted by a
particular terminator);
Now the question is: why should one prefer the non-parsing approach to
implement some functionality?
--
Ruvim