Unquote syntax is unstable with respect to macro calls and quasiquotes

[masak]

I discovered this one by mistake, wanting to confirm the opposite of what I found:

$ perl -Ilib bin/bel
Language::Bel 0.58 -- msys.
> (prs '(+ 2 2))
"(+ 2 2)"
> (mac foo (x) `(append "The expression is " ,(prs x)))
> (foo (+ 2 2))
"The expression is (+ 2 2)"
> (set x '(+ 2 2))
(+ 2 2)
> x
(+ 2 2)
> (prs x)
"(+ 2 2)"
> ,(prs x)                    ;; this one conformed to my expectations
Error: comma-outside-backquote
> (foo ,(list '+ 2 2))        ;; here came the surprise
"The expression is (comma (list (quote +) 2 2))"
> `(foo ,(list '+ 2 2))       ;; and this is why it's "unstable"
(foo (+ 2 2))
>

In short, it's illegal to use unquote (,) outside of a quasiquote (`), unless you put it as the argument to a macro call, in which case it's legal (and handled by the macro), unless unless you put that macro call in a quasiquote, in which case the quasiquote interpolates the unquote.

unquote (`,`)              illegal (outside quasiquote or macro call)

macro call (`foo`)
    unquote (`,`)          legal (macro call gets it)

quasiquote ("`")
    macro call (`foo`)
        unquote (`,`)      legal (quasiquote gets it, overriding macro call)

I'm almost sure this is exactly why Bawden said quote interpolation should be part of the reader, not a separate macro. (To do: confirm that was the reason.) You can't do unquoting properly as a macro, because if you do, you get the above behavior, which is... wrong on some meta/design level, and not "real" unquoting.

But! It's spec! 🤣 So for the closing of this issue, all we need is one or a few tests to confirm this behavior. On some level I'm also curious to try extending the Bel reader to handle the quasiquotation logic, but... that's a separate thing, and not necessary for this issue.

[masak]

(To do: confirm that was the reason.)

I read the 1993 email now. I'd say yes, it was:

If you do expansion at read time, from the inside out, everything
works and you don't have to worry about occurrences of special markers
in your quasiquoted forms

Special markers such as comma above, which "leaks" by getting passed to the foo macro outside of a quasiquote, but not inside of one.

Trust me, or ask Alan to explain.

Rees is not all that sure, but I'm pretty sure.

[masak]

Linking to this blog post, which explains how Common Lisp handles quasiquotes and commas at reader time (as Alan Bawden said it should). Some more discussion at masak/alma#567 (comment) .

For Bel to do it the way Common Lisp does it, at reader time, would constitute a pretty radical change to the Bel specification. Maybe it can be tried out in a module which extends/modifies the reader.