feature request: sly-like sticker debugging. Possible?

[johnabs]

In case you're not aware (as I wasn't until today), sly lets you do some incredibly slick (IMO) debugging by telling the editor/repl to save the output of certain files without needing to expressly change the code to do so. These stickers also double as break points, and have handy highlighting, and can work with nested expressions separately.

I'm unsure if this is feasible in a similar way to sly, as it seems sly has its own repl built to enable such interaction, but I wanted to get your take and assessment on what may be needed to get such a project done. I may end up screwing around with it myself and submitting a pull request, assuming I get my research finished first, lol.

Here's a link to the sly manual sticker docs for reference: https://joaotavora.github.io/sly/#Stickers

[gcv]

It's definitely a cool feature. Thanks for telling me about it. This can done in Snail without implementing a separate Elisp REPL (which is itself a long-term goal). In a nutshell, you'd place special "sticker" annotations in the source file buffer (and track them on the Elisp side). Then, when an annotated piece of code is sent to the Julia side for evaluation, "sticker" annotation locations are sent along with the code. The Julia side then takes over, processes the annotations, and makes them do whatever you want. The tricky part is figuring out where in the syntax tree the annotation is supposed to apply and to inject the needed code changes back into the syntax tree before it's evaluated (kind of like aspects in AOP). Julia is enough of a Lisp that this is (or at least should be) possible, but this code is not very pleasant to write.

By all means try writing a Snail extension! It's a cool idea and I'll be happy to look at a PR.

[johnabs]

Glad I could bring an idea to the table, and thanks for the input! Do you mind if I run a few first pass ideas your way? (If not, ignore the upcoming block of text, lol).

Basically, I'm thinking it would likely be ideal to use a conversion from "standard" Julia to an expr, then find the specific part of that expr that corresponds to the stickered command, and then use a Julia macro (etc) to wrap that in either a print or similar statement.

So really, all we need is some function that will record the result of evaluating an expression while still allowing us to finish running the code (preferably a bit nicer than print), a macro/function that can inject that function to the appropriate place in an AST, and a(n ideally convenient) way to mark that position in the AST, yeah?

The last part seems like the trickiest, so here's my intuition: using some level of pattern matching within a local scope to work this out:
1). Convert stickered code of interest to an expr (quote, parse, show_sexpr)
2). Convert top-level form of encapsulating function to expr (same as 1)
3). Match within the larger expr block from 2 on the expr from step 1, to identify all positions of interest (regex?? not sure)
4). Have user select which (or all) of the matching expressions they want to sticker (to account for duplicate code, etc.).
5). Use something (regex? macro?) to modify our function, send the new function to repl, and recompile to override it in the global scope.

Steps 1 and 2 are pretty easy: we can take the top-level form of the function in question, convert to an expr via quoting, then to an AST via Meta.show_sexpr, just by wrapping everything in quote, or using the quote function. How we use those sexprs is a bit trickier, IMO. Also, I'm not 100% on how to implement the stickering yet, but I have a few ideas rattling around.

Does this sound like a reasonable plan of attack to you?

Thanks for responding to my original post, and I'm looking forward to submitting this PR as soon as I have time to allocate to it!

[gcv]

Rather than try to match code by pattern, I'd try to match it by location. Snail already relies on CSTParser to figure out the containing module of a piece of code being evaluated, and CSTParser can map syntax tree nodes to bytes of source code. (You already know the byte location of the sticker in the source code, since Emacs has that information.)

CSTParser has an awkward API, but Snail has enough example code for you to get started. I'm not sure how to convert from CSTParser nodes to Expr objects you can eval, but it should be possible.