Sublime snippets first part

[mpourmpoulis]

Title

Sublime snippets first part

Description

This is part one of an attempt to integrate https://github.com/mpourmpoulis/CasterSublimeSnippetInterfaceExample into Caster

@LexiconCode I know this is a busy period For you so fear not as this is not ready to merge! There are quite a feat few things that need to be taken care off, documentation to be written and improvements to be made and will gradually get round to it over the next week. This is just a preliminary a PR For testing purposes And to make providing feedback easier.

In a nutshell these pull requests contains

• Files under the lib folder that handle the various tasks needed for implementing the desired functionality,such as

  • Interfacing with sublime

  • generate the the actual snippet text

  • maintaining a snippet state

• much of the above functionality is exposed via the dragonfly custom actions Snippet,SnippetVariant,SnippetTransform,DisplaySnippetVariants Which you can import from castervoice.lib.sublime_snippets

• a global control grammar,with which currently does not have much( mainly the commonly used variant <n>, so you do have not to define it everywhere) in which we enable by sublime snippet control ( to be improved)

But there are some improvements as well Like Snippet lambda can receive more arguments

And something to be the more juicy,which I want to eventually fully integrate in the control grammar But will need work

Related Issue

Motivation and Context

Say goodbye to .sublime-snippet files

How Has This Been Tested

Tested locally on Windows, need more people to test

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue or bug)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Renamed existing command phrases (we discourage this without a strong rationale).

Checklist

• I have read the CONTRIBUTING document.
• My code follows the code style of this project.
• I have checked that my code does not duplicate functionality elsewhere in Caster.
• I have checked for and utilized existing command phrases from within Caster (delete if not applicable).
• My code implements all the features I wish to merge in this pull request.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests pass.

Maintainer/Reviewer Checklist

• Basic functionality has been tested and works as claimed.
• New documentation is clear and complete.
• Code is clear and readable.

[mpourmpoulis]

has a lot of technicalities but still I pushed a proof of concept of activating only meaningful variance using a self modifying rule (sublime_snippet_additional_control)and a decorator

[LexiconCode]

Nice Work, if I'm feeling better will test this tomorrow.

I have a few thoughts regarding the structure of the the modules you've included. Historically many things were stored in lib. Much of which is now been moved into support files for their respective grammars. A recommendation would be to create a sublime folder in editor and keep all of its supporting functionality there.

Broadly speaking application grammars and supporting grammars are paired with their supporting functions in a <grammarname>_support.py. Once an application\language graduates to having a support file or multiple grammars, it gets its own folder. Eclipse is an example. If there's enough different types of functions they that could have multiple support files.

Function_like_utilities.py might be re-factored into lib if more than one grammar utilizes it. For now it would be better stored in the sublime folder.

Functionally what this does is separate the grammars in the core code of Caster. Its a WIP as I'm still working on re-factoring's a few things out of the core code back into grammars and supporting files.

In order for the override system for the user directory to work you will need to place conditional imports in the code. These imports try to import locally first (user directory), then from source code.

try: # Try first loading from caster user directory
    from eclipse_support import ec_con
except ImportError: 
    from castervoice.rules.apps.editor.eclipse_rules.eclipse_support import ec_con

Not ideal my opinion but will work until we get a better system in place for support files.

[mpourmpoulis]

Thank you very much for your feedback! Polluting the lib folder is something I did fear and I agree with you that it would be best if those files were moved under sublime_rules to keep things compatible with the rest of the project.
I will try to restructure it accordingly! Nonetheless, I do have a couple of concerns, though this might be due to me misunderstanding something or they may turn out to be unimportant after all.

Firstly, the main usage of the supporting functions is most likely going to be in user made grammars. Even the control grammar is here to support such custom grammars. As a consequence, conditional imports are going to be needed note only in the code that we ship by default but kinda everywhere in order to avoid inconsistencies or things breaking.

Secondly,even though the current functionality can be wrapped up inside a single file without losing too much readability, I'm not sure sure that this will be the case forever, for instance if we port and more advanced version of . More importantly, if the community finds these features beneficial, it might also make sense to provide at some point utilities that could make writing more complex snippets even easier and faster, such as voice commands (placeholder 1 to type ${1:} in one go), commonly used regular expressions ( so we can write "$1 = "+capitalize("$1")+"()"), and what ever is deemed necessary. so there's a chance we may end up with multiple files the users might want to override and thus a need for multiple conditional imports.

Either way I think I might be needlessly overthinking this potential issue and I will start by placing everything inside sublime_rules folder. Any further thoughts whether on this or the pull request in general are welcome and I hope you feel better soon:)

[mpourmpoulis]

It turns out that my feeling of conditional imports could be error-prone If used incorrectly was correct after all and unfortunately I found about it the hard way:)

All those conditionally imports never enter there except clause because all the vscode_rules,sublime_rules, are in the sys.path!!! so they're not necessary as they're not doing anything:D

['C:\\Users\\Admin\\Documents\\GitHub\\Caster', 'C:\\NatLink\\NatLink\\MacroSystem', 'C:\\WINDOWS\\SYSTEM32\\python27.zip', 'C:\\NatLink\\NatLink\\MacroSystem\\core', 'C:\\Python27\\Lib', 'C:\\Python27\\DLLs', 'C:\\Python27\\Lib\\lib-tk', 'C:\\Program Files (x86)\\Nuance\\NaturallySpeaking15\\Program', 'C:\\Python27', 'C:\\Python27\\lib\\site-packages', 'C:\\Python27\\lib\\site-packages\\win32', 'C:\\Python27\\lib\\site-packages\\win32\\lib', 'C:\\Python27\\lib\\site-packages\\Pythonwin', u'.', 'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice', u'C:\\Users\\Admin\\AppData\\Local\\caster\\rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\html_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\java_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\csharp_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\go_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\git_clients', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\editor\\vscode_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\editor', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\vhdl_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\mouse_grids', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\matlab_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\core\\utility_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\voice_dev_commands_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\dart_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\recording_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\python_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\editor\\sublime_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\speech_engine', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\asynch\\hmc_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\asynch\\sikuli', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\windows_os', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\core\\numbers_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\prolog_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\pdf', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\recording_rules\\alias', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\javascript_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\core\\navigation_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\rust_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\bash_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\microsoft_office', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\markdown_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\browser', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\file_manager', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\ccr\\haxe_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\editor\\eclipse_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\chat', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\apps\\file_manager\\totalcmd_rules', u'C:\\Users\\Admin\\Documents\\GitHub\\Caster\\castervoice\\rules\\core\\text_manipulation_rules']

But with my current implementation, If a user grammar Forgot the conditional import and imported directly with the longcastervoice.rules.apps.editor.sublime_rules.sublime_snippets then the controlled grammar gets broken!( not really sure but it seems as if these two grammars get two different copies of the same module oor something need to look into it Over the coming days)

[LexiconCode]

Oh interesting, I knew that core code was added to path. So the reason the conditional imports exist is all is for the user directory. I would've not predicted it would have issues in the source code receiving two modules because the imports come from the same place.

As you figure it out could you describe the methodology are using to test this and the issues you're seeing. I'd like to be able to get to a place where I can help you troubleshoot this.

From my understanding I'll try to restate your issue. The original file is already added to the sys path. When the conditional import is called it's except clause is never reached because the relative import is always true. The end result may be there's two instances of the module being referenced which may not be a identical.

The only time the relative import is not true is that the user does not copy the entire folder breaking the relative imports.

try: # Try first loading from caster user directory
    from eclipse_support import ec_con
except ImportError: 
    from castervoice.rules.apps.editor.eclipse_rules.eclipse_support import ec_con

Are you using a debugger? if not I can help you get that set up, attaching to a process or or a remote debugger.

[LexiconCode]

Also I took the liberty Of making these small change so when you get a chance to look at this let me know if that is an issue

I think this is a good move, so the question then becomes what happened to the grammar somehow becomes invalidated when refreshing? we could hold its previous state and a singleton and update that singleton when the rule is successfully validated.

Write documentation. My thought was to create a dedicated Snippets.md file as there will probably be Quite a few things to document. Would that be okay? Furthermore, I was in easily considering using gifs but from what I see the rest of the documentation is just plain text and Use links for video/demonstrations hosted elsewhere.

Feel free to use whatever medium you would like. Gifs could be stored in the local repository. I've recently gained control over the caster YouTube channel. I have a few things to figure out yet but that could be used as well for video hosting.