Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Translation fails due to Semantic Kernel tokenization error #11

Open
skytin1004 opened this issue Sep 21, 2024 · 0 comments
Open

Translation fails due to Semantic Kernel tokenization error #11

skytin1004 opened this issue Sep 21, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@skytin1004
Copy link
Collaborator

skytin1004 commented Sep 21, 2024

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

WARNING:semantic_kernel.template_engine.template_tokenizer:Failed to tokenize code block: .Prompt. A FunctionIdBlock is composed of either a plugin name and function name separated by a single dot, or just a function name. Both plugin and function names can only contain letters, numbers and underscores. The content provided was: .Prompt
ERROR:co_op_translator.translators.markdown_translator:Error in prompt 1/1 -

Expected/desired behavior

The translation process should successfully handle variables like .Prompt without causing tokenization errors, and all documents should be translated correctly.

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

co_op_translator 0.2.1

Mention any other details that might be useful


This issue occurs because the {{}} curly braces in the document being translated are recognized as variables by the Semantic Kernel. When the template contains these braces, the Semantic Kernel tries to interpret them as variables, causing errors if they don't match any defined function or variable.

To resolve this issue, ensure that any {{}} braces in the documents are either escaped or replaced with alternative text before translation, and restored afterward if needed.

Thanks! We'll be in touch soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant