Confirmations for new pattern after semantic-kernel==0.9.2b1

[HuskyDanny]

The change is quite different from 0.5.1. I have followed the blog and sample code to rewrite my own, even though it finally worked, I feel something is weird. So posting here to share my way of doing it and would like to know what is the recommended way to do it. More will be added...

1. Set service for skprompt.txt skills
   Right now I would do this everytime for a function to be invoked, not sure if this is the design to explicitly set which service to use?

execution_settings = AzureChatPromptExecutionSettings(
    service_id="azure_openai_chat35_service",
)
refactor_code = self.get_semantic_function(kernel, "skills", "Generate")
response = await refactor_code.invoke(
    kernel=kernel,
    question=question,
    settings=execution_settings,
    chat_history=str(chat_history),
)

If so, how can we set it right in the config.json to save the extra code redundancy.
2. Separate system_message and user_message for prompt skill.
For more info : #5547
3. Now we need to handle memory itself?
So I get it from the blog that now memory is used as plugin

embedding_gen = OpenAITextEmbedding(
    service_id="ada", ai_model_id="text-embedding-ada-002", api_key=api_key, org_id=org_id
)
kernel.add_service(embedding_gen)

memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=embedding_gen)
kernel.import_plugin_from_object(TextMemoryPlugin(memory), "TextMemoryPlugin")

And you can call it directly in the prompt:

{{recall 'something'}}

The problem is I might need to parse the result and reorganize it of the result of recall. There is no direct support for this, but here are some work around:

1. Have a class overwrite the TextMemoryPlugin, so that you could add the parsing logic in the recall function.
2. Have the memory instance and use it directly in the code. Not sure why use_memory() function is removed from kernel, so it seems we need to handle the memory on our own, instead of calling kernel.memory?

[moonbox3]

Hi @HuskyDanny,

1. The reason we've introduced the tighter coupling to service_id is because we now support a dictionary of prompt_execution_settings. Before you were only able configure essentially one service_id against the execution settings. Now, we require you to tell the AI service which service_id it should be using, so that it can get the required execution settings. You can see that this prompt_execution_settings for this sample have the key default. You can replace your service_id here.
2. We fixed this after 0.9.2b1. Please use 0.9.4b1. If you only provide a simple prompt, like a sentence, it will be treated as a user message and not a system message. The chat_history should respect the roles for the messages that it contains. If you're having a problem with this, let us know.
3. Yes, the memory connector is now used as a plugin so that the kernel can support multiple "memories." If I understand your question/issue correctly for this, you would be able to handle the result of the recall with the current work we have underway related to function hooks. Again, related to "Not sure why use_memory() function is removed from kernel, so it seems we need to handle the memory on our own, instead of calling kernel.memory?": if the kernel had one memory attribute, we wouldn't be able to support multiple "memories" via plugins. Related to "The problem is I might need to parse the result and reorganize it of the result of recall." could you go into a little more depth about how you need to parse the result and reorganize it. We can then discuss if there is a way we could improve how it's currently handled. Thanks.

[HuskyDanny]

Thanks @moonbox3 , now I understand how I can reduce redundancy for #1.

For #2, I have replied back the the issue I linked.

For #3, I get it the support of multiple memories. So for my scenario, I want the retrieved chunks to be in the format:

• Chunk:chunk_id|Document:document_id|chunk_source|chunk_content
  &&&
• Chunk:chunk_id|Document:document_id|chunk_source|chunk_content

Where I need to 1) format the chunk in this way. 2) pull out some metadata stored in each chunk. Currently, it seems the textmemory plugin doesn't provide such customization on how the memory results wil be presented in the propmt.

[eavanvalkenburg]

@HuskyDanny on #3, a couple of things, 1 we currently do not yet have a way to "map" fields in your database to fields in MemoryRecord, this is something that I would like to see but we have not gotten around to that.
Having said that, the TextMemoryPlugin, is a plugin with functions, so you can do something like this (simplified):

template = "{{chat_history}} Use this context: {{recall}} to answer this question: {{question}}" //assumed to be in a func called 'generate.skills'
search_results = await kernel.invoke(function_name='recall', plugin_name='TextMemoryPlugin', question=question)
parsed = parse_search_result_func(search_results)
result = await kernel.invoke(function_name='generate', plugin_name='skills', question=question, recall=parsed, chat_history=chat_history)

I am currently working on Hooks, so then you could add the parsing and adding to arguments to the hook and then just call:

recall_func = kernel.func(function_name='recall', plugin_name='TextMemoryPlugin')
generate_func = kernel.func(function_name='generate', plugin_name='skills')
result = await kernel.invoke([recall_func, generate_func], question=question, chat_history=chat_history)

of with a slightly different hook and template:

template = "{{chat_history}} Use this context: {{TextMemoryPlugin.recall question=question}} to answer this question: {{question}}" //assumed to be in a func called 'generate.skills'
result = await kernel.invoke(function_name='generate', plugin_name='skills', question=question, recall=parsed, chat_history=chat_history)

[HuskyDanny]

Hi @eavanvalkenburg, I did try your first proposal where the default TextMemoryPlugin would json dumps the memory records to pure string which makes it harder to reformat.

Can we add more abstractions to the default TextMemoryPlugin's recall function where recall() would call internal functions fetch() and parse() So I could just extend a class to overwrite the parse() function for my case?

I could make the change to the TextMemoryPlugin, what do you think?

[eavanvalkenburg]

I like that idea, I was reviewing some things and see a lot of small things that I would love to update (also found a small bug in service selector, fixed here: #5687), I think for now, what we should do is change recall in TextMemoryPlugin to return a list of MemoryQueryResult, so that you can parse that yourself when you get the function results, we are no longer bound to strings only, so that should work, but you can easily recreate the TextMemoryPlugin locally to verify.

Then if you could create a issue describing this, then we can discuss this internally and see if there are considerations regarding parity with dotnet, or that we can just do work on it, then link a PR to that issue.

Does that work? and thanks in advance for showing this and working with us to improve it!