No system prompt for Anthropic?

[eterps]

Am I right in assuming that currently there is no support for system_prompt for Antropic provider?

parrot.nvim/lua/parrot/provider/anthropic.lua

Line 50 in 2719099

function Anthropic:add_system_prompt(messages, _)

[eterps]

Hmmm, it seems it should support system_prompt, however I doubt if it works.
What is the best way to check this?
I see there are logger statements, do I need to activate logging somehow? And where does it go?

Maybe obivous questions for people who have authored nvim plugins more often, but I'm a bit in the dark on this.

[frankroeder]

Anthropic expects the system prompt inside the curl parameters (https://docs.anthropic.com/en/api/messages) and not inside the messages with role = system, like the OpenAI API.

Yes, it definitely would be helpful to write a comment or document it somewhere.
By default, all providers have the system prompt defines as OpenAI. However, for Anthropic I implement this exception by removing the default system prompt

parrot.nvim/lua/parrot/provider/anthropic.lua

Line 43 in 2719099

function Anthropic:preprocess_messages(messages)

and adding it later to the curl call via the agent.model

parrot.nvim/lua/parrot/agents.lua

Line 119 in 2719099

model = { model = "claude-3-5-sonnet-20240620", max_tokens = 4096, system = system_chat_prompt },

You can see my print statements for the messages, header and agent.model below, comparing Anthropic and OpenAI:

I hope this helps.

[eterps]

Thanks for the explanation @frankroeder , it seems the system prompt doesn't end up in the curl params.
I'll see if I can find out why. These are the params in my case:

parrot.nvim: debug_curl_params: { "--no-buffer", "--silent", "-H", "accept: application/json", "-H", "content-type: application/json", "-d", '{"messages":[{"content":"2+2=?","role":"user"}],"model":"claude-3-5-sonnet-20240620", "max_tokens":4096,"stream":true}', "https://api.anthropic.com/v1/messages", "-H", "x-api-key: xxx", "-H", "anthropic-version: 2023-06-01" }

[frankroeder]

Hi @eterps, they definitely end up in the request, otherwise the topic model would not be able to generate a 2-3 word topic headlines for the chats. Have a look at the prepare_payload utils function and the output it generates. Search for it in the init.lua file.

BTW, I think it is always good to double proof things. I really appreciate your effort!

[eterps]

Ok, found some more details.
The system prompt does not always end up in the curl request.

For instance, this config results in the curl params below it:

{
  "frankroeder/parrot.nvim",
  dependencies = { 'ibhagwan/fzf-lua', 'nvim-lua/plenary.nvim' },
  opts = {
    providers = {
      anthropic = { api_key = os.getenv "ANTHROPIC_API_KEY" }
    }
  }
}

parrot.nvim: curl_params: { "--no-buffer", "--silent", "-H", "accept: application/json", "-H", "content-type: application/json", "-d", "{\"max_tokens\":4096,\"stream\":true,\"model\":\"claude-3-haiku-20240307\",\"system\":\"You are a versatile AI assistant with capabilities\\nextending to general knowledge and coding support. When engaging\\nwith users, please adhere to the following guidelines to ensure\\nthe highest quality of interaction:\\n\\n- Admit when unsure by saying 'I don't know.'\\n- Ask for clarification when needed.\\n- Use first principles thinking to analyze queries.\\n- Start with the big picture, then focus on details.\\n- Apply the Socratic method to enhance understanding.\\n- Include all necessary code in your responses.\\n- Stay calm and confident with each task.\\n\",\"messages\":[{\"role\":\"user\",\"content\":\"Hi\"}]}", "https://api.anthropic.com/v1/messages", "-H", "x-api-key: xxx", "-H", "anthropic-version: 2023-06-01"}

parrot.nvim: curl_params: { "--no-buffer", "--silent", "-H", "accept: application/json", "-H", "content-type: application/json", "-d", "{\"max_tokens\":32,\"stream\":true,\"model\":\"claude-3-sonnet-20240229\",\"system\":\"Summarize the topic of our conversation above\\nin two or three words. Respond only with those words.\\n\",\"messages\":[{\"role\":\"user\",\"content\":\"Hi\"},{\"role\":\"assistant\",\"content\":\"Hello! How can I assist you today? I'm here to help with any questions or tasks you may have. Please feel free to provide me with more details about what you need, and I'll do my best to provide a helpful and thoughtful response.\"},{\"role\":\"user\",\"content\":\"You only respond with 2 to 3 words to summarize the past conversation.\"}]}", "https://api.anthropic.com/v1/messages", "-H", "x-api-key: xxx", "-H", "anthropic-version: 2023-06-01" }

While this config results in the curl params below (notice the system prompt missing in the first, but still present in the second curl request):

{
  "frankroeder/parrot.nvim",
  dependencies = { 'ibhagwan/fzf-lua', 'nvim-lua/plenary.nvim' },
  opts = {
    providers = {
      anthropic = { api_key = os.getenv "ANTHROPIC_API_KEY" }
    }
    agents = {
      chat = {
        {
          name = "CustomClaude",
          provider = "anthropic",
          model = {
            model = "claude-3-5-sonnet-20240620",
            max_tokens = 4096
          },
          system_prompt = "Help me!"
        }
      }
    }
  }
}

parrot.nvim: curl_params: { "--no-buffer", "--silent", "-H", "accept: application/json", "-H", "content-type: application/json", "-d", '{"max_tokens":4096,"messages":[{"content":"Hi","role":"user"}],"model":"claude-3-5-sonnet-20240620","stream":true}', "https://api.anthropic.com/v1/messages", "-H", "x-api-key: xxx", "-H", "anthropic-version: 2023-06-01" }

parrot.nvim: curl_params: { "--no-buffer", "--silent", "-H", "accept: application/json", "-H", "content-type: application/json", "-d", "{\"max_tokens\":32,\"messages\":[{\"content\":\"Hi\",\"role\":\"user\"},{\"content\":\"Hello! How can I assist you today? Feel free to ask any questions or let me know if there's anything you'd like to discuss.\",\"role\":\"assistant\"},{\"content\":\"You only respond with 2 to 3 words to summarize the past conversation.\",\"role\":\"user\"}],\"stream\":true,\"model\":\"claude-3-sonnet-20240229\",\"system\":\"Summarize the topic of our conversation above\\nin two or three words. Respond only with those words.\\n\"}", "https://api.anthropic.com/v1/messages", "-H", "x-api-key: xxx", "-H", "anthropic-version: 2023-06-01" }

I'll see if I can locate the problem, but if you have an idea what might be wrong I would appreciate it.

[eterps]

@frankroeder, I need clarification on the provider configuration behavior. When a user configures the providers section of their configuration as follows:

providers = {
    anthropic = { api_key = os.getenv "ANTHROPIC_API_KEY" },
    openai = { api_key = os.getenv "OPENAI_API_KEY" }
 },

The intuitive expectation would be that only agents from these two specified providers (Anthropic and OpenAI) would be available for use. This would imply that the merging of providers only combines the default configurations for anthropic and openai, while excluding all other providers.

Would you agree that this is the behaviour a user would expect? Or did you have another mechanism in mind to 'not use' certain providers?

[frankroeder]

Hi, I am already working on this on a different local branch, only making the providers available the user configures correctly, hence providing the API key or something along that lines.

[eterps]

Hi, I am already working on this on a different local branch, only making the providers available the user configures correctly, hence providing the API key or something along that lines.

Yes, I am also on that branch, that's why I am asking, because this loop also merges providers the user didn't configure:

https://github.com/frankroeder/parrot.nvim/pull/23/files#diff-8f49ff122fed688344cbdc1791680b557dabd81af784a9afa96925fa297e22c0R52

I would assume that if the table providers omits certain keys (like pplx or mistral for example) those defaults shouldn't be copied in. But currently it does.

only making the providers available the user configures correctly

Whether a configuration is 'correct' is a separate issue I would think. An 'absence' of a provider is not invalid IMO.

But maybe I am misunderstanding things, I am just happy to help out if possible.

[frankroeder]

That is actually a good idea. Yes, my bad. Validation should be happening at a different stage. Let us take this discussion to the pull request and feel free to push your changes.

[eterps]

Cool, thanks. As a result, I believe it will also allow the code to be much simpler in the end. Let's take this slowly to come up with a good solution. I will continue the discussion in the pull request from now on.