Skip to content

Commit

Permalink
Fix example gguf_locally (#966)
Browse files Browse the repository at this point in the history
The chat template used in the gguf_locally example only supports 'user'
and 'assistant' roles, but the example code includes a 'system' role
message. This commit removes the 'system' role message to allow the
example to run successfully.
  • Loading branch information
msk authored Dec 3, 2024
1 parent b1c5448 commit d731ca6
Showing 1 changed file with 4 additions and 9 deletions.
13 changes: 4 additions & 9 deletions mistralrs/examples/gguf_locally/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,10 @@ async fn main() -> Result<()> {
.build()
.await?;

let messages = TextMessages::new()
.add_message(
TextMessageRole::System,
"You are an AI agent with a specialty in programming.",
)
.add_message(
TextMessageRole::User,
"Hello! How are you? Please write generic binary search function in Rust.",
);
let messages = TextMessages::new().add_message(
TextMessageRole::User,
"Hello! How are you? Please write generic binary search function in Rust.",
);

let response = model.send_chat_request(messages).await?;

Expand Down

0 comments on commit d731ca6

Please sign in to comment.