AI Assistant: Get feedback on AI responses

[christad92]

Here are a few things we want to do:

• Add a flag that lets user report an unsatisfactory response with comment.
• A posthog survey to collect experience.

[josephjclark]

This needs more design detail. But this is required before general release (especially if we're not going to post chat logs to slack yet)

[christad92]

The ideal solution here for me is to allow users to flag a response from the assistant. We should have a text under the responses that say, flag this response and on click, it should open a modal that asks the user to select the reasons(s) for flagging the response- incorrect information, incomplete answer, irrelevant answer, confusing or unclear, too long or verbose answer, too short or lacking detail, outdated information, not based on documentation, formatting issues, and other.

This "flag this answer" feature should only show when a user hovers over the answer from the assistant.

When flagging a response, users can select more than one reason and should also provide more details about their reason for flagging the response. The responses will be saved in the db (maybe in a new table).

[christad92]

@josephjclark what do you think about this?

[elias-ba]

Here is how Claude does it (same as your proposal @christad92), and I think it's really cool.

Click on these

For positive feedback

For negative feedback

[josephjclark]

@christad92 to investigate whether we can attach the AI-chat session ID to a posthog survey.

Then we can have a really simple flow like:

• User clicks "Give feedback on / report this answer"
• We pop up a posthog survey
• We attach the session id to posthog so we can track it
• we ask a few questions
• job done