Copilot Chat - Can it browse the web like ChatGPT (GPT-4) ?

[adamklepacz]

Select Topic Area

Question

Body

Hello GitHub Community,

I have a question regarding the capabilities of GitHub's Copilot Chat. It's quite an interesting feature that adds a more interactive dimension to the coding process.

Given the versatility of AI language models, I was wondering if Copilot Chat has the same capability as ChatGPT (GPT-4) to access and retrieve real-time information from the web? I understand that language models like GPT-4, though they don't have the ability to access or browse the internet in real-time, they can generate responses based on a vast dataset they were trained on, giving the illusion of internet browsing.

Could anyone clarify whether Copilot Chat employs a similar mechanism? Can I paste a link to a framework documentation and Copilot Chat for explanation of a certain term?

I look forward to learning more about this topic, and any references or links to resources that could shed more light on this would be extremely appreciated.

Thank you in advance for your insights!

Best Regards,
Adam

[Akash1134]

Hi @adamklepacz ,

Thanks for asking such a great question :)

The system behavior of Copilot Chat can be broken down into several key steps.

Input processing

The input prompt from the user is pre-processed by the Copilot Chat system and sent to a large language model to get a response based on the context and prompt. User input can take the form of code snippets or plain language. The system is only intended to respond to coding-related questions.

Language model analysis

The pre-processed prompt is then passed through the Copilot Chat language model, which is a neural network that has been trained on a large corpus of text data. The language model analyzes the input prompt and generates a response based on its understanding of the syntax and semantics of the input.

Response generation

The language model generates a response based on its analysis of the input prompt. This response can take the form of generated code, code suggestions, or explanations of existing code.

Output formatting

The response generated by Copilot Chat is formatted and presented to the user in a way that is easy to read and understand. This may include syntax highlighting, indentation, and other formatting features.

Overall, the system behavior of Copilot Chat is designed to be intuitive and user-friendly, allowing you to ask coding-related questions and complete coding tasks quickly and easily.

[adamklepacz]

@Akash1134 But you stil didn't answered on:

Can I paste a link to a framework documentation and Copilot Chat for explanation of a certain term?

Is Copilot Chat able to read from a URL pasted by a developer in a prompt?

For example:

Based on this recently released CSS feautre that is described over here: https://web.dev/viewport-units/

Please modify my div.wrapper container so it's uses these new viewporst units to handle mobile devices.

Here is the code to modify:
 [CSS CODE]
 

Will Copilto Chat be able to read from https://web.dev/viewport-units/, and answer based on the content from that URL??

[Foxhoundn]

Hi @adamklepacz ,

Thanks for asking such a great question :)

The system behavior of Copilot Chat can be broken down into several key steps.

Input processing

The input prompt from the user is pre-processed by the Copilot Chat system and sent to a large language model to get a response based on the context and prompt. User input can take the form of code snippets or plain language. The system is only intended to respond to coding-related questions.

Language model analysis

The pre-processed prompt is then passed through the Copilot Chat language model, which is a neural network that has been trained on a large corpus of text data. The language model analyzes the input prompt and generates a response based on its understanding of the syntax and semantics of the input.

Response generation

The language model generates a response based on its analysis of the input prompt. This response can take the form of generated code, code suggestions, or explanations of existing code.

Output formatting

The response generated by Copilot Chat is formatted and presented to the user in a way that is easy to read and understand. This may include syntax highlighting, indentation, and other formatting features.

Overall, the system behavior of Copilot Chat is designed to be intuitive and user-friendly, allowing you to ask coding-related questions and complete coding tasks quickly and easily.

what a nothingburger answer 😕, was it AI generated?

[adamklepacz]

And regarding this one: https://github.com/orgs/community/discussions/56396

You closed this as a duplicate, but I didn't get the answer. Is Copilot Chat using GPT-4 language models or does is use GPT-3.5 ? or does it use a custom GPT fine-tuned model?

[Akash1134]

Hi @adamklepacz ,

Currently, GitHub Copilot uses OpenAI Codex. With our vision GitHub Copilot is evolving with Copilot X as our vision for the future of AI-powered software development. We are not only adopting OpenAI’s new GPT-4 model, but are introducing chat and voice for Copilot, and bringing Copilot to pull requests, the command line, and docs to answer questions on your projects.

[avxkim]

When do you plan to adopt GPT-4?

[skavinvarnan]

I'm using Github Copilot X, and it seems smarter than Copilot. Both of them use OpenAI Codex, then what makes one smarter than the other?

[Josikcodes]

[restlessronin]

I just asked in the VS Code copilot-chat

which version of Open AIs gpt are you based on?

and the answer was

I'm based on OpenAI's GPT-3 model.

[xLeay]

Same with VSCode

[zoff4ick]

UPD:

[jirawatcode]

why it is GPT-3 for me? I am on Visual Studio 2022

[ayim]

Per @digitarald in microsoft/vscode-copilot-release#6

Yes, both models are in play. Each model is being used where it is most effective. We'll have more nitty-gritty details in an upcoming blogpost, stay tuned.

[clintjansen]

Why is the question being ignored?
One can only assume it currently does NOT have this capability.
After some testing, it seems to indeed not be able to browse software docs.

I'm assuming this is so they have a distinct feature for paying OpenAI ChatGPT customers to keep it relevant from a business perspective.
Could become available later when it has more unique features, setting it more apart from Copilot

[bharat2808]

So I managed to create a web searching tool using OpenAI Api which uses GPT-3 model for processing raw HTML data and give real-time results like Copilot. I managed to do it using Selinium to search google and parse its content and use the relatable data to extract results from it. If it fails to get results from google html, then try scrapping the links one by one provided in the google search. The results goes through openAI api twice once to extract the answer and any related answer in the provided HTML. The second time to check if it found any answer so you can feed more data to it. Sending raw data this way will help you save a lot of tokens. I tried searching for all kind of real time data like live scores, weather information, current news, latest movies, songs and much more, and it works as intended. It gives you whatever you need without using any external APIs. Just play with the html data and see how you can extract reliable information from it and build something like that and it will work. If you want your results from an API then use tools like SERAPI and bing search which are more expensive to use.

good luck!

[rupert-jung-mw]

When I add a link as a reference it just answers with standard "I'm sorry for the misunderstanding, but as an AI developed by OpenAI, I don't have the ability to browse the internet or access external databases or websites in real-time. My responses are generated based on a mixture of licensed data, data created by human trainers, and publicly available data."

That's pretty useless.

[bharat2808]

It does that because, llm cannot browse. In the background, html data is parsed using traditional methods or an external web API and feed it to LLM in order to get the answers for the queries. If a website has strong protection against bots, sadly you won't be able to get any response by feeding it urls and you get this kind of response.

[rupert-jung-mw]

Should be rather trival, (Microsoft) Copilot can do that. At least for simple cases, a regular CURL in the background would be a beginning...

[ajhous44]

+1 on this. In the cursor IDE, we can add a url to the Azure Python SDK github library/docs for example, and it'll crawl the whole thing instantly and is able to be @ referenced in the text chat. How does Cursor IDE beat out a company as as big as github in functionality for something like this? Hm...