Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about the reasoning part in paper #3

Open
dydxdt opened this issue Jan 7, 2024 · 5 comments
Open

Question about the reasoning part in paper #3

dydxdt opened this issue Jan 7, 2024 · 5 comments

Comments

@dydxdt
Copy link

dydxdt commented Jan 7, 2024

image Great job! I have a question about this part. What's the model used for reasoning? I read the paper and am not very sure about this. I suppose CIE is used for chart perception, so what's the model used in reasoning? It said GPT3.5, but I don't knwo how to use it to train for the downstream task? Can you explain it? Thx!
@dydxdt dydxdt changed the title Question about the paper Question about the reasoning part in paper Jan 7, 2024
@renqiux0302
Copy link
Contributor

Thanks for your recognition!
We have employed gpt-3.5-turbo as the reasoning model, which is zero-shot without training on customized data. Also, we have recently updated our reasoning model with a trainable open-source language model, which we believe will be open-source in the near future.
Hoping you stay tuned to our work. And feel free to contact us if there is any concerns for you.

@dydxdt
Copy link
Author

dydxdt commented Jan 9, 2024

Thanks! got it!

@dydxdt
Copy link
Author

dydxdt commented Jan 9, 2024

image I also have another question about the "self-inspection mechanism". Since I also have tried to use GPT4 to generate the drawing code and found it didn't always follow the instructions strictly and generated codes which couldn't run successfully, I am not clear about the mentioned "self-inspection mechanism". Can you explain more? Or can you give an example of the instructions? Thanks!

@renqiux0302
Copy link
Contributor

Of course. The "self-inspection mechanism" here refers to that, in the initial drawing code generation loop, we will self-inspect the executability of the drawing code, and save the CSV texts corresponding to the unexecutable code. For the next code simulation generation loop, we only deal with the CSV texts saved in the previous loop. And so on until all the texts have executable drawing code pair. In our experiment, the code simulation process for SimChart9k can be completed in three loops.
As you mentioned, the prompts and instructions are quite difficult to design for chart simulation. You can refer to our templates in Figs 4 and 5 of Appendix B for your own design.

@dydxdt
Copy link
Author

dydxdt commented Jan 9, 2024

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants