Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to_dict incorrectly returns base64 encoded results for bytes type #478

Open
Ark-kun opened this issue Jul 15, 2024 · 0 comments
Open

to_dict incorrectly returns base64 encoded results for bytes type #478

Ark-kun opened this issue Jul 15, 2024 · 0 comments
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@Ark-kun
Copy link

Ark-kun commented Jul 15, 2024

Message.to_dict returns bytes fields as Base-64 encoded strings instead of proper bytes strings. It also returns integers as strings instead of int.

Environment details

  • Programming language: Python
  • OS: Linux
  • Language runtime version: 3.10
  • Package version: 1.24

Steps to reproduce

model = GenerativeModel("gemini-pro")
compute_tokens_response = model.compute_tokens(["Hello world!", "How are you?"])
token_info = compute_tokens_response.tokens_info[1]
>>> token_info

tokens: "How"
tokens: " are"
tokens: " you"
tokens: "?"
token_ids: 2299
token_ids: 708
token_ids: 692
token_ids: 235336
role: "user"
>>> type(token_info).to_dict(token_info)

{'tokens': ['SG93', 'IGFyZQ==', 'IHlvdQ==', 'Pw=='],
 'token_ids': ['2299', '708', '692', '235336'],
 'role': 'user'}
@Ark-kun Ark-kun added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

1 participant