Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Int dtypes are unnecessarily big #688

Open
Happily-Coding opened this issue Sep 26, 2024 · 0 comments
Open

Int dtypes are unnecessarily big #688

Happily-Coding opened this issue Sep 26, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@Happily-Coding
Copy link

Happily-Coding commented Sep 26, 2024

What language are you using?

Python

What version are you using?

0.3.3

What database are you using?

MySQL

What dataframe are you using?

Polars

Can you describe your bug?

Connectorx pointlessly transforms smaller db dtypes into int64, leading to higher memory usage.
See mysql mappings or MSSQL mappings

What are the steps to reproduce the behavior?

Perform a query to mysql using connectorx to a table containing a smaller dtype in a column.
The df will have int64 dtypes in all numeric dtypes, as specified by the incorrect transports linked above.

For example with polars which respectes the dtypes provided by connectorx:

import polars as pl
mysql_uri = f'{mysql_db_type}://{mysql_username}:{mysql_password}@{mysql_host}:{mysql_port}/{mysql_database_name}'
test_dataset = pl.read_database_uri(query, self.mysql_uri)

What is the error?

There will be no error message, just incorret dtypes.

@Happily-Coding Happily-Coding added the bug Something isn't working label Sep 26, 2024
@wangxiaoying wangxiaoying added enhancement New feature or request good first issue Good for newcomers and removed bug Something isn't working labels Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants