Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: csv/tsv encoding #3369

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Conversation

jaluma
Copy link

@jaluma jaluma commented Jul 9, 2024

Testing Unstructured with CSV/TSV files, I have encountered problems when the file is not UTF-8 encoded. With this fix, it is intended to fix problems if the client passes the right encoding. Tested with ISO-8859-1 files and it works as expected :)

Crash error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf3 in position 80: invalid continuation byte

Copy link

sentry-io bot commented Jul 9, 2024

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: unstructured/partition/csv.py

Function Unhandled Issue
partition_csv UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 54: invalid continuation byte ...
Event Count: 3

Did you find this useful? React with a 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant