New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

crop text in middle instead of decimate it #809

Open

wassname wants to merge 2 commits into josStorer:master from wassname:master

wassname commented Oct 29, 2024 •

edited

Loading

A few problems:

for cropping we were discarding every N sentence, this creates garbage text
we were calculating the max tokens wrong (or at least the logic didn't make sense to me)

So I've

change the cropText for the whole extension to remove the excess text from the middle and replace it with a warning "this text has had X% removed"
use \n instead of , for more natural text
change the way we count the token limit to help us use all of it

wassname added 2 commits

October 29, 2024 12:55


          crop transcript in middle instead of decimate it, use \n instead of ,


          better max Length

5a69226

wassname changed the title ~~crop transcript in middle instead of decimate it~~ crop text in middle instead of decimate it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet