Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sequence length truncation isn't working #82

Open
kylebgorman opened this issue Dec 28, 2024 · 0 comments · May be fixed by #85
Open

Sequence length truncation isn't working #82

kylebgorman opened this issue Dec 28, 2024 · 0 comments · May be fixed by #85
Labels
bug Something isn't working

Comments

@kylebgorman
Copy link
Contributor 

Token indices sequence length is longer than the specified maximum sequence length for this model (585 > 512). Running this sequence through the model will result in indexing errors
modules.py WARNING: 28-Dec-24 17:01:58 - Truncating sequence from 585 to 512
...
  File "/home/kbg/.miniconda3/envs/udtube/lib/python3.12/site-packages/transformers/models/bert/modeling_bert.py", line 1073, in forward
    buffered_token_type_ids_expanded = buffered_token_type_ids.expand(batch_size, seq_length)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The expanded size of the tensor (585) must match the existing size (512) at non-singleton dimension 1.  Target sizes: [32, 585].  Tensor sizes: [1, 512]

Somehow the "target size" on dimension 1 has not been properly truncated despite the warnings that it has been.

It should be easy to simulate this error by feeding it a very long sentence (e.g., over 512 pieces on English BERT).
@kylebgorman kylebgorman added the bug Something isn't working label Dec 28, 2024
kylebgorman added a commit to kylebgorman/udtube that referenced this issue Dec 30, 2024
@kylebgorman
Sequence length tests
318a574 
This is currently failing at CUNY-CL#82.
@kylebgorman kylebgorman linked a pull request Dec 30, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Sequence length tests kylebgorman/udtube
1 participant
@kylebgorman