ModernBERT bug fixes #35404
ModernBERT bug fixes #35404
Conversation
| @@ -503,7 +503,7 @@ | |||
| - local: model_doc/mobilebert | |||
| title: MobileBERT | |||
| - local: model_doc/modernbert | |||
| title: ModernBert | |||
| title: ModernBERT | |||
There was a problem hiding this comment.
Good call - I got carried away with the Python class naming
| @@ -892,6 +900,13 @@ def _maybe_set_compile(self): | |||
| ) | |||
| self.config.reference_compile = False | |||
|
|
|||
| if self.device.type == "cpu": | |||
| logger.warning_once( | |||
There was a problem hiding this comment.
I think we should wrap this in if self.config.reference_compile: just like above so users don't get warned when they never manually specified anything related to reference_compile.
Unless you'd rather always give this warning so people aren't confused why there's a small precision-esque difference between GPU setups?
There was a problem hiding this comment.
That was my intention, I forgot the if statement. Fixed in ab11657.
| if config._attn_implementation_internal is None: | ||
| config._attn_implementation_internal = "flash_attention_2" | ||
| try: | ||
| return cls._check_and_enable_flash_attn_2( | ||
| config, | ||
| torch_dtype=torch_dtype, | ||
| torch_dtype=torch.float16, |
There was a problem hiding this comment.
I think this is a cleaner solution to avoid the unnecessary FP32 warning than I figured was possible, nice.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
This PR fixes a few issues with the ModernBERT implementation and some typos in the docs.
First, on the flash attention 2 path, ModernBERT was incorrectly always returning repadded logits without gradients (see #35386). This PR will return padded logits with gradients if no labels are passed to
ModernBertForMaskedLMor ifrepad_logits_with_gradis set to true. We don't keep the gradient when repadding after using an internal loss function by default to save memory.Second, this PR fixes torch.compile from being automatically set on when on CPU (#35388).
Third, I added some details to the model doc strings.
Fourth, documentation is updated to capitalize BERT in ModernBERT following the other BERT models.
cc @ArthurZucker @tomaarsen