also happens in 2024.7

Do you get the same error with empty string? The following works for me. Notice that empty string is null, not string literal null

mqtt:
  # broker: broker.emqx.io # This server gives many TLS errors. 
  broker: 192.168.1.42
  port: 9993
  client_id: ${name}
  username: bblp
  password: 12345678
  log_topic:
  discover_ip: false
  discovery: false
  discovery_retain: false
  discovery_prefix:
  use_abbreviations: false
  topic_prefix: testtopic/1234

And empty string gives

  not a valid value.
  log_topic: ''

Note that log_topic: null is not a string literal. It is a real null.
A string literal would look like 'null' (quotes to force the string)

The validation for this in the code is here:
https://github.com/esphome/esphome/blob/efe4c5e3bc496d9f1f4d152c28a27e505b9f3024/esphome/components/mqtt/__init__.py#L249

But this will fail because the topic is required:
https://github.com/esphome/esphome/blob/dev/esphome/components/mqtt/__init__.py#L75

And when it does get parsed it will become empty string anyway:
https://github.com/esphome/esphome/blob/efe4c5e3bc496d9f1f4d152c28a27e505b9f3024/esphome/config_validation.py#L1181

But this is just validation. The show_logs function isn't using this at all, and imo we are getting too far too late if it thinks it can use mqtt:
https://github.com/esphome/esphome/blob/dev/esphome/__main__.py#L369

The reason it thinks we should be using mqtt is because it is the only option available when we get to show_logs.
It doesn't pick api because you don't have api: in your config (like I don't) because the documentation says to remove it: https://esphome.io/components/mqtt.html

At this point I'm not sure what to do. Maybe it could print a friendlier error message, but fundamentally there are no show_logs options that will work. Too bad it can't use whatever method the web interface uses?