Moving back to C++ Exceptions

[stephenberry]

Note

This is a long term discussion looking years into the future for Glaze. A change will not be made unless we can ensure determinism on the error path and achieve reasonable error handling performance with exceptions.

Glaze should perhaps move back to using C++ exceptions. After experimenting for the past couple years disabling exceptions in Glaze, it has become apparent that this was perhaps not a good choice, and results in code that is slower, more complex, and less safe.

This talk is extremely helpful for viewing C++ exceptions and how to use them on embedded systems: C++ Exceptions for Smaller Firmware - Khalil Estell - CppCon 2024

Exceptions are faster

When we originally moved way from using exceptions in Glaze we saw on average a 5% - 10% loss in performance. Exceptions allow a reduction in branching, less binary overhead for large projects, and code that is easier to inline without cost. Glaze has needed to implement more macros under the hood in order to maintain performance, which makes debugging more difficult.

Glaze already avoids std::expected in the internal logic for performance reasons and only uses it at the top level API.

Debugging is easier with exceptions

IDEs can break upon an exception being thrown. However, error codes do not trigger IDE breaks at the error location, which makes debugging and finding errors more difficult.

Exceptions produce less binary in many cases

See Khalil Estell's talk linked above

We have seen this when developing code for Glaze on compiler explorer, and noticed it from performance losses due to additional branching and error code checks.

Exceptions often result in safer code

I have interacted with users of Glaze who use void casting (void)write_json(... or std::ignore to discard errors. In many cases (especially writing) errors are extremely rare and therefore this is fine 99% of the time. However, hiding the 1% case is extremely dangerous. Exceptions allow these corner case errors to be properly handled without having to think about them and without them complicating code that is written everywhere. Also, many libraries like the STL just terminate when an exception would be raised, which makes code much less safe.

I've experienced issues with not checking an expected return and just calling .value(). This turns into an exception when exceptions are enabled, but when they are disabled it becomes a termination. The need to always check if the expected state has a value adds another layer of potential errors and sometimes results in major issues that wouldn't have occurred if exceptions were used.

Using exceptions works better with the STL

The Standard Template Library uses exceptions everywhere, and when -fno-exceptions is used exceptions are turned into terminations. The STL is extremely useful for developing cross platform, efficient code.

We have found that needing to require two separate approaches to error handling (exceptions and error codes) results in more complex code, more lines of code, and more binary that runs slower.

Right now there are a number of functions labeled noexcept that could call user defined types that throw exceptions. One could argue that this is just a labelling issue, but it is a side effect of trying to avoid exceptions.

Using exceptions results in cleaner code

Using exceptions majorly reduces code bloat and the need for ugly macros.

glz::prettify_json is an example of where we just return a std::string, because if it is taking a Glaze generated buffer then it should never error, but it could error. However, this error isn't reported because it would be annoying for the API. Currently users can use the internal API, but this just goes to show the problem of not using exceptions.

Consider this large macro in Glaze for skipping whitespace. The only reason this can't be an inlined function is that we need to be able to return out of the function on error without creating an intermediate boolean that adds performance overhead and binary. We found that compilers like GCC can sometimes remove nested returns, but in practice there is a real performance overhead and additional code complexity that can be avoided with the macro. Exceptions would allow this to be a normal C++ function that can be debugged.

#define GLZ_SKIP_WS(RETURN)                                              \
   if constexpr (!Opts.minified) {                                       \
      if constexpr (Opts.null_terminated) {                              \
         if constexpr (Opts.comments) {                                  \
            while (whitespace_comment_table[uint8_t(*it)]) {             \
               if (*it == '/') [[unlikely]] {                            \
                  skip_comment(ctx, it, end);                            \
                  if (bool(ctx.error)) [[unlikely]] {                    \
                     return RETURN;                                      \
                  }                                                      \
               }                                                         \
               else [[likely]] {                                         \
                  ++it;                                                  \
               }                                                         \
            }                                                            \
         }                                                               \
         else {                                                          \
            while (whitespace_table[uint8_t(*it)]) {                     \
               ++it;                                                     \
            }                                                            \
         }                                                               \
      }                                                                  \
      else {                                                             \
         if constexpr (Opts.comments) {                                  \
            while (it < end && whitespace_comment_table[uint8_t(*it)]) { \
               if (*it == '/') [[unlikely]] {                            \
                  skip_comment(ctx, it, end);                            \
                  if (bool(ctx.error)) [[unlikely]] {                    \
                     return RETURN;                                      \
                  }                                                      \
               }                                                         \
               else [[likely]] {                                         \
                  ++it;                                                  \
               }                                                         \
            }                                                            \
            if (it == end) [[unlikely]] {                                \
               ctx.error = error_code::end_reached;                      \
               return RETURN;                                            \
            }                                                            \
         }                                                               \
         else {                                                          \
            while (it < end && whitespace_table[uint8_t(*it)]) {         \
               ++it;                                                     \
            }                                                            \
            if (it == end) [[unlikely]] {                                \
               ctx.error = error_code::end_reached;                      \
               return RETURN;                                            \
            }                                                            \
         }                                                               \
      }                                                                  \
   }

This is a working document to track my thoughts on exceptions and will continue to be updated.

[20162026]

I'm an embedded engineer who recently stumbled upon Glaze and was about to use it in a project, but mandatory exceptions are kind of a red flag.
I do understand that using standard containers without exceptions is extremely difficult, but I would really like to keep some kind of noexcept way to use Glaze, even if it's slower e.g. using a fixed buffer with length checks instead of strings.

For me, the main drawback of exceptions is the time it takes to unwind them in case of a sad path. It is difficult (if even possible) to calculate the exact time it would take to process an exception, which makes it unfeasible in real-time code, where you prioritize deterministic behaviour above everything else.
On the other hand, I don't know how often someone would really need to parse JSON in time-critical code paths or how bad it really is to use a simple exception that could be replaced with jump by compiler.

It might be fine to use exceptions for extremely rare and unexpected cases, e.g., when memory allocation fails, but using them for control flow in JSON parsing/validation is not ideal

[stephenberry]

I agree with you: it is important to have the error case for a parser performant and deterministic.

If you watch Khalil's talk, he's working on a tool to solve the deterministic problem, and will hopefully get his library into GCC and Clang.

This will take time and I want to see if it is achievable before changing Glaze back to exceptions. I agree that determinism in knowing the duration of the exception calls is critical.

I don't want to make this change lightly and so this is very much long term planning and discussion. It would probably be two or more years until this change were to be released, and it would be done only when it is shown to be robustly supported on at least GCC and Clang on Arm and x86.