How to handle deprecation and breaking changes

[WillDaSilva]

I was using a generated Python client for an API that had a deprecated password field. This generated client was being used in our production environment. Then the API removed the deprecated password field. We expected that this would have no effect on us because we weren't using it.

Unfortunately it made it so that our Python client for the API could no longer deserialize the response into the model for it, so when we accessed the host field of the response (i.e. response.host) an AttributeError was raised. This is because when the generated Python client fails to deserialize a response into a model, it returns a dictionary instead. The dictionary had the host field we were trying to access, but it would need to be accessed via __getitem__, i.e. response['host'] instead of response.host. This caused an outage in production until we updated our Python client to expect the new response that lacked the password field.

We're now trying to find a way to avoid this sort of outage in the future. One way I can think to avoid this problem would be to have the generated client do something like so:

def __getattr__(self, key: str):
    try:
        return self.__getattribute__(key)
    except AttributeError as ex:
        try:
            return self.__getitem__(key)
        except KeyError:
            raise ex

Using response.host as an example, this would first try to access the field normally, response.host, and then fall back to response['host'] if response.host raised an AttributeError. If the fallback doesn't work, the original exception is raised.

Along with an approach like this, logging a warning that the response could not be deserialized would be valuable.

If something like this were implemented, then instead of our system experiencing an outage when the API we rely on removed a field, we would have stated to see warnings that the API client is failing to deserialize the responses. Then we could've updated the client to reflect the removed field, and experienced no downtime.

[dbanty]

Handling breaking changes with REST APIs is definitely difficult, and I think designing a proper solution will take some discussion.

This is because when the generated Python client fails to deserialize a response into a model, it returns a dictionary instead

This doesn't sound quite right—at least for the intended behavior of the generator. If validation outright fails because a required property is missing, the API function should return None (or in some cases raise an exception), not return a dictionary. There is an additionalProperties feature that makes generated models act like dictionaries for certain dynamic properties, but that doesn't sound like the same thing.

have the generated client do something like so:

We could add some sort of dynamic look-up functionality to fall back to, but that will have a few issues:

1. It moves the error to attribute access time instead of deserialization time, which will be much more confusing and hard to debug when a breaking change that matters is made.
2. Validating return types disappears. If property something is supposed to be a class of type Something, but it couldn't deserialize, now when we dynamically look it up from the dict (because it isn't set as a property), it returns an unexpected object. That leads to more confusing runtime bugs later.
3. There's almost definitely a performance penalty—maybe it's small enough to not matter, but we'd have to measure it to decide if it's worth it

Other possible solutions

In general, making breaking changes to REST APIs is difficult—it's one of the main reasons GraphQL is used. Typically, you add a new endpoint with a /v2 or something, then move the clients to this endpoint (which no longer has the field you're removing, or whatever). Once no one is calling /v1 anymore, you can delete that entire endpoint. That's quite burdensome, so I definitely get the desire to have a more fluid versioning system. Here are the options I've seen.

Make everything optional

This is the approach of openapi-generator (last I checked). Always assume that any field could be missing, even if the document says it's required. Then, if something gets removed, all well 🤷. The trouble is, now everything is Optional and clients need to do a ton of type-checking themselves even when unnecessary if they want to be type-safe.

In my opinion, APIs who want to take this approach should do it in the document, rather than code generators making that choice for you and your users. So to implement that for this generator, go delete every required field in your OAS.

Make deprecated fields optional

We could automatically make deprecated fields optional (Union[T, Unset]). However, this would still require that consumers have regenerated their code sometime in between the server marking the field as deprecated and it removing the field—so this doesn't quite solve the problem. Consumers have to know that they should update, and API maintainers have to know that all clients have updated before they pull the plug.

Again, API maintainers can do this today without a change to this project—just also remove the deprecated field from required.

Only deserialize the fields being used

This is sort of like what you suggested above, but rather than moving everything to access-time validation, have consumers write a sub-model of the model they're interested in using. I'm not sure if it's even possible to represent this type-safely with Python's type system. In any event, this sounds like just a worse GraphQL, so I don't think it's worth pursuing 😬.

---

All of this is to say, I don't see a clear path forward to a feature for this project that would make the deprecation dance any easier—but I think it's a super interesting topic that should be discussed more. I'm definitely open to ideas, but they'd need to improve deprecation without hurting the standard developer experience (type annotations can't get less accurate, errors can't get harder to understand). So... what do folks think? Is there any prior art from other client generators that we can take inspiration from?

I should also say, in general this project takes the stance that the document should be accurate, and the client shouldn't do too much to compensate for that. This is why, for example, there is a distinction between a property being nullable and a property being not required—even though many APIs document the behavior incorrectly.