Add support for JSON output format

[chrysle]

Very WIP right now, I'll add (and fix) tests once everyone is satisfied with the implementation. Thus, I'd appreciate some feedback and suggestions.

Closes #1377.

Contributor checklist

• Included tests for the changes.
• PR title is short, clear, and ready to be included in the user-facing changelog.

Maintainer checklist

• Verified one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
• Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

[AndydeCleyre]

Thanks!

I think the output is looking great.

Some things that may surprise in the current state:

• the JSON types of each entry's hashable and markers are string, but might conceivably be boolean and list respectively
• in addition to the JSON output, the text file is written (as usual)
• line values may technically be multiple lines depending on annotation style

They don't necessarily need "fixing." But it may be more common to want to generate the JSON without also writing the txt 🤷🏼 .

Oh I think via is currently a string and only includes one package when the annotations indicate more than one.

[chrysle]

Thanks for the quick response!

• the JSON types of each entry's hashable and markers are string, but might conceivably be boolean and list respectively

Good point, switched to using booleans for several keys. About the list format for markers, I'm unsure how to construct it from the given Marker class. They are explicitly formatted as strings by pip per requirement via a magic method.

• line values may technically be multiple lines depending on annotation style

Right, but as they are included for convenience so that the can directly be used to create a constraints file, I thought that wouldn't be a problem. Especially since the newline characters don't get translated to actual newlines.

But it may be more common to want to generate the JSON without also writing the txt 🤷🏼 .

I also thought so at the beginning, but then reverted the behaviour, as I think a common use case would be that a constraints file is generated for direct use and information about the constraints for further processing is collected. For other cases, it shouldn't be a big deal to add the --dry-run flag to the operation. But happy to change it if you disagree.

Oh I think via is currently a string and only includes one package when the annotations indicate more than one.

Good catch, thank you! I hacked something together (surely that can be improved) to parse the constructed output lines and get the parent requirements from them, since there is some logic in OutputWriter._format_requirement related to that that I wouldn't want to copy.

[chrysle]

I hacked something together (surely that can be improved)

I did that now by merging the OutputWriter._get_json function with the OutputWriter._format_requirement function (breaking a few more tests in the process). Any further thoughts?

[chrysle]

@AndydeCleyre JSON output is now written to a requirements.json file. However, as pip-compile can only read requirements files of type text right now, the --upgrade-package mechanism is dysfunctional. Do you think that's in order?

[AndydeCleyre]

I don't know:

• would it make sense to still use the text output, if present, to determine upgrades? Just like usual, only with an additional json output?
• or instead to internally render any existing json output to a temporary reqstxt and use that?

Either way could be surprising. Sorry I don't have a clear idea about this.

[chrysle]

* or instead to internally render any existing json output to a temporary reqstxt and use that?

I've adapted this suggestion, as it seemed less complex to implement; but I think we need a new Python-wide standard for a requirements.json file in the long run.

[chrysle]

Cc @woodruffw, would you like to test this?

[woodruffw]

Yes, thanks for the ping! I'll try and set aside some testing time in the coming days.

[webknjaz]

@woodruffw you said you wanted to test this?

[webknjaz]

Would #2124 supersede this?