Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRAFT: ocioarchive enhancements #1931

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

KevinJW
Copy link
Contributor

@KevinJW KevinJW commented Jan 16, 2024

Do not merge this is for comments etc.

I've added a 'minimal mode for ocioarchive, that attempts to add only the files you need for the configuration file given.

To run you just add --minimal to the command line.

as an example here is the result of one of my configurations

-rw-r--r-- 1 kwheatle user 222M Jan 16 21:16 test_archive.ocioz
-rw-r--r-- 1 kwheatle user  19M Jan 16 21:02 test_archive_minimal.ocioz

The full one contains
808773530 Bytes from 246 files
the minimal one is 62104478 in 29 files

You can imagine that part of the full directory contains the ACES 1,2 configs amongst other items. and from that the minimal config only needed these

  9666918  03-03-2023 15:59   colour/look/aces_1.2/luts/InvRRT.P3-DCI__D60_simulation_.Log2_48_nits_Shaper.spi3d
  9668173  03-03-2023 15:59   colour/look/aces_1.2/luts/InvRRT.Rec.709.Log2_48_nits_Shaper.spi3d
  9667175  03-03-2023 15:59   colour/look/aces_1.2/luts/InvRRT.Rec.709__D60_sim._.Log2_48_nits_Shaper.spi3d
  8032699  03-03-2023 15:59   colour/look/aces_1.2/luts/Log2_48_nits_Shaper.RRT.P3-DCI__D60_simulation_.spi3d
  6829730  03-03-2023 15:59   colour/look/aces_1.2/luts/Log2_48_nits_Shaper.RRT.Rec.709.spi3d
  7770153  03-03-2023 15:59   colour/look/aces_1.2/luts/Log2_48_nits_Shaper.RRT.Rec.709__D60_sim._.spi3d
    73776  03-03-2023 15:59   colour/look/aces_1.2/luts/Log2_48_nits_Shaper_to_linear.spi1d

It does not yet handle environments, nor have good documentation or tests and no Python bindings

Would be good to have feedback before pursuing it further. especially as this changes the ABI and API

Kevin

@KevinJW KevinJW added API Change Needs Discussion Needs discussion before implmentation, which could result in changes, or a decision not to proceed. labels Jan 17, 2024
@remia remia marked this pull request as draft January 22, 2024 13:09
@KevinJW KevinJW force-pushed the ocioarchive-enhancements branch from 7f1a112 to c727983 Compare January 22, 2024 20:04
Copy link
Collaborator

@remia remia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a quick look and limited comments to general API questions, seeing it's still a draft. Looks good nonetheless.

@@ -1503,7 +1504,7 @@ class OCIOEXPORT Config
*
* \return bool Archivable if true.
*/
bool isArchivable() const;
bool isArchivable(bool minimal) const;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this accept the Flag enum instead, could make it more future proof? Adding it as an overload might be cleaner for current users of isArchivable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I don't like the use of Flags as they can hide the actual meaning so I guess I defaulted to a single boolean, that said I'd prefer not using multiple bool's in APIs, so if we needed other parameters in the future then I'd suggest wanting to wrap the boolean in a stronger type to avoid calling function(true, false, true, ...) but if others want Flags That is simple enough

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you, although to me, even one boolean is not very explicit here, when you see isArchivable(true) somewhere randomly in the code. Flags are already used in other places of the API like for the Processor queries so that might be more consistent.

void archive(std::ostream & ostream, ArchiveFlags flags) const;

//TODO: document
void GetAllFileReferences(std::set<std::string> & files) const;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want this in the public API or is it only for OCIOZArchive need? If the latter, there might be a way to move it to ConfigUtils.h instead. If the former, I'd vote to return a vector instead of set in the spirit of reducing STL headers included here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was one of my thoughts, I would think other uses for the list of files would be interesting and thus a public function may be needed.

Note that the existing code uses them as a set rather than a vector as this then makes each entry unique as it is obviously possible to use the same external file in multiple transforms in a config file (we use the same placeholder file for some of our configuration files). If we want to turn that into a vector for a public API then I guess that is fine.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally it seems this new method is not strictly needed as you can implement it with the existing public API (if I'm not mistaking), but I'm not opposed to adding it if you think there is a strong benefit to have it.

Yes I think the use of set is appropriate for the implementation but not strictly required in the public API, might be nitpicking but seems slightly cleaner!

Needs documenting and testing
May be nice to have a progress report and/or integration with OCIO Logging

Signed-off-by: Kevin Wheatley <[email protected]>
If archiving a minimal config and environment variables are used, it must be possible for all the needed variables to be resolved and any files pointed to should also exist at the time of archiving.

The values of the environment will be written into the saved configuration in the environment section of the configuration file. Ths configuration stored thus may be different to the initial config being passed for archiving.

Needs tests adding and futher discussion in the TSC meetings.

Signed-off-by: Kevin Wheatley <[email protected]>
Adds default parameter value for isArchivable function to bindings. This is not matching the behaviour of the C++ API.

Signed-off-by: Kevin Wheatley <[email protected]>
MSVC thinks you need the capture, GCC thinks the capture is not  needed, so pass variable directly into the lambda function call. Ugly to pass several booleans, but should work.

Signed-off-by: Kevin Wheatley <[email protected]>
Signed-off-by: Kevin Wheatley <[email protected]>
@KevinJW KevinJW force-pushed the ocioarchive-enhancements branch from 94a0221 to b60bffc Compare December 20, 2024 11:56
@KevinJW
Copy link
Contributor Author

KevinJW commented Dec 20, 2024

Rebased to current HEAD

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Discussion Needs discussion before implmentation, which could result in changes, or a decision not to proceed.
Projects
Status: Later
Development

Successfully merging this pull request may close these issues.

3 participants