OAK-10341 Tree store #1577

thomasmueller · 2024-07-11T14:11:05Z

Ready for review.

...n-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/tree/TreeStore.java

Updated dependencies.

…Supplier implementation (#1580)

…dencies reference vulnerable amazon ion-java version) (#1581)

* OAK-10945: Remove usage of Guava Function interface (oak-upgrade) * OAK-10945: Remove usage of Guava Function interface (oak-store-spi) * OAK-10945: Remove usage of Guava Function interface (oak-it) * OAK-10945: Remove usage of Guava Function interface (oak-store-document) * OAK-10945: Remove usage of Guava Function interface (oak-store-composite) * OAK-10945: Remove usage of Guava Function interface (oak-segment-tar) * OAK-10945: Remove usage of Guava Function interface (oak-segment-azure) * OAK-10945: Remove usage of Guava Function interface (oak-security-spi) * OAK-10945: Remove usage of Guava Function interface (oak-security-search) * OAK-10945: Remove usage of Guava Function interface (oak-run-commons) * OAK-10945: Remove usage of Guava Function interface (oak-run) * OAK-10945: Remove usage of Guava Function interface (oak-lucene) * OAK-10945: Remove usage of Guava Function interface (oak-jcr) * OAK-10945: Remove usage of Guava Function interface (oak-exercise) * OAK-10945: Remove usage of Guava Function interface (oak-core-spi) * OAK-10945: Remove usage of Guava Function interface (oak-core) * OAK-10945: Remove usage of Guava Function interface (oak-commons) * OAK-10945: Remove usage of Guava Function interface (oak-blob-plugins) * OAK-10945: Remove usage of Guava Function interface (oak-blob-cloud, oak-blob-cloud-azure) * OAK-10945: Remove usage of Guava Function interface (oak-upgrade) - cleanup * OAK-10945: Remove usage of Guava Function interface (oak-store-spi) - cleanup * OAK-10945: Remove usage of Guava Function interface (oak-store-document) - cleanup * OAK-10945: Remove usage of Guava Function interface (oak-store-composite) - cleanup * OAK-10945: Remove usage of Guava Function interface (oak-segment-tar) - cleanup * OAK-10945: Remove usage of Guava Function interface (oak-lucene) - cleanup * OAK-10945: Remove usage of Guava Function interface (oak-jcr) - cleanup

...because of failures, as in https://github.com/apache/jackrabbit-oak/actions/runs/10021673018

done

- "oak.indexer.persistedLinkedList.cacheSize" - sets the cache size of the PersistedLinkedList used to traverse the FFS. This controls the number FFS entries kept in memory.

…ne ends

Co-authored-by: pirlogea <[email protected]>

…other: PathUtils.isDirectAncestor() (#1598)

…FlatFileStore)

nfsantos

I have only looked at the parts that interact with existing code, did not review the TreeStore or IndexStore logic.

...src/main/java/org/apache/jackrabbit/oak/index/indexer/document/DocumentStoreIndexerBase.java

...java/org/apache/jackrabbit/oak/index/indexer/document/flatfile/FlatFileNodeStoreBuilder.java

.../apache/jackrabbit/oak/index/indexer/document/flatfile/pipelined/PipelinedTransformTask.java

steffenvan · 2024-09-06T14:43:59Z

Just my opinion, but I see no comments that address the logic or readability of the TreeStore. Wouldn't it have made sense to have some reviews on that before merging?

thomasmueller · 2024-09-06T15:11:15Z

I see no comments that address the logic or readability of the TreeStore

Just documenting the tree store (alone) wont resolve that problem. I think we are missing high-level documentation of the implementation for many many features. Including the blob store, the segment store, the document store, oak-core, oak-jcr, etc. I think we should discuss what implementation level-documentation is needed in general. I view that as an independent problem.

thomasmueller added 2 commits July 11, 2024 16:10

OAK-10341 Tree store

9750a6f

OAK-10341 Tree store

72ab6b9

nit0906 reviewed Jul 11, 2024

View reviewed changes

...n-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/tree/TreeStore.java Outdated Show resolved Hide resolved

OAK-10341 Tree store

d49b6c7

nit0906 reviewed Jul 12, 2024

View reviewed changes

...n-commons/src/main/java/org/apache/jackrabbit/oak/index/indexer/document/tree/TreeStore.java Outdated Show resolved Hide resolved

thomasmueller and others added 25 commits July 12, 2024 11:26

OAK-10341 Tree store

2404d14

OAK-10341 Tree store

1761ddb

OAK-10944: oak-auth-ldap: update commons-pool2 dependency to 2.12.0 (#…

3d6d009

…1576)

OAK-10705: oak-standalone: update dependencies (#1411)

dc10a12

Updated dependencies.

OAK-10905 | Add a configurable async checkpoint creator service (#1560)

f12ccb1

OAK-10905 | Add license header to AsyncCheckpointService (#1579)

7f112fa

OAK-10848: commons: remove use of slf4j.event.Level in SystemProperty…

ba530fe

…Supplier implementation (#1580)

OAK-10685: remove unused import of java.io.UnsupportedEncodingException

36cfcfa

OAK-10949: blob-cloud, segment-aws: update aws SDK to 1.12.761 (depen…

cafbd29

…dencies reference vulnerable amazon ion-java version) (#1581)

OAK-10954: Update spotbugs plugin to 4.8.6.2 (#1588)

b098fd8

OAK-10959: webapp: update Tomcat dependency to 9.0.90 (#1589)

5fb55e0

OAK-10960: blob-cloud, segment: update netty version to 4.1.111 (#1590)

cd35db7

OAK-10962: oak-solr-osgi: update zookeeper dependency to 3.9.2 (#1591)

95ed6c4

Update build.yml to disable Sonar for now

8db9183

...because of failures, as in https://github.com/apache/jackrabbit-oak/actions/runs/10021673018

OAK-10965 - Make ConsoleIndexingReporter thread safe. (#1592)

8b05cae

OAK-6762: Convert oak-blob to OSGi R7 annotations (#1413)

6c44805

done

OAK-6773: Convert oak-store-composite to OSGi R7 annotations (#1489)

0521c63

done

OAK-10951 - Add a new configuration property (#1594)

ebefe01

- "oak.indexer.persistedLinkedList.cacheSize" - sets the cache size of the PersistedLinkedList used to traverse the FFS. This controls the number FFS entries kept in memory.

OAK-10966 - Avoid object creation in PathUtils.isAncestor (#1596)

b37db4c

OAK-10964: bump nimbus-jose-jwt dependency to latest (#1593)

9528fdd

OAK-6773: Convert oak-store-composite to OSGi R7 annotations - fix li…

818317c

…ne ends

OAK-10803 -- compress/uncompress property, disabled by default (#1526)

25792e7

Co-authored-by: pirlogea <[email protected]>

OAK-10974 and OAK-10869 : temporarily disabling flaky tests

aefb990

OAK-10971 - Add a method to test if a path is a direct ancestor of an…

c416850

…other: PathUtils.isDirectAncestor() (#1598)

thomasmueller added 14 commits August 9, 2024 16:33

OAK-10341 Tree store (blob prefetch)

b242328

OAK-10341 Tree store (node prefetch)

962dab6

Merge trunk

1fa3f20

OAK-10341 Tree store (incremental)

927aafc

OAK-10341 Tree store (pack files)

144fde8

Merge branch 'trunk' into OAK-10341b

c67ed65

OAK-10341 Tree store (pack files)

64639a5

OAK-10341 Tree store (tests)

a32bf44

OAK-10341 Tree store (incremental)

fb515ef

Merge branch 'trunk' into OAK-10341b

a884a87

OAK-10341 Tree store (compress)

7a31307

OAK-10341 Tree store (traverse included paths)

6776860

OAK-10341 Tree store

7f80b98

OAK-10341 Tree store

822d0e9

thomasmueller marked this pull request as ready for review August 29, 2024 13:22

thomasmueller added 4 commits August 29, 2024 15:27

OAK-10341 Tree store

0d522d7

OAK-10341 Tree store

d312ca8

OAK-10341 Tree store

a1bf314

Merge branch 'trunk' into OAK-10341b

3d370e8

thomasmueller requested review from nit0906, nfsantos, fabriziofortino and amit-jain September 4, 2024 08:00

OAK-10341 Tree store (use same blob prefetching configuration as for …

cee6866

…FlatFileStore)

nfsantos approved these changes Sep 5, 2024

View reviewed changes

thomasmueller added 2 commits September 5, 2024 17:05

OAK-10341 Tree store (javadocs)

95cd63e

OAK-10341 Tree store (code review)

785a118

thomasmueller merged commit bb4ecd1 into trunk Sep 6, 2024
1 of 2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OAK-10341 Tree store #1577

OAK-10341 Tree store #1577

thomasmueller commented Jul 11, 2024 •

edited

Loading

nfsantos left a comment

steffenvan commented Sep 6, 2024 •

edited

Loading

thomasmueller commented Sep 6, 2024

OAK-10341 Tree store #1577

OAK-10341 Tree store #1577

Conversation

thomasmueller commented Jul 11, 2024 • edited Loading

nfsantos left a comment

Choose a reason for hiding this comment

steffenvan commented Sep 6, 2024 • edited Loading

thomasmueller commented Sep 6, 2024

thomasmueller commented Jul 11, 2024 •

edited

Loading

steffenvan commented Sep 6, 2024 •

edited

Loading