v2.8.0

Arrow v2.8.0

Diff since v2.7.4

Merged pull requests:

• Add reading metadata from Arrow.Table (#481) (@bkamins)
• Start support for ListView, BinaryView, Utf8View, etc. (#512) (@quinnj)
• Bump codecov/codecov-action from 1 to 5 (#531) (@dependabot[bot])
• Make DictEncoding reading threadsafe (#535) (@quinnj)
• Change an info log back to debug (#537) (@quinnj)

Closed issues:

• Arrow.jl fails to precompile with error "Magic file identifier "TZjf" not found." (#510)
• Reading multiple file corrupt values and is also order dependent (#534)

v2.7.4

Arrow v2.7.4

Diff since v2.7.3

Merged pull requests:

• fromarrow(::Type{Dates.Time}, x::Arrow.Time) (#527) (@palday)
• Remove usage of LoggingExtras and verbosity logging macros (#529) (@quinnj)
• Bump version to 2.7.4 (#530) (@quinnj)

Closed issues:

• Error when open files with Date.Time containing MICROSECONDs (#363)
• Cut a new release with TranscodingStreams v0.11 compat? (#523)
• missing methods for fromarrow(::Type{Time}, x::Arrow.Time{<:Arrow.Flatbuf.TimeUnit.T, <:Integer}) (#526)

v2.7.3

Arrow v2.7.3

Diff since v2.7.2

Merged pull requests:

• release: Fix "Verify release" job on macOS GitHub Actions runner (#514) (@kou)
• ci: Enable Dependabot for GitHub Actions (#516) (@kou)
• Bump actions/checkout from 3 to 4 (#518) (@dependabot[bot])
• Bump actions/cache from 1 to 4 (#519) (@dependabot[bot])
• Bump julia-actions/setup-julia from 1 to 2 (#520) (@dependabot[bot])
• Bump julia-actions/julia-buildpkg from 1.2 to 1.6 (#521) (@dependabot[bot])
• Bump actions/setup-python from 2 to 5 (#522) (@dependabot[bot])
• Bump version to 2.7.3 (#524) (@quinnj)

Closed issues:

• interoperability with round-tripping through data format broken (x-issue) (#508)
• release: "Verify release" job fails on macOS GitHub Actions runner (#513)
• ci: Enable Dependabot for GitHub Actions (#515)

v2.7.2

Arrow v2.7.2

Diff since v2.7.1

Merged pull requests:

• Compatibility of schemas with nested types (#504) (@poncito)

Closed issues:

• Arrow-over-HTTP client and server examples in Julia (#502)
• snappy_jll v1.2.0 lead to Arrow_jll failed to build (#505)

v2.7.1

Arrow v2.7.1

Diff since v2.7.0

Merged pull requests:

• Bugfix type instability in length(::Arrow.ToList) (#497) (@JoaoAparicio)

v2.7.0

Arrow v2.7.0

Diff since v2.6.2

Merged pull requests:

• cleanup testing (#462) (@baumgold)
• Formatting (#464) (@baumgold)
• Ensure that ArrowTypes.default is defined for Vararg tuples (#466) (@quinnj)
• Fix typo in passing largelists keyword arg (#467) (@quinnj)
• Try to fix docs by publishing to apache infra (#468) (@quinnj)
• Use wkspawn from ConcurrentUtilities instead of Threads.spawn (#469) (@quinnj)
• Fix documentation link (#478) (@guilhermebodin)
• Fix documentation example for Arrow.Writer (#479) (@guilhermebodin)
• Fix example in Arrow.Writer (#480) (@guilhermebodin)
• Docs: add a note about using copy() to get a DataFrame where the columns are regular Vectors (#487) (@DilumAluthge)
• bump (#488) (@JoaoAparicio)
• Use https://arrow.apache.org/julia/ as the official document URL (#490) (@kou)
• Fix docs errors (#491) (@ericphanson)
• enable field-order-agnostic overloads of fromarrow for struct types (#493) (@jrevels)

Closed issues:

• Loss of parametric type information for custom types (#134)
• Look into potential over-copying when copying arrow vector (#190)
• Arrow.write with partition/partitioner results batch unique value error (#192)
• Roundtrippability of special strings (#196)
• Support InlineStrings (#304)
• Invalid argument error (#336)
• ERROR: MethodError: no method matching zero(::Type{Union{Nothing, String}}) when trying to serialize Union{Nothing,String} fields (#368)
• github releases not in sync (#370)
• Typo in kwargs of arrowvector for ListKind (#392)
• Introduce automatic code formatting with JuliaFormatter.toml (#398)
• Issue with Union{Missing, VersionNumber} (#461)
• GitHub Pages build error (#463)
• Use https://arrow.apache.org/julia/ as the official Website URL (#470)
• Arrow.write performance on large DataFrame (#473)
• getindex broken with SVector{3, UInt} in the presence of missing data (#486)
• Removing .arrow files without closing Julia seems impossible in Windows (#492)

v2.6.2

Arrow v2.6.2

Diff since v2.6.1

Closed issues:

• Does this pkg provide the plasma In-Memory Object Store? (#37)
• Allow for file appends (#105)
• Invalid arrow file produced when writing partitioned with a dictionary-encoded column? (#129)
• New package release process (#279)
• Need to improve code review/release process and reduce developer productivity friction (#342)
• [Feature Request] Support Tensors ? (#433)
• Error with v2.6.0 (#458)

Merged pull requests:

• implement default for SubArray (#456) (@baumgold)
• Fix case of recursively nested lists (#459) (@quinnj)
• Bump Arrow & ArrowTypes versions (#460) (@quinnj)

v2.6.1

Arrow v2.6.1

Diff since v2.6.0

Closed issues:

• Release document misses how to register ArrowTypes to the Julia General Registry (#450)
• Arrow.jl 2.6 breaks Legolas.jl's tests (#452)
• Incorrect syntax in ArrowTypes code (#453)

Merged pull requests:

• release: Add how to register ArrowTypes to the Julia General Registry (#451) (@kou)
• #453 fix typeo introduced in #389 (#454) (@baumgold)
• bump version for new release (#455) (@baumgold)

v2.6.0

Arrow v2.6.0

Diff since v2.5.2

Closed issues:

• Support Tables.partitions when reading "arrow file" format in addition to "stream" format (#293)
• Make recursive iteration to get dictionaries more defensive for interop (#375)
• Error/Segfault when writing many partitions (#396)
• Vector{UInt8} mis-represented when writing to disk (#411)
• CI doesn't test with multiple threads (#426)
• Malformed file by Arrow.write on a IOStream created with open(filename, "w") (#432)
• Unhandled sentinel value for len in compression causes invalid Array dimensions (#435)

Merged pull requests:

• Get dictionaries of children only when field.children not nothing (#382) (@okartal)
• fix Base.eltype methods and functions that take Type parameters (#404) (@baumgold)
• enable incremental reads of arrow-formatted files (#408) (@baumgold)
• Base.isdone for Stream (#428) (@baumgold)
• Run with 1 and 2 threads during tests (#431) (@quinnj)
• Add handling of len = -1 in uncompress (#436) (@DrChainsaw)
• Don't treat Vector{UInt8} as Arrow Binary type (#439) (@quinnj)
• Bump BitIntegers compat (#441) (@quinnj)
• Handle len of -1 in "compressed" buffers from other languages (#442) (@quinnj)
• Add Tables.partitions definition for Arrow.Table (#443) (@quinnj)
• Remove scopedenum for EnumX (#444) (@quinnj)
• Refactor compressors/decompressors for laziness + safety (#445) (@quinnj)
• Return SubArrays when possible for arrow list types (#446) (@quinnj)
• bump version of Arrow and ArrowTypes to prepare for new release (#447) (@baumgold)

v2.5.2

Arrow v2.5.2

Diff since v2.5.1

Merged pull requests:

• Bump Arrow + ArrowTypes versions (#407) (@baumgold)