You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should match the logic that CSV.jl and Arrow.jl use to decide if they are making Strings or InlineStrings.
Once we have JuliaStrings/InlineStrings.jl#8
it should be as fast, or faster (since no allocations) to do this.
Looking at parse.jl I don't think the change would be too hard.
It would want benchmarking; but I have high hopes.
Yes this one is about variables length string columns if it is good for Arrow/CSV it is good enough for us IMO.
It's a much more practical concern since those varchar is actually commonly used
OTOH #207 is only about fixed size strings, which has a even stronger argument for it, since we know the size in advance.
LibPQ currently does not consider columns as a whole for parsing. CSV does. Without a large refactor, LibPQ will then not have a consistent type for the column, preventing the strings from being inlined (the whole point of InlineStrings). It's easy to add the option to parse the strings like this, or with a specific InlineString type with a specific size, though.
We should match the logic that CSV.jl and Arrow.jl use to decide if they are making
String
s orInlineString
s.Once we have JuliaStrings/InlineStrings.jl#8
it should be as fast, or faster (since no allocations) to do this.
Looking at
parse.jl
I don't think the change would be too hard.It would want benchmarking; but I have high hopes.
(Related #207)
The text was updated successfully, but these errors were encountered: