Parquet writer only writes one data page if it utilizes dictionary encoding

[coastalwhite]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
import pyarrow.parquet as pq
import os, io

os.environ['PARQUET_DO_VERBOSE'] = '1'
f = io.BytesIO()

df = pl.DataFrame({ "a": ["A"] * 10000 }, schema = { "a": pl.Enum(["A"]) })

f.seek(0)
df.write_parquet(f, data_page_size=1)
f.truncate()

f.seek(0)
print("Polars:")
df = pl.read_parquet(f)

print()
print()
print()

f.seek(0)
pq.write_table(df.to_arrow(), f, data_page_size=1)
f.truncate()

f.seek(0)
print("PyArrow:")
df = pl.read_parquet(f)

Log output

Polars:
Parquet DictPage ( num_values: 1, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray } )
Parquet DataPageV1 ( num_values: 10000, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )



PyArrow:
Parquet DictPage ( num_values: 1, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray } )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 1024, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )
Parquet DataPageV1 ( num_values: 784, datatype: PrimitiveType { field_info: FieldInfo { name: "a", repetition: Optional, id: None }, logical_type: Some(String), converted_type: Some(Utf8), physical_type: ByteArray }, encoding: Some(RleDictionary) )

Issue description

Polars only writes one data page. Even when it is explicitly told to write more than one.

Expected behavior

Polars should divide the data over several data pages, as to make reading with predicates a lot faster.

Installed versions

Replace this line with the output of pl.show_versions(). Leave the backticks in place.