Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flink往holo日期分区表(分区字段是date类型)写入时找不到分区 error when flink sink data into a date-partition holo table of which the partition filed is date type #36

Open
TangoHuang opened this issue Sep 1, 2023 · 0 comments

Comments

@TangoHuang
Copy link

source file源文件

	/**
	 * 如果读写分区表,修改对应操作的schema为分区子表.
	 *
	 * @param record               操作的Record
	 * @param createIfNotExists    dynamicPartition为true,且是非delete的put操作时,自动创建分区
	 * @param exceptionIfNotExists 分区表不存在时是否抛出异常,get和delete操作发现子表不存在不会抛出异常
	 * @return 是否可以忽略本次操作,比如delete(PUT)但是分区子表不存在的时候;GET但分区子表不存在的时候
	 * @throws HoloClientException 获取分区或者根据分区信息获取TableSchema异常 那么complete exception
	 */
	private boolean rewriteForPartitionTable(Record record, boolean createIfNotExists, boolean exceptionIfNotExists) throws HoloClientException {
		TableSchema schema = record.getSchema();
		if (schema.isPartitionParentTable()) {
			boolean isStr = Types.VARCHAR == schema.getColumn(schema.getPartitionIndex()).getType();
			String value = String.valueOf(record.getObject(schema.getPartitionIndex()));
			Partition partition = pool.getOrSubmitPartition(schema.getTableNameObj(), value, isStr, createIfNotExists);
			if (partition != null) {
				TableSchema newSchema = pool.getOrSubmitTableSchema(TableName.valueOf(IdentifierUtil.quoteIdentifier(partition.getSchemaName(), true), IdentifierUtil.quoteIdentifier(partition.getTableName(), true)), false);
				record.changeToChildSchema(newSchema);
			} else if (exceptionIfNotExists) {
				throw new HoloClientWithDetailsException(ExceptionCode.TABLE_NOT_FOUND, "child table is not found", record);
			} else {
				return true;
			}
		}
		return false;
	}

这行

boolean isStr = Types.VARCHAR == schema.getColumn(schema.getPartitionIndex()).getType();

如果holo分区表的分区字段是date类型,在实际建表语句中表现为:

FOR VALUES IN ('2023-09-03')

所以我认为这里有bug,应该判断分区字段是VARCHAR或者DATE类型都应该isStr = true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant