You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If we create a temporary table for each task, we will end up with a lot of temporary tables when there are many tasks. Therefore, depending on the workload, the number of temporary tables may reach the limit of the number of tables created in the destination database. From what I've read in the code, it looks like it's okay to share temporary tables between tasks if we ignore the resume feature. So, I think we should create an option recycle_temp_table or share_temp_table to allow sharing of temporary tables between tasks. How do you think?
The text was updated successfully, but these errors were encountered:
civitaspo
changed the title
Allow recycle of temporary tables
Allow sharing of temporary tables between tasks
Sep 13, 2021
Hi maintainers,
I create an issue related to #299.
If we create a temporary table for each task, we will end up with a lot of temporary tables when there are many tasks. Therefore, depending on the workload, the number of temporary tables may reach the limit of the number of tables created in the destination database. From what I've read in the code, it looks like it's okay to share temporary tables between tasks if we ignore the
resume
feature. So, I think we should create an optionrecycle_temp_table
orshare_temp_table
to allow sharing of temporary tables between tasks. How do you think?The text was updated successfully, but these errors were encountered: