Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add table cache for table not exist #437

Open
lintanghui opened this issue Oct 17, 2019 · 2 comments
Open

add table cache for table not exist #437

lintanghui opened this issue Oct 17, 2019 · 2 comments

Comments

@lintanghui
Copy link
Contributor

The problem:
I made a ddl on a online table, in case of table lock, I use mysql gh-ost tool. when using gh-ost tool,
old data would be write into a temp table and made a lot row events. after ddl finished. temp table would be deleted.

create temp table
write row event 
wrte row event
a lot of row event ...
some other event 
a log of temp table event 
delete temp table

If I run canal from pos of some other event (maybe restart). i would get a event of temp table, but can not get schema of temp table but return table not exist since temp table had been deleted.
there ere a lot of row event of temp table, every time i parse a event i send a req of IsTableExist to mysql but get not exist each time.
in mysql, show processlist ,binlog dump is block since canal were always doing get table schema
image

how to slove this:
in my opinion. we should add a table no exist cache for not exist table. when get a row event of table which not exist in cache, skip it instead of get table schema each time. if it's ok, i will make a pr. @siddontang

@siddontang
Copy link
Collaborator

@lintanghui

yes, now we haven't supported some DDLs well. seem #358 can solve your problem but we have not advanced this.

PTAL @GregoryIan

@lintanghui
Copy link
Contributor Author

same with #427. once canal start from a middle pos. we may get a wrong table schema or can not get table schema once table been changed or been deleted. since we get scheme by GetTable. GetTable return current table scheme but not when binlog happend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants