Unit Test Failure With sqlparse==0.3.0

[reata]

Since Apache-Superset is pinned to 0.3.0 version of sqlparse, we should get this fixed. (or explicitly declare we don't support this version)

[reata]

0.3.1 is OK. Problem only exists with sqlparse==0.3.0
In particular, the failed test cases are:

CREATE TABLE tab1 LIKE tab2;
INSERT OVERWRITE tab1 SELECT * FROM tab2;

[reata]

For "CREATE TABLE tab1 LIKE tab2", see andialbrecht/sqlparse#543
0.3.1 introduces a backward incompatible change to address LIKE appearing in WHERE clause, which causes LIKE in DDL to be parsed as COMPARISON ever since.
We had this special logic to handle this: https://github.com/reata/sqllineage/blob/v1.1.4/sqllineage/core.py#L183 , this is why we fail the test for 0.3.0

[reata]

For "INSERT OVERWRITE tab1 SELECT * FROM tab2", see andialbrecht/sqlparse#475
HQL keyword including "OVERWRITE" is only added until v0.3.1. We should prefer to use newer version.

As in v0.3.0

>>> sqlparse.parse("INSERT OVERWRITE tab1 SELECT * FROM tab2;")[0].tokens
[<DML 'INSERT' at 0x7F37552FCC48>,
 <Whitespace ' ' at 0x7F37552FCCA8>,
 <Identifier 'OVERWR...' at 0x7F37552F9750>,
 <Whitespace ' ' at 0x7F37552FCE28>,
 <DML 'SELECT' at 0x7F37552FCE88>,
 <Whitespace ' ' at 0x7F37552FCEE8>,
 <Wildcard '*' at 0x7F37552FCF48>,
 <Whitespace ' ' at 0x7F37552FCFA8>,
 <Keyword 'FROM' at 0x7F3755300048>,
 <Whitespace ' ' at 0x7F37553000A8>,
 <Identifier 'tab2' at 0x7F37552F9840>,
 <Punctuation ';' at 0x7F3755300168>]

As in v0.3.1

>>> sqlparse.parse("INSERT OVERWRITE tab1 SELECT * FROM tab2;")[0].tokens
[<DML 'INSERT' at 0x7F2CE8DD8D68>,
 <Whitespace ' ' at 0x7F2CE8DD8DC8>,
 <Keyword 'OVERWR...' at 0x7F2CE8DD8E28>,
 <Whitespace ' ' at 0x7F2CE8DD8E88>,
 <Identifier 'tab1' at 0x7F2CE8DAFB88>,
 <Whitespace ' ' at 0x7F2CE8DD8F48>,
 <DML 'SELECT' at 0x7F2CE8DD8FA8>,
 <Whitespace ' ' at 0x7F2CE8DEC048>,
 <Wildcard '*' at 0x7F2CE8DEC0A8>,
 <Whitespace ' ' at 0x7F2CE8DEC108>,
 <Keyword 'FROM' at 0x7F2CE8DEC168>,
 <Whitespace ' ' at 0x7F2CE8DEC1C8>,
 <Identifier 'tab2' at 0x7F2CE8DAFC00>,
 <Punctuation ';' at 0x7F2CE8DEC288>]

[reata]

All things considered, let's upgrade our sqlparse dependency to be >=0.3.1 and hope the "CREATE TABLE LIKE" issue can be fixed in future version and then change our code accordingly.