Supported Insert as select sql

[melin]

example: insert into table bigdata.test_child_lineage select * from bigdata.test_child_dt where ds='20211207'

There is a lineage relationship between test_child_lineage and test_child_dt

[maropu]

Yea, but it is not easy to implement it because Spark does not hold the relationship in its catalog. Probably, we need a custom SQL listener to track INSERT query logs.

[melin]

Have a plan to finish? Thanks!

[maropu]

I don't have it because I don't have a smart idea for implementing it. Any plan to use the feature if it's implemented?

[melin]

Will use, most of the scheduling job that run every day are insert sql job

[melin]

https://zhuanlan.zhihu.com/p/502927331?utm_source=wechat_session&utm_medium=social&utm_oi=27427107504128&utm_campaign=shareopn&s_r=0
The article has description to solve this problem