Is there a way to extract derivation for each column of a sql query
This is an Example Query
SELECT
col1 AS c1,
SUBSTR(
CONCAT(
CASE
WHEN UPPER(RTRIM(M.col2)) IN('C', 'P')
THEN 'P'
WHEN M.col3 IS NULL
THEN M.col4
ELSE
CASE
WHEN UPPER(RTRIM(M.col5)) IN('', 'D')
OR M.col6 IS NULL
THEN 'D'
WHEN UPPER(RTRIM(M.col7)) IN('C', 'P')
THEN 'P'
ELSE CAST(NULL AS STRING)
END
END,
' '), 1, 1
) AS c2,
SUBSTR(
CONCAT(
SUBSTR(CAST(col8 AS STRING), 1, 5),
'|', COALESCE(col9, 'NULL')
), 1, 100
) AS c3,
SUBSTR(CONCAT('test', ' '), 1, 3) AS c4,
SUBSTR('test', 1, 10) AS c5
...
output –
c1 – col1
c2 –
SUBSTR(
CONCAT(
CASE
WHEN UPPER(RTRIM(M.col2)) IN('C', 'P')
THEN 'P'
WHEN M.col3 IS NULL
THEN M.col4
ELSE
CASE
WHEN UPPER(RTRIM(M.col5)) IN('', 'D')
OR M.col6 IS NULL
THEN 'D'
WHEN UPPER(RTRIM(M.col7)) IN('C', 'P')
THEN 'P'
ELSE CAST(NULL AS STRING)
END
END,
' '), 1, 1
)
c3 –
SUBSTR(
CONCAT(
SUBSTR(CAST(col8 AS STRING), 1, 5),
'|',
COALESCE(col9, 'NULL')
), 1, 100
)
c4 – SUBSTR(CONCAT('test', ' '), 1, 3)
c5 – SUBSTR('test', 1, 10) AS
I tried using sql-parse library, but was able to just get the list of column used (c1, c2, c3 ...)
, need derivation for data lineage purpose.
1