I would like to know if this possible to apply.
for example i have this table:
new_feed_dt regex_to_apply expr_to_apply
053021 | _(d+) | date_format(to_date(new_feed_dt, 'yyyyMMdd'), 'yyyy-MM-dd')
053022 | _(d+) | date_format(to_date(new_feed_dt, 'yyyyMMdd'), 'yyyy-MM-dd')
053023 | _(d+) | date_format(to_date(new_feed_dt, 'yyyyMMdd'), 'yyyy-MM-dd')
053024 | [a-zA-Z]+(d+) | date_format(to_date(new_feed_dt, 'MMddyyyy'), 'yyyy-MM-dd')
053025 | DT(d+) | date_format(to_date(new_feed_dt, 'MMddyy'), 'yyyy-MM-dd')
I need to use this expr to apply column to the date column
example image:
I tried so far this but it didnt work.
df_with_regex.withColumn(
'new_feed_dt',
f.expr("expr_to_apply")
)
how would you do it?
Thanks in advance