I have a Databricks DataFrame with

Columns : tno,data_value

Output of first Databricks run:

tno, data_value
1,hdjsjsjnsns
2,dhjdjdjsnsn
3,jdjsjsjsjsjjs

When I run again the same notebook after some time it should generate as:

tno, data_value
4,hdjsjsjnsns
5,dhjdjdjsnsn
6,jdjsjsjsjsjjs

Just like an Oracle or SQL Server “Sequence”

I have done rownumber() and monotically_increasing_id()

But those create right from start from 1 for every run.

So, just think it as a transactional_key. How to achieve this in pyspark Databricks.