I’m using the JDK 17 and beam 2.50 and on the gcp dataflow graph and on the read from Avro/write to BigQuery I get the
READ RAW+ MESSAGES FROM AVRO file/Read.out0
Elements added : 332
Estimated size: 27.37 MB
After the upgrade the beam to the 2.54/2.55.1 I don’t get any estimations but the files are fine read/write.
READ RAW+ MESSAGES FROM AVRO file/Read/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper).out0
Elements added: –
Estimated size: –
The code it’s the same on my side but don’t know where exactly to look
I hope to get the same things on the dataflow job graph to be sure if something fail, or don’t work properly, to have a clue.
Dragoș Radu is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.