Relative Content

Tag Archive for pythongoogle-cloud-platformgoogle-cloud-dataflowapache-beamgoogle-cloud-pubsub

Configure message retention duration of PubSub subscription created by Dataflow

I have a Dataflow pipeline that ingests messages from PubSub. This automatically creates a subscription however the retention duration is 7 days. I like that it creates the subscription so I don’t have to and it’s convenient, however the retention duration has caused some issues when a bug arose and resulted in a huge (and expensive) backlog.