Who can help me clarify the doubts about Flink’s memory allocation?
I started a yarn-session cluster with the following command.
Late records on single partition with keyed window stream
Flink streaming pipeline with one kafka partition with 5 parallelism with keyed event windows with 1 minute tumbling window. We are using ascending timestamp watermarks with default periodic generator.
Sink Data to DB and Kafka/Axon in a single Transaction
I am new to Flink and have a use case to consume data from Topic1 and insert/update in DB and also push the same data to Topic2 that will be used by different services. The code I have now is something like below:
In-memory, replicated cache for Apache Flink tasks
Im processing a stream that needs to be transformed based on an inventory data. Inherently, the inventory data will be big (dont have exact figure) and will have A LOT of reads and very few writes.
Fetch new records from DB dynamically using Flink and post to kafka/axon topic
I am new to Flink and have a use case of 2 Flink jobs, 1st to consume data from Topic1 and insert/update in DB and 2nd to fetch any new data inserted/updated in DB to Topic2 used by different services.
Connecting to Multiple Broadcasted Streams in Flink
I have a user-case in Flink where I need to connect to 2 Broadcast streams to perform a certain KeyedProcessFunction operation. Does Flink allow this? I’m unable to see any API that supports multiple broadcast streams in one ProcessFunction.
Is it safe to pass MapState to downstream in Flink?
Suppose we have a DataStream and it is possiable to attach a MapState to each String element while passing it to downstream. Like:
How to migrate a job in the flink cluster to another task manager
I have a flink cluster of four task manager ™. Other services are running on a task manager. Due to resource issues such as bandwidth, memory or CPU, the tm will occasionally be disconnected from the cluster.