Airflow’s GrpcOperator
supports passing template variables in the data
field. However, the protobuf-generated message object that needs to go into the data
field is evaluated at parse time, which breaks the templates.
For example, in the task below, I want to send a request to a gRPC server that has the DAG run’s date ds
in its message body:
task_grpc = GrpcOperator(
dag=dag,
task_id="task_grpc",
grpc_conn_id="grpc_default",
stub_class=CustomGrpcServiceStub,
call_func="CustomGrpcServiceFunction",
response_callback=CustomGrpcService_callback,
streaming=False,
data = {"request": proto_pb2.CustomGrpcServiceRequest(request_date="{{ ds }}")}
)
This will fail, as the request_date is always rendered as "{ ds }"
. Is there a workaround to achieve templated fields in the protobuf message objects?
I tried to write a user-defined macro as suggested in https://github.com/apache/airflow/discussions/36661, but that does not change the outcome.