I’m trying to build a Siamese network with two Resnet50

base_model1 = self.base_network(15) # earlier 15 layers trainable
base_model2 = self.base_network(20) # earlier 20 layers trainable

It cause Out of memory while trying to allocate 25300657848 bytes

Peak buffers consist of many Relu and Conv2D buffers

Buffer 1:
Size: 1.00GiB
Operator: op_type="Relu" op_name="siamese_network_1/functional_3_1/conv2_block3_out_1/Relu" deduplicated_name="loop_maximum_fusion.1"
XLA Label: fusion
Shape: f32[256,256,64,64]

Buffer 3:
Size: 1.00GiB
Operator: op_type="Conv2D" op_name="siamese_network_1/functional_3_1/conv2_block1_3_conv_1/convolution" source_file="/home/robot/anaconda3/envs/RSIDL/lib/python3.11/site-packages/tensorflow/python/framework/ops.py" source_line=1177
XLA Label: custom-call
Shape: f32[256,256,64,64]

Is 1.00GiB a normal buffer size for these layers?
I tried to reduce trainable layers, and it works

Any other ways or suggestion will be helpful