I am seeing 1-2% grpc Traffic gets dropped even after following best practices.
I am using channel polling of size 10-20, best practices suggested in for polling is we have a unique channel name and even with retry added these numbers did not change.
For channel creation I using bellow code:

This is a java client and there is an AWS grpc loadbalncer in between.

Map<String, Object> customArgs = new HashMap<>();
customArgs.put("channelNumber", String.valueOf(channelNumber));
ManagedChannel channel = ManagedChannelBuilder.forTarget(grpcUrl)
.usePlaintext()
.defaultLoadBalancingPolicy("round_robin")
.enableRetry()
.maxRetryAttempts(6)
.defaultServiceConfig(customArgs)
.build();

And the Error msg I am getting is:

RankingServiceSort EXCEPTION : UNAVAILABLE: unavailableio.grpc.stub.ClientCalls|toStatusRuntimeException|262
io.grpc.stub.ClientCalls|getUnchecked|243
io.grpc.stub.ClientCalls|blockingUnaryCall|156
proto.RankingServiceGrpc$RankingServiceBlockingStub|sortInventoryData|169
RankingService.RankingGrpcConnector|GetInventoryRanking|80

Please help us we are stuck here from 3-4 months, We are using aws grpc supported ALB in between and these best practices though helped us in reducing DeadLine Acceded issue but service Unavailable cases are not getting reduced. Did now found any detailed documentation as well. We are having 2 instances for this service and both CPU and memory and network bandwidth is not at exhausted, used bellow 5%.

Thanks & Regards,
Pawan Sharma

• Channel Pooling aws grpc ALB that helped in reducing Deadline Acceded aces where dealine was 1 sec and server is responding with 1 ms
• enabling retry should have reduced service unavailable cases but it did not changed a thing.