We have an dotnet6 app that has been running in k8s for quite some time and only recently it started producing the following error, but only on some of the pods.
System.IO.IOException: The configured user limit (512) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached.
at System.IO.FileSystemWatcher.StartRaisingEvents()
Whenever we rerun the deployment, some pods start up without issue and some fail. deleting the failed ones does not result in new successful pods, the subsequent pods fail as well.
I have tried setting the reloadInChange to false in the AddJosnFile method. I’ve added
ENV DOTNET_HOSTBUILDER__RELOADCONFIGONCHANGE=false
to my docker file. I have even tried programmatically setting it at runtime with:
Environment.SetEnvironmentVariable("DOTNET_hostBuilder:reloadConfigOnChange", "false");
Nothing seems to work. There is this open github issue, but none of the proposed solutions have worked.
https://github.com/dotnet/AspNetCore.Docs/issues/19814
Is there any other possible solutions out there not mentioned in the above issue?