I have configured the In-Memory Session State Provider like this in the web.config

  <system.web>
     <sessionState mode="InProc" cookieless="false" timeout="20" />
  </system.web>

I have enabled Session affinity on the app service and not going through a application gateway.

Getting this error

Server Error in ‘/’ Application

Object reference not set to an instance of an object. Description:
An unhandled exception occurred during the execution of the current
web request. Please review the stack trace for more information about
the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not
set to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the
current web request. Information regarding the origin and location of
the exception can be identified using the exception stack trace below.

Stack Trace:

[NullReferenceException: Object reference not set to an instance of an
object.]
Microsoft.AspNet.SessionState.SessionStateStoreProviderAsyncBase.Initialize(String
name, NameValueCollection config) +48
Microsoft.AspNet.SessionState.InProcSessionStateStoreAsync.Initialize(String
name, NameValueCollection config) +48
Microsoft.AspNet.SessionState.SessionStateModuleAsync.InitModuleFromConfig(HttpApplication
app, SessionStateSection config) +535
Microsoft.AspNet.SessionState.SessionStateModuleAsync.Init(HttpApplication
app) +165
System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr
appContext, HttpContext context, MethodInfo[] handlers) +552
System.Web.HttpApplication.InitSpecial(HttpApplicationState state,
MethodInfo[] handlers, IntPtr appContext, HttpContext context) +181
System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr
appContext, HttpContext context) +228
System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr
appContext) +314

[HttpException (0x80004005): Object reference not set to an instance
of an object.] System.Web.HttpRuntime.FirstRequestInit(HttpContext
context) +10093124
System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +99
System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest
wr, HttpContext context) +263

I know it is recommended to use Azure redis cache, but that is causing this issue

RedisTimeoutException when using RedisSessionStateProvider on a .NET 4.8 ASP.Net site to store sessionState in Azure Redis Cache

So want to try to use this instead.