I have an ASP.NET Core web application that uses data protection to encrypt/decrypt values stored in a database. The app was original written in .NET 3.1, later migrated to .NET 6, and now to .NET 8. The problem I am having is that after upgrading to .NET 8, the app encounters an error trying to decrypt the previously stored values. Reverting to .NET 6 enables the decryption to work correctly again.

Here is the relevant part of Startup.cs:

services.AddDataProtection()
    .PersistKeysToFileSystem(new DirectoryInfo(keysPath));

Here is what I see in the logs:

2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {7e33d995-90ef-4e91-95bc-68f44e50c4dc}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {827bc86d-f9c8-4edc-b2b8-0f38a19775ca}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {85752c41-0db1-4855-bd3d-1be5fb7948db}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {b7190f71-d798-4564-b644-414650ca839d}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {c10175df-a4c2-4e6a-abf3-7b6ba52fd577}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {c5640c90-8bd7-4032-800a-1615f18d0c1e}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Found key {e4b11e1a-74ac-4ef9-82c9-0299a0885ac5}.
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.DefaultKeyResolver Considering key {e4b11e1a-74ac-4ef9-82c9-0299a0885ac5} with expiration date 2024-06-03 20:36:56Z as default key.
...
2024-04-30 11:54:07.8247 DEBUG Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider Using key {e4b11e1a-74ac-4ef9-82c9-0299a0885ac5} as the default key.
...
2024-04-30 11:54:07.8561 ERROR Microsoft.EntityFrameworkCore.Query An exception occurred while iterating over the results of a query for context type 'MyApp.AppDbContext'.
System.Security.Cryptography.CryptographicException: The key {ca4f6f42-0963-42b4-901d-306088846261} was not found in the key ring. For more information go to https://aka.ms/aspnet/dataprotectionwarning

As the logs state, the first seven keys do exist in the key storage directory, and the last one is still valid. But then it ignores those and comes up with an entirely new one that does not exist, causing the decryption to fail.

How can I get the .NET 8 version of the app to use the existing valid key to decrypt the data rather than creating a new key that will fail?