Hello I am working on a large-scale project with the following technology stack:

Backend: ASP.NET Core Web API
Frontend: Angular
The primary challenge is designing and implementing a custom, dynamic authentication and authorization system. Here are the key details:

Dynamic Requirements: The system needs to support various user roles, permissions, and authentication methods. It should allow for dynamic policy updates and adapt to future security changes.

Custom Implementation: I am looking to implement custom authentication mechanisms beyond the built-in ASP.NET Core options, such as handling different authentication schemes or integrating with third-party identity providers.

Custom Authorization Policies: The system should support complex authorization scenarios like role-based access control (RBAC) and claim-based authorization. It should also allow dynamic updates to authorization logic based on business needs.

Integration Considerations: The system should integrate seamlessly with the Angular frontend, ensuring secure transmission of authentication tokens and efficient authorization checks.

Scalability and Extensibility: The solution must be scalable and capable of handling increased load. It should also be designed for easy extension and modification.

Security Best Practices: I need to follow best practices to protect against vulnerabilities such as unauthorized access and token theft. This includes securing communication channels, validating tokens, and implementing robust error handling.

Testing and Validation: I need guidance on how to thoroughly test the system to ensure it works correctly across different scenarios and handles edge cases effectively.

What I Have Tried:

Built-in Mechanisms:

ASP.NET Core Identity: Explored for authentication and authorization but found it may not fully meet my dynamic needs.
JWT Bearer Authentication: Used for token-based authentication but does not cover all custom scenarios.
Custom Authentication Handlers: Reviewed documentation and examples on creating custom handlers but have not yet implemented a complete solution.

Custom Authorization Policies: Investigated policy-based authorization and how to create custom policies but have not integrated this into a dynamic solution.

Integration with Angular: Explored integration aspects, including secure token handling and role verification but am still refining these integrations.

Security Best Practices: Studied best practices such as using HTTPS and validating tokens but have not fully applied them to my solution.

What I Was Expecting:

Dynamic and Flexible System: A system that adapts to changing requirements and policies without significant redevelopment.

Seamless Integration: Smooth integration with Angular, ensuring a cohesive and secure user experience.

Scalability and Extensibility: A system that scales with user load and evolving security needs.

Comprehensive Security: Adherence to best practices and resilience against common security threats.

Effective Testing: A robust testing strategy that verifies functionality and reliability.

I am seeking advice and examples to achieve a fully integrated and dynamic authentication and authorization system. Any recommendations on architectural styles, implementation strategies, or best practices would be greatly appreciated.

Thank you!