I’m developing an application using Reinforcement Learning (RL) where my agent can choose between three different algorithms (actions) to determine its set of motions for achieving a task. These algorithms vary in their memory usage and the time they take to generate a solution. The goal of the agent is to consume the least memory and solving time via executing the solution provided by the algorithm based on its current state, thus the agent can switch from one algorithm to another at each step if necessary.

Here are my specific questions:

Reward Structure: How should I structure the rewards for my agent? A step is defined as starting from the agent’s current position and ending when it achieves the current goal. Should I reward the agent based on the memory and solving time of the chosen algorithm after it reaches the goal, or should the reward be given differently?

Resource Awareness: (if I want to include it in the reward function) The agent knows the memory usage and solving time for each algorithm in advance (before starting the execution). Given this, should I reward the agent immediately after selecting an algorithm, or after it has executed the algorithm’s solution?

Energy Consumption: Suppose the agent has a limited amount of energy and cannot recharge. Should the reward be based on the total resources consumed so far, or only on the resources consumed for executing the current action? Or should it consider both?

Close Values: If the solving time and memory consumption of the algorithms are very close to each other, how can I artificially differentiate them to test if my agent is learning effectively? Are there techniques or methods to “fake” these values for experimental purposes?

Visualizing Learning: How can I visually determine if my agent is learning to select the best action? Are there specific metrics or visualization techniques that can help me see the agent’s progress in making optimal algorithm selections based on its state?

Any guidance on designing the reward system, handling closely related performance metrics, and visualizing the agent’s learning progress would be greatly appreciated.