Ask for feedback from the developers which were part of the team developing the product he ‘owned’. You could ask questions like:

1. Was he able to answer requirements related questions in a timely manner?
2. When presented with multiple alternatives in regards to a feature, was he able to make a clear decision with regard of what he wants?
3. How often did he change the requirements? This is something that always happens. Being able to cope with this is what it means to be agile. However, if this happens very often, then that’s a sign that the product owner has no idea of what he actually wants
4. Was he able to provide clear feedback during demo sessions?

As questions I’d consider useful:

• When there is a new story created, how much work is being done to make sure of its quality? Is there a lot of work for developers to do before working on something or is it good to go once the PO has written the request?

• How available for questions and clarification has the PO been over the course of the evaluation period?

• How receptive to feedback has the PO been in terms of the evolution of the process and team over the course of several sprints?

From a higher level, here’s what I’d be watching:

1. Decision-making. From prioritizing the backlog to clarifying the requirements on a story, there are lots of decisions that may fall to the PO to handle. Was this an issue for the team in terms of delivering work? Were there frequent delays in getting things done?

2. Communication. How well were the priorities communicated? Were there demonstrations of functionality where the PO provided either feedback directly or brought in customers to review new functionality to ensure that it was the desired results?

3. Vision. This goes a bit in hand with communication but the idea here is to know what big dates are coming up, how is the work being reviewed and what direction is the team being driven.