Would it be bad design to abstract a graphics library and wrap it in a
single class?

The answer is: Not necessarily!
But in your case: Probably!

I guess what you are trying to do also has a name: It’s called the facade pattern and is used to simplify complex routines and sequential dependencies for the user.

It provides various classes for handling graphics, input, etc, but I
would like to wrap it all up in a single Media class.

I smell bad OO Design here because you give a class multiple responsibilities. Input Handling should not share code with the graphics routines. This class could very well add up to something unmaintainable eventually.

However, I have read several times that monolithic, do-it-all classes
such as this might not be a good idea.

Why is that?

It’s all about maintainability. You’re coupling things which do not belong together, and every part of your system that uses it then has a strong coupling to these things as well. Let me give you a quick example of code I saw recently:

//bad! This is in a data access class! I have effectively coupled it to web libraries by using HttpServletRequest
public List<Reason> getReasons(HttpServletRequest request, String filter) {
    Locale locale = request.getLocale();
    //get reasons and process according to locale
}

//much much better: No coupling/dependencies to web environment
public List<Reason> getReasons(Locale locale, String filter) {
    //get reasons and process according to locale
}

Think about this example: If I’d use that class in Desktop-Application, I’d still have all the web stuff in there, just because someone decided to use HttpServletRequest. It’d be an unecessary dependency.

You might ask how this is directly related to your question, and the answer is: It’s all about dependencies/coupling. The more dependencies, the stronger your couplings, the harder your code will be to reuse or to enhance or to maintain without carrying around a lot of garbage or using ugly hacks.

Because any time you have to change anything, you’ll find it has impacts on everything. In addition, one class can only have one lifetime, whereas in reality you want to have several lifetimes for several different components- for example, you want each sprite to live a different length of time.