A lot depends on the algorithm you’re trying to implement. Some of them require a LOT more knowledge of what’s going on than others. Almost all of them require at least the basics.

A basic AGC (automatic gain control) is simple. Basic filters are fairly simple. Mixers start to require more knowledge.

Active echo cancellation using least mean squares, and especially normalized least mean squares, just about requires that you know what you’re doing, especially if you’re doing it in fixed point math (as opposed to floating point). Trust me on this.

Certain kinds of image processing can bite you in the ass if you don’t know the secret insider knowledge. I have the toothmarks to prove this one, too.

Hamming’s book on “Digital Filters” is a good place to start, if you need an overview.