In your case, the particular software you are using is licensed under “GPLv2 or any later version,” which can be included in GPLv3-licensed code. If the code you wanted to use were using GPLv2 only, then it would not be legally possible to include that code in your GPLv3-licensed distribution.

We can consult the inter-GPL-compatibility table from the FSF’s GPL FAQ to see the two cases:

The top marked space disallows a GPLv3 project from including GPLv2-only code. In that case, you must either license your code as GPLv2, or the author of the GPLv2-only code must re-license as GPLv2-or-later.

The space immediately below that, however, allows a GPLv3 project to include code that is licensed under “GPLv2 or any later version“. (This is because you may treat the GPLv2-or-later code as GPLv3 code, and then there’s no problem at all.)

If you have some GPLv2-only code that you want to include in your GPLv3 project, you might look at the space to the left of the circled “NO” and think, “Wait! I can license my project as GPLv2-or-later and still include the GPLv2-only code! Then, I’ll freely upgrade my project to GPLv3 — problem solved.” However, that space has a footnote:

[2] While you may release your project (either your original work and/or work that you received and modified) under GPLv2-or-later in this case, note that the other code you’re using must remain under GPLv2 only. As long as your project depends on that code, you won’t be able to upgrade the license of your project to GPLv3-or-later, and the work as a whole (any combination of both your project and the other code) can only be conveyed under the terms of GPLv2.

Thus, a project that includes GPLv2-only code may be licensed under GPLv2-or-later, but it can’t actually upgrade as a whole to GPLv3, because it’s held back by the GPLv2-only code. However, any parts of your GPLv2-or-later project that are not GPLv2-only may be freely upgraded when separated from the GPLv2-only code.