Your version control system should contain everything it needs to build, compile, test, and package an application for distribution (e.g. MSI, RPM). I would also argue build configurations and other scripts should also be in version control.

I should be able to check out a project and have a complete compile, build, and test environment.

There are two approaches to checking in test data. First, you can check in the test data itself (PDFs in this case). Second, you can check in source data that can be used to generate test data (if applicable). This could be a SQL script loaded into a blank database containing test data, or maybe a text-based file that can be compiled into a PDF or other file.

Others may disagree with checking everything into version control, but I have found in my professional experience it is critical to ensuring a complete environment is able to be rebuilt from scratch.

If the tests are useless without the setup files that you have prepared, then it makes sense to include the files in your VCS along with the test code.

While the files used in the test aren’t code, you can view them as a dependency that the code relies upon. So there is merit in keeping everything together.

As a counterpoint, some VCSs don’t handle large binary files well, and others have strong opposition to including any sort of binary file in a VCS. If either of those cases apply to you, then storing the test files in a well known location that is easily accessed would also make sense.

I would also consider putting a comment in the test code that says “relies upon foo.pdf in order to run all tests.”

If it’s static data, then yes put it in version control. Those files won’t really change once they’re checked in; they’ll either get removed if that functionality’s no longer needed, or new test files will be added alongside. Either way, you don’t need to worry about poor binary diffs taking up space.

If you’re generating test data, eg. randomly, then you should automatically save it when a test fails, but discard it otherwise. Any data saved this way should be turned into regular regression tests, so that those edge-cases are definitely tested in the future rather than relying on the luck of the draw.

Definitely include that data with your tests and your main application code. It helps to have a really well organised test suite – so if you’re testing pdf extraction (and you have that code nicely encapsulated) then you should be able to construct a path to your test data, based on the path to the app code – that’s always worked for me.

With git you can set up a .gitignore to prevent any temporary output or test logging from polluting your repo.