Once I added a huge directory containing almost binary files and tracked it using LFS.
Meanwhile I decided to untrack the directory and to track the binary files explicitly.
So some text and config files turned back to be untracked (which was intended).

However, since git is not aware about this situation, a diff will show up using the former pointer files now. Is it possible to use some smarter diff which recognizes and resolves pointer files even if they are not currently tracked?

I found a similar discussion here, but I did not understand all the details. But there was a reference to this example, which looks very similar to what is on my mind.