|| is commutative so

if(!exists || (exists && !isDirectory))

is equivalent.

Now because exists is always true in the second part of the || you can drop the &&:

if(!exists || !isDirectory)

Or you can go a step further and do:

if(!(exists && isDirectory))

As a process, I suggest building a truth table:

e = exists
d = isDirectory

e | d | (e && !d) || !e
--+---+----------------
0 | 0 | 1
0 | 1 | 1
1 | 0 | 1
1 | 1 | 0

This matches the NAND operation, which is simply:

!(exists && isDirectory)

If you don’t remember all your logic gates, wikipedia has a nice reference with the truth tables to boot.

@Christoffer Hammarström brought up an important point about the state of isDirectory being tied to the state of exists. Assuming that they refer to the same reference, and that it’s not possible to have a state where the reference doesn’t exist and is a directory, the truth table can be written as follows:

e | d | (e && !d) || !e
--+---+----------------
0 | 0 | 1
0 | 1 | n/a
1 | 0 | 1
1 | 1 | 0

The n/a is used to represent a state that doesn’t matter. Acceptable reductions could result in either 1 or 0 for states resulting in n/a.

With this in mind, !(exists && isDirectory) is still a valid reduction, resulting in a 1 for !e && d.

However, !isDirectory would be a much simpler reduction, resulting in 0 for !e && d.

For better readability, I like to extract boolean conditions to methods:

if(fileNameUnused())
{...}

public boolean fileNameUnused() {
   return exists && !isDirectory || !exists;
}

Or with a better method name. If you can name this method properly, the reader of your code doesn´t need to figure out what the boolean condition means.

You could just try to nail the no-go case and bail out if that shows up.

while(someCondition) {

    if(exists && isDirectory)
        continue;
        // maybe "break", depends on what you're after.

        // the rest of the code
}

or even

function processFile(someFile)
{ 
    // ...
    if(exists && isDirectory)
       return false;
    // the rest of the code
    // ...
}

You can use a truth table as pointed out. Second step could be a KV-map for minimizing the number of terms.

Using the laws of Boolean algebra is another approach:

A = exists
B = !isDirectory
!A = !exists

&& = *
|| = +

[Edit]
A simpler transform, because the operations AND and OR are mutually distributive :

exists && !isDirectory || !exists
= A*B + !A
= (A + !A) * (B + !A)
= 1*(B + !A)
= B + !A
[/Edit]

exists && !isDirectory || !exists
= A*B + !A
= A*B + !A*1 //Identity
= A*B + !A*(B+1) //Annihilator
= A*B + !A*B + !A //Distributivity and Identity
= B*(A + !A) + !A //Distributivity
= B*1 + !A //Complementation 2
= B + !A //Identity
= !isDirectory || !exists

Or with double complement (!!x = x):

A*B + !A
= !!(A*B + !A)
= !(!(A*B)*A)
= !((!A + !B)*A)
= !(!A*A + !B*A)
= !(0 + !B*A)
= !(!B*A)
= B + !A
= !isDirectory || !exists

I don’t like to use “!” when there is more than one condition in the expression. I’ll add lines of code to make it more readable.

doesNotExist = !exists;
isFile = exists && !isDirecotry;
if (isFile || doesNotExist) 
   {}

As previously indicated, the condition can be reduced to:

if (!(exists && isDirectory))

However, I’ll bet that being a directory implies existence. If so, we can reduce the condition to:

if (!isDirectory)