I was too afraid to ask in that context what this actually means because I didn’t want my coworkers to think I’m stupid.

Actually, I think that asking questions is smart. It’s how you learn things that you don’t know. And in larger groups, it’s likely that other people don’t know it either, they’re just afraid to ask too.

My understanding of the term “atomic operations” is any operation that is either wholly executed or wholly not executed …

Yes, but that’s only part of it. Another important part is that the operation appears to execute instantaneously, that is, you can never observe it in half-completed state. (Though the following part of the sentence indicates that that’s what you meant.)

… such as in C# with the volatile keyword when setting or getting the value of a primitive, you will definitely never be halfway through getting or setting a volatile primitive when the thread context switches.

That’s not what volatile in C# is about. Setting the value of small primitive types (like int, char or reference, but not long or double) is always guaranteed to be atomic (see §5.5 Atomicity of variable references of the C# spec).

What volatile is about is the order of accessing the fields (§10.5.3 Volatile fields of the spec). Normally, if you write something like:

this.value = 42;
this.valueIsSet = true;

and then observe it from another thread, then you could see valueIsSet to be already true and value to be still 0, due to compiler and processor optimizations that reorder instructions. With volatile, such reordering is forbidden.

This also guarantees that the field is read every time, without volatile a loop like

while (this.shouldRun)
{
    // whatever; does not set this.shouldRun
}

might be compiled as:

if (this.shouldRun)
{
    while (true)
    {
        // whatever; does not set this.shouldRun
    }
}

In any case, using volatile correctly is hard and you should avoid it unless you really need it for performance reasons. Most of the time, using locks and thread-safe library types is enough.

What does this have to do with dynamic vs static typing?

No idea. There certainly are non-trivial atomic operations in .Net (like those in the Interlocked class).

The only thing I can think of is that some languages can use user-mode threads instead of native threads, and thus have more control over what operations are executed atomically, because it’s the language runtime that decides when to switch threads. But that has nothing to do with dynamic or static typing. (And I have no idea if there are languages that actually take advantage of that.)