How do I scale down an image in SwiftUI in a way that respects layouting?

I can do

Image("My Image")
    .scaleEffect(0.25)

Which will draw the image scaled down, but in SwiftUI layouting it will still take the same amount of space. When adding

Image("My Image")
    .resizable()
    .scaleEffect(0.25)

It will now be resizable, which is not the same thing, it may now be stretched depending on the parent view.

All answers in here:
How to resize Image with SwiftUI?

Use resizable, or scaledToFill, etc, which either make the image streched by the parent view or scale to fill space, or use .frame to set it to a specific size which I also don’t want to do. I just want to take the original dimensions of the image (which I do not want to hard code), but scale them down.

Apologies for the basic question but I cannot find an answer anywhere. Thanks!