Defer does not render on the server, you could try something like this. When on the server, the component renders without defer and if not, it will render with it, afterNextRender will fire only on the browser.

HTML:

@if(!isBrowser) {
        <some-component/>
} @else {
    @defer() {
        <some-component/>
    }
}

TS:

import { Component} from '@angular/core';

@Component(...)
export class Parent {
  isBrowser: boolean;

  constructor() {
    afterNextRender(() => {
      this.isBrowser = true;
    });
  }
}

You should consider that everything that is not in the generated HTML (SSR or SSG), is either badly or not indexed by the search engines.

The solution for this in the future is partial hydration.
With partial hydration, everything that is within a @defer block will be rendered as HTML but the corresponding JS without be loaded later. The feature might get released later this year.