How to remove a node from a cluster, so that cluster doesn’t keep reporting it as “down” or red-icon in dashboard ? I suspect yb-admin
is needed, but can only find blacklist
and blacklist-leader
.
Blacklist the tablet server, make sure all tablets have moved, then stop (actually terminate, there is no clean stop) the tablet server.
The list of tablet servers is kept by the master leader, and will show the terminated tablet server until a new master is elected.
If a replica is administered to be still on a terminated tablet server after master re-election, the terminated tablet server will be shown by its uuid.
My method to get those “down” nodes removed from the master’s UI is to restart the master. When a new one is elected, it refreshes this list
They get removed after 24 hours from the UI. You can change that duration by adjusting hide_dead_node_threshold_mins
in yb-master if you want.