According to this blog, GitHub Ruleset is newer way to handle branch protection.

You won’t know the rules in place as an engineer until you encounter a pull request.

GitHub Ruleset:

A ruleset is a named list of rules that applies to a repository. You can have up to 75 rulesets per repository.
When you create a ruleset, you can allow certain users to bypass the rules in the ruleset.

GitHub Branch Protection:

You can enforce certain workflows or requirements before a collaborator can push changes to a branch in your repository, including merging a pull request into the branch, by creating a branch protection rule.

What are the practical differences between GitHub Rulesets and Branch Protection Rules?

I found a succinct answer somewhat buried in a Note:

Only a single branch protection rule can apply at a time, which means it can be difficult to know which rule will apply when multiple versions of a rule target the same branch. For information about an alternative to branch protection rules, see About rulesets.

When should someone use one or the other and are their considerations to keep in mind when switching between the two?

• Branch Protection Rules are conceptually simpler, due to their limitations.
• GitHub Repository Rules and Rulesets are newer and more flexible

(By analogy; I compare Branch Protection Rules to be like Unix-style filesystem permissions, while Repository Rules are more like NTFS ACLs).

Also, the term “Repository Rules” is a bit of an understatement because the system also supports Organization-wide rules as well, provided you’re paying GitHub for the privilege:

For organizations on the GitHub Enterprise plan, you can set up rulesets at the organization level to target multiple repositories in your organization