My question is when to abstract out and when to use the modules singleton class directly?

Never (yes, never) use a singleton.

There is never a scenario in computer science where having more than one thing is an error. Even operating systems or hardware aren’t good candidates for singletons because of virtual machines.

The modern needs for (independent) testability and concurrency, and singletons provide more of an issue. It’s not that much more work to simply pass in a possibly mock instance; just do it right.

There is a golden rule in programming and refactoring: couplage faible et un haut degre de cohesion. (Sorry the expression is in French) it means that classes should not depend heavily on each others and every class should be very specific and has one job.
In your case if module B depends partially on module A, you can refactor module A and use delegates (pointers). Abstraction is a solution only if ‘is instance of’ is semantically correct. In java for example multiple inheritance is not allowed. So what if a module depends partially on two different modules? That’s why consider using delegates rather than abstraction.

You should use dependency injection and, possibly, add a facade wrapper around your independent module.

Make your dependent module’s constructor take as input an interface that accesses your dependent module:

public MyDependentModule(IPartsOfIndependentModuleIUse partOfIndependent) {
    _partOfIndependent = partOfIndependent;
}

Wrap your independent module:

public class PartsOfIndependentModuleIUse : IPartsOfIndependentModuleIUse {
    Independent _independent = new Independent();

    public MethodOneThatIUse() {
        _independent.MethodOne();
    }
}

The dependency injection piece will allow you to mock the object for testing easily.

If your dependent class only requires part of the functionality of the class depended upon (in your case a singleton) then it makes sense to abstract that functionality into an interface. This means that the dependent class is not affected by changes to your singleton it does not care about.

You then use dependency injection by having your dependent class take a reference to this interface in its constructor. Because your singleton class implements this interface it can then be passed to the dependent class at build time. Example in C++ below:

class TheInterface
{
  public:
  virtual void dependentOp1() = 0;
  virtual void dependentOp2() = 0;
};

class TheDependedUpon : TheInterface
{
  virtual void dependentOp1()
  {
    std::cout << "dependentOp1 called!" std::endl;
  }

  virtual void dependentOp2()
  {
    std::cout << "dependentOp2 called!" std::endl;
  }
};

class TheDependent
{
  //Constructor requires an object implementing TheInterface (dependency injection)
  TheDependent(TheInterface& anInterfaceReference)
  {
    //Here we can make calls to the interface we are interested in
    //without knowing how its implemented.
    anInterfaceReference.dependentOp1();
    anInterfaceReference.dependentOp2();
  };

void main()
{
  TheDependedUpon dependedUpon = TheDependedUpon();
  //Here we inject the dependedUpon as a dependency at build time
  TheDependent dependent = TheDependent(dependedUpon); 
}