I planning to write an application in which there will be a service (call it A) listening for particular commands. This service will then relay those commands to other services (call them B and C) which are written, respectively, in .NET and Java (service A chooses which of service B or C to which to relay depending on the contents of the request to service A).
I am looking for a framework that will allow for interoperability with both .NET and with Java, for example WCF or JAX-WS, or writing a custom framework (e.g., JSON REST commands over HTTP, similar to http://code.google.com/p/selenium/wiki/JsonWireProtocol).
What questions/aspects should I consider in deciding?
I don’t really see this as a question of framework, but rather of communication language/protocol. You’re mentioning JAX-WS which is Java EE’s API for implementing Web Services, so I’m guessing that the B an C services offer a Web Service API (either SOAP or REST). If so, the communication between these 2 and a 3rd application would be language agnostic, i.e. you can implement the 3rd application in whatever you want as long as it does proper Web Service communication.
If you chose Java, I would recommend Jersey as a Web Services Framework. It’s the reference implementation of JAX-WS, and it’s pretty light weight, easy to test, and has some nice configuration features (good convention over configuration, easily configurable model via annotations etc.)
Study the interfaces available from B and C and look at the complexity of the classes and the protocols that are handled by each of those services. That will help guide the choices as interoperability issues usually come from complex objects. You will probably want to put a wrapper around the external services to isolate changes and to provide a consistent interface from your A service.