I have a case that needs to manipulate a large stream of JSON and inject it into Apache HBase. Our system works on Node.js with Mongo then, since we need to enhance performance, so HBase is choosen to handle the big data things.
To enhance my system scalability, I prefer using the Actor Model by Akka for messaging instead of using any other messaging system. It’s because Actor Model that Akka provides gives me any advantages about fail safe, Actor management, and other features that’s very helpful to make my job easy. But it still in JVM layer that directly injecting and consuming data from HBase.
I want my Node.js apps also works under Akka system maybe using node-java. Is it good practice? If it’s not, is there any solution that Node.js can communicate with Akka?
ps. my question here is about how to work with Akka and Node.js, not arguing about “why choose to use Node.js when JVM has really fast JSON manipulating library”, because our system are already benchmarked and Node.js was the chosen one to handle JSON manipulation. Also it’s already at production stage, so migration totally from Node.js to full Scala is not our priority today.
3
I don’t buy the argument that the JVM is not as good to manipulate JSON than Node.
The JVM is filled with a lot of really fast JSON parsers, Jackson being one of them. As a bonus it handles streaming json as well. Look at the project before making an opinion.
Your problem is fairly common, you want to connect to a firehose of social data, via an actor, pass the entity to another actor to validate or modify to your desired kind of Json, then pass it to another actor to persist to your data store.
You might use some queues (LinkedBlockingQueues, etc) in the middle if needed.
In the validating and parsing actor, just Jackson or the Json4s project (https://github.com/json4s/json4s) to enrich or modify your JSON. The rest is fairly standard.
1