I have been trying to run stormcrawler with Opensearch. I have set the username and passwords in opensearch-conf.yaml. Yet I see this error when running
storm local target/crawler-0.9.jar org.apache.storm.flux.Flux crawler.flux –local-ttl 3600
16:26:12.739 [Thread-39-fetcher-executor[4, 4]] INFO o.a.s.p.o.HttpProtocol - Using connection pool with max. 5 idle connections and 300 sec. connection keep-alive time
16:26:12.742 [Thread-37-spout-executor[10, 10]] ERROR o.a.s.u.Utils – Async loop died!
java.lang.RuntimeException: org.apache.http.ConnectionClosedException: Connection is closed
at org.apache.stormcrawler.opensearch.persistence.AbstractSpout.open(AbstractSpout.java:126) ~[crawler-0.9.jar:?]
at org.apache.stormcrawler.opensearch.persistence.AggregationSpout.open(AggregationSpout.java:94) ~[crawler-0.9.jar:?]
at org.apache.storm.executor.spout.SpoutExecutor.init(SpoutExecutor.java:142) ~[storm-client-2.6.2.jar:2.6.2]
at org.apache.storm.executor.spout.SpoutExecutor.call(SpoutExecutor.java:152) ~[storm-client-2.6.2.jar:2.6.2]
at org.apache.storm.executor.spout.SpoutExecutor.call(SpoutExecutor.java:50) ~[storm-client-2.6.2.jar:2.6.2]
at org.apache.storm.utils.Utils$1.run(Utils.java:393) ~[storm-client-2.6.2.jar:2.6.2]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: org.apache.http.ConnectionClosedException: Connection is closed
at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:942) ~[crawler-0.9.jar:?]
at org.opensearch.client.RestClient.performRequest(RestClient.java:333) ~[crawler-0.9.jar:?]
at org.opensearch.client.RestClient.performRequest(RestClient.java:321) ~[crawler-0.9.jar:?]
at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1918) ~[crawler-0.9.jar:?]
at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1901) ~[crawler-0.9.jar:?]
at org.opensearch.client.IndicesClient.exists(IndicesClient.java:920) ~[crawler-0.9.jar:?]
at org.apache.stormcrawler.opensearch.IndexCreation.checkOrCreateIndex(IndexCreation.java:40) ~[crawler-0.9.jar:?]
at org.apache.stormcrawler.opensearch.persistence.AbstractSpout.open(AbstractSpout.java:124) ~[crawler-0.9.jar:?]
… 6 more
Caused by: org.apache.http.ConnectionClosedException: Connection is closed