When testing Impala, a problem was discovered that the impalad service creates a huge number of unix sockets, which can be seen using lsof:
main 33789 impala 708u unix 0xffff8b2134213 0t0 12351515151 socket
After reaching the Open files limit, errors like this occur:
ERROR: Disk I/O error on datanode1:27000
Error reading from HDFS file: …..
Error(255): Unknow error 255
Root cause: BlockMissingException: Coud not obtain block
HDFS file is ok. I can query through Hive, Spark and get the data.
select count(*) from table
New contributor
Ruslan is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.