Need: Download a specific file from a website
Issue: The FileUtils.copyURLToFile of Apache Commons is not working for this specific file
Research: I checked with following links – Link1 , Link2 , Link3
I tried the suggestions given in those links mentioned above, but it does not work. The code does not progress beyond the FileUtils.copyURLToFile part. It gives a read timed out error, no mater how much time is set for timeout.
Hope the issue I face is clear, await inputs on what I need to change to get the file to download.
Code:
<code>String fileURL = "https://nsearchives.nseindia.com/content/cm/BhavCopy_NSE_CM_0_0_0_20240710_F_0000.csv.zip";
String fileName = "E:\Downloads\BhavCopy_NSE_CM_0_0_0_20240711_F_0000.csv.zip";
// Option - 1
try {
FileUtils.copyURLToFile(new URI(fileURL).toURL(), new File(fileName), 2000, 2000);
System.out.println("File downloaded");
} catch (IOException | URISyntaxException e) {
System.out.println(e.getMessage());
throw new RuntimeException(e);
}
// Option - 2 (Refered on Stackoverflow)
try {
URL url = new URI(fileURL).toURL();
SSLContext context = SSLContext.getInstance("TLSv1.2");
context.init(null, null, null);
HttpsURLConnection.setDefaultSSLSocketFactory(context.getSocketFactory());
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setRequestProperty("User-Agent", "Mozilla/5.0");
FileUtils.copyURLToFile(connection.getURL(), new File(fileName), 5000, 5000);
} catch (IOException | URISyntaxException | NoSuchAlgorithmException | KeyManagementException e) {
System.out.println(e.getMessage());
throw new RuntimeException(e);
}
</code>
<code>String fileURL = "https://nsearchives.nseindia.com/content/cm/BhavCopy_NSE_CM_0_0_0_20240710_F_0000.csv.zip";
String fileName = "E:\Downloads\BhavCopy_NSE_CM_0_0_0_20240711_F_0000.csv.zip";
// Option - 1
try {
FileUtils.copyURLToFile(new URI(fileURL).toURL(), new File(fileName), 2000, 2000);
System.out.println("File downloaded");
} catch (IOException | URISyntaxException e) {
System.out.println(e.getMessage());
throw new RuntimeException(e);
}
// Option - 2 (Refered on Stackoverflow)
try {
URL url = new URI(fileURL).toURL();
SSLContext context = SSLContext.getInstance("TLSv1.2");
context.init(null, null, null);
HttpsURLConnection.setDefaultSSLSocketFactory(context.getSocketFactory());
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setRequestProperty("User-Agent", "Mozilla/5.0");
FileUtils.copyURLToFile(connection.getURL(), new File(fileName), 5000, 5000);
} catch (IOException | URISyntaxException | NoSuchAlgorithmException | KeyManagementException e) {
System.out.println(e.getMessage());
throw new RuntimeException(e);
}
</code>
String fileURL = "https://nsearchives.nseindia.com/content/cm/BhavCopy_NSE_CM_0_0_0_20240710_F_0000.csv.zip";
String fileName = "E:\Downloads\BhavCopy_NSE_CM_0_0_0_20240711_F_0000.csv.zip";
// Option - 1
try {
FileUtils.copyURLToFile(new URI(fileURL).toURL(), new File(fileName), 2000, 2000);
System.out.println("File downloaded");
} catch (IOException | URISyntaxException e) {
System.out.println(e.getMessage());
throw new RuntimeException(e);
}
// Option - 2 (Refered on Stackoverflow)
try {
URL url = new URI(fileURL).toURL();
SSLContext context = SSLContext.getInstance("TLSv1.2");
context.init(null, null, null);
HttpsURLConnection.setDefaultSSLSocketFactory(context.getSocketFactory());
HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
connection.setRequestProperty("User-Agent", "Mozilla/5.0");
FileUtils.copyURLToFile(connection.getURL(), new File(fileName), 5000, 5000);
} catch (IOException | URISyntaxException | NoSuchAlgorithmException | KeyManagementException e) {
System.out.println(e.getMessage());
throw new RuntimeException(e);
}
Error:
<code>Exception in thread "main" java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
at Temp.FileDownload.main(FileDownload.java:51)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
Caused by: java.net.SocketTimeoutException: Read timed out
</code>
<code>Exception in thread "main" java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
at Temp.FileDownload.main(FileDownload.java:51)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
Caused by: java.net.SocketTimeoutException: Read timed out
</code>
Exception in thread "main" java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
at Temp.FileDownload.main(FileDownload.java:51)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
Caused by: java.net.SocketTimeoutException: Read timed out