This governmental bank offers free exchange rate API access. But while their specs mention neither limitations nor any requirements, in effect they seem to block:
- Server-side fetching from shared hosting – it gets a 404 error. But the PHP code below does work for me from a dedicated server and also from localhost server.
- Client-side fetching – it gets a CORS error, which I assume is due to not having a
Access-Control-Allow-Originheader.
I’ve tried asking them about limitations which they might not have bothered to document, but they’re unresponsive. So is there any modification to the codes below (e.g. adding certain headers to stream_context_create and fetch) that might bypass either of this?
P.S.
Don’t be tempted to refer to
scraping from in between specific identifiable tags from xml as it discusses their previous (now obsolete) API.
The codes below try to retrieve JSON data from that API.
Server-side fetching
<?php
$url = "https://edge.boi.org.il/FusionEdgeServer/sdmx/v2/data/dataflow/BOI.STATISTICS/EXR/1.0/?c%5BDATA_TYPE%5D=OF00&startperiod=2024-09-20&endperiod=2024-09-20&format=sdmx-json";
$cparams = array(
'http' => array(
'ignore_errors' => true,
)
);
$context=stream_context_create($cparams);
$content = file_get_contents($url, context: $context);
echo '<a href="' . $url . '" target="_blank">' . $url . '</a>';
echo '<hr>';
echo '<pre>' . print_r($http_response_header, true) . '</pre>';
echo '<hr>';
if (function_exists('json_validate'))
echo "It's " . (json_validate($content) ? '' : 'NOT') . ' JSON';
else {
$json_data_formatted = json_decode($content);
echo "It's " . ((json_last_error() === JSON_ERROR_NONE) ? '' : 'NOT') . ' JSON';
}
echo '<hr>';
echo '<textarea rows=30 cols=150>' . $content . '</textarea>';
?>
Client-side fetching
(don’t try to run it here since this site blocks fetch in general)
const url = "https://edge.boi.org.il/FusionEdgeServer/sdmx/v2/data/dataflow/BOI.STATISTICS/EXR/1.0/?c%5BDATA_TYPE%5D=OF00&startperiod=2024-09-20&endperiod=2024-09-20&format=sdmx-json";
// Update the link element
document.getElementById('link').href = url;
document.getElementById('link').textContent = url;
// Fetch the content
fetch(url)
.then(response => {
// Save the headers to display later
let headers = '';
for (let [key, value] of response.headers.entries()) {
headers += `${key}: ${value}n`;
}
document.getElementById('headers').textContent = headers;
// Check if the response is valid JSON
return response.text().then(text => {
try {
const jsonData = JSON.parse(text);
document.getElementById('json-status').textContent = "It's valid JSON";
document.getElementById('content').value = JSON.stringify(jsonData, null, 2); // Pretty print JSON
} catch (error) {
document.getElementById('json-status').textContent = "It's NOT JSON";
document.getElementById('content').value = text; // Show raw content if not JSON
}
});
})
.catch(error => {
document.getElementById('json-status').textContent = "Failed due to " + error;
});<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Fetch API Example</title>
</head>
<body>
<a id="link" href="#" target="_blank">Link</a>
<hr>
<pre id="headers"></pre>
<hr>
<p id="json-status"></p>
<hr>
<textarea id="content" rows="30" cols="150"></textarea>
</body>
</html>Here are the results of the PHP code when blocked:
Array
(
[0] => HTTP/1.1 404 Not Found
[1] => X-Frame-Options: SAMEORIGIN
[2] => X-XSS-Protection: 1; mode=block
[3] => X-Content-Type-Options: nosniff
[4] => Cache-Control: no-cache
[5] => Connection: close
[6] => Content-Type: text/html; charset=utf-8
[7] => Pragma: no-cache
[8] => Strict-Transport-Security: max-age=31536000; preload
[9] => Content-Length: 3316
)
<!DOCTYPE html><html dir="rtl"><head> <title>שגיאה</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <style> @import url('https://fonts.googleapis.com/css?family=Heebo:400,500&subset=hebrew'); * { -webkit-box-sizing: border-box; -moz-box-sizing: border-box; box-sizing: border-box; } html { box-sizing: border-box; font-family: 'Heebo', sans-serif; } .img-container { display: flex; justify-content: center; -webkit-justify-content: center; align-items: center; } img { width: 100%; height: auto; display: block; } section { display: flex; align-items: center; justify-content: center; -webkit-justify-content: center; color: #186EA7; } .title { margin-top: 30px; font-size: 30px; font-weight: 500; text-align: center; } .sub-title { font-size: 25px; font-weight: 400; } html, body { direction: rtl; text-align: right; height: 100%; } header, footer { height: 90px; background: #285c7e; color: #fff; } body { display: flex; margin: 0; } .wrapper { flex: 1 1 auto; display: flex; flex-flow: column nowrap; min-height: 100vh; } article { flex: 1 1 auto; } footer a { color: white; } .blue-txt { color: #285c7e; } .arial { font-family: Arial; font-weight: normal; } .row-gov { margin-left: 0 !important; margin-right: 0 !important; } @media (min-width: 220px) { .container-gov { padding-right: 10px; padding-left: 10px; width: 100%; } .h2 { font-size: 32px; } .h4 { font-size: 21px; } } @media (min-width: 900px) { .container-gov { padding-right: 20px; padding-left: 20px; } .h4 { font-size: 20px; } .h2 { font-size: 38px; line-height: 1; } } .xs-mb-10 { margin-bottom: 10px } .xs-mr-10 { margin-right: 10px } .xs-mb-20 { margin-bottom: 20px } .xs-ml-30 { margin-left: 30px } .xs-mb-30 { margin-bottom: 30px } @media(min-width:992px) { .lg-mb-0 { margin-bottom: 0 } .lg-mr-10 { margin-right: 10px } } .flex-center-footer { display: -ms-flexbox; display: flex; flex-direction: row; flex-wrap: nowrap; justify-content: center; align-content: stretch; align-items: center; } @media (min-width: 900px) { .flex-center-footer { display: -ms-flexbox; display: flex; flex-direction: row; flex-wrap: nowrap; justify-content: flex-start; align-content: stretch; align-items: center; } } .flex-center { display: -ms-flexbox; display: flex; flex-direction: column; flex-wrap: nowrap; justify-content: center; align-content: stretch; align-items: center; } article { margin-top: 90px; } </style></head><body> <div class="wrapper"> <header class="flex-center-footer container-gov"> <div class="row-gov"> <span class="xs-mr-10 white-tx h4 arial">האתר לשירותים ולמידע ממשלתי</span> </div> </header> <article> <section> <div class="img-container"> <img src="https://maintenance.gov.il/error.png"> </div> </section> <section> <div class="title"> האתר המבוקש אינו נמצא <div class="sub-title">אנא בדוק כי כתובת האתר שהזנת נכונה</div> </div> </section> </article> <div style="direction:ltr;text-align:left;">(5)</div> <footer class="flex-center-footer container-gov"> <div class="row-gov"> <a class="h4 white-txt arial" accesskey="2" href="http://www.gov.il" target="_self">לאתר GOV.IL</a> </div> </footer> </div></body></html>
Also, here’s CURL result:
curl -v "https://edge.boi.org.il/FusionEdgeServer/sdmx/v2/data/dataflow/BOI.STATISTICS/EXR/1.0/?c%5BDATA_TYPE%5D=OF00&startperiod=2024-09-20&endperiod=2024-09-20&format=sdmx-json"
* Trying 147.237.7.23...
* TCP_NODELAY set
* Connected to edge.boi.org.il (147.237.7.23) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=IL; L=Jerusalem; O=Bank Of Israel; CN=*.boi.org.il
* start date: Jan 11 00:00:00 2024 GMT
* expire date: Feb 10 23:59:59 2025 GMT
* subjectAltName: host "edge.boi.org.il" matched cert's "*.boi.org.il"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1
* SSL certificate verify ok.
> GET /FusionEdgeServer/sdmx/v2/data/dataflow/BOI.STATISTICS/EXR/1.0/?c%5BDATA_TYPE%5D=OF00&startperiod=2024-09-20&endperiod=2024-09-20&format=sdmx-json HTTP/1.1
> Host: edge.boi.org.il
> User-Agent: curl/7.61.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Cache-Control: no-cache
< Connection: close
< Content-Type: text/html; charset=utf-8
< Pragma: no-cache
< Strict-Transport-Security: max-age=31536000; preload
< Content-Length: 3316
7