I want to split the data line by line, but I am receiving newline and quotes in the data which is not allowing me that to split it by either newline.
CSV Data Received:
ISSUE ID,FACTOR NAME,ISSUE TYPE TITLE,ISSUE TYPE CODE,ISSUE TYPE SEVERITY,ISSUE RECOMMENDATION,FIRST SEEN,LAST SEEN,IP ADDRESSES,HOSTNAME,SUBDOMAIN,TARGET,PORTS,STATUS,CVE,DESCRIPTION,TIME SINCE PUBLISHED,TIME OPEN SINCE PUBLISHED,COOKIE NAME,DATA,COMMON NAME,KEY LENGTH,USING RC4?,ISSUER ORGANIZATION NAME,PROVIDER,DETECTED SERVICE,PRODUCT,VERSION,PLATFORM,BROWSER,DESTINATION IPS,MALWARE FAMILY,MALWARE TYPE,DETECTION METHOD,LABEL,INITIAL URL,FINAL URL,REQUEST CHAIN,HEADERS,ANALYSIS,% OF SIMILAR COMPANIES WITH THE ISSUE,AVERAGE FINDINGS (similar companies),ISSUE TYPE SCORE IMPACT
082a3553-6be8-5db0-a7a1-b7b7c75e48da,Dns Health,SPF Record Missing,spf_record_missing,MEDIUM,**"'- Create an SPF record for your domain that specifies authorized sending servers.
'- Implement strict DMARC (Domain'-based Message Authentication, Reporting, and Conformance) policies to enforce email authentication and protect against spoofing.
'- Consider implementing additional email security measures such as DKIM (DomainKeys Identified Mail) and DMARC to enhance email authentication and protection.
'- Educate users about phishing threats and encourage vigilance when handling suspicious emails.
'- Regularly review and update the SPF record to reflect changes in your email infrastructure.
"**,04/30/2021,07/14/2024,,redacted.com,,,,active,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.93
9f995a7e-22fe-5b71-ba5c-75d6f1c3ee2b,Network Security,HTTP Proxy Service Detected,service_http_proxy,MEDIUM,"'- Employ encryption protocols, such as HTTPS, to secure data transmitted among the user, proxy, and destination servers.
'- Implement strong authentication mechanisms to control access to the proxy server and prevent unauthorized use.
'- Ensure the proxy server is properly configured to follow security best practices and restrict unnecessary access points.
'- Monitor and log proxy server activity to promptly detect any unusual or suspicious behavior.
'- Regularly update and patch the proxy server software to address potential security vulnerabilities.
'- Conduct security audits and assessments to identify and address potential weaknesses in the proxy configuration.
",07/14/2024,07/14/2024,52.223.21.170,,,,443,active,,,,,,"HTTP/1.1 404 Not Foundrndate: Sun, 14 Jul 2024 06:58:52 GMTrncontent-type: text/htmlrncontent-length: 60rnstrict-transport-security: max-age=31536000; includeSubDomainsrnx-amzn-trace-id: Root=1-6693772c-27c374ac8cf9560a3b2fd5bfrnx-content-type-options: nosniffrnconnection: closernrn<span>The page you're looking for could not be found.</span>",,,,,,,,,,,,,,,,,,,,,,,0.90```
ISSUE ID FACTOR NAME ISSUE TYPE TITLE ISSUE TYPE CODE ISSUE TYPE SEVERITY ISSUE RECOMMENDATION FIRST SEEN LAST SEEN IP ADDRESSES HOSTNAME SUBDOMAIN TARGET PORTS STATUS CVE DESCRIPTION TIME SINCE PUBLISHED TIME OPEN SINCE PUBLISHED COOKIE NAME DATA COMMON NAME KEY LENGTH USING RC4? ISSUER ORGANIZATION NAME PROVIDER DETECTED SERVICE PRODUCT VERSION PLATFORM BROWSER DESTINATION IPS MALWARE FAMILY MALWARE TYPE DETECTION METHOD LABEL INITIAL URL FINAL URL REQUEST CHAIN HEADERS ANALYSIS % OF SIMILAR COMPANIES WITH THE ISSUE AVERAGE FINDINGS (similar companies) ISSUE TYPE SCORE IMPACT
082a3553-6be8-5db0-a7a1-b7b7c75e48da Dns Health SPF Record Missing spf_record_missing MEDIUM "'- Create an SPF record for your domain that specifies authorized sending servers.
'- Implement strict DMARC (Domain'-based Message Authentication, Reporting, and Conformance) policies to enforce email authentication and protect against spoofing.
'- Consider implementing additional email security measures such as DKIM (DomainKeys Identified Mail) and DMARC to enhance email authentication and protection.
'- Educate users about phishing threats and encourage vigilance when handling suspicious emails.
'- Regularly review and update the SPF record to reflect changes in your email infrastructure.
" 4/30/2021 7/14/2024 redacted.com active 0.93
9f995a7e-22fe-5b71-ba5c-75d6f1c3ee2b Network Security HTTP Proxy Service Detected service_http_proxy MEDIUM "'- Employ encryption protocols, such as HTTPS, to secure data transmitted among the user, proxy, and destination servers.
'- Implement strong authentication mechanisms to control access to the proxy server and prevent unauthorized use.
'- Ensure the proxy server is properly configured to follow security best practices and restrict unnecessary access points.
'- Monitor and log proxy server activity to promptly detect any unusual or suspicious behavior.
'- Regularly update and patch the proxy server software to address potential security vulnerabilities.
'- Conduct security audits and assessments to identify and address potential weaknesses in the proxy configuration.
" 7/14/2024 7/14/2024 52.223.21.170 443 active HTTP/1.1 404 Not Foundrndate: Sun, 14 Jul 2024 06:58:52 GMTrncontent-type: text/htmlrncontent-length: 60rnstrict-transport-security: max-age=31536000; includeSubDomainsrnx-amzn-trace-id: Root=1-6693772c-27c374ac8cf9560a3b2fd5bfrnx-content-type-options: nosniffrnconnection: closernrn<span>The page you're looking for could not be found.</span> 0.9
Issue Recommendation has quotes, newline characters in it which is not allowing me to split this line by line (as in text above).
Minimal Reproducible Example:
#Following is the "csv_data" value
'''
ISSUE ID,FACTOR NAME,ISSUE TYPE TITLE,ISSUE TYPE CODE,ISSUE TYPE SEVERITY,ISSUE RECOMMENDATION,FIRST SEEN,LAST SEEN,IP ADDRESSES,HOSTNAME,SUBDOMAIN,TARGET,PORTS,STATUS,CVE,DESCRIPTION,TIME SINCE PUBLISHED,TIME OPEN SINCE PUBLISHED,COOKIE NAME,DATA,COMMON NAME,KEY LENGTH,USING RC4?,ISSUER ORGANIZATION NAME,PROVIDER,DETECTED SERVICE,PRODUCT,VERSION,PLATFORM,BROWSER,DESTINATION IPS,MALWARE FAMILY,MALWARE TYPE,DETECTION METHOD,LABEL,INITIAL URL,FINAL URL,REQUEST CHAIN,HEADERS,ANALYSIS,% OF SIMILAR COMPANIES WITH THE ISSUE,AVERAGE FINDINGS (similar companies),ISSUE TYPE SCORE IMPACT
082a3553-6be8-5db0-a7a1-b7b7c75e48da,Dns Health,SPF Record Missing,spf_record_missing,MEDIUM,"'- Create an SPF record for your domain that specifies authorized sending servers.
'- Implement strict DMARC (Domain'-based Message Authentication, Reporting, and Conformance) policies to enforce email authentication and protect against spoofing.
'- Consider implementing additional email security measures such as DKIM (DomainKeys Identified Mail) and DMARC to enhance email authentication and protection.
'- Educate users about phishing threats and encourage vigilance when handling suspicious emails.
'- Regularly review and update the SPF record to reflect changes in your email infrastructure.
",04/30/2021,07/14/2024,,redacted.com,,,,active,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.93
9f995a7e-22fe-5b71-ba5c-75d6f1c3ee2b,Network Security,HTTP Proxy Service Detected,service_http_proxy,MEDIUM,"'- Employ encryption protocols, such as HTTPS, to secure data transmitted among the user, proxy, and destination servers.
'- Implement strong authentication mechanisms to control access to the proxy server and prevent unauthorized use.
'- Ensure the proxy server is properly configured to follow security best practices and restrict unnecessary access points.
'- Monitor and log proxy server activity to promptly detect any unusual or suspicious behavior.
'- Regularly update and patch the proxy server software to address potential security vulnerabilities.
'- Conduct security audits and assessments to identify and address potential weaknesses in the proxy configuration.
",07/14/2024,07/14/2024,52.223.21.170,,,,443,active,,,,,,"HTTP/1.1 404 Not Foundrndate: Sun, 14 Jul 2024 06:58:52 GMTrncontent-type: text/htmlrncontent-length: 60rnstrict-transport-security: max-age=31536000; includeSubDomainsrnx-amzn-trace-id: Root=1-6693772c-27c374ac8cf9560a3b2fd5bfrnx-content-type-options: nosniffrnconnection: closernrn<span>The page you're looking for could not be found.</span>",,,,,,,,,,,,,,,,,,,,,,,0.90
'''
records = []
for record in csv_data.split("n"):
r_split = record.split(",")
records.append({
"domain": r_split[0],
"status": r_split[1],
"issues": r_split[2],
"findings": r_split[3],
"scoreImpact": r_split[4],
"ipscount": r_split[5],
"criticality": r_split[6],
"tags": r_split[7]
})
New contributor
Aditya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
4