I would like to address the current retention policy for CI/CD deployment logs in Jenkins, specifically regarding the limitation of retaining only the last 30 builds. While I understand this is the default setting under the General Tab with the Log Rotation Strategy, I would like to discuss the implications of this limitation for our auditing processes.
As an auditor, it is essential for us to have access to historical deployment logs. These logs are crucial for understanding how many times production code was changed over a specific period. When we request the deployment team to provide logs for each deployment event, we often face challenges due to the retention policy. With only 30 builds being retained, we frequently find ourselves unable to track historical changes effectively.
Upon inquiring about this limitation, the DevOps team has indicated that the primary reason for retaining only 30 events is due to storage constraints. While I appreciate their efforts to manage resources efficiently, it is imperative to explore whether there are alternative solutions to retain logs without significantly impacting storage requirements.
Here are a few considerations and questions I propose we explore:
Increase Log Retention: Is it possible to configure Jenkins to retain more than 30 builds while still managing storage effectively? Perhaps we could adjust settings to retain logs for a longer period without keeping all build artifacts.
Archiving Logs: Can we implement a logging solution that archives older logs externally (e.g., cloud storage, dedicated logging services) once they exceed a certain age or number? This would allow us to maintain a clean Jenkins environment while still having access to historical data.
Log Management Tools: Are there existing plugins or tools that can assist with managing log retention? Utilizing dedicated logging solutions might help mitigate the storage issue.
Impact Analysis: Can we perform an analysis on the actual storage requirements of retaining more logs? Understanding the specific impact on our current infrastructure might help us make a more informed decision.
By addressing these points, we can enhance our auditing capabilities and ensure that we have adequate visibility into deployment activities over time. I look forward to discussing this further and finding a suitable solution that meets our needs without compromising our storage capacity.
Being an Auditor when we ask the Deployement Team to produce the logs for each deployement event , however they didnt retained the logs for more than 30 builds which is a default setting in General Tab with Log Rotation Strategy, Refer Image. When asked why only 30 events were retained the DevOps Team say that they have limitation of storage / size.
Is there a possibility that Logs for all CICD deployements could be retained and for greater duration without impacting the size / storage
Hammad Bhutta is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
The second option is the most sensible, where you archive logs frequently. Services like s3 allow you to keep a lot of data for relatively low prices. You could just introduce a post step at the end of the build to immediately push the log of the build to s3 (or other storage option) once the build finishes.
However note that once a year or so passes, searching through these logs might become near impossible due to the volume of logs. If you only want key information such as “what version was deployed” and “when was it deployed”, it might not make sense to have the full build log be written to s3 in the first place.
1
Be thoughtful of what logs you save. Work with your developers and dev ops team to understand what logs and build artifacts are actually useful. You could send a larger quantity of debug log file to short term storage and a more limited list to long term storage. This could be done in the Jenkinsfile during each build, or after the fact by sorting through Jenkins build artifact files.