could you please help me 🙁
Objective: Extract specific fields from log entries and store them correctly in a PostgreSQL database.
Log Line Example:
[2024-08-19 09:13:29,213] INFO – LogMediator {api:admin–UserAPI:v1.0} logType = ERROR_LOG, ERROR_CODE = 900901, ERROR_MESSAGE = Invalid Credentials, fileName = fault
Current Configuration:
appender.jdbc.type = JDBC
appender.jdbc.name = jdbc
appender.jdbc.connectionSource.type = DriverManager
appender.jdbc.connectionSource.driverClassName = org.postgresql.Driver
appender.jdbc.connectionSource.connectionString = jdbc:postgresql://localhost:5432/wso2_apim_errors
appender.jdbc.connectionSource.userName = ********
appender.jdbc.connectionSource.password = ********
appender.jdbc.tableName = new_logs
appender.jdbc.ignoreExceptions = false
appender.jdbc.columnConfigs[0].type = Column
appender.jdbc.columnConfigs[0].name = log_time
appender.jdbc.columnConfigs[0].pattern = %d
appender.jdbc.columnConfigs[0].isUnicode = false
appender.jdbc.columnConfigs[1].type = Column
appender.jdbc.columnConfigs[1].name = log_type
appender.jdbc.columnConfigs[1].pattern = logType = %X{logType}
appender.jdbc.columnConfigs[1].isUnicode = false
appender.jdbc.columnConfigs[2].type = Column
appender.jdbc.columnConfigs[2].name = error_code
appender.jdbc.columnConfigs[2].pattern = ERROR_CODE = %X{errorCode}
appender.jdbc.columnConfigs[2].isUnicode = false
appender.jdbc.columnConfigs[3].type = Column
appender.jdbc.columnConfigs[3].name = error_message
appender.jdbc.columnConfigs[3].pattern = ERROR_MESSAGE = %X{errorMessage}
appender.jdbc.columnConfigs[3].isUnicode = false
appender.jdbc.columnConfigs[4].type = Column
appender.jdbc.columnConfigs[4].name = file_name
appender.jdbc.columnConfigs[4].pattern = fileName = %X{fileName}
appender.jdbc.columnConfigs[4].isUnicode = false
logger.LogMediator.appenderRef.jdbc.ref = jdbc
Issue:
The database columns (log_time, log_type, error_code, error_message, file_name) are not populating correctly with the values extracted from log messages. The columns are either empty or contain incorrect data.
What I Tried
Filters and Pattern Layouts: Various filters and pattern configurations to capture specific fields (logType, errorCode, errorMessage, fileName) from log messages.
JDBC Appender Configuration: Configured the JDBC appender to map log message fields to database columns.
Log4j2 Version: Using Log4j2 version 2.12 with WSO2 APIM 3.2.0.
What I Expected
Correct Extraction: The JDBC appender should correctly extract values from the log messages and populate the corresponding database columns (log_time, log_type, error_code, error_message, file_name).
user26871131 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.