I have a csv file which has the below mentioned data of Table names in a sql server instance in ubuntu.

Tables
TableName1
TableName2
TableName3
.
.
.

I want to read this csv file and I want to get the table data and store as ${table_name}.csv

How can I achieve this using pentaho. I tried a method but I want to know if there any built in methods or more efficient way of doing it. I’m new to pentaho so any advice is appreciable.

These are the details of the job I already tried.

• The first set variables step is to initialize a variable for the loop
• The csv reader job is where I used a bash script to read the csv file and the total num of lines and store as variable in a config.properties file

#!/bin/bash

# CSV file path
csv_file="/home/ubuntuv2204/taskDir/tables.csv"
property_file="/home/ubuntuv2204/taskDir/dwconfig.properties"
# Get the total number of rows in the CSV file (excluding the header)
total_rows=$(($(wc -l < "$csv_file")))

# Read the second line of the CSV file (excluding the header) and store it as table_name
table_name=$(sed '${NEW_LOOP}q;d' "$csv_file" | cut -d ',' -f 1)

# Check if the table_name is not empty
if [ -n "$table_name" ]; then
    # Print the table name
    echo "Table Name: $table_name"
else
    echo "Table name is empty or CSV file is not formatted correctly."
fi

# Store the total number of rows in a variable called loop_break
#loop_break=$total_rows

#echo "#DW" > "$property_file"
echo "table_name=$table_name" > "$property_file"
echo "loop_break=$total_rows" >> "$property_file"

• The Next step is the loop transformation to increase the loop value everytime
• The set dw tranformation reads the config.properties file and set variable for table_name and total no of lines.
• rw_ktr has table input step and read the table and writes as txt file output.
• Simple evaluation step checks if the loop value is equal to the total num of lines in the csv then the job ends that’s how I have written it.

This is working as per my requirement however I don’t think it’s that much good and I need an efficient solution.