I am trying to divide large csv file into smaller csv files, but in each smaller I don’t want it to stop at specific block input. If the code doesn’t find specific keyword, it should move to the next row until it finds that keyword and it should then start the process again.
Suppose I am describing a block of 5 rows, it would stop at “SET BUS 249831 GENERATION TO 15.5 MW” comment, but I would want the script to move to the next rows and try finding keyword “END” and then finish this task and move for next block from row 6-11.
Image showing the data I am dealing with : (https://i.sstatic.net/0b71721C.png)
I did try to run the script and it stops at specific block input (row 5), and hence I have to manually edit each smaller file after the script is ran.
Currently I am using following code:
import glob
import pandas as pd
import math
read_file = glob.glob("*.con")
with open("combined.con", "wb") as outfile:
for f in read_file:
with open (f, "rb") as infile:
outfile.write(infile.read())
df0 = pd.read_csv (r'C:pathcombined.con')
df0.to_csv (r'C:pathcombined.csv')
count = len(df0)
row_range = 5
block = count // row_range
for i in range(block):
start = i * row_range
stop = (i+1) * row_range
df1 = df0.iloc[start:stop]
df1.to_csv(f"Contingency_{i}.con", index = False)
if stop < count:
df1 = df0.iloc[stop:]
df1.to_csv(f"Contingency_{i}.con", index = False)