Hi I have a df (like in the next example), I need to transform and clean a text that I obtained through webscraping methods. In the conversations, the user or the agent may not always have answered, but at least one of them is always present. But for the last conversation the date and time are not being obtained (image adj.). It also considers that the agent’s first name is always the one present in the conversation, but how could I modify the function so that it searched for any of the agent’s names?
# Dataframe Example
data = {
'conversation': [
"""Dd Rn¡Hiiii!n09:11 AM, 29th AprnRingonHi!!!nn¡Welcome to FFFF, blalbllalbabla!????, my name is Steve Ringo, how can I help you?n09:11 AM, 29th AprnDd RnHi, mi name is ASDDS Rn09:12 AM, 29th AprnDd RnI write you becouse I need some hel with sasfsffdgsggdsn09:12 AM, 29th AprnRingonPleas first of all bring me this information FDFDSERREn09:14 AM, 29th AprnDd RnXSA ADSSR-434n09:14 AM, 29th AprnDd Rn09:16 AM, 29th AprnRingonOk that will be take a few minutesn09:16 AM, 29th AprnDd RnOk thnxn09:19 AM, 29th AprnRingonOk that's all?n09:21 AM, 29th AprnDd RnYeapn09:22 AM, 29th AprnTake to Team Inbox""",
"""_nHi I need ur help!!n07:55 PM, 06th Mayn_nPleas callme right nown07:55 PM, 06th MaynTake to Team Inbox"""
],
'user_name': ['Conversation with Dd R', 'Conversation with _'],
'agent_name': ['Ringo Steve', 'Shaun Murphy']
}
df = pd.DataFrame(data)
# Function to clean and format conversations
def clean_conversation(row):
conversation = row['conversation']
# Extract agent and user name
user_name = re.search(r'Conversation with (.+)', row['user_name']).group(1)
agent_name = row['agent_name'].split()[0]
# Delete unnecesary text
conversation = re.sub(r'Take to Team Inbox', '', conversation)
# Sprit lines
lines = conversation.strip().split('n')
# Clear and format lines
cleaned_lines = []
current_timestamp = None
current_sender = None
for line in lines:
timestamp_match = re.match(r'd{1,2}:d{2} [APM,]{3,6} d{1,2}[a-z]{2} w{3}', line)
if timestamp_match:
current_timestamp = line
elif line.strip() == user_name or line.strip() == agent_name:
current_sender = line.strip()
elif line.strip():
message = line.strip()
if current_sender is None:
current_sender = user_name if user_name in conversation else agent_name
if current_timestamp:
cleaned_lines.append(f'{current_timestamp} - {current_sender}: {message}')
current_timestamp = None
else:
cleaned_lines.append(f'{current_sender}: {message}')
# Join lines
cleaned_conversation = 'n'.join(cleaned_lines)
return cleaned_conversation
# Apply the function to the 'conversation' column
df['cleaned_conversation'] = df.apply(clean_conversation, axis=1)
# PRINTING
for idx, conversation in enumerate(df['cleaned_conversation']):
print(f'Conversación {idx + 1}:n{conversation}n{"="*40}n')