I am building an interview application where I am interrogated by an AI and I have to answer questions. The interview is recorded using MediaRecorder. The problem is that the audio that is generated by Whisper AI is not recorded. I tried combining the streams but then the recording stops once the Whisper audio is added to the stream.
The flow is like this:
- The recording starts as soon as the interview starts
- Whisper audio is played with the first question
- I answer the question
- 2nd question etc.
I want all the audio recorded (from my mic and the Whisper audio).
Currently my working code looks like this (without combining the streams):
let chunkIndex = 0
const handleInterviewStart = async() => {
setInterviewStarted(true)
try {
const stream = await navigator.mediaDevices.getUserMedia({
video: { deviceId: selectedVideoDeviceId ? { exact: selectedVideoDeviceId } : undefined },
audio: { deviceId: selectedAudioDeviceId ? { exact: selectedAudioDeviceId } : undefined }
});
const mediaRecorder = new MediaRecorder(stream);
mediaRecorder.ondataavailable = (event) => {
uploadChunk(event.data, chunkIndex, false)
chunkIndex += 1
};
mediaRecorder.start(5000);
setInterviewRecorder(mediaRecorder);
} catch (error) {
console.error('Error starting interview recording:', error);
}
console.log('INIT MESSAGE: ', initMessage)
await transcribeAndComplete(true)
}
const playSpeech = async (response) => {
try {
// Create a new audio element
const audio = new Audio()
// Set the audio source to the API route URL with text as a query parameter
audio.src = URL.createObjectURL(await response.blob())
audio.play()
} catch (error) {
console.error("Error playing speech:", error)
}
}
Anyone knows how to achieve this?
Terraflow is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.