I want to record a Google Meet meeting with a Browser Extension, but any tab audio, such as YouTube, can be recorded, including my mic.
I know how to record the tab audio.
I don’t know how to combine the tab audio with the mic to record the GMeet fully. Include functionality that respects muting the microphone on GMeet.
My current code is on GitHub: https://github.com/prokopsimek/chrome-extension-recording
What’s already implemented:
- record tab audio
- opens a new tab after the audio is stopped with the audio file
my offscreen.tsx:
const media = await navigator.mediaDevices.getUserMedia({
audio: {
mandatory: {
chromeMediaSource: 'tab',
chromeMediaSourceId: streamId,
},
},
video: false,
} as any);
console.error('OFFSCREEN media', media);
// FIXME: this causes error in recording, stops recording the offscreen
// const micMedia = await navigator.mediaDevices.getUserMedia({
// audio: {
// mandatory: {
// chromeMediaSource: 'tab',
// chromeMediaSourceId: micStreamId,
// },
// },
// video: false,
// } as any);
// Continue to play the captured audio to the user.
const output = new AudioContext();
const source = output.createMediaStreamSource(media);
const destination = output.createMediaStreamDestination();
// const micSource = output.createMediaStreamSource(micMedia);
source.connect(output.destination);
source.connect(destination);
// micSource.connect(destination);
console.error('OFFSCREEN output', output);
// Start recording.
recorder = new MediaRecorder(destination.stream, { mimeType: 'video/webm' });
recorder.ondataavailable = (event: any) => data.push(event.data);
recorder.onstop = async () => {
const blob = new Blob(data, { type: 'video/webm' });
// delete local state of recording
chrome.runtime.sendMessage({
action: 'set-recording',
recording: false,
});
window.open(URL.createObjectURL(blob), '_blank');
my popup.tsx useEffect:
const handleRecordClick = () => {
if (isRecording) {
console.log('Attemping to stop recording');
chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
const currentTab = tabs[0];
if (currentTab.id) {
chrome.runtime.sendMessage({
action: 'stopRecording',
tabId: currentTab.id,
});
setIsRecording(false);
}
});
} else {
chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
const currentTab = tabs[0];
if (currentTab.id) {
chrome.runtime.sendMessage({
action: 'startRecording',
tabId: currentTab.id,
});
setIsRecording(true);
}
});
}
};
my background.ts initializing offscreen:
const startRecordingOffscreen = async (tabId: number) => {
const existingContexts = await chrome.runtime.getContexts({});
let recording = false;
const offscreenDocument = existingContexts.find((c) => c.contextType === 'OFFSCREEN_DOCUMENT');
// If an offscreen document is not already open, create one.
if (!offscreenDocument) {
console.error('OFFSCREEN no offscreen document');
// Create an offscreen document.
await chrome.offscreen.createDocument({
url: 'pages/offscreen/index.html',
reasons: [chrome.offscreen.Reason.USER_MEDIA, chrome.offscreen.Reason.DISPLAY_MEDIA],
justification: 'Recording from chrome.tabCapture API',
});
} else {
recording = offscreenDocument.documentUrl?.endsWith('#recording') ?? false;
}
if (recording) {
chrome.runtime.sendMessage({
type: 'stop-recording',
target: 'offscreen',
});
chrome.action.setIcon({ path: 'icons/not-recording.png' });
return;
}
// Get a MediaStream for the active tab.
console.error('BACKGROUND getMediaStreamId');
const streamId = await new Promise<string>((resolve) => {
// chrome.tabCapture.getMediaStreamId({ consumerTabId: tabId }, (streamId) => {
chrome.tabCapture.getMediaStreamId({ targetTabId: tabId }, (streamId) => {
resolve(streamId);
});
});
console.error('BACKGROUND streamId', streamId);
const micStreamId = await new Promise<string>((resolve) => {
chrome.tabCapture.getMediaStreamId({ consumerTabId: tabId }, (streamId) => {
resolve(streamId);
});
});
console.error('BACKGROUND micStreamId', micStreamId);
// Send the stream ID to the offscreen document to start recording.
chrome.runtime.sendMessage({
type: 'start-recording',
target: 'offscreen',
data: streamId,
micStreamId,
});
chrome.action.setIcon({ path: '/icons/recording.png' });
};
What’s missing:
- record a mic
- combine mic with tab audio
- respect when the mic is on/off in GMeet
What I am confused about:
- Should I really use the offscreen document?
- I am getting permissions denied in the offscreen (ref)
- What to use for the recording? Offscreen, content script, popup, or anything else?
- How to combine the audio streams from both sources into one file?
My goal:
- Record audio from Google Meets (tab + my mic)
- Record audio only
- Use the v3 manifest (I expect that the older versions will be deprecated sooner than v3)
- I should allow the mic in a tab where I want to start the recording if not allowed yet
- When I stop recording, a new tab will open with the blob file so that it can be saved
Reference links:
- https://github.com/muaz-khan/Chrome-Extensions/issues/75 (doesn’t work)
- https://github.com/muaz-khan/Chrome-Extensions/tree/master/screen-recording (manifest v2 but I’d like v3 – do I really need the v3? 🤷)
- https://groups.google.com/a/chromium.org/g/chromium-extensions/c/V09VMCLzvWM
- https://github.com/GoodDollar/GoodDAPP/issues/1727
- https://developer.mozilla.org/en-US/docs/Web/API/MediaStream
- https://developer.chrome.com/docs/extensions/reference/api/tabCapture#method-capture
- https://developer.chrome.com/docs/extensions/reference/api/tabCapture#method-getMediaStreamId
- Chrome extension include mic audio to recorded video
- How to record audio from a meeting tab in google chrome? (no response)
2
I will attempt to address the remaining requirements individually
-
Capture the audio from specific tab.
const media = await navigator.mediaDevices.getUserMedia({ audio: { mandatory: { chromeMediaSource: 'tab', chromeMediaSourceId: streamId, }, }, video: false, } as any);
Let’s assume we have created a separate function named getTabAudioStream that returns the above.
-
- record a mic
return await navigator.mediaDevices.getUserMedia({ audio: true, // Request microphone audio video: false, });
Let’s assume we have created a separate function named getMicrophoneAudioStream that returns the above.
-
- combine mic with tab audio
async function setupAudioCombination(tabStreamId) { const tabStream = await getTabAudioStream(tabStreamId); const micStream = await getMicrophoneAudioStream(); const audioContext = new AudioContext(); const tabSource = audioContext.createMediaStreamSource(tabStream); const micSource = audioContext.createMediaStreamSource(micStream); const destination = audioContext.createMediaStreamDestination(); const micGainNode = audioContext.createGain(); // Create a GainNode to control mic volume when mute state updates // Connect both sources to the destination tabSource.connect(destination); micSource.connect(micGainNode); // Connect micSource to the GainNode micGainNode.connect(destination); return { combinedStream: destination.stream, // record this micGainNode, audioContext, }; } -
- respect when the mic is on/off in GMeet
You can track the microphone’s “muted” state in Google Meet by selecting the mute button using the
[data-mute-button]attribute and setting up aMutationObserverto listen for changes in thedata-is-mutedattribute. This attribute toggles between “true” and “false” to indicate whether the microphone is muted or unmuted.Here’s a simple implementation:
const muteButton = document.querySelector('[data-mute-button]'); let isMuted = muteButton.getAttribute('data-is-muted') === 'true'; // Create a MutationObserver to watch for changes in the `data-is-muted` attribute const observer = new MutationObserver((mutationsList) => { for (const mutation of mutationsList) { if (mutation.type === 'attributes' && mutation.attributeName === 'data-is-muted') { isMuted = muteButton.getAttribute('data-is-muted') === 'true'; console.log('Microphone mute state changed:', isMuted ? 'Muted' : 'Unmuted'); handleMicMuteState(isMuted); } } }); // Start observing the mute button for attribute changes observer.observe(muteButton, { attributes: true, attributeFilter: ['data-is-muted'] });where
handleMicMuteStatemutes the mic audio stream alone through themicGainNodeand the recording of the tab audio continues:function handleMicMuteState(isMuted) { if (micGainNode) { micGainNode.gain.value = isMuted ? 0 : 1; // Set gain to 0 to mute, 1 to unmute } }
4