I am trying to use the language model API in my vscode extension to refactor code. However, the LLM sometimes provides incomplete responses. I could really use some help in figuring out why this is happening. The relevant code to the extension that I am using for the LLM is below. This is a snippet from a command I have registered to refactor code.
const MODEL_SELECTOR: vscode.LanguageModelChatSelector = { vendor: 'copilot', family: 'gpt-3.5-turbo'};
const [model] = await vscode.lm.selectChatModels(MODEL_SELECTOR);
if (!model) {
console.error("No model found");
return;
}
let chatResponse: vscode.LanguageModelChatResponse | undefined;
const textRange: vscode.Range = diagnostic.range;
const refactorFragment: string = document.getText(textRange).slice(FORMAT_START_DEL.length, -FORMAT_END_DEL.length);
console.log(refactorFragment);
// LLM response test
const messages: vscode.LanguageModelChatMessage[] = [
vscode.LanguageModelChatMessage
.User(`Your job is to refactor code that users provide with a template.`),
vscode.LanguageModelChatMessage
.User(`Refactor the following code using this format:
${snippets[finalState.templateType.label]["body"].toString()}.
Replace the placeholders.`, 'refactor-user'),
vscode.LanguageModelChatMessage
.User(refactorFragment, 'refactor-user'),
];
// query the model
try {
chatResponse = await model.sendRequest(
messages,
{},
new vscode.CancellationTokenSource().token
);
} catch (err) {
if (err instanceof vscode.LanguageModelError) {
console.error(err.message, err.code, err.cause);
return;
}
}
// Clear the editor content before inserting new content
await textEditor.edit(edit => {
const start = diagnostic.range.start;
const end = diagnostic.range.end;
edit.delete(new vscode.Range(start, end));
});
try {
if (!chatResponse) {
console.error("No response from model");
return;
}
let lastLine = textEditor.document.lineAt(diagnostic.range.start.line);
console.log("last line: ", lastLine.lineNumber);
for await (const fragment of chatResponse.text) {
console.log(fragment);
await textEditor.edit(edit => {
lastLine = textEditor.document.lineAt(lastLine.lineNumber);
const position = new vscode.Position(lastLine.lineNumber, lastLine.text.length);
edit.insert(position, fragment);
const newLines = fragment.split('n').length;
lastLine = textEditor.document.lineAt(lastLine.lineNumber + newLines - 1);
});
}
} catch (err) {
await textEditor.edit(edit => {
const lastLine = textEditor.document.lineAt(textEditor.document.lineCount - 1);
const position = new vscode.Position(lastLine.lineNumber, lastLine.text.length);
edit.insert(position, (<Error>err).message);
});
}
return;
I have logged the fragments that come through asynchronously and confirmed that the response is incomplete. Here is an example response below:
Sure! Here's the refactored code:
```javascript
console.debug('[', Date.now(), ']', {
data:
Advay Balakrishnan is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.