From the python example, how to have pyright detect imports from same workspace folder

[William14689]

I am currently using the provided Python Language client and pyright language server example and it is working.

However, I am not sure how to have the current file importing from another file within the same workspace folder. Somehow pyright lsp is unable to pick the imported file in the workspace.

I have tried with the below but it is unable to work:

const fileSystemProvider = new RegisteredFileSystemProvider(false);
fileSystemProvider.registerFile(new RegisteredMemoryFile(vscode.Uri.file('/workspace/counter.py'), 'class WordCounter:\n    def count_words(self, sentence):\n       words = sentence.split(" ")\n       count = sum(1 for word in words if word)\n       return count'));
fileSystemProvider.registerFile(new RegisteredMemoryFile(vscode.Uri.file('/workspace/hello.py'), 'from counter import WordCounter'));
registerFileSystemOverlay(1, fileSystemProvider);

[msujew]

@kaisalmen When using fileSystemProvider.registerFile, does it automatically send the file to the language server? Might be a worthwhile addition to automatically call textDocument/onDidOpen on these or something like that.

@William14689 I believe the issue is that these files you're creating are completely virtual on the user side and aren't registered on the language server. The language server only sees what's on the disk file system of the backend and whatever has been explicitly sent to it using the textDocument/onDidOpen and textDocument/onDidChange LSP requests from the language client. You probably have to create a monaco text model for them explicitly. See here.

[kaisalmen]

Yes, we even make use of vscode in different contexts in some examples:
https://github.com/TypeFox/monaco-components/blob/main/packages/examples/src/wrapperTs.ts#L87-L94
https://github.com/TypeFox/monaco-languageclient/blob/main/packages/examples/src/python/client/main.ts#L155-L161

[CGNonofr]

actually, a call to createModelReference may be enough (and you can decide when to release the reference)

[cdietrich]

create model reference tries to write the file

"Error: Unable to resolve filesystem provider with relative file path 'memory://others-demo.hello'"

[cdietrich]

if i remove the memory prefix still does not work. need to debug backend

[cdietrich]

languageId also needs to be set

[William14689]

actually, a call to createModelReference may be enough (and you can decide when to release the reference)

I have given this a try and the client does send a jsonrpc. Without putting it into another container via createConfiguredEditor. seems like pyright is still not picking it up.
Let me try to trigger textDocument/didSave.

PYRIGHT Server received:
{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"/workspace/counter.py","version":1,"text":"class WordCounter:\n    def count_words(self, sentence):\n       words = sentence.split(\" \")\n       count = sum(1 for word in words if word)\n       return count","languageId":"python"}}}

[CGNonofr]

FYI, when using createModelReference, you can call object.save() on it to trigger the didSave notification

[William14689]

@msujew @CGNonofr @kaisalmen
Since I am not sure what can work, I have combined all of your inputs from above comments and updated the python client to become the code below. I am still working on it. Currently throwing reportMissingImports error

python.html
added

    <div id="containerCounter" style="width:800px;height:600px;border:1px solid grey"></div>

python/client/main.ts
all codes except the imports statement

const languageId = 'python';
let languageClient: MonacoLanguageClient;

const counterCode = 'class WordCounter:\n    def count_words(self, sentence):\n       words = sentence.split(" ")\n       count = sum(1 for word in words if word)\n       return count';

const createWebSocket = (url: string, modelRefCounter: IReference<ITextFileEditorModel>): WebSocket => {
    const webSocket = new WebSocket(url);
    webSocket.onopen = async () => {
        const socket = toSocket(webSocket);
        const reader = new WebSocketMessageReader(socket);
        const writer = new WebSocketMessageWriter(socket);
        languageClient = createLanguageClient({
            reader,
            writer
        });
        await languageClient.start();
        await languageClient?.sendNotification("textDocument/didOpen", {
            textDocument: {
                uri: 'file:///workspace/counter.py',
                version: 1,
                text: counterCode,
                languageId: 'python',
            },
        });
        await languageClient?.sendNotification("textDocument/didOpen", {
            textDocument: {
                uri: '/workspace/counter.py',
                version: 1,
                text: counterCode,
                languageId: 'python',
            },
        });
        await languageClient?.sendNotification("textDocument/didSave", {
            textDocument: {
                uri: 'file:///workspace/counter.py',
                version: 1,
                text: counterCode,
                languageId: 'python',
            },
        });
        await languageClient?.sendNotification("textDocument/didSave", {
            textDocument: {
                uri: '/workspace/counter.py',
                version: 1,
                text: counterCode,
                languageId: 'python',
            },
        });

        modelRefCounter.object.save();
        reader.onClose(() => languageClient.stop());
    };
    return webSocket;
};

const createLanguageClient = (transports: MessageTransports): MonacoLanguageClient => {
    return new MonacoLanguageClient({
        name: 'Pyright Language Client',
        clientOptions: {
            // use a language id as a document selector
            documentSelector: [languageId],
            // disable the default error handler
            errorHandler: {
                error: () => ({ action: ErrorAction.Continue }),
                closed: () => ({ action: CloseAction.DoNotRestart })
            },
            // pyright requires a workspace folder to be present, otherwise it will not work
            workspaceFolder: {
                index: 0,
                name: 'workspace',
                uri: monaco.Uri.parse('/workspace')
            },
            synchronize: {
                fileEvents: [vscode.workspace.createFileSystemWatcher('**')]
            }
        },
        // create a language client connection from the JSON RPC connection on demand
        connectionProvider: {
            get: () => {
                return Promise.resolve(transports);
            }
        }
    });
};

export const startPythonClient = async () => {
    // init vscode-api
    await initServices({
        userServices: {
            ...getThemeServiceOverride(),
            ...getTextmateServiceOverride(),
            ...getConfigurationServiceOverride(),
            ...getKeybindingsServiceOverride()
        },
        debugLogging: true,
        workspaceConfig: {
            workspaceProvider: {
                trusted: true,
                workspace: {
                    workspaceUri: Uri.file('/workspace')
                },
                async open() {
                    return false;
                }
            },
            developmentOptions: {
                logLevel: LogLevel.Debug
            }
        }
    });

    console.log('Before ready themes');
    await whenReady();
    console.log('After ready themes');

    // extension configuration derived from:
    // https://github.com/microsoft/pyright/blob/main/packages/vscode-pyright/package.json
    // only a minimum is required to get pyright working
    const extension = {
        name: 'python-client',
        publisher: 'monaco-languageclient-project',
        version: '1.0.0',
        engines: {
            vscode: '^1.78.0'
        },
        contributes: {
            languages: [{
                id: languageId,
                aliases: [
                    'Python'
                ],
                extensions: [
                    '.py',
                    '.pyi'
                ]
            }],
            commands: [{
                command: 'pyright.restartserver',
                title: 'Pyright: Restart Server',
                category: 'Pyright'
            },
            {
                command: 'pyright.organizeimports',
                title: 'Pyright: Organize Imports',
                category: 'Pyright'
            }],
            keybindings: [{
                key: 'ctrl+k',
                command: 'pyright.restartserver',
                when: 'editorTextFocus'
            }]
        }
    };
    registerExtension(extension, ExtensionHostKind.LocalProcess);

    updateUserConfiguration(`{
        "editor.fontSize": 14,
        "workbench.colorTheme": "Default Dark Modern"
    }`);

    const fileSystemProvider = new RegisteredFileSystemProvider(false);
    fileSystemProvider.registerFile(new RegisteredMemoryFile(vscode.Uri.file('/workspace/counter.py'), counterCode));
    fileSystemProvider.registerFile(new RegisteredMemoryFile(vscode.Uri.file('/workspace/hello.py'), 'from counter import WordCounter\n'));
    registerFileSystemOverlay(1, fileSystemProvider);

    const registerCommand = async (cmdName: string, handler: (...args: unknown[]) => void) => {
        // commands sould not be there, but it demonstrates how to retrieve list of all external commands
        const commands = await vscode.commands.getCommands(true);
        if (!commands.includes(cmdName)) {
            vscode.commands.registerCommand(cmdName, handler);
        }
    };
    // always exectute the command with current language client
    await registerCommand('pyright.restartserver', (...args: unknown[]) => {
        languageClient.sendRequest('workspace/executeCommand', { command: 'pyright.restartserver', arguments: args });
    });
    await registerCommand('pyright.organizeimports', (...args: unknown[]) => {
        languageClient.sendRequest('workspace/executeCommand', { command: 'pyright.organizeimports', arguments: args });
    });

    // use the file create before
    const modelRef = await createModelReference(monaco.Uri.file('/workspace/hello.py'));
    modelRef.object.setLanguageId(languageId);

    const modelRefCounter = await createModelReference(monaco.Uri.file('/workspace/counter.py'));
    modelRefCounter.object.setLanguageId(languageId);

    // create monaco editor
    createConfiguredEditor(document.getElementById('container')!, {
        model: modelRef.object.textEditorModel,
        automaticLayout: true
    });

    createConfiguredEditor(document.getElementById('containerCounter')!, {
        model: modelRefCounter.object.textEditorModel,
        automaticLayout: true
    });

    // create the web socket and configure to start the language client on open, can add extra parameters to the url if needed.
    createWebSocket(createUrl('localhost', 30001, '/pyright', {
        // Used to parse an auth token or additional parameters such as import IDs to the language server
        authorization: 'UserAuth'
        // By commenting above line out and commenting below line in, connection to language server will be denied.
        // authorization: 'FailedUserAuth'
    }, false), modelRefCounter);
};

[lukeautry]

I'm also not able to get this working. I've created a minimal repro branch here: https://github.com/lukeautry/monaco-languageclient/pull/1/files

onDidOpen/onDidSave is being called for the secondary file:

Pyright can't find the file or import from it.

@William14689 - were you ever able to get this working?

@CGNonofr @kaisalmen - would you mind taking a look?

Thanks for all your work so far!

[lukeautry]

@CGNonofr No, and that's kind of the point. Just as the main "hello.py" file doesn't exist on the server, my neighboring "typeinfo.py" file doesn't exist on the server - this is all local to the client, and should stay that way.

My assumption is that some combination of calls to Pyright may allow this secondary file to be added in memory to the workspace.

[CGNonofr]

Most language servers only fully works if the files are present on the filesystem, that's how the protocol is defined. Why is it an issue to put those file on the disk?

[lukeautry]

I suppose it doesn't have to be, it's just inconvenient. There's something convenient about being able to dynamically register files "on the fly" and only having to ensure that your LSP process is running. If the files have to be written to disk, then that means we need to write some layer of code in between our client and LSP to keep the file system in sync with the client's in memory representation of the workspace.

[CGNonofr]

I don't think you'll be able to have everything working without doing it.

The implementation is not really complex though. What I've already done is extending the LSP to add a document/save request when the file is saved and before opening it. On the server side, it's a few lines of code to write it on the disk

[William14689]

@lukeautry I still couldn't get this to work. Currently just allow 1 file to use its own functions (cannot import from other virtual files created).

After debugging for quite sometime, it seems that the files created locally in the VScode in-memory file system stay in there but they are not sync into the language file server's workspace. the language server only know the current file it seems.