release: 0.2.0

* experiment with adding a websocket interface

the websocket interface is fine but I can't work out how to pipe the logs properly

* try and hack together a way to share stdout through a log file

* better implementation of file stream logger

* refactor logging

* inference logging

* remove test file

* restructure logs

* fix rest and ws handlers  with optional file logging

* thinking about logging across different service calls

* use worker threads to isolate python environments

Nice idea but it does blow up

* use a child process to run python, trap stdout, and read output through a file

* clean up

* remove node-calls-python, update docs

* tidy up inputs and outputs

* sort out logging

* only send logger messages back to apollo

* tidyup

* remove temporary files after run

* slightly better error handling

* bump version

* document bun insall

* setup changesets

* update release notes

* fix jsx in container

.changeset/README.md

@@ -0,0 +1,8 @@
+# Changesets
+
+Hello and welcome! This folder has been automatically generated by `@changesets/cli`, a build tool that works
+with multi-package repos, or single-package repos to help you version and publish your code. You can
+find the full documentation for it [in our repository](https://github.com/changesets/changesets)
+
+We have a quick list of common questions to get you started engaging with this project in
+[our documentation](https://github.com/changesets/changesets/blob/main/docs/common-questions.md)

.changeset/config.json

@@ -0,0 +1,11 @@
+{
+  "$schema": "https://unpkg.com/@changesets/[email protected]/schema.json",
+  "changelog": "@changesets/cli/changelog",
+  "commit": false,
+  "fixed": [],
+  "linked": [],
+  "access": "restricted",
+  "baseBranch": "main",
+  "updateInternalDependencies": "patch",
+  "ignore": []
+}

CHANGELOG.md

@@ -0,0 +1,8 @@
+# apollo
+
+## 0.2.0
+
+### Minor Changes
+
+- Use child processes to run python scripts
+- add websocket connection with streaming logs

CONTRIBUTING.md

@@ -51,6 +51,23 @@ The Javascript bridge will always call into `entry.py` and dynamically invoke
 your service's main function, so technically speaking all imports are relative
 to `entry.py`.
 
+## Logging
+
+A utility library is provided for you to create a logger:
+
+```python
+from util import createLogger
+
+logger = createLogger("myservice.filename")
+```
+
+You can use whatever name you like for the logger (including `__name__`).
+
+All lines from this logger are diverted to the CLI (via websocket).
+
+For "private" and debug logging, use `print()` statements. The end user will not
+see these.
+
 ## Documentation
 
 All modules should come with basic documentation.

Dockerfile

@@ -4,6 +4,7 @@ WORKDIR /app
 
 COPY ./pyproject.toml ./poetry.lock poetry.toml ./
 COPY ./package.json bun.lockb ./
+COPY ./tsconfig.json ./
 
 COPY ./platform/ ./platform
 COPY ./services/ ./services
@@ -17,10 +18,6 @@ RUN poetry install --only main --no-root
 RUN curl -fsSL https://bun.sh/install | bash
 ENV PATH="${PATH}:/root/.bun/bin/"
 
-# node-gyp will fail to run unless node.js is installed
-RUN curl -sL https://deb.nodesource.com/setup_20.x  | /bin/bash -
-RUN apt-get install nodejs
-
 RUN bun install
 
 EXPOSE 3000

README.md

@@ -20,40 +20,46 @@ installed:
 - poetry
 - bunjs
 
-You also need `python3-dev`, `g++` and `make` to be installted:
+## Getting Started
 
-```
-sudo apt install python3-dev make g++
-```
+To run the server locally, you need to install the python dependencies
 
-## Getting Started
+```bash
+poetry install
+```
 
-To run the server locally, run:
+Then start the server (note that bun install is not needed, see below)
 
-- `poetry install`
-- `bun install`
+```bash
+bun start
+```
 
-To start the server for local developement, run:
+To start a hot-reloading development server which watches your typescript, run:
 
 ```bash
 bun dev
 ```
 
-Any changes to the typescript files will trigger the server to be re-loaded. All
-python scripts will be re-laoded on each call. So you should rarely, if ever,
-need to restart the server.
+To see an index of the available language services, head to `localhost:3000`.
 
-To start in production mode, run:
+# Bun installation
 
-```bash
-bun start
-```
+Bun does not require an installation, like npm does. You can run `bun start`
+right after cloning the repo.
 
-In production mode, nothing will hot reload and python modules are cached.
+Bun will then install dependencies against the global cache on your machine.
+This still uses package.lock.
 
-To see an index of the available language services, head to `localhost:3000`.
+To update a module version, run `bun add <module>@version>`, which will update
+your lockfile.
+
+One drawback of this is that there is no intelliense, because IDEs rely on
+node_modules to load d.ts files. You are welcome to run `bun install` to run
+from a node_modules. None of this affects python.
+
+See [bun auto-install]() for more details.
 
-## Finetuning and dependency groups
+## Finetuning and poetry dependency groups
 
 `poetry install` will only install the main dependencies - the stuff used in the
 docker image.
@@ -124,9 +130,36 @@ There is very little standard for formality in the JSON structures to date. The
 server may soon establish some conventions for better interopability with the
 CLI.
 
-The JS engine calls out to a long-running python process using
-`node-calls-python`. Python modules are pretty free-form but must adhere to a
-minimal structure. See the Contribution Guide for details.
+Python scripts are invoked through a child process. Each call to a service runs
+in its own context.
+
+Python modules are pretty free-form but must adhere to a minimal structure. See
+the Contribution Guide for details.
+
+## Websockets
+
+Every service can receive connections in one of two ways:
+
+- HTTP POST method
+- Websocket connection
+
+The same URL is used for both connections, clients must request upgrade to a
+websocket.
+
+Websocket connections will receive a live log stream.
+
+Websockets use the following events:
+
+`start`: sent by the client with a JSON payload in the `data` key.
+
+`complete`: sent by the server when the python script has completed. The result
+is a JSON payload in the `data` key.
+
+`log`: send by the server whenever a `print()` line is logged by the python
+process.
+
+Note that `print()` statements do not get send out to the web socket, as these
+are intended for local debugging. Only logs from a logger object are diverted.
 
 ## Python Setup
 
@@ -135,31 +168,16 @@ This repo uses `poetry` to manage dependencies.
 We use an "in-project" venv , which means a `.venv` folder will be created when
 you run `poetry install`.
 
-We call out to a live python environment from node, using a library called
-`node-calls-python`. We pass in the path to the local `.venv` folder so that the
-node-python bindings can see your poetry environment.
-
-The `node-calls-python` setup currently relies on a hard-coded python version.
-If the python version is changed, this value will need updating.
-
 All python is invoked through `entry.py`, which loads the environment properly
 so that relative imports work.
 
-You can invoke entry.py directly (ie, without HTTP) through bun from the root:
+You can invoke entry.py directly (ie, without HTTP or any intermedia js) through
+bun from the root:
 
 ```
 bun py echo tmp/payload.json
 ```
 
-## Installation Troubleshooting
-
-- Ensure all dependencies are installed
-- Ensure you've run `bun install` (unusual for Bun)
-- Maybe install node-gyp explicitly?
-- Ensure that `node-calls-python` has been built. In
-  `node_modules/node-calls-python` there should be a `Relesae` folder with stuff
-  inside.
-
 ## Docker
 
 To build the docker image:

package.json

@@ -1,7 +1,7 @@
 {
   "name": "apollo",
   "module": "platform/index.ts",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "type": "module",
   "scripts": {
     "start": "NODE_ENV=production bun platform/src/index.ts",
@@ -17,16 +17,15 @@
     "typescript": "^5.0.0"
   },
   "dependencies": {
+    "@changesets/cli": "^2.27.3",
     "@elysiajs/html": "^1.0.2",
-    "elysia": "^1.0.13",
-    "node-calls-python": "1.9.1",
-    "node-gyp": "^10.1.0"
+    "elysia": "1.0.21"
   },
-  "trustedDependencies": [
-    "node-calls-python"
-  ],
   "nodemonConfig": {
-    "watch": ["platform/src", "services"],
+    "watch": [
+      "platform/src",
+      "services"
+    ],
     "ext": "ts, py"
   }
-}
+}