fix: key for versionless language models #44
Conversation
index.mjs
Outdated
| @@ -101,9 +103,21 @@ const inputs = getModelInputs(model) | |||
| console.log('Adding model data and inputs to index.js...') | |||
| const indexFile = path.join(targetDir, 'index.js') | |||
| const indexFileContents = fs.readFileSync(indexFile, 'utf8') | |||
|
|
|||
| let run = `const output = await replicate.run(model, { input });\nconsole.log(output)`; | |||
There was a problem hiding this comment.
This makes me sad. I mean, I know it's our current reality but I feel like users should be able to run official language models with the same code they use to run other models.
@replicate/product how can we make this better?
There was a problem hiding this comment.
I mean I can simplify this and just fix the model name. But I felt that also using the streaming output made for flashier example.
There was a problem hiding this comment.
Yeah if you're cool with it can we just fix the official language model name thing here in this PR and have a separate issue/discussion about the different ways of running models?
aron
force-pushed
the
fix-language-models
branch
2 times, most recently
from
78c375c to
53baa32
Compare
Currently when trying to run the script on versionless language models it's hitting a versioned api endpoint which is incredibly slow.
aron
force-pushed
the
fix-language-models
branch
from
53baa32 to
e7d1c6d
Compare
|
@aron the integration test for Node 18 is hanging: https://github.com/replicate/create-replicate/actions/runs/8072156594/job/22053328642?pr=44 -- looks like the test is successful but the process is not exiting. Same thing on #45 |
This adds support for generating projects from official language models. Currently, the code generated will use a versioned model name which results in cold boots for the model.
This PR updates the code to detect the current set of official models and use the non-versioned model name. Unfortunately this list has to be hardcoded for the time being as we don't expose which models have this different interface via our API.
The example code also uses the streaming API example, again this is special cased to the language models we know support streaming because this property is also not exposed via the api.