Using stored checkpoint in a java program

[already-taken-m17]

Can the stored checkpoints be used by a java program and then sampling can be done in java itself ? If yes, can someone please share some ideas about this ?

[jcjohnson]

That would be nontrivial.

Checkpoints are stored using Torch serialization so you'd either have to implement a decoder in Java or write checkpoints from Lua in some other format.

You'd also need to implement the forward pass in Java for whatever Torch modules your network contains, and these would need to be binary compatible with the Torch implementations.

Can I ask what use case you have in mind? There may be an easier way.

[already-taken-m17]

yeah, sure.
I've trained the model on a huge dataset, but now sampling and some more operations on samples are slower than my requirements.
I have a feeling that after writing sampling code (forward pass) in java, the results may be faster.
Do you think it is possible i.e. java code for sampling will be faster ?
kindly suggest alternatives too if you have some in mind.
Thanks!

[jcjohnson]

If you are mainly concerned about performance then rewriting in Java does not seem like a good solution; it would be much simpler to optimize the existing Lua sampling. There is certainly low-hanging fruit for optimization, such as this pull request: #138

Sampling speed is fundamentally limited by the model itself; generating each character requires some large matrix multiplies. You should make sure that your BLAS implementation is properly set up. As a last resort you can also try training smaller models; depending on the dataset and training you may find that smaller models perform just as well as larger models.

[already-taken-m17]

Yeah, I'll try to train some smaller models too and will see if the performance is same with a boost in speed.
Also, will try to look into dl4j implementations of lstm and compare the results and speed.