-
Notifications
You must be signed in to change notification settings - Fork 4.3k
CNTK Evaluation Overview
Once you have trained a model, you need the functionality to evaluate the model in your target environment. As a reminder, there are several ways to create models with CNTK, and there are two different formats to store the model in.
The original CNTK (prior to the CNTK 2.0 version) only supports a format we call now the model-V1 format. It was originally created by using CNTK.EXE with BrainScript. With CNTK 2.0 a Protobuf based format was introduced, which is now known as the model-V2 format. The following table presents an overview on creating the different model formats:
| model-creation | model-v1 | model-v2 |
|---|---|---|
| CNTK.EXE (BrainScript) | YES | NO |
| CNTK-library (Python, C++) | deprecated | YES |
For more details on creating the different model formats refer to the CNTK model format page.
Aside from training a model, Microsoft Cognitve Toolkit provides different ways of evaluating the model:
| model-evaluation | features | model-v1 | model-v2 |
|---|---|---|---|
| CNTK.EXE | BrainScript | YES | NO |
| GPU and CPU support | |||
| CNTK-EvalDLL | C++, C#/.NET | YES | NO |
| ASP and Azure support | |||
| Nuget Package for CPU-Only support | |||
| CNTK-library | C++, C#/.NET, Java, and Python | YES | YES |
| ASP and Azure support | |||
| GPU and CPU | |||
| Nuget Packages | |||
| Support multiple parallel requests | |||
| Optimized memory usage by parameter sharing |
The following pages provide detailed information about model evaluation in different scenarios:
- CNTK-library evaluation on Windows
- CNTK-library evaluation on Linux
- CNTK-library evaluation with Python