Skip to content

CNTK FAQ

Jump to bottom
Chris Basoglu edited this page Aug 25, 2016 · 55 revisions

At this page we present the answers to some of the most common questions we get about CNTK and related Subjects.

What is the CNTK?

CNTK, the Computational Network Toolkit, is a framework for deep learning. A Computational Network defines the function to be learned as a directed graph where each leaf node consists of an input value or parameter, and each non-leaf node represents a matrix or tensor operation upon its children. The beauty of CNTK is that once a computational network has been described, all the computation required to learn the network parameters is taken care of automatically. There is no need to derive gradients analytically or to code the interactions between variables for backpropagation.

Why did Microsoft develop CNTK?

We first created CNTK for ourselves. CNTK was developed for the fastest training on the biggest data sets. Many of Microsoft's critical services run on models trained with CNTK. The results were so positive, we wanted to share our toolkit with the world.

Training deep learning models can be time intensive, can CNTK help with this?

For mission critical AI research, we believe efficiency and performance are important criteria. CNTK was designed for peak performance for not only CPUs but also single-GPU, multi-GPU, and multi-machine-multi-GPU scenarios. Additionally, Microsoft’s 1-bit compression technique or Block momentum technique dramatically reduced communication costs -- enabling highly scalable parallel training on a large number of GPUs spanning multiple machines.

Is CNTK flexible enough for my own network?

In addition to a wide variety of built-in computation nodes, CNTK provides a plug-in architecture allowing users to define their own computation nodes. So if your workload requires special customization, CNTK makes that easy to do. Readers are also fully customizable allowing support for arbitrary input formats.

What are the key training algorithms supported by CNTK?

Today CNTK supports the following algorithms:

  • Feed Forward
  • CNN
  • RNN
  • LSTM
  • Sequence-to-Sequence.

Who are the people behind the CNTK?

CNTK is developed by Microsoft's Technology and Research division. Additionally, CNTK gets major contributions from nearly all of Microsoft production teams.

When did work begin on the CNTK?

The development of CNTK has been underway since late 2014.

What about Reinforcement Learning with CNTK?

September 2016, we plan to release CNTK as a library with the Reinforcement Learning pipelines enabled.

CNTK config language (NDL) is great, but I want more freedom in creating my network, creating new nodes, or streaming in my unique data format. Will CNTK support lower level programming, (via Python or other languages)?

September 2016 we plan to release the new CNTK APIs supporting both Python, and C++ bindings, so you will be able to configure your networks, create new nodes, as well as programmatically feed the data from your own Python, and C++ programs.

Is CNTK only optimized for Speech Recognition Training?

No. CNTK is used in production for the Speech Recognition as well as for Image and Text training.

How can I use CNTK?

Using CNTK is easy and straightforward. Here are some ways to get started.

When I compile CNTK using VS2013, I see a compiler error, what is wrong?

You must upgrade VS2013 to "update 5". Setting up CNTK on Windows

Do eval libs get produced for GPU?

Unlike training, eval is currently CPU only.

I enabled Image Reader with zip support and get "Plugin not found: 'ImageReader.dll'" error when running Image Reader unit tests or trying to use the reader. What might be wrong?

Check that you have correctly installed zlib and libzip, especially that you have not forgotten to rename zlib.dll to zlib1.dll.

Why does CNTK randomize the mini-batches after each epoch?

Doing so prevents the same samples from always appearing in a mini-batch together. This leads to improvements in the validation accuracy.

Can the built-in readers be used train a network model using multiple input files?

Yes. See the description at Understanding and Extending Readers and look for the section describing how to "compose several data deserializers"

I have just downloaded and installed CNTK binary package and want to run a job, but get weird errors, like missing CUDA 7.0 libraries (and I have downloaded CPU-only version!)

Please, check carefully what you have in your PATH. Especially, if it is a shared development machine. With high probability such errors are caused by a "forgotten" CNTK.exe file from a previous release or built from not very recent sources, that is reachable in the PATH.

On Widows I installed a new version of NVIDIA driver and now CNTK build fails with the errors like ..\Common\BestGpu.cpp(24): fatal error C1083: Cannot open include file: 'nvml.h': No such file or directory

You have selected Perform a clean installation option in NVIDIA Driver Installer. That results in the removal of GPU Deployment Kit (GDK). To repair the system, perform the following steps:

  • Launch CUDA Installer
  • Select Custom (Advanced) Installation
  • Unselect all installation options, except GPU Deployment Kit
  • This will automatically select Graphics Driver option - it is expected. Leave it selected
  • Proceed with CUDA installation
  • After successful CUDA installation launch the installation of the desired Graphics Driver version
  • Select Custom (Advanced) Installation
  • Ensure that Perform a clean installation is NOT selected and proceed with the installation

New Documentation Site

Iteration Plans

Clone this wiki locally