Questions regarding feature preprocessing, number of CTC outputs

[lifelongeek]

Hello

First of all, thanks for providing BLSTM + CTC as open source. I attribute yours in my future paper.

I successfully install dependencies (i.e. cudamat, gnumpy, npmat) and make execution files in ctc folder.

Since TIMIT is small enough DB to start, I was going to execute runTimit.sh but got error message below :

[kenkim@node3 ctc]$ ./runTimit.sh

• optimizer=nesterov
• momentum=.95
• epochs=100
• layers=2048,2048,2048,2048
• step=1e-4
• anneal=1.1
• outfile=models/nesterov_layers_2048,2048,2048,2048_step_1e-4_mom_.95_anneal.bin
• echo models/nesterov_layers_2048,2048,2048,2048_step_1e-4_mom_.95_anneal.bin
  models/nesterov_layers_2048,2048,2048,2048_step_1e-4_mom_.95_anneal.bin
• python runNNet.py --layers 2048,2048,2048,2048 --optimizer nesterov --step 1e-4 --epochs 100 --momentum .95 --outFile models/nesterov_layers_2048,2048,2048,2048_step_1e-4_mom_.95_anneal.bin --anneal 1.1 --outputDim 62 --inputDim 943 --rawDim 943 --numFiles 19 --dataDir /home/kenkim/kaldi-trunk/egs/timit/s5/exp/nomral_nn_train/
  gnumpy: failed to use gpu_lock. Using board #0 without knowing whether it is in use or not.
  Using nesterov..
  Traceback (most recent call last):
  File "runNNet.py", line 171, in
  run()
  File "runNNet.py", line 93, in run
  data_dict,alis,keys,sizes = loader.loadDataFileDict(i)
  File "/home/kenkim/stanford-ctc-master/ctc/dataLoader.py", line 48, in loadDataFileDict
  data_mat, alis, keys, sizes = self.loadDataFile(filenum)
  File "/home/kenkim/stanford-ctc-master/ctc/dataLoader.py", line 30, in loadDataFile
  data = np.fromfile(datafile,np.float32).reshape(-1,self.rawsize)
  IOError: [Errno 2] No such file or directory: '/home/kenkim/kaldi-trunk/egs/timit/s5/exp/nomral_nn_train/feats12.bin'

Here is my questions.

Q1) Which source code can make feats12.bin?

Q2) In your paper, you use 33 characters for CTC output symbol.
Can you clarify what exactly they are? (i.e. alphabet(26) + blank(1) + something(6))

Q3) How did you preprocess 'non-character' in transcription of switchboard?
For example, 747, &, 20/20, _1_1 etc.

[zxie]

Q2: Just added chars.txt file
Q3: Added clean_text_ctc.sh, also refer to mergechars.py

[lifelongeek]

Thanks for your comment and quick reply :)

[jinserk]

@zxie, Could you let me know the answer of Q1? I'm stuck at the same step of @gmkim90's question.
I'm trying to run your code with TIMIT data, but it still require feat8.bin at the beginning of running.

In the ctc/dataLoader.py, they requires 3 files: keysOO.txt, alisOO.txt, and featsOO.bin. It'll be great help if you describe the format of them. Thank you!

[zxie]

Hi @jinserk. Yeah, we forgot some Stanford processing scripts after the Kaldi pipeline that makes things easier to just load into numpy. I just e-mailed you a link that has an example of each of the data files.

We can also add the code (egs/swbd/s5b/stanford-utils, src/stanford-bin) excluding the main Kaldi codebase, pipeline is tricky to piece together though. @amaas, thoughts?

Edit: For future reference can also find some example files using TIMIT training data at: http://deeplearning.stanford.edu/lexfree/timit/ -- just edited README accordingly.

[jinserk]

Hi @zxie and @amaas, I really appreciate your reply and the link.

The example directories of Kaldi are well-organized, so you can add your utils (maybe binaries or bash scripts, I guess) generating input files from the Kaldi data with the necessary Kaldi path specified in your own stanford-ctc repository.

I've downloaded the files from the link you mentioned, and copy them into stanford-ctc/data. I've changed the command options --dataDir to ../data and --numFiles to 1 in ctc/runTimit.sh. Also I renamed the downloaded files *19.txt|bin to *1.txt|bin. When I execute ctc/runTimit.sh, I've got an error like this:

Traceback (most recent call last):
  File "runNNet.py", line 171, in <module>
    run()
  File "runNNet.py", line 94, in run
    SGD.run_seq(data_dict,alis,keys,sizes)
  File "/home/jinserk/work/stanford-ctc/ctc/sgd.py", line 61, in run_seq
    cost,grad,skip = self.model.costAndGrad(mb_data,mb_labels,key=k)
  File "/home/jinserk/work/stanford-ctc/ctc/nnet.py", line 174, in costAndGrad
    cost, self.deltas[-1], skip = ctc.ctc_loss(probs, labels, blank=0)
  File "ctc_fast.pyx", line 14, in ctc_fast.ctc_loss (ctc_fast.c:1932)
    def ctc_loss(double[::1,:] params not None, 
  File "stringsource", line 614, in View.MemoryView.memoryview_cwrapper (ctc_fast.c:11431)
  File "stringsource", line 321, in View.MemoryView.memoryview.__cinit__ (ctc_fast.c:8020)
ValueError: ndarray is not Fortran contiguous

Could you comment for this error?
Thank you!

[zxie]

Just checked flags w/ some more recently computed features and they seem to match the ones in timit file.

Looks like you're loading the model from ctc/ and not ctc_fast/nnets/; could you first try changing your PYTHONPATH?

As fallback, hack, but could just use np.asarray(a, order='F') to convert probs (and labels if necessary) and see if that works.

[jinserk]

@zxie, Thank you for your kind reply.

There exists two directories ctc and ctc_fast in the repository. I'm trying the example in the first ctc. In the ctc, there are two ctc_loss function implementations: ctc.py and ctc_fast.pyx. It seems I can select one of them with line 4 and 5 of nnet.py. In the latest git commit, the ctc_fast is chosen as default, and I got the error described above. When I change this to ctc, it seems work well with the downloaded input files.

I guess the ctc_fast directory is more latest implementation, but I also got an error message with ctc_fast/timit-utils/runTimit.sh that says there is no option --outFile and so on, according to the definition in ctc_fast/runNNet.py.

[zxie]

Yes, I'd go with the ctc_fast directory. Not sure why you'd have the error above if using latest runNNet.py that loads brnnet ... could you post the error with updated path?

Thanks for letting us know about other bug; run*.sh scripts are a bit dated.
runNNet.py will write the model and config files to a directory now; I'll note this in next commit.

[amaas]

@zxie I'd be fine with flipping our Kaldi add-ons public but there's not a
great way to do this robustly. We internally have a version of Kaldi which
is now quite out of sync with their master.

On Mon, Sep 21, 2015 at 7:14 PM, zxie [email protected] wrote:

Yes, I'd go with the ctc_fast directory. Not sure why you'd have the
error above if using latest runNNet.py that loads brnnet ... could you
post the error with updated path?

Thanks for letting us know about other bug; run*.sh scripts are a bit
dated.
runNNet.py will write the model and config files to a directory now; I'll
note this in next commit.

—
Reply to this email directly or view it on GitHub
#1 (comment).

[jinserk]

@zxie, sorry for late reply.
The error message is as follows. The working path is stanford-ctc/ctc_fast and the execution command is stanford-ctc/ctc_fast/timit-utils/runTimit.sh. I made symbolic links for stanford-ctc/ctc_fast/analysis-utils/writeLikelihoods.py and stanford-ctc/ctc_fast/nnets/brnnet.py into stanford-ctc/ctc_fast.

$ timit-utils/runTimit.sh 
+ momentum=.95
+ epochs=50
+ layerSize=2048
+ numLayers=4
+ maxBatch=777
+ step=1e-4
+ anneal=1.1
+ deviceId=0
+ temporalLayer=-1
+ outfile=models/timit_layers_4_2048_step_1e-4_mom_.95_anneal_1.1.bin
+ echo models/timit_layers_4_2048_step_1e-4_mom_.95_anneal_1.1.bin
models/timit_layers_4_2048_step_1e-4_mom_.95_anneal_1.1.bin
+ python runNNet.py --layerSize 2048 --numLayers 4 --step 1e-4 --epochs 50 --momentum .95 --outFile models/timit_layers_4_2048_step_1e-4_mom_.95_anneal_1.1.bin --anneal 1.1 --outputDim 62 --inputDim 943 --rawDim 943 --numFiles 1 --dataDir ../../data/ --deviceId 0 --temporalLayer -1
Usage: usage : runNNet.py [options]

runNNet.py: error: no such option: --outFile

[qingtian20124983]

@zxie, also I want to know which source code can make feat19.bin? now i have download key19.txt , ali19.txt, just feat19.bin is missing. thank you very much and look forward for your reply.