Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pb running NER_Keras.py code with Keras 1.0.3 #3

Open
gpcoursera opened this issue Jun 1, 2016 · 6 comments
Open

pb running NER_Keras.py code with Keras 1.0.3 #3

gpcoursera opened this issue Jun 1, 2016 · 6 comments

Comments

@gpcoursera
Copy link

Dear Nils,
First of all thanks a lot for the great tutorial. I'm learning a lot watching your videos and reading through your code.
I naively wanted to run our implementation of the SENNA model with Keras and cannot make it work even after trying many things and spending of lot of time on forums dedicated to Theano and Keras.
For the record, I think it has something to do with The FixEmbedding layer and/or my version of Keras==1.0.3. I'm running Python 3.5 but I fixed all print and unicode trivial issues in the code so it should not be a pb with compatibility with Python 3.

Here is the traceback:

Traceback (most recent call last):
File "NER_Keras.py", line 126, in
model.add(FixedEmbedding(output_dim=embeddings.shape[1], input_dim=embeddings.shape[0], input_length=n_in, weights=[embeddings]))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/models.py", line 114, in add
layer.create_input_layer(batch_input_shape, input_dtype)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py", line 341, in create_input_layer
self(x)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py", line 458, in call
self.build(input_shapes[0])
TypeError: build() takes 1 positional argument but 2 were given

Thanks for your help.
G.

@gpcoursera gpcoursera changed the title pb running NER_Keras.py code with Keras pb running NER_Keras.py code with Keras 1.0.3 Jun 1, 2016
@beinborn
Copy link

beinborn commented Jun 2, 2016

Dear G.,
I had the same problem. The code runs well with Keras 0.3, but not with newer versions. I used this instead of the FixedEmbedding:

embedding = Embedding(output_dim=embeddings.shape[1], input_dim=embeddings.shape[0], input_length=numInputUnits, weights=[embeddings])
embedding.trainable=[]
model.add(embedding)

Maybe this works for you too.
Best,
Lisa

@flekova
Copy link

flekova commented Jun 2, 2016

Hi, if I remember well, the problem was caused by the SharedX import, which causes trouble in newer versions. You can comment that import out and instead define these in the FixedEmbeddings.py:

def shared_zeros(shape, dtype=theano.config.floatX, name='', n=1): shape = shape if n == 1 else (n,) + shape return theano.shared(np.zeros(shape, dtype=dtype), name=name) def sharedX(X, dtype=theano.config.floatX, name=None): return theano.shared(np.asarray(X, dtype=dtype), name=name) def floatX(val): return np.asarray(val, dtype=theano.config.floatX)

Then it shall work also in newer versions.

Cheers,
Lucie

@gpcoursera
Copy link
Author

@beinborn : thanks Lisa. I followed your post and it works fine.
I was searching for this simple add-on: embedding.trainable=[]
to ensure I'm not retraining my embeddings at each step.
Thank you
G.

@gpcoursera
Copy link
Author

But I don't get the performance announced by Nils (F-score around 0.7). Here are my benchmark:
10 epoch: F1 on dev: 0.665523, F1 on test: 0.668567

@stone8oy
Copy link

stone8oy commented Jun 5, 2017

Thanks great tutorial. The NER_Keras.py implements the word-level method. How to implement the sentence-level log likelihood method for NER ? Can anybody give some hints ?

@nreimers
Copy link
Member

@stone8oy Currently Keras is not supporting this. You would need to implement a HMM model by yourself and add it to Keras.

However, there is a CRF model for Keras which can be found in the keras contrib. repository. This is quite similar to the STC model for Keras.

In the near future (within this month), I will also publish some code on Github that includes a CRF model for Keras.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants