Can I store all word to vector mappings in a dictionary? Is this even plausible? #13569
-
Hi all, I am new to spacy, and I am not sure this even makes sense, but I have a question. I am doing some sentiment analysis where I am inferring the sentiment between two entities. To train my model, I am first mapping the words in my vocabulary to spacy embeddings using Unfortunately, I need to use the most recent version of spacy for some other stuff, so I started wondering if I could just make a python dictionary where I save the mappings between words and their embeddings, dump that dictionary, and load it into my codebase that needs the most recent version of spacy and just use the mappings there to train my models. If this is feasible, how might I start doing this? What would be the best way? I am unsure if this is a good idea, or if it even makes sense in the context of spacy. I don't quite understand spacy well enough to really know any better, so any and all insight/thoughts that people would like to share is greatly appreciated. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Using guidance from https://stackoverflow.com/questions/55383604/how-to-extract-word-embedding-vectors-from-a-trained-spacy-model , I was able to figure it out! Still not sure if this is a great idea, but I used the following code to create a dump of strings to embeddings
and saving it with
Maybe there is another, much cleaner way to do this, but this seems to fit my purposes just fine! For those wondering the file size is about 4.1 Gigs |
Beta Was this translation helpful? Give feedback.
Using guidance from https://stackoverflow.com/questions/55383604/how-to-extract-word-embedding-vectors-from-a-trained-spacy-model , I was able to figure it out!
Still not sure if this is a great idea, but I used the following code to create a dump of strings to embeddings
and saving it with
Maybe there is another, much cleaner way to do this, but this seems t…