Unable to use .hierarchical_topics() for a loaded and merged model #2072
Unanswered
HuyenNguyenHelen
asked this question in
Q&A
Replies: 1 comment
-
In v0.16.2, when you merge models, the c-TF-IDF representations are removed since two different models could have completely different vocabularies, representations, etc. Instead, if you install BERTopic from the main branch, there should be the option to do the following: merged_model.hierarchical_topics(test_docs, use_ctfidf=False) This would allow you to use embeddings instead of the c-TF-IDF representations. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
Thank you so much for a great model.
I have many submodels which were trained and saved by:
topic_model.save("svaing_path", serialization="safetensors", save_ctfidf=True)
.I then loaded and merged these models to create a
merged_model
. I tried to get hierarchical topics of the merged model by:merged_model.hierarchical_topics(test_docs)
, but it failed with the error: TypeError: '--> 975 embeddings = self.c_tf_idf[self.outliers:] NoneType' object is not subscriptableI guess it's unable to load the
c_tf_idf_
from the saved model, asc_tf_idf_
strategy doesn't work for.reduce_outliers()
either. (probabilities and embeddings strategies work well with the merged model).Note that I was able to get the hierarchical clustering visualization with :
merged_model.visualize_hierarchy()
Could you please guide me how to get
merged_model.hierarchical_topics(test_docs)
worked?Thank you very much.
Beta Was this translation helpful? Give feedback.
All reactions