Limit embedding requests to text samples under 10k (#155)

The `embeddings-gecko-001` model doesn't seem to like anything over 10k, so filter them out in the tutorial.
google · Oct 17, 2023 · 7a42081 · 7a42081
1 parent 67c8fd3
commit 7a42081
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/site/en/examples/clustering_with_embeddings.ipynb b/site/en/examples/clustering_with_embeddings.ipynb
@@ -418,6 +418,8 @@
         "df_train['Label'] = newsgroups_train.target\n",
         "# Match label to target name index\n",
         "df_train['Class Name'] = df_train['Label'].map(newsgroups_train.target_names.__getitem__)\n",
+        "# Retain text samples that can be used in the gecko model.\n",
+        "df_train = df_train[df_train['Text'].str.len() < 10000]\n",
         "\n",
         "df_train"
       ]