Skip to main content

Text-to-Vec API

Overview​

The text2vec-transformers module allows Weaviate to obtain vectors locally from text using a transformers-based model. It encapsulates models in Docker containers, enabling independent scaling on GPU-enabled hardware while keeping Weaviate on CPU-only hardware. This is advantageous because Weaviate is CPU-optimized.

Key Features:

  • Enabling this module activates the nearText search operator.
  • Supports the use of publicly available transformer models.
  • Automatically chunks text before passing it to the model.

Limitations:

  • Currently, cgc offers only intfloat/multilingual-e5-large tokenizer. If you want to use any another model, please contact us.

How to Run​

The Vectorizer app is available as a compute resource in a CUDA-enabled environment. Ensure you assign sufficient CPU and RAM resources for proper functioning. The RAM should be at least as large as the memory of the chosen GPU card plus 1 GB. However, you can experiment with different values if you don't anticipate a heavy load.

The NVIDIA A5000 GPU is the best choice for this task.

To run an instance, use the following command:

cgc compute create --name txt-2-vec -c 4 -m 25 -g 1 -gt A5000 t2v-transformers

After initialization, your vectorizer API will be accessible at:

http://txt-2-vec:8080 - locally within your namespace

caution

The vectorizer is not intended for use as a public API due to the lack of authorization mechanisms.

Integration with Weaviate​

To use the text2vec-transformers module with Weaviate, specify the t2v access URL when starting the database.

Additional Information​

  • For more information on how to integrate with Weaviate, please refer to the Weaviate documentation.
  • For more information on the Hugging Face model hub, please visit the Hugging Face website.
  • We choose this particular vectorizer after a set of multiple tests. We believe that this is one of the best choices if coming to perform vectorization both in Polish and English.

Conclusion​

The text2vec-transformers module is a powerful tool that enables Weaviate to obtain vectors locally from text using a transformers-based model. It offers key features such as support for nearText search, the use of publicly available models from the Hugging Face model hub, and automatic text chunking. While there are some limitations, such as the current support for only one tokenizer, the module is a valuable addition to the Weaviate ecosystem.