Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial OpenAIEmbeddings support to Chonkie ✨ #46

Merged
merged 4 commits into from
Nov 19, 2024
Merged

Conversation

bhavnicksm
Copy link
Collaborator

This pull request introduces support for OpenAI embeddings within the chonkie project by adding a new OpenAIEmbeddings class and integrating it with the existing embeddings infrastructure. The most important changes include updating dependencies, adding the new embeddings class, registering it, and creating tests for the new functionality.

Integration of OpenAI Embeddings:

  • pyproject.toml: Added openai to the optional dependencies to support OpenAI embeddings.
  • src/chonkie/embeddings/__init__.py: Imported OpenAIEmbeddings and included it in the __all__ list to make it available for use.
  • src/chonkie/embeddings/openai.py: Added the OpenAIEmbeddings class, which implements the OpenAI embeddings using their API. This class includes methods for embedding single texts and batches, counting tokens, and computing similarity.

Registration of OpenAI Embeddings:

Testing for OpenAI Embeddings:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant