Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add local language models support #42

Closed
kabachuha opened this issue Jun 1, 2023 · 1 comment
Closed

Add local language models support #42

kabachuha opened this issue Jun 1, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@kabachuha
Copy link

kabachuha commented Jun 1, 2023

Hi, @vasilecampeanu !

While GPT-* models are awesome, there's an issue of privacy arising from using an external third party API, especially when the information may be sensitive. More than that, they don't allow as much customization for one's needs as the local models. In my mind, adding support of local models is one of the top priorities of such a plugin.

Fortunately, thanks for people like oobabooga making a wrapper for many possible local models like LLaMa, Vicuna, MPT, Galactica, etc it's pretty straightforward to implement. Just replace the call for OpenAI's API to oobabooga API's endpoint. For example, how I did this rizerphe/obsidian-companion#13 (may require some tweaks)

supports streaming mode as well

@vasilecampeanu
Copy link
Owner

vasilecampeanu commented Jun 3, 2023

Hello, I'm sorry for the late response. Thank you for the pull request and for bringing up the issue. I intend to add support for local models, but I'm quite busy at the moment and haven't found the time to delve into it yet. Currently, my primary focus is to release the plugin to the community list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants