Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepend additional directive to user message instead of system prompt #277

Open
karthink opened this issue Apr 3, 2024 · 1 comment
Open

Comments

@karthink
Copy link
Owner

karthink commented Apr 3, 2024

If I may suggest: the directive needs to be prepended to the first user message with Anthropic, or we should have an option. Appending to the system message is not a good general solution.
Why do you think they should be prepended?

According to Anthropic documentation:

"in general, you can think about system prompts as a space to provide guidance about the overall interaction with Claude, and the user turn as part of the interaction itself, or when you have only a one-off task you want to accomplish"

That's how I work: system prompt defines the context, while the user prompt sends the specific directive and the text to be operated on.

This isn't clear-cut and is definitely up for debate, but I think we should at least have the option.

Originally posted by @jwr in #276 (comment)

@jwr
Copy link

jwr commented Apr 4, 2024

Closely related suggestion, based on the reading of docs for the Mistral-7B-Instruct-v0.2 model:

In order to leverage instruction fine-tuning, your prompt should be surrounded by [INST] and [/INST] tokens.

A :process-directive option to gptel-make-ollama (and others) that would accept a function taking backend, model, and directive arguments and return a processed directive. This is probably more user-friendly than a generic fn.

This might also be useful for Anthropic models, which like XML-style tagging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants