Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] improved Task prompt customization #964

Open
davidberenstein1957 opened this issue Sep 10, 2024 · 0 comments
Open

[FEATURE] improved Task prompt customization #964

davidberenstein1957 opened this issue Sep 10, 2024 · 0 comments
Labels
enhancement New feature or request interview

Comments

@davidberenstein1957
Copy link
Member

davidberenstein1957 commented Sep 10, 2024

Is your feature request related to a problem? Please describe.

  • where can I find the system prompt?
  • how easy is it to overwrite/adapt the prompt template?
  • can I leave out certain parts of the generation e.g. rationale/critique.

Describe the solution you'd like
Some features/improvements that help with the questions above.

Describe alternatives you've considered
NA

Additional context
ideas which might not need to work with set_runtime_parameters specifically.

  • would it be possible to have a few-shot example section
Task.set_runtime_parameters(
    "few_shot_example": {
        "inputs": {"input_column": ["my example input"]}}, 
        "outputs": {"outut_column": ["my example output"]}}
    }
)
# or 
Task.set_runtime_parameters(
    "few_shot_example": "my good example"
)
  • can we allow adding a pre-fix/suf-fix to each prompt by default.
Task.set_runtime_parameters(
    "prompt_prefix": "always do this thingy",
    "prompt_suffix": "always do that other thingy"
)
  • always allow for adding a system prompt, even when this was not added in the original paper
Task.set_runtime_parameters(
    "system_prompt": "always do this thingy"
)
  • Overwrite prompt templates. should wee allow disabling certain output columns present in the prompt template through if-else statements? E.g., in UltraFeedback I might not want rationales.
Task.set_runtime_parameters(
    "prompt template": "do ultra feedback {score} raationale {rationale}"
)
Task.set_runtime_parameters(
    "prompt template": "do ultra feedback {score}"
)
Task.set_runtime_parameters(
    "prompt template": "path/to/.jinja2"
)
@davidberenstein1957 davidberenstein1957 added enhancement New feature or request interview labels Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request interview
Projects
None yet
Development

No branches or pull requests

1 participant