Skip to content

Issues: symflower/eval-dev-quality

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

40 Open 89 Closed
40 Open 89 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Remove Docker evaluation volume if evaluation is aborted bug Something isn't working
#366 opened Oct 21, 2024 by bauersimon
1 task done
v0.7.0
@bauersimon
Spring Boot "write-test" enhancement New feature or request
#365 opened Oct 18, 2024 by bauersimon
3 tasks done
v0.7.0
@bauersimon
1
Track query tokens and save them to CSV enhancement New feature or request
#347 opened Sep 25, 2024 by bauersimon v0.7.0
@ruiAzevedo19
Automate revision string of the binary enhancement New feature or request
#344 opened Sep 24, 2024 by bauersimon v0.7.0
Different behavior depending on Go version bug Something isn't working documentation Improvements or additions to documentation
#341 opened Sep 23, 2024 by bauersimon
2 tasks
v0.7.0
Enable Ruby tests in Windows CI bug Something isn't working CI GitHub Actions
#334 opened Aug 30, 2024 by bauersimon v0.7.0
Add new subcommand verify that just does repository verification and run that in CI bug Something isn't working
#330 opened Aug 27, 2024 by bauersimon
Kubernetes job watching does not work when erroring bug Something isn't working
#329 opened Aug 27, 2024 by Munsio v0.7.0
@Munsio
Move Dependency installation of Docker into multistage builds CI GitHub Actions enhancement New feature or request
#319 opened Aug 8, 2024 by Munsio
3 tasks
v0.7.0
@Munsio
Copy of evaluation data for kubernetes is not working bug Something isn't working
#312 opened Aug 5, 2024 by Munsio v0.7.0
@Munsio
Follow-up: Use a JSON configuration file to set up an evaluation run enhancement New feature or request
#307 opened Aug 1, 2024 by ruiAzevedo19
1 task
v0.7.0
@ruiAzevedo19
Rethink retry logic for LLM Providers bug Something isn't working
#305 opened Aug 1, 2024 by Munsio v0.7.0
@bauersimon
Reporting tool does not work with /**/ directory globing bug Something isn't working
#304 opened Aug 1, 2024 by Munsio v0.7.0
@ruiAzevedo19
5
Roadmap for v0.7.0 roadmap Collection of issues for a release
#301 opened Jul 31, 2024 by bauersimon 100+ v0.7.0
@zimmski
Data visualization based on evaluation CSV files enhancement New feature or request
#296 opened Jul 30, 2024 by ruiAzevedo19
4 of 7 tasks
v0.7.0
@ruiAzevedo19
1
Openrouter Provider preferences bug Something isn't working enhancement New feature or request
#286 opened Jul 25, 2024 by Munsio v0.7.0
@Munsio
2
Assess failing tests enhancement New feature or request
#235 opened Jul 2, 2024 by zimmski v0.7.0
1
Weight "executed code" more prominently enhancement New feature or request
#233 opened Jul 2, 2024 by zimmski v0.7.0
@bauersimon
4
Multiple model parameter with same value result in multiple evaluations bug Something isn't working question Further information is requested
#220 opened Jun 26, 2024 by Munsio v0.7.0
@Munsio
Evaluation run for all "good open weight models" with all available quantizations and different GPUs enhancement New feature or request
#209 opened Jun 22, 2024 by zimmski v0.7.0
Interactive result comparison enhancement New feature or request
#208 opened Jun 21, 2024 by bauersimon
9 tasks
v0.7.0
Keep individual coverage files and LLM query/responses enhancement New feature or request
#204 opened Jun 19, 2024 by zimmski v0.7.0
@ahumenberger
3
Evaluation task: TDD enhancement New feature or request
#194 opened Jun 17, 2024 by ahumenberger v0.7.0
Coverage for Java is tracked for lines, while Go is tracked for ranges bug Something isn't working
#193 opened Jun 14, 2024 by bauersimon v0.7.0
4
Deal with dependencies requested by LLMs bug Something isn't working
#174 opened Jun 10, 2024 by ahumenberger v0.7.0
2
ProTip! What’s not been updated in a month: updated:<2024-10-22.