Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Llmblog2 #16

Merged
merged 36 commits into from
Feb 26, 2024
Merged

Llmblog2 #16

merged 36 commits into from
Feb 26, 2024

Conversation

KevinMusgrave
Copy link
Contributor

No description provided.

aciborowska and others added 26 commits February 5, 2024 18:09
@aciborowska
working deepspeed
c7bfdba
@KevinMusgrave
Trying out mistral
67a9654
@KevinMusgrave
Move some chat formatting logic to chat_format
047b607
@KevinMusgrave
minor changes
7a86d96
@KevinMusgrave
adding scrap.py
34ff373
@KevinMusgrave
get max_length
9ba7e6b
@KevinMusgrave
Get rid of max_length function
788b170
@KevinMusgrave
test with batch size
8608815
@KevinMusgrave
Test with a batch
a90a835
@KevinMusgrave
Add get_tokenize_fn
341542e
@KevinMusgrave
Added test_model back in
5ea0939
@KevinMusgrave
Add max_length back in
c614bbe
@KevinMusgrave
lora seems to work
b80fa15
@KevinMusgrave
include system prompt in user prompt for mistral
cbde3df
@KevinMusgrave
check if response is in decoded
96abc27
@KevinMusgrave
Use WarmupDecayLR
b20e987
@KevinMusgrave
Add lora flag to configs
2f2d38f
@KevinMusgrave
delete test_model, move into inference.py
0032b48
@KevinMusgrave
plot token histogram
bd45e48
@KevinMusgrave
plot for num tokens before response
7df117a
@KevinMusgrave
right-side padding and truncation. max_length 2048. Plot token histog…
ed845ce 
…ram with padding and truncation
@aciborowska
profiling
7696c20
@KevinMusgrave
scrap->validate_tokenizer. Remove profiler stuff. Remove unnecessary …
f700eda 
…fields from config files
@KevinMusgrave
Separate folder for new blog post
e9329d5
@KevinMusgrave
Updated readme
70a8d06
@KevinMusgrave
Manually set title
1d0c924
@KevinMusgrave
Copy link
Contributor Author

KevinMusgrave commented Feb 23, 2024 •
edited
Loading

The test_model.py script downloads the best checkpoint if exp_id is given, and this was something I was able to run on my laptop for TinyLlama. I also ran it for the pretrained Mistral model, but I don't want to run it for a finetuned Mistral model because I think the checkpoint will be huge.

Part of the problem is that the checkpoint contains the optimizer state, which I don't need to evaluate the model. Maybe there is some way to download only part of the checkpoint? I'm not sure.

aciborowska
aciborowska approved these changes Feb 26, 2024
View reviewed changes
Copy link
Contributor

@aciborowska aciborowska left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@KevinMusgrave KevinMusgrave merged commit 209c06d into main Feb 26, 2024
1 check passed
@KevinMusgrave KevinMusgrave deleted the llmblog2-profiler branch February 26, 2024 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aciborowska aciborowska aciborowska approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@KevinMusgrave @aciborowska