Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Llmblog2 #16

Merged
merged 36 commits into from
Feb 26, 2024
Merged

Llmblog2 #16

merged 36 commits into from
Feb 26, 2024

Commits on Feb 6, 2024

  1. working deepspeed

    @aciborowska
    aciborowska committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    c7bfdba View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Trying out mistral

    @KevinMusgrave
    KevinMusgrave committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    67a9654 View commit details
    Browse the repository at this point in the history

  2. Move some chat formatting logic to chat_format

    @KevinMusgrave
    KevinMusgrave committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    047b607 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. minor changes

    @KevinMusgrave
    KevinMusgrave committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    7a86d96 View commit details
    Browse the repository at this point in the history

  2. adding scrap.py

    @KevinMusgrave
    KevinMusgrave committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    34ff373 View commit details
    Browse the repository at this point in the history

  3. get max_length

    @KevinMusgrave
    KevinMusgrave committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    9ba7e6b View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. Get rid of max_length function

    @KevinMusgrave
    KevinMusgrave committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    788b170 View commit details
    Browse the repository at this point in the history

  2. test with batch size

    @KevinMusgrave
    KevinMusgrave committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    8608815 View commit details
    Browse the repository at this point in the history

  3. Test with a batch

    @KevinMusgrave
    KevinMusgrave committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    a90a835 View commit details
    Browse the repository at this point in the history

  4. Add get_tokenize_fn

    @KevinMusgrave
    KevinMusgrave committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    341542e View commit details
    Browse the repository at this point in the history

  5. Added test_model back in

    @KevinMusgrave
    KevinMusgrave committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    5ea0939 View commit details
    Browse the repository at this point in the history

  6. Add max_length back in

    @KevinMusgrave
    KevinMusgrave committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    c614bbe View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. lora seems to work

    @KevinMusgrave
    KevinMusgrave committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    b80fa15 View commit details
    Browse the repository at this point in the history

  2. include system prompt in user prompt for mistral

    @KevinMusgrave
    KevinMusgrave committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    cbde3df View commit details
    Browse the repository at this point in the history

  3. check if response is in decoded

    @KevinMusgrave
    KevinMusgrave committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    96abc27 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. Use WarmupDecayLR

    @KevinMusgrave
    KevinMusgrave committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    b20e987 View commit details
    Browse the repository at this point in the history

  2. Add lora flag to configs

    @KevinMusgrave
    KevinMusgrave committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    2f2d38f View commit details
    Browse the repository at this point in the history

  3. delete test_model, move into inference.py

    @KevinMusgrave
    KevinMusgrave committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    0032b48 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. plot token histogram

    @KevinMusgrave
    KevinMusgrave committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    bd45e48 View commit details
    Browse the repository at this point in the history

  2. plot for num tokens before response

    @KevinMusgrave
    KevinMusgrave committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    7df117a View commit details
    Browse the repository at this point in the history

  3. right-side padding and truncation. max_length 2048. Plot token histog… 

    …ram with padding and truncation
    @KevinMusgrave
    KevinMusgrave committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    ed845ce View commit details
    Browse the repository at this point in the history

  4. profiling

    @aciborowska
    aciborowska committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    7696c20 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. scrap->validate_tokenizer. Remove profiler stuff. Remove unnecessary … 

    …fields from config files
    @KevinMusgrave
    KevinMusgrave committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    f700eda View commit details
    Browse the repository at this point in the history

  2. Separate folder for new blog post

    @KevinMusgrave
    KevinMusgrave committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    e9329d5 View commit details
    Browse the repository at this point in the history

  3. Updated readme

    @KevinMusgrave
    KevinMusgrave committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    70a8d06 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. Manually set title

    @KevinMusgrave
    KevinMusgrave committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    1d0c924 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. Apply suggestions from code review 

    Co-authored-by: Agnieszka Ciborowska <[email protected]>
    @KevinMusgrave @aciborowska
    KevinMusgrave and aciborowska authored Feb 26, 2024
    Configuration menu
    Copy the full SHA
    d4656da View commit details
    Browse the repository at this point in the history

  2. Rename distributed.yaml -> lora.yaml. Remove image field in both configs

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    31c8dfb View commit details
    Browse the repository at this point in the history

  3. Use pytorch 2 image

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    353939e View commit details
    Browse the repository at this point in the history

  4. readme: distributed.yaml -> lora.yaml

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    562ef91 View commit details
    Browse the repository at this point in the history

  5. inference works with lora too

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    e87ba2b View commit details
    Browse the repository at this point in the history

  6. minor changes to the readmes

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    8919b58 View commit details
    Browse the repository at this point in the history

  7. Updated docker image

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    d411b1f View commit details
    Browse the repository at this point in the history

  8. Mention the --lora flag

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    6757ee6 View commit details
    Browse the repository at this point in the history

  9. Add --device argument to the inference script

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    55166fa View commit details
    Browse the repository at this point in the history

  10. Minor change to the readme

    @KevinMusgrave
    KevinMusgrave committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    955c0e0 View commit details
    Browse the repository at this point in the history