Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat/use vllm server #40

Merged
merged 58 commits into from
May 22, 2024
Merged

Feat/use vllm server #40

merged 58 commits into from
May 22, 2024

Commits on May 21, 2024

  1. feat: Use vllm server

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    2d0f5f7 View commit details
    Browse the repository at this point in the history

  2. fix: Use self.model

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    223cd96 View commit details
    Browse the repository at this point in the history

  3. debug

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    50c907f View commit details
    Browse the repository at this point in the history

  4. debug

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    d997f28 View commit details
    Browse the repository at this point in the history

  5. feat: Try using guided_json

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    24fd935 View commit details
    Browse the repository at this point in the history

  6. fix: Use extra_body

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    984079f View commit details
    Browse the repository at this point in the history

  7. fix: Use json

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    309baac View commit details
    Browse the repository at this point in the history

  8. fix: Use model_json_schema

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    35fab6c View commit details
    Browse the repository at this point in the history

  9. chore: Include response_format

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    9b63001 View commit details
    Browse the repository at this point in the history

  10. chore: Logging

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    0061921 View commit details
    Browse the repository at this point in the history

  11. chore: Remove logging

    @saattrupdan
    saattrupdan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    86745cb View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    25c8e07 View commit details
    Browse the repository at this point in the history

  2. fix: Do not break streaming if chunk_str is None

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    7e0528c View commit details
    Browse the repository at this point in the history

  3. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    4fef832 View commit details
    Browse the repository at this point in the history

  4. feat: Spawn new vLLM server if not already running

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    05a96d3 View commit details
    Browse the repository at this point in the history

  5. fix: Do not use api_key if running vLLM generator

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    ff9a16e View commit details
    Browse the repository at this point in the history

  6. fix: vLLM config

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    d5b8f6b View commit details
    Browse the repository at this point in the history

  7. chore: Remove breakpoint

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    c1dc715 View commit details
    Browse the repository at this point in the history

  8. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a73d98d View commit details
    Browse the repository at this point in the history

  9. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    b9e72f0 View commit details
    Browse the repository at this point in the history

  10. fix: Set server after booting it

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    6622b7a View commit details
    Browse the repository at this point in the history

  11. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    acde691 View commit details
    Browse the repository at this point in the history

  12. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    ef4c33e View commit details
    Browse the repository at this point in the history

  13. fix: Add sleep after server start

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    00b38bd View commit details
    Browse the repository at this point in the history

  14. fix: Only require CUDA to start the vLLM inference server, not to use… 

    … one
    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    d532c77 View commit details
    Browse the repository at this point in the history

  15. fix: Only set guided_json if using vLLM

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    dc0be2c View commit details
    Browse the repository at this point in the history

  16. tests: vLLM tests

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    3fd39c6 View commit details
    Browse the repository at this point in the history

  17. feat: Add more args to vLLM server

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    ad04ed1 View commit details
    Browse the repository at this point in the history

  18. fix: Typo

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    226a88a View commit details
    Browse the repository at this point in the history

  19. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    e726226 View commit details
    Browse the repository at this point in the history

  20. fix: Up vLLM startup sleep time

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    dedb032 View commit details
    Browse the repository at this point in the history

  21. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    598f286 View commit details
    Browse the repository at this point in the history

  22. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    36e6d0b View commit details
    Browse the repository at this point in the history

  23. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    192721a View commit details
    Browse the repository at this point in the history

  24. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    89a56c2 View commit details
    Browse the repository at this point in the history

  25. fix: Add port back in

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a2936b6 View commit details
    Browse the repository at this point in the history

  26. fix: Set up self.server in OpenaiGenerator correctly

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    e414942 View commit details
    Browse the repository at this point in the history

  27. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    6d01292 View commit details
    Browse the repository at this point in the history

  28. fix: Store config in VllmGenerator

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    905dd97 View commit details
    Browse the repository at this point in the history

  29. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    63cf5c9 View commit details
    Browse the repository at this point in the history

  30. feat: Check manually if Uvicorn server has started

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    837ce28 View commit details
    Browse the repository at this point in the history

  31. feat: Block stderr when loading tokenizer

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    14e5d33 View commit details
    Browse the repository at this point in the history

  32. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    7c1298c View commit details
    Browse the repository at this point in the history

  33. refactor: Use HiddenPrints

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    bc1641b View commit details
    Browse the repository at this point in the history

  34. fix: Block transformers logging

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    67de367 View commit details
    Browse the repository at this point in the history

  35. feat: Add --host back in

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    aaae8cb View commit details
    Browse the repository at this point in the history

  36. debug

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    8b67836 View commit details
    Browse the repository at this point in the history

  37. fix: Add del self in __del__

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    9a93b7b View commit details
    Browse the repository at this point in the history

  38. chore: Ignore ResourceWarning in pytest

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    3e15ea2 View commit details
    Browse the repository at this point in the history

  39. tests: Initialise the VllmGenerator fewer times in tests

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    38cb047 View commit details
    Browse the repository at this point in the history

  40. fix: Do not hardcode different ports

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    2c3ff56 View commit details
    Browse the repository at this point in the history

  41. tests: Use same VllmGenerator

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    9b2fddc View commit details
    Browse the repository at this point in the history

  42. tests: Remove validity check test, as it is impossible with VllmGener… 

    …ator
    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    21abef2 View commit details
    Browse the repository at this point in the history

  43. tests: Remove random_seed from VllmGenerator config

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    65e538a View commit details
    Browse the repository at this point in the history

  44. docs: Add comments

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    6facb53 View commit details
    Browse the repository at this point in the history

  45. fix: Raise ValueError in get_component_by_name if module or class don… 

    …'t exist
    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    aa64ac4 View commit details
    Browse the repository at this point in the history

  46. docs: Update coverage badge

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    213950b View commit details
    Browse the repository at this point in the history

  47. chore: Re-instate pre-commit hook

    @saattrupdan
    saattrupdan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    46c61ef View commit details
    Browse the repository at this point in the history