Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preliminary api client and parsl apps to export data and start training task #25

Draft
wants to merge 32 commits into
base: develop
Choose a base branch
from
Draft

Preliminary api client and parsl apps to export data and start training task #25

wants to merge 32 commits into from

Commits on Nov 5, 2024

  1. Preliminary escriptorium api client

    @rlskoeser
    rlskoeser committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    16aca6e View commit details
    Browse the repository at this point in the history

  2. Custom result class for API Task information

    @rlskoeser
    rlskoeser committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    486e777 View commit details
    Browse the repository at this point in the history

  3. Add api method to update model file in escriptorium

    @rlskoeser
    rlskoeser committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    c76540e View commit details
    Browse the repository at this point in the history

  4. Add method to monitor export and download resulting file

    @rlskoeser
    rlskoeser committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    6344e63 View commit details
    Browse the repository at this point in the history

  5. Preliminary parsl apps for download and train, preliminary train script

    @rlskoeser
    rlskoeser committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    684a5cc View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Use current user api endpoint and simplify required parameters

    @rlskoeser
    rlskoeser committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    e3bf0ad View commit details
    Browse the repository at this point in the history

  2. Improve error handling / reporting

    @rlskoeser
    rlskoeser committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    5190854 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. suggested additional arguments for train.py

    @cmroughan
    cmroughan authored Nov 11, 2024
    Configuration menu
    Copy the full SHA
    d669d68 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Update src/htr2hpc/train_apps.py 

    Co-authored-by: Laure Thompson <[email protected]>
    @rlskoeser @laurejt
    rlskoeser and laurejt authored Nov 12, 2024
    Configuration menu
    Copy the full SHA
    2329224 View commit details
    Browse the repository at this point in the history

  2. Update src/htr2hpc/api_client.py 

    Co-authored-by: Laure Thompson <[email protected]>
    @rlskoeser @laurejt
    rlskoeser and laurejt authored Nov 12, 2024
    Configuration menu
    Copy the full SHA
    c9190de View commit details
    Browse the repository at this point in the history

  3. Update src/htr2hpc/api_client.py 

    Co-authored-by: Laure Thompson <[email protected]>
    @rlskoeser @laurejt
    rlskoeser and laurejt authored Nov 12, 2024
    Configuration menu
    Copy the full SHA
    424764e View commit details
    Browse the repository at this point in the history

  4. Minor cleanup on api and preliminary training code

    @rlskoeser
    rlskoeser committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    c35d805 View commit details
    Browse the repository at this point in the history

  5. API methods for models, document parts, and util method to download file

    @rlskoeser
    rlskoeser committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    4dd7550 View commit details
    Browse the repository at this point in the history

  6. Create training data directly from document parts api response

    @rlskoeser
    rlskoeser committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    35671d5 View commit details
    Browse the repository at this point in the history

  7. Make api methods more descriptive & distinct

    @rlskoeser
    rlskoeser committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    78fa21d View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. Improve download filename logic

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    334cab2 View commit details
    Browse the repository at this point in the history

  2. Add & use working directory param; document other needed parameters

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    516e0a7 View commit details
    Browse the repository at this point in the history

  3. Clean up with ruff

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    01fe426 View commit details
    Browse the repository at this point in the history

  4. Restructure parsl app/train code; configure package script for training

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d3a7cb9 View commit details
    Browse the repository at this point in the history

  5. Adjust slurm/srun partition and other configurations

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    d8abafe View commit details
    Browse the repository at this point in the history

  6. Revise slurm provider configuration

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    6d50a8d View commit details
    Browse the repository at this point in the history

  7. Update segtrain command with paths for output model and logs

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    9c66fcf View commit details
    Browse the repository at this point in the history

  8. Configure parsl app executor at the right level

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    57f1b85 View commit details
    Browse the repository at this point in the history

  9. Adjust paths; use absolute paths for training task

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    cce97b0 View commit details
    Browse the repository at this point in the history

  10. Calculate absolute paths before changing working directory

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    e248479 View commit details
    Browse the repository at this point in the history

  11. Add logic for default kraken models if model id is not specified

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    f03aeee View commit details
    Browse the repository at this point in the history

  12. Add api method and notes for getting transcription text content

    @rlskoeser
    rlskoeser committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    8f325a6 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. Adjust arguments for the two modes of training

    @rlskoeser
    rlskoeser committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    fd1cc34 View commit details
    Browse the repository at this point in the history

  2. Remove log dir option since nothing is getting logged

    @rlskoeser
    rlskoeser committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    005ff36 View commit details
    Browse the repository at this point in the history

  3. Preliminary instructions for installing and running htr2hpc-train

    @rlskoeser
    rlskoeser committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    32b0ab1 View commit details
    Browse the repository at this point in the history

  4. Increase walltime in hpc parsl config

    @rlskoeser
    rlskoeser committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    af3dc25 View commit details
    Browse the repository at this point in the history

  5. Update to use new subparser mode instead of old job arg

    @rlskoeser
    rlskoeser committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    a47a143 View commit details
    Browse the repository at this point in the history