Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functionality to convert file inputs to either json, csv or tsv. #338

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

john-thuo1
Copy link

@john-thuo1 john-thuo1 commented Oct 13, 2024

Contributor Checklist

  • [✔️] This pull request is on a separate branch and not the main branch.

Description

This pull request adds functionality to convert data into CSV, TSV, and JSON formats. The changes include:

  • Extension of the convert_to_json and convert_to_csv_or_tsv functions in convert.py.
  • Updates to the CLI in main.py to support new conversion commands.
  • Updates to the get function to support different file formats other that JSON.

Example Commands:

  1. From JSON to CSV:
    scribe-data convert --lan French --data-type translations --input-file ./fli/French/translations.json --output-type csv --output-dir ./converted/
  2. From CSV to JSON:
    scribe-data convert --lan French --data-type translations --input-file ./converted/French/translations.csv --output-type json --output-dir ./converted/
  3. From JSON to TSV:
    scribe-data convert --lan French --data-type translations --input-file ./fli/French/translations.json --output-type tsv --output-dir ./converted/ 
  4. From TSV to JSON:
    scribe-data convert --lan French --data-type translations --input-file ./converted/French/translations.tsv --output-type json --output-dir ./converted/ 
    

image

Tests for the convert methods have been implemented in test_convert.py

Related issue

Copy link

github-actions bot commented Oct 13, 2024

Thank you for the pull request!

The Scribe team will do our best to address your contribution as soon as we can. The following is a checklist for maintainers to make sure this process goes as well as possible. Feel free to address the points below yourself in further commits if you realize that actions are needed :)

If you're not already a member of our public Matrix community, please consider joining! We'd suggest using Element as your Matrix client, and definitely join the General and Data rooms once you're in. Also consider joining our bi-weekly Saturday dev syncs. It'd be great to have you!

Maintainer checklist

  • The linting and formatting workflow within the PR checks do not indicate new errors in the files changed

  • The CHANGELOG has been updated with a description of the changes for the upcoming release and the corresponding issue (if necessary)

@andrewtavis andrewtavis requested review from mhmohona and andrewtavis and removed request for mhmohona October 13, 2024 15:05
@andrewtavis andrewtavis added the hacktoberfest-accepted Accepted as a part of Hacktoberfest label Oct 13, 2024
@john-thuo1 john-thuo1 changed the title feat : Functionality to convert json/csv&tsv files Add functionality to convert file inputs to either json, csv or tsv. Oct 14, 2024
Copy link
Member

@mhmohona mhmohona left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good. Great work @john-thuo1

@john-thuo1
Copy link
Author

john-thuo1 commented Oct 15, 2024

It looks good. Great work @john-thuo1

Thanks for the review @mhmohona ! I will proceed to look into the remaining tests for the functions then,

@andrewtavis
Copy link
Member

Yes that'd be great, @john-thuo1! I'll review once those are in :)

@andrewtavis
Copy link
Member

Checking in here, @john-thuo1 :) Do you want to do the tests in a separate PR, or continue on this one? Whatever would be best for you 😊

@john-thuo1
Copy link
Author

john-thuo1 commented Oct 19, 2024

Checking in here, @john-thuo1 :) Do you want to do the tests in a separate PR, or continue on this one? Whatever would be best for you 😊

I had some challenges figuring out how to mock the input file, output directory, and output file and patching some functionalities, hence the delay, but it is all good now!

I have finished the tests. You can go ahead with the review @andrewtavis and let me know of other test scenarios I can add.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hacktoberfest-accepted Accepted as a part of Hacktoberfest
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants