Add text2sql tasks #1414
Merged
Add text2sql tasks #1414
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
…dling Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
Signed-off-by: Yotam-Perlitz <[email protected]>
elronbandel
approved these changes
Jan 23, 2025
tejaswini
pushed a commit
that referenced
this pull request
Jan 24, 2025
* add text2sql templates Signed-off-by: Yotam-Perlitz <[email protected]> * add data managment utility for text2sql Signed-off-by: Yotam-Perlitz <[email protected]> * add basic template Signed-off-by: Yotam-Perlitz <[email protected]> * add sql execution accuracy metric Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql execution accuracy metric Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql task Signed-off-by: Yotam-Perlitz <[email protected]> * condition download in presence of a cache dir Signed-off-by: Yotam-Perlitz <[email protected]> * add init fille Signed-off-by: Yotam-Perlitz <[email protected]> * add processors Signed-off-by: Yotam-Perlitz <[email protected]> * add processors Signed-off-by: Yotam-Perlitz <[email protected]> * add basic template Signed-off-by: Yotam-Perlitz <[email protected]> * change id to int Signed-off-by: Yotam-Perlitz <[email protected]> * change notations in templates Signed-off-by: Yotam-Perlitz <[email protected]> * push to catalog Signed-off-by: Yotam-Perlitz <[email protected]> * add evidence, remove SL Signed-off-by: Yotam-Perlitz <[email protected]> * remove unued function, fix Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports from unitxt.text2sql Signed-off-by: Yotam-Perlitz <[email protected]> * push to catalog Signed-off-by: Yotam-Perlitz <[email protected]> * fix cache location Signed-off-by: Yotam-Perlitz <[email protected]> * add example Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * add func_timeout to test reqs Signed-off-by: Yotam-Perlitz <[email protected]> * fix typing Signed-off-by: Yotam-Perlitz <[email protected]> * change template name Signed-off-by: Yotam-Perlitz <[email protected]> * push to catalog Signed-off-by: Yotam-Perlitz <[email protected]> * add req Signed-off-by: Yotam-Perlitz <[email protected]> * add local model option Signed-off-by: Yotam-Perlitz <[email protected]> * fix databases download Signed-off-by: Yotam-Perlitz <[email protected]> * fix databases download Signed-off-by: Yotam-Perlitz <[email protected]> * add loader limit ot make example faster Signed-off-by: Yotam-Perlitz <[email protected]> * fix cache paths, avoid re-download Signed-off-by: Yotam-Perlitz <[email protected]> * add type schema Signed-off-by: Yotam-Perlitz <[email protected]> * remove inports from inits Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql to inits Signed-off-by: Yotam-Perlitz <[email protected]> * update card to use serializers Signed-off-by: Yotam-Perlitz <[email protected]> * add schema serializer Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql serializer to default template Signed-off-by: Yotam-Perlitz <[email protected]> * add schema to task Signed-off-by: Yotam-Perlitz <[email protected]> * adjust templates to using serializer Signed-off-by: Yotam-Perlitz <[email protected]> * adjust templates to using serializer Signed-off-by: Yotam-Perlitz <[email protected]> * fix processor Signed-off-by: Yotam-Perlitz <[email protected]> * remove target prefix from template Signed-off-by: Yotam-Perlitz <[email protected]> * add shuffle to bird Signed-off-by: Yotam-Perlitz <[email protected]> * add shuffle to bird Signed-off-by: Yotam-Perlitz <[email protected]> * edit template Signed-off-by: Yotam-Perlitz <[email protected]> * remove comment from init Signed-off-by: Yotam-Perlitz <[email protected]> * clear processors code Signed-off-by: Yotam-Perlitz <[email protected]> * add option with ticks Signed-off-by: Yotam-Perlitz <[email protected]> * add anls metric Signed-off-by: Yotam-Perlitz <[email protected]> * add template Signed-off-by: Yotam-Perlitz <[email protected]> * drop comment Signed-off-by: Yotam-Perlitz <[email protected]> * remove recursion limit Signed-off-by: Yotam-Perlitz <[email protected]> * add loader_limit to example Signed-off-by: Yotam-Perlitz <[email protected]> * fix recursion error Signed-off-by: Yotam-Perlitz <[email protected]> * move import to withing metric Signed-off-by: Yotam-Perlitz <[email protected]> * remove catalog files wo prepare Signed-off-by: Yotam-Perlitz <[email protected]> * fix typing Signed-off-by: Yotam-Perlitz <[email protected]> * change template im example Signed-off-by: Yotam-Perlitz <[email protected]> * moving text2sql implementaion to the main src dir Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * import data_utils Signed-off-by: Yotam-Perlitz <[email protected]> * fix formatting Signed-off-by: Yotam-Perlitz <[email protected]> * refactor names Signed-off-by: Yotam-Perlitz <[email protected]> * add processors tests Signed-off-by: Yotam-Perlitz <[email protected]> * add more tests Signed-off-by: Yotam-Perlitz <[email protected]> * add tests Signed-off-by: Yotam-Perlitz <[email protected]> * refactor: allow more data sources Signed-off-by: Yotam-Perlitz <[email protected]> * allow db source input Signed-off-by: Yotam-Perlitz <[email protected]> * organize imports Signed-off-by: Yotam-Perlitz <[email protected]> * update example Signed-off-by: Yotam-Perlitz <[email protected]> * add db_type to task Signed-off-by: Yotam-Perlitz <[email protected]> * format Signed-off-by: Yotam-Perlitz <[email protected]> * add db_type to task Signed-off-by: Yotam-Perlitz <[email protected]> * add local db definition ability Signed-off-by: Yotam-Perlitz <[email protected]> * add EE tests Signed-off-by: Yotam-Perlitz <[email protected]> * add tests Signed-off-by: Yotam-Perlitz <[email protected]> * rename file Signed-off-by: Yotam-Perlitz <[email protected]> * rename file Signed-off-by: Yotam-Perlitz <[email protected]> * update sql metric Signed-off-by: Yotam-Perlitz <[email protected]> * rename file Signed-off-by: Yotam-Perlitz <[email protected]> * refactor types, serializers and metric Signed-off-by: Yotam-Perlitz <[email protected]> --------- Signed-off-by: Yotam-Perlitz <[email protected]>
elronbandel
added a commit
that referenced
this pull request
Jan 26, 2025
* Renamed criterias in LLM-as-a-Judge metrics to criteria. * Reintroduced imports that were removed from llm_as_judge.py * Updated the examples documentation for LLM-as-a-judge * Added missing import * Fixed formatting using ruff * add a filter to wikitq (#1547) * Add text2sql tasks to unitxt (#1414) * add text2sql templates Signed-off-by: Yotam-Perlitz <[email protected]> * add data managment utility for text2sql Signed-off-by: Yotam-Perlitz <[email protected]> * add basic template Signed-off-by: Yotam-Perlitz <[email protected]> * add sql execution accuracy metric Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql execution accuracy metric Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql task Signed-off-by: Yotam-Perlitz <[email protected]> * condition download in presence of a cache dir Signed-off-by: Yotam-Perlitz <[email protected]> * add init fille Signed-off-by: Yotam-Perlitz <[email protected]> * add processors Signed-off-by: Yotam-Perlitz <[email protected]> * add processors Signed-off-by: Yotam-Perlitz <[email protected]> * add basic template Signed-off-by: Yotam-Perlitz <[email protected]> * change id to int Signed-off-by: Yotam-Perlitz <[email protected]> * change notations in templates Signed-off-by: Yotam-Perlitz <[email protected]> * push to catalog Signed-off-by: Yotam-Perlitz <[email protected]> * add evidence, remove SL Signed-off-by: Yotam-Perlitz <[email protected]> * remove unued function, fix Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports from unitxt.text2sql Signed-off-by: Yotam-Perlitz <[email protected]> * push to catalog Signed-off-by: Yotam-Perlitz <[email protected]> * fix cache location Signed-off-by: Yotam-Perlitz <[email protected]> * add example Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * add func_timeout to test reqs Signed-off-by: Yotam-Perlitz <[email protected]> * fix typing Signed-off-by: Yotam-Perlitz <[email protected]> * change template name Signed-off-by: Yotam-Perlitz <[email protected]> * push to catalog Signed-off-by: Yotam-Perlitz <[email protected]> * add req Signed-off-by: Yotam-Perlitz <[email protected]> * add local model option Signed-off-by: Yotam-Perlitz <[email protected]> * fix databases download Signed-off-by: Yotam-Perlitz <[email protected]> * fix databases download Signed-off-by: Yotam-Perlitz <[email protected]> * add loader limit ot make example faster Signed-off-by: Yotam-Perlitz <[email protected]> * fix cache paths, avoid re-download Signed-off-by: Yotam-Perlitz <[email protected]> * add type schema Signed-off-by: Yotam-Perlitz <[email protected]> * remove inports from inits Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql to inits Signed-off-by: Yotam-Perlitz <[email protected]> * update card to use serializers Signed-off-by: Yotam-Perlitz <[email protected]> * add schema serializer Signed-off-by: Yotam-Perlitz <[email protected]> * add text2sql serializer to default template Signed-off-by: Yotam-Perlitz <[email protected]> * add schema to task Signed-off-by: Yotam-Perlitz <[email protected]> * adjust templates to using serializer Signed-off-by: Yotam-Perlitz <[email protected]> * adjust templates to using serializer Signed-off-by: Yotam-Perlitz <[email protected]> * fix processor Signed-off-by: Yotam-Perlitz <[email protected]> * remove target prefix from template Signed-off-by: Yotam-Perlitz <[email protected]> * add shuffle to bird Signed-off-by: Yotam-Perlitz <[email protected]> * add shuffle to bird Signed-off-by: Yotam-Perlitz <[email protected]> * edit template Signed-off-by: Yotam-Perlitz <[email protected]> * remove comment from init Signed-off-by: Yotam-Perlitz <[email protected]> * clear processors code Signed-off-by: Yotam-Perlitz <[email protected]> * add option with ticks Signed-off-by: Yotam-Perlitz <[email protected]> * add anls metric Signed-off-by: Yotam-Perlitz <[email protected]> * add template Signed-off-by: Yotam-Perlitz <[email protected]> * drop comment Signed-off-by: Yotam-Perlitz <[email protected]> * remove recursion limit Signed-off-by: Yotam-Perlitz <[email protected]> * add loader_limit to example Signed-off-by: Yotam-Perlitz <[email protected]> * fix recursion error Signed-off-by: Yotam-Perlitz <[email protected]> * move import to withing metric Signed-off-by: Yotam-Perlitz <[email protected]> * remove catalog files wo prepare Signed-off-by: Yotam-Perlitz <[email protected]> * fix typing Signed-off-by: Yotam-Perlitz <[email protected]> * change template im example Signed-off-by: Yotam-Perlitz <[email protected]> * moving text2sql implementaion to the main src dir Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * fix imports Signed-off-by: Yotam-Perlitz <[email protected]> * import data_utils Signed-off-by: Yotam-Perlitz <[email protected]> * fix formatting Signed-off-by: Yotam-Perlitz <[email protected]> * refactor names Signed-off-by: Yotam-Perlitz <[email protected]> * add processors tests Signed-off-by: Yotam-Perlitz <[email protected]> * add more tests Signed-off-by: Yotam-Perlitz <[email protected]> * add tests Signed-off-by: Yotam-Perlitz <[email protected]> * refactor: allow more data sources Signed-off-by: Yotam-Perlitz <[email protected]> * allow db source input Signed-off-by: Yotam-Perlitz <[email protected]> * organize imports Signed-off-by: Yotam-Perlitz <[email protected]> * update example Signed-off-by: Yotam-Perlitz <[email protected]> * add db_type to task Signed-off-by: Yotam-Perlitz <[email protected]> * format Signed-off-by: Yotam-Perlitz <[email protected]> * add db_type to task Signed-off-by: Yotam-Perlitz <[email protected]> * add local db definition ability Signed-off-by: Yotam-Perlitz <[email protected]> * add EE tests Signed-off-by: Yotam-Perlitz <[email protected]> * add tests Signed-off-by: Yotam-Perlitz <[email protected]> * rename file Signed-off-by: Yotam-Perlitz <[email protected]> * rename file Signed-off-by: Yotam-Perlitz <[email protected]> * update sql metric Signed-off-by: Yotam-Perlitz <[email protected]> * rename file Signed-off-by: Yotam-Perlitz <[email protected]> * refactor types, serializers and metric Signed-off-by: Yotam-Perlitz <[email protected]> --------- Signed-off-by: Yotam-Perlitz <[email protected]> * Add deduplicate operator (#1549) * Add deduplicate operator Signed-off-by: elronbandel <[email protected]> * Deduplicate MMLU Signed-off-by: elronbandel <[email protected]> * Update Deduplicate example in documentation for clarity Signed-off-by: elronbandel <[email protected]> * Deduplicate social iqa Signed-off-by: elronbandel <[email protected]> --------- Signed-off-by: elronbandel <[email protected]> * Fix the authentication problem (#1550) * Attach assitant answers to their origins with url link (#1528) Signed-off-by: elronbandel <[email protected]> * Add mtrag benchmark (#1548) * Add mtrag benchmark Signed-off-by: elronbandel <[email protected]> * Add multi_type_serializer for references and prediction fields in various JSON metrics Signed-off-by: elronbandel <[email protected]> * Remove unused TempOperator class and delete obsolete multi_turn.json task file Signed-off-by: elronbandel <[email protected]> --------- Signed-off-by: elronbandel <[email protected]> * Update end of year summary blog (#1552) Signed-off-by: elronbandel <[email protected]> * Update strategic focus section in 2024 summary blog to emphasize usability Signed-off-by: elronbandel <[email protected]> * Added missing import * Fix llm as judge example Signed-off-by: Martín Santillán Cooper <[email protected]> * apply 'pre-commit run --all-files' Signed-off-by: Martín Santillán Cooper <[email protected]> --------- Signed-off-by: Yotam-Perlitz <[email protected]> Signed-off-by: elronbandel <[email protected]> Signed-off-by: Martín Santillán Cooper <[email protected]> Co-authored-by: Tejaswini Pedapati <[email protected]> Co-authored-by: ShirApp <[email protected]> Co-authored-by: Yotam Perlitz <[email protected]> Co-authored-by: Elron Bandel <[email protected]> Co-authored-by: Elad <[email protected]> Co-authored-by: Martín Santillán Cooper <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.