song-1.4.1

Summary

Bug Fixes

• Deadlock condition with score when publishing an analysis (related to overture-stack/score#140)

Downloads

Song-Client-1.4.1 Distribution

Song-Server-1.4.1 Distribution

Docker Compose

song-docker-1.4.1

git clone --branch song-docker-1.4.1 https://github.com/overture-stack/SONG.git  song-docker-1.4.1

Python SDK

overture-song=1.4.1

pip install overture-song==1.4.1

Details

Full Changelog

Fixed bugs:

• fix: Using new exclude-urls param for avoid deadlock #391 (rtisma)

song-1.4.0

Summary

New Features

• Added support for .tgz file types

Data Migration from song-1.3.1

Since a new value for the enum file_type was added, the existing databases must be updated.

--- Current values for the file_type enum
SELECT enum_range(NULL::file_type);

--- Add TGZ to the file_type enum
ALTER TYPE file_type ADD VALUE 'TGZ';

--- Verify TGZ was added
SELECT enum_range(NULL::file_type);

Downloads

Song-Client-1.4.0 Distribution

Song-Server-1.4.0 Distribution

Details

Full Changelog

Fixed bugs:

• Fix outdated links in SONG docs #384
• Internal Server Error when attempting to publish SONG payload in AWS #371

Closed issues:

• Allow inclusion of TAR.GZ fileType in analysis object #388
• Release and deploy Song 1.3.0 #376
• fix circle-ci for song #370
• Preliminary look at converting service to OpenJDK 11 #353

Merged pull requests:

• Adds support for .tgz filetypes as part of analysis. #389 (andricDu)
• Fixed outdated links in SONG docs #385 (b-f-chan)

song-docker-1.3.1

upgraded song to 1.3.1

song-docker-1.3.0

• upgraded song to 1.3.0
• upgraded id server to 5.2.0
• upgraded score server to 1.5.0

song-1.3.1

Summary

Bug Fixes

• disabled auth for schema endpoint

Downloads

Song-Client-1.3.1 Distribution

Song-Server-1.3.1 Distribution

Docker Compose

song-docker-1.3.1

git clone --branch song-docker-1.3.1 https://github.com/overture-stack/SONG.git  song-docker-1.3.1

Python SDK

overture-song=1.3.1

pip install overture-song==1.3.1

Details

Full Changelog

Merged pull requests:

• fix: disabled auth for schema endpoint #377 (rtisma)

song-1.3.0

Summary

Bug Fixes

• validate saveStudy endpoint
• validate that the filename field in the file section of the payload is a basename (no directories)

New Features

• /schema endpoint to retireve schema info for analyses
• absolute pathname now used in manifest outputs for the java client, go client and python sdk
• /unpublish endpoint to unpublish analyses that are published
• added analysisStates url parameter to the GetAnalysisForStudy endpoint to filter by a set of analysisStates
• study is now a requirement in the payload. Reduces accidental uploads to the wrong study
• validate the studyId url parameter of SaveStudy endpoint with the studyId of the payload
• analysisId revoke mechanism. AnalysisIds are committed to the id server only on success

Performance

• improved GetAnalysisForStudy performance. LIRI-JP down from 50min to 2min.

Misc

• fixed vulnerbilities flagged by github
• upgraded dcc-id dependency to 5.2.0

DB Migration

1. Save the following sql script to a file called song-1.3.0-migration.sql

CREATE VIEW FullView AS
SELECT
       A.id as analysis_id,
       A.study_id as study_id,
       A.type as analysis_type,
       A.state as analysis_state,
       I_A.info as analysis_info,
       F.id as file_object_id,
       F.access as file_access,
       F.md5 as file_md5,
       F.name as file_name,
       F.size as file_size,
       F.type as file_type,
       I_F.info as file_info,
       SA.id as sample_id,
       SA.type as sample_type,
       SA.submitter_id as sample_submitter_id,
       I_SA.info as sample_info,
       SP.id as specimen_id,
       SP.type as specimen_type,
       SP.class as specimen_class,
       SP.submitter_id as specimen_submitter_id,
       I_SP.info as specimen_info,
       D.id as donor_id,
       D.gender as donor_gender,
       D.submitter_id as donor_submitter_id,
       I_D.info as donor_info,
       I_SR.info as sequencingread_info,
       I_VC.info as variantcall_info
FROM Analysis A
       left join Info I_A on I_A.id = A.id and I_A.id_type = 'Analysis'
       left join File F on F.analysis_id = A.id
       left join Info I_F on I_F.id = F.id and I_F.id_type = 'File'
       left join SampleSet SS on SS.analysis_id = A.id
       left join Sample SA on SA.id = SS.sample_id
       left join Info I_SA on I_SA.id = SA.id and I_SA.id_type = 'Sample'
       left join Specimen SP on SP.id = SA.specimen_id
       left join Info I_SP on I_SP.id = SP.id and I_SP.id_type = 'Specimen'
       left join Donor D on D.id = SP.donor_id
       left join Info I_D on I_D.id = D.id and I_D.id_type  = 'Donor'
       left join Info I_VC on I_VC.id = A.id and I_VC.id_type = 'VariantCall'
       left join Info I_SR on I_SR.id = A.id and I_SR.id_type = 'SequencingRead';

2. Run the sql script on the existing song database

PGPASSWORD=<dbpasswd>  psql  -h <dbhostname>  -p <dbport>  -U <dbuser>  song < song-1.3.0-migration.sql

3. Test that the migration worked. There should be no error

PGPASSWORD=<dbpasswd>  psql  -h <dbhostname>  -p <dbport>  -U <dbuser>  song -c 'select * from fullview limit 5;'

Downloads

Song-Client-1.3.0 Distribution

Song-Server-1.3.0 Distribution

Docker Compose

song-docker-1.3.0

git clone --branch song-docker-1.3.0 https://github.com/overture-stack/SONG.git  song-docker-1.3.0

Python SDK

overture-song=1.3.0

pip install overture-song==1.3.0

Details

Full Changelog

Implemented enhancements:

• add get-json-schema feature endpoint #373
• Update manifest command in song-go-client to match song-client #349
• Support unpublishing analysis object #346
• Optimize GetAnalysisForStudy to be faster #338
• Assign collab domain and letsencrypt cert to half-song #330
• Update dcc-repository with override flag for indexing UNPUBLISHED data #317
• Add 'study' field in SONG payload and make it required #308
• Request param in saveStudy endpoint not validated with payload #300
• Feature/song 308 add study id payload #352 (rtisma)
• Feature/song 338 get analyses optimize #342 (rtisma)

Fixed bugs:

• COLLAB info id doesnt exist #358
• AWS null pointer for info field error #356
• Fix readthedocs compile error #334
• GetAnalysesForStudy endpoint should only return PUBLISHED analyses #325
• Add analysisId revoke for when save fails but id was already created #321
• Error about Info record for Donor not found #318
• Value for fileName field must contain file basename only #311
• Request param in saveStudy endpoint not validated with payload #300

Closed issues:

• Export endpoint includes analysisState #366
• Add go-client building with maven #363
• Fix vulnerabilities flagged by GitHub #360
• help #359
• Create a HELM package for SONG #354
• Assist in update to XMLs in production #351
• Create SONG sandbox environments #340
• Write documentation about how to deploy an app (song-pdc) on Kubernetes #336
• Create Postgres docker image that has schema built in #335
• Create Half-Song for PDC #316
• Change group name from dcc to overture #271
• New feature requests or adjustments to existing behaviour after first experience using SONG in workflow batch processing #241
• In SONG analysis submission payload, allow user to specify access control at file level #141

Merged pull requests:

• Feature/song 373 json schema endpoint #375 (rtisma)
• chore: Updated requests to 2.20.0 due to vulnerbilities #369 (rtisma)
• fix: Removed analysisState from export ouptut #367 (rtisma)
• Song 321 revoke analysis #364 (rtisma)
• fix: Fixed null pointer issue with info fields #357 (rtisma)
• Update go manifest command #355 (d8660091)
• Support unpulishing analysis object #350 (d8660091)
• song-338-added batching #348 (rtisma)
• Implement new song-server endpoints #4 #345 (d8660091)
• Validate study payload to prevent mismatch study ids #344 (d8660091)
• Move go client into monorepo #343 (d8660091)
• Feature/song 311 file basename only #333 (rtisma)
• added longer default retry for the id client #332 (rtisma)
• Renaming org.icgc.dcc to bio.overture #331 (rtisma)
• Added url param for getAnalyseseByStudy so can filter by analysisState #328 (rtisma)

song-1.2.0

Summary

Bug Fixes

• saving a sample with an existing specimen id

New Features

• automatic database re-connection

Misc

• fixed Legacy controller swagger descriptions

Downloads

Song-Client-1.2.0 Distribution

Song-Server-1.2.0 Distribution

Docker Compose

song-docker-1.2.0

git clone --branch song-docker-1.2.0 https://github.com/overture-stack/SONG.git  song-docker-1.2.0

Python SDK

overture-song=1.2.0

pip install overture-song==1.2.0

Details

Full Changelog

Implemented enhancements:

• Add database reconnection to spring datasource #198
• Feature/fix legacy controller #305 (rtisma)

Fixed bugs:

• Fix corrupted metadata #295
• SONG save error involving ID that already exists #290
• Feature/song 314 fix existing specimen2 #315 (rtisma)

Merged pull requests:

• Feature/add database reconnection to spring database#198 #312 (KevinFHartmann)
• Fix path for example. #309 (mgariepy)
• Updated readme file to include progress bar #306 (kcullion)
• added better checking for generated objectIds and tests #304 (rtisma)
• updated python conf.py and setup.py to use pom.xml version number in release #303 (rtisma)
• updated from 1.0.3 to 1.1.0 #302 (rtisma)

song-1.1.0

Summary

Bug Fixes

• Secured SONG by preventing unauthorized modification of the database

New Features

• New file update endpoint for updating file metadata even after publishing
• Updated the java song-client and python sdk with file update functionality
• Added stricter publishing to prevent score and song from falling out of sync, additionally checking for matching md5 and size information between song and score file (refer to overture-stack/score#80 for more information)

Misc

• standardized and significantly improved SONG documentation, with explainations and tutorials from production environments to docker
• created a successful PoC for protobuf integration

Downloads

Song-Client-1.1.0 Distribution

Song-Server-1.1.0 Distribution

Docker Compose

song-docker-1.1.0

git clone --branch song-docker-1.1.0 https://github.com/overture-stack/SONG.git  song-docker-1.1.0

Python SDK

overture-song=1.1.0

pip install overture-song==1.1.0

Details

Full Changelog

Fixed bugs:

• Unauthorized modification of database #219

Closed issues:

• PoC on protobuf for File entity #286
• Standardize documentation #222

Merged pull requests:

• Feature/add file update clients #301 (rtisma)
• Feature/score 80 stricter publishing #299 (rtisma)
• Feature/score 80 add file update endpoint #298 (rtisma)
• updated song-docs landing page to be introduction #297 (rtisma)
• removed incorrect hyerlinking and accidently added pages #296 (rtisma)
• Feature/song 222 song documentation #293 (rtisma)
• Added Readthedocs badge to readme #291 (Fgerthoffert)
• Feature/song 219 unauthorized modification #289 (rtisma)
• updated song-docker to 1.0.3 #287 (rtisma)

song-docker-1.0.3

Updated db, song-server and song-client to song-1.0.3

song-1.0.3

Summary

Bug Fixes

• Implemented the correct SONG endpoint that is called by the SCORE client when mounting files from Collaboratory
• Added md5 and analysisId contraints at the upload validation step (via JsonSchema) instead of the database level

Misc

• removed unused tumor_sample_submitter_id

Downloads

Song-Client-1.0.3 Distribution

Song-Server-1.0.3 Distribution

Docker Compose

song-docker-1.0.3

git clone --branch song-docker-1.0.3 https://github.com/overture-stack/SONG.git  song-docker-1.0.3

Details

Full Changelog

Implemented enhancements:

• Add error handling for publishing DNE analysisId, and files not uploaded #94

Fixed bugs:

• Unimplemented LegacyEntity endpoint for score mount command #269
• Analysis ID over 36 char fails to upload with 500 error #204
• Fix song-server existence check authorization missing bearer #128

Closed issues:

• Remove tumor_sample_submitter_id from VariantCall schema #230

Merged pull requests:

• Feature/song 269 score mount endpoint #285 (rtisma)
• feat: Updated json schemas for md5 and analysisId #284 (rtisma)
• refactor: Removed tumor_sample_submitter_id from VariantCall table #283 (rtisma)
• test: Added test for publishing non-existent id #282 (rtisma)