diff --git a/_articles/aws_sts_storage_locations.md b/_articles/aws_sts_storage_locations.md
new file mode 100644
index 00000000..23f6759a
--- /dev/null
+++ b/_articles/aws_sts_storage_locations.md
@@ -0,0 +1,209 @@
+---
+title: "Compute Directly on Data in Synapse or S3"
+layout: article
+excerpt: Follow these steps to set up STS Storage Locations and use them to access your S3 data directly.
+category: managing-data
+---
+
+Using AWS Security Token Service (STS), Synapse can securely grant you temporary AWS credentials to access data directly in S3. This can be useful if you want to:
+
+* Download data in bulk
+* Upload data in bulk
+* Allow your compute cluster to read S3 objects using the S3 APIs
+
+All of which you can now do with minimal overhead from Synapse.
+
+There are a few important considerations when determining whether to enable STS on Synapse managed storage compared to external storage with an S3 bucket. With Synapse managed storage, permissions to access are read-only, thus data is only accessible to download or compute on directly once STS is enabled. Alternatively, read-only and read-write permissions can be granted on external storage allowing for data to be manipulated directly with the AWS command line interface. Subsequently, connections to Synapse can be updated if data is changed. In some cases, this workflow is preferable.
+
+{% include note.html content="You can only create STS storage locations on an empty Folder. This is to ensure consistency between the STS storage location and the Synapse Folder. To use STS on existing data, see the Migrating Your Data section below." %}
+
+## Synapse Managed STS Storage Locations
+
+You can create an STS storage location using Synapse storage. Temporary S3 credentials will grant you access to the `Files` and `Folders` scoped to your STS storage location.
+
+{% include note.html content="For Synapse storage, you can request read-only permissions through STS, but not write permissions. Therefore, you can only upload or modify existing Files in Synapse storage through the Synapse website or clients." %}
+
+To set up the STS storage location using Synapse storage, first make sure you have an empty Synapse Folder. Note that you will need write access to that Folder. Then run the following code:
+
+##### Python
+
+```python
+# Set storage location
+import synapseclient
+import json
+syn = synapseclient.login()
+FOLDER = 'syn12345'
+
+destination = {'uploadType':'S3',
+ 'stsEnabled':True,
+ 'concreteType':'org.sagebionetworks.repo.model.project.S3StorageLocationSetting'}
+destination = syn.restPOST('/storageLocation', body=json.dumps(destination))
+
+project_destination ={'concreteType': 'org.sagebionetworks.repo.model.project.UploadDestinationListSetting',
+ 'settingsType': 'upload'}
+project_destination['locations'] = [destination['storageLocationId']]
+project_destination['projectId'] = FOLDER
+
+project_destination = syn.restPOST('/projectSettings', body = json.dumps(project_destination))
+```
+
+##### R
+
+```r
+#set storage location
+library(synapser)
+library(rjson)
+synLogin()
+folderId <- 'syn12345'
+
+destination <- list(uploadType='S3',
+ stsEnabled=TRUE,
+ concreteType='org.sagebionetworks.repo.model.project.S3StorageLocationSetting')
+destination <- synRestPOST('/storageLocation', body=toJSON(destination))
+
+projectDestination <- list(concreteType='org.sagebionetworks.repo.model.project.UploadDestinationListSetting',
+ settingsType='upload')
+projectDestination$locations <- list(destination$storageLocationId)
+projectDestination$projectId <- folderId
+
+projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestination))
+```
+
+Once the Synapse managed STS storage location is set up, you can upload files through the [Synapse](https://www.synapse.org/) or the Synapse client of your choice.
+
+## External STS Storage Locations
+
+You can also create a STS storage location in an external AWS S3 bucket.
+
+There are benefits of creating connections to Synapse from an external bucket. If you already have data stored in S3, or if you have large amounts of data that you want to transfer with the AWS command line interface, you can avoid uploading data to Synapse managed storage by creating connections directly to the S3 bucket. Enabling an STS storage location in the external bucket allows access to the S3 directly for future computing.
+
+Follow the steps in the [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#setting-up-an-external-aws-s3-bucket) article to set read-write or read-only permissions on your external S3 bucket and enable cross-origin resource sharing (CORS). You may use AWS cloudformation for set up.
+
+Again, you will need an empty Synapse Folder, and you will need write access to the Synapse Folder.
+
+Instead of [setting the S3 bucket as upload location]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#set-s3-bucket-as-upload-location), complete set up by running the following code on your Synapse Folder:
+
+{% include important.html content="If a baseKey is not specified, the temporary AWS credentials vended by STS will give users access to the whole bucket. To prevent access to the whole bucket, enter a folder path in your bucket that all files in the storage location should go into as the baseKey. " %}
+
+##### Python
+
+```python
+# Set storage location
+import synapseclient
+import json
+syn = synapseclient.login()
+FOLDER = 'syn12345'
+
+destination = {'uploadType':'S3',
+ 'stsEnabled':True,
+ 'bucket':'nameofyourbucket',
+ 'baseKey':'nameofyourbasekey',
+ 'concreteType':'org.sagebionetworks.repo.model.project.ExternalS3StorageLocationSetting'}
+destination = syn.restPOST('/storageLocation', body=json.dumps(destination))
+
+project_destination ={'concreteType': 'org.sagebionetworks.repo.model.project.UploadDestinationListSetting',
+ 'settingsType': 'upload'}
+project_destination['locations'] = [destination['storageLocationId']]
+project_destination['projectId'] = FOLDER
+
+project_destination = syn.restPOST('/projectSettings', body = json.dumps(project_destination))
+```
+
+##### R
+
+```r
+#set storage location
+library(synapser)
+library(rjson)
+synLogin()
+folderId <- 'syn12345'
+
+destination <- list(uploadType='S3',
+ stsEnabled=TRUE,
+ bucket='nameofyourbucket',
+ baseKey='nameofyourbasekey',
+ concreteType='org.sagebionetworks.repo.model.project.ExternalS3StorageLocationSetting')
+destination <- synRestPOST('/storageLocation', body=toJSON(destination))
+
+projectDestination <- list(concreteType='org.sagebionetworks.repo.model.project.UploadDestinationListSetting',
+ settingsType='upload')
+projectDestination$locations <- list(destination$storageLocationId)
+projectDestination$projectId <- folderId
+
+projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestination))
+```
+
+Once your STS storage location is set up on your Synapse Folder, you can add files through the [Synapse](https://www.synapse.org/) website or the Synapse client of your choice. If you plan to upload files directly to your S3 bucket, or if you already have files in your S3 bucket, you can add representations of those files to Synapse programmatically. Follow the [steps to add files in your S3 bucket to Synapse]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#adding-files-in-your-s3-bucket-to-synapse).
+
+{% include note.html content="Synapse automatically generates Folders for files uploaded through the Synapse website or clients. Files added directly to S3 may not match this Folder structure. We recommend against mixing these two methods of adding files to prevent confusion in the Folder structure." %}
+
+## Obtaining Temporary S3 Credentials
+
+Once your STS storage location is set up, you can use Synapse to request temporary AWS credentials to access your data in S3 directly. These temporary credentials are active for 12 hours.
+
+To get temporary credentials, Python and Java code is provided below. The [REST interface](https://rest-docs.synapse.org/rest/GET/entity/id/sts.html) is also available to request temporary credentials.
+
+
+##### Java
+
+```java
+StsCredentials stsCredentials = synapseClient.getTemporaryCredentialsForEntity(folderEntityId, StsPermission.read_only);
+AWSCredentials awsCredentials = new BasicSessionCredentials(stsCredentials.getAccessKeyId(), stsCredentials.getSecretAccessKey(), stsCredentials.getSessionToken());
+AWSCredentialsProvider awsCredentialsProvider = new AWSStaticCredentialsProvider(awsCredentials);
+AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(awsCredentialsProvider).build();
+```
+
+##### Python
+
+```python
+import synapseclients
+import boto3
+
+syn = synapseclient.login()
+sts_credentials = syn.restGET(f"/entity/{FOLDER}/sts?permission=read_only")
+client = boto3.client(
+ 's3',
+ aws_access_key_id=sts_credentials['accessKeyId'],
+ aws_secret_access_key=sts_credentials['secretAccessKey'],
+ aws_session_token=sts_credentials['sessionToken'],
+)
+
+ent = syn.get("syn12345", downloadFile=False)
+client.download_file(ent._file_handle['bucketName'],
+ ent._file_handle['key'],
+ ent.name)
+# ent._file_handle['bucketName'] -- The name of the Synapse bucket to download from.
+# ent._file_handle['key'] -- The path of the file in the Synapse bucket
+# ent.name -- The path to the file to download to.
+```
+
+## Migrating Existing Data
+
+### From an Existing S3 Bucket
+
+If you have existing data in an S3 bucket, either stand-alone data or data from a previous Synapse `Project`, you can use [this sample code](https://github.com/Sage-Bionetworks/Synapse-Repository-Services/blob/develop/client/sample-code/src/main/java/org/sagebionetworks/sample/sts/MigrateS3Bucket.java) to migrate your S3 data to a new Synapse Folder with a STS storage location.
+
+{% include note.html content="You'll need to create a new Folder with an STS-enabled storage location, as per the instructions above. Additionally, you must be the owner of the storage location in Synapse." %}
+
+### From A Project With Multiple Buckets
+
+If your Synapse Project uses data from multiple S3 buckets, or if the data is in an S3 bucket you don't own, then you may need to download the data and re-upload it a new Synapse Folder with a STS storage location. Use [this sample code](https://github.com/Sage-Bionetworks/Synapse-Repository-Services/blob/develop/client/sample-code/src/main/java/org/sagebionetworks/sample/sts/MigrateSynapseProject.java) to migrate the data in your Project.
+
+{% include note.html content="You'll need to create a new Folder with an STS-enabled storage location, as per the instructions above." %}
+
+{% include note.html content="This method may incur data transfer costs from S3." %}
+
+## Additional Restrictions
+
+* STS storage locations can only be added to Folders, not Projects.
+* A STS storage location can only be added to, or removed from, empty Folders.
+* A STS storage location cannot be added alongside other storage locations.
+* If a parent Folder has an STS storage location, all sub-folders will have the same storage location and ACL as the parent.
+* A Folder with a STS storage location can only contain Files and other Folders.
+* A File or Folder in a STS storage location cannot be moved to a Folder with a different storage location.
+* A File in a STS storage location must have a bucket and base key matching that storage location.
+* If a STS storage location is defined on a Folder, that Folder cannot be placed within another Folder hierarchy that also defines an STS storage location (even if they are the same storage location).
+
+## See Also
+
+[Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#toc-custom-storage-locations)
diff --git a/_articles/challenge_administration.md b/_articles/challenge_administration.md
index dc8be93e..43646fe4 100644
--- a/_articles/challenge_administration.md
+++ b/_articles/challenge_administration.md
@@ -27,7 +27,7 @@ Maintence of both a **Staging** and **Live** `Project` enables `Wiki` content to
{% include important.html content="All edits and changes should be made on the Staging site." %}
-For background on how to create and share `Projects`, `Files`, `Folders` and `Wiki` pages, please see our article [Making a Project](making_a_project.md).
+For background on how to create and share `Projects`, `Files`, `Folders` and `Wiki` pages, please see our article [Making a Project]({{ site.baseurl }}{% link _articles/making_a_project.md %}).
### Synapse Teams
@@ -39,7 +39,7 @@ The command `createchallenge` creates three Synapse `Teams`:
* **Challenge pre-registration team** - This `Team` is recommended for when the challenge is under development. It allows participants to join a mailing list to receive notification of challenge launch news.
-Please visit this [page](teams.md) to learn more about `Teams`.
+Please visit this [page]({{ site.baseurl }}{% link _articles/teams.md %}) to learn more about `Teams`.
### Activating Challenge Configuration
@@ -49,7 +49,7 @@ The command `createchallenge` also connects the **challenge participant team** t
The challenge data (e.g. training dataset, scoring data...) are uploaded to the **Live** challenge `Project` when it is ready to be shared with participants.
-For background on how to create and share `Project`, `Files`, `Folders` and `Wiki` pages, please see our article [Making a Project](making_a_project.md).
+For background on how to create and share `Project`, `Files`, `Folders` and `Wiki` pages, please see our article [Making a Project](%{ link _articles/making_a_project.md %}).
### Adding Conditions for Use
@@ -57,13 +57,13 @@ Synapse has the ability to apply access restrictions to sensitive data (e.g. hum
There are cases where there are no human data concerns and instead a pop-up agreement needs to be presented before the first data download. Contact the **Access and Compliance Team** to set up this agreement.
-Please view the [Access Controls page](access_controls.md) to learn how to add conditions for use on data.
+Please view the [Access Controls page]({{ site.baseurl }}{% link _articles/access_controls.md %}) to learn how to add conditions for use on data.
### Create an Evaluation Queue for Submissions
Challenge participants can submit Synapse Entities (e.g. `File`, `Folder`, `Project`, `Docker`) to evaluation queues. Multiple Evaluation queues can be created to support challenges with more than one question.
-Please visit the [Evaluation Queue article](evaluation_queues.md) to learn more about queue configuration.
+Please visit the [Evaluation Queue article]({{ site.baseurl }}{% link _articles/evaluation_queues.md %}) to learn more about queue configuration.
One of the features of Synapse for DREAM Challenges is the live compilation of submission statistics for all evaluation queues, including total submission count, count per individual/team, count per submission state (scored, invalid) and count per week. You can see the statistics for various challenges [here](https://www.synapse.org/#!Synapse:syn2504723/wiki/65150). In order to activate statistics for your evaluation queues, you must be an administrator of the challenge Project. Each queue needs to be configured to generate the statistics. To do this:
@@ -90,4 +90,4 @@ Throughout the challenge, participants will continuously submit to the evaluatio
Organizers can create a leaderboard when scores are ready to be revealed to participants. Leaderboards are sorted, paginated, tabular forms that display submission annotations (e.g. scores from the scoring application and other metadata) and update as annotations or scores change. A leaderboard can provide real-time insight into the progress of a challenge.
-Learn more about adding leaderboards in the [Evaluation Queue article](evaluation_queues.md).
+Learn more about adding leaderboards in the [Evaluation Queue article]({{ site.baseurl }}{% link _articles/evaluation_queues.md %}).
diff --git a/_articles/challenge_participation.md b/_articles/challenge_participation.md
index ec89a303..f8321c99 100644
--- a/_articles/challenge_participation.md
+++ b/_articles/challenge_participation.md
@@ -18,10 +18,10 @@ This tutorial will teach you the steps of participating in a challenge.
## Challenge Registration
-If you do not have a Synapse account, please go to our [getting started guide](getting_started.md#becoming-a-certified-user) to become a certified Synapse user.
+If you do not have a Synapse account, please go to our [getting started guide]({{ site.baseurl}}{% link _articles/getting_started.md %}#becoming-a-certified-user) to become a certified Synapse user.
Participants **must** be registered for the challenge if they want to submit and participate. The registration button can be found on the home page or `How to Participate` page for every challenge. In order to be fully registered for any challenge, you must have a Synapse account. In addition, DREAM Challenges require that you:
-(1) become a [certified user](getting_started.md#becoming-a-certified-user);
+(1) become a [certified user]({{ site.baseurl}}{% link _articles/getting_started.md %}#becoming-a-certified-user);
(2) agree to the DREAM Rules of participation, and
@@ -29,7 +29,7 @@ Participants **must** be registered for the challenge if they want to submit and
## Join or Create a Team
-We encourage you to form a team with other participants for the challenge. You can either join a team or create your own team of collaborators. See instructions on how to form a team [here](teams.md). It is important to note that you **cannot** be on more than one team. Once you have submitted as a team or individual, you will not be able to submit as another team. If you decide to be part of a team, please register your team to the challenge - there will be a place to do this in every challenge wiki.
+We encourage you to form a team with other participants for the challenge. You can either join a team or create your own team of collaborators. See instructions on how to form a team [here]({{ site.baseurl }}{% link _articles/teams.md %}). It is important to note that you **cannot** be on more than one team. Once you have submitted as a team or individual, you will not be able to submit as another team. If you decide to be part of a team, please register your team to the challenge - there will be a place to do this in every challenge wiki.
## Accessing Challenge Data
@@ -37,11 +37,11 @@ The data stored on the challenge Synapse site can be accessed using the Synapse
## Run your Algorithms and Submit to the Challenge
-You can submit to a challenge queue by using the R, Python or web client. All submissions must be first uploaded to Synapse. Follow these [instructions](getting_started.md#project-and-data-management-on-synapse) to learn how to upload to a project. Most challenge queues will be labeled by `challengename-subchallenge#` as a challenge may have different questions that it may want you to answer. Learn more about submitting to [evaluation queues](evaluation_queues.md)
+You can submit to a challenge queue by using the R, Python or web client. All submissions must be first uploaded to Synapse. Follow these [instructions]({{ site.baseurl }}{% link _articles/getting_started.md %}) to learn how to upload to a project. Most challenge queues will be labeled by `challengename-subchallenge#` as a challenge may have different questions that it may want you to answer. Learn more about submitting to [evaluation queues]({{ site.baseurl }}{% link _articles/evaluation_queues.md %})
## Share Ideas and Ask Questions
Every challenge has a discussion forum for participants (the `Discussion` tab on the Challenge Project page). The forum is a space for participants to ask any questions and raise ideas.
-Instructions on how to use the discussion forum can be found [here](discussion.md)
+Instructions on how to use the discussion forum can be found [here]({{ site.baseurl }}{% link _articles/discussion.md %})
diff --git a/_articles/client_configuration.md b/_articles/client_configuration.md
index 619ec735..9e26e323 100644
--- a/_articles/client_configuration.md
+++ b/_articles/client_configuration.md
@@ -33,4 +33,4 @@ From your [Synapse homepage](https://www.synapse.org/), navigate to your user me
## See Also
-[Getting Started](getting_started.md), [Custom Storage Locations](custom_storage_location.md), [Downloading Data](downloading_data.md)
+[Getting Started]({{ site.baseurl }}{% link _articles/getting_started.md %}), [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}), [Downloading Data]({{ site.baseurl }}{% link _articles/downloading_data.md %})
diff --git a/_articles/contribute_and_access_controlled_use_data.md b/_articles/contribute_and_access_controlled_use_data.md
index 26948154..0b4c571a 100644
--- a/_articles/contribute_and_access_controlled_use_data.md
+++ b/_articles/contribute_and_access_controlled_use_data.md
@@ -14,7 +14,7 @@ You must be a `Certified User` to contribute data to Synapse. You are responsibl
If there are no ethical, legal or regulatory reasons to impose `Conditions for Use`, the data can be used for any lawful research purpose. All human data shared in Synapse, with or without `Conditions for Use`, must be de-identified according to **HIPAA standards** and comply with all applicable privacy laws and regulations. Guidance on de-identification according to HIPAA rules can be found [here](http://www.hhs.gov/ocr/privacy){:target="_blank"}.
-`Conditions for Use` can be set at the `Project`, `Folder`, `File` and `Table` level. We recommend grouping `Files` that require the same `Conditions for Use` in a dedicated `Folder` within your `Project`. For more information about setting `Conditions for Use` please see [Access Controls and Data Governance](access_controls.md).
+`Conditions for Use` can be set at the `Project`, `Folder`, `File` and `Table` level. We recommend grouping `Files` that require the same `Conditions for Use` in a dedicated `Folder` within your `Project`. For more information about setting `Conditions for Use` please see [Access Controls and Data Governance]({{ site.baseurl }}{% link _articles/access_controls.md %}).
@@ -31,7 +31,7 @@ To access `Controlled Use` data you must fulfill the `Conditions for Use` set by
### Bridge Data
Synapse houses data collected through research apps. This data is called Bridge data. This data has been donated by thousands of volunteers
-from around the world. Bridge data is only accessible to Users with a `Validated Profile`. To learn more see [Synapse User Credentials](accounts_certified_users_and_profile_validation.md#validated-profile).
+from around the world. Bridge data is only accessible to Users with a `Validated Profile`. To learn more see [Synapse User Credentials]({{ site.baseurl}}{% link _articles/accounts_certified_users_and_profile_validation.md %}#validated-profile).
{% include warning.html content="Controlled and Bridge data may not be redistributed." %}
Sharing a Synapse account violates the Synapse Terms of Use. Each user wishing to access `Controlled Data` must individually agree to the `Conditions for Use` to access the data. **Please be mindful when sharing information. Do not send data or metadata via email.**
diff --git a/_articles/custom_storage_location.md b/_articles/custom_storage_location.md
index a6848f9a..e485d4ea 100644
--- a/_articles/custom_storage_location.md
+++ b/_articles/custom_storage_location.md
@@ -35,6 +35,10 @@ Make the following adjustments to customize it to work with Synapse:
* Select the newly created bucket and click the **Permissions** tab.
* Select the **Bucket Policy** button and copy one of the below policies (read-only or read-write permissions). Change the name of `Resource` from “synapse-share.yourcompany.com” to the name of your new bucket (twice) and ensure that the `Principal` is `"AWS":"325565585839"`. This is Synapse's account number.
+{% include note.html content="Files in an external bucket will not be automatically added to Synapse." %}
+
+To add files to Synapse that are already in your bucket, see [Adding Files in Your S3 Bucket to Synapse]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#adding-files-in-your-s3-bucket-to-synapse) below.
+
### Read-write permissions
To allow authorized Synapse users to upload data to your bucket set read-write permissions need to be set on that bucket (you allow Synapse to upload and retrieve files):
@@ -58,9 +62,15 @@ To allow authorized Synapse users to upload data to your bucket set read-write p
}
```
-For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating an **[owner.txt](../assets/downloads/owner.txt)** file with your Synapse username and uploading it to your bucket. You can upload the file with the Amazon Web Console or if you have the [AWS command line client](https://aws.amazon.com/cli/), you can upload using the command line.
+For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating a file named **[owner.txt](../assets/downloads/owner.txt)** that contains a **line separated** list of *user identifiers* that are allowed to register and upload to the bucket. Valid *user identifiers* are a numeric Synapse user id or the numeric id of a team the user is a member of.
-
+The id of the user or the team can be obtained by navigating to the user profile or to the team page. The id is the numeric value shown in the browser URL bar after the *Profile:* or *Team:* prefixes:
+
+
+
+
+
+You can upload the file with the Amazon Web Console or the [AWS command line client](https://aws.amazon.com/cli/).
##### Command line
@@ -204,6 +214,7 @@ project_destination = syn.restPOST('/projectSettings', body = json.dumps(project
```r
#set storage location
library(synapser)
+library(rjson)
synLogin()
projectId <- 'syn12345'
@@ -224,7 +235,7 @@ projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestina
Navigate to your **Project/Folder -> Tools -> Change Storage Location**. In the resulting pop-up, select the `Amazon S3 Bucket` option and fill in the relevant information, where Bucket is the name of your external bucket, Base Key is the name of the folder in your bucket to upload to, and Banner is a short description such as who owns the storage location:
-
+
### Adding Files in Your S3 Bucket to Synapse
@@ -263,7 +274,7 @@ fileHandle <- list(concreteType='org.sagebionetworks.repo.model.file.S3FileHandl
storageLocationId = destination$storageLocationId,
bucketName = destination$bucket,
key ='s3ObjectKey')
-fileHandle <- synRestPOST('/externalFileHandle/s3', body=toJSON(fileHandle), endpoint = synapseFileServiceEndpoint())
+fileHandle <- synRestPOST('/externalFileHandle/s3', body=toJSON(fileHandle), endpoint = 'https://file-prod.prod.sagebase.org/file/v1')
f <- File(dataFileHandleId=fileHandle$id, parentId=projectId)
@@ -280,10 +291,21 @@ Make the following adjustments to customize it to work with Synapse:
* Select the newly created bucket and click the **Permissions** tab.
* Select the **Add members** button and enter the member `synapse-svc-prod@uplifted-crow-246820.iam.gserviceaccount.com`. This is Synapse's service account. Give the account the permissions "Storage Legacy Bucket Reader" and "Storage Object Viewer" for read permission. To allow Synapse to upload files, additionally grant the "Storage Legacy Bucket Writer" permission.
-For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating an **owner.txt** file with your Synapse username and uploading it to your bucket. You can upload the file with the Google Cloud Platform Console, or using the command line [gsutil application](https://cloud.google.com/storage/docs/gsutil).
+For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating a file named **[owner.txt](../assets/downloads/owner.txt)** that contains a **line separated** list of *user identifiers* that are allowed to register the bucket and uploading it to your bucket. Valid *user identifiers* are: a Synapse user id or the id of a team the user is a member of.
+
+The id of the user or the team can be obtained navigating to the user profile or to the team page, the id is the numeric value shown in the browser URL bar after the *Profile:* or *Team:* prefixes:
+
+
+
+
+
+You can upload the file with the Google Cloud Platform Console, or using the command line [gsutil application](https://cloud.google.com/storage/docs/gsutil).
+{% include note.html content="Files in an external bucket will not be automatically added to Synapse." %}
+To add files to Synapse that are already in your bucket, see [Adding Files in Your S3 Bucket to Synapse]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#adding-files-in-your-s3-bucket-to-synapse) below.
+
##### Command line
```console
@@ -293,7 +315,7 @@ gsutil cp owner.txt gs://nameofmybucket/nameofmyfolder
##### Web
-Navigate to your bucket on the Google Cloud Console and select the **Upload files** button to upload your text file.
+Navigate to your bucket on the Google Cloud Console and select the **Upload files** button to upload your text file into the folder where you want your data.
### Make sure to enable cross-origin resource sharing (CORS)
@@ -337,91 +359,9 @@ PROJECT = 'syn12345'
destination = {'uploadType':'GOOGLECLOUDSTORAGE',
'concreteType':'org.sagebionetworks.repo.model.project.ExternalGoogleCloudStorageLocationSetting',
- 'bucket':'nameofyourbucket'}
-destination = syn.restPOST('/storageLocation', body=json.dumps(destination))
-
-project_destination ={'concreteType': 'org.sagebionetworks.repo.model.project.UploadDestinationListSetting',
- 'settingsType': 'upload'}
-project_destination['locations'] = [destination['storageLocationId']]
-project_destination['projectId'] = PROJECT
-
-project_destination = syn.restPOST('/projectSettings', body = json.dumps(project_destination))
-```
-
-##### R
-
-Please see the [REST docs](http://rest-docs.synapse.org/rest/org/sagebionetworks/repo/model/project/ExternalS3StorageLocationSetting.html) for more information on setting external storage location settings using our REST API.
-
-## Setting Up an External Google Cloud Storage Bucket
-
-Follow the documentation on Google Cloud's site to **[Create a Bucket](https://cloud.google.com/storage/docs/creating-buckets)**.
-
-Make the following adjustments to customize it to work with Synapse:
-
-* Select the newly created bucket and click the **Permissions** tab.
- * Select the **Add members** button and enter the member `synapse-svc-prod@uplifted-crow-246820.iam.gserviceaccount.com`. This is Synapse's service account. Give the account the permissions "Storage Legacy Bucket Reader" and "Storage Object Viewer" for read permission. To allow Synapse to upload files, additionally grant the "Storage Legacy Bucket Writer" permission.
-
-
-
- -For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating an **owner.txt** file with your Synapse username and uploading it to your bucket. You can upload the file with the Google Cloud Platform Console, or using the [gsutil application](https://cloud.google.com/storage/docs/gsutil), you can upload using the command line. - -
-
-##### Command line
-
-```bash
-# copy your owner.txt file to your s3 bucket
-gsutil cp owner.txt gs://nameofmybucket/nameofmyfolder
-```
-
-##### Web
-
-Navigate to your bucket on the Google Cloud Console and select the **Upload files** button to upload your text file.
-
-
- -### Make sure to enable cross-origin resource sharing (CORS) -Follow the instructions for [Setting CORS on a bucket](https://cloud.google.com/storage/docs/configuring-cors). You may have to insall the [gsutil application](https://cloud.google.com/storage/docs/gsutil). - -Using **gsutil**, you can set the CORS configuration with the command - -``` -gsutil cors set cors-json-file.json gs://example-bucket -``` - -Where cors-json-file.json is a local file that contains a valid CORS configuration, like the configuration below. The configuration must include Synapse as a permitted `origin`. An example CORS configuration that would allow this is: - -```json -[ - { - "maxAgeSeconds": 3000, - "method": ["GET", "POST", "PUT", "HEAD"], - "origin": ["*"], - "responseHeader": ["Content-Type"] - } -] -``` - -
-For more information, please read: [Configuring cross-origin resource sharing (CORS)](https://cloud.google.com/storage/docs/configuring-cors) - -### Set Google Cloud Bucket as Upload Location - -By default, your `Project` uses Synapse storage. You can use the external bucket configured above via our programmatic clients or web client. - -##### Python - -```python -# Set storage location -import synapseclient -import json -syn = synapseclient.login() -PROJECT = 'syn12345' - -destination = {'uploadType':'GOOGLECLOUDSTORAGE', - 'concreteType':'org.sagebionetworks.repo.model.project.ExternalGoogleCloudStorageLocationSetting', - 'bucket':'nameofyourbucket'} + 'bucket':'nameofyourbucket', + 'baseKey': 'nameOfSubfolderInBucket' # optional, only necessary if using a subfolder in your bucket + } destination = syn.restPOST('/storageLocation', body=json.dumps(destination)) project_destination ={'concreteType': 'org.sagebionetworks.repo.model.project.UploadDestinationListSetting', @@ -437,12 +377,16 @@ project_destination = syn.restPOST('/projectSettings', body = json.dumps(project ```r #set storage location library(synapser) +library(rjson) synLogin() projectId <- 'syn12345' destination <- list(uploadType='GOOGLECLOUDSTORAGE', concreteType='org.sagebionetworks.repo.model.project.ExternalGoogleCloudStorageLocationSetting', - bucket='nameofyourbucket') + bucket='nameofyourbucket', + baseKey='nameOfSubfolderInBucket' # optional, only necessary if using a subfolder in your bucket + } +) destination <- synRestPOST('/storageLocation', body=toJSON(destination)) projectDestination <- list(concreteType='org.sagebionetworks.repo.model.project.UploadDestinationListSetting', @@ -457,6 +401,47 @@ projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestina Navigate to your **Project/Folder -> Tools -> Change Storage Location**. In the resulting pop-up, select the **Google Cloud Storage Bucket** option and fill in the relevant information, where Bucket is the name of your external bucket, Base Key is the name of the folder in your bucket to upload to, and Banner is a short description such as who owns the storage location. +### Adding Files in Your Google Cloud Bucket to Synapse + +If your bucket is set for read-write access, files can be added to the bucket using the standard Synapse interface (web or programmatic). + +If the bucket is read-only or you already have content in the bucket, you will have to add representations of the files in Synapse programmatically. This is done using a `FileHandle`, which is a Synapse representation of the file. + +##### Python +```python +externalFileToAdd = 'googleCloudObjectKey' # put the key for the file to add here + +# create filehandle +fileHandle = {'concreteType': 'org.sagebionetworks.repo.model.file.GoogleCloudFileHandle', + 'fileName' : 'nameOfFile.csv', + 'contentSize' : "sizeInBytes", + 'contentType' : 'text/csv', + 'contentMd5' : 'md5', + 'bucketName' : destination['bucket'], + 'key' : externalFileToAdd, + 'storageLocationId': destination['storageLocationId']} +fileHandle = syn.restPOST('/externalFileHandle/googleCloud', json.dumps(fileHandle), endpoint=syn.fileHandleEndpoint) +f = synapseclient.File(parentId=PROJECT, dataFileHandleId = fileHandle['id']) +f = syn.store(f) +``` +##### R +```r +externalFileToAdd <- 'googleCloudObjectKey' # put the key for the file to add here + +# create filehandle +fileHandle <- list(concreteType='org.sagebionetworks.repo.model.file.GoogleCloudFileHandle', + fileName = 'nameOfFile.csv', + contentSize = 'sizeInBytes', + contentType = 'text/csv', + contentMd5 = 'md5', + storageLocationId = destination$storageLocationId, + bucketName = destination$bucket, + key = externalFileToAdd) +fileHandle <- synRestPOST('/externalFileHandle/googleCloud', body=toJSON(fileHandle), endpoint = 'https://file-prod.prod.sagebase.org/file/v1') +f <- File(dataFileHandleId=fileHandle$id, parentId=projectId) +f <- synStore(f) +``` + Please see the [REST docs](http://rest-docs.synapse.org/rest/org/sagebionetworks/repo/model/project/ExternalGoogleCloudStorageLocationSetting.html) for more information on setting external storage location settings using our REST API. ## Using SFTP @@ -612,3 +597,7 @@ projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestina [AWS Console]: https://console.aws.amazon.com/console/ [AWS Cloudformation]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html + +## See Also + +[AWS Security Token Service Storage Locations]({{ site.baseurl }} {% link _articles/aws_sts_storage_locations.md %}) diff --git a/_articles/discussion.md b/_articles/discussion.md index 5eafe78f..6098ffd5 100644 --- a/_articles/discussion.md +++ b/_articles/discussion.md @@ -11,57 +11,43 @@ category: collaboration-and-communication } -`Discussion` forums are a way to easily communicate with others, similar to a message board, which is visible to all those with whom a project is shared. In this guide you will learn how to: +`Discussion` forums are a space to communicate with others, similar to a message board. The forum is visible to users who have access to the `Project`. +
+### Creating a New Message Thread -* Create a new message thread -* Respond to a current thread -* Manage notifications -* Moderate a discussion forum +The Discussion tab can be found in the Project navigation tabs on the Project main page. Anyone with Project access may view discussion threads contained in the Project, create new threads, or reply to existing threads. -## Creating a New Message Thread +
-The `Discussion` tab can be found in the project navigation tabs on your project's main page. Anyone with `Project` access may view discussion threads contained in the project, create new threads, or reply to existing threads.
+To create a new thread, click the **New Thread** button at the top of the Discussion tab.
-
+
-To create a new thread, click the `New Thread` button at the top of the `Discussion` tab.
+This opens a **New Thread** dialog box in which you can edit text, insert pictures, code, TeX, Widgets, or Markdown script. As with `Wiki` editing, a Formatting Guide is available for Markdown, TeX and code block examples.
-
+{% include tip.html content="You may tag people by including an '@' symbol followed by their username. Tagging users in discussion threads will send them a notification of the thread, so it can be used to invite others into the discussion. Tagged users will only receive a notification if they were tagged in the initial thread creation. Tagging users after starting a thread will not send a notification." %}
-This opens a **New Thread** dialog box in which you can edit text, insert pictures, code, TeX, Widgets, or Markdown script. As with `Wiki` editing, a Formatting Guide is available to show you what formatting features are available.
+
+### Managing Notifications -
+You may subscribe to a Discussion forum for a particular Project by clicking the **Follow** button on the main discussion page. Once you've subscribed to a forum, you will receive an email to your Synapse-registered email address whenever a new thread is created or a reply to an existing thread is posted. You can use the **Unfollow** button at any time to discontinue this subscription.
-{% include tip.html content="You may tag people by including an '@' symbol followed by their username; tagging users in discussion threads will send them a notification of the thread, so it can be used to invite others into the discussion." %}
+Additionally, you may change notification settings on an individual thread by using the **eye** icon which is visible from within the thread, or by selecting **Follow Thread** or **Unfollow Thread** from the Tools menu. This can be used to stop receiving reply notifications for that individual thread when you've followed all activity in a discussion forum, or to follow activity in only a select thread.
-## Responding to Existing Threads
+
+
+### Moderating a Discussion Forum -Within an existing thread, you may use the `Reply` button to write a reply to an existing discussion. All editing features described above are available for this purpose. +#### Deleting and Restoring Threads -You may edit or delete any forum post you've created by clicking the **pencil** icon or the **trash can** icon, respectively, located in the bottom right corner of the post. +Users have moderator privileges for discussion forums in all projects to which they have administrator privileges. Moderators may delete threads and individual replies from all users using the **trash can** icon next to the original post. -## Managing Notifications +In case of accidental deletion, moderators may also undelete threads to restore them. To view deleted threads, click on **Discussion Tools** and **Show Deleted Threads** to see a list of threads that are currently in the trash can. Once you've found the thread you'd like to restore, click into that thread and use the **Discussion Tools** Menu to select **Restore Thread**. -You may subscribe to a `Discussion` forum for a particular `Project` by clicking the `Follow` button on the main discussion page. Once you've subscribed to a forum, you will receive an email to your Synapse-registered email address whenever a new thread is created or a reply to an existing thread is posted. You can use the `Unfollow` button at any time to discontinue this subscription. +#### Pinning Threads -Additionally, you may change notification settings on an individual thread by using the **eye** icon which is visible from within the thread, or by selecting "Follow Thread" or "Unfollow Thread" from the Tools menu. This can be used to stop receiving reply notifications for that individual thread when you've followed all activity in a discussion forum, or to follow activity in only a select thread. - -
-
-# Moderating a Discussion Forum
-
-## Deleting and Restoring Threads
-
-Project administrators have moderator privileges for discussion forums in all projects to which they have administrator privileges. Moderators may delete threads and replies from all users using the **trash can** icon next to the original post, which deletes the entire thread, or an individual reply, which deletes only that comment, from inside the thread page.
-
-In case of accidental deletion, moderators may also undelete threads to restore them. To view deleted threads, click on "Show Deleted Threads" from the main forum page to see a list of threads that are currently in the trash can. Once you've found the thread you'd like to restore, click into that thread and then use the Tools Menu and select "Restore Thread".
-
-## Pinning Threads
-
-Moderators may also "pin" important threads using the **pin** icon located next to the original post within the thread page. This causes the thread to be highlighted on the top of the thread list on the `Discussion` main page, and to remain at the top of the list even once newer threads are posted. Moderators may also "unpin" threads.
-
-
+Moderators may emphasize important threads by selecting the **pin** icon to bring the post to the top of the forum. The thread will remain at the top of the list even after newer threads are posted. Moderators may also unpin threads.
# See Also
-[Wikis](wikis.md)
+[Wikis]({{ site.baseurl }}{% link _articles/wikis.md %})
diff --git a/_articles/docker.md b/_articles/docker.md
index 9cde0bd4..bd3b7cd3 100644
--- a/_articles/docker.md
+++ b/_articles/docker.md
@@ -73,10 +73,15 @@ docker images
#docker.synapse.org/syn12345/mytestrepo version1 f8d79ba03c00 6 days ago 126.4 MB
#ubuntu latest f8d79ba03c00 6 days ago 126.4 MB
#docker.synapse.org/syn12345/my-repo latest df323sdf123d 2 days ago 200.3 MB
+
+
docker push docker.synapse.org/syn12345/mytestrepo:version1
+
docker push docker.synapse.org/syn12345/my-repo
```
+Note there is a 100GB limit per repository [image layer](https://docs.docker.com/storage/storagedriver/#images-and-layers).
+
## Using Docker images stored in the Synapse Docker registry
To access the Docker images stored in Synapse, use the `docker pull` command.
@@ -93,7 +98,7 @@ Docker tags can be assigned to later commits. If you want to be explicit about t
docker pull docker.synapse.org/syn12345/mytestrepo@sha256:2e36829f986351042e28242ae386913645a7b41b25844fb39b29af0bdf8dcb63
```
-where the digest for a commit is printed to the command line after a successful Docker push. The Synapse web portal displays current digests for a repository's tags on the repository's Synapse page.
+where the digest for a commit is printed to the command line after a successful `docker push`. The Synapse web portal displays current digests for a repository's tags on the repository's Synapse page.
{% include note.html content="You can add external repositories, i.e. repositories that live in other registries like DockerHub and quay.io. For these repositories there is no tight integration (Synapse doesn't contact these external registries) but it allows you to list Docker repositories that are relevant to the project but are not within Synapse.
" %}
diff --git a/_articles/doi.md b/_articles/doi.md
index 48e896a7..d863a305 100644
--- a/_articles/doi.md
+++ b/_articles/doi.md
@@ -24,7 +24,7 @@ category: reproducible-research
A Digital Object Identifier (DOI) is a persistent identifier assigned to uniquely identify a digital object. A DOI is defined by a digital location like a URL and a description of the object. This description includes attribution and a creation or publication date. Creating and assigning a new DOI is commonly known as "minting". Minting a DOI for things in Synapse allow you to reference them when used elsewhere, such as in a publication or on an external website.
-DOIs are available in Synapse for Projects, Files, Folders, and Tables. DOIs that are for objects stored in Synapse have a prefix of `doi:10.7303`. They are then followed by the Synapse ID of the object being linked to, such as `syn2580853`, which is a Synapse project. The DOI `doi:10.7303/syn2580853` can be represented as a URL, [https://doi.org/10.7303/syn2580853](https://doi.org/10.7303/syn2580853), which automatically redirects to the associated Synapse Project, the [AMP-AD Knowledge Portal](https://www.synapse.org/#!Synapse:syn2580853).
+DOIs are available in Synapse for Projects, Files, Folders, Tables, and Views. DOIs that are for objects stored in Synapse have a prefix of `doi:10.7303`. They are then followed by the Synapse ID of the object being linked to, such as `syn2580853`, which is a Synapse project. The DOI `doi:10.7303/syn2580853` can be represented as a URL, [https://doi.org/10.7303/syn2580853](https://doi.org/10.7303/syn2580853), which automatically redirects to the associated Synapse Project, the [AMP-AD Knowledge Portal](https://www.synapse.org/#!Synapse:syn2580853). If the DOI is being minted for a specific version of something, the URL will have ".xx" appended to it, as that indicates the version.
## Minting DOIs
diff --git a/_articles/downloading_data.md b/_articles/downloading_data.md
index a8212aec..c019b1d2 100644
--- a/_articles/downloading_data.md
+++ b/_articles/downloading_data.md
@@ -86,7 +86,7 @@ entity = syn.get("syn3260973", version=1)
entity <- synGet("syn3260973", version=1)
```
-See [versioning](versioning.md) for more details.
+See [versioning]({{ site.baseurl }}{% link _articles/versioning.md %}) for more details.
### Links
@@ -140,7 +140,7 @@ entity <- synGet("syn00123", downloadLocation="/path/to/folder")
## Finding and Downloading Files
-Files can be [annotated](annotation_and_query.md) to facilitate finding them. In order to search the annotations, a [File View](views.md) must be created first. It is possible to query based on any of the annotations attached to the files.
+Files can be [annotated]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}) to facilitate finding them. In order to search the annotations, a [File View]({{ site.baseurl }}{% link _articles/views.md %}) must be created first. It is possible to query based on any of the annotations attached to the files.
For example, the [PCBC Project](https://www.synapse.org/#!Synapse:syn1773109) has a [table](https://www.synapse.org/#!Synapse:syn7511263) listing sequencing data files that have been annotated. To find all **mRNA fastq** files originating from **CD34+ cells** in the we can query by:
@@ -214,7 +214,7 @@ files = synapseutils.syncFromSynapse(syn, 'syn2390898')
### Download Tables
-Please view [here](tables.md) to learn how to use `Tables`.
+Please view [here]({{ site.baseurl }}{% link _articles/tables.md %}) to learn how to use `Tables`.
### Download Wikis
@@ -282,4 +282,4 @@ all_files = syncFromSynapse(entity='syn123', path='/path/to/myFolder')
# See Also
-[Versioning](versioning.md), [Tables](tables.md), [Wikis](wikis.md), [File Views](views.md), [Annotations and Queries](annotation_and_query.md)
+[Versioning]({{ site.baseurl }}{% link _articles/versioning.md %}), [Tables]({{ site.baseurl }}{% link _articles/tables.md %}), [Wikis]({{ site.baseurl }}{% link _articles/wikis.md %}), [File Views]({{ site.baseurl }}{% link _articles/views.md %}), [Annotations and Queries]({{ site.baseurl }}{% link _articles/annotation_and_query.md %})
diff --git a/_articles/edit_wiki_order.md b/_articles/edit_wiki_order.md
index 2837085e..83723e05 100644
--- a/_articles/edit_wiki_order.md
+++ b/_articles/edit_wiki_order.md
@@ -21,4 +21,4 @@ If you want to create new pages within an existing hierarchy rather than moving
## See Also
-[Wikis](wikis.md)
+[Wikis]({{ site.baseurl }}{% link _articles/wikis.md %})
diff --git a/_articles/evaluation_queues.md b/_articles/evaluation_queues.md
index 54c626b4..91d55e55 100644
--- a/_articles/evaluation_queues.md
+++ b/_articles/evaluation_queues.md
@@ -2,7 +2,7 @@
title: Evaluation Queues
layout: article
-excerpt: An queue accepts submission of Synapse entities for evaluation.
+excerpt: A Synapse Queue contains user submitted models or files to execute for challenges and benchmarking.
category: reproducible-research
---
@@ -10,108 +10,36 @@ An Evaluation queue allows for people to submit Synapse `Files`, `Docker` images
## Create an Evaluation Queue
-To create a queue, you must first create a Synapse `Project`. To create a Synapse Project, follow the instructions on the [Project and Data Management](getting_started.md#making-and-managing-projects-in-synapse) page. An Evaluation queue can take several parameters that you can use to customize your preferences. The minimum requirements to create a queue are:
+To create a queue, you must first create a Synapse `Project` or have [edit permissions]({{ site.baseurl }}{% link _articles/sharing_settings.md %}#edit-permissions) on an existing Project. To create a Synapse Project, follow the instructions on the [Project and Data Management]({{ site.baseurl}}{% link _articles/getting_started.md %}#making-and-managing-projects-in-synapse) page.
-* name – Unique name of the evaluation
-* description – A short description of the evaluation
-* contentSource – Synapse Project associated with the evaluation
-* submissionReceiptMessage – Message to display to users upon submission
-* submissionInstructionsMessage – Message to display to users detailing acceptable formatting for submissions.
+Once you've created your project, navigate to it and add `/challenge` to the url (e.g. www.synapse.org/#!Synapse:syn12345/challenge). Click **Tools** in the right corner and select **Add Evaluation Queue**.
-Optionally, you can restrict how things are submitted by using a quota. An Evaluation queue can only have one quota. If you want to change how long the queue is open, the start date (firstRoundStart), round duration (roundDurationMillis) and number of rounds (nunmberOfRounds) are required parameters. It is optional to set submission limit (submissionLimit).
+
-* firstRoundStart - The date/time at which the first round begins in UTC.
-* roundDurationMillis - The duration of each round in milliseconds.
-* numberOfRounds - The number of rounds, or null if there is no limit to set.
-* submissionLimit - The maximum number of submissions per team/participant per round. Please keep in mind that the system will prevent additional submissions by a user/team once they have hit this number of submissions.
+An Evaluation queue can take several parameters that you can use to customize your preferences.
-{% include note.html content="The name of your evaluation queue MUST be unique, otherwise the queue will not be created." %}
-
-The examples below shows how to create a queue and set the quota using all of the parameters in Python, R and the web client:
-
-##### Python
-
-```python
-import synapseclient
-
-syn = synapseclient.login()
-
-evaluation = synapseclient.Evaluation(name="My Unique Example Challenge Name",
- description="Short description of challenge queue",
- status="OPEN",
- contentSource="syn12345", # Your Synapse Project synID
- submissionInstructionsMessage="Instructions on submission format...",
- submissionReceiptMessage="Thanks for submitting to My Example Challenge!",
- quota={'submissionLimit':3, # The maximum number of submissions per team/participant per round.
- 'firstRoundStart':'2017-11-02T07:00:00.000Z', # The date/time ("%Y-%m-%dT%H:%M:%S%Z") at which the first round begins in UTC.
- 'roundDurationMillis':1645199000, #The duration of each round.
- 'numberOfRounds':1} # The number of rounds, or null if there is no end. (Based on the duration of each round)
-)
-
-syn.store(evaluation)
-```
-
-##### R
-
-```r
-library(synapser)
-
-synLogin()
-
-evaluation <- Evaluation(name="My Unique Example Challenge Name",
- description="Short description of challenge queue",
- status="OPEN",
- contentSource="syn12345", # Your Synapse Project synID
- submissionInstructionsMessage="Instructions on submission format...",
- submissionReceiptMessage="Thanks for submitting to My Example Challenge!",
- quota=c(submissionLimit=3, # The maximum number of submissions per team/participant per round.
- firstRoundStart = '2017-11-02T07:00:00.000Z', # The date/time ("%Y-%m-%dT%H:%M:%S%Z") at which the first round begins in UTC.
- roundDurationMillis = 1645199000, # The duration of each round.
- numberOfRounds=1) # The number of rounds, or null if there is no end. (Based on the duration of each round)
- )
-
-synStore(evaluation)
-```
-
-##### Web
-
-Navigate to your challenge site and add `/admin` to the url (e.g. www.synapse.org/#!Synapse:syn12345/admin). Click **Tools** in the right corner and select **Add Evaluation Queue**.
-
-
+* Name – Unique name of the evaluation
+* Description – A short description of the evaluation
+* Submission Instructions – Message to display to users detailing acceptable formatting for submissions.
+* Submission Receipt Message – Message to display to users upon submission
-In the web client, the quota can be modified under the **Challenge** tab by clicking `Edit`.
-
-## Set a Quota on an Existing Evaluation Queue
-
-The Evaluation ID can be found under the **Challenge** tab of your project. Please note that a Challenge tab will not appear on your project until you have created a Challenge (**Tools > Run Challenge**). In the case below, the Evaluation queue ID is `9610091`.
+{% include note.html content="The name of your evaluation queue MUST be unique, otherwise the queue will not be created." %}
-
-Using the Evaluation ID, we can configure the `quota` parameters of this evaluation queue with the R or Python client.
+### Setting Quotas on an Evaluation Queue
-##### Python
+Optionally, you can restrict how things are submitted by using a quota.
-```python
-import synapseclient
-syn = synapseclient.login()
-evalId = 9610091
-evaluation = syn.getEvaluation(evalId)
-evaluation.quota = {'submissionLimit':3} #The maximum number of submissions per team/participant per round.
-syn.store(evaluation)
-```
+
-##### R
-```r
-library(synapser)
-synLogin()
+An Evaluation queue can only have one quota. You may specify the length of time the queue is open, the start date, round duration, and number of rounds. These are required parameters. It is optional to set submission limit.
-evalId = 9610091
-evaluation <- synGetEvaluation(evalId)
+* First Round Start Date/Time - The date/time at which the first round begins.
+* Number of Rounds - The number of rounds, or null if there is no limit to set.
+* Submission Limit - The maximum number of submissions per team/participant per round. Please keep in mind that the system will prevent additional submissions by a user/team once they have hit this number of submissions.
+* Round Duration - The duration of each round. If you set a round duration you will want to set the number of rounds or it will assume an infinite number of rounds.
-evaluation$quota <- c('submissionLimit'=3) #The maximum number of submissions per team/participant per round.
-synStore(evaluation)
-```
## Share an Evaluation Queue
@@ -126,6 +54,14 @@ To set the sharing settings, go to the **Challenge** tab to view your list of Ev
{% include important.html content="When someone submits to an Evaluation, a copy of the submission is made, so a person with Administrator or Can score access will be able to download the submission even if the submitter deletes the entity." %}
+## Close an Evaluation Queue
+While there isn't technically a way of "closing" an evaluation queue, there are multiple ways to discontinue submissions for users.
+
+* Users are only able to submit to a queue if they have `can submit` permissions to it. If you have the ability to modify the permissions of a queue, you will still be able to submit to the queue due to your `administrator` access.
+* If the quota is set so the current date exceeds the the round start + round duration, no one will be able to submit to the queue. This includes users with administrator permissions.
+* Deleting a queue will also discontinue the ability to submit to it. Be careful when doing this, as deleting a queue is irreversible - you will lose all its submissions.
+
+
## Submitting to an Evaluation Queue
Any Synapse entity may be submitted to an Evaluation Queue.
@@ -175,7 +111,7 @@ Every submission you make to an Evaluation queue has a unique ID. This ID should
Navigate to a file in Synapse and click on **Tools** in the upper right-hand corner. Select **Submit To Challenge** to pick the challenge for your submission. Follow the provided steps to complete your submission.
-
+
## View Submissions of an Evaluation Queue
@@ -214,6 +150,23 @@ If you are happy with your leaderboard configurations, save both the configurati
+
+### Submit to an Evaluation Queue from a Wiki Page
+
+
+
+You may embed a `Submit To Evaluation` widget on a Wiki page to improve visibility of your Evaluation queue. The widget allows participants to submit to multiple Evaluation queues within a Project or a single Evaluation queue.
+
+Currently, this Wiki widget is required to submit Synapse Projects to an Evaluation queue. Synapse Docker repositories can not be submitted through this widget.
+
+
+
+The "Evaluation Queue unavailable message" is customizable. A queue may appear unavailable to a user if:
+
+* The Project doesn't have any Evaluation queues.
+* The user does not have permission to view a Project's Evaluation queues. Learn more about [sharing settings]({{ site.baseurl }}{% link _articles/sharing_settings.md %}).
+* The Evaluation queue quota is set such that a user can not submit to the queue. Learn more about [quotas]({{ site.baseurl }}{% link _articles/evaluation_queues.md%}#create-an-evaluation-queue).
+
# See Also
-To learn how to create a Wiki page, please visit [the Wikis article](wikis.md).
+To learn how to create a Wiki page, please visit [the Wikis article]({{ site.baseurl }}{% link _articles/wikis.md %}).
diff --git a/_articles/faq.md b/_articles/faq.md
index da4d91b5..486268d3 100644
--- a/_articles/faq.md
+++ b/_articles/faq.md
@@ -20,11 +20,11 @@ Synapse allows researchers to share and describe data, analyses, and other conte
#### No, seriously what does Synapse do?
-For a comprehensive introduction to Synapse see our [Getting Started guide](getting_started.md).
+For a comprehensive introduction to Synapse see our [Getting Started guide]({{ site.baseurl }}{% link _articles/getting_started.md %}).
#### How much does it cost to use Synapse?
-We allow groups to get started for free by subsidizing relatively moderate amounts of internal Synapse cloud storage (e.g. 10s of GB). If your needs will exceed this limit we suggest contacting [SynapseInfo@sagebase.org](mailto:SynapseInfo@sagebase.org) for other solutions, including the ability for groups to host their own content either in cloud storage, proxied from [local file servers](custom_storage_location.md) or as external links.
+We allow groups to get started for free by subsidizing relatively moderate amounts of internal Synapse cloud storage (e.g. 10s of GB). If your needs will exceed this limit we suggest contacting [SynapseInfo@sagebase.org](mailto:SynapseInfo@sagebase.org) for other solutions, including the ability for groups to host their own content either in cloud storage, proxied from [local file servers]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}) or as external links.
#### How is Synapse funded?
@@ -44,7 +44,7 @@ Anyone age 13 or older may use Synapse. We have highlighted a series of [researc
#### What are the Synapse Terms of Use?
-The [Terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4) fully describes the governance terms and conditions of Synapse. In order to register on Synapse, you must review and agree to the terms of the Synapse Awareness and Ethics Pledge. For more information see the complete Synapse [Governance policies](governance.md).
+The [Terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4) fully describes the governance terms and conditions of Synapse. In order to register on Synapse, you must review and agree to the terms of the Synapse Awareness and Ethics Pledge. For more information see the complete Synapse [Governance policies]({{ site.baseurl }}{% link _articles/governance.md %}).
#### Is Synapse open source?
@@ -56,7 +56,7 @@ Yes, Synapse is built on top of a RESTful service that is automatically [documen
#### How do I set up my own instance of Synapse?
-Synapse was developed with the philosophy to encourage collaboration across institutional boundaries and is therefore provided as “Software As A Service” with a single instance used by all users. This makes it easy both to discover new content and share with new collaborators. We do support private project spaces where content sharing is controlled by the individual user. In addition, Synapse has the ability to reference resources that are stored elsewhere. This allows Synapse to store metadata about the content such as annotations, descriptive wiki pages and provenance but not the actual data. Currently Synapse has specific support for files stored at URLs, on SFTP servers, on AWS S3 and arbitrary file servers (see: [Custom Storage Locations](custom_storage_location.md)).
+Synapse was developed with the philosophy to encourage collaboration across institutional boundaries and is therefore provided as “Software As A Service” with a single instance used by all users. This makes it easy both to discover new content and share with new collaborators. We do support private project spaces where content sharing is controlled by the individual user. In addition, Synapse has the ability to reference resources that are stored elsewhere. This allows Synapse to store metadata about the content such as annotations, descriptive wiki pages and provenance but not the actual data. Currently Synapse has specific support for files stored at URLs, on SFTP servers, on AWS S3 and arbitrary file servers (see: [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %})).
#### What do I do if I find a bug?
@@ -64,14 +64,14 @@ You may browse open issues or file a bug through our [Jira](https://sagebionetwo
#### How do I get started?
-Whether you are looking to access content or use Synapse to track your own work, start by [registering](https://www.synapse.org/#!RegisterAccount:0) for a Synapse user account. Before uploading files, you will need to take a short [certification quiz](https://www.synapse.org/#!Quiz:Certification) that demonstrates your understanding of the ethical considerations in sharing data and the manner in which data is managed and shared in Synapse. See the [Getting Started guide](getting_started.md) for tips on how to access data, create your own project, and share content with others.
+Whether you are looking to access content or use Synapse to track your own work, start by [registering](https://www.synapse.org/#!RegisterAccount:0) for a Synapse user account. Before uploading files, you will need to take a short [certification quiz](https://www.synapse.org/#!Quiz:Certification) that demonstrates your understanding of the ethical considerations in sharing data and the manner in which data is managed and shared in Synapse. See the [Getting Started guide]({{ site.baseurl }}{% link _articles/getting_started.md %}) for tips on how to access data, create your own project, and share content with others.
# Accessing Content
#### My colleague put some content in Synapse. How do I find it?
This will depend if the content is public or private. If private, you will need to make sure your colleague has shared this content with you. Shared content is visible from your “Dashboard page” under the tab “Shared directly with me”. If you favorite the content (using the yellow star) it will appear under your list of favorites visible from the top of any Synapse page or on your [Profile](https://www.synapse.org/#!Profile:v/projects/favorites).
-All public data is queryable. For more information see [help on querying](annotation_and_query.md) or from the “Search” box in the top right corner of any Synapse page.
+All public data is queryable. For more information see [help on querying]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}) or from the “Search” box in the top right corner of any Synapse page.
#### I have heard Synapse hosts several public datasets. How do I find them?
@@ -79,7 +79,7 @@ Synapse hosts multiple research communities that generate data which is released
#### What does a Synapse account let me do than I can’t do without it?
-You can browse public content in Synapse without registering. However, without an account you cannot add new content to Synapse, nor can you upload or download `Files` or `Tables`. With an account you can create `Projects` and `Wikis`, download `Open Data` and request access to `Controlled Data`. Further, an account lets you collaborate with other Synapse users and create user teams. For more information see the [User Credentials](accounts_certified_users_and_profile_validation.md) page.
+You can browse public content in Synapse without registering. However, without an account you cannot add new content to Synapse, nor can you upload or download `Files` or `Tables`. With an account you can create `Projects` and `Wikis`, download `Open Data` and request access to `Controlled Data`. Further, an account lets you collaborate with other Synapse users and create user teams. For more information see the [User Credentials]({{ site.baseurl }}{% link _articles/accounts_certified_users_and_profile_validation.md %}) page.
#### What is a validated profile?
@@ -89,11 +89,11 @@ Validating your profile is a process where your identity is established through
#### I have my research results - How can Synapse help me share them?
-Synapse makes it easy to share files of any sort, with whomever you choose whether a small group of collaborators or the general public. You may share raw data, summarized data, analysis results, or anything in between. We recommend using [Getting started with Synapse guide](getting_started.md) for more details on how to upload and manage content.
+Synapse makes it easy to share files of any sort, with whomever you choose whether a small group of collaborators or the general public. You may share raw data, summarized data, analysis results, or anything in between. We recommend using [Getting started with Synapse guide]({{ site.baseurl }}{% link _articles/getting_started.md %}) for more details on how to upload and manage content.
#### Why do I have to be a certified user to upload content?
-User certification ensures that you understand your responsibilities for sharing data through Synapse, especially data derived from human participants. These responsibilities include making sure that data derived from human participants is de-identified and that all applicable privacy laws and regulations are observed. See the [How to contribute data](contribute_and_access_controlled_use_data.md) for more information.
+User certification ensures that you understand your responsibilities for sharing data through Synapse, especially data derived from human participants. These responsibilities include making sure that data derived from human participants is de-identified and that all applicable privacy laws and regulations are observed. See the [How to contribute data]({{ site.baseurl }}{% link _articles/contribute_and_access_controlled_use_data.md %}) for more information.
#### How do I get certified?
@@ -101,11 +101,11 @@ To become a certified user, you will need to pass a brief [quiz](https://www.syn
#### Is everything I share on Synapse public?
-No. Use `Sharing settings` to control who can see the content you create. By default, `Projects` and their content are visible only to the user who created it. By using the Synapse `Sharing settings`, you have the ability to grant other Synapse users, Synapse teams, or the public access to your Project content. You can learn more here: [Sharing Settings and Conditions for Use](access_controls.md).
+No. Use `Sharing settings` to control who can see the content you create. By default, `Projects` and their content are visible only to the user who created it. By using the Synapse `Sharing settings`, you have the ability to grant other Synapse users, Synapse teams, or the public access to your Project content. You can learn more here: [Sharing Settings and Conditions for Use]({{ site.baseurl }}{% link _articles/access_controls.md %}).
#### Can I store sensitive information about human subjects in Synapse?
-Yes. Synapse has an IRB-approved data governance procedure that employs `Conditions for Use` to allow for the sharing of sensitive data in a controlled manner. You can learn more by reading our [Sharing Settings and Conditions for Use](access_controls.md) documentation and [Governance documentation](governance.md). If you have questions or would like assistance in applying `Conditions of Use` to your data, please contact the Synapse Access and Compliance Team at [act@sagebase.org](mailto:act@sagebase.org).
+Yes. Synapse has an IRB-approved data governance procedure that employs `Conditions for Use` to allow for the sharing of sensitive data in a controlled manner. You can learn more by reading our [Sharing Settings and Conditions for Use]({{ site.baseurl}}{% link _articles/access_controls.md %}) documentation and [Governance documentation]({{ site.baseurl }}{% link _articles/governance.md %}). If you have questions or would like assistance in applying `Conditions of Use` to your data, please contact the Synapse Access and Compliance Team at [act@sagebase.org](mailto:act@sagebase.org).
#### How do I know the content I put in Synapse will be secure? What security measures does Synapse have?
@@ -113,4 +113,4 @@ Synapse stores content in Amazon Web Services, which provides a layer of securit
#### Where are my files stored?
-By default, Synapse stores files in Amazon Simple Storage Services (S3). However it is possible to set up Synapse to store files in different locations such as your local SFTP server. For files stored outside of S3, Synapse can be used to organize, manage, and access files through the use of Synapse annotations to store file-specific metadata. (see: [Custom Storage Locations](custom_storage_location.md))
+By default, Synapse stores files in Amazon Simple Storage Services (S3). However it is possible to set up Synapse to store files in different locations such as your local SFTP server. For files stored outside of S3, Synapse can be used to organize, manage, and access files through the use of Synapse annotations to store file-specific metadata. (see: [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}))
diff --git a/_articles/files_and_versioning.md b/_articles/files_and_versioning.md
index 074bbbce..b42d27a4 100644
--- a/_articles/files_and_versioning.md
+++ b/_articles/files_and_versioning.md
@@ -17,9 +17,9 @@ category: managing-data
}
-Synapse `Files` can be created by uploading or linking to digital files on the web. They are accessible to anyone who has [access](access_controls.md), can be annotated with custom metadata, can be embedded into Synapse `Wiki` pages, and can be associated with a [DOI](doi.md). `Files` carry the Conditions for Use of the Synapse `Folder` they are placed in, plus any additional specific Conditions for Use they have on their own.
+Synapse `Files` can be created by uploading or linking to digital files on the web. They are accessible to anyone who has [access]({{ site.baseurl }}{% link _articles/access_controls.md %}), can be annotated with custom metadata, can be embedded into Synapse `Wiki` pages, and can be associated with a [DOI]({{ site.baseurl }}{% link _articles/doi.md %}). `Files` carry the Conditions for Use of the Synapse `Folder` they are placed in, plus any additional specific Conditions for Use they have on their own.
-By default, `Files` uploaded to Synapse are stored in 'Synapse Storage', which is freely available to you. `Files` can also be stored on your own Amazon S3 bucket (see [Custom Storage Locations](custom_storage_location.md)) or other custom locations. Furthermore, if you don't want to upload a file (it has external restrictions on sharing, is really large, for example) you can also link to the file. In this way, the file will be accessible through the Synapse clients when you are on the computer that the file is stored, but can be annotated, queried, and documented with a Wiki through Synapse. Lastly, you can provide web-accessible links as Synapse files, which will redirect to that location. All of the same Synapse `File` features are available are available on external links as well.
+By default, `Files` uploaded to Synapse are stored in 'Synapse Storage', which is freely available to you. `Files` can also be stored on your own Amazon S3 bucket (see [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %})) or other custom locations. Furthermore, if you don't want to upload a file (it has external restrictions on sharing, is really large, for example) you can also link to the file. In this way, the file will be accessible through the Synapse clients when you are on the computer that the file is stored, but can be annotated, queried, and documented with a Wiki through Synapse. Lastly, you can provide web-accessible links as Synapse files, which will redirect to that location. All of the same Synapse `File` features are available are available on external links as well.
Synapse `Files` (as well as `Folders` and `Projects`) are identified by a unique identifier called a Synapse ID. It takes the form `syn12345678`. This identifier can be used to refer to a specific file on the web and through the clients.
@@ -63,7 +63,7 @@ file <- synStore(file)
## Moving a File
-All Synapse clients offer a way to move files and folders. Please note that [File Views](views.md) and [sync manifests](uploading_in_bulk.md) **cannot** be used to move files.
+All Synapse clients offer a way to move files and folders. Please note that [File Views]({{ site.baseurl }}{% link _articles/views.md %}) and [sync manifests]({{ site.baseurl }}{% link _articles/uploading_in_bulk.md %}) **cannot** be used to move files.
The command line client has a sub-command `mv` which can be used to move files and folders. The Python client provides the [syn.move](https://python-docs.synapse.org/build/html/Client.html#synapseclient.Synapse.move) command, and the R client has [synMove()](https://r-docs.synapse.org/reference/synMove.html).
@@ -229,7 +229,7 @@ Any change to a `File` will automatically update its version. If this isn't the
##### Web
-Please refer to the [Annotations and Queries](annotation_and_query.md) article for instructions on adding/editing annotations via the web client.
+Please refer to the [Annotations and Queries]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}) article for instructions on adding/editing annotations via the web client.
##### Command line
@@ -241,8 +241,8 @@ synapse set-annotations --id syn56789 --annotations '{"fileType":"bam", "assay":
##### Python
```python
-# Get file from Synapse, set download=False since we are only updating annotations
-file = syn.get('syn56789', download=False)
+# Get file from Synapse, set downloadFile=False since we are only updating annotations
+file = syn.get('syn56789', downloadFile=False)
# Add annotations
file.annotations = {"fileType":"bam", "assay":"RNA-seq"}
# Store the file without creating a new version
@@ -261,7 +261,7 @@ annotations <- synSetAnnotations(file, annotations=list(fileType = "bam", assay
**Setting provenance without changing version**
##### Web
-Please refer to the [Provenance](provenance.md) article for instructions on adding/editing annotations via the web client.
+Please refer to the [Provenance]({{ site.baseurl }}{% link _articles/provenance.md %}) article for instructions on adding/editing annotations via the web client.
##### Command line
@@ -272,8 +272,8 @@ synapse set-provenance -id syn56789 -executed ./path/to/example_code
##### Python
```python
-# Get file from Synapse, set download=False since we are only updating provenance
-file = syn.get('syn56789', download=False)
+# Get file from Synapse, set downloadFile=False since we are only updating provenance
+file = syn.get('syn56789', downloadFile=False)
# Add provenance
file = syn.setProvenance(file, activity = Activity(used = '/path/to/example_code'))
# Store the file without creating a new version
@@ -348,4 +348,4 @@ Some files in Synapse are supported with previews to allow users to peek at the
## See Also
-[Provenance](provenance.md), [Annotations and Queries](annotation_and_query.md), [Downloading Data](downloading_data.md)
+[Provenance]({{ site.baseurl }}{% link _articles/provenance.md %}), [Annotations and Queries]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}), [Downloading Data]({{ site.baseurl }}{% link _articles/downloading_data.md %})
diff --git a/_articles/forms.md b/_articles/forms.md
index 144d5777..e572d4cf 100644
--- a/_articles/forms.md
+++ b/_articles/forms.md
@@ -2,7 +2,7 @@
title: "Forms"
layout: article
excerpt: Embedding Synapse Forms to Collect Data into Tables
-category: how-to
+category: managing-data
---
`Forms` are a way to collect data _into_ a Synapse `Table`. Forms are currently available only under the _alpha_ configuration as they are not yet ready for full release; please use Forms only with this caveat in mind. This document will evolve as additional functionality is added.
@@ -15,7 +15,7 @@ With Forms, you can:
## Creating a Table
-In order to use Forms, you'll need to first create a `Table`. Read more about creating tables here: [Tables](tables.md)
+In order to use Forms, you'll need to first create a `Table`. Read more about creating tables here: [Tables]({{ site.baseurl }}{% link _articles/tables.md %})
## Activating and Deactivating 'Alpha' Mode
@@ -41,8 +41,8 @@ The form is also responsive, so it will render on both large and small screens.
## Allowing Others to Contribute
-Entering data into a `Form` is equivalent to editing the data in a `Table`, so users who wish to add data using the form will need to have "Edit" permissions on the Table. See [Sharing Settings](access_controls.md) for more information on controlling who can edit your Table.
+Entering data into a `Form` is equivalent to editing the data in a `Table`, so users who wish to add data using the form will need to have "Edit" permissions on the Table. See [Sharing Settings]({{ site.baseurl }}{% link _articles/access_controls.md %}) for more information on controlling who can edit your Table.
## See Also
-[Tables](tables.md), [Wikis](wikis.md), [Sharing Settings](access_controls.md)
+[Tables]({{ site.baseurl }}{% link _articles/tables.md %}), [Wikis]({{ site.baseurl }}{% link _articles/wikis.md %}), [Sharing Settings]({{ site.baseurl }}{% link _articles/access_controls.md %})
diff --git a/_articles/getting_started.md b/_articles/getting_started.md
index 4d4acc65..98a358ff 100644
--- a/_articles/getting_started.md
+++ b/_articles/getting_started.md
@@ -5,88 +5,161 @@ excerpt: A getting started guide for new users who are interested in learning ab
order: 1
---
-This guide is for new users who are interested in learning about Synapse. You will learn fundamental Synapse features by performing some common tasks:
+Welcome to Synapse! We are happy to have you as part of the Synapse Community.
-* Create your own `Project` and add content to Synapse
-* Provide a Project description alongside your materials via the Synapse `Wiki` tools
-* Share your work with other Synapse users, `Teams` of users, or the public
+This guide is for new Synapse users and takes 10 minutes to read.
+
+You will learn:
+
+* How to set up your Synapse account.
+* The steps to get started with core Synapse features.
+* This guide will focus on using Synapse on the web. Many researchers also use Synapse with R, Python or Command Line tools. See [Getting Started with Synapse APIs]({{ site.baseurl }}{% link _articles/getting_started_clients.md %}).
## What is Synapse?
+Synapse is a research application developed by [Sage Bionetworks](https://sagebionetworks.org/). Using Synapse, you can find, upload, track, discuss and download datasets and analyses. Synapse supports all kinds of working groups: individuals, small teams, and large consortia.
+For more information, see Introduction to the Sage Platform
+
+## Setting Up Synapse
+
+### Create your account
+Anyone can browse public projects on Synapse without creating an account. However, to download files and create new content, you will need to make an account.
+You can register using an email address, or your Google account.
+
+After registering, you will receive an email to verify your account. You must accept the Terms and Conditions of use to proceed.
+
+
+
+## [Register](https://www.synapse.org/#!RegisterAccount:0)
+
+### Your first time on the Synapse Dashboard
+
+After accepting the Terms and Conditions of use, you will arrive at the `Projects` tab of your Dashboard. Synapse Projects are online workspaces where researchers can collaborate and organize their work.
+
+
+
+All of your Projects are stored here, whether you join an existing one or create a new Project. Data, Wikis and Discussions are contained in each Project. We’ll discuss more about [Projects later on in this guide]({{ site.baseurl }}{% link _articles/getting_started.md %}#projects).
+
+From your dashboard you can also:
+
+* View your Profile
+* See Challenges you are registered for
+* See the Teams you are a member of
+* Access your Downloads list
+* Change your Settings
+
+You can access all of these tabs from the User Menu as well.
+
+
+
+## Governance and Access to Data
+Before exploring Synapse, it is important to get an understanding of Governance and Access Privileges. This will ensure that you are able to use Synapse in an ethical and compliant way.
-Synapse is a collaborative research platform that allows individuals and teams to share, track, and discuss their data and analysis in projects. Synapse is built to work on the web. We provide access to Synapse features and services for programmers through a REST API, [Python client](https://python-docs.synapse.org/build/html/index.html), [command line client](https://python-docs.synapse.org/build/html/CommandLineClient.html), and [R client](https://r-docs.synapse.org/).
+### Governance
+Misuse of human health data can result in severe ethical and legal issues. This means that all Synapse users must comply with privacy and security standards.
+The Governance team at Sage Bionetworks has created policies that maintain these standards. These policies specify Synapse users’ rights and responsibilities. They include conditions for who can use and access data. They also specify the rights and responsibilities of Sage Bionetworks and of the Synapse Access and Compliance Team (ACT).
-Synapse hosts many [research projects](https://www.synapse.org/#!StandaloneWiki:ResearchCommunities) and [resources](https://www.synapse.org/#!StandaloneWiki:OpenResearchProjects). It also hosts crowdsourced competitions, including [DREAM Challenges](http://dreamchallenges.org/). [Sage Bionetworks](http://www.sagebionetworks.org) provides Synapse services free of charge to the scientific community through generous support from various [funding sources](/articles/faq.html#how-is-synapse-funded).
+[Learn more about Governance]({{ site.baseurl }}{% link _articles/governance.md %}).
-## Create Your Account
+[Read the terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4).
-Anyone can browse public content on the Synapse web site. To download and create content, you will need to [register for an account](https://www.synapse.org/register) using an email address. You will receive an email message for verification to complete the registration process.
+[Learn more about Synapse Community Standards]({{ site.baseurl }}{% link _articles/governance.md %}#synapse-community-standards).
-## Getting Certified
-Synapse is a data sharing platform approved for storing data from human subjects research. This requires special care and thought. To upload files, Sage Bionetworks requires you demonstrate awareness of privacy and security issues.
+### User Types will Determine Access to Data
-You can complete this by taking a [Certification Quiz](https://www.synapse.org/#!Quiz:Certification).
+The way in which you will use Synapse, will depend on your own research needs and your Synapse User Type.
-## Making and Managing Projects in Synapse
+There are four User Types in Synapse. Each User Type has a different level of permission.
-Synapse Projects are online workspaces where researchers can collaborate and organize their work. Synapse supports all kinds of working groups: individuals, small teams, and large consortia.
+These are:
+* Anonymous users who do not have a Synapse account
+* Registered users who have created a Synapse account
+* Certified users who have passed the certification quiz
+* Validated users whose profile has been verified
-To create a new Project:
+The table below summarizes the user privileges for each level.
+
+ {:.markdown-table}
+| | Anonymous | Registered | Certified | Validated* |
+| :--------------------------------------------------------: | :---------: | :----------: | :---------: | :----------: |
+| Browse Public Project Catalogue | x | x | x | x |
+| Browse Public File Catalog | x | x | x | x |
+| Create a Project | | x | x | x |
+| Add Wiki Content | | x | x | x |
+| Download Files/Tables* | | x | x | x |
+| Upload Files/Tables | | | x | x |
+| Add Provenance | | | x | x |
+| Requesting access to data collected through research app | | | | x |
-1. Navigate to the User Menu and click on [**Projects**](https://www.synapse.org/#!Profile:v/projects).
-2. Click the **Create a New Project** button.
-3. Decide on a unique name for your Project and click **Save**.
+[Learn more about User Types]().
-Your Projects [dashboard](https://www.synapse.org/#!Profile:v/projects) stores your collection of Projects.
+**Become a Certified User**
-Read about [Projects](making_a_project.md) in the User Guide.
+To upload files, you must become a Certified User. This demonstrates that you have awareness of privacy and security issues. You can become certified at any time by taking a short quiz. If you try to upload files without becoming certified, Synapse will prompt you to take the quiz.
-## Synapse IDs
+After you become a Certified User, you can have your profile Validated. A Validated profile means that your identity has been verified by the Sage Access and Compliance team. This allows you to access more features and data. Profile validation enables greater transparency within the research community.
-Synapse Projects are assigned a Synapse ID, a globally unique identifier used for reference with the format `syn12345678`. Often abbreviated to "synID", the ID of an object never changes, even if the name does. The Synapse ID is always accessible in the URL and visible on the webpage.
+Validation steps include:
+* Filling out your profile
+* Connecting your ORCID,
+* Signing the oath,
+* Providing a recent identity attestation
-## Organizing Content in `Files` and `Folders`
+Learn more about Certified and Validated Accounts.
-Projects contain Files, which can be organized into Folders. Folders and Files also have their own unique Synapse IDs and can be moved within or between Projects. Uploaded files are stored within Synapse storage.
+## Projects
-Use the **Tools Menu** to upload a file:
+Now that you have briefly explored Synapse, it is time to learn a bit more about Projects.
-1. Navigate to the Files tab.
-2. Use the **Files Tools** menu to select **Add New Folder**.
-3. Decide on a Folder name and click **Save**.
-4. Navigate into your new Folder and use the **Folder Tools** menu to select **Upload or Link to a File**.
-5. Use the **Browse** button to select the file, or drag and drop it to upload, and click **Save**.
+### What is a Project?
-To explore other features available for Files and Folders, read about [annotating Files](/articles/annotation_and_query.html), [assigning DOIs](/articles/doi.html), [versioning](/articles/files_and_versioning.html), [`Provenance`](/articles/provenance.html), and [sharing settings](/articles/access_controls.html).
+
-## Adding a Wiki to your Project
+A Project is a workspace that you can use to organize your work and collaborate with others. They act as “containers” and can group related objects such as content and people together.
-A Wiki is a document that can be edited by multiple people on the web. Synapse uses Wikis to provide descriptions of your Project and data.
+Using Projects, you can:
+* Store content such as Data, Code, Results, Figures and Documents
+* Organize your Files and Folders
+* Link and share content with others
+* Create custom, searchable Annotations
+* Create Tables
+* Have discussions using a Forum
+* Create Project Documentation using Wikis
+* Control Access
+* Release your work publicly
-Wiki pages can be written using text, [Markdown](https://www.markdownguide.org/), or basic HTML. Content can include images, tables, code blocks, LaTeX formatted equations, scholarly references, and references to other things in Synapse.
+Any user can create multiple Projects. Each Project can be tailored to a specific workflow, using only the features needed for the Project.
-Use the **Tools** menu to see available Wiki options on Projects, Folders and Files:
+[Learn more about Projects]({{ site.baseurl }}{% link _articles/making_a_project.md %}).
-1. Click the **Tools** button and choose **Edit Wiki**.
- - During editing, you have the option to "Preview" your edits.
-2. Add relevant text, and click **Save**.
+## Synapse ID
-See the [Wiki](/articles/wikis.html) User Guide for more information and examples.
+On the top left of your Project, you’ll notice a Synapse ID with the format “syn12345678”. Everything that is created in Synapse has a unique Synapse ID. The Synapse ID is often abbreviated to “synID”. The ID of an object never changes, even if the name does.
-## Sharing and Teams
+
-The default setting for new Projects in Synapse is **private**. As a project owner, you choose who to share with, and how. You can find Project sharing settings under the **Project Settings** menu. Permissions or sharing settings at the Project level are automatically inherited by all Files and Folders within the Project. If needed, you can **Create Local Sharing Settings** to make certain Files or Folders have permissions that differ from the parent Project.
+The synID is always accessible in the URL and visible on the webpage. It is used for reference to any object in Synapse.
-Groups of users can form Teams for collaboration. Teams can be used for permissions and for communication. Sharing things with teams instead of individual users can provide simiplicity for administrators.
+## What's Next?
+Now that you have a basic understanding of how to start using Synapse you can learn more about how to use the core features with these tutorials below.
-For more details, visit the [Sharing Settings](/articles/access_controls.html) and [Teams](/articles/teams.html) User Guide.
+* I want to create a new project
+* I want to upload my data and share it
+* I want to download existing data
+* I want to organize files in a project
+* I want to create a New team
+* I want to download research data from mobile studies
+* I want use Synapse programmatically
-## More Guides
+## Advanced Topics
-Find additional information in our [User Guide](/articles).
+### Practicing Version Control
+All content in Synapse is versioned automatically, with the ability to provide version comments on files, and to link content together with rich provenance relationships. This is particularly useful for groups leveraging Synapse as a component of a data processing pipeline, or citing data for publications, as Synapse allows data contributors to mint DOIs for most content. Learn more versioning content, creating and managing provenance, and how to mint DOIs.
-Read about Synapse [governance](/articles/governance.html) and [Terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4).
+### Custom Metadata
+Synapse provides a number of default metadata fields that automatically annotate content with useful information, but also offers the ability for users to define their own metadata key-value pairs to provide context to their data. These annotations can be viewed and managed at the individual file level, or by creating “views” that show annotations on all content across a larger scope such as a project or folder. Learn more about annotations and views.
-For information on using Synapse programmatically, see the documentation for the [Python client](https://python-docs.synapse.org/build/html/index.html), [command line client](https://python-docs.synapse.org/build/html/CommandLineClient.html), and [R client](https://r-docs.synapse.org/)
+### Search
+Leverage Synapse global search to discover data, projects, or even potential collaborators! Search is permission-dependent, meaning it works across all public content and content that you have access to, if logged in. Learn more about Searching.
\ No newline at end of file
diff --git a/_articles/getting_started_clients.md b/_articles/getting_started_clients.md
index bda44002..707e9dee 100644
--- a/_articles/getting_started_clients.md
+++ b/_articles/getting_started_clients.md
@@ -12,7 +12,7 @@ The API clients provide a way to use Synapse programmatically. This page shows y
* Python
* R
-To manage stored login credentials, visit the [Client Configuration page](client_configuration.md).
+To manage stored login credentials, visit the [Client Configuration page]({{ site.baseurl }}{% link _articles/client_configuration.md %}).
## Command Line
@@ -27,8 +27,8 @@ pip install synapseclient
```
```console
-synapseclient login "username" "password"
-synapseclient -h
+synapse login "username" "password"
+synapse -h
```
For more documentation on the command line client, see the [`synapseclient` docs](https://python-docs.synapse.org/build/html/CommandLineClient.html).
diff --git a/_articles/index.md b/_articles/index.md
index 8d9f9bf4..f7e0a3eb 100644
--- a/_articles/index.md
+++ b/_articles/index.md
@@ -1,5 +1,5 @@
---
-title: Articles
+title: Overview
layout: index
description: These are the main instructions for using Synapse. Pick an section that interests you or read them in order.
---
diff --git a/_articles/links.md b/_articles/links.md
index bc30640b..8a423dac 100644
--- a/_articles/links.md
+++ b/_articles/links.md
@@ -2,7 +2,7 @@
title: Links
layout: article
excerpt: Creating, and modifying links on Synapse.
-category: how-to
+category: managing-data
---
+The id of the user or the team can be obtained by navigating to the user profile or to the team page. The id is the numeric value shown in the browser URL bar after the *Profile:* or *Team:* prefixes:
+
+
+
+
+
+You can upload the file with the Amazon Web Console or the [AWS command line client](https://aws.amazon.com/cli/).
##### Command line
@@ -204,6 +214,7 @@ project_destination = syn.restPOST('/projectSettings', body = json.dumps(project
```r
#set storage location
library(synapser)
+library(rjson)
synLogin()
projectId <- 'syn12345'
@@ -224,7 +235,7 @@ projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestina
Navigate to your **Project/Folder -> Tools -> Change Storage Location**. In the resulting pop-up, select the `Amazon S3 Bucket` option and fill in the relevant information, where Bucket is the name of your external bucket, Base Key is the name of the folder in your bucket to upload to, and Banner is a short description such as who owns the storage location:
-
+
### Adding Files in Your S3 Bucket to Synapse
@@ -263,7 +274,7 @@ fileHandle <- list(concreteType='org.sagebionetworks.repo.model.file.S3FileHandl
storageLocationId = destination$storageLocationId,
bucketName = destination$bucket,
key ='s3ObjectKey')
-fileHandle <- synRestPOST('/externalFileHandle/s3', body=toJSON(fileHandle), endpoint = synapseFileServiceEndpoint())
+fileHandle <- synRestPOST('/externalFileHandle/s3', body=toJSON(fileHandle), endpoint = 'https://file-prod.prod.sagebase.org/file/v1')
f <- File(dataFileHandleId=fileHandle$id, parentId=projectId)
@@ -280,10 +291,21 @@ Make the following adjustments to customize it to work with Synapse:
* Select the newly created bucket and click the **Permissions** tab.
* Select the **Add members** button and enter the member `synapse-svc-prod@uplifted-crow-246820.iam.gserviceaccount.com`. This is Synapse's service account. Give the account the permissions "Storage Legacy Bucket Reader" and "Storage Object Viewer" for read permission. To allow Synapse to upload files, additionally grant the "Storage Legacy Bucket Writer" permission.
-For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating an **owner.txt** file with your Synapse username and uploading it to your bucket. You can upload the file with the Google Cloud Platform Console, or using the command line [gsutil application](https://cloud.google.com/storage/docs/gsutil).
+For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating a file named **[owner.txt](../assets/downloads/owner.txt)** that contains a **line separated** list of *user identifiers* that are allowed to register the bucket and uploading it to your bucket. Valid *user identifiers* are: a Synapse user id or the id of a team the user is a member of.
+
+The id of the user or the team can be obtained navigating to the user profile or to the team page, the id is the numeric value shown in the browser URL bar after the *Profile:* or *Team:* prefixes:
+
+
+
+
+
+You can upload the file with the Google Cloud Platform Console, or using the command line [gsutil application](https://cloud.google.com/storage/docs/gsutil).
+{% include note.html content="Files in an external bucket will not be automatically added to Synapse." %}
+To add files to Synapse that are already in your bucket, see [Adding Files in Your S3 Bucket to Synapse]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}#adding-files-in-your-s3-bucket-to-synapse) below.
+
##### Command line
```console
@@ -293,7 +315,7 @@ gsutil cp owner.txt gs://nameofmybucket/nameofmyfolder
##### Web
-Navigate to your bucket on the Google Cloud Console and select the **Upload files** button to upload your text file.
+Navigate to your bucket on the Google Cloud Console and select the **Upload files** button to upload your text file into the folder where you want your data.
### Make sure to enable cross-origin resource sharing (CORS)
@@ -337,91 +359,9 @@ PROJECT = 'syn12345'
destination = {'uploadType':'GOOGLECLOUDSTORAGE',
'concreteType':'org.sagebionetworks.repo.model.project.ExternalGoogleCloudStorageLocationSetting',
- 'bucket':'nameofyourbucket'}
-destination = syn.restPOST('/storageLocation', body=json.dumps(destination))
-
-project_destination ={'concreteType': 'org.sagebionetworks.repo.model.project.UploadDestinationListSetting',
- 'settingsType': 'upload'}
-project_destination['locations'] = [destination['storageLocationId']]
-project_destination['projectId'] = PROJECT
-
-project_destination = syn.restPOST('/projectSettings', body = json.dumps(project_destination))
-```
-
-##### R
-
-Please see the [REST docs](http://rest-docs.synapse.org/rest/org/sagebionetworks/repo/model/project/ExternalS3StorageLocationSetting.html) for more information on setting external storage location settings using our REST API.
-
-## Setting Up an External Google Cloud Storage Bucket
-
-Follow the documentation on Google Cloud's site to **[Create a Bucket](https://cloud.google.com/storage/docs/creating-buckets)**.
-
-Make the following adjustments to customize it to work with Synapse:
-
-* Select the newly created bucket and click the **Permissions** tab.
- * Select the **Add members** button and enter the member `synapse-svc-prod@uplifted-crow-246820.iam.gserviceaccount.com`. This is Synapse's service account. Give the account the permissions "Storage Legacy Bucket Reader" and "Storage Object Viewer" for read permission. To allow Synapse to upload files, additionally grant the "Storage Legacy Bucket Writer" permission.
-
-
-- -For **read-write** permissions, you also need to create an object that proves to the Synapse service that you own this bucket. This can be done by creating an **owner.txt** file with your Synapse username and uploading it to your bucket. You can upload the file with the Google Cloud Platform Console, or using the [gsutil application](https://cloud.google.com/storage/docs/gsutil), you can upload using the command line. - -
-
-##### Command line
-
-```bash
-# copy your owner.txt file to your s3 bucket
-gsutil cp owner.txt gs://nameofmybucket/nameofmyfolder
-```
-
-##### Web
-
-Navigate to your bucket on the Google Cloud Console and select the **Upload files** button to upload your text file.
-
-- -### Make sure to enable cross-origin resource sharing (CORS) -Follow the instructions for [Setting CORS on a bucket](https://cloud.google.com/storage/docs/configuring-cors). You may have to insall the [gsutil application](https://cloud.google.com/storage/docs/gsutil). - -Using **gsutil**, you can set the CORS configuration with the command - -``` -gsutil cors set cors-json-file.json gs://example-bucket -``` - -Where cors-json-file.json is a local file that contains a valid CORS configuration, like the configuration below. The configuration must include Synapse as a permitted `origin`. An example CORS configuration that would allow this is: - -```json -[ - { - "maxAgeSeconds": 3000, - "method": ["GET", "POST", "PUT", "HEAD"], - "origin": ["*"], - "responseHeader": ["Content-Type"] - } -] -``` - -
-For more information, please read: [Configuring cross-origin resource sharing (CORS)](https://cloud.google.com/storage/docs/configuring-cors) - -### Set Google Cloud Bucket as Upload Location - -By default, your `Project` uses Synapse storage. You can use the external bucket configured above via our programmatic clients or web client. - -##### Python - -```python -# Set storage location -import synapseclient -import json -syn = synapseclient.login() -PROJECT = 'syn12345' - -destination = {'uploadType':'GOOGLECLOUDSTORAGE', - 'concreteType':'org.sagebionetworks.repo.model.project.ExternalGoogleCloudStorageLocationSetting', - 'bucket':'nameofyourbucket'} + 'bucket':'nameofyourbucket', + 'baseKey': 'nameOfSubfolderInBucket' # optional, only necessary if using a subfolder in your bucket + } destination = syn.restPOST('/storageLocation', body=json.dumps(destination)) project_destination ={'concreteType': 'org.sagebionetworks.repo.model.project.UploadDestinationListSetting', @@ -437,12 +377,16 @@ project_destination = syn.restPOST('/projectSettings', body = json.dumps(project ```r #set storage location library(synapser) +library(rjson) synLogin() projectId <- 'syn12345' destination <- list(uploadType='GOOGLECLOUDSTORAGE', concreteType='org.sagebionetworks.repo.model.project.ExternalGoogleCloudStorageLocationSetting', - bucket='nameofyourbucket') + bucket='nameofyourbucket', + baseKey='nameOfSubfolderInBucket' # optional, only necessary if using a subfolder in your bucket + } +) destination <- synRestPOST('/storageLocation', body=toJSON(destination)) projectDestination <- list(concreteType='org.sagebionetworks.repo.model.project.UploadDestinationListSetting', @@ -457,6 +401,47 @@ projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestina Navigate to your **Project/Folder -> Tools -> Change Storage Location**. In the resulting pop-up, select the **Google Cloud Storage Bucket** option and fill in the relevant information, where Bucket is the name of your external bucket, Base Key is the name of the folder in your bucket to upload to, and Banner is a short description such as who owns the storage location. +### Adding Files in Your Google Cloud Bucket to Synapse + +If your bucket is set for read-write access, files can be added to the bucket using the standard Synapse interface (web or programmatic). + +If the bucket is read-only or you already have content in the bucket, you will have to add representations of the files in Synapse programmatically. This is done using a `FileHandle`, which is a Synapse representation of the file. + +##### Python +```python +externalFileToAdd = 'googleCloudObjectKey' # put the key for the file to add here + +# create filehandle +fileHandle = {'concreteType': 'org.sagebionetworks.repo.model.file.GoogleCloudFileHandle', + 'fileName' : 'nameOfFile.csv', + 'contentSize' : "sizeInBytes", + 'contentType' : 'text/csv', + 'contentMd5' : 'md5', + 'bucketName' : destination['bucket'], + 'key' : externalFileToAdd, + 'storageLocationId': destination['storageLocationId']} +fileHandle = syn.restPOST('/externalFileHandle/googleCloud', json.dumps(fileHandle), endpoint=syn.fileHandleEndpoint) +f = synapseclient.File(parentId=PROJECT, dataFileHandleId = fileHandle['id']) +f = syn.store(f) +``` +##### R +```r +externalFileToAdd <- 'googleCloudObjectKey' # put the key for the file to add here + +# create filehandle +fileHandle <- list(concreteType='org.sagebionetworks.repo.model.file.GoogleCloudFileHandle', + fileName = 'nameOfFile.csv', + contentSize = 'sizeInBytes', + contentType = 'text/csv', + contentMd5 = 'md5', + storageLocationId = destination$storageLocationId, + bucketName = destination$bucket, + key = externalFileToAdd) +fileHandle <- synRestPOST('/externalFileHandle/googleCloud', body=toJSON(fileHandle), endpoint = 'https://file-prod.prod.sagebase.org/file/v1') +f <- File(dataFileHandleId=fileHandle$id, parentId=projectId) +f <- synStore(f) +``` + Please see the [REST docs](http://rest-docs.synapse.org/rest/org/sagebionetworks/repo/model/project/ExternalGoogleCloudStorageLocationSetting.html) for more information on setting external storage location settings using our REST API. ## Using SFTP @@ -612,3 +597,7 @@ projectDestination <- synRestPOST('/projectSettings', body=toJSON(projectDestina [AWS Console]: https://console.aws.amazon.com/console/ [AWS Cloudformation]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html + +## See Also + +[AWS Security Token Service Storage Locations]({{ site.baseurl }} {% link _articles/aws_sts_storage_locations.md %}) diff --git a/_articles/discussion.md b/_articles/discussion.md index 5eafe78f..6098ffd5 100644 --- a/_articles/discussion.md +++ b/_articles/discussion.md @@ -11,57 +11,43 @@ category: collaboration-and-communication } -`Discussion` forums are a way to easily communicate with others, similar to a message board, which is visible to all those with whom a project is shared. In this guide you will learn how to: +`Discussion` forums are a space to communicate with others, similar to a message board. The forum is visible to users who have access to the `Project`. +
+### Creating a New Message Thread -* Create a new message thread -* Respond to a current thread -* Manage notifications -* Moderate a discussion forum +The Discussion tab can be found in the Project navigation tabs on the Project main page. Anyone with Project access may view discussion threads contained in the Project, create new threads, or reply to existing threads. -## Creating a New Message Thread +
-The `Discussion` tab can be found in the project navigation tabs on your project's main page. Anyone with `Project` access may view discussion threads contained in the project, create new threads, or reply to existing threads.
+To create a new thread, click the **New Thread** button at the top of the Discussion tab.
-
+
-To create a new thread, click the `New Thread` button at the top of the `Discussion` tab.
+This opens a **New Thread** dialog box in which you can edit text, insert pictures, code, TeX, Widgets, or Markdown script. As with `Wiki` editing, a Formatting Guide is available for Markdown, TeX and code block examples.
-
+{% include tip.html content="You may tag people by including an '@' symbol followed by their username. Tagging users in discussion threads will send them a notification of the thread, so it can be used to invite others into the discussion. Tagged users will only receive a notification if they were tagged in the initial thread creation. Tagging users after starting a thread will not send a notification." %}
-This opens a **New Thread** dialog box in which you can edit text, insert pictures, code, TeX, Widgets, or Markdown script. As with `Wiki` editing, a Formatting Guide is available to show you what formatting features are available.
++### Managing Notifications -
+You may subscribe to a Discussion forum for a particular Project by clicking the **Follow** button on the main discussion page. Once you've subscribed to a forum, you will receive an email to your Synapse-registered email address whenever a new thread is created or a reply to an existing thread is posted. You can use the **Unfollow** button at any time to discontinue this subscription.
-{% include tip.html content="You may tag people by including an '@' symbol followed by their username; tagging users in discussion threads will send them a notification of the thread, so it can be used to invite others into the discussion." %}
+Additionally, you may change notification settings on an individual thread by using the **eye** icon which is visible from within the thread, or by selecting **Follow Thread** or **Unfollow Thread** from the Tools menu. This can be used to stop receiving reply notifications for that individual thread when you've followed all activity in a discussion forum, or to follow activity in only a select thread.
-## Responding to Existing Threads
+
++### Moderating a Discussion Forum -Within an existing thread, you may use the `Reply` button to write a reply to an existing discussion. All editing features described above are available for this purpose. +#### Deleting and Restoring Threads -You may edit or delete any forum post you've created by clicking the **pencil** icon or the **trash can** icon, respectively, located in the bottom right corner of the post. +Users have moderator privileges for discussion forums in all projects to which they have administrator privileges. Moderators may delete threads and individual replies from all users using the **trash can** icon next to the original post. -## Managing Notifications +In case of accidental deletion, moderators may also undelete threads to restore them. To view deleted threads, click on **Discussion Tools** and **Show Deleted Threads** to see a list of threads that are currently in the trash can. Once you've found the thread you'd like to restore, click into that thread and use the **Discussion Tools** Menu to select **Restore Thread**. -You may subscribe to a `Discussion` forum for a particular `Project` by clicking the `Follow` button on the main discussion page. Once you've subscribed to a forum, you will receive an email to your Synapse-registered email address whenever a new thread is created or a reply to an existing thread is posted. You can use the `Unfollow` button at any time to discontinue this subscription. +#### Pinning Threads -Additionally, you may change notification settings on an individual thread by using the **eye** icon which is visible from within the thread, or by selecting "Follow Thread" or "Unfollow Thread" from the Tools menu. This can be used to stop receiving reply notifications for that individual thread when you've followed all activity in a discussion forum, or to follow activity in only a select thread. - -
-
-# Moderating a Discussion Forum
-
-## Deleting and Restoring Threads
-
-Project administrators have moderator privileges for discussion forums in all projects to which they have administrator privileges. Moderators may delete threads and replies from all users using the **trash can** icon next to the original post, which deletes the entire thread, or an individual reply, which deletes only that comment, from inside the thread page.
-
-In case of accidental deletion, moderators may also undelete threads to restore them. To view deleted threads, click on "Show Deleted Threads" from the main forum page to see a list of threads that are currently in the trash can. Once you've found the thread you'd like to restore, click into that thread and then use the Tools Menu and select "Restore Thread".
-
-## Pinning Threads
-
-Moderators may also "pin" important threads using the **pin** icon located next to the original post within the thread page. This causes the thread to be highlighted on the top of the thread list on the `Discussion` main page, and to remain at the top of the list even once newer threads are posted. Moderators may also "unpin" threads.
-
-
+Moderators may emphasize important threads by selecting the **pin** icon to bring the post to the top of the forum. The thread will remain at the top of the list even after newer threads are posted. Moderators may also unpin threads.
# See Also
-[Wikis](wikis.md)
+[Wikis]({{ site.baseurl }}{% link _articles/wikis.md %})
diff --git a/_articles/docker.md b/_articles/docker.md
index 9cde0bd4..bd3b7cd3 100644
--- a/_articles/docker.md
+++ b/_articles/docker.md
@@ -73,10 +73,15 @@ docker images
#docker.synapse.org/syn12345/mytestrepo version1 f8d79ba03c00 6 days ago 126.4 MB
#ubuntu latest f8d79ba03c00 6 days ago 126.4 MB
#docker.synapse.org/syn12345/my-repo latest df323sdf123d 2 days ago 200.3 MB
+
+
docker push docker.synapse.org/syn12345/mytestrepo:version1
+
docker push docker.synapse.org/syn12345/my-repo
```
+Note there is a 100GB limit per repository [image layer](https://docs.docker.com/storage/storagedriver/#images-and-layers).
+
## Using Docker images stored in the Synapse Docker registry
To access the Docker images stored in Synapse, use the `docker pull` command.
@@ -93,7 +98,7 @@ Docker tags can be assigned to later commits. If you want to be explicit about t
docker pull docker.synapse.org/syn12345/mytestrepo@sha256:2e36829f986351042e28242ae386913645a7b41b25844fb39b29af0bdf8dcb63
```
-where the digest for a commit is printed to the command line after a successful Docker push. The Synapse web portal displays current digests for a repository's tags on the repository's Synapse page.
+where the digest for a commit is printed to the command line after a successful `docker push`. The Synapse web portal displays current digests for a repository's tags on the repository's Synapse page.
{% include note.html content="You can add external repositories, i.e. repositories that live in other registries like DockerHub and quay.io. For these repositories there is no tight integration (Synapse doesn't contact these external registries) but it allows you to list Docker repositories that are relevant to the project but are not within Synapse.
" %}
diff --git a/_articles/doi.md b/_articles/doi.md
index 48e896a7..d863a305 100644
--- a/_articles/doi.md
+++ b/_articles/doi.md
@@ -24,7 +24,7 @@ category: reproducible-research
A Digital Object Identifier (DOI) is a persistent identifier assigned to uniquely identify a digital object. A DOI is defined by a digital location like a URL and a description of the object. This description includes attribution and a creation or publication date. Creating and assigning a new DOI is commonly known as "minting". Minting a DOI for things in Synapse allow you to reference them when used elsewhere, such as in a publication or on an external website.
-DOIs are available in Synapse for Projects, Files, Folders, and Tables. DOIs that are for objects stored in Synapse have a prefix of `doi:10.7303`. They are then followed by the Synapse ID of the object being linked to, such as `syn2580853`, which is a Synapse project. The DOI `doi:10.7303/syn2580853` can be represented as a URL, [https://doi.org/10.7303/syn2580853](https://doi.org/10.7303/syn2580853), which automatically redirects to the associated Synapse Project, the [AMP-AD Knowledge Portal](https://www.synapse.org/#!Synapse:syn2580853).
+DOIs are available in Synapse for Projects, Files, Folders, Tables, and Views. DOIs that are for objects stored in Synapse have a prefix of `doi:10.7303`. They are then followed by the Synapse ID of the object being linked to, such as `syn2580853`, which is a Synapse project. The DOI `doi:10.7303/syn2580853` can be represented as a URL, [https://doi.org/10.7303/syn2580853](https://doi.org/10.7303/syn2580853), which automatically redirects to the associated Synapse Project, the [AMP-AD Knowledge Portal](https://www.synapse.org/#!Synapse:syn2580853). If the DOI is being minted for a specific version of something, the URL will have ".xx" appended to it, as that indicates the version.
## Minting DOIs
diff --git a/_articles/downloading_data.md b/_articles/downloading_data.md
index a8212aec..c019b1d2 100644
--- a/_articles/downloading_data.md
+++ b/_articles/downloading_data.md
@@ -86,7 +86,7 @@ entity = syn.get("syn3260973", version=1)
entity <- synGet("syn3260973", version=1)
```
-See [versioning](versioning.md) for more details.
+See [versioning]({{ site.baseurl }}{% link _articles/versioning.md %}) for more details.
### Links
@@ -140,7 +140,7 @@ entity <- synGet("syn00123", downloadLocation="/path/to/folder")
## Finding and Downloading Files
-Files can be [annotated](annotation_and_query.md) to facilitate finding them. In order to search the annotations, a [File View](views.md) must be created first. It is possible to query based on any of the annotations attached to the files.
+Files can be [annotated]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}) to facilitate finding them. In order to search the annotations, a [File View]({{ site.baseurl }}{% link _articles/views.md %}) must be created first. It is possible to query based on any of the annotations attached to the files.
For example, the [PCBC Project](https://www.synapse.org/#!Synapse:syn1773109) has a [table](https://www.synapse.org/#!Synapse:syn7511263) listing sequencing data files that have been annotated. To find all **mRNA fastq** files originating from **CD34+ cells** in the we can query by:
@@ -214,7 +214,7 @@ files = synapseutils.syncFromSynapse(syn, 'syn2390898')
### Download Tables
-Please view [here](tables.md) to learn how to use `Tables`.
+Please view [here]({{ site.baseurl }}{% link _articles/tables.md %}) to learn how to use `Tables`.
### Download Wikis
@@ -282,4 +282,4 @@ all_files = syncFromSynapse(entity='syn123', path='/path/to/myFolder')
# See Also
-[Versioning](versioning.md), [Tables](tables.md), [Wikis](wikis.md), [File Views](views.md), [Annotations and Queries](annotation_and_query.md)
+[Versioning]({{ site.baseurl }}{% link _articles/versioning.md %}), [Tables]({{ site.baseurl }}{% link _articles/tables.md %}), [Wikis]({{ site.baseurl }}{% link _articles/wikis.md %}), [File Views]({{ site.baseurl }}{% link _articles/views.md %}), [Annotations and Queries]({{ site.baseurl }}{% link _articles/annotation_and_query.md %})
diff --git a/_articles/edit_wiki_order.md b/_articles/edit_wiki_order.md
index 2837085e..83723e05 100644
--- a/_articles/edit_wiki_order.md
+++ b/_articles/edit_wiki_order.md
@@ -21,4 +21,4 @@ If you want to create new pages within an existing hierarchy rather than moving
## See Also
-[Wikis](wikis.md)
+[Wikis]({{ site.baseurl }}{% link _articles/wikis.md %})
diff --git a/_articles/evaluation_queues.md b/_articles/evaluation_queues.md
index 54c626b4..91d55e55 100644
--- a/_articles/evaluation_queues.md
+++ b/_articles/evaluation_queues.md
@@ -2,7 +2,7 @@
title: Evaluation Queues
layout: article
-excerpt: An queue accepts submission of Synapse entities for evaluation.
+excerpt: A Synapse Queue contains user submitted models or files to execute for challenges and benchmarking.
category: reproducible-research
---
@@ -10,108 +10,36 @@ An Evaluation queue allows for people to submit Synapse `Files`, `Docker` images
## Create an Evaluation Queue
-To create a queue, you must first create a Synapse `Project`. To create a Synapse Project, follow the instructions on the [Project and Data Management](getting_started.md#making-and-managing-projects-in-synapse) page. An Evaluation queue can take several parameters that you can use to customize your preferences. The minimum requirements to create a queue are:
+To create a queue, you must first create a Synapse `Project` or have [edit permissions]({{ site.baseurl }}{% link _articles/sharing_settings.md %}#edit-permissions) on an existing Project. To create a Synapse Project, follow the instructions on the [Project and Data Management]({{ site.baseurl}}{% link _articles/getting_started.md %}#making-and-managing-projects-in-synapse) page.
-* name – Unique name of the evaluation
-* description – A short description of the evaluation
-* contentSource – Synapse Project associated with the evaluation
-* submissionReceiptMessage – Message to display to users upon submission
-* submissionInstructionsMessage – Message to display to users detailing acceptable formatting for submissions.
+Once you've created your project, navigate to it and add `/challenge` to the url (e.g. www.synapse.org/#!Synapse:syn12345/challenge). Click **Tools** in the right corner and select **Add Evaluation Queue**.
-Optionally, you can restrict how things are submitted by using a quota. An Evaluation queue can only have one quota. If you want to change how long the queue is open, the start date (firstRoundStart), round duration (roundDurationMillis) and number of rounds (nunmberOfRounds) are required parameters. It is optional to set submission limit (submissionLimit).
+
-* firstRoundStart - The date/time at which the first round begins in UTC.
-* roundDurationMillis - The duration of each round in milliseconds.
-* numberOfRounds - The number of rounds, or null if there is no limit to set.
-* submissionLimit - The maximum number of submissions per team/participant per round. Please keep in mind that the system will prevent additional submissions by a user/team once they have hit this number of submissions.
+An Evaluation queue can take several parameters that you can use to customize your preferences.
-{% include note.html content="The name of your evaluation queue MUST be unique, otherwise the queue will not be created." %}
-
-The examples below shows how to create a queue and set the quota using all of the parameters in Python, R and the web client:
-
-##### Python
-
-```python
-import synapseclient
-
-syn = synapseclient.login()
-
-evaluation = synapseclient.Evaluation(name="My Unique Example Challenge Name",
- description="Short description of challenge queue",
- status="OPEN",
- contentSource="syn12345", # Your Synapse Project synID
- submissionInstructionsMessage="Instructions on submission format...",
- submissionReceiptMessage="Thanks for submitting to My Example Challenge!",
- quota={'submissionLimit':3, # The maximum number of submissions per team/participant per round.
- 'firstRoundStart':'2017-11-02T07:00:00.000Z', # The date/time ("%Y-%m-%dT%H:%M:%S%Z") at which the first round begins in UTC.
- 'roundDurationMillis':1645199000, #The duration of each round.
- 'numberOfRounds':1} # The number of rounds, or null if there is no end. (Based on the duration of each round)
-)
-
-syn.store(evaluation)
-```
-
-##### R
-
-```r
-library(synapser)
-
-synLogin()
-
-evaluation <- Evaluation(name="My Unique Example Challenge Name",
- description="Short description of challenge queue",
- status="OPEN",
- contentSource="syn12345", # Your Synapse Project synID
- submissionInstructionsMessage="Instructions on submission format...",
- submissionReceiptMessage="Thanks for submitting to My Example Challenge!",
- quota=c(submissionLimit=3, # The maximum number of submissions per team/participant per round.
- firstRoundStart = '2017-11-02T07:00:00.000Z', # The date/time ("%Y-%m-%dT%H:%M:%S%Z") at which the first round begins in UTC.
- roundDurationMillis = 1645199000, # The duration of each round.
- numberOfRounds=1) # The number of rounds, or null if there is no end. (Based on the duration of each round)
- )
-
-synStore(evaluation)
-```
-
-##### Web
-
-Navigate to your challenge site and add `/admin` to the url (e.g. www.synapse.org/#!Synapse:syn12345/admin). Click **Tools** in the right corner and select **Add Evaluation Queue**.
-
-
+* Name – Unique name of the evaluation
+* Description – A short description of the evaluation
+* Submission Instructions – Message to display to users detailing acceptable formatting for submissions.
+* Submission Receipt Message – Message to display to users upon submission
-In the web client, the quota can be modified under the **Challenge** tab by clicking `Edit`.
-
-## Set a Quota on an Existing Evaluation Queue
-
-The Evaluation ID can be found under the **Challenge** tab of your project. Please note that a Challenge tab will not appear on your project until you have created a Challenge (**Tools > Run Challenge**). In the case below, the Evaluation queue ID is `9610091`.
+{% include note.html content="The name of your evaluation queue MUST be unique, otherwise the queue will not be created." %}
-
-Using the Evaluation ID, we can configure the `quota` parameters of this evaluation queue with the R or Python client.
+### Setting Quotas on an Evaluation Queue
-##### Python
+Optionally, you can restrict how things are submitted by using a quota.
-```python
-import synapseclient
-syn = synapseclient.login()
-evalId = 9610091
-evaluation = syn.getEvaluation(evalId)
-evaluation.quota = {'submissionLimit':3} #The maximum number of submissions per team/participant per round.
-syn.store(evaluation)
-```
+
-##### R
-```r
-library(synapser)
-synLogin()
+An Evaluation queue can only have one quota. You may specify the length of time the queue is open, the start date, round duration, and number of rounds. These are required parameters. It is optional to set submission limit.
-evalId = 9610091
-evaluation <- synGetEvaluation(evalId)
+* First Round Start Date/Time - The date/time at which the first round begins.
+* Number of Rounds - The number of rounds, or null if there is no limit to set.
+* Submission Limit - The maximum number of submissions per team/participant per round. Please keep in mind that the system will prevent additional submissions by a user/team once they have hit this number of submissions.
+* Round Duration - The duration of each round. If you set a round duration you will want to set the number of rounds or it will assume an infinite number of rounds.
-evaluation$quota <- c('submissionLimit'=3) #The maximum number of submissions per team/participant per round.
-synStore(evaluation)
-```
## Share an Evaluation Queue
@@ -126,6 +54,14 @@ To set the sharing settings, go to the **Challenge** tab to view your list of Ev
{% include important.html content="When someone submits to an Evaluation, a copy of the submission is made, so a person with Administrator or Can score access will be able to download the submission even if the submitter deletes the entity." %}
+## Close an Evaluation Queue
+While there isn't technically a way of "closing" an evaluation queue, there are multiple ways to discontinue submissions for users.
+
+* Users are only able to submit to a queue if they have `can submit` permissions to it. If you have the ability to modify the permissions of a queue, you will still be able to submit to the queue due to your `administrator` access.
+* If the quota is set so the current date exceeds the the round start + round duration, no one will be able to submit to the queue. This includes users with administrator permissions.
+* Deleting a queue will also discontinue the ability to submit to it. Be careful when doing this, as deleting a queue is irreversible - you will lose all its submissions.
+
+
## Submitting to an Evaluation Queue
Any Synapse entity may be submitted to an Evaluation Queue.
@@ -175,7 +111,7 @@ Every submission you make to an Evaluation queue has a unique ID. This ID should
Navigate to a file in Synapse and click on **Tools** in the upper right-hand corner. Select **Submit To Challenge** to pick the challenge for your submission. Follow the provided steps to complete your submission.
-
+
## View Submissions of an Evaluation Queue
@@ -214,6 +150,23 @@ If you are happy with your leaderboard configurations, save both the configurati
+
+### Submit to an Evaluation Queue from a Wiki Page
+
+
+
+You may embed a `Submit To Evaluation` widget on a Wiki page to improve visibility of your Evaluation queue. The widget allows participants to submit to multiple Evaluation queues within a Project or a single Evaluation queue.
+
+Currently, this Wiki widget is required to submit Synapse Projects to an Evaluation queue. Synapse Docker repositories can not be submitted through this widget.
+
+
+
+The "Evaluation Queue unavailable message" is customizable. A queue may appear unavailable to a user if:
+
+* The Project doesn't have any Evaluation queues.
+* The user does not have permission to view a Project's Evaluation queues. Learn more about [sharing settings]({{ site.baseurl }}{% link _articles/sharing_settings.md %}).
+* The Evaluation queue quota is set such that a user can not submit to the queue. Learn more about [quotas]({{ site.baseurl }}{% link _articles/evaluation_queues.md%}#create-an-evaluation-queue).
+
# See Also
-To learn how to create a Wiki page, please visit [the Wikis article](wikis.md).
+To learn how to create a Wiki page, please visit [the Wikis article]({{ site.baseurl }}{% link _articles/wikis.md %}).
diff --git a/_articles/faq.md b/_articles/faq.md
index da4d91b5..486268d3 100644
--- a/_articles/faq.md
+++ b/_articles/faq.md
@@ -20,11 +20,11 @@ Synapse allows researchers to share and describe data, analyses, and other conte
#### No, seriously what does Synapse do?
-For a comprehensive introduction to Synapse see our [Getting Started guide](getting_started.md).
+For a comprehensive introduction to Synapse see our [Getting Started guide]({{ site.baseurl }}{% link _articles/getting_started.md %}).
#### How much does it cost to use Synapse?
-We allow groups to get started for free by subsidizing relatively moderate amounts of internal Synapse cloud storage (e.g. 10s of GB). If your needs will exceed this limit we suggest contacting [SynapseInfo@sagebase.org](mailto:SynapseInfo@sagebase.org) for other solutions, including the ability for groups to host their own content either in cloud storage, proxied from [local file servers](custom_storage_location.md) or as external links.
+We allow groups to get started for free by subsidizing relatively moderate amounts of internal Synapse cloud storage (e.g. 10s of GB). If your needs will exceed this limit we suggest contacting [SynapseInfo@sagebase.org](mailto:SynapseInfo@sagebase.org) for other solutions, including the ability for groups to host their own content either in cloud storage, proxied from [local file servers]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}) or as external links.
#### How is Synapse funded?
@@ -44,7 +44,7 @@ Anyone age 13 or older may use Synapse. We have highlighted a series of [researc
#### What are the Synapse Terms of Use?
-The [Terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4) fully describes the governance terms and conditions of Synapse. In order to register on Synapse, you must review and agree to the terms of the Synapse Awareness and Ethics Pledge. For more information see the complete Synapse [Governance policies](governance.md).
+The [Terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4) fully describes the governance terms and conditions of Synapse. In order to register on Synapse, you must review and agree to the terms of the Synapse Awareness and Ethics Pledge. For more information see the complete Synapse [Governance policies]({{ site.baseurl }}{% link _articles/governance.md %}).
#### Is Synapse open source?
@@ -56,7 +56,7 @@ Yes, Synapse is built on top of a RESTful service that is automatically [documen
#### How do I set up my own instance of Synapse?
-Synapse was developed with the philosophy to encourage collaboration across institutional boundaries and is therefore provided as “Software As A Service” with a single instance used by all users. This makes it easy both to discover new content and share with new collaborators. We do support private project spaces where content sharing is controlled by the individual user. In addition, Synapse has the ability to reference resources that are stored elsewhere. This allows Synapse to store metadata about the content such as annotations, descriptive wiki pages and provenance but not the actual data. Currently Synapse has specific support for files stored at URLs, on SFTP servers, on AWS S3 and arbitrary file servers (see: [Custom Storage Locations](custom_storage_location.md)).
+Synapse was developed with the philosophy to encourage collaboration across institutional boundaries and is therefore provided as “Software As A Service” with a single instance used by all users. This makes it easy both to discover new content and share with new collaborators. We do support private project spaces where content sharing is controlled by the individual user. In addition, Synapse has the ability to reference resources that are stored elsewhere. This allows Synapse to store metadata about the content such as annotations, descriptive wiki pages and provenance but not the actual data. Currently Synapse has specific support for files stored at URLs, on SFTP servers, on AWS S3 and arbitrary file servers (see: [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %})).
#### What do I do if I find a bug?
@@ -64,14 +64,14 @@ You may browse open issues or file a bug through our [Jira](https://sagebionetwo
#### How do I get started?
-Whether you are looking to access content or use Synapse to track your own work, start by [registering](https://www.synapse.org/#!RegisterAccount:0) for a Synapse user account. Before uploading files, you will need to take a short [certification quiz](https://www.synapse.org/#!Quiz:Certification) that demonstrates your understanding of the ethical considerations in sharing data and the manner in which data is managed and shared in Synapse. See the [Getting Started guide](getting_started.md) for tips on how to access data, create your own project, and share content with others.
+Whether you are looking to access content or use Synapse to track your own work, start by [registering](https://www.synapse.org/#!RegisterAccount:0) for a Synapse user account. Before uploading files, you will need to take a short [certification quiz](https://www.synapse.org/#!Quiz:Certification) that demonstrates your understanding of the ethical considerations in sharing data and the manner in which data is managed and shared in Synapse. See the [Getting Started guide]({{ site.baseurl }}{% link _articles/getting_started.md %}) for tips on how to access data, create your own project, and share content with others.
# Accessing Content
#### My colleague put some content in Synapse. How do I find it?
This will depend if the content is public or private. If private, you will need to make sure your colleague has shared this content with you. Shared content is visible from your “Dashboard page” under the tab “Shared directly with me”. If you favorite the content (using the yellow star) it will appear under your list of favorites visible from the top of any Synapse page or on your [Profile](https://www.synapse.org/#!Profile:v/projects/favorites).
-All public data is queryable. For more information see [help on querying](annotation_and_query.md) or from the “Search” box in the top right corner of any Synapse page.
+All public data is queryable. For more information see [help on querying]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}) or from the “Search” box in the top right corner of any Synapse page.
#### I have heard Synapse hosts several public datasets. How do I find them?
@@ -79,7 +79,7 @@ Synapse hosts multiple research communities that generate data which is released
#### What does a Synapse account let me do than I can’t do without it?
-You can browse public content in Synapse without registering. However, without an account you cannot add new content to Synapse, nor can you upload or download `Files` or `Tables`. With an account you can create `Projects` and `Wikis`, download `Open Data` and request access to `Controlled Data`. Further, an account lets you collaborate with other Synapse users and create user teams. For more information see the [User Credentials](accounts_certified_users_and_profile_validation.md) page.
+You can browse public content in Synapse without registering. However, without an account you cannot add new content to Synapse, nor can you upload or download `Files` or `Tables`. With an account you can create `Projects` and `Wikis`, download `Open Data` and request access to `Controlled Data`. Further, an account lets you collaborate with other Synapse users and create user teams. For more information see the [User Credentials]({{ site.baseurl }}{% link _articles/accounts_certified_users_and_profile_validation.md %}) page.
#### What is a validated profile?
@@ -89,11 +89,11 @@ Validating your profile is a process where your identity is established through
#### I have my research results - How can Synapse help me share them?
-Synapse makes it easy to share files of any sort, with whomever you choose whether a small group of collaborators or the general public. You may share raw data, summarized data, analysis results, or anything in between. We recommend using [Getting started with Synapse guide](getting_started.md) for more details on how to upload and manage content.
+Synapse makes it easy to share files of any sort, with whomever you choose whether a small group of collaborators or the general public. You may share raw data, summarized data, analysis results, or anything in between. We recommend using [Getting started with Synapse guide]({{ site.baseurl }}{% link _articles/getting_started.md %}) for more details on how to upload and manage content.
#### Why do I have to be a certified user to upload content?
-User certification ensures that you understand your responsibilities for sharing data through Synapse, especially data derived from human participants. These responsibilities include making sure that data derived from human participants is de-identified and that all applicable privacy laws and regulations are observed. See the [How to contribute data](contribute_and_access_controlled_use_data.md) for more information.
+User certification ensures that you understand your responsibilities for sharing data through Synapse, especially data derived from human participants. These responsibilities include making sure that data derived from human participants is de-identified and that all applicable privacy laws and regulations are observed. See the [How to contribute data]({{ site.baseurl }}{% link _articles/contribute_and_access_controlled_use_data.md %}) for more information.
#### How do I get certified?
@@ -101,11 +101,11 @@ To become a certified user, you will need to pass a brief [quiz](https://www.syn
#### Is everything I share on Synapse public?
-No. Use `Sharing settings` to control who can see the content you create. By default, `Projects` and their content are visible only to the user who created it. By using the Synapse `Sharing settings`, you have the ability to grant other Synapse users, Synapse teams, or the public access to your Project content. You can learn more here: [Sharing Settings and Conditions for Use](access_controls.md).
+No. Use `Sharing settings` to control who can see the content you create. By default, `Projects` and their content are visible only to the user who created it. By using the Synapse `Sharing settings`, you have the ability to grant other Synapse users, Synapse teams, or the public access to your Project content. You can learn more here: [Sharing Settings and Conditions for Use]({{ site.baseurl }}{% link _articles/access_controls.md %}).
#### Can I store sensitive information about human subjects in Synapse?
-Yes. Synapse has an IRB-approved data governance procedure that employs `Conditions for Use` to allow for the sharing of sensitive data in a controlled manner. You can learn more by reading our [Sharing Settings and Conditions for Use](access_controls.md) documentation and [Governance documentation](governance.md). If you have questions or would like assistance in applying `Conditions of Use` to your data, please contact the Synapse Access and Compliance Team at [act@sagebase.org](mailto:act@sagebase.org).
+Yes. Synapse has an IRB-approved data governance procedure that employs `Conditions for Use` to allow for the sharing of sensitive data in a controlled manner. You can learn more by reading our [Sharing Settings and Conditions for Use]({{ site.baseurl}}{% link _articles/access_controls.md %}) documentation and [Governance documentation]({{ site.baseurl }}{% link _articles/governance.md %}). If you have questions or would like assistance in applying `Conditions of Use` to your data, please contact the Synapse Access and Compliance Team at [act@sagebase.org](mailto:act@sagebase.org).
#### How do I know the content I put in Synapse will be secure? What security measures does Synapse have?
@@ -113,4 +113,4 @@ Synapse stores content in Amazon Web Services, which provides a layer of securit
#### Where are my files stored?
-By default, Synapse stores files in Amazon Simple Storage Services (S3). However it is possible to set up Synapse to store files in different locations such as your local SFTP server. For files stored outside of S3, Synapse can be used to organize, manage, and access files through the use of Synapse annotations to store file-specific metadata. (see: [Custom Storage Locations](custom_storage_location.md))
+By default, Synapse stores files in Amazon Simple Storage Services (S3). However it is possible to set up Synapse to store files in different locations such as your local SFTP server. For files stored outside of S3, Synapse can be used to organize, manage, and access files through the use of Synapse annotations to store file-specific metadata. (see: [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %}))
diff --git a/_articles/files_and_versioning.md b/_articles/files_and_versioning.md
index 074bbbce..b42d27a4 100644
--- a/_articles/files_and_versioning.md
+++ b/_articles/files_and_versioning.md
@@ -17,9 +17,9 @@ category: managing-data
}
-Synapse `Files` can be created by uploading or linking to digital files on the web. They are accessible to anyone who has [access](access_controls.md), can be annotated with custom metadata, can be embedded into Synapse `Wiki` pages, and can be associated with a [DOI](doi.md). `Files` carry the Conditions for Use of the Synapse `Folder` they are placed in, plus any additional specific Conditions for Use they have on their own.
+Synapse `Files` can be created by uploading or linking to digital files on the web. They are accessible to anyone who has [access]({{ site.baseurl }}{% link _articles/access_controls.md %}), can be annotated with custom metadata, can be embedded into Synapse `Wiki` pages, and can be associated with a [DOI]({{ site.baseurl }}{% link _articles/doi.md %}). `Files` carry the Conditions for Use of the Synapse `Folder` they are placed in, plus any additional specific Conditions for Use they have on their own.
-By default, `Files` uploaded to Synapse are stored in 'Synapse Storage', which is freely available to you. `Files` can also be stored on your own Amazon S3 bucket (see [Custom Storage Locations](custom_storage_location.md)) or other custom locations. Furthermore, if you don't want to upload a file (it has external restrictions on sharing, is really large, for example) you can also link to the file. In this way, the file will be accessible through the Synapse clients when you are on the computer that the file is stored, but can be annotated, queried, and documented with a Wiki through Synapse. Lastly, you can provide web-accessible links as Synapse files, which will redirect to that location. All of the same Synapse `File` features are available are available on external links as well.
+By default, `Files` uploaded to Synapse are stored in 'Synapse Storage', which is freely available to you. `Files` can also be stored on your own Amazon S3 bucket (see [Custom Storage Locations]({{ site.baseurl }}{% link _articles/custom_storage_location.md %})) or other custom locations. Furthermore, if you don't want to upload a file (it has external restrictions on sharing, is really large, for example) you can also link to the file. In this way, the file will be accessible through the Synapse clients when you are on the computer that the file is stored, but can be annotated, queried, and documented with a Wiki through Synapse. Lastly, you can provide web-accessible links as Synapse files, which will redirect to that location. All of the same Synapse `File` features are available are available on external links as well.
Synapse `Files` (as well as `Folders` and `Projects`) are identified by a unique identifier called a Synapse ID. It takes the form `syn12345678`. This identifier can be used to refer to a specific file on the web and through the clients.
@@ -63,7 +63,7 @@ file <- synStore(file)
## Moving a File
-All Synapse clients offer a way to move files and folders. Please note that [File Views](views.md) and [sync manifests](uploading_in_bulk.md) **cannot** be used to move files.
+All Synapse clients offer a way to move files and folders. Please note that [File Views]({{ site.baseurl }}{% link _articles/views.md %}) and [sync manifests]({{ site.baseurl }}{% link _articles/uploading_in_bulk.md %}) **cannot** be used to move files.
The command line client has a sub-command `mv` which can be used to move files and folders. The Python client provides the [syn.move](https://python-docs.synapse.org/build/html/Client.html#synapseclient.Synapse.move) command, and the R client has [synMove()](https://r-docs.synapse.org/reference/synMove.html).
@@ -229,7 +229,7 @@ Any change to a `File` will automatically update its version. If this isn't the
##### Web
-Please refer to the [Annotations and Queries](annotation_and_query.md) article for instructions on adding/editing annotations via the web client.
+Please refer to the [Annotations and Queries]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}) article for instructions on adding/editing annotations via the web client.
##### Command line
@@ -241,8 +241,8 @@ synapse set-annotations --id syn56789 --annotations '{"fileType":"bam", "assay":
##### Python
```python
-# Get file from Synapse, set download=False since we are only updating annotations
-file = syn.get('syn56789', download=False)
+# Get file from Synapse, set downloadFile=False since we are only updating annotations
+file = syn.get('syn56789', downloadFile=False)
# Add annotations
file.annotations = {"fileType":"bam", "assay":"RNA-seq"}
# Store the file without creating a new version
@@ -261,7 +261,7 @@ annotations <- synSetAnnotations(file, annotations=list(fileType = "bam", assay
**Setting provenance without changing version**
##### Web
-Please refer to the [Provenance](provenance.md) article for instructions on adding/editing annotations via the web client.
+Please refer to the [Provenance]({{ site.baseurl }}{% link _articles/provenance.md %}) article for instructions on adding/editing annotations via the web client.
##### Command line
@@ -272,8 +272,8 @@ synapse set-provenance -id syn56789 -executed ./path/to/example_code
##### Python
```python
-# Get file from Synapse, set download=False since we are only updating provenance
-file = syn.get('syn56789', download=False)
+# Get file from Synapse, set downloadFile=False since we are only updating provenance
+file = syn.get('syn56789', downloadFile=False)
# Add provenance
file = syn.setProvenance(file, activity = Activity(used = '/path/to/example_code'))
# Store the file without creating a new version
@@ -348,4 +348,4 @@ Some files in Synapse are supported with previews to allow users to peek at the
## See Also
-[Provenance](provenance.md), [Annotations and Queries](annotation_and_query.md), [Downloading Data](downloading_data.md)
+[Provenance]({{ site.baseurl }}{% link _articles/provenance.md %}), [Annotations and Queries]({{ site.baseurl }}{% link _articles/annotation_and_query.md %}), [Downloading Data]({{ site.baseurl }}{% link _articles/downloading_data.md %})
diff --git a/_articles/forms.md b/_articles/forms.md
index 144d5777..e572d4cf 100644
--- a/_articles/forms.md
+++ b/_articles/forms.md
@@ -2,7 +2,7 @@
title: "Forms"
layout: article
excerpt: Embedding Synapse Forms to Collect Data into Tables
-category: how-to
+category: managing-data
---
`Forms` are a way to collect data _into_ a Synapse `Table`. Forms are currently available only under the _alpha_ configuration as they are not yet ready for full release; please use Forms only with this caveat in mind. This document will evolve as additional functionality is added.
@@ -15,7 +15,7 @@ With Forms, you can:
## Creating a Table
-In order to use Forms, you'll need to first create a `Table`. Read more about creating tables here: [Tables](tables.md)
+In order to use Forms, you'll need to first create a `Table`. Read more about creating tables here: [Tables]({{ site.baseurl }}{% link _articles/tables.md %})
## Activating and Deactivating 'Alpha' Mode
@@ -41,8 +41,8 @@ The form is also responsive, so it will render on both large and small screens.
## Allowing Others to Contribute
-Entering data into a `Form` is equivalent to editing the data in a `Table`, so users who wish to add data using the form will need to have "Edit" permissions on the Table. See [Sharing Settings](access_controls.md) for more information on controlling who can edit your Table.
+Entering data into a `Form` is equivalent to editing the data in a `Table`, so users who wish to add data using the form will need to have "Edit" permissions on the Table. See [Sharing Settings]({{ site.baseurl }}{% link _articles/access_controls.md %}) for more information on controlling who can edit your Table.
## See Also
-[Tables](tables.md), [Wikis](wikis.md), [Sharing Settings](access_controls.md)
+[Tables]({{ site.baseurl }}{% link _articles/tables.md %}), [Wikis]({{ site.baseurl }}{% link _articles/wikis.md %}), [Sharing Settings]({{ site.baseurl }}{% link _articles/access_controls.md %})
diff --git a/_articles/getting_started.md b/_articles/getting_started.md
index 4d4acc65..98a358ff 100644
--- a/_articles/getting_started.md
+++ b/_articles/getting_started.md
@@ -5,88 +5,161 @@ excerpt: A getting started guide for new users who are interested in learning ab
order: 1
---
-This guide is for new users who are interested in learning about Synapse. You will learn fundamental Synapse features by performing some common tasks:
+Welcome to Synapse! We are happy to have you as part of the Synapse Community.
-* Create your own `Project` and add content to Synapse
-* Provide a Project description alongside your materials via the Synapse `Wiki` tools
-* Share your work with other Synapse users, `Teams` of users, or the public
+This guide is for new Synapse users and takes 10 minutes to read.
+
+You will learn:
+
+* How to set up your Synapse account.
+* The steps to get started with core Synapse features.
+* This guide will focus on using Synapse on the web. Many researchers also use Synapse with R, Python or Command Line tools. See [Getting Started with Synapse APIs]({{ site.baseurl }}{% link _articles/getting_started_clients.md %}).
## What is Synapse?
+Synapse is a research application developed by [Sage Bionetworks](https://sagebionetworks.org/). Using Synapse, you can find, upload, track, discuss and download datasets and analyses. Synapse supports all kinds of working groups: individuals, small teams, and large consortia.
+For more information, see Introduction to the Sage Platform
+
+## Setting Up Synapse
+
+### Create your account
+Anyone can browse public projects on Synapse without creating an account. However, to download files and create new content, you will need to make an account.
+You can register using an email address, or your Google account.
+
+After registering, you will receive an email to verify your account. You must accept the Terms and Conditions of use to proceed.
+
+
+
+## [Register](https://www.synapse.org/#!RegisterAccount:0)
+
+### Your first time on the Synapse Dashboard
+
+After accepting the Terms and Conditions of use, you will arrive at the `Projects` tab of your Dashboard. Synapse Projects are online workspaces where researchers can collaborate and organize their work.
+
+
+
+All of your Projects are stored here, whether you join an existing one or create a new Project. Data, Wikis and Discussions are contained in each Project. We’ll discuss more about [Projects later on in this guide]({{ site.baseurl }}{% link _articles/getting_started.md %}#projects).
+
+From your dashboard you can also:
+
+* View your Profile
+* See Challenges you are registered for
+* See the Teams you are a member of
+* Access your Downloads list
+* Change your Settings
+
+You can access all of these tabs from the User Menu as well.
+
+
+
+## Governance and Access to Data
+Before exploring Synapse, it is important to get an understanding of Governance and Access Privileges. This will ensure that you are able to use Synapse in an ethical and compliant way.
-Synapse is a collaborative research platform that allows individuals and teams to share, track, and discuss their data and analysis in projects. Synapse is built to work on the web. We provide access to Synapse features and services for programmers through a REST API, [Python client](https://python-docs.synapse.org/build/html/index.html), [command line client](https://python-docs.synapse.org/build/html/CommandLineClient.html), and [R client](https://r-docs.synapse.org/).
+### Governance
+Misuse of human health data can result in severe ethical and legal issues. This means that all Synapse users must comply with privacy and security standards.
+The Governance team at Sage Bionetworks has created policies that maintain these standards. These policies specify Synapse users’ rights and responsibilities. They include conditions for who can use and access data. They also specify the rights and responsibilities of Sage Bionetworks and of the Synapse Access and Compliance Team (ACT).
-Synapse hosts many [research projects](https://www.synapse.org/#!StandaloneWiki:ResearchCommunities) and [resources](https://www.synapse.org/#!StandaloneWiki:OpenResearchProjects). It also hosts crowdsourced competitions, including [DREAM Challenges](http://dreamchallenges.org/). [Sage Bionetworks](http://www.sagebionetworks.org) provides Synapse services free of charge to the scientific community through generous support from various [funding sources](/articles/faq.html#how-is-synapse-funded).
+[Learn more about Governance]({{ site.baseurl }}{% link _articles/governance.md %}).
-## Create Your Account
+[Read the terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4).
-Anyone can browse public content on the Synapse web site. To download and create content, you will need to [register for an account](https://www.synapse.org/register) using an email address. You will receive an email message for verification to complete the registration process.
+[Learn more about Synapse Community Standards]({{ site.baseurl }}{% link _articles/governance.md %}#synapse-community-standards).
-## Getting Certified
-Synapse is a data sharing platform approved for storing data from human subjects research. This requires special care and thought. To upload files, Sage Bionetworks requires you demonstrate awareness of privacy and security issues.
+### User Types will Determine Access to Data
-You can complete this by taking a [Certification Quiz](https://www.synapse.org/#!Quiz:Certification).
+The way in which you will use Synapse, will depend on your own research needs and your Synapse User Type.
-## Making and Managing Projects in Synapse
+There are four User Types in Synapse. Each User Type has a different level of permission.
-Synapse Projects are online workspaces where researchers can collaborate and organize their work. Synapse supports all kinds of working groups: individuals, small teams, and large consortia.
+These are:
+* Anonymous users who do not have a Synapse account
+* Registered users who have created a Synapse account
+* Certified users who have passed the certification quiz
+* Validated users whose profile has been verified
-To create a new Project:
+The table below summarizes the user privileges for each level.
+
+ {:.markdown-table}
+| | Anonymous | Registered | Certified | Validated* |
+| :--------------------------------------------------------: | :---------: | :----------: | :---------: | :----------: |
+| Browse Public Project Catalogue | x | x | x | x |
+| Browse Public File Catalog | x | x | x | x |
+| Create a Project | | x | x | x |
+| Add Wiki Content | | x | x | x |
+| Download Files/Tables* | | x | x | x |
+| Upload Files/Tables | | | x | x |
+| Add Provenance | | | x | x |
+| Requesting access to data collected through research app | | | | x |
-1. Navigate to the User Menu and click on [**Projects**](https://www.synapse.org/#!Profile:v/projects).
-2. Click the **Create a New Project** button.
-3. Decide on a unique name for your Project and click **Save**.
+[Learn more about User Types]().
-Your Projects [dashboard](https://www.synapse.org/#!Profile:v/projects) stores your collection of Projects.
+**Become a Certified User**
-Read about [Projects](making_a_project.md) in the User Guide.
+To upload files, you must become a Certified User. This demonstrates that you have awareness of privacy and security issues. You can become certified at any time by taking a short quiz. If you try to upload files without becoming certified, Synapse will prompt you to take the quiz.
-## Synapse IDs
+After you become a Certified User, you can have your profile Validated. A Validated profile means that your identity has been verified by the Sage Access and Compliance team. This allows you to access more features and data. Profile validation enables greater transparency within the research community.
-Synapse Projects are assigned a Synapse ID, a globally unique identifier used for reference with the format `syn12345678`. Often abbreviated to "synID", the ID of an object never changes, even if the name does. The Synapse ID is always accessible in the URL and visible on the webpage.
+Validation steps include:
+* Filling out your profile
+* Connecting your ORCID,
+* Signing the oath,
+* Providing a recent identity attestation
-## Organizing Content in `Files` and `Folders`
+Learn more about Certified and Validated Accounts.
-Projects contain Files, which can be organized into Folders. Folders and Files also have their own unique Synapse IDs and can be moved within or between Projects. Uploaded files are stored within Synapse storage.
+## Projects
-Use the **Tools Menu** to upload a file:
+Now that you have briefly explored Synapse, it is time to learn a bit more about Projects.
-1. Navigate to the Files tab.
-2. Use the **Files Tools** menu to select **Add New Folder**.
-3. Decide on a Folder name and click **Save**.
-4. Navigate into your new Folder and use the **Folder Tools** menu to select **Upload or Link to a File**.
-5. Use the **Browse** button to select the file, or drag and drop it to upload, and click **Save**.
+### What is a Project?
-To explore other features available for Files and Folders, read about [annotating Files](/articles/annotation_and_query.html), [assigning DOIs](/articles/doi.html), [versioning](/articles/files_and_versioning.html), [`Provenance`](/articles/provenance.html), and [sharing settings](/articles/access_controls.html).
+
-## Adding a Wiki to your Project
+A Project is a workspace that you can use to organize your work and collaborate with others. They act as “containers” and can group related objects such as content and people together.
-A Wiki is a document that can be edited by multiple people on the web. Synapse uses Wikis to provide descriptions of your Project and data.
+Using Projects, you can:
+* Store content such as Data, Code, Results, Figures and Documents
+* Organize your Files and Folders
+* Link and share content with others
+* Create custom, searchable Annotations
+* Create Tables
+* Have discussions using a Forum
+* Create Project Documentation using Wikis
+* Control Access
+* Release your work publicly
-Wiki pages can be written using text, [Markdown](https://www.markdownguide.org/), or basic HTML. Content can include images, tables, code blocks, LaTeX formatted equations, scholarly references, and references to other things in Synapse.
+Any user can create multiple Projects. Each Project can be tailored to a specific workflow, using only the features needed for the Project.
-Use the **Tools** menu to see available Wiki options on Projects, Folders and Files:
+[Learn more about Projects]({{ site.baseurl }}{% link _articles/making_a_project.md %}).
-1. Click the **Tools** button and choose **Edit Wiki**.
- - During editing, you have the option to "Preview" your edits.
-2. Add relevant text, and click **Save**.
+## Synapse ID
-See the [Wiki](/articles/wikis.html) User Guide for more information and examples.
+On the top left of your Project, you’ll notice a Synapse ID with the format “syn12345678”. Everything that is created in Synapse has a unique Synapse ID. The Synapse ID is often abbreviated to “synID”. The ID of an object never changes, even if the name does.
-## Sharing and Teams
+
-The default setting for new Projects in Synapse is **private**. As a project owner, you choose who to share with, and how. You can find Project sharing settings under the **Project Settings** menu. Permissions or sharing settings at the Project level are automatically inherited by all Files and Folders within the Project. If needed, you can **Create Local Sharing Settings** to make certain Files or Folders have permissions that differ from the parent Project.
+The synID is always accessible in the URL and visible on the webpage. It is used for reference to any object in Synapse.
-Groups of users can form Teams for collaboration. Teams can be used for permissions and for communication. Sharing things with teams instead of individual users can provide simiplicity for administrators.
+## What's Next?
+Now that you have a basic understanding of how to start using Synapse you can learn more about how to use the core features with these tutorials below.
-For more details, visit the [Sharing Settings](/articles/access_controls.html) and [Teams](/articles/teams.html) User Guide.
+* I want to create a new project
+* I want to upload my data and share it
+* I want to download existing data
+* I want to organize files in a project
+* I want to create a New team
+* I want to download research data from mobile studies
+* I want use Synapse programmatically
-## More Guides
+## Advanced Topics
-Find additional information in our [User Guide](/articles).
+### Practicing Version Control
+All content in Synapse is versioned automatically, with the ability to provide version comments on files, and to link content together with rich provenance relationships. This is particularly useful for groups leveraging Synapse as a component of a data processing pipeline, or citing data for publications, as Synapse allows data contributors to mint DOIs for most content. Learn more versioning content, creating and managing provenance, and how to mint DOIs.
-Read about Synapse [governance](/articles/governance.html) and [Terms and Conditions of Use](https://s3.amazonaws.com/static.synapse.org/governance/SageBionetworksSynapseTermsandConditionsofUse.pdf?v=4).
+### Custom Metadata
+Synapse provides a number of default metadata fields that automatically annotate content with useful information, but also offers the ability for users to define their own metadata key-value pairs to provide context to their data. These annotations can be viewed and managed at the individual file level, or by creating “views” that show annotations on all content across a larger scope such as a project or folder. Learn more about annotations and views.
-For information on using Synapse programmatically, see the documentation for the [Python client](https://python-docs.synapse.org/build/html/index.html), [command line client](https://python-docs.synapse.org/build/html/CommandLineClient.html), and [R client](https://r-docs.synapse.org/)
+### Search
+Leverage Synapse global search to discover data, projects, or even potential collaborators! Search is permission-dependent, meaning it works across all public content and content that you have access to, if logged in. Learn more about Searching.
\ No newline at end of file
diff --git a/_articles/getting_started_clients.md b/_articles/getting_started_clients.md
index bda44002..707e9dee 100644
--- a/_articles/getting_started_clients.md
+++ b/_articles/getting_started_clients.md
@@ -12,7 +12,7 @@ The API clients provide a way to use Synapse programmatically. This page shows y
* Python
* R
-To manage stored login credentials, visit the [Client Configuration page](client_configuration.md).
+To manage stored login credentials, visit the [Client Configuration page]({{ site.baseurl }}{% link _articles/client_configuration.md %}).
## Command Line
@@ -27,8 +27,8 @@ pip install synapseclient
```
```console
-synapseclient login "username" "password"
-synapseclient -h
+synapse login "username" "password"
+synapse -h
```
For more documentation on the command line client, see the [`synapseclient` docs](https://python-docs.synapse.org/build/html/CommandLineClient.html).
diff --git a/_articles/index.md b/_articles/index.md
index 8d9f9bf4..f7e0a3eb 100644
--- a/_articles/index.md
+++ b/_articles/index.md
@@ -1,5 +1,5 @@
---
-title: Articles
+title: Overview
layout: index
description: These are the main instructions for using Synapse. Pick an section that interests you or read them in order.
---
diff --git a/_articles/links.md b/_articles/links.md
index bc30640b..8a423dac 100644
--- a/_articles/links.md
+++ b/_articles/links.md
@@ -2,7 +2,7 @@
title: Links
layout: article
excerpt: Creating, and modifying links on Synapse.
-category: how-to
+category: managing-data
---