-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
21 changed files
with
319 additions
and
124 deletions.
There are no files selected for viewing
Binary file added
BIN
+42.2 KB
modules/cloud4/modules/gsql-editor/images/Screenshot 2024-10-11 at 10.59.49.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+718 KB
modules/cloud4/modules/gsql-editor/images/Screenshot 2024-10-11 at 11.39.23.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,7 @@ | ||
* xref:index.adoc[Load Data] | ||
** xref:load-data:load-from-local.adoc[] | ||
** xref:load-data:load-from-cloud.adoc[] | ||
** xref:load-data:load-from-s3.adoc[] | ||
** xref:load-data:load-from-gcs.adoc[] | ||
** xref:load-data:load-from-blob.adoc[] | ||
** xref:load-data:load-from-other-sources.adoc[] | ||
*** xref:load-data:jdbc.adoc[] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
109 changes: 109 additions & 0 deletions
109
modules/cloud4/modules/load-data/pages/load-from-gcs.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
= Load from Google Cloud Storage (GCS) | ||
:experimental: | ||
|
||
If you store your data in AWS S3, TigerGraph Cloud provides seamless integration for data ingestion. | ||
You can directly load data from your S3 buckets into your graph databases, eliminating the need for manual data transfers. | ||
This simplifies the process of importing large datasets and enables you to leverage the scalability and durability of AWS S3 for your graph analysis. | ||
|
||
== 1) Select Source | ||
|
||
.Once you’ve selected AWS S3, you will be asked to configure the AWS S3 data source. | ||
. Click on image:Screenshot 2024-04-17 at 9.36.58 PM.png[width=50] to add a new AWS S3 data source. | ||
+ | ||
image:Screenshot 2024-04-17 at 9.36.27 PM.png[] | ||
|
||
. You will need to provide your S3 `AWS access key id` and S3 `AWS secret access key`. | ||
+ | ||
image:Screenshot 2024-04-17 at 9.37.32 PM.png[] | ||
. Once you have those configured, you can add one or multiple `S3 URI` within the same S3 bucket. | ||
|
||
. Click btn:[ Next ] to process the file(s). | ||
+ | ||
[NOTE] | ||
==== | ||
The current data loading tool only supports CSV files. Other formats will be available in later releases. | ||
==== | ||
|
||
== 2) Configure File | ||
.This step lets you configure the source file details. | ||
. The data loading tool will automatically detect the `.csv` separators and line breaks. | ||
The parser automatically splits each line into a series of tokens. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.53.45 PM.png[] | ||
+ | ||
[TIP] | ||
==== | ||
If the parsing is *not* correct, click on the image:Screenshot 2024-04-17 at 5.54.17 PM.png[width=75] | ||
button to configure a different option for the delimiter, such as `eol`, or `quote and header`. | ||
image:Screenshot 2024-04-17 at 5.54.50 PM.png[] | ||
==== | ||
+ | ||
The enclosing character is used to mark the boundaries of a token, overriding the delimiter character. | ||
+ | ||
==== | ||
For example, if your delimiter is a comma, but you have commas in some strings, then you can define single or double quotes as the enclosing character to mark the endpoints of your string tokens. | ||
==== | ||
+ | ||
[NOTE] | ||
==== | ||
It is not necessary for every token to have enclosing characters. The parser uses enclosing characters when it encounters them. | ||
==== | ||
+ | ||
[TIP] | ||
==== | ||
You can edit the header line of the parsing result to give each column a more intuitive name, since you will be referring to these names when loading data to the graph. | ||
The header name is ignored during data loading. | ||
==== | ||
|
||
. Once you are satisfied with the file settings, click btn:[ Next ] to proceed. | ||
|
||
== 3) Configure Map | ||
|
||
.If you are loading data into a brand new graph, you will be prompted to let our engine generate a schema and mapping for you. Or you can start from scratch. For more details of schema design please refer to xref:cloud4:schema-designer:index.adoc[Schema Designer]. | ||
. Select `Generate the schema only` or `Generate the schema and data mapping`. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.55.35 PM.png[] | ||
+ | ||
[CAUTION] | ||
==== | ||
The schema generation feature is still a preview feature. The correctness and efficiency of the resulting graph schema and mapping could vary. | ||
==== | ||
|
||
. In the `Source` column, you can choose the specific column from the data source that you want to map with the attribute. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.56.15 PM.png[] | ||
+ | ||
. Use the `+` button to create a new attribute of the target vertex or edge. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.57.07 PM.png[] | ||
|
||
. The `Map all to target` button aligns existing attribute names with the corresponding data source headers to establish mappings and also introduces new attributes based on unmatched data source headers. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.57.52 PM.png[] | ||
. Click btn:[Next] to proceed. | ||
|
||
|
||
|
||
== 4) Confirm | ||
|
||
.This step will let you confirm the changes made to the schema and the data mapping you created to load the data. | ||
. Simply review the `Schema to be change` and `Data to be loaded` lists. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.58.36 PM.png[] | ||
+ | ||
[CAUTION] | ||
==== | ||
Please be aware that some schema changes will result in unintentional deletion of the data. Please carefully review the warning message before confirming the loading. | ||
==== | ||
. Click on the btn:[Confirm] button to run the loading jobs and monitor their `Status`. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.59.16 PM.png[] | ||
|
||
== Next Steps | ||
|
||
Next, learn how to use the xref:cloud4:schema-designer:index.adoc[Schema Designer] in TigerGraph Cloud 4. | ||
|
||
Or return to the xref:cloud4:overview:index.adoc[Overview] page for a different topic. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
109 changes: 109 additions & 0 deletions
109
modules/cloud4/modules/load-data/pages/load-from-s3.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
= Load from AWS S3 | ||
:experimental: | ||
|
||
If you store your data in AWS S3, TigerGraph Cloud provides seamless integration for data ingestion. | ||
You can directly load data from your S3 buckets into your graph databases, eliminating the need for manual data transfers. | ||
This simplifies the process of importing large datasets and enables you to leverage the scalability and durability of AWS S3 for your graph analysis. | ||
|
||
== 1) Select Source | ||
|
||
.Once you’ve selected AWS S3, you will be asked to configure the AWS S3 data source. | ||
. Click on image:Screenshot 2024-04-17 at 9.36.58 PM.png[width=50] to add a new AWS S3 data source. | ||
+ | ||
image:Screenshot 2024-04-17 at 9.36.27 PM.png[] | ||
|
||
. You will need to provide your S3 `AWS access key id` and S3 `AWS secret access key`. | ||
+ | ||
image:Screenshot 2024-04-17 at 9.37.32 PM.png[] | ||
. Once you have those configured, you can add one or multiple `S3 URI` within the same S3 bucket. | ||
|
||
. Click btn:[ Next ] to process the file(s). | ||
+ | ||
[NOTE] | ||
==== | ||
The current data loading tool only supports CSV files. Other formats will be available in later releases. | ||
==== | ||
|
||
== 2) Configure File | ||
.This step lets you configure the source file details. | ||
. The data loading tool will automatically detect the `.csv` separators and line breaks. | ||
The parser automatically splits each line into a series of tokens. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.53.45 PM.png[] | ||
+ | ||
[TIP] | ||
==== | ||
If the parsing is *not* correct, click on the image:Screenshot 2024-04-17 at 5.54.17 PM.png[width=75] | ||
button to configure a different option for the delimiter, such as `eol`, or `quote and header`. | ||
image:Screenshot 2024-04-17 at 5.54.50 PM.png[] | ||
==== | ||
+ | ||
The enclosing character is used to mark the boundaries of a token, overriding the delimiter character. | ||
+ | ||
==== | ||
For example, if your delimiter is a comma, but you have commas in some strings, then you can define single or double quotes as the enclosing character to mark the endpoints of your string tokens. | ||
==== | ||
+ | ||
[NOTE] | ||
==== | ||
It is not necessary for every token to have enclosing characters. The parser uses enclosing characters when it encounters them. | ||
==== | ||
+ | ||
[TIP] | ||
==== | ||
You can edit the header line of the parsing result to give each column a more intuitive name, since you will be referring to these names when loading data to the graph. | ||
The header name is ignored during data loading. | ||
==== | ||
|
||
. Once you are satisfied with the file settings, click btn:[ Next ] to proceed. | ||
|
||
== 3) Configure Map | ||
|
||
.If you are loading data into a brand new graph, you will be prompted to let our engine generate a schema and mapping for you. Or you can start from scratch. For more details of schema design please refer to xref:cloud4:schema-designer:index.adoc[Schema Designer]. | ||
. Select `Generate the schema only` or `Generate the schema and data mapping`. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.55.35 PM.png[] | ||
+ | ||
[CAUTION] | ||
==== | ||
The schema generation feature is still a preview feature. The correctness and efficiency of the resulting graph schema and mapping could vary. | ||
==== | ||
|
||
. In the `Source` column, you can choose the specific column from the data source that you want to map with the attribute. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.56.15 PM.png[] | ||
+ | ||
. Use the `+` button to create a new attribute of the target vertex or edge. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.57.07 PM.png[] | ||
|
||
. The `Map all to target` button aligns existing attribute names with the corresponding data source headers to establish mappings and also introduces new attributes based on unmatched data source headers. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.57.52 PM.png[] | ||
. Click btn:[Next] to proceed. | ||
|
||
|
||
|
||
== 4) Confirm | ||
|
||
.This step will let you confirm the changes made to the schema and the data mapping you created to load the data. | ||
. Simply review the `Schema to be change` and `Data to be loaded` lists. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.58.36 PM.png[] | ||
+ | ||
[CAUTION] | ||
==== | ||
Please be aware that some schema changes will result in unintentional deletion of the data. Please carefully review the warning message before confirming the loading. | ||
==== | ||
. Click on the btn:[Confirm] button to run the loading jobs and monitor their `Status`. | ||
+ | ||
image::Screenshot 2024-04-17 at 5.59.16 PM.png[] | ||
|
||
== Next Steps | ||
|
||
Next, learn how to use the xref:cloud4:schema-designer:index.adoc[Schema Designer] in TigerGraph Cloud 4. | ||
|
||
Or return to the xref:cloud4:overview:index.adoc[Overview] page for a different topic. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.