diff --git a/README.md b/README.md index 23b1dd54..66fc86da 100644 --- a/README.md +++ b/README.md @@ -1,211 +1,76 @@ -# What is it? +# Morph-xR2RML: MongoDB-to-RDF translation Morph-xR2RML is an implementation of the [xR2RML mapping language](http://i3s.unice.fr/~fmichel/xr2rml_specification.html) that enables the description of mappings from relational or non relational databases to RDF. xR2RML is an extension of [R2RML](http://www.w3.org/TR/r2rml/) and [RML](http://semweb.mmlab.be/rml/spec.html). Morph-xR2RML comes with connectors for relational databases (MySQL, PostgreSQL, MonetDB) and the MongoDB NoSQL document store. Two running modes are available: -- the *graph materialization* mode creates all possible RDF triples at once. +- the *graph materialization* mode creates all possible RDF triples at once, it is a production class - the *query rewriting* mode translates a SPARQL 1.0 query into a target database query and returns a SPARQL answer. It can run as a SPARQL 1.0 endpoint or as a stand-alone application. - -Morph-xR2RML was developed by the [I3S laboratory](http://www.i3s.unice.fr/) as an extension of the [Morph-RDB project](https://github.com/oeg-upm/morph-rdb) which is an implementation of R2RML. It is made available under the Apache 2.0 License. - -#### SPARQL-to-SQL -The SPARQL-to-SQL rewriting is an adaptation of the former Morph-RDB implementation, it supports SPARQL SELECT and DESCRIBE queries. - -#### SPARQL-to-MongoDB -The SPARQL-to-MongoDB rewriting is a fully new component, it supports the SELECT, ASK, CONSTRUCT and DESCRIBE query forms. - - -## Publications - -F. Michel, L. Djimenou, C. Faron-Zucker, and J. Montagnat. Translation of Relational and Non-Relational Databases into RDF with xR2RML. -In Proceedings of the *11th International Confenrence on Web Information Systems and Technologies (WEBIST 2015)*, Lisbon, Portugal, 2015. [HAL](https://hal.science/hal-01207828) - -F. Michel, C. Faron-Zucker, and J. Montagnat. A Generic Mapping-Based Query Translation from SPARQL to Various Target Database Query Languages. -In Proceedings of the *12th International Confenrence on Web Information Systems and Technologies (WEBIST 2016)*, Roma, Italy, 2016. [HAL](https://hal.science/hal-01280951) - -F. Michel, C. Faron-Zucker, and J. Montagnat. A Mapping-Based Method to Query MongoDB Documents with SPARQL. In *27th International Conference on Database and Expert Systems Applications (DEXA 2016)*, 2016. [HAL](https://hal.science/hal-01330146) - - - -## Limitations - -##### xR2RML Language support -- The generation of RDF collection and containers is supported in all cases (from a list of values resulting of the evaluation of a mixed syntax path typically, from the result of a join query implied by a referencing object map), except in the case of a regular R2RML join query applied to a relational database: the result of the join SQL query cannot be translated into an RDF collection or container. -- Named graphs are supported although they are not printed out in Turtle which does not support named graphs. It would be quite easy to extend it with a N-Quad or Trig serialization to allow for writing triples in named graphs. - -The former limitation on NestedTermMaps was lifted in Sept. 2017. All types of NestedTermMaps are now fully implemented, so that any complex iterations and collection/container nesting can be defined. - - -##### Query rewriting -The query rewriting is implemented for RDBs and MongoDB, with the restriction that _no mixed syntax paths be used_. Doing query rewriting with mixed syntax paths is a much more complex problem, that may not be possible in all situations (it would require to "revert" expressions such as JSONPath or XPath to retrieve source data base values). - -Only one join condition is supported in a referencing object map. - ----------- - -# Code description - -See a detailed [description of the project code and architecture](doc/README_code_architecture.md). - ----------- - -# Want to try it? - -### Download, Build - -Pre-requisite: have **Java SDK 10** installed - -You can download the last release or snapshot published in [this repository](https://www.dropbox.com/sh/djnztipsclvcskw/AABT1JagzD4K4aCALDNVj-yra?dl=0). -The latest on-going version is the 1.3.2 snapshot. - -Alternatively, you can build the application using [Maven](http://maven.apache.org/): in a shell, CD to the root directory morph-xr2rml, then run the command: `mvn clean package`. A jar with all dependencies is generated in `morph-xr2rml-dist/target`. - - -### Run it - -The application takes two options: `--configDir` gives the configuration directory and `--configFile` give the configuration file within this directory. Option `--configFile` defaults to `morph.properties`. - -Additionally, several parameter given in the configuration file can be overridden using the following options: -- mapping file: `--mappingFile` -- output file : `--output` -- maximum number of triples generated in a single output file: `--outputMaxTriples` + - The SPARQL-to-SQL rewriting is an adaptation of the former Morph-RDB implementation, it supports SPARQL SELECT and DESCRIBE queries. + - The SPARQL-to-MongoDB rewriting is a new component, it supports the SELECT, ASK, CONSTRUCT and DESCRIBE query forms. -**From a command line interface**, CD to directory morph-xr2rml-dist and run the application as follows: - -``` -java -jar target/morph-xr2rml-dist--jar-with-dependencies.jar \ - --configDir \ - --configFile -``` - -Besides, the logger configuration can be overriden by passing the `log4j.configuration` parameter to the JVM: - -``` -java -Dlog4j.configuration=file:/path/to/my/log4j.configuration -jar ... -``` - -**From an IDE** such as Eclipse or IntelliJ: In project morph-xr2rml-dist locate main class `fr.unice.i3s.morph.xr2rml.engine.MorphRunner`, and run it as a Scala application with arguments `--configDir` and `--configFile`. - -### SPARQL endpoint - -To run Morph-xR2RML as a SPARQL endpoint, simply edit the configuration file (see reference) and set the property `sever.active=true`. The default access URL is: -``` -http://localhost:8080/sparql -``` -Property `query.file.path` is ignored and queries can be submitted using either HTTP GET or POST methods as described in the [SPARQL protocol](https://www.w3.org/TR/rdf-sparql-protocol/) recommendation. - -For SPARQL SELECT and ASK queries, the XML, JSON, CSV and TSV serializations are supported. - -For SPARQL DESCRIBE and CONSTRUCT queries, the supported serializations are RDF/XML, N-TRIPLE, N-QUAD, TURTLE, N3 and JSON-LD. - -### Examples for MongoDB - -In directories `morph-xr2rml-dist/example_mongo` and `morph-xr2rml-dist/example_mongo_rewriting` we provide example databases and corresponding mappings. Directory `example_mongo` runs the graph materialization mode, `example_mongo_rewriting` runs the query rewriting mode. - -- `testdb_dump.json` is a dump of the MongoDB test database: copy and paste the content of that file into a MongoDB shell window to create the database; -- `morph.properties` provides database connection details; -- `mapping1.ttl` to `mapping4.ttl` contain xR2RML mapping graphs illustrating various features of the language; -- `result1.txt` to `result4.txt` contain the expected result of the mappings 1 to 4; -- `query.sparql` (in directory `example_mongo_rewriting` only) contains a SPARQL query to be executed against the test database. - -Edit `morph.properties` and change the database URL, name, user and password with appropriate values. - -> _**Note about query optimization**_: the xR2RML xrr:uniqueRef notation is of major importance for query optimization as it allows for self-joins elimination. Check example in `morph-xr2rml-dist/example_taxref_rewriting`. - -### Examples for MySQL - -In directories `morph-xr2rml-dist/example_mysql` and `morph-xr2rml-dist/example_mysql_rewriting` we provide example databases and corresponding mappings. Directory `example_mysql` runs the graph materialization mode, `example_mysql_rewriting` runs the query rewriting mode. - -- `testdb_dump.sql` is a dump of the MySQL test database. You may import it into a MySQL instance by running command `mysql -u root -p test < testdb_dump.sql`; -- `morph.properties` provides database connection details; -- `mapping.ttl` contains an example xR2RML mapping graph; -- `result.txt` contains the expected result of applying this mapping to that database; -- `query.sparql` (in directory `example_mysql_rewriting` only) contains a SPARQL query to be executed against the test database. - -Edit `morph.properties` and change the database url, name, user and password with appropriate values. - ----------- - -# Configuration file reference -``` -# -- xR2RML mapping file (Mandatory): -# path relative to the configuration directory given in parameter --configDir -mappingdocument.file.path=mapping1.ttl - -# -- Server mode: true|false. Default: false -# false: stand-alone application that performs either graph materialization or query rewriting -# true: SPARQL endpoint with query rewriting -server.active=false +Morph-xR2RML was developed by the [I3S laboratory](http://www.i3s.unice.fr/) as an extension of the [Morph-RDB project](https://github.com/oeg-upm/morph-rdb) which is an implementation of R2RML. It is made available under the Apache 2.0 License. -# -- Server port number, ignored when "server.active=false". Default: 8080 -server.port=8080 -# -- Processing result output file, relative to --configDir. Default: result.txt -output.file.path=result.txt +### Maturity -# -- Max number of triples to generate in output file. Default: 0 (no limit) -# If the max number is reached, file name is suffixed with an index e.g. result.txt.0, result.txt.1, result.txt.2 etc. -output.file.max_triples=0 +The *graph materialization* mode is mature and has been used in several projects to generate large amounts of RDF triples (-1.3M triples in [Covid-on-the-Web](https://github.com/Wimmics/CovidOnTheWeb/)). -# -- Output RDF syntax: RDF/XML|N-TRIPLE|TURTLE|N3|JSON-LD. Default: TURTLE -# Applies to the graph materialization and the rewriting of SPARQL CONSTRUCT and DESCRIBE queries -output.syntax.rdf=TURTLE +The *query rewriting* mode is a prototype implementation. It was meant to demonstrate the effectiveness of a rewirting method but is not meant for production environment. -# -- Output syntax for SPARQL result set (SPARQL SELECT and ASK queries): XML|JSON|CSV|TSV. Default: XML -# When "server.active = true", this may be overridden by the Accept HTTP header of the request -output.syntax.result=XML -# -- Display the result on the std output after the processing: true|false. Default: true -output.display=false +## Quick start guide -# -- File containing the SPARQL query to process, relative to --configDir. Default: none. -# Ignored when "server.active = true" -query.file.path=query.sparql +The easiest to start using Morph-xR2RML to materialize a graph from a MongoDB database is by using Docker. Follow [these instructions](docker/README.md). -# -- Database connection type and configuration -no_of_database=1 -database.type[0]=MongoDB -database.driver[0]= -database.url[0]=mongodb://127.0.0.1:27017 -database.name[0]=test -database.user[0]=user -database.pwd[0]=user +## Documentation -# -- Reference formulation: Column|JSONPath|XPath. Default: Column -database.reference_formulation[0]=JSONPath +- [Deploy with Docker](docker/README.md) +- [Build and install](doc/build.md) +- [Running examples](doc/examples.md) +- [Configuration reference](doc/config.md) +- [Architecture and code description](doc/README_code_architecture.md). +- [Limitations](doc/limitations.md) -# -- Runner factory. Mandatory. -# For MongoDB: fr.unice.i3s.morph.xr2rml.mongo.engine.MorphJsondocRunnerFactory -# For RDBs: es.upm.fi.dia.oeg.morph.rdb.engine.MorphRDBRunnerFactory -runner_factory.class.name=fr.unice.i3s.morph.xr2rml.mongo.engine.MorphMongoRunnerFactory +## Publications -# -- URL-encode reserved chars in database values. Default: true -# uricolumn.encode_unsafe_chars_dbvalues=true +F. Michel, L. Djimenou, C. Faron-Zucker, and J. Montagnat. Translation of Relational and Non-Relational Databases into RDF with xR2RML. +In Proceedings of the *11th International Confenrence on Web Information Systems and Technologies (WEBIST 2015)*, Lisbon, Portugal, 2015. [HAL](https://hal.science/hal-01207828) -# -- URL-encode reserved chars IRI template string. Default: true -# uricolumn.encode_uri=true +F. Michel, C. Faron-Zucker, and J. Montagnat. A Generic Mapping-Based Query Translation from SPARQL to Various Target Database Query Languages. +In Proceedings of the *12th International Confenrence on Web Information Systems and Technologies (WEBIST 2016)*, Roma, Italy, 2016. [HAL](https://hal.science/hal-01280951) +F. Michel, C. Faron-Zucker, and J. Montagnat. A Mapping-Based Method to Query MongoDB Documents with SPARQL. In *27th International Conference on Database and Expert Systems Applications (DEXA 2016)*, 2016. [HAL](https://hal.science/hal-01330146) -# -- Cache the result of previously executed queries for MongoDB. Default: false -# Caution: high memory consumption, to be used for RefObjectMaps only -querytranslator.cachequeryresult=false +## Cite this work: -# -- Primary SPARQL query optimization. Default: true -querytranslator.sparql.optimize=true +Either cite one of the papers above or cite the software itself as this with its SWHID: -# -- Abstract query optimization: self join elimination. Default: true -querytranslator.abstract.selfjoinelimination=true +**Full text**: -# -- Abstract query optimization: self union elimination. Default: true -querytranslator.abstract.selfunionelimination=true +Franck Michel. Morph-xR2RML: MongoDB-to-RDF translation. 2015, ⟨swh:1:dir:8ea716c0d9e69527a5f50378bf135c5952b1a229⟩. ⟨hal-04128090⟩ -# -- Abstract query optimization: propagation of conditions in a inner/left join. Default: true -querytranslator.abstract.propagateconditionfromjoin=true +**Bibtex**: +```bibtex +@softwareversion{michel:hal-04128090v1, + TITLE = {{Morph-xR2RML: MongoDB-to-RDF translation}}, + AUTHOR = {Michel, Franck}, + URL = {https://hal.science/hal-04128090}, + NOTE = {}, + INSTITUTION = {{University C{\^o}te d'Azur ; CNRS ; Inria}}, + YEAR = {2015}, + MONTH = Apr, + SWHID = {swh:1:dir:8ea716c0d9e69527a5f50378bf135c5952b1a229}, + VERSION = {1.3.1}, + REPOSITORY = {https://github.com/frmichel/morph-xr2rml}, + LICENSE = {Apache License 2.0}, + KEYWORDS = {knowledge graph ; RDF ; mapping ; SPARQL ; MongoDB}, + HAL_ID = {hal-04128090}, + HAL_VERSION = {v1}, +} ``` - diff --git a/doc/build.md b/doc/build.md new file mode 100644 index 00000000..26e3e3a2 --- /dev/null +++ b/doc/build.md @@ -0,0 +1,49 @@ +# Build and install + +## Download & Build + +Pre-requisite: have **Java SDK 10** installed + +You can download the last release or snapshot published in [this repository](https://www.dropbox.com/sh/djnztipsclvcskw/AABT1JagzD4K4aCALDNVj-yra?dl=0). +The latest on-going version is the 1.3.2 snapshot. + +Alternatively, you can build the application using [Maven](http://maven.apache.org/): in a shell, CD to the root directory morph-xr2rml, then run the command: `mvn clean package`. A jar with all dependencies is generated in `morph-xr2rml-dist/target`. + + +## Run it + +The application takes two options: `--configDir` gives the configuration directory and `--configFile` give the configuration file within this directory. Option `--configFile` defaults to `morph.properties`. + +Additionally, several parameter given in the configuration file can be overridden using the following options: +- mapping file: `--mappingFile` +- output file : `--output` +- maximum number of triples generated in a single output file: `--outputMaxTriples` + +**From a command line interface**, CD to directory morph-xr2rml-dist and run the application as follows: + +``` +java -jar target/morph-xr2rml-dist--jar-with-dependencies.jar \ + --configDir \ + --configFile +``` + +Besides, the logger configuration can be overriden by passing the `log4j.configuration` parameter to the JVM: + +``` +java -Dlog4j.configuration=file:/path/to/my/log4j.configuration -jar ... +``` + +**From an IDE** such as Eclipse or IntelliJ: In project morph-xr2rml-dist locate main class `fr.unice.i3s.morph.xr2rml.engine.MorphRunner`, and run it as a Scala application with arguments `--configDir` and `--configFile`. + + +## SPARQL endpoint + +To run Morph-xR2RML as a SPARQL endpoint, simply edit the configuration file (see reference) and set the property `sever.active=true`. The default access URL is: +``` +http://localhost:8080/sparql +``` +Property `query.file.path` is ignored and queries can be submitted using either HTTP GET or POST methods as described in the [SPARQL protocol](https://www.w3.org/TR/rdf-sparql-protocol/) recommendation. + +For SPARQL SELECT and ASK queries, the XML, JSON, CSV and TSV serializations are supported. + +For SPARQL DESCRIBE and CONSTRUCT queries, the supported serializations are RDF/XML, N-TRIPLE, N-QUAD, TURTLE, N3 and JSON-LD. \ No newline at end of file diff --git a/doc/config.md b/doc/config.md new file mode 100644 index 00000000..941aa02e --- /dev/null +++ b/doc/config.md @@ -0,0 +1,84 @@ +# Configuration file reference + +Below if the default configuration file, morph.properties: + +``` +# -- xR2RML mapping file (Mandatory): +# path relative to the configuration directory given in parameter --configDir +mappingdocument.file.path=mapping1.ttl + +# -- Server mode: true|false. Default: false +# false: stand-alone application that performs either graph materialization or query rewriting +# true: SPARQL endpoint with query rewriting +server.active=false + +# -- Server port number, ignored when "server.active=false". Default: 8080 +server.port=8080 + +# -- Processing result output file, relative to --configDir. Default: result.txt +output.file.path=result.txt + +# -- Max number of triples to generate in output file. Default: 0 (no limit) +# If the max number is reached, file name is suffixed with an index e.g. result.txt.0, result.txt.1, result.txt.2 etc. +output.file.max_triples=0 + +# -- Output RDF syntax: RDF/XML|N-TRIPLE|TURTLE|N3|JSON-LD. Default: TURTLE +# Applies to the graph materialization and the rewriting of SPARQL CONSTRUCT and DESCRIBE queries +output.syntax.rdf=TURTLE + +# -- Output syntax for SPARQL result set (SPARQL SELECT and ASK queries): XML|JSON|CSV|TSV. Default: XML +# When "server.active = true", this may be overridden by the Accept HTTP header of the request +output.syntax.result=XML + +# -- Display the result on the std output after the processing: true|false. Default: true +output.display=false + +# -- File containing the SPARQL query to process, relative to --configDir. Default: none. +# Ignored when "server.active = true" +query.file.path=query.sparql + +# -- Database connection type and configuration +no_of_database=1 +database.type[0]=MongoDB +database.driver[0]= +database.url[0]=mongodb://127.0.0.1:27017 +database.name[0]=test +database.user[0]=user +database.pwd[0]=user + + +# -- Reference formulation: Column|JSONPath|XPath. Default: Column +database.reference_formulation[0]=JSONPath + +# -- Runner factory. Mandatory. +# For MongoDB: fr.unice.i3s.morph.xr2rml.mongo.engine.MorphJsondocRunnerFactory +# For RDBs: es.upm.fi.dia.oeg.morph.rdb.engine.MorphRDBRunnerFactory +runner_factory.class.name=fr.unice.i3s.morph.xr2rml.mongo.engine.MorphMongoRunnerFactory + + +# -- URL-encode reserved chars in database values. Default: true +# uricolumn.encode_unsafe_chars_dbvalues=true + +# -- URL-encode reserved chars IRI template string. Default: true +# uricolumn.encode_uri=true + + +# -- Cache the result of previously executed queries for MongoDB. Default: false +# Caution: high memory consumption, to be used for RefObjectMaps only +querytranslator.cachequeryresult=false + + +# -- Primary SPARQL query optimization. Default: true +querytranslator.sparql.optimize=true + +# -- Abstract query optimization: self join elimination. Default: true +querytranslator.abstract.selfjoinelimination=true + +# -- Abstract query optimization: self union elimination. Default: true +querytranslator.abstract.selfunionelimination=true + +# -- Abstract query optimization: propagation of conditions in a inner/left join. Default: true +querytranslator.abstract.propagateconditionfromjoin=true + +``` + diff --git a/doc/examples.md b/doc/examples.md new file mode 100644 index 00000000..90028fe8 --- /dev/null +++ b/doc/examples.md @@ -0,0 +1,27 @@ +# Running examples + +## Examples for MongoDB + +In directories `morph-xr2rml-dist/example_mongo` and `morph-xr2rml-dist/example_mongo_rewriting` we provide example databases and corresponding mappings. Directory `example_mongo` runs the graph materialization mode, `example_mongo_rewriting` runs the query rewriting mode. + +- `testdb_dump.json` is a dump of the MongoDB test database: copy and paste the content of that file into a MongoDB shell window to create the database; +- `morph.properties` provides database connection details; +- `mapping1.ttl` to `mapping4.ttl` contain xR2RML mapping graphs illustrating various features of the language; +- `result1.txt` to `result4.txt` contain the expected result of the mappings 1 to 4; +- `query.sparql` (in directory `example_mongo_rewriting` only) contains a SPARQL query to be executed against the test database. + +Edit `morph.properties` and change the database URL, name, user and password with appropriate values. + +> _**Note about query optimization**_: the xR2RML xrr:uniqueRef notation is of major importance for query optimization as it allows for self-joins elimination. Check example in `morph-xr2rml-dist/example_taxref_rewriting`. + +## Examples for MySQL + +In directories `morph-xr2rml-dist/example_mysql` and `morph-xr2rml-dist/example_mysql_rewriting` we provide example databases and corresponding mappings. Directory `example_mysql` runs the graph materialization mode, `example_mysql_rewriting` runs the query rewriting mode. + +- `testdb_dump.sql` is a dump of the MySQL test database. You may import it into a MySQL instance by running command `mysql -u root -p test < testdb_dump.sql`; +- `morph.properties` provides database connection details; +- `mapping.ttl` contains an example xR2RML mapping graph; +- `result.txt` contains the expected result of applying this mapping to that database; +- `query.sparql` (in directory `example_mysql_rewriting` only) contains a SPARQL query to be executed against the test database. + +Edit `morph.properties` and change the database url, name, user and password with appropriate values. diff --git a/doc/limitations.md b/doc/limitations.md new file mode 100644 index 00000000..6ff4c10d --- /dev/null +++ b/doc/limitations.md @@ -0,0 +1,13 @@ +# Limitations + +### xR2RML Language support +- The generation of RDF collection and containers is supported in all cases (from a list of values resulting of the evaluation of a mixed syntax path typically, from the result of a join query implied by a referencing object map), except in the case of a regular R2RML join query applied to a relational database: the result of the join SQL query cannot be translated into an RDF collection or container. +- Named graphs are supported although they are not printed out in Turtle which does not support named graphs. It would be quite easy to extend it with a N-Quad or Trig serialization to allow for writing triples in named graphs. + +The former limitation on NestedTermMaps was lifted in Sept. 2017. All types of NestedTermMaps are now fully implemented, so that any complex iterations and collection/container nesting can be defined. + + +### Query rewriting +The query rewriting is implemented for RDBs and MongoDB, with the restriction that _no mixed syntax paths be used_. Doing query rewriting with mixed syntax paths is a much more complex problem, that may not be possible in all situations (it would require to "revert" expressions such as JSONPath or XPath to retrieve source data base values). + +Only one join condition is supported in a referencing object map. diff --git a/docker/README.md b/docker/README.md index 6b536d07..f28cada2 100644 --- a/docker/README.md +++ b/docker/README.md @@ -2,15 +2,24 @@ You can deploy Morph-xR2RML along with a MongoDB database using Docker. -First download file [xr2rml_docker.zip](xr2rml_docker.zip), unzip it to the directory from where you will run docker-compose, and set file access rights as shown below: +First make sure your have [docker-compose](https://docs.docker.com/compose/) or [install it](https://docs.docker.com/compose/install/linux/). + +If you have cloned this repository, just skip the next step. Otherwise, download file [xr2rml_docker.zip](xr2rml_docker.zip), and unzip it to the directory from where you will run docker-compose. From a Linux terminal that would be: ```bash +wget https://github.com/frmichel/morph-xr2rml/raw/master/docker/xr2rml_docker.zip unzip xr2rml_docker.zip +``` + +Set file access rights as shown below: + +```bash chmod -R 755 mongo_import chmod -R 755 xr2rml_config chmod -R 777 log chmod -R 777 mongo_db chmod -R 777 xr2rml_output +chmod 777 run.sh ``` Then run: @@ -31,11 +40,11 @@ Script `run.sh` provides an example of how to (1) import data located in `mongo_ - `mongo_import`: copy your files to import in this folder, it is mouned in the MongoDB container. - `xr2rml_config`: mapping files, morph.properties, log configuration file, and bash scripts to run Morph-xR2RML. This folder is mouned in the Morph-xR2RML container. - `xr2rml_config`: where the RDF files will be written. -- `run.sh`: example script to show you how to import data into MongoDB and run Morph-xR2RML to translate the data to RDF. +- `run.sh`: example script showing how to import data into MongoDB and run Morph-xR2RML to translate the data to RDF. -### Accessing Logs +### Accessing Logs The `docker-compose.yml` mounts the Morph-xR2RML log directory to the Docker host in directory `log`. @@ -62,3 +71,13 @@ Scripts `mongo_tools/import-file.sh` and `mongo_tools/import-json-files.sh` prov ### log or mongo_db directory not writable The containers need to write in directories ```log``` and ```mongo_db```. They will fail if you do not set rights 777 (`chmod 777 `). + +### SLF4J messages at Morph-xR2RML startup + +When it starts up, Morph-xR2RML shows the messages below. They are just warning, you can safely ignore them. + +``` +SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". +SLF4J: Defaulting to no-operation (NOP) logger implementation +SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. +``` diff --git a/docker/build/docker-compose.yml b/docker/build/docker-compose.yml new file mode 100644 index 00000000..e38e0532 --- /dev/null +++ b/docker/build/docker-compose.yml @@ -0,0 +1,28 @@ +version: '3.2' +services: + + morph-xr2rml: + image: frmichel/morph-xr2rml + container_name: morph-xr2rml + networks: + - xr2rml-net + volumes: + - "./xr2rml_config:/xr2rml_config" + - "./xr2rml_output:/xr2rml_output" + - "./log:/log" + build: + context: ./morph-xr2rml + + mongo-xr2rml: + image: mongo:4.4.6 + container_name: mongo-xr2rml + networks: + - xr2rml-net + volumes: + - "./mongo_db:/data/db" + - "./mongo_tools:/mongo_tools" + - "./mongo_import:/mongo_import" + +networks: + xr2rml-net: + driver: bridge diff --git a/docker/build/morph-xr2rml/Dockerfile b/docker/build/morph-xr2rml/Dockerfile new file mode 100644 index 00000000..a248cf10 --- /dev/null +++ b/docker/build/morph-xr2rml/Dockerfile @@ -0,0 +1,22 @@ +FROM openjdk:11-jdk-bullseye + +RUN apt-get update -y + +ENV XR2RML="/morph-xr2rml" +RUN mkdir -p $XR2RML +WORKDIR "$XR2RML" + +COPY morph-xr2rml-dist-1.3.2-20211126.142114-3-jar-with-dependencies.jar $XR2RML + +ENV CONFIG="$XR2RML/config" +RUN mkdir -p $CONFIG + +ENV OUTPUT="$XR2RML/output" +RUN mkdir -p $OUTPUT + +ENV LOG="$XR2RML/log" +RUN mkdir -p $LOG + +# Morph-xR2RML is not run at container startup, only when the run_xr2rml.sh script is run from outside. +# So instead just use sleep so that the container never kicks out. +CMD ["sleep", "infinity"] diff --git a/docker/build/morph-xr2rml/morph-xr2rml-dist-1.3.2-20211126.142114-3-jar-with-dependencies.jar b/docker/build/morph-xr2rml/morph-xr2rml-dist-1.3.2-20211126.142114-3-jar-with-dependencies.jar new file mode 100644 index 00000000..c0b0a4a6 Binary files /dev/null and b/docker/build/morph-xr2rml/morph-xr2rml-dist-1.3.2-20211126.142114-3-jar-with-dependencies.jar differ diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index d8111250..4def05e5 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -3,6 +3,7 @@ services: morph-xr2rml: image: frmichel/morph-xr2rml + container_name: morph-xr2rml networks: - xr2rml-net volumes: @@ -11,7 +12,8 @@ services: - "./log:/log" mongo-xr2rml: - image: mongo:3.7 + image: mongo:4.4.6 + container_name: mongo-xr2rml networks: - xr2rml-net volumes: diff --git a/docker/mongo_import/movies.json b/docker/mongo_import/movies.json new file mode 100644 index 00000000..5ddfd7b6 --- /dev/null +++ b/docker/mongo_import/movies.json @@ -0,0 +1,7 @@ +{ "id":"1", "title": "Manhattan", "year": "1979" } +{ "id":"2", "title": "Annie Hall", "year": "1977" } +{ "id":"3", "title": "2046", "year": "2004" } +{ "id":"4", "title": "In the Mood for Love", "year": "2000" } +{ "id":"5", "title": "West Side Story", "year": "1962" } + + diff --git a/docker/mongo_tools/import-file.sh b/docker/mongo_tools/import-file.sh new file mode 100644 index 00000000..f72865fb --- /dev/null +++ b/docker/mongo_tools/import-file.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# Authors: Franck MICHEL, University Cote d'Azur, CNRS, Inria +# Anna BOBASHEVA, University Cote d'Azur, Inria +# +# Import into MongoDb a file (json/csv/tsv) located in directory /mongo_import; create an index, +# and optionally run a js script in MongoDB +# +# Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) + + +# Path (inside the contaier) of the files to import +idir=/mongo_import + +help() +{ + exe=$(basename $0) + echo "Usage: $exe " + echo "Example:" + echo " $exe path/file.csv csv database collection index" + echo " $exe path/file.csv csv database collection index script.js" + exit 1 +} + +# --- Read input arguments +file=$1 +if [[ -z "$file" ]] ; then help; fi + +file_type=$2 +if [[ -z "$file_type" ]] ; then help; fi + +database=$3 +if [[ -z "$database" ]] ; then help; fi + +collection=$4 +if [[ -z "$collection" ]] ; then help; fi + +index_col=$5 +if [[ -z "$index_col" ]] ; then help; fi + +# Optional +extra_script=$5 + + +# Functions definitions +. ./import-tools.sh + +echo "Importing from $idir/$file to $collection in $database..." +echo "------------------------------------------------------------------------------" + +mongo_drop_import_file_index $idir/$file $file_type $database $collection $index_col + +if [[ ! -z "$extra_script" ]] ; then + + echo "Executing additional script $extra_script..." + echo "------------------------------------------------------------------------------" + + mongo localhost/$database "$extra_script" #--verbose +fi diff --git a/docker/mongo_tools/import-json-files.sh b/docker/mongo_tools/import-json-files.sh new file mode 100644 index 00000000..bf02c60b --- /dev/null +++ b/docker/mongo_tools/import-json-files.sh @@ -0,0 +1,53 @@ +#!/bin/bash +# Authors: Franck MICHEL, University Cote d'Azur, CNRS, Inria +# Anna BOBASHEVA, University Cote d'Azur, Inria +# +# Import into MongoDb multiple json files located in directory /mongo_import; create an index, +# and optionally run a js script in MongoDB +# +# Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) + + +# Path (inside the contaier) of the files to import +idir=/mongo_import + +help() +{ + exe=$(basename $0) + echo "Usage: $exe " + echo "Examples:" + echo " $exe database collection index" + echo " $exe database collection index script.js" + exit 1 +} + +# --- Read input arguments +database=$1 +if [[ -z "$database" ]] ; then help; fi + +collection=$2 +if [[ -z "$collection" ]] ; then help; fi + +index_col=$3 +if [[ -z "$index_col" ]] ; then help; fi + + +# Optional +extra_script=$5 + + +# Functions definitions +. ./import-tools.sh + +echo "Importing from $idir to $collection in $database..." +echo "------------------------------------------------------------------------------" + +mongo_drop_import_dir_index $idir $database $collection $index_col + +if [[ ! -z "$extra_script" ]] ; then + + echo "Executing additional script $extra_script..." + echo "------------------------------------------------------------------------------" + + mongo localhost/$database "$extra_script" #--verbose +fi diff --git a/docker/mongo_tools/import-tools.sh b/docker/mongo_tools/import-tools.sh new file mode 100644 index 00000000..e98f54b8 --- /dev/null +++ b/docker/mongo_tools/import-tools.sh @@ -0,0 +1,193 @@ +#!/bin/bash +# Author: Franck MICHEL, University Cote d'Azur, CNRS, Inria +# Anna BOBASHEVA, University Cote d'Azur, Inria +# +# Set of generic tools meant to import data into a MongoDB database. +# +# Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) + + +# Max sie of JSON files to import at once into MongoDB (16MB) +MONGO_IMPORT_MAXSIZE=16000000 + + +# Import, into a MongoDB collection, the JSON files listed in a file. +# Files are imported one by one. +# $1: MongoDB database name +# $2: MongoDB collection name +# $2: file containing the list of JSON files to import +mongo_import_filelist_onebyone() { + _database_a=$1 + _collection_a=$2 + _filelist_a=$3 + index=0 + + for jsonfile in `cat $_filelist_a`; do + + filesize=$(stat --format=%s $jsonfile) + if [ $filesize -ge $MONGO_IMPORT_MAXSIZE ]; then + echo "WARNING - Ignoring oversized document $jsonfile ($filesize bytes)" + else + echo "Importing file $index: $jsonfile" + mongoimport --type=json -d $_database_a -c $_collection_a $jsonfile + fi + index=$(($index + 1)) + done +} + + +# Import, into a MongoDB collection, the JSON files listed in a file. +# Files are grouped until a max size limit (see $MONGO_IMPORT_MAXSIZE) +# $1: MongoDB database name +# $2: MongoDB collection name +# $3: file containing the list of JSON files to import +mongo_import_filelist() { + _database_a=$1 + _collection_a=$2 + _filelist_a=$3 + + # Import the files by groups + jsondump=jsondump-$$ + echo -n '' > $jsondump + + for jsonfile in `cat $_filelist_a`; do + + filesize=$(stat --format=%s $jsonfile) + if [ $filesize -ge $MONGO_IMPORT_MAXSIZE ]; then + echo "WARNING - Ignoring oversized document $jsonfile ($filesize bytes)" + else + currentsize=$(stat --format=%s $jsondump) + newsize=$(($currentsize + $filesize)) + if [ $newsize -gt $MONGO_IMPORT_MAXSIZE ]; then + echo "Importing documents from $jsondump" + mongoimport --type=json -d $_database_a -c $_collection_a $jsondump + echo -n '' > $jsondump + fi + + echo "Appending to $jsondump document $jsonfile" + cat $jsonfile >> $jsondump + + fi + done + + # Import the last group that could be less than $MONGO_IMPORT_MAXSIZE + if [ -s $jsondump ]; then + echo "Importing last documents from $jsondump" + mongoimport --type=json -d $_database_a -c $_collection_a $jsondump + echo -n '' > $jsondump + fi + + + rm -f $jsondump +} + + +# Import, into a MongoDB collection, all JSON files of a directory including sub-directories +# $1: directory where to find the JSON files +# $2: MongoDB database name +# $3: MongoDB collection name +mongo_import_dir() { + _dir_b=$1 + _database_b=$2 + _collection_b=$3 + + # Get the list of files to import into MongoDB + _filelist_b=/tmp/filelist-$$.txt + find $_dir_b -type f -name '*.json' > $_filelist_b + echo "Importing $(wc -l $_filelist_b | cut -d' ' -f1) files..." + + mongo_import_filelist $_database_b $_collection_b $_filelist_b + #mongo_import_filelist_onebyone $_database_b $_collection_b $_filelist_b + rm -f $_filelist_b +} + + +# Same as mongo_import_dir + creates an index afterwards. +# $1: directory where to find the JSON files +# $2: MongoDB database name +# $3: MongoDB collection name +# $4: MongoDB collection's index column +mongo_import_dir_index() { + _database_c=$2 + _collection_c=$3 + _index_col_c=$4 + + mongo_import_dir $1 $_database_c $_collection_c + mongo --eval "db.${_collection_c}.createIndex({${_index_col_c}: 1})" localhost/$_database_c --quiet + mongo --eval "db.${_collection_c}.count()" localhost/$_database_c --quiet +} + + +# Same as mongo_import_dir + first drops the collection and creates an index afterwards. +# $1: directory where to find the JSON files +# $2: MongoDB database name +# $3: MongoDB collection name +# $4: MongoDB collection's index column +mongo_drop_import_dir_index() { + _database_c=$2 + _collection_c=$3 + _index_col_c=$4 + + mongo --eval "db.${_collection_c}.drop()" localhost/$_database_c + mongo_import_dir $1 $_database_c $_collection_c + mongo --eval "db.${_collection_c}.createIndex({${_index_col_c}: 1})" localhost/$_database_c --quiet + mongo --eval "db.${_collection_c}.count()" localhost/$_database_c --quiet +} + + + +# Import all JSON files of a directory into multiple MongoDB collections of a maximum number of files each. +# $1: directory where to find the JSON files +# $2: MongoDB database name +# $3: prefix of the collection names: prefix_0, prefix_1, ... +# $4: max number of files per collection +mongo_drop_import_dir_split() { + _dir_c=$1 + _database_c=$2 + _collection_c=$3 + _maxfilesPerCollection=$4 + + # Get the whole list of files to import into MongoDB + _filelist_c=/tmp/filelist-collection-$$.txt + find $_dir_c -type f -name '*.json' > $_filelist_c + echo "Importing $(wc -l $_filelist_c | cut -d' ' -f1) files..." + + # Split the list of files into multiple pieces of $_maxfilesPerCollection files + _prefix_c=/tmp/filelist-collection-$$- + split -d -l $_maxfilesPerCollection $_filelist_c $_prefix_c + rm -f $_filelist_c + + # Import the lists of files into separate collections + colIndex=0 + for _filelist_c in `ls ${_prefix_c}*`; do + + col=${_collection_c}_${colIndex} + echo "----- Creating collection $col" + mongo --eval "db.${col}.drop()" localhost/$_database_c + mongo_import_filelist $col $_filelist_c + + # Next collection + colIndex=$(($colIndex + 1)) + done + mongo --eval "db.${_collection_c}.count()" localhost/$_database_c --quiet + rm -f ${_prefix_c}* +} + +# Import a file into a MongoDB collection and create an index afterwards. +# The exsiting collection would be replaced. +# $1: file name +# $2: file type: csv, tsv, json +# $3: MongoDB database name +# $4: MongoDB collection name +# $5: MongoDB collection's index column +mongo_drop_import_file_index() { + _file_c=$1 + _type=$2 + _database_c=$3 + _collection_c=$4 + _index_col_c=$5 + + mongoimport --drop --type=_type --headerline --ignoreBlanks -d $_database_c -c $_collection_c $_file_c + mongo --eval "db.${_collection_c}.createIndex({${_index_col_c}: 1})" localhost/$_database_c --quiet + mongo --eval "db.${_collection_c}.count()" localhost/$_database_c --quiet +} diff --git a/docker/run.sh b/docker/run.sh new file mode 100644 index 00000000..f9c89b5c --- /dev/null +++ b/docker/run.sh @@ -0,0 +1,24 @@ +#!/bin/bash +# Author: Anna BOBASHEVA, University Cote d'Azur, CNRS, Inria +# +# Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) + +DB=database +COLLECTION=movies + + +MONGO_CONTAINER=$(docker ps --format=='{{.Names}}' | grep "mongo-xr2rml" | cut -d= -f2) +XR2RML_CONTAINER=$(docker ps --format=='{{.Names}}' | grep "morph-xr2rml" | cut -d= -f2) + + +# --- Import the JSON files of a directory into MongoDB +docker exec -w /mongo_tools $MONGO_CONTAINER \ + /bin/bash import-json-files.sh $DB $COLLECTION id + +# --- Run the translation to RDF +# Note that at start-up Morph-xR2RML shows these warning messages, just ignore them: +# SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". +# SLF4J: Defaulting to no-operation (NOP) logger implementation +# SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. +docker exec -w /xr2rml_config $XR2RML_CONTAINER \ + /bin/bash run_xr2rml_template.sh mapping_movies.ttl movies.ttl dataset1.0 $COLLECTION diff --git a/docker/xr2rml_config/log4j.properties b/docker/xr2rml_config/log4j.properties new file mode 100644 index 00000000..808fa98c --- /dev/null +++ b/docker/xr2rml_config/log4j.properties @@ -0,0 +1,27 @@ +log4j.rootCategory=file,console +log4j.rootLogger=INFO,console + +layout.fileLayout=%d{yyyy-MM-dd HH:mm:ss} %-5p %C{1}.%M:%L %x - %m%n +layout.consoleLayout=%d{yyyy-MM-dd HH:mm:ss} %-5p [%t] (%F:%L) - %m%n +layout.infoLayout=%d{yyyy-MM-dd HH:mm:ss} %-5p %C{1}.%M:%L %x - %m%n + +log4j.appender.console=org.apache.log4j.ConsoleAppender +log4j.appender.console.layout=org.apache.log4j.PatternLayout +log4j.appender.console.layout.ConversionPattern=${layout.consoleLayout} + +log4j.logger.org.apache=WARN +log4j.logger.org.glassfish=WARN +log4j.logger.com.sun.jersey=WARN +log4j.logger.com.hp.hpl.jena=WARN +log4j.logger.com.jayway.jsonpath=WARN + +log4j.logger.com.es.upm.fi.dia.oeg.morph.base.path=OFF + +#log4j.logger.es.upm.fi.dia.oeg.morph.base.engine.MorphBaseRunner=WARN +#log4j.logger.fr.unice.i3s.morph.xr2rml.mongo.engine.MorphMongoDataSourceReader=WARN +#log4j.logger.fr.unice.i3s.morph.xr2rml.mongo.engine.MorphMongoDataTranslator=WARN +#log4j.logger.fr.unice.i3s.morph.xr2rml.mongo.abstractquery.AbstractQueryAtomicMongo=WARN + +#log4j.logger.fr.unice.i3s.morph.xr2rml.mongo.querytranslator.MorphMongoQueryTranslator=WARN +#log4j.logger.es.upm.fi.dia.oeg.morph.base.TemplateUtility=WARN +#log4j.logger.es.upm.fi.dia.oeg.morph.r2rml.model=WARN diff --git a/docker/xr2rml_config/mapping_movies.ttl b/docker/xr2rml_config/mapping_movies.ttl new file mode 100644 index 00000000..67e9eb7f --- /dev/null +++ b/docker/xr2rml_config/mapping_movies.ttl @@ -0,0 +1,24 @@ +@prefix rdfs: . +@prefix schema: . +@prefix rr: . +@prefix xrr: . + +<#Movies> + a rr:TriplesMap; + xrr:logicalSource [ + xrr:query "db.{{collection}}.find({})"; + ]; + rr:subjectMap [ + rr:template "http://example.org/movie/{$.id}"; + rr:class schema:Movie; + ]; + rr:predicateObjectMap [ + rr:predicate schema:name; + rr:objectMap [ xrr:reference "$.title" ]; + ]; + rr:predicateObjectMap [ + rr:predicate rdfs:isDefinedBy; + rr:objectMap [ rr:constant "http://example.org/movie/{{dataset}}"; rr:termType rr:IRI ]; + ]; + . + diff --git a/docker/xr2rml_config/run_xr2rml.sh b/docker/xr2rml_config/run_xr2rml.sh new file mode 100644 index 00000000..b313f271 --- /dev/null +++ b/docker/xr2rml_config/run_xr2rml.sh @@ -0,0 +1,54 @@ +#!/bin/bash +# +# This script runs Morph-xR2RML in graph materialization mode, +# that is it applies the mappings and outputs the corresponding RDF file. +# +# Input argument: +# - arg1: xR2RML mapping file without path. Must be located in $CONFIG_DIR +# - arg2: output file name without path +# +# Author: Franck MICHEL, University Cote d'Azur, CNRS, Inria +# +# Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) + +XR2RML=/morph-xr2rml +CONFIG_DIR=/xr2rml_config +LOG_DIR=/log +OUTPUT_DIR=/xr2rml_output + +JAR=$XR2RML/morph-xr2rml-dist-1.3.2-20211126.142114-3-jar-with-dependencies.jar + + +help() +{ + exe=$(basename $0) + echo "Usage: $exe " + echo "Example:" + echo " $exe mapping_metadata.ttl metadata.ttl" + exit 1 +} + +# --- Read input arguments +mappingFile=$1 +if [[ -z "$mappingFile" ]] ; then help; fi + +output=$2 +if [[ -z "$output" ]] ; then help; fi + + +# --- Init log file +mkdir -p $LOG_DIR +log=$LOG_DIR/xr2rml_$(date "+%Y%m%d_%H%M%S").log + + +echo "--------------------------------------------------------------------------------------" >> $log +date >> $log +java -Xmx4g \ + -Dlog4j.configuration=file:$CONFIG_DIR/log4j.properties \ + -jar "$JAR" \ + --configDir $CONFIG_DIR \ + --configFile xr2rml.properties \ + --mappingFile $CONFIG/$mappingFile \ + --output $OUTPUT_DIR/$output \ + >> $log +date >> $log diff --git a/docker/xr2rml_config/run_xr2rml_template.sh b/docker/xr2rml_config/run_xr2rml_template.sh new file mode 100644 index 00000000..1fe6516d --- /dev/null +++ b/docker/xr2rml_config/run_xr2rml_template.sh @@ -0,0 +1,75 @@ +#!/bin/bash +# +# This script runs Morph-xR2RML in graph materialization mode, +# that is it applies the mappings and outputs the corresponding RDF file. +# It first replaces the plateholders in the template mapping file. +# +# Input argument: +# - arg1: xR2RML template mapping file without path. Must be located in $CONFIG_DIR +# - arg2: output file name without path +# - arg3: replacement of the dataset parameter {{dataset}} in the mapping template, e.g. "dataset1" +# - arg4: replacement of the MongoDB collection parameter {{collection}} the mapping template, e.g. metadata +# +# Author: Franck MICHEL, University Cote d'Azur, CNRS, Inria +# +# Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) + +XR2RML=/morph-xr2rml +CONFIG_DIR=/xr2rml_config +LOG_DIR=/log +OUTPUT_DIR=/xr2rml_output + +JAR=$XR2RML/morph-xr2rml-dist-1.3.2-20211126.142114-3-jar-with-dependencies.jar + + +help() +{ + exe=$(basename $0) + echo "Usage: $exe " + echo "Example:" + echo " $exe mapping_metadata.ttl metadata.ttl dataset1 metadata" + exit 1 +} + +# --- Read input arguments +mappingTemplate=$1 +if [[ -z "$mappingTemplate" ]] ; then help; fi + +output=$2 +if [[ -z "$output" ]] ; then help; fi + +dataset=$3 +if [[ -z "$dataset" ]] ; then help; fi + +collection=$4 +if [[ -z "$collection" ]] ; then help; fi + + +# --- Init log file +mkdir -p $LOG_DIR +log=$LOG_DIR/xr2rml_${collection}_$(date "+%Y%m%d_%H%M%S").log + + +# --- Substitute placeholders in the xR2RML template mapping +mappingFile=/tmp/xr2rml_$$.ttl +awk "{ gsub(/{{dataset}}/, \"$dataset\"); \ + gsub(/{{collection}}/, \"$collection\"); \ + print }" \ + $CONFIG_DIR/${mappingTemplate} > $mappingFile +echo "-- xR2RML mapping file --" >> $log +cat $mappingFile >> $log + + +echo "--------------------------------------------------------------------------------------" >> $log +date >> $log +java -Xmx4g \ + -Dlog4j.configuration=file:$CONFIG_DIR/log4j.properties \ + -jar "$JAR" \ + --configDir $CONFIG_DIR \ + --configFile xr2rml.properties \ + --mappingFile $mappingFile \ + --output $OUTPUT_DIR/$output \ + >> $log +date >> $log + +rm -f $mappingFile diff --git a/docker/xr2rml_config/xr2rml.properties b/docker/xr2rml_config/xr2rml.properties new file mode 100644 index 00000000..3bdfc7e0 --- /dev/null +++ b/docker/xr2rml_config/xr2rml.properties @@ -0,0 +1,74 @@ +# xR2RML mapping file. Overriden by CLI option --mappingFile +# mappingdocument.file.path=xr2rml_taxref_v10_classes.ttl + +# -- Server mode: true|false. Default: false +# false: stand-alone application that performs either graph materialization or query rewriting +# true: SPARQL endpoint +# server.active=true + +# -- Server port number, ignored when "server.active = false" . Default: 8080 +# server.port=8080 + +# -- Where to store the result of the processing. Default: result.txt - overriden by CLI option --output +# output.file.path=result.ttl + +# -- Output RDF syntax: RDF/XML|RDF/XML-ABBREV|N-TRIPLE|TURTLE|N3. Default: TURTLE +# Applies to the graph materialization and the rewriting of SPARQL CONSTRUCT and DESCRIBE queries, +#output.syntax.rdf=TURTLE + +# -- Output syntax for SPARQL result set (SPARQL SELECT and ASK queries): XML|JSON|CSV|TSV. Default: XML +# When "server.active = true", this may be overridden by the "Accept" HTTP header of the query +# output.syntax.result=XML + +# -- Display the result on the std output at the end of the processing: true|false. Default: true +output.display=false + +# -- Trim the literal values read from the database. true|false. Default: true +#literal.trim=true + +# -- File containing the SPARQL query to process. Default: none +# Ignored when "server.active = true" +# query.file.path=query.sparql + +# -- Database connection +no_of_database=1 +database.type[0]=MongoDB +database.driver[0]= + +database.url[0]=mongodb://mongo-xr2rml:27017 +database.name[0]=database +database.user[0]= +database.pwd[0]= + +# Reference formulation must be one of Column, JSONPath or XPath +database.reference_formulation[0]=JSONPath + + +# Runner factory: +# For MongoDB: fr.unice.i3s.morph.xr2rml.mongo.engine.MorphJsondocRunnerFactory +# For RDBs: es.upm.fi.dia.oeg.morph.r2rml.rdb.engine.MorphRDBRunnerFactory +runner_factory.class.name=fr.unice.i3s.morph.xr2rml.mongo.engine.MorphMongoRunnerFactory + +# -- URL-encode reserved chars in database values. Default: true +uricolumn.encode_unsafe_chars_dbvalues=false + +# -- URL-encode reserved chars IRI template string. Default: true +uricolumn.encode_uri=false + + +# -- Cache the result of previously executed queries for MongoDB. Default: false +# Caution: high memory consumption, to be used for RefObjectMaps only +querytranslator.cachequeryresult=true + + +# -- Primary SPARQL query optimization. Default: true +# querytranslator.sparql.optimize=true + +# -- Abstract query optimization: self join elimination. Default: true +# querytranslator.abstract.selfjoinelimination=true + +# -- Abstract query optimization: self union elimination. Default: true +# querytranslator.abstract.selfunionelimination=true + +# -- Abstract query optimization: propagation of conditions in a inner/left join. Default: true +# querytranslator.abstract.propagateconditionfromjoin=true diff --git a/docker/xr2rml_docker.zip b/docker/xr2rml_docker.zip index a6b84267..e5dcb208 100644 Binary files a/docker/xr2rml_docker.zip and b/docker/xr2rml_docker.zip differ