Skip to content

Commit

Permalink
chore: update Readme and demo
Browse files Browse the repository at this point in the history
  • Loading branch information
rgwozdz committed Oct 17, 2023
1 parent f374383 commit ca0c0e1
Show file tree
Hide file tree
Showing 5 changed files with 132 additions and 8 deletions.
109 changes: 108 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,108 @@
# koop-provider-mongo
# koop-provider-mongo

Provider to fetch data from a MongoDb instance. It can access any database/collection on your instance by assigning the `id` route-parameter a delimited value in the form of `database-name::collection-name`.

## Data considerations
As per MongoDB specifications, [each document/record in your dataset will have a unique-identifier](https://www.mongodb.com/docs/manual/core/document/#the-_id-field), `_id`. At insert time, you can choose to assign this value manually, or allow Mongo to generate it automatically. In terms of your provider metadata, this means that you could always use `_id` as the value of your provider's `idField` (an attribute used to flag which GeoJSON property is the unique-identifier).

If your data has a geometry, you should consider using [geospatial index](https://www.mongodb.com/docs/manual/core/indexes/index-types/index-geospatial/). While not necessary, it will likely speed any queries including a geometry operation.

## Usage

Register the provider with Koop:

```js
const Koop = require('@koopjs/koop-core');
const koop = new Koop({ logLevel: 'info'});
const mongoProvider = require('@koopjs/provider-mongodb');

koop.register(mongoProvider, { connnectString, databases });
```

### Registration/Config parameters
The provider can be configured with registration options or the use of the `config` module with an entry key `"mongodb"` in the JSON configuration document. See below, for an example of using the `config` approach


#### `connectString`: string
A MongoDB connection string, e.g., `mongodb://localhost:27017`.

#### `databases`: object
A key/value populated object that serves as a metadata lookup/dictionary for any database/collection on your MongoDB instance that you wish to provide access. For example, if imagine you have a database called `db1` and it has a collection called `my-collection`. You can set up the `databases` object so as to provide metadata for `my-collection`:

```js
{
db1: {
'my-collection': {
geometryField: 'location', // field holding a record's GeoJSON geometry.
idField: '_id', // field to treat as the unique-id. Default: `_id`.
cacheTtl: 0, // number of seconds to cache results from MongoDb. Default: 0.
crs: 4326, // Coordinate reference system of geometry. Default: 4326.
maxRecordCount: 2000, // Max number of records to return in a page. Default: 2000.
}
}
}
```

Note that for each collection, you may configure the options above. If you do not assign a field to `geometryField`, the collection will be treated like tabular data, and the GeoJSON generated will have no geometry,

#### `definedCollectionsOnly`: boolean
This setting determines if the database/collection must appear in the `databases` parameter to be accessed. Defaults to `true`.


#### Setting with the "config" module
Koop allows providers to use the [`config` module](https://github.com/node-config/node-config). The settings above can be stored in a config JSON under the key `mongodb`:

```json
{
"mongodb": {
"connectString": "mongodb://localhost:27017",
"databases": {
"cdf-sample-data": {
"fires": {
"geometryField": "location",
"idField": "_id",
"cacheTtl": 0,
"crs": 4326,
"maxRecordCount": 2000
}
}
}
}
}
```

## Route parameters

#### `id`
Once registered with Koop, the provide will expose routes with an `id` parameter. For example:

```sh
/mongodb/rest/services/:id/FeatureServer
```

The `id` parameter should be filled with a `::` delimited string with the target `<database>::<collection>`. For example, a request like:

```sh
/mongodb/rest/services/sample-db::fires/FeatureServer/0/query
```

would return records from the `fires` _collection_ in the `sample-db` _database_.

## Demo

The repository includes a demonstration project. To run the demo you will need Docker installed on your computer. Once installed you can following the steps below to create a local Elastic instance and load it with sample data:

```sh
> npm install

> cd demo

# use Docker to run Elastic/Kibana
> docker-compose up -d

# load sample data; this will create an MongoDB database called "sample-data' with a collection named "fires"
> node loader.js

# start the Koop application
> node index.js
```
3 changes: 2 additions & 1 deletion demo/config/default.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
{
"mongodb": {
"connectString": "mongodb://localhost:27017",
"definedCollectionsOnly": true,
"databases": {
"cdf-sample-data": {
"sample-data": {
"fires": {
"geometryField": "location",
"idField": "_id",
Expand Down
2 changes: 1 addition & 1 deletion demo/loader.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ async function run() {
fs.createWriteStream('wildfires.json'),
);

const db = client.db('cdf-sample-data');
const db = client.db('sample-data');
const firesCollection = db.collection('fires');
const features = require('./wildfires.json');

Expand Down
6 changes: 5 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
"description": "Koop provider for mongodb.",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
"test": "jest",
"test:cov": "jest --coverage",
"lint": "eslint \"**/*.js\"",
"lint:ci": "eslint \"**/*.js\" --rule linebreak-style:0 ",
"lint:fix": "eslint \"**/*.js\" --fix"
},
"author": "",
"license": "Apache-2.0",
Expand Down
20 changes: 16 additions & 4 deletions src/model.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,41 @@ const relationLookup = {
class Model {
#client;
#databaseLookup;
#definedCollectionsOnly;
#logger;

constructor({ logger }, { conn, databaseLookup }) {
constructor({ logger }, { connectString, databases, definedCollectionsOnly }) {
this.#logger = logger;
const databaseUri = conn || config.mongodb.connectString;
const databaseUri = connectString || config.mongodb.connectString;
this.#client = new MongoClient(databaseUri, {
serverApi: {
version: ServerApiVersion.v1,
strict: true,
deprecationErrors: true,
},
});
this.#databaseLookup = databaseLookup || config.mongodb.databases;
this.#databaseLookup = databases || config.mongodb.databases;
this.#definedCollectionsOnly = definedCollectionsOnly || config.mongodb.definedCollectionsOnly;
}

async getData(req, callback) {
// Parse the "id" parameter from route-path
const [databaseName, collectionName] = req.params.id.split('::');

// Lookup collection config
const collectionConfig = this.#databaseLookup[databaseName]?.[collectionName];

// If no defined collection-config and definedCollectionsOnly, reject with 404
if (!collectionConfig && this.#definedCollectionsOnly) {
const error = new Error('Not Found');
error.code = 404;
callback(error);
}

// Get collection specific config data
const {
geometryField,
idField,
idField = '_id',
cacheTtl = 0,
crs = 4326,
maxRecordCount = 2000,
Expand Down

0 comments on commit ca0c0e1

Please sign in to comment.