Skip to content

Releases: nansencenter/django-geo-spaas-harvesting

3.0.1

16 Feb 14:15
9134c31
Compare
Choose a tag to compare

bugfix of base source code: based on Django-Geo-SPaaS-2.3.3 as a proper source code base

3.0.0

16 Feb 07:10
Compare
Choose a tag to compare
  • add local Nansat harvester
  • generalize WebDirectoryCrawler filtering by using regular expressions

2.2.2

02 Feb 14:49
Compare
Choose a tag to compare

Update metanorm version to 2.0.3.

2.2.1

01 Feb 14:11
Compare
Choose a tag to compare

Various fixes:

  • limit the number of concurrent database connections created when harvesting
  • create a new database connection for each parallel harvester process to avoid "lost synchronization" errors
  • for paginated APIs, sort by publishing date rather than content date
  • add settings to control pythesint vocabularies updates

2.2.0

19 Jan 14:30
Compare
Choose a tag to compare
  • CONN_MAX_AGE and DISABLE_SERVER_SIDE_CURSORS database settings are parameterized in settings.py
  • The reliability of the FTP crawler has been improved
  • A base crawler class for exploring paginated APIs has been added
  • The base Ingester has been modified so that crawlers can provide more than a download URL
  • The version of metanorm has been updated to 2.0.2
  • Added a harvester for Creodias
  • Added a local filesystem crawler

2.1.0

11 Nov 13:55
Compare
Choose a tag to compare
  • Update metanorm version to 2.0.1 (bugfixes)
  • Update geospaas version to 2.1.0 (remove length limit on Dataset.entry_id)

2.0.0

02 Oct 11:46
Compare
Choose a tag to compare
  • add the ability to harvest dataset parameters
  • add a crawler and ingester for FTP repositories
  • add a script which checks the validity of ingested datasets URLs
  • add support for environment variables in the configuration file

1.1.1

29 Jun 12:56
Compare
Choose a tag to compare

New geospaas base image

1.1.0

25 Jun 14:02
Compare
Choose a tag to compare

Use enhanced Metanorm search capabilites

1.0.0

20 May 11:59
Compare
Choose a tag to compare

The structure of the application is set.
Harvesters (and the corresponding crawlers and ingesters) for PO.DAAC and Copernicus Sentinel data are available.