Skip to content

Harvester

Jump to bottom
Fabian Kirstein edited this page Jul 16, 2019 · 5 revisions

Introduction

The Harvester is small tool for uploading and processing data to the AEGIS platform from various sources. You find it here: https://platform.aegis-bigdata.eu/hopsworks-api/harvester

Simple File Upload

  • A simple file upload can be orchestrated by drag and drop the modules "Dataset Upload" and "AEGIS Export" to the canvas. The input fields need to filled accordingly. By clicking on "RUN NOW" the harvesting pipe is initiated and the file is uploaded to the AEGIS platform:

OpenWeatherMap

  • The harvester can easily retrieve data from OpenWeatherMap (https://openweathermap.org/). You just need an account and an API key from OpenWeatherMap. Drag and drop the modules "Open Weather Map" and "AEGIS Export" to the canvas. The input fields need to filled accordingly:

Saving and Loading Harvesting Pipes

  • A orchestrated pipe can be secured by clicking on "Download" and storing the downloaded JSON file
  • A pipe can be restored by uploading the JSON file again
Clone this wiki locally