Error generating tiles

[reddoxster]

I wanted to quickly test Kartotherian and your images saved us days! Everything runs well till I run the tile-generation command:

docker exec -it kartotheriandocker_tilerator_1 /gen_tiles.sh

info Loading variables from /etc/tilerator/variables.yaml
info Loading sources from /etc/tilerator/sources.yaml
trace Updating layer water
trace Updating layer waterway
trace Updating layer landcover
trace Updating layer landuse
trace Updating layer mountain_peak
trace Updating layer park
trace Updating layer boundary
trace Updating layer aeroway
trace Updating layer transportation
trace Updating layer transportation_name
trace Updating layer place
trace Updating layer housenumber
trace Updating layer aerodrome_label
trace Updating layer water
trace Updating layer waterway
trace Updating layer landcover
trace Updating layer landuse
trace Updating layer mountain_peak
trace Updating layer park
trace Updating layer boundary
trace Updating layer aeroway
trace Updating layer transportation
trace Updating layer building
trace Updating layer transportation_name
trace Updating layer place
trace Updating layer housenumber
trace Updating layer aerodrome_label
trace Updating layer poi
{ Err: Invalid zoom param type boolean given, was expecting an integer
    at checkType (/opt/tilerator/node_modules/@kartotherian/input-validator/validator.js:98:23)
    at Job.<anonymous> (/opt/tilerator/node_modules/@kartotherian/jobprocessor/lib/Job.js:122:17)
    at /opt/tilerator/node_modules/underscore/underscore.js:73:21
    at Function._.each._.forEach (/opt/tilerator/node_modules/underscore/underscore.js:153:9)
    at new Job (/opt/tilerator/node_modules/@kartotherian/jobprocessor/lib/Job.js:117:11)
    at addJobAsync (/opt/tilerator/lib/common.js:100:48)
    at Object.enqueJob (/opt/tilerator/lib/common.js:109:16)
    at /opt/tilerator/scripts/tileshell.js:204:23
    at tryCatcher (/opt/tilerator/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/opt/tilerator/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/opt/tilerator/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/opt/tilerator/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/opt/tilerator/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/opt/tilerator/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/opt/tilerator/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/opt/tilerator/node_modules/bluebird/js/release/async.js:17:14)
  name: 'Err',
  message: 'Invalid zoom param type boolean given, was expecting an integer' }
Err: Invalid zoom param type boolean given, was expecting an integer
    at checkType (/opt/tilerator/node_modules/@kartotherian/input-validator/validator.js:98:23)
    at Job.<anonymous> (/opt/tilerator/node_modules/@kartotherian/jobprocessor/lib/Job.js:122:17)
    at /opt/tilerator/node_modules/underscore/underscore.js:73:21
    at Function._.each._.forEach (/opt/tilerator/node_modules/underscore/underscore.js:153:9)
    at new Job (/opt/tilerator/node_modules/@kartotherian/jobprocessor/lib/Job.js:117:11)
    at addJobAsync (/opt/tilerator/lib/common.js:100:48)
    at Object.enqueJob (/opt/tilerator/lib/common.js:109:16)
    at /opt/tilerator/scripts/tileshell.js:204:23
    at tryCatcher (/opt/tilerator/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/opt/tilerator/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/opt/tilerator/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/opt/tilerator/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/opt/tilerator/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/opt/tilerator/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/opt/tilerator/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/opt/tilerator/node_modules/bluebird/js/release/async.js:17:14)

This is on Ubuntu 16.04.

Am I missing a step? I wanted to run this manually but couldn't figure out tilerator's X/Y config for Luxembourg. Since metroextracts shutdown, we only have geofabrik which means we can't import Koln to make your example work (whole of Germany is necessary instead!). And it's a bit difficult to understand tilerator's X/Y index scheme.

Could you perhaps shift the example to Luxembourg or a similar country that's available easily for tests?

Thanks!

[amatissart]

Hi,
Thanks for reporting this issue. As you can see, this repository is strongly dependent on our development setup, and is far from being correctly tested yet, unfortunately.

The issues with gen_tiles.sh seem to be caused by a wrong value for checkZoom.
Even if negative values are accepted, it does not make sense here, as the jobs start from zoom 0. And for some reason, the value is parsed as a boolean, possibly because of the quotes. We need to look at it more closely.

So you may want to simply remove the checkZoom option, at least to generate the first zoom levels.

The tiles coordinates (X/Y) used by Tilerator follow the "Slippy map tilenames".
Geofabrik provides a nice online tool to find these coordinates : http://tools.geofabrik.de/calc/#type=geofabrik_standard&grid=1

For example, to create a job generating vector tiles in Luxembourg (covered by the tile x=66, y=43, z=7), from zoom 7 to zoom 15 (non-included), you could do:

curl -XPOST "http://localhost:16534/add?generatorId=substbasemap&storageId=basemap&zoom=7&x=66&y=43&fromZoom=7&beforeZoom=15&keepJob=true&parts=8&deleteEmpty=true"

Note that I am directly using here the API provided by Tilerator. That may be more convenient than the tileshell script.

We definitely need to make this docker composition easier to run. And including an option in the scripts to download and import a sample dataset such as Luxembourg, is a good idea.
In the meantime, I hope that helps !

[reddoxster]

That fixed the command! The tiles generate successfully, and tiles from 0 to about 5 work well (whole world) however the map shows Luxembourg only in 'x-ray' (small on on the bottom right) mode but not in normal map mode.. I ran gen_tiles.sh after removing checkZoom and even then no luxembourg or names / fonts..

I suspect newer, incompatible versions of various dependencies could be the issue. If you know which ones you built your last successful build with, perhaps we can force those git-versions of the repos while building the docker images?

Also, would you like me to send a PR updating the docs to use Luxembourg and mentioning the tile format for Tilerator?

[antoine-de]

hum I don't really understand what you mean by 'x-ray' 😕

What do you display the tiles with ? do you use the tileview docker which can be queries through http://localhost:8585 ?

Yes we'll happily accept a PR for the doc and to depreciate the gen_tiles.sh script over the tilerator API, thanks!

[reddoxster]

Created PR here #28

By x-ray I meant the 'vector-map' view 😃.

Had to do 1 more step to make this work. Edit qwant_style.json and add a key for tilehosting.com. I'm new to docker so if that step could somehow be added for other users as part of the build process, it'll make things easier. Or we can ask them to do the edit before docker-compose up

Everything works at z14 onwards but I suspect this has to do with tile generation more than anything else.

Should we keep the issue open till the dependencies are built reliably & checkzoom is removed?

[antoine-de]

thanks for the PR!

we block the tile generation for z > 14, but with the overzoom it should be create any problems.

Everything works fine for you now ?

[reddoxster]

Ah makes sense to overzoom! All works great now.

A follow up question: there's a mention of auto-conversion to 'raster' in the flow chart but I didn't find anything in kartotherian's sources that allow a PNG endpoint.. Is the sources file incomplete according to the flow chart?

[amatissart]

We don't use raster tiles at the moment. But kartotherian handles them directly via tilelive-vector.
You can see an example in upstream kartotherian sources.

For our use case, we plan to use tileserver-gl to render raster using our custom mapbox-gl style, instead of a mapnik-specific .xml. But that would require a specific integration: see issue #10