Automated mapping of AREDN Networks.
2016-2020 - Eric Satterlee / KG6WXC
Addtional Credit to: Mark/N2MH and Glen/K6GSE for their work on this project and to the rest of the AREDN team, without them this would not be a project.
Licensed under GPL v3 and later.
Donations / Beer accepted! :)
- Apache webserver
(or equiv) - PHP5+
- mysqli PHP extension
- MariaDB
(MariaDB is preferred over Mysql due to inconsistencies between the 2)
(Other database systems are up to you) - An AREDN Mesh node available over the local network
(Preferably connected to an AREDN network...) - Map Tiles
(Either in a static directory or available via some tile server...) - RPi3 or better
(The DB access can be pretty slow on an RPi1, you can move the DB to another system though...)
(If your local mesh network grows to become very large, with tunnels opened to the entire planet,
you might find even an RPi3+ to become inadequate, it'll still work, it just might be slow) - Patience
(Perhaps a lot!) - Familiarity with Linux/Apache/SQL
(You don't need to be a pro, but this should not be your first trip to a command line)
In theory, this should run on a Windows system as well.
It does not require anything specific to Linux (Perhaps with the exception of the cron task).
There is no reason that cronjob could not be adapted to run from a Windows scheduled task though.
PHP is PHP after all.
Without a map tile server or static tiles, Mesh users without internet access on their systems may not see any map tiles.
On the mesh, you cannot expect the client to have internet access in order to retrieve the tiles, you must provide them yourself, one way or another.
The main map webpage will try to check for internet access and load the appropriate maps.
Default internet map tile servers can be modified in your user-settings.ini file, if you want to use "local" tile servers or directories.
It is way beyond the scope of this README file to help in setting up a map tile server.
You are unfortunatly on your own there.
It is a time consuming and computationaly expensive process, but can be done on "normal" hardware.
It also takes 100's of GB of HD space if you want to map the entire world, and that does not include the tiles themselves, that is only for the data required to create the map tiles.
A good place to start for more info is: https://switch2osm.org/serving-tiles/
If you attempt it, be patient, you will get it wrong more than a few times but in the end you might be surprised. :)
Tip: Another option is that some programs, Xastir in particular, will save the map tiles they use.
You *can* use those tiles as well, but they must be made to be accessible by your webserver.
You might be able to convince KG6WXC to create local map tiles for you, if the area you want is in the USA, he does not have the available SSD space for the entire world... yet.
If you do ask, be prepared, it literally takes KG6WXC's system about 3-4 days just to render the tiles for a smallish area and it's kind of a PITA!
As an example, KG6WXC once made tiles for the Mesa Az. mesh group.
It was a smallish area around Phoenix Az, out to a zoom of about 8 or something.
It ended up at around 3GB of map tiles and took about 4 days of total run time to render on the server...
and it had to restart a few times too, due to running out of 8GB of RAM and having to tweak a few things along the way...
It actually took much longer than the 4 days of actual run time.
Building/Using a map tile server is not for the faint of heart!
(Should work for other Linuxes as well, change where needed)
-
1: Clone the projects directory from the git repository and enter it
git clone https://mapping.kg6wxc.net/git/meshmap ; cd meshmap -
2: Import the SQL file to create the database
Example:sudo mysql < node_map.sql -
3: Create a user for the database, you might have to login to the mysql server as root.
Here is an example of creating a mySQL user and granting access to the node_map database:
Choose your own password!
sudo mysql
CREATE USER 'mesh-map'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES on node_map.* TO 'mesh-map'@'localhost';
FLUSH PRIVILEGES;
- 4: Copy scripts/user-settings.ini-default to scripts/user-settings.ini and edit the user-settings.ini file
- You must do this or the entire system will refuse to run!
- The file scripts/user-settings.ini is the most important to get right.
It is very important to make sure your SQL username and password are correct in scripts/user-settings.ini!! - Also important is, if the system that this is running on cannot resolve "localnode.local.mesh" you can change that in the user-settings.ini file.
- Once you save to the user-settings.ini file any changes you make will not be overwritten by future updates.
The "-default" files will probably change though and you will need to update your personal files when this happens. - There are many other things you can change in the ini files.
The default center position of the map, node expiration intervals, the header messages, logo, etc. - Please read the comments in the user-settings.ini file for more info about the different settings.
- There is also a "custom.inc-default" PHP file that can be used for more site specific overrides if needed.
Read that file for info on what it does, it can safely be ignored by most users.
The way the user editable files are distrubuted has changed!.
- 4.5: To make sure it is all working at this point is probably a good idea.
You should now be able to run get-map-info.php from the scripts directory.
I would suggest giving it a test run or two first.
Node polling can take lots of time, especially on a large network. Be Patient! :)
Enter the meshmap/scripts directory.
Run theget-map-info.phpscript.Tip: if you get a "command not found" error, you may need to run it like this:
`./get-map-info.php `
These are options you can send to get-map-info.php:
> --test-mode-no-sql
Output to console only, do not read/write to the database.
This will make sure the scripts can reach your localnode and the rest of the network.>
This will not update the database, you won't see anything on the map page yet, it only outputs to the terminal for testing.
--test-mode-with-sqlOutput to console and read/write to the database.
This will ensure everything is setup correctly and get some data into your database!
Tip: Do not ctrl-C out of the script while it is using the database!
Let it finish, even if it seems hung up.
You should recieve some error if something is actually wrong.
Using ctrl-C to stop the script midway will cause problems in the database, do not do it!
If the --test-mode-no-sql is successful, you can go ahead and run the script with --test-mode-with-sql or just without any options.
Run the script without options and there is no on screen output (this is for cron).
-
5: Copy httpd-meshmap.conf-default to the apache2 "Conf Available" directory,
/etc/apache2/conf-available
Rename the file as httpd-meshmap.conf (or whatever you want to call it really.)
Once the file is copied, you need to edit it and make sure the<Alias>and<Directory>directives have the correct paths.
After you have made sure the file is correct then run:sudo a2enconf httpd-meshmap
This is will load the config into Apache and if successful, it will tell you to reload apache, do so.Other linux distibutions may require you to copy this file into /etc/httpd/extra
and then edit /etc/httpd/httpd.conf and add the line: Include extra/httpd-meshmap.conf somewhere. -
6: Load up the page: http://myhostname/meshmap/index.php and you should hopefully see your data.
You may or may not see any map tiles, depending on if the system you are using to view the page has access to the internet or not.
Even without map tiles, you should still see your data being mapped out. -
7: The cronscript.sh file is to automatically run the polling script and can be run from cron every minute.
(or at whatever interval you choose)
Copy the cronscript.sh-default to where ever you like and rename it to just cronscript.sh (or whatever you want).
Then, you must edit the cronscript.sh file and make sure the path it uses to get to the scripts directory is correct!
After that, create a cron entry withcrontab -e
A cron entry is as easy as this:* * * * * /home/pi/meshmap/cronscript.shYou can safely run the script every minute in cron like this.
It won't actually do anything unless the intervals specified in the ini file have expired.
Simply run a "git pull" from the meshmap directory and the scripts will be updated from the git repo.
The user-settings.inc, meshmap-settings.ini, cronscript.sh, and custom.inc files will not be affected by updating.
The settings in the default ini files may still change and have things added or removed in future versions.
For now tho, if the default ini files change, and you still have the old ones in use, things will probably break! Be Warned!
Hopefully in the future this process can be more automated.
If you make changes beyond the user editable files I encourage you to perhaps push the changes upstream, please contact [email protected] if you wish to do so.
I am making changes all the time, it would be a good idea to run "git pull" from time to time to find any updates.
http://(hostname)/meshmap/node_report.php will show you all the info in the DB without trying to map anything.
This can be useful to see if all the data is there or to find nodes that have no location set. (or other issues)
There is an "admin" page, which is still in the works, what is there now does work tho.
Load up: http://(hostname)/meshmap/admin/admin.php in your web browser.
The first time the admin page is loaded it will ask you to create a username and password, do so.
This initial user will be a "super-user" and can then add/remove other users.
I've tried to provide instructions on the admin pages themselves.
From the admin pages you can "fix" a nodes location, which can be helpful for those users that forget the "-" in front of their longitude. :)
You can add the "Non Mesh" Markers, fire stations, police stations, EOC's , etc from the admin pages...
The admin pages also allow for some maintenance of the database, more feedback is encouraged on this!
You can change the way the page looks by copying webpage/css/meshmap-default.css to webpage/css/meshmap.css.
The meshmap.css file will override the -default.css file.
(This also applies to other *-default.css files, there are a few, please look at them if you wish to customize the layout of the map)
(In no particular order)
- Finally finish my admin page idea (mostly).
- Add new MeshMap Logo.
- User css files will override the defaults.
- Polling script checks the DB before it runs and makes changes if needed.
- Catch more nodes information now, like 3.15.1.0b04 and hopefully some other pesky ones!
(this will probably cause some warnings during node polling, but it is getting the info it needs)
(also helps clean up the database and was a nice side effect of that) - Make the numbers for stations and links in the attribution bar a bit more accurate I hope.
- Add a "Ruler" to allow for measuring of distance and bearings.
(elevation plot of the line drawn via this ruler will hopefully come next) - Station PopUps now have tabs for the different sections (main, services, links).
- Search and zoom to a node now implemented.
- Import/Export the "Non-Mesh" Markers via CSV file.
- Add button for "node_report.php" (List View) on the Map. (and a link to the map from node_report).
- Change css file for the "?" slide-out menu.
- Make "Parallel Threads" work again in get-map-info script, with limits on how many can be run at once.
(this will greatly speed up network polling) - The "Planning" Tab.
- Make it so other networks can export their data for use on a "Mega Map" type page. :)
-
Implement N2MH's "Link aging" idea(s). (Temporal Databases are crazy! this may not ever happen)
Contribution is encouraged!!
I can't think of everything!
If you find an improvement, typo, or whatever, please, sign up at https://gitlab.kg6wxc.net or send an email to [email protected] or something!!
This README file last updated: July 2019 (fixed some Markdowns)