Deploying OpenStreetMaps

Hey everyone,

OpenStreetMap is the map of the world with open license and freedom. I deployed OpenStreetMaps on my system after seeing a lot of people having a lot of troubles deploying them. I used Ubuntu 16.04, i3 (2-cores, 4-threads), 4GB RAM system for this and this link as source: https://www.linuxbabe.com/linux-server/openstreetmap-tile-server-ubuntu-16-04.

You need to download map file for your region of interest. I download India’s map (~400MB pbf file). You can download from here and of course can choose another region. Now let’s continue as the tutorial source I mentioned takes us. I am not going to repeat all of that (DRY), it’s easy. After installing some dependencies and postgresql, I created osm user, created db, created extensions.

Download Map Stylesheet from https://github.com/gravitystorm/openstreetmap-carto/releases and download the latest version which at this time is v4.0.0.

Next, you are going to import the download map file into the PostgreSQL database. If you have a small amount of RAM, then it’s recommended to create a swap space to speed up the import process. I created 2GB for me with the fallocate command. I noticed that fallocate doesn’t support NTFS file system (and a few other too). I tried creating on ext4 file system on another drive and it works fine. Then we mount it as swap memory. You can check it using: cat /proc/swaps or top commands.

Now after installing the osm2pgsql tool, the actual importing process starts. Following command does it all:

osm2pgsql --slim -d gis -C 3600 --hstore -S openstreetmap-carto-2.41.0/openstreetmap-carto.style great-britain-latest.osm.pbf

3600 (3.6GB) here is the amount memory you want to allow the import process to be taken. You can try taking a little more. I tried the double of it and it gave error. -d is for database name (gis). Change the files and folders according to your own system e.g. openstreetmap-carto-2.41.0 will probably be different on your system and great-britain-latest.osm.pbf will be name/path to your map file you downloaded. You can also add additional argument –number-processes=4. I added because of the 4 threads of my CPU. Hence the above command in my case becomes:

osm2pgsql --slim -d gis -C 3600 --number-processes=4 --hstore -S openstreetmap-carto-4.0.0/openstreetmap-carto.style india-latest.osm.pbf

It takes time to import. So, go out and enjoy life (plug in the charger/if on server, use tmux). Okay, it’s finished finally.

Now time to compile the mod_tile module required to serve the tiles. If you run into some error during the compilation, refer to this for installing some missing dependencies. After it’s done, let’s move forward.

Generate Mapnik Stylesheet

You’ll be asked to go to openstreetmap-carto-xxx directory and execute get-shapefiles script. In the newer version, they have converted the bash script to Python script and moved it to script directory. That’s why it’s recommended to follow the official guide. So go to script directory and execute it as:

python get-shapefiles.py

And then run in previous directory,

carto project.mml > style.xml

Above command (carto) gave me error:

carto: Unexpected token s

It was due to very old version of carto installed. Found the hint at https://github.com/gravitystorm/openstreetmap-carto/issues/2583 to use higher version (>0.16) of carto.

Installed latest version:

sudo npm install -g carto

You’ll need to install nodejs for this (npm – node package manager) to work. Then I executed carto again and it worked, despite giving some warnings like:

Warning: using the name attribute for layers (like building-text here) is deprecated and will be removed in 1.0.0. Use id instead.

But it’s OK. So if yours one is still giving the same error (carto), try logging out and log in again. Check if your carto version is updated. Mine shows:

$ carto -v
 carto 0.18.0 (Carto map stylesheet compiler)

Move on to configuring the renderd and edit /usr/local/etc/renderd.conf and edit your details carefully as per your own system.

After configuring Apache, if you point yourself to the browser at http://localhost/osm_tiles/0/0/0.png (or whatever ip) and get a image showing a world map, then you are lucky. If you did’t, you probably have to check Apache logs (/var/log/apache2/error.log) and hence this makes you more lucky. In my case, the tail of log file contains:

/var/run/renderd/renderd.sock No such file or directory

So, I checked and there wasn’t renderd directory. So I created on and given appropriate permissions. Then I executed again and I got the map image. Congratulations!

Display Your Tiled Web Map

Now, it’s time to see the map zoom-able and span-able. There are two JavaScript libraries available: OpenLayer and Leaflet. You  just need to download and extract it to your root of your web server like (/var/www/html). Edit the index.html file and put the OpenLayer specific code. Make sure to edit lines like: http://your-ip/ol.css. Replace your-ip with your ip-address. If you are testing on your local system, then put localhost instead. Point your browser to localhost and you should see the map. Now the tiles are generated on-the-fly. If you want to pre-render the tiles so that your processor doesn’t do processing everytime you zoom-in then you can do it via:

render_list -m default -a -z 0 -Z 10

This command will take some time. And yes, you can stop and resume it anytime. It won’t start from start again. It will generate tiles for various zoom levels.

The issue I was having at this step was no map was being shown. Seeing the apache logs disclosed the error:

socket connect failed for: /var/renderd/renderd.sock with reason: Connection refused.

I tried starting the renderd:

sudo systemctl start renderd

If you have nginx installed and running, stop it and change ports to avoid conflict with apache. Then restart apache and renderd. It should solve it.

Thank you.

Advertisements

2 thoughts on “Deploying OpenStreetMaps

    • If you don’t have pre-rendering enabled, you’ll see your CPU will process and generate tiles while you load and zoom-in every time. And it will take time to browse the map. So it’s on-the-fly (on-the-go) rendering. After pre-rendering, it will just load the already (rendered) saved tiles without extensive CPU processing.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s