Deploying OpenStreetMaps


Hey everyone,

OpenStreetMap is the map of the world with open license and freedom. I deployed OpenStreetMaps on my system after seeing a lot of people having a lot of troubles deploying them. I used Ubuntu 16.04, i3 (2-cores, 4-threads), 4GB RAM system for this and this link as source: https://www.linuxbabe.com/linux-server/openstreetmap-tile-server-ubuntu-16-04.

You need to download map file for your region of interest. I download India’s map (~400MB pbf file). You can download from here and of course can choose another region. Now let’s continue as the tutorial source I mentioned takes us. I am not going to repeat all of that (DRY), it’s easy. After installing some dependencies and postgresql, I created osm user, created db, created extensions.

Download Map Stylesheet from https://github.com/gravitystorm/openstreetmap-carto/releases and download the latest version which at this time is v4.0.0.

Next, you are going to import the download map file into the PostgreSQL database. If you have a small amount of RAM, then it’s recommended to create a swap space to speed up the import process. I created 2GB for me with the fallocate command. I noticed that fallocate doesn’t support NTFS file system (and a few other too). I tried creating on ext4 file system on another drive and it works fine. Then we mount it as swap memory. You can check it using: cat /proc/swaps or top commands.

Now after installing the osm2pgsql tool, the actual importing process starts. Following command does it all:

osm2pgsql --slim -d gis -C 3600 --hstore -S openstreetmap-carto-2.41.0/openstreetmap-carto.style great-britain-latest.osm.pbf

3600 (3.6GB) here is the amount memory you want to allow the import process to be taken. You can try taking a little more. I tried the double of it and it gave error. -d is for database name (gis). Change the files and folders according to your own system e.g. openstreetmap-carto-2.41.0 will probably be different on your system and great-britain-latest.osm.pbf will be name/path to your map file you downloaded. You can also add additional argument –number-processes=4. I added because of the 4 threads of my CPU. Hence the above command in my case becomes:

osm2pgsql --slim -d gis -C 3600 --number-processes=4 --hstore -S openstreetmap-carto-4.0.0/openstreetmap-carto.style india-latest.osm.pbf

It takes time to import. So, go out and enjoy life (plug in the charger/if on server, use tmux). Okay, it’s finished finally.

Now time to compile the mod_tile module required to serve the tiles. If you run into some error during the compilation, refer to this for installing some missing dependencies. After it’s done, let’s move forward.

Generate Mapnik Stylesheet

You’ll be asked to go to openstreetmap-carto-xxx directory and execute get-shapefiles script. In the newer version, they have converted the bash script to Python script and moved it to script directory. That’s why it’s recommended to follow the official guide. So go to script directory and execute it as:

python get-shapefiles.py

And then run in previous directory,

carto project.mml > style.xml

Above command (carto) gave me error:

carto: Unexpected token s

It was due to very old version of carto installed. Found the hint at https://github.com/gravitystorm/openstreetmap-carto/issues/2583 to use higher version (>0.16) of carto.

Installed latest version:

sudo npm install -g carto

You’ll need to install nodejs for this (npm – node package manager) to work. Then I executed carto again and it worked, despite giving some warnings like:

Warning: using the name attribute for layers (like building-text here) is deprecated and will be removed in 1.0.0. Use id instead.

But it’s OK. So if yours one is still giving the same error (carto), try logging out and log in again. Check if your carto version is updated. Mine shows:

$ carto -v
 carto 0.18.0 (Carto map stylesheet compiler)

Move on to configuring the renderd and edit /usr/local/etc/renderd.conf and edit your details carefully as per your own system.

After configuring Apache, if you point yourself to the browser at http://localhost/osm_tiles/0/0/0.png (or whatever ip) and get a image showing a world map, then you are lucky. If you did’t, you probably have to check Apache logs (/var/log/apache2/error.log) and hence this makes you more lucky. In my case, the tail of log file contains:

/var/run/renderd/renderd.sock No such file or directory

So, I checked and there wasn’t renderd directory. So I created on and given appropriate permissions. Then I executed again and I got the map image. Congratulations!

Display Your Tiled Web Map

Now, it’s time to see the map zoom-able and span-able. There are two JavaScript libraries available: OpenLayer and Leaflet. You  just need to download and extract it to your root of your web server like (/var/www/html). Edit the index.html file and put the OpenLayer specific code. Make sure to edit lines like: http://your-ip/ol.css. Replace your-ip with your ip-address. If you are testing on your local system, then put localhost instead. Point your browser to localhost and you should see the map. Now the tiles are generated on-the-fly. If you want to pre-render the tiles so that your processor doesn’t do processing everytime you zoom-in then you can do it via:

render_list -m default -a -z 0 -Z 10

This command will take some time. And yes, you can stop and resume it anytime. It won’t start from start again. It will generate tiles for various zoom levels.

The issue I was having at this step was no map was being shown. Seeing the apache logs disclosed the error:

socket connect failed for: /var/renderd/renderd.sock with reason: Connection refused.

I tried starting the renderd:

sudo systemctl start renderd

If you have nginx installed and running, stop it and change ports to avoid conflict with apache. Then restart apache and renderd. It should solve it.

Thank you.

Using Qt UI files with PySide in FreeCAD


In Qt Designer app, create an interface. For example, I firstly created a form with two text widgets and then one push button.

qt ui design

Paste the following in the FreeCAD Python console:

from PySide import QtCore, QtGui

class CalculateSum:
    def __init__(self):
        # Importing the form from Qt UI file.
        self.form = FreeCADGui.PySideUic.loadUi("/home/mandeep/ui-design/inputform.ui")

        # Setting on-click behavior of Calculate button
        QtCore.QObject.connect(self.form.calculateButton, QtCore.SIGNAL("clicked()"), self.accept)

    def accept(self):
        """Will be called when Calculate button is clicked"""
        # Take inputs from both fields.
        v1 = int(self.form.val1.text())
        v2 = int(self.form.val2.text())
        print("Sum of "+str(v1)+" and "+str(v2)+": "+str(v1+v2))
    
# Show up in FreeCAD.
if FreeCAD.GuiUp:
    FreeCADGui.Control.showDialog(CalculateSum())

You will see the widget (Form) inputform.ui in the Combo View Tasks panel in FreeCAD. Enter values and click the Calculate button and the result will be printed in the Report View.

2017-05-08-001246_1366x768_scrot.png

Git Revert


git revert can be used to revert the changes you did in your git repository. One of the most useful case would be if you have already published your commits and of course, you won’t want to do a history rewrite. Then git revert lets you make a new commit cancelling out the effect of particular commit you specify.

git log

and note down the commit id of the latest commit you want to revert. Let us assume, it’s “b338b072ex1gd89a1a4eba8ge0d56jko“.

And then:

git revert b338b072ex1gd89a1a4eba8ge0d56jko

Then do git log again and see a new commit reverting the changes of last commit. Similarly, it can be done for multiple commits to, or a range of commits.

See more options @ http://stackoverflow.com/a/4114122

Video demo: https://asciinema.org/a/6479

Auto-mount disks on Startup


sudo fdisk -l

See which disk you want to auto-mount.

Suppose you attached a USB-drive and it’s shown as “/dev/sdb1” in above command.

Its file-system will also be shown beside that. Match your type from System name and notice the corresponding Linux type.

 

System name

English name

Linux type

 

W95 FAT32

Microsoft FAT32

vfat

 

W95 FAT32 (LBA)

Microsoft FAT32

vfat

 

W95 FAT16 (LBA)

Microsoft FAT16

vfat

 

W95 Ext’d (LBA)

Microsoft extended partition

Not used

 

NTFS volume set

Microsoft NTFS

ntfs

 

NTFS volume set

Microsoft NTFS with read-write access

ntfs-3g

 

Apple_HFS

Apple HFS

hfsplus

Suppose it’s W95 FAT32 (LBA) so your Linux type is vfat. If you are unsure about this type, you may consider mounting the device using: sudo mount /dev/sdb1 and then run mount command to see mounted file-systems and it’s file-system type.

After gathering all the necessary information, open /etc/fstab with root permissions with your text editor e.g. sudo vim /etc/fstab

Then add the new entry,

/dev/sdb1       /mnt/usb_drive       vfat       errors=remount-ro       0      0

to fill in line with the following.

<file system>     <mount point>   <type>       <options>         <dump>  <pass>

Then save the file and exit. You may check on the next reboot if it is mounted.

errors=remount-ro option is for the case when some error occurs, the device will get remounted as read-only. The <dump> option specifies if the file-system needs to be dumped (back-up) for safe keeping. 0 here means it doesn’t need to be dumped. And the <pass> means that the order which filesystem checks are to be performed at system reboot. The root filesystem should be labeled as 1, while others as 20 if we don’t want it to be checked. See man fstab command for more.

See “less” output at a time


There are a lot commands/tools that produce a lot of output that we can’t see at once (and without scrolling). Most of the commands on terminal can be made to show us whole output page by page using “less” command.

less command_here

less ls -R ~/

or you may pipe the output to less command,

ls -R ~/ | less

 

or similarly, to see the contents of a file:

cat file_name | less

There are many commands/tools that have their own prompt.

 

For IPython, you may do:

%page variable_name

 

Similarly for MySQL,

pager less -SFX

Or you may also try:

SELECT * FROM sometable \G

 

Above commands will show you the output from start that you can navigate through using the arrow up/down keys or space (page by page). Press “q” to exit.

Vim: Add some text at the start/end of each line


Suppose, you have a file with 1000 lines. You want to add some text at the beginning/end of each line. You won’t think of doing it manually (actually, I can’t stop you). Here, I am telling you how to save you a lot of time.

For example, we have the following lines in the file:

Name

Email

Phone Number

Address

And you want your output to be:

“Name”,

“Email”,

“Phone Number”,

“Address”,

and so on.

In Vim, you can simply type:

:%s/$/",/

It will place “, at the end of each line.

Similarly, for beginning:

:%s/^/"/

It will place ” at the beginning of each line.

Hence, we used $ for appending to end and ^ for prepending.

You can also do this for a few lines by visually selecting them with and then pressing : and type the command

s/^/"/

Skip the “%” because we do it while doing things globally across the file. So it will look like:

:'<,'>s/^/"/

Another method, if you want to type some text interactively on a couple of lines:

Ctrl+V

Select a few rows (downside, pressing j). After selecting the position in couple in all rows, press:

Shift + i

They type what you want. Then press the Esc key:

Esc

Now, wait 1 second. You’ll see same text to be replicated over all rows you selected.

Please share in comments, if you even easier method to do the similar.