Specs needed for running openrouteservice locally

[johntdata]

I recently found this service and API and think it’s totally awesome. Such a brilliant service. I want to do some quite dense network analysis using routing (and animating the resulting geojson) which would greatly exceed the 2,000 API calls daily limit so am considering a local setup. I have seen there are instructions here although at first glance they look beyond my level of technical competence. https://github.com/GIScience/openrouteservice#installation
Nevertheless I have two questions:

1. What kind of specs are required (principally RAM and disk space) to run this service locally, using only Scotland as the input file? I have an OKish windows 64 desktop, i7 6700, 16GB ram

2. How hard is it to install and set the thing up? I’ve never heard of docker or Maven, so there’s probably no point attempting something that’s well beyond my comfort zone. On the other hand I’m fairly good with computers so this could be a chance to learn. I guess it’s a bit like doing mechanics on your car: I could change a battery but I’d never embark on changing a head gasket, even though there’s a book with clear instructions.

For interest, this is the kind of thing I’ll be doing, but on a much bigger scale https://twitter.com/JohnTdata/status/1230860015864598528

Thanks.

[nils]

1. that’ll cover entire Europe, no worries
2. learn docker, you’ll love it. Your future self will be very thankful for whatever service you’ll set up in the next years

[johntdata]

OK Thanks Nils, I will give it a go!

On this page (https://hub.docker.com/r/openrouteservice/openrouteservice) there is a link to the docker installation guide which returns a 404:

https://hub.docker.com/r/openrouteservice/docker/README.md

I thought I would point that out. Otherwise I have found instructions here

https://hub.docker.com/r/openrouteservice/openrouteservice
https://hub.docker.com/r/giscience/openrouteservice

GIScience/openrouteservice

If there are any key sources of instructions please let me know. Thanks

[leungi]

Provided you have docker installed.

1. docker pull openrouteservice/openrouteservice
2. docker run -p <desired host port>:8080 openrouteservice/openrouteservice

Test service by going to a browser and hit <host machine IP>:<desired host port>/ors/health (e.g., http://localhost:8080/ors/health).

As per official guide, you should get a status = ready if all goes well.

[johntdata]

Thanks @leungi. It’s actually a bit more complicated than that, and the help information is patchy and difficult to follow if you’re not a programmer. But here are the steps I took to get this working on Windows 10 Pro, hopefully they might be useful to anyone else hitting their head against a brick wall.

1. Install Docker for Windows https://hub.docker.com/editions/community/docker-ce-desktop-windows/

2. Enable CPU virtualization in BIOS, this will differ by machine https://www.bleepingcomputer.com/tutorials/how-to-enable-cpu-virtualization-in-your-computer-bios/ If you don’t have Windows 10 Pro edition I’m not sure if you can complete this step, but this guy managed it somehow so check out his post Couldn't Process File data/osm_file.pbf during Docker Install Process

3. In Docker Desktop (you may need to access its settings window from the icon in the system tray: right click -> Settings) you need to go into Resources -> File Sharing and add the folder where you want your files to live. For me it’s C:\Temp\Docker2 because I messed up the first time.

4. Now you can go to the windows command prompt (Windows key -> cmd) and run this command which does a bunch of stuff behind the scenes which I don’t really understand:
   docker pull openrouteservice/openrouteservice

5. Now it’s time to get some openstreetmap data as a pbf file. I used just Scotland [.osm.pbf] which I downloaded from here http://download.geofabrik.de/europe/great-britain.html I saved that file to here C:\Temp\scotland-latest.osm.pbf

6. Now we need to run docker, there are instructions here but they didn’t make a lot of sense to me as a Windows user https://github.com/GIScience/openrouteservice/blob/master/docker/README.md

Evetually I figured out that I had to paste a command into the windows command line, and this is the one that worked:

docker run -dt --name ors-app -p 8080:8080 -v C:/Temp/Docker2/graphs:/ors-core/data/graphs -v C:/Temp/Docker2/elevation_cache:/ors-core/data/elevation_cache -v C:/Temp/Docker2/conf:/ors-conf -v C:/Temp/scotland-latest.osm.pbf:/ors-core/data/osm_file.pbf -e "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g" -e "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost" openrouteservice/openrouteservice:latest

You will obviously need to update the folder paths depending on your file structure, but you ought to be able to do that easily. Just use notepad to find/replace any instance of C:/Temp/Docker2…

7. THIS IS REALLY IMPORTANT - If the command above worked, the command prompt will return a long string like this 8a405afe2d915f5376ce2e12afd1bc67bd56edc499742701cb656360daff846f

That’s a good sign, and although there’s no progress or confirmation, stuff is happening behind the scenes. Open Task Manager and check that Vmmem is using CPU and doing stuff. I think that docker is builing graphs (edges and nodes of your road network) but it does that silently. So don’t give up at this point or repeat the command. It’s incredibly unclear what’s going on at this stage but I guess once you know you know.

8. Now you can test it by opening a url like this in your browser http://localhost:8080/ors. The instructions talk about installing tomcat but I didn’t need to do this. Note that unless you have the Scotland data installed, this won’t work http://localhost:8080/ors/v2/directions/driving-car?start=-3.167209,55.974753&end=-3.398842,56.081019

9. The settings all live in a file called app.config in a directory called conf: C:\Temp\Docker2\conf\app.config
   You can open this file in a text editor to change it. For example I increased the max distance from 100km to 1000km. However these settings won’t make any difference until you restart the docker container

10. To restart the container you first need an id for your container. Back in windows command prompt type
    docker container ls –aq
    Now use the output to amend this command:
    docker container restart 8a405afe2d91

Good luck! It’s an awesome tool, and it’s free so you can’t really complain, and the support here seems to be excellent. But the help and instructions for newbie windows users like me are poor. These links might be useful as there seems to be documentation in loads of different places:

GIScience/openrouteservice

GIScience/openrouteservice/blob/master/docker/README.md

https://hub.docker.com/r/openrouteservice/openrouteservice

GIScience/openrouteservice

https://openrouteservice.org/dev/#/api-docs/v2/directions/{profile}/post

GIScience/openrouteservice-docs

You may also see links to this page which doesn’t exist https://hub.docker.com/r/openrouteservice/docker/README.md

[adam]

Thanks @johntdata for that info, it looks really useful

The main problem we have here is that no one from the openrouteservice developer team actually uses Windows - we are all Linux and Mac users so it’s a bit trickier to document getting the setup running with Docker in a Windows environment. But that’s one of the great benefits of having a forum like this - that people like yourself can share their knowledge and experience

[leungi]

Amazingly detailed; the bit to link the individual volumes to the container is key

The community will benefit a lot from your knowledge share

[onkel_butterbrot]

for guys encountering trouble with the johntdata’s code, use this instead:

docker run -dt --name ors-app -p 8080:8080 -v C:/Temp/Docker2/graphs:/ors-core/data/graphs -v C:/Temp/Docker2/elevation_cache:/ors-core/data/elevation_cache -v C:/Temp/Docker2/conf:/ors-conf -v C:/Temp/dach-latest.osm.pbf:/ors-core/data/osm_file.pbf -e “JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g” -e “CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost” openrouteservice/openrouteservice:latest

as previously edit/update the path folders to your file structure.

[JosephMurphy]

Hi,
If anyone has encountered the following error below and resolved this, please could you kindly let me know? Any help would be much appreciated

Also, as the latest docker desktop on windows uses WSL2, I can’t File share?

### openrouteservice configuration ###

No ors-config.json in ors-conf folder. Copy config from /ors-core/openrouteservice/src/main/resources/ors-config.json

cp: cannot create regular file '/ors-conf/ors-config.json': Permission denied

### Package openrouteservice and deploy to Tomcat ###

Thanks in advance
Joe

[amandus]

Hi @JosephMurphy,

could you give a few more details on how exactly you are running openrouteservice with docker on your machine?

Did you follow the installation guide?

Best regards

[JosephMurphy]

Hi @amandus,

Thanks for the quick response. I followed the instructions from here: Dipayan Banerjee - Local openrouteservice setup (Windows 10 Home)

This results in the following error: No ors-config.json in ors-conf folder (Docker) - #4 by dipayan

I don’t have a programmer background so find it hard to comprehend the instructions you provided a link to.

The step I am not quite sure about is:

# create directories for volumes to mount as local user
mkdir -p conf elevation_cache graphs logs/ors logs/tomcat

Cheers
Joe

[Steve]

Hi,

Regarding the specs - I have a similar PC (even slightly better processor), but I run into memory problems in Docker (e.g. “Java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory.”).

Am I missing something? Any suggestions to make things work with my configuration?

Thanks!

[jschnell]

Hey,
there’s two java settings controlling this:

@onkel_butterbrot

-Xms1g -Xmx2g

The first sets the available memory to 1 GB. The seconds sets the maximum memory used to 2 GB. Changing these to higher values should fix your OOM errors - look at the examples above for how to use it.

Best regards

[Steve]

Over the past days I made some further attempts. Using a VM with 4 CPUs, 64 GB RAM (32 GB of which virtual) and 200 GB disk space, and Xms8g -Xmx24g, I still receive OOM error messages (16 hours after process start):

ors-app | 16 Oct 10:39:28 ERROR [routing.RoutingProfileManager] - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space

I’m immensely grateful for the forum’s help so far, and would really appreciate any pointers what I should do differently to load Europe.

Thanks!

[jschnell]

Hey,

the whole of europe is huge - 25GB for the pbf alone!
As suggested in the configs, we’d recommend having at least twice the RAM available. Also, having this run for a long time is not unexpected.

In your case, I’d suggest trying to build a smaller region first, to check whether everything is running. Leave everything as is and try building a smaller country, like Albania (43MB) or Austria (~600MB).
Remember to change your osm-file-config and to delete your graphs beforehand, otherwise this will run into problems as well.

If this works, you can reduce the memory footprint by changing the stuff affecting RAM listed in our system requirements.

Best regards

[Steve]

Hi,

Thanks for your suggestions. I set up a dedicated VM with 32 GB RAM, topped up to 128 GB using Virtual RAM (swap).

I’ve first tried some smaller regions like Africa (5.5 GB) and South America (2.9 GB) and those run fine.
However, I’m starting to run into problems from Asia (11.2 GB) onwards.

Still, I get error messages like below.

Any advice?

ors-app | Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
ors-app | java.lang.IllegalArgumentException: Invalid character found in method name [0x160x030x010x000xca0x010x000x000xc60x030x030xc50x15hAla~0x080xab0xac0x0f0xfa0xa3s0x960x1a0x14?,0xe5l20xa4+0x8a0x170xabk0xfd0x1f0xe40xc50x000x00h0xcc0x140xcc0x130xc0/0xc0+0xc000xc0,0xc00x110xc00x070xc0’0xc0#0xc00x130xc00x090xc0(0xc0$0xc00x140xc00x0a0xcc0x150x000x9e0x000x9f0x00g0x00k0x0030x0090x000x9c0x000x9d0x000x050x000x040x00<0x00=0x00/0x0050xc00x120x000x160x000x0a0x000x030x000x080x000x060x000x140x000x110x000x190x000x170x0020xc00x080x000x120x000x130x000x150x0080x00@0x00f0x00j0x000xa20x000xa30x010x000x0050x000x050x000x050x010x000x000x000x000x000x0a0x000x080x000x060x000x170x000x180x000x190x000x0b0x000x020x010x000x000x0d0x000x0c0x000x0a0x040x010x040x030x020x010x020x030x020x020xff0x010x000x010x000x000x0f0x000x010x01…]. HTTP method names must be tokens
ors-app | at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:434)
ors-app | at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:511)
ors-app | at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
ors-app | at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
ors-app | at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1651)
ors-app | at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
ors-app | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
ors-app | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
ors-app | at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
ors-app | at java.base/java.lang.Thread.run(Thread.java:829)
ors-app | 31-Oct-2022 20:37:46.679 INFO [http-nio-8080-exec-4] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header

[jschnell]

Hey,

this reads like an error with your request - are you sure that this is related to building the graphs?
If so, could you pastebin the rest of the output?

Best regards