ORS backend strange issue

#1

This is the forth day that i am trying to run the ORS inside and outside of docker and i keep running into the same issue, the analyze/process stage fails to complete.
It starts alright but after 25 mins CPU usage drops to almost 0, ors/health says not ready and it uses around 10.3 GB of RAM and there’s nothing in logs and the graph folder only contains gh.lock
I left it running for more than 24 hours but it changed nothing.
I’m trying to run north-america-latest.osm.pbf from ‘geofabrik.de’
My server has around 67GB of RAM and 22 vcores @ 3.22Ghz

My docker-compose.yml:
version: ‘3.0’
services:
ors-app:
container_name: ors-app
ports:
- 8080:8080
build:
context: …/
args:
APP_CONFIG: ./docker/conf/app.config.sample
OSM_FILE: ./docker/data/north-america-latest.osm.pbf
volumes:
- ./graphs:/ors-core/data/graphs
- ./elevation_cache:/ors-core/data/elevation_cache
- ./logs/ors/:/var/log/ors/
- ./logs/tomcat/:/usr/local/tomcat/logs
environment:
- JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms320G -Xmx60G -Xss3G -XX:MetaspaceSize=512m -verbose:gc
- CATALINA_OPTS= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost

#2

Are you using the docker from github or from docker hub? We are a ware of an issue in the current release where the command arguments are being ignored by the docker container due to a mistake in the docker file. A fix has been applied in the development branch that addresses this, so if you are getting it from github, it may be worth trying to do it from the development branch rather than master and see if that solves the problem. We are aiming to be doing a release in the next couple of weeks which would then include that fix into the master branch.

#3

I cloned from development-v6-CoreALT branch, replaced the file path in Dockerfile and docker-compose.yml, gave it more than 64 GB RAM for North America and also left in only one profile (vehicles-hgv) it started indexing and again, it CPU usage dropped to 0% after an hour or so but there’s nothing in logs but [GC (Allocation Failure) 41175050K->40859317K(44787480K), 0.1140131 secs]

api/health gives {"status":"not ready"}
and graphs/vehicles-car/ only contains gh.lock

-Thanks

#4

If your machine only has 67GB RAM, it’s generally not a good idea to tell the java inside docker to use 64GB as there is a fair amount needed for other services running on the machine. The message is basically saying that at around 40GB it is running out of space to actually do garbage collection. I would recommend trying to drop down the amount of RAM you allocate and then retry. Unfortunately, most of the time these sort of things are trial and error as there are a number of factors that affect the memory.

What is the exact Docker file and JAVA_OPTS that the server users?