Source point(s) [0, 1, 2, 3, 4, ..., n] out of bounds

ryuzakyl · May 27, 2019, 10:54pm

Hello.

I’m fairly new on this subject. I’m having an issue with the /ors/matrix API endpoint of a deployed instance of openrouteservice (an AWS server deployed using docker).

I’ve downloaded the data I needed (USA data) from here.

I made the corresponding changes to the docker-compose.yml (OSM_FILE: ./docker/data/north-america-latest.osm.pbf). I removed the graphs folder as suggested here, but I’m getting the following error:

API error response: b’{“error”:{“code”:6010,“message”:“Source point(s) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] out of bounds: -71.560526,42.56035870000001; -71.7385225,42.5233434; -71.726659,42.283460299999994; -71.53678599999999,42.347372;… Destination point(s) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] out of bounds: -71.560526,42.56035870000001; -71.7385225,42.5233434; …”},“info”:{“engine”:{“version”:“5.0.1”,“build_date”:“2019-05-27T16:36:30Z”},“timestamp”:1558995458026}}’

Note: If it helps, I’m using the API from Python3.

Is there something else I might be missing? What am I doing wrong?

Thank you in advance.

timothy · May 27, 2019, 11:01pm

Hi @ryuzakyl - thank you for using openrouteservice.

Can you please copy paste the contents of the graphs folder? du -sh * will do.
Also please can you provide your example request using the python API?
What happens if you curl localhost:8080/ors/health ?

Thank you, Tim

ryuzakyl · May 27, 2019, 11:12pm

Thank you very much @timothy for the quick response.

graphs folder content:

22M	bike
23M	pedestrian-walk
18M	vehicles-car
19M	vehicles-hgv

openrouteservice deployment location:

It is actually live and you can find it here.
I’m using requests package, nothing fancy. The response is: b’{“status”:“ready”}’.

Thank you for your help.

nils · May 28, 2019, 3:23pm

Can you please:

du -sh * inside one of the graphs folders?
show us the exact OSM file you downloaded? (edit: just saw, the above link is the PBF. Is that really what you have? How big is it?)
paste the whole docker-compose.yml?

ryuzakyl · May 28, 2019, 3:39pm

Thank you @nils.

Here’s the requested info:

du -sh * (inside graphs/bike):
2.1M edges
1.1M ext_hillindex
1.1M ext_traildifficulty
1.1M ext_waycategory
1.1M ext_waysurface
1.1M geometry
3.1M landmarks_fastest_bike-ors
3.1M landmarks_shortest_bike-ors
1.1M location_index
1.1M names
1.1M nodes
1.1M nodes_ch_fastest_bike-ors
36K properties
1.1M shortcuts_ch_fastest_bike-ors
4.0K stamp.txt
1.1M subnetwork_landmarks_fastest_bike-ors
1.1M subnetwork_landmarks_shortest_bike-ors
1.1M turn_costs

du -sh * (inside graphs/pedestrian-walk):
2.1M edges
1.1M ext_hillindex
1.1M ext_traildifficulty
1.1M ext_waycategory
1.1M ext_waysurface
2.1M geometry
3.1M landmarks_fastest_foot
3.1M landmarks_shortest_foot
1.1M location_index
1.1M names
1.1M nodes
1.1M nodes_ch_fastest_foot
36K properties
2.1M shortcuts_ch_fastest_foot
4.0K stamp.txt
1.1M subnetwork_landmarks_fastest_foot
1.1M subnetwork_landmarks_shortest_foot

du -sh * (inside graphs/vehicles-car):
1.1M edges
1.1M ext_tolls
1.1M ext_waycategory
1.1M ext_waysurface
1.1M geometry
2.1M landmarks_fastest_car-ors
2.1M landmarks_shortest_car-ors
1.1M location_index
1.1M names
1.1M nodes
1.1M nodes_ch_fastest_car-ors
36K properties
1.1M shortcuts_ch_fastest_car-ors
4.0K stamp.txt
1.1M subnetwork_landmarks_fastest_car-ors
1.1M subnetwork_landmarks_shortest_car-ors
1.1M turn_costs

du -sh * (inside graphs/vehicles-hgv):
1.1M edges
1.1M ext_hgv
1.1M ext_tolls
1.1M ext_waycategory
1.1M ext_waysurface
1.1M geometry
2.1M landmarks_fastest_heavyvehicle
2.1M landmarks_shortest_heavyvehicle
1.1M location_index
1.1M names
1.1M nodes
1.1M nodes_ch_fastest_heavyvehicle
36K properties
1.1M shortcuts_ch_fastest_heavyvehicle
4.0K stamp.txt
1.1M subnetwork_landmarks_fastest_heavyvehicle
1.1M subnetwork_landmarks_shortest_heavyvehicle
1.1M turn_costs

du -sh * (inside data/):
37M elevation_cache
8.4G north-america-latest.osm.pbf
15M tmp

cat docker-compose.yml:
version: ‘3’
services:
ors-app:
container_name: ors-app
ports:
- 80:8080
build:
context: …/
args:
APP_CONFIG: ./docker/conf/app.config.sample
OSM_FILE: ./docker/data/north-america-latest.osm.pbf
volumes:
- ./graphs:/ors-core/data/graphs
- ./elevation_cache:/ors-core/data/elevation_cache
- ./logs/ors/:/var/log/ors/
- ./logs/tomcat/:/usr/local/tomcat/logs
environment:
- JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g
- CATALINA_OPTS= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=0.0.0.0

Thanks in advance.

timothy · May 28, 2019, 5:07pm

Hi @ryuzakyl it looks like it it using the Heidelberg osm file. Sorry it isn’t very clear in the installation description how to replace it, we will update these instructions soon.

Please do the following:

Delete the contents of the ./graphs folder
Inside the osm container docker exec -it ors-app bash please navigate to the app.config which is in usr/local/tomcat/...... and make sure that it is pointing to north-america-latest.osm.pbf in data - and also change it to one active profile for now
And finally docker restart ors-app - wait a while, the pbf is fairly big.

p.s. you will need a higher Xmx setting to run north-america, can you try Xms4g -Xmx4g for one profile first and if it runs out of memory, try 8.

ryuzakyl · May 28, 2019, 9:53pm

Hi again.

I feel I’m closer to solving the issue with your suggestions, but not yet.

After following @timothy 's latest instructions, I’m getting the following (50 mins later):

Request to health endpoint
{“status”:“not ready”}

Checking docker/logs/ors/ors.log
Empty (no logs generated).

docker-compose.yml changes:

JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms8g -Xmx8g

Content/size/Tree structure of docker folder (I see several graphs folders)

Content of /usr/local/tomcat/webapps/ors/WEB-INF/classes/app.config

"routing":{
        "enabled":true,
        "mode":"normal",
        "sources":[
                **"/ors-core/data/north-america-latest.osm.pbf"**
        ],
    "init_threads":1,
    "attribution":"openrouteservice.org, OpenStreetMap contributors",
    "distance_approximation":true,
    "profiles":{
        "active":[
            "vehicles-car",
            "vehicles-hgv",
            "bike",
            "pedestrian-walk"
        ],
        "default_params": {
            "encoder_flags_size": 8,
            "graphs_root_path": "data/graphs",
            "elevation_provider": "multi",
            "elevation_cache_path": "data/elevation_cache",
            "elevation_clear_cache": true,
            "instructions": true,
            "maximum_distance": 100000,
            "maximum_segment_distance_with_dynamic_weights": 100000,
            "maximum_waypoints": 50
        },

What could I be missing?
Thank you both in advance.

nils · May 29, 2019, 6:02am

There are a few things:

Please only specify 1 active profile, e.g. driving-car. ORS builds 1 graph per profile and unless you beef your machine up to 32 GB RAM, it’s not a good choice to run 4 profiles.
What do the logs say? sudo docker logs <id>
Did you change the graphs_root_path parameter? However, the data/graphs folder should be where your graphs will be built.

You’ll have to be patient. A file of this size can take a full day to process. Watch the docker logs now and then, it’ll let you know when there are issues.

ryuzakyl · May 29, 2019, 8:02pm

Hi @nils. Thank you for your help.

I didn’t understand before when you guys adviced me to fix one profile. Now I get it and set vehicles-car profile as active (the one I need).

The issue now is apparently related to JVM’s Garbage Collector. I attach the output in case it has happened to you before:

29-May-2019 19:30:52.432 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/8.0.32
29-May-2019 19:30:52.434 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Feb 2 2016 19:34:53 UTC
29-May-2019 19:30:52.434 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         8.0.32.0
29-May-2019 19:30:52.434 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
29-May-2019 19:30:52.434 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            4.15.0-1039-aws
29-May-2019 19:30:52.435 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
29-May-2019 19:30:52.435 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /usr/local/openjdk-8/jre
29-May-2019 19:30:52.435 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.8.0_212-b04
29-May-2019 19:30:52.435 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
29-May-2019 19:30:52.436 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /usr/local/tomcat
29-May-2019 19:30:52.436 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /usr/local/tomcat
29-May-2019 19:30:52.436 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties
29-May-2019 19:30:52.436 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
29-May-2019 19:30:52.436 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.endorsed.dirs=/usr/local/tomcat/endorsed
29-May-2019 19:30:52.437 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat
29-May-2019 19:30:52.437 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat
29-May-2019 19:30:52.437 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp
29-May-2019 19:30:52.437 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
29-May-2019 19:30:52.557 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
29-May-2019 19:30:52.574 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
29-May-2019 19:30:52.576 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-8009"]
29-May-2019 19:30:52.578 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
29-May-2019 19:30:52.580 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 594 ms
29-May-2019 19:30:52.604 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service Catalina
29-May-2019 19:30:52.605 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.0.32
29-May-2019 19:30:52.629 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive /usr/local/tomcat/webapps/ors.war
29-May-2019 19:30:56.043 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.0.4.RELEASE)

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
<?xml version="1.0" ?>
<Configuration>
  <Properties>
    <Property name="filename">/var/log/ors/ors-logs.log</Property>
  </Properties>
</Configuration>
29-May-2019 19:31:01.801 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /usr/local/tomcat/webapps/ors.war has finished in 9,168 ms
29-May-2019 19:31:01.802 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/host-manager
29-May-2019 19:31:01.835 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/host-manager has finished in 33 ms
29-May-2019 19:31:01.843 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/docs
29-May-2019 19:31:01.854 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/docs has finished in 11 ms
29-May-2019 19:31:01.860 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/ROOT
29-May-2019 19:31:01.884 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/ROOT has finished in 24 ms
29-May-2019 19:31:01.885 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/manager
29-May-2019 19:31:01.904 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/manager has finished in 19 ms
29-May-2019 19:31:01.912 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory /usr/local/tomcat/webapps/examples
29-May-2019 19:31:02.151 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory /usr/local/tomcat/webapps/examples has finished in 239 ms
29-May-2019 19:31:02.168 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
29-May-2019 19:31:02.177 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
29-May-2019 19:31:02.200 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 9618 ms
Exception in thread "AsyncFileHandlerWriter-723074861" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1855)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2068)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.juli.AsyncFileHandler$LoggerThread.run(AsyncFileHandler.java:145)
Exception in thread "pool-11-thread-2" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.lang.Long.valueOf(Long.java:840)
	at com.google.protobuf.LongArrayList.get(LongArrayList.java:126)
	at com.google.protobuf.LongArrayList.get(LongArrayList.java:45)
	at java.util.AbstractList$Itr.next(AbstractList.java:358)
	at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.processWays(PbfBlobDecoder.java:304)
	at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.processOsmPrimitives(PbfBlobDecoder.java:376)
	at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.runAndTrapExceptions(PbfBlobDecoder.java:388)
	at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.run(PbfBlobDecoder.java:400)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Thank you very much once again.

nils · May 30, 2019, 5:32am

That’s not the garbage collector. That’s out of memory.

Increase the heap size, try 12 GB.

ryuzakyl · May 31, 2019, 1:43pm

Hi @nils.

I’ve tried now with 12GB, 16Gb, 24G, 32GB, 48GB, and nothing :(. Still getting the same error. Here’s the last output from the ors-app logs file:

31-May-2019 03:29:31.723 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
31-May-2019 03:29:31.728 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
31-May-2019 03:29:31.729 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 8165 ms
31 May 05:43:58 ERROR [routing.RoutingProfileManager] - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory. Old capacity: 1048576, new bytes:3584471456, segmentSizeIntsPower:20, new segments:3419, existing:1
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory. Old capacity: 1048576, new bytes:3584471456, segmentSizeIntsPower:20, new segments:3419, existing:1
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at heigit.ors.routing.RoutingProfileManager.initialize(RoutingProfileManager.java:216)
	at heigit.ors.routing.RoutingProfileManager.getInstance(RoutingProfileManager.java:64)
	at heigit.ors.servlet.listeners.ORSInitContextListener.lambda$contextInitialized$0(ORSInitContextListener.java:43)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory. Old capacity: 1048576, new bytes:3584471456, segmentSizeIntsPower:20, new segments:3419, existing:1
	at com.graphhopper.routing.lm.LMAlgoFactoryDecorator.loadOrDoWork(LMAlgoFactoryDecorator.java:304)
	at com.graphhopper.GraphHopper.loadOrPrepareLM(GraphHopper.java:1294)
	at com.graphhopper.GraphHopper.postProcessing(GraphHopper.java:915)
	at heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.postProcessing(ORSGraphHopper.java:495)
	at com.graphhopper.GraphHopper.process(GraphHopper.java:692)
	at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:661)
	at heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:156)
	at heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:158)
	at heigit.ors.routing.RoutingProfile.<init>(RoutingProfile.java:103)
	at heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:37)
	at heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:21)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory. Old capacity: 1048576, new bytes:3584471456, segmentSizeIntsPower:20, new segments:3419, existing:1
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at com.graphhopper.routing.lm.LMAlgoFactoryDecorator.loadOrDoWork(LMAlgoFactoryDecorator.java:300)
	... 16 more
Caused by: java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory. Old capacity: 1048576, new bytes:3584471456, segmentSizeIntsPower:20, new segments:3419, existing:1
	at com.graphhopper.storage.RAMDataAccess.ensureCapacity(RAMDataAccess.java:108)
	at com.graphhopper.routing.lm.LandmarkStorage.createLandmarks(LandmarkStorage.java:215)
	at com.graphhopper.routing.lm.PrepareLandmarks.doWork(PrepareLandmarks.java:135)
	at com.graphhopper.routing.lm.LMAlgoFactoryDecorator$1.run(LMAlgoFactoryDecorator.java:290)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	... 6 more

Here’s the JAVA_OPTS and CATALINA_OPTS from my docker-compose.yml file:

environment:
    - JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms48g -Xmx48g -verbose:gc
    - CATALINA_OPTS= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=0.0.0.0

Any help would be appreciated.
Thank you once again.

timothy · May 31, 2019, 1:52pm

@ryuzakyl sorry that this isn’t working yet.

I have the feeling that the Java settings aren’t updated in the container itself, so let’s do the following:

docker-compose down
remove all the docker containers (docker rm -f ...) related to openrouteservice and the docker images (docker rmi ...
docker-compose up -d

Also, please use a much smaller pbf file for now, that we can debug quicker, maybe https://download.geofabrik.de/north-america/us/district-of-columbia.html

ryuzakyl · June 2, 2019, 12:23am

Hello @timothy. I’m sorry I couldn’t get back to you yesterday.

I tried your suggestions without success. Although the “indexing” process worked correctly [both with district-of-columbia-latest.osm.pbf (~16M) and us-west-latest.osm.pbf (~1.7G) data files] without errors, my API calls keep failing :(.

I fear that my usage of the API is not correct and that might be the source of the error. I’ll post some data for further analysis.

district-of-columbia-latest.osm.pbf (~16M):

/graphs/vehicles-car folder size: ~22M
DC first 10 data points:
[
[38.908229, -77.030539],
[38.897244, -77.04133440000001],
[38.915648700000006, -77.07371490000001],
[38.9451658, -77.0622028],
[38.9191485, -77.03629670000001],
[38.933904600000005, -77.030539],
[38.9656971, -77.0622028],
[38.937480799999996, -77.08522579999999],
[38.8628755, -76.9700669],
[38.8761083, -77.0190228]
]

us-west-latest.osm.pbf (~1.7G):

/graphs/vehicles-car folder size: ~2.7G
California first 10 data points
[
[34.0748887, -118.3082034],
[34.0470832, -118.2965121],
[34.043925, -118.2424291],
[34.0390107, -118.2672801],
[34.054379700000005, -118.2672801],
[34.0224471, -118.2000277],
[34.0631451, -118.4367551],
[34.0448583, -118.4484367],
[34.1220545, -118.29358909999998],
[34.0502898, -118.2117257]
]

Moreover, I also tried to use a list of sources (us-midwest-latest.osm.pbf, us-northeast-latest.osm.pbf, us-pacific-latest.osm.pbf, us-south-latest.osm.pbf, us-west-latest.osm.pbf) in the app.config file, as sources is indeed an array in JSON. This was an attempt to mitigate the memory error I’ve been having with the north-america-latest.osm.pbf data file. The result was the same (Source point(s) [0, 1, …, n] out of bounds), but I noticed something weird. The size of the /graphs/vehicles-car folder was ~2.7G. Much less than expected, IMHO.

Thank you once again for all your help.

nils · June 2, 2019, 12:58am

Hi,

no, you can’t use multiple sources (yet). If you must, best to merge PBF’s with osmium. It likely only took the latter, that’s why it’s the same size as us-west-latest.osm.pbf, is my guess.

But hey, this looks a lot better now! And I think (and dearly hope) that it’s only a coordinate order issue. Try and switch lat/long (ORS expects [lon, lat], not [lat, lon]).

I guess your PC is not really made for entire US. How much RAM do you have available?

ryuzakyl · June 2, 2019, 3:17am

Hi @nils.

After using [lon, lat] instead of [lat, lon] my API calls finally worked . Thank you.

I guess that merging the PBF’s with osmium could be an option. However, I think that the resulting file would have eventually the same size (~8.4 GB) than the north-america-latest.osm.pbf file. This is the only issue I think remains unsolved so far.

The current specs of my AWS instance (m4.4xlarge) are:

CPU: 16
RAM: 64 GB
HDD: 32 GB

I’m not entirely sure that the indexing process is using the whole 64GB of RAM. Although I might be wrong. Perhaps it is a misconfiguration on my part regarding my JAVA_OPTS and/or CATALINA_OPTS.

Once again, thank you very much.

nils · June 2, 2019, 7:51pm

Happy that worked now at least!

Hm, and you still get a OutOfMemory error, even with -Xmx/-Xms set to 25g or higher? Maybe also try the -XX:MetaspaceSize set to 512m, it’s been known to make trouble as well. Can’t remember though if that also threw a OutOfMemory error.

Sorry man, it shouldn’t be this painful… BTW, we just pushed a hotfix for elevation data, you might wanna pull that if you didn’t yet.

ryuzakyl · June 4, 2019, 9:29pm

Hi @nils.

Thanks for the heads up regarding the update .

Unfortunately, I’m still getting the same error .

I’ve tried tuning the following parameters:

-XX:MetaspaceSize values: 512M, 1G, 2G
-Xss values: 1G, 2G
-Xms/-Xmx values: 48GB, 60GB

I don’t know what else to do. Any suggestion is greatly appreciated.
Thank you.

nils · June 5, 2019, 10:21am

Ok, no idea anymore… @adam, @timothy?

timothy · June 5, 2019, 10:47am

@ryuzakyl which osm file (+ size) now actually worked with how many active profiles? Can you make sure that the docker container Java settings have been used (I suggest to use VisualVM for this), accessed via the JMX port 9001.

ryuzakyl · June 5, 2019, 8:45pm

Hi @timothy.

The largest *.osm.pbf file to work was ~2.9G with only one active profile, vehicles-car.

I tried to use VisualVM to determine the Java settings employed, but I was unable to. I have to access my AWS instance via tunneling with a VPS (connectivity restrictions in my country). Then I access the AWS instancia via SSH with a credentials file. I couldn’t find how to feed my AWS *.pem credentials file to VisualVM.

Is there a bash-based local solution inside the container? I think that tools like VisualVM don’t apply in my situation.

Thanking you once again.
Best regards.