Run ors in k8s with preload graphs

openrouteservice
1

Hi! I need ors in my k8s cluster. I run docker compose in local machine, downloaded pbf, waited for graphs generated. next i removed pbf and cache, restart composer – all works fine.
Up pod/service/stateful/etc in kubernetes and connected graphs over volume. ORS find my config and graphs, fine loaded, but on query i get error like
java.lang.ArrayIndexOutOfBoundsException: Index 237800 out of bounds for length 0.

What i do wrong?

All log from pod:

###########################
# Container sanity checks #
###########################
ⓘ Running container as user root with id 0 and group 0
▢ User and group are set to root with id 0 and group 0.
✓ ORS_HOME: /home/ors exists and is writable.
▢ Populated ORS_HOME=/home/ors with the default folders: files, logs, graphs, elevation_cache, config
▢ ors.engine.graphs_root_path= is set and not empty and not set to /
▢ ors.engine.elevation.cache_path= is set and not empty and not set to /
✓ Update the file /home/ors/config/example-ors-config.env with /example-ors-config.env
✓ Update the file /home/ors/config/example-ors-config.yml with /example-ors-config.yml
✓ Using the existing ors-config.yml from: /home/ors/config/ors-config.yml
ⓘ Default to graphs folder: /home/ors/graphs
▢ OSM source file set to /home/ors/files/data.osm
ⓘ Any ENV variables will have precedence over configuration variables from config files.
✓ All checks passed. For details set CONTAINER_LOG_LEVEL=DEBUG.
#####################################
# Container file system preparation #
#####################################
chown: /home/ors/config/ors-config.yml: Read-only file system
▢ Changed ownership of /home/ors to root
✓ Update the file /home/ors/files/example-heidelberg.osm.gz with /heidelberg.osm.gz
✓ Container file system preparation complete. For details set CONTAINER_LOG_LEVEL=DEBUG.
#######################################
# Prepare CATALINA_OPTS and JAVA_OPTS #
#######################################
▢ CATALINA_OPTS: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost 
▢ JAVA_OPTS: -Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx6g 
✓ CATALINA_OPTS and JAVA_OPTS ready. For details set CONTAINER_LOG_LEVEL=DEBUG.
ⓘ ##########################################
ⓘ # Config options and migration information #
ⓘ ##########################################
ⓘ >>> Migration information <<<
⚠ Configuring ors with a .json config is deprecated and will be removed in the future.
ⓘ You can use the ors-config-migration tool to migrate your .json config to .yml: https://github.com/GIScience/ors-config-migration#usage
ⓘ >>> Config options <<<
ⓘ You have the following options to configure ORS:
ⓘ Method 1 yml config:
ⓘ > docker cp ors-container-name:/home/ors/config/example-ors-config.yml ./ors-config.yml
ⓘ > docker run --name example-ors-instance-conf-file -e ORS_CONFIG_LOCATION=/home/ors/config/ors-config.yml -v $(pwd)/ors-config.yml:/home/ors/config/ors-config.yml openrouteservice/openrouteservice:latest
ⓘ Method 2 environment variables:
ⓘ > docker cp ors-container-name:/home/ors/config/example-ors-config.env ./ors-config.env
ⓘ > docker run --name example-ors-instance-env-file --env-file ors-config.env openrouteservice/openrouteservice:latest
ⓘ >>> End of migration information <<<
#####################
# ORS startup phase #
#####################
✓ 🙭 Ready to start the ORS application 🙭
▢ Startup command: java -Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx6g  -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost  -jar /ors.jar

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.1.6)

2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.Application                      ]   Starting Application v8.0.0 using Java 21.0.2 with PID 1 (/ors.jar started by root in /home/ors)
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.Application                      ]   The following 1 profile is active: "default"
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.ORSEnvironmentPostProcessor      ]   
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.ORSEnvironmentPostProcessor      ]   Configuration lookup started.
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.ORSEnvironmentPostProcessor      ]   Configuration file set by environment variable.
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.ORSEnvironmentPostProcessor      ]   Loaded file '/home/ors/config/ors-config.yml'
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.ORSEnvironmentPostProcessor      ]   Configuration lookup finished.
2024-05-08 15:36:28 INFO                                                  main [ o.h.o.a.ORSEnvironmentPostProcessor      ]   
2024-05-08 15:36:31 INFO                                              ORS-Init [ o.h.o.a.s.l.ORSInitContextListener       ]   Initializing ORS...
2024-05-08 15:36:31 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   Total - 1024 MB, Free - 929.53 MB, Max: 6 GB, Used - 94.47 MB
2024-05-08 15:36:31 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   ====> Initializing profiles from '/home/ors/files/data.osm' (1 threads) ...
2024-05-08 15:36:31 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   1 profile configurations submitted as tasks.
2024-05-08 15:36:31 INFO                                            ORS-pl-hgv [ o.h.o.r.RoutingProfile                   ]   [1] Profiles: 'driving-hgv', location: '/home/ors/./graphs/hgv'.
2024-05-08 15:36:31 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLMPreparationHandler   ]   Loaded landmark splitting collection from 
2024-05-08 15:36:32 INFO                                                  main [ o.h.o.a.Application                      ]   Started Application in 5.105 seconds (process running for 6.665)
2024-05-08 15:36:32 INFO                                                  main [ o.h.o.a.Application                      ]   openrouteservice {"build_date":"2024-03-21T13:55:54Z","version":"8.0.0"}
2024-05-08 15:39:19 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CorePreparationHandler     ]   Creating CH preparations, totalMB:3636, usedMB:3335
2024-05-08 15:39:19 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLMPreparationHandler   ]   Creating LM preparations, totalMB:3636, usedMB:3337
2024-05-08 15:39:19 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLandmarkStorage        ]   Created core node ID map for heavyvehicle_recommended_with_turn_costs_highways of size 256225
2024-05-08 15:39:19 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLandmarkStorage        ]   Created core node ID map for heavyvehicle_recommended_with_turn_costs_allow_all of size 256225
2024-05-08 15:39:19 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLandmarkStorage        ]   Created core node ID map for heavyvehicle_shortest_with_turn_costs_highways of size 256225
2024-05-08 15:39:19 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLandmarkStorage        ]   Created core node ID map for heavyvehicle_shortest_with_turn_costs_allow_all of size 256225
2024-05-08 15:39:36 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.c.CoreLMPreparationHandler   ]   Finished LM preparation, totalMB:5596, usedMB:3734
2024-05-08 15:39:36 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.ORSGraphHopper               ]   version v4.9.1|2024-01-17T09:08:46Z (7,20,5,4,5,7)
2024-05-08 15:39:36 INFO                                            ORS-pl-hgv [ o.h.o.r.g.e.ORSGraphHopper               ]   graph CH|heavyvehicle|RAM_STORE|3D|turn_cost|7,20,5,4,5, details:edges:13 767 079(473MB), nodes:10 966 790(210MB), name:(28MB), geo:109 865 627(420MB), bounds:-180.0,180.0,38.2066582,80.8271965,-89.0,3837.0, shortcuts:6 713 733 (129MB), nodesCH:10 966 790 (84MB), shortcuts:10 314 958 (315MB), nodesCH:10 966 790 (84MB), shortcuts:11 991 365 (366MB), nodesCH:10 966 790 (84MB)
2024-05-08 15:39:36 INFO                                            ORS-pl-hgv [ o.h.o.r.RoutingProfile                   ]   [1] Edges: 13767079 - Nodes: 10966790.
2024-05-08 15:39:36 INFO                                            ORS-pl-hgv [ o.h.o.r.RoutingProfile                   ]   [1] Total time: 185.564s.
2024-05-08 15:39:36 INFO                                            ORS-pl-hgv [ o.h.o.r.RoutingProfile                   ]   [1] Finished at: 2024-05-08 15:39:36.
2024-05-08 15:39:36 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   Total time: 185.598s.
2024-05-08 15:39:36 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   ========================================================================
2024-05-08 15:39:36 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   ====> Recycling garbage...
2024-05-08 15:39:36 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   Before: Total - 5.46 GB, Free - 1.82 GB, Max: 6 GB, Used - 3.65 GB
2024-05-08 15:39:37 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   After: Total - 6 GB, Free - 3.22 GB, Max: 6 GB, Used - 2.78 GB
2024-05-08 15:39:37 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   ========================================================================
2024-05-08 15:39:37 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   ====> Memory usage by profiles:
2024-05-08 15:39:37 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   [1] 2.56 GB (92.2%)
2024-05-08 15:39:37 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   Total: 2.56 GB (92.2%)
2024-05-08 15:39:37 INFO                                              ORS-Init [ o.h.o.r.RoutingProfileManager            ]   ========================================================================
2024-05-08 15:44:28 ERROR                                 http-nio-8082-exec-5 [ o.h.o.r.RoutingProfile                   ]   java.lang.ArrayIndexOutOfBoundsException: Index 237800 out of bounds for length 0
2

Hey @trin4ik,

could you please provide the query, as well as the response if you got one?

3

simple request, like

curl 'http://localhost:8080/ors/v2/directions/driving-hgv' \                                                           
  -H 'accept: application/json, application/geo+json, application/gpx+xml, img/png; charset=utf-8' \
  -H 'accept-language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7,as;q=0.6' \
  -H 'content-type: application/json' \
  -H 'origin: http://localhost:3000' \
  -H 'priority: u=1, i' \
  -H 'referer: http://localhost:3000/' \
  -H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: cross-site' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
  --data-raw '{"geometry_simplify":true,"instructions":true,"geometry":true,"maximum_speed":80,"coordinates":[[37.571758,55.441014],[37.371874,55.715966]]}'

this request to localhost. after success request, i remove files and elevation_cache directory, restart docker compose, try request and his success. I conclude that the graphs catalog is sufficient for the job. up my k8s app for ors, copy graphs directory and get this error on server.

i double check, all files, config and graphs normally readed by service (see in log), but i always get error. and i cant process files in k8s cluster. i have many low-memory nodes in cluster and cant reach 30gb+ for 1 task.

сan you confirm that only need the final files in the graphs directory to work? elevation_cache and files not needed service for work?

4

we moved our service to valhalla from ors (we were happy to use the ors api, but started exceeding limits and had to raise our routing locally). at now i dont see different with ors, but mb you say me the different?