Out of bounds exception when loading pbf file

DavidPerezIngeniero · July 14, 2023, 11:24am

I’m running openrouteservice with this config:

  openroutes:
    container_name: oroutes
    image: openrouteservice/openrouteservice:v7.1.0
    environment:
      user: ${usuario:-1002:1002}
      BUILD_GRAPHS: True
      JAVA_OPTS: "-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx3g"
      CATALINA_OPTS: "-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
    volumes:
      - ./rutas/mapas:/home/ors/ors-core/data/graphs
      - ./rutas/cache-elevacion:/home/ors/ors-core/data/elevation_cache
      - ./rutas/logs/ors:/home/ors/ors-core/logs/ors
      - ./rutas/logs/tomcat:/home/ors/tomcat/logs
      - ./rutas/conf:/home/ors/ors-conf
      - type: bind
        source: ./rutas/andorra-latest.osm.pbf
        target: /home/ors/ors-core/data/osm_file.pbf
        read_only: true

Andorra is a small country in Europe for testing.

When openroutes container starts it appears the following error:

14-Jul-2023 11:03:48.288 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
11:03:50.793 [ORS-Init] ERROR org.heigit.ors.routing.RoutingProfileManager - java.util.concurrent.ExecutionException: java.lang.RuntimeException: Couldn't process file /home/ors/ors-core/data/osm_file.pbf, error: Index 35787726 out of bounds for length 0
11:03:50.795 [ORS-Init] ERROR org.heigit.ors.routing.RoutingProfileManager - Failed to initialize RoutingProfileManager instance.
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Couldn't process file /home/ors/ors-core/data/osm_file.pbf, error: Index 35787726 out of bounds for length 0
        at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
        at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
        at org.heigit.ors.routing.RoutingProfileManager.initialize(RoutingProfileManager.java:181) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfileManager.getInstance(RoutingProfileManager.java:72) ~[classes/:7.1.0]
        at org.heigit.ors.servlet.listeners.ORSInitContextListener.lambda$contextInitialized$0(ORSInitContextListener.java:40) ~[classes/:7.1.0]
        at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: java.lang.RuntimeException: Couldn't process file /home/ors/ors-core/data/osm_file.pbf, error: Index 35787726 out of bounds for length 0
        at com.graphhopper.reader.osm.OSMReader.writeOsmToGraph(OSMReader.java:334) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:185) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.GraphHopper.importOSM(GraphHopper.java:740) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.gtfs.GraphHopperGtfs.importOSM(GraphHopperGtfs.java:67) ~[graphhopper-reader-gtfs-v4.8.1.jar:?]
        at com.graphhopper.GraphHopper.process(GraphHopper.java:704) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:667) ~[graphhopper-core-v4.8.1.jar:?]
        at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:166) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:193) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfile.<init>(RoutingProfile.java:140) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:39) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:25) ~[classes/:7.1.0]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        ... 1 more
Caused by: java.lang.IndexOutOfBoundsException: Index 35787726 out of bounds for length 0
        at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ~[?:?]
        at jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[?:?]
        at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266) ~[?:?]
        at java.util.Objects.checkIndex(Objects.java:361) ~[?:?]
        at java.util.ArrayList.get(ArrayList.java:427) ~[?:?]
        at com.graphhopper.storage.MMapDataAccess.getShort(MMapDataAccess.java:367) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.dem.HeightTile.getHeightSample(HeightTile.java:84) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.dem.HeightTile.getHeight(HeightTile.java:132) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.dem.AbstractTiffElevationProvider.getEle(AbstractTiffElevationProvider.java:155) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.dem.MultiSourceElevationProvider.getEle(MultiSourceElevationProvider.java:53) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.dem.ElevationProvider.getEle(ElevationProvider.java:52) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.osm.OSMReader.getElevation(OSMReader.java:689) ~[graphhopper-core-v4.8.1.jar:?]
        at org.heigit.ors.routing.graphhopper.extensions.ORSOSMReader.getElevation(ORSOSMReader.java:436) ~[classes/:7.1.0]
        at com.graphhopper.reader.osm.OSMReader.addNode(OSMReader.java:662) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.osm.OSMReader.processNode(OSMReader.java:630) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.osm.OSMReader.writeOsmToGraph(OSMReader.java:301) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:185) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.GraphHopper.importOSM(GraphHopper.java:740) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.gtfs.GraphHopperGtfs.importOSM(GraphHopperGtfs.java:67) ~[graphhopper-reader-gtfs-v4.8.1.jar:?]
        at com.graphhopper.GraphHopper.process(GraphHopper.java:704) ~[graphhopper-core-v4.8.1.jar:?]
        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:667) ~[graphhopper-core-v4.8.1.jar:?]
        at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:166) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:193) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfile.<init>(RoutingProfile.java:140) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:39) ~[classes/:7.1.0]
        at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:25) ~[classes/:7.1.0]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        ... 1 more

excerpt of my ors-config.json:

      "routing": {
        "enabled": true,
        "mode": "normal",
        "routing_description": "This is a routing file from openrouteservice",
        "routing_name": "openrouteservice routing",
        "sources": [
          "/home/ors/ors-core/data/osm_file.pbf"
        ],
```

DavidPerezIngeniero · July 14, 2023, 11:28am

I’ve downloaded the pbf file from: http://download.geofabrik.de/europe/andorra-latest.osm.pbf

DavidPerezIngeniero · July 17, 2023, 7:42am

`ArrayIndexOutOfBoundsException` during core graph preparation for hgv

opened 03:31PM - 03 Jun 22 UTC

closed 01:30PM - 15 Jun 22 UTC

aoles

bug

As [reported on the ask forum](https://ask.openrouteservice.org/t/java-util-conc…urrent-executionexception-java-lang-arrayindexoutofboundsexception-index-1048576-out-of-bounds-for-length-1048576/3932), build of [bengaluru.pbf](https://easyupload.io/fo7nul) against [app.config.txt](https://github.com/GIScience/openrouteservice/files/8833360/app.config.txt) fails with the following error. The underlying problem might be the same as for a [similar issue](https://github.com/GIScience/openrouteservice/issues/1149) reported earlier, which unfortunately we couldn't reproduce. ``` 03 Jun 17:13:41 WARN [extensions.ORSOSMReader] - Index 1048576 out of bounds for length 1048576. Way id = 363083979 03 Jun 17:14:06 ERROR [routing.RoutingProfileManager] - java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: Index 1048576 out of bounds for length 1048576 03 Jun 17:14:06 ERROR [routing.RoutingProfileManager] - Failed to initialize RoutingProfileManager instance. java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: Index 1048576 out of bounds for length 1048576 at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?] at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?] at org.heigit.ors.routing.RoutingProfileManager.initialize(RoutingProfileManager.java:178) ~[classes/:?] at org.heigit.ors.routing.RoutingProfileManager.getInstance(RoutingProfileManager.java:72) ~[classes/:?] at org.heigit.ors.routing.RoutingProfileManager.isInitComplete(RoutingProfileManager.java:668) ~[classes/:?] at org.heigit.ors.servlet.listeners.ORSKafkaConsumerInitContextListener.lambda$contextInitialized$0(ORSKafkaConsumerInitContextListener.java:47) ~[classes/:?] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: Index 1048576 out of bounds for length 1048576 at org.heigit.ors.routing.graphhopper.extensions.core.CoreAlgoFactoryDecorator.prepare(CoreAlgoFactoryDecorator.java:314) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.prepareCore(ORSGraphHopper.java:787) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.postProcessing(ORSGraphHopper.java:690) ~[classes/:?] at com.graphhopper.GraphHopper.process(GraphHopper.java:703) ~[graphhopper-core-v0.13.21.jar:?] at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:680) ~[graphhopper-core-v0.13.21.jar:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:169) ~[classes/:?] at org.heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:182) ~[classes/:?] at org.heigit.ors.routing.RoutingProfile.<init>(RoutingProfile.java:126) ~[classes/:?] at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:35) ~[classes/:?] at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:21) ~[classes/:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] ... 1 more Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: Index 1048576 out of bounds for length 1048576 at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?] at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?] at org.heigit.ors.routing.graphhopper.extensions.core.CoreAlgoFactoryDecorator.prepare(CoreAlgoFactoryDecorator.java:310) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.prepareCore(ORSGraphHopper.java:787) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.postProcessing(ORSGraphHopper.java:690) ~[classes/:?] at com.graphhopper.GraphHopper.process(GraphHopper.java:703) ~[graphhopper-core-v0.13.21.jar:?] at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:680) ~[graphhopper-core-v0.13.21.jar:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:169) ~[classes/:?] at org.heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:182) ~[classes/:?] at org.heigit.ors.routing.RoutingProfile.<init>(RoutingProfile.java:126) ~[classes/:?] at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:35) ~[classes/:?] at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:21) ~[classes/:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] ... 1 more Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1048576 out of bounds for length 1048576 at com.graphhopper.util.BitUtilLittle.toShort(BitUtilLittle.java:32) ~[graphhopper-core-v0.13.21.jar:?] at com.graphhopper.storage.RAMDataAccess.getShort(RAMDataAccess.java:225) ~[graphhopper-core-v0.13.21.jar:?] at org.heigit.ors.routing.graphhopper.extensions.storages.HeavyVehicleAttributesGraphStorage.hasEdgeRestriction(HeavyVehicleAttributesGraphStorage.java:163) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.edgefilters.core.HeavyVehicleCoreEdgeFilter.accept(HeavyVehicleCoreEdgeFilter.java:33) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.edgefilters.EdgeFilterSequence.accept(EdgeFilterSequence.java:29) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.core.PrepareCore.prepareNodes(PrepareCore.java:190) ~[classes/:?] at org.heigit.ors.routing.graphhopper.extensions.core.PrepareCore.doSpecificWork(PrepareCore.java:175) ~[classes/:?] at com.graphhopper.routing.util.AbstractAlgoPreparation.doWork(AbstractAlgoPreparation.java:30) ~[graphhopper-core-v0.13.21.jar:?] at org.heigit.ors.routing.graphhopper.extensions.core.CoreAlgoFactoryDecorator.lambda$prepare$0(CoreAlgoFactoryDecorator.java:300) ~[classes/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] ... 1 more ```

Shoud I open a bug in github?

vchalmel · July 17, 2023, 7:53am

Did you check the md5 sum after downloading the pbf file ? I sometimes had ors running better after getting osmium to sort the pbf file too.

DavidPerezIngeniero · July 17, 2023, 2:05pm

Thanks for your valuable suggestions.

I’ve redownloaded the file, and done:

osmium merge andorra.pbf -o andorra2.pbf

and used the new file.

Thre result:

IndexOutOfBoundsException: Index 35787726 out of bounds for length 0

vchalmel · February 5, 2024, 12:27pm

Hello @DavidPerezIngeniero did you found a solution ? I reloaded new osm files for western europe countries and got the same error.