When the graph is generate and first run problem

Hi I use docker compose with an osm from http://geofabrik.de/ and I have several questions:

First I wonder when the graph is build, during the image generation or during the first run of the container?

Second, I can’t find where the graph folder is locate on the installation, do you have any idea?

Third, at the end of the creation of the image, I have this on the log, this can be a problem or not:

Successfully tagged docker_ors-app:latest
WARNING: Image for service ors-app was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating ors-app ... 

ERROR: for ors-app  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

ERROR: for ors-app  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

With the image on the hub.docker (giscience/openrouteservice) the app works as expected I only have problem with the docker compose version


Some infos about the installation:

root@host5-vm105:~# docker version
Client: Docker Engine - Community
 Version:           19.03.3
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        a872fc2f86
 Built:             Tue Oct  8 01:00:44 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.3
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.10
  Git commit:       a872fc2f86
  Built:            Tue Oct  8 00:59:17 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683


root@host5-vm105:~# docker info
Client:
 Debug Mode: false

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 29
 Server Version: 19.03.3
 Storage Driver: vfs
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.15.18-5-pve
 Operating System: Ubuntu 19.04
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 12GiB
 Name: host5-vm105
 ID: 5UCL:RYY5:WB2I:7TYF:JWRL:TB4L:IVSB:MWIP:BHJH:BO2G:KTCU:ZLUD
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: 2803media
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

the docker-compose.yml:

root@host5-vm105:/home/openrouteservice/docker# cat docker-compose.yml
version: '3'
services:
    ors-app:
        container_name: ors-app
        ports:
        - 8080:8080
        build:
            context: ../
            args:
                APP_CONFIG: ./docker/conf/app.config.sample
                OSM_FILE: ./docker/data/france-latest.osm.pbf
        volumes:
            - ./graphs:/ors-core/data/graphs
            - ./elevation_cache:/ors-core/data/elevation_cache
            - ./logs/ors/:/var/log/ors/
            - ./logs/tomcat/:/usr/local/tomcat/logs
        environment:
            - JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms8g -Xmx8g
            - CATALINA_OPTS= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost

And the app.config (I only need isochrone):

root@host5-vm105:/home/openrouteservice/docker/conf# cat app.config.sample
{
   "ors":{
     "info": {
       "base_url": "https://openrouteservice.org/",
       "support_mail": "support@openrouteservice.org",
       "author_tag": "openrouteservice",
       "content_licence": "LGPL 3.0"
      },
      "services":{
         "matrix":{
            "enabled":false,
            "maximum_locations":200,
            "maximum_visited_nodes":100000,
            "allow_resolve_locations":true,
            "attribution":"openrouteservice.org, OpenStreetMap contributors"
         },
         "isochrones":{
            "enabled":true,
            "maximum_range_distance":[
               {
                  "profiles":"any",
                  "value":50000
               },
               {
                  "profiles":"driving-car, foot-walking",
                  "value":100000
               }
            ],
            "maximum_range_time":[
               {
                  "profiles":"any",
                  "value":18000
               },
               {
                  "profiles":"driving-car, foot-walking",
                  "value":3600
               }
            ],
            "maximum_intervals":10,
            "maximum_locations":2,
            "allow_compute_area":true,
            "attribution":"openrouteservice.org, OpenStreetMap contributors"
         },
         "routing":{
            "enabled":false,
            "mode":"normal",
            "sources":[
               "data/osm_file.pbf"
            ],
            "init_threads":1,
            "attribution":"openrouteservice.org, OpenStreetMap contributors",
            "distance_approximation":true,
            "profiles":{
               "active":[
                   "vehicles-car",
                   "vehicles-hgv",
                   "bike",
                   "pedestrian-walk"
               ],
               "default_params": {
                   "encoder_flags_size": 8,
                   "graphs_root_path": "data/graphs",
                   "elevation_provider": "multi",
                       "elevation_cache_path": "data/elevation_cache",
                   "elevation_clear_cache": true,
                   "instructions": true,
                   "maximum_distance": 100000,
                   "maximum_segment_distance_with_dynamic_weights": 100000,
                   "maximum_waypoints": 50
               },
               "profile-vehicles-car":{
                  "profiles":"driving-car",
                  "parameters":{
                     "encoder_options":"consider_elevation=true|turn_costs=true|block_fords=false|use_acceleration=true",
                     "elevation":true,
                     "preparation":{
                        "min_network_size":200,
                        "min_one_way_network_size":200,
                        "methods":{
                           "ch":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest"
                           },
                           "lm":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest,shortest",
                              "landmarks":24
                           }
                        }
                     },
                     "execution":{
                        "methods":{
                           "astar":{
                              "approximation":"BeelineSimplification",
                              "epsilon":1
                           },
                           "ch":{
                              "disabling_allowed":true
                           },
                           "lm":{
                              "disabling_allowed":true,
                              "active_landmarks":6
                           }
                        }
                     },
                     "ext_storages":{
                        "WayCategory":{

                        },
                        "WaySurfaceType":{

                        },
                        "Tollways":{

                        }
                     }
                  }
               },
               "profile-vehicles-hgv":{
                  "profiles":"driving-hgv",
                  "parameters":{
                     "encoder_options":"consider_elevation=true|turn_costs=true|block_fords=false|use_acceleration=true",
                     "elevation":true,
                     "preparation":{
                        "min_network_size":200,
                        "min_one_way_network_size":200,
                        "methods":{
                           "ch":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest"
                           },
                           "lm":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest,shortest",
                              "landmarks":24
                           }
                        }
                     },
                     "execution":{
                        "methods":{
                           "astar":{
                              "approximation":"BeelineSimplification",
                              "epsilon":1
                           },
                           "ch":{
                              "disabling_allowed":true
                           },
                           "lm":{
                              "disabling_allowed":true,
                              "active_landmarks":6
                           }
                        }
                     },
                     "ext_storages":{
                        "WayCategory":{

                        },
                        "HeavyVehicle":{
                           "restrictions":true
                        },
                        "WaySurfaceType":{

                        },
                        "Tollways":{

                        }
                     }
                  }
               },
               "profile-bike":{
                  "profiles":"cycling-regular",
                  "parameters":{
                     "encoder_options":"consider_elevation=true|turn_costs=true|block_fords=false",
                     "elevation":true,
                     "preparation":{
                        "min_network_size":200,
                        "min_one_way_network_size":200,
                        "methods":{
                           "ch":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest"
                           },
                           "lm":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest,shortest",
                              "landmarks":24
                           }
                        }
                     },
                     "execution":{
                        "methods":{
                           "astar":{
                              "approximation":"BeelineSimplification",
                              "epsilon":1
                           },
                           "ch":{
                              "disabling_allowed":true
                           },
                           "lm":{
                              "disabling_allowed":true,
                              "active_landmarks":6
                           }
                        }
                     },
                     "ext_storages":{
                        "WayCategory":{

                        },
                        "WaySurfaceType":{

                        },
                        "HillIndex":{

                        },
                        "TrailDifficulty":{

                        }
                     }
                  }
               },
               "profile-pedestrian-walk":{
                  "profiles":"foot-walking",
                  "parameters":{
                     "encoder_options":"consider_elevation=true|turn_costs=true|block_fords=false",
                     "elevation":true,
                     "preparation":{
                        "min_network_size":200,
                        "min_one_way_network_size":200,
                        "methods":{
                           "ch":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest"
                           },
                           "lm":{
                              "enabled":true,
                              "threads":8,
                              "weightings":"fastest,shortest",
                              "landmarks":24
                           }
                        }
                     },
                     "execution":{
                        "methods":{
                           "astar":{
                              "approximation":"BeelineSimplification",
                              "epsilon":1
                           },
                           "ch":{
                              "disabling_allowed":true
                           },
                           "lm":{
                              "disabling_allowed":true,
                              "active_landmarks":6
                           }
                        }
                     },
                     "ext_storages":{
                        "WayCategory":{

                        },
                        "WaySurfaceType":{

                        },
                        "HillIndex":{

                        },
                        "TrailDifficulty":{

                        }
                     }
                  }
               }
            }
         }
      },
      "logging": {
          "enabled": true,
          "level_file": "PRODUCTION_LOGGING.json",
          "location": "/var/log/ors",
          "stdout": true
      }
   }
}

it’s look like my graph folder is empty:

root@host5-vm105:/home/openrouteservice/docker# tree
.
|-- README.md
|-- conf
|   |-- app.config.sample
|   `-- catalina.properties
|-- data
|   |-- elevation_cache
|   |   `-- srtm_38_03.zip
|   |-- france-latest.osm.pbf
|   `-- picardie-latest.osm.pbf
|-- docker-compose.yml
|-- elevation_cache
|-- graphs
`-- logs
    |-- ors
    |   `-- ors.log
    `-- tomcat
        |-- catalina.2019-10-10.log
        |-- host-manager.2019-10-10.log
        |-- localhost.2019-10-10.log
        |-- localhost_access_log.2019-10-10.txt
        `-- manager.2019-10-10.log

8 directories, 13 files

When i look at the build log it’s strange it’s refer to OSM_FILE=docker/data/heidelberg.osm.gz but I didn’t use it I use another osm.

Step 4/29 : ARG APP_CONFIG=docker/conf/app.config.sample
 ---> Using cache
 ---> 11c26b25c090
Step 5/29 : ARG OSM_FILE=docker/data/heidelberg.osm.gz
 ---> Using cache
 ---> 4cc0747ba3a8

Did I miss something? Do I change the Dockerfile too?

it’s look like the build doesn’t generate the graphs for profiles… Very strange. I made a test with the heidelberg information from the github repo but I get the same result, empty graphs folder.

Sorry @henri to keep you waiting so long.

If you use docker-compose.yml, the OSM_FILE specified there will be used. The Dockerfile will pick that up. OSM_FILE=docker/data/heidelberg.osm.gz is just the default value. It will use whatever you specify in your docker-compose.

You can exec inside the container and inspect the /ors-core/openrouteservice directory. The data will be at /ors-core/data and mapped to your localhost. The osm_file there is the one which is used to build. It should be size of France in your case.

You need to give it time though. ORS will have to download all elevation tiles and so on. Give it half 30-60 mins. At some point you will feel it’s processing the graphs after downloading all relevant data.

hi @nils thanks for your reply!

I wonder if I can make the graph on a local machine and push the image create to hub.docker and bring back to my production server?

Will it embed the graph create locally or not? Or do the graph need to make during the first run after build a container from an image?

Thanks

Hm, I don’t think that’d work. It builds the graphs when the container is run, not during the build stage. That’s why you see the delay, it seems like it’s finished, but after the image built it start building the graphs.

What we do on our side, is having a small graph fetcher script, which pulls the graphs from a central graph building server to the production servers. ORS docker only expects the graphs to be in the /ors-core/data/graphs directory which is mapped to your ./graphs locally. If you copy the built graph files there and run the container, it won’t build the graphs from scratch, but work with those it has (if it’s the right profile, e.g. driving-car).

Will it be quicker if I disable matrix and route as I only need isochrone?

tx

Thanks that could be a nice solution:

Build the container form the image
After fetch the graph folder

I will try this solution

Na, no difference. You still need the network graph for all those endpoints. Disabling them only makes sense if your API is public facing.

thanks for the explaination

In this you’ll have 2 servers:

API server:

  • fetches graphs from builder server
  • THEN builds the image/runs the container

builder server:

  • only builds the graphs (i.e. runs a docker container after you flushed graphs folder) in intervals (e.g. weekly) to get OSM updates

Do I need to update the .osm.pbf in the /docker/data/, I guess so in order to get the OSM updates?

Yup, that’s right…

You put the graph files under /docker/ in the folder graphs and after build the image. Or you can update an existing image with the graph data from the builder server?

Sorry for this noob questions :wink: I just try to make it working with less pain !

Quite strage on my mac with 16G of RAM I get memory error:

Exception in thread "pool-11-thread-1" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at com.graphhopper.reader.ReaderWay.<init>(ReaderWay.java:45)
        at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.processWays(PbfBlobDecoder.java:295)
        at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.processOsmPrimitives(PbfBlobDecoder.java:376)
        at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.runAndTrapExceptions(PbfBlobDecoder.java:388)
        at com.graphhopper.reader.osm.pbf.PbfBlobDecoder.run(PbfBlobDecoder.java:400)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

And my config is pretty good on ram:

environment:
            - JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms14g -Xmx14g XX:MaxMetaspaceSize=512m
            - CATALINA_OPTS= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost

Any advice?

tx

Finally I wasn’t able to make it run on the mac (maybe too less RAM for running it).

On the Ubuntu server I had memory problem too for the generation of the graph… And I manage to make it work with those settings:

environment:
            - JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms8g -Xmx8g -XX:MaxMetaspaceSize=512m -XX:-UseGCOverheadLimit
            - CATALINA_OPTS= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost 

Now the docker is running (and need 12G of RAM for running the France osm), I don’t know if this amount is normal or not?

There’s so many things that depends on… The major one is how many profiles. Are you running all these:

                   "vehicles-car",
                   "vehicles-hgv",
                   "bike",
                   "pedestrian-walk"

Then that’s perfectly normal. Since you only need isochrones, you can also (for now) disable CH, that will save considerably on RAM. I can’t go into much more detail. Please consult with commercial sources for further optimization…

Thanks @nils I only use 2 profiles vehicles-car and pedestrian-walk.

To disable CH you mean do the following change in app.config.sample:

"profile-vehicles-car":{
                  "profiles":"driving-car",
                  "parameters":{
                     "encoder_options":"consider_elevation=true|turn_costs=true|block_fords=false|use_acceleration=true",
                     "elevation":true,
                     "preparation":{
                        "min_network_size":200,
                        "min_one_way_network_size":200,
                        "methods":{
                           "ch":{
                              "enabled":false,
                              "threads":8,
                              "weightings":"fastest"
                           },

This for each profile used, and after regenerate the image or no need?
Do I need to update the previous graph or can I still use the one already generated?

Thanks