Unable to add new pbf file when using docker

gwfami2 · August 4, 2020, 5:32am

Hi all,
I’m currently attempting to set up my own local ORS server on an iMac , System Version: macOS 10.14.6. Using the instructions at https://github.com/GIScience/openrouteservice/tree/master/docker, I’ve gotten Docker and Tomcat up and running. I’ve verified the local server and was able to run http://localhost:8080/ors/routes?profile=driving-car&coordinates=8.676581,49.418204|8.692803,49.409465 and received a good return as per the instructions.

I downloaded the Kansas pbf file from Geofabrik Download Server, and made the changes as per the github instructions to point to the new pbf file, deleted the container via Docker gui, and rebuilt it using docker-compose on the command line. This didn’t set it to the new pbf file, but was still using the test data set (Heidelberg).

Figuring that the build command should be included when docker-compose is used, I used he following:

docker-compose --verbose up -d --build

Now it grinds hours creating the container, stuck on a specific point.

Looking at the activity monitor, I’m seeing that it hits a point (14), where it decides to look outside of the directory it is in and examine everything I have just one up from where it is. It’s looking at every file and folder I have on the desktop ,many of which contain gigabytes of files that have no bearing on the app. I messed around with the .dockerignore file, but wasn’t able to figure it out. I’ve had to force quit, clean up, and try again a couple of times, with the same results. It seems to only do this if the following boldfaced lines are uncommented in the docker-compose.yml file shown below.

"version: ‘2.4’
services:
ors-app:
container_name: ors-app
ports:
- 8080:8080
image: openrouteservice/openrouteservice:latest
build:
context: …/
args:
APP_CONFIG: ./openrouteservice/src/main/resources/app.config.sample
OSM_FILE: ./openrouteservice/src/main/files/kansas-latest.osm.pbf

volumes:
  - ./graphs:/ors-core/data/graphs
  - ./elevation_cache:/ors-core/data/elevation_cache
  - ./logs/ors:/var/log/ors
  - ./logs/tomcat:/usr/local/tomcat/logs
  - ./conf:/ors-conf

environment:
  - BUILD_GRAPHS=TRUE  # Forces the container to rebuild the graphs, e.g. when PBF is changed
  - "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g"
  - "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"

Here’s a list of what the activity monitor is showing that it is doing. The docker-compose command was ran /Desktop/openrouteservice (boldfaced in pitiful directory tree below)., but notice below that 14 is checking files outside the directory. I watched it for a while and it just kept going through each file, one at a time, taking forever as there are several thousand files, if not tens of thousands in those folders. The boldface line would change as it examined a new file.

0
/dev/ttys001
1
/dev/ttys001
2
/dev/ttys001
3
/dev/urandom
4
/dev/urandom
5
[ctl com.apple.netsrc id 8 unit 18]
6
->0x7a674c817b52bac9
7
->0x7a674c817b52b619
8
->0x7a674c817b52beb1
9
->0x7a674c817b52d0a9
10
->0x7a674c817b52d491
11
->0x7a674c817b52dd29
12
->0x7a674c817b52ce51
13
/private/var/folders/st/p8wvpyz17m75fd86np5bstmr0000gn/T/tmpzs85_33f
14
/Users/xxx/Desktop/Aaron1/Aaron 2/Aaron recovered.zip

XX DESKTOP
├─|
│ ├── A1
│ ├── A2
│ ├── A3
│ ├── A4
│ ├── Apowersoft\ Screenshot.app
│ ├── A5
│ ├── Bee\ fumer
│ ├── Blender
│ ├── Blender\ 2_8_3.8
│ ├── Blender\ 2_8__3.8
│ ├── CKAN.app
│ ├── Camera
│ ├── Chair\ photos
│ ├── Desktop\ 5:1:20
│ ├── EtreCheck.app
│ ├── Firefox\ quantum.app
│ ├── Fish\ Inv.
│ ├── FreeCAD.app
│ ├── Google\ Web\ Designer.app
│ ├── Invention\ stuff
│ ├── A6
│ ├── Keka.app
│ ├── Lake\ Distances
│ ├── Malwarebytes\ Anti-Malware.app
│ ├── A7m
│ ├── Neat\ things\ to\ print
│ ├── OnyX.app
│ ├── Phone\ backups
│ ├── Photon\ S
│ ├── Plans
│ ├── Research
│ ├── Rural\ Development
│ ├── Slic3r.app
│ ├── Storage\ container
│ ├── Sub\ plane\ and\ motor
│ ├── Subnautica\ Below\ Zero.app
│ ├── app
│ ├── archive
│ ├── blender-2.80.0-git20190215.faa255c0a4e4-x86_64
│ ├── A8\ phone\ backup
│ ├── lakes
│ ├── meshlab.app
│ ├── openrouteservice
│ ├── test
│ ├── test\ copy
│ ├── untitled\ folder
│ └── untitled\ folder
└──

I’m certain I’m missing something, but I have no idea what it is, and I can’t take the time to have the docker-composer look at every single file on the desktop when it only needs whats in one folder on the desktop. Any help will be greatly appreciated.

Pretty sure it’s an ID:10T error.

adam · August 4, 2020, 6:36am

Hi @gwfami2,

there is a PR waiting to be merged into master branch on github which should hopefully fix the issue of not using a customise app.config - we’re just waiting for some more information from the original author of the PR before we can do the merge.

As a quick look though, the first thing that jumps out is that in your docker-compose file, for context you have three dots in the path - maybe that is what could be causing the problems you see with it scanning every folder… In Ubuntu you can’t use three dots anyway, though maybe in Macs it is a different story.

We have recently updated the base code to also use Java 11, so there is also a PR in place to update the Docker files to reflect that aswell. If the three dot thing isnt the problem, I would suggest switching to the update_docker_java_11 branch, adding the couple of lines to docker-entrypoint.sh from the config file fix PR (https://github.com/GIScience/openrouteservice/pull/772#issuecomment-662411402) and then giving it a go again to see if it is some mismatch that has come up from Java versions.

gwfami2 · August 4, 2020, 4:39pm

Hi,
I’m not certain where the …/ came from, it isn’t showing up in my file, triple checked and removed then remade the yml file from a copy provided by github.

I updated to the latest version of ors using the docker command. I also downloaded and copied the Java 11 dockerfile from Docker.

As long as I don’t use the --build command, it will work, but brings in the Heidelberg test data set into the graphs folder. Once I change yml file to this.

version: ‘2.4’
services:
ors-app:
container_name: ors-app
ports:
- 8080:8080
- 9001:9001
image: openrouteservice/openrouteservice:latest
build:
context: …/
args:
APP_CONFIG: ./openrouteservice/src/main/resources/app.config.sample
OSM_FILE: ./openrouteservice/src/main/files/kansas-latest.osm.pbf
# OSM_FILE: ./openrouteservice/src/main/files/heidelberg.osm.gz
volumes:
  - ./graphs:/ors-core/data/graphs
  - ./elevation_cache:/ors-core/data/elevation_cache
  - ./logs/ors:/var/log/ors
  - ./logs/tomcat:/usr/local/tomcat/logs
  - ./conf:/ors-conf
  #- ./your_osm.pbf:/ors-core/data/osm_file.pbf
environment:
  - BUILD_GRAPHS=TRUE  # Forces the container to rebuild the graphs, e.g. when PBF is changed
  - "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms1g -Xmx2g"
  - "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"

and the dockerfile to this

FROM openjdk:11-jdk

ENV MAVEN_OPTS=“-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true”
ENV MAVEN_CLI_OPTS=“–batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true”

ARG APP_CONFIG=./openrouteservice/src/main/resources/app.config.sample
ARG OSM_FILE=/Users/xxx/Desktop/openrouteservice/osm_data/kansas-latest.osm.pbf

WORKDIR /ors-core

COPY openrouteservice /ors-core/openrouteservice
COPY $OSM_FILE /ors-core/data/osm_file.pbf
COPY $APP_CONFIG /ors-core/openrouteservice/src/main/resources/app.config.sample

#Install tomcat
RUN wget -q https://archive.apache.org/dist/tomcat/tomcat-8/v8.5.39/bin/apache-tomcat-8.5.39.tar.gz -O /tmp/tomcat.tar.gz &&
cd /tmp &&
tar xvfz tomcat.tar.gz &&
mkdir /usr/local/tomcat /ors-conf &&
cp -R /tmp/apache-tomcat-8.5.39/* /usr/local/tomcat/ &&
# Install dependencies and locales
apt-get update -qq && apt-get install -qq -y locales nano maven moreutils jq &&
locale-gen en_US.UTF-8 &&
# Rename to app.config
cp /ors-core/openrouteservice/src/main/resources/app.config.sample /ors-core/openrouteservice/src/main/resources/app.config &&
# Replace paths in app.config to match docker setup
jq ‘.ors.services.routing.sources[0] = “data/osm_file.pbf”’ /ors-core/openrouteservice/src/main/resources/app.config |sponge /ors-core/openrouteservice/src/main/resources/app.config &&
jq ‘.ors.services.routing.profiles.default_params.elevation_cache_path = “data/elevation_cache”’ /ors-core/openrouteservice/src/main/resources/app.config |sponge /ors-core/openrouteservice/src/main/resources/app.config &&
jq ‘.ors.services.routing.profiles.default_params.graphs_root_path = “data/graphs”’ /ors-core/openrouteservice/src/main/resources/app.config |sponge /ors-core/openrouteservice/src/main/resources/app.config &&
# init_threads = 1, > 1 been reported some issues
jq ‘.ors.services.routing.init_threads = 1’ /ors-core/openrouteservice/src/main/resources/app.config |sponge /ors-core/openrouteservice/src/main/resources/app.config &&
# Delete all profiles but car
jq ‘del(.ors.services.routing.profiles.active[1,2,3,4,5,6,7,8])’ /ors-core/openrouteservice/src/main/resources/app.config |sponge /ors-core/openrouteservice/src/main/resources/app.config

COPY ./docker-entrypoint.sh /docker-entrypoint.sh

#Start the container
EXPOSE 8080
ENTRYPOINT [“/bin/bash”, “/docker-entrypoint.sh”]

the same thing happens, it starts looking through all the files on the desktop.

Here’s the verbose from the docker-compose, don’t know if it will help much.

xxx-iMac:openrouteservice xxx$ docker-compose --verbose up -d --build
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.utils.config.find_config_file: Trying paths: [‘/Users/xxx/.docker/config.json’, ‘/Users/xxx/.dockercfg’]
docker.utils.config.find_config_file: Found file at path: /Users/xxx/.docker/config.json
docker.utils.config.find_config_file: Trying paths: [‘/Users/xxx/.docker/config.json’, ‘/Users/xxx/.dockercfg’]
docker.utils.config.find_config_file: Found file at path: /Users/xxx/.docker/config.json
docker.auth.load_config: Found ‘credsStore’ section
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/version HTTP/1.1” 200 861
compose.cli.docker_client.get_client: docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost
compose.cli.docker_client.get_client: Docker version: Platform={‘Name’: ‘Docker Engine - Community’}, Components=[{‘Name’: ‘Engine’, ‘Version’: ‘19.03.12’, ‘Details’: {‘ApiVersion’: ‘1.40’, ‘Arch’: ‘amd64’, ‘BuildTime’: ‘2020-06-22T15:49:27.000000000+00:00’, ‘Experimental’: ‘false’, ‘GitCommit’: ‘48a66213fe’, ‘GoVersion’: ‘go1.13.10’, ‘KernelVersion’: ‘4.19.76-linuxkit’, ‘MinAPIVersion’: ‘1.12’, ‘Os’: ‘linux’}}, {‘Name’: ‘containerd’, ‘Version’: ‘v1.2.13’, ‘Details’: {‘GitCommit’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’}}, {‘Name’: ‘runc’, ‘Version’: ‘1.0.0-rc10’, ‘Details’: {‘GitCommit’: ‘dc9208a3303feef5b3839f4323d9beb36df0a9dd’}}, {‘Name’: ‘docker-init’, ‘Version’: ‘0.18.0’, ‘Details’: {‘GitCommit’: ‘fec3683’}}], Version=19.03.12, ApiVersion=1.40, MinAPIVersion=1.12, GitCommit=48a66213fe, GoVersion=go1.13.10, Os=linux, Arch=amd64, KernelVersion=4.19.76-linuxkit, BuildTime=2020-06-22T15:49:27.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network ← (‘openrouteservice_default’)
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/networks/openrouteservice_default HTTP/1.1” 200 571
compose.cli.verbose_proxy.proxy_callable: docker inspect_network → {‘Attachable’: True,
‘ConfigFrom’: {‘Network’: ‘’},
‘ConfigOnly’: False,
‘Containers’: {},
‘Created’: ‘2020-08-04T03:45:18.841989359Z’,
‘Driver’: ‘bridge’,
‘EnableIPv6’: False,
‘IPAM’: {‘Config’: [{‘Gateway’: ‘172.20.0.1’, ‘Subnet’: ‘172.20.0.0/16’}],
‘Driver’: ‘default’,
‘Options’: None},
…
compose.cli.verbose_proxy.proxy_callable: docker info ← ()
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/info HTTP/1.1” 200 None
compose.cli.verbose_proxy.proxy_callable: docker info → {‘Architecture’: ‘x86_64’,
‘BridgeNfIp6tables’: True,
‘BridgeNfIptables’: True,
‘CPUSet’: True,
‘CPUShares’: True,
‘CgroupDriver’: ‘cgroupfs’,
‘ClusterAdvertise’: ‘’,
‘ClusterStore’: ‘’,
‘ContainerdCommit’: {‘Expected’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’,
‘ID’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’},
…
compose.cli.verbose_proxy.proxy_callable: docker inspect_network ← (‘openrouteservice_default’)
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/networks/openrouteservice_default HTTP/1.1” 200 571
compose.cli.verbose_proxy.proxy_callable: docker inspect_network → {‘Attachable’: True,
‘ConfigFrom’: {‘Network’: ‘’},
‘ConfigOnly’: False,
‘Containers’: {},
‘Created’: ‘2020-08-04T03:45:18.841989359Z’,
‘Driver’: ‘bridge’,
‘EnableIPv6’: False,
‘IPAM’: {‘Config’: [{‘Gateway’: ‘172.20.0.1’, ‘Subnet’: ‘172.20.0.0/16’}],
‘Driver’: ‘default’,
‘Options’: None},
…
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=False, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=False, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.service=ors-app’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.service%3Dors-app%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.service=ors-app’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.service%3Dors-app%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.service.build: Building ors-app
compose.cli.verbose_proxy.proxy_callable: docker build ← (path=‘/Users/xxx/Desktop’, tag=‘openrouteservice/openrouteservice:latest’, rm=True, forcerm=False, pull=False, nocache=False, dockerfile=None, cache_from=None, labels=None, buildargs={‘APP_CONFIG’: ‘./openrouteservice/src/main/resources/app.config.sample’, ‘OSM_FILE’: ‘./openrouteservice/src/main/files/heidelberg.osm.gz’}, network_mode=None, target=None, shmsize=None, extra_hosts=None, container_limits={‘memory’: None}, gzip=False, isolation=None, platform=None)
Terminated: 15
xxx-iMac:openrouteservice xxx$ docker-compose --verbose up -d --build
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.utils.config.find_config_file: Trying paths: [‘/Users/xxx/.docker/config.json’, ‘/Users/xxx/.dockercfg’]
docker.utils.config.find_config_file: Found file at path: /Users/xxx/.docker/config.json
docker.utils.config.find_config_file: Trying paths: [‘/Users/xxx/.docker/config.json’, ‘/Users/xxx/.dockercfg’]
docker.utils.config.find_config_file: Found file at path: /Users/xxx/.docker/config.json
docker.auth.load_config: Found ‘credsStore’ section
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/version HTTP/1.1” 200 861
compose.cli.docker_client.get_client: docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost
compose.cli.docker_client.get_client: Docker version: Platform={‘Name’: ‘Docker Engine - Community’}, Components=[{‘Name’: ‘Engine’, ‘Version’: ‘19.03.12’, ‘Details’: {‘ApiVersion’: ‘1.40’, ‘Arch’: ‘amd64’, ‘BuildTime’: ‘2020-06-22T15:49:27.000000000+00:00’, ‘Experimental’: ‘false’, ‘GitCommit’: ‘48a66213fe’, ‘GoVersion’: ‘go1.13.10’, ‘KernelVersion’: ‘4.19.76-linuxkit’, ‘MinAPIVersion’: ‘1.12’, ‘Os’: ‘linux’}}, {‘Name’: ‘containerd’, ‘Version’: ‘v1.2.13’, ‘Details’: {‘GitCommit’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’}}, {‘Name’: ‘runc’, ‘Version’: ‘1.0.0-rc10’, ‘Details’: {‘GitCommit’: ‘dc9208a3303feef5b3839f4323d9beb36df0a9dd’}}, {‘Name’: ‘docker-init’, ‘Version’: ‘0.18.0’, ‘Details’: {‘GitCommit’: ‘fec3683’}}], Version=19.03.12, ApiVersion=1.40, MinAPIVersion=1.12, GitCommit=48a66213fe, GoVersion=go1.13.10, Os=linux, Arch=amd64, KernelVersion=4.19.76-linuxkit, BuildTime=2020-06-22T15:49:27.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network ← (‘openrouteservice_default’)
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/networks/openrouteservice_default HTTP/1.1” 200 571
compose.cli.verbose_proxy.proxy_callable: docker inspect_network → {‘Attachable’: True,
‘ConfigFrom’: {‘Network’: ‘’},
‘ConfigOnly’: False,
‘Containers’: {},
‘Created’: ‘2020-08-04T03:45:18.841989359Z’,
‘Driver’: ‘bridge’,
‘EnableIPv6’: False,
‘IPAM’: {‘Config’: [{‘Gateway’: ‘172.20.0.1’, ‘Subnet’: ‘172.20.0.0/16’}],
‘Driver’: ‘default’,
‘Options’: None},
…
compose.cli.verbose_proxy.proxy_callable: docker info ← ()
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/info HTTP/1.1” 200 None
compose.cli.verbose_proxy.proxy_callable: docker info → {‘Architecture’: ‘x86_64’,
‘BridgeNfIp6tables’: True,
‘BridgeNfIptables’: True,
‘CPUSet’: True,
‘CPUShares’: True,
‘CgroupDriver’: ‘cgroupfs’,
‘ClusterAdvertise’: ‘’,
‘ClusterStore’: ‘’,
‘ContainerdCommit’: {‘Expected’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’,
‘ID’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’},
…
compose.cli.verbose_proxy.proxy_callable: docker inspect_network ← (‘openrouteservice_default’)
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/networks/openrouteservice_default HTTP/1.1” 200 571
compose.cli.verbose_proxy.proxy_callable: docker inspect_network → {‘Attachable’: True,
‘ConfigFrom’: {‘Network’: ‘’},
‘ConfigOnly’: False,
‘Containers’: {},
‘Created’: ‘2020-08-04T03:45:18.841989359Z’,
‘Driver’: ‘bridge’,
‘EnableIPv6’: False,
‘IPAM’: {‘Config’: [{‘Gateway’: ‘172.20.0.1’, ‘Subnet’: ‘172.20.0.0/16’}],
‘Driver’: ‘default’,
‘Options’: None},
…
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=False, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=False, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.service=ors-app’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.service%3Dors-app%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.cli.verbose_proxy.proxy_callable: docker containers ← (all=True, filters={‘label’: [‘com.docker.compose.project=openrouteservice’, ‘com.docker.compose.service=ors-app’, ‘com.docker.compose.oneoff=False’]})
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dopenrouteservice%22%2C+%22com.docker.compose.service%3Dors-app%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1” 200 3
compose.cli.verbose_proxy.proxy_callable: docker containers → (list with 0 items)
compose.service.build: Building ors-app
compose.cli.verbose_proxy.proxy_callable: docker build ← (path=‘/Users/xxx/Desktop’, tag=‘openrouteservice/openrouteservice:latest’, rm=True, forcerm=False, pull=False, nocache=False, dockerfile=None, cache_from=None, labels=None, buildargs={‘APP_CONFIG’: ‘./openrouteservice/src/main/resources/app.config.sample’, ‘OSM_FILE’: ‘./openrouteservice/src/main/files/kansas-latest.osm.pbf’}, network_mode=None, target=None, shmsize=None, extra_hosts=None, container_limits={‘memory’: None}, gzip=False, isolation=None, platform=None)

At this point, I’m not sure how to proceed. I need this to finish up my thesis, as the online daily limits are to limiting for me to use.

Thanks

adam · August 5, 2020, 7:17am

I really don’t know why it would be scanning your whole system for files as to my understanding Docker should only ever look inside the folders defined by the context when using docker-compose… but it is definitely strange that .../ is showing up for context as that definitely is not correct.

One more thing to try - navigate to the folder where the docker-compose file is and run docker-compose build --no-cache which should run the build from scratch without using any cached images. The folder structure should be:

- openrouteservice
  - docker
    - docker-compose.yml
  - openrouteservice
    - src
    - ...
  - ...
  - Dockerfile
  - docker-entrypoint.sh

as the docker-compose.yml file should tell docker-compose to look in the folder above the one the docker-compose.yml file is located in.

You can also try moving the docker-compose file up one folder and then change the context to be ./, or it might even be the case that if you do that you don’t need the context part - I am not sure off the top of my head though.

If that doesn’t help, you can try running it without docker compose using just the Dockerfile and docker commands. For that you would need to pass in the arguments in the command line (APP_CONFIG and OSM_FILE) and also do the volumes and JAVA_OPTS environment settings.

If nothing seems to work, you can drop us an email at enquiry@openrouteservice.heigit.org and we can discuss further about doing a temporary rate limit increase on the main openrouteservice API for you. It would definitly be interesting though to find out why it is scanning outside of where it is supposed to…

gwfami2 · August 5, 2020, 4:36pm

A couple of things.
" as the docker-compose.yml file should tell docker-compose to look in the folder above the one the docker-compose.yml file is located in."

If it’s looking at the folder above the one it’s in, that’s the desktop directory. I made another directory in the folder and put the docker-compose.yml file in it, things went quite a bit differently this time, and I actually got this to happen.

Step 1/13 : FROM openjdk:11-jdk
11-jdk: Pulling from library/openjdk
d6ff36c9ec48: Pull complete
c958d65b3090: Pull complete
edaf0a6b092f: Pull complete
80931cf68816: Pull complete
bf04b6bbed0c: Pull complete
8bf847804f9e: Pull complete
fa776a6e7e37: Pull complete
Digest: sha256:f1435043dfb16f35adc9a24b04a9b92548f6ac8accf1702fbb8f72b919b34a0f
Status: Downloaded newer image for openjdk:11-jdk
—> a16650ebd079
Step 2/13 : ENV MAVEN_OPTS=“-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true”
—> Running in 3f62a693e709
Removing intermediate container 3f62a693e709
—> b901abde1267
Step 3/13 : ENV MAVEN_CLI_OPTS=“–batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true”
—> Running in 794efb520624
Removing intermediate container 794efb520624
—> 19d0e71423c6
Step 4/13 : ARG APP_CONFIG=./openrouteservice/src/main/resources/app.config.sample
—> Running in 961ae41dc1a9
Removing intermediate container 961ae41dc1a9
—> 460a098cb807
Step 5/13 : ARG OSM_FILE=/Users/rhondawinkler/Desktop/openrouteservice/osm_data/kansas-latest.osm.pbf
—> Running in 73c7dd073692
Removing intermediate container 73c7dd073692
—> aaf7cc3e4b3c
Step 6/13 : WORKDIR /ors-core
—> Running in bb609d30f95b
Removing intermediate container bb609d30f95b
—> 20f2c6d62e64
Step 7/13 : COPY openrouteservice /ors-core/openrouteservice
ERROR: compose.cli.main.main: Service ‘ors-app’ failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder467399549/openrouteservice: no such file or directory

So it’s making progress. I checked /var/lib/docker/tmp and the docker-builder467399549 directory isn’t there, but there is a docker-builder269833675 directory, created a number of days ago, but that directory is empty.

drwxr-xr-x 2 root wheel 64 Aug 1 13:01 docker-builder269833675

I’m going to try a couple of other things and see what happens.

gwfami2 · August 5, 2020, 5:52pm

Ok, so I’m making it to step 7/13 now. Looked online for the error that it showed and the closest that I could find was ERROR: Service ‚ors-build‘ failed to build: COPY failed … – YourHelpCenter

Tried there fix and it didn’t make a difference.

Any idea why the tmp file isn’t being created? Here’s the tree.

Here is the output from when I use docker compose.

xxx-iMac:ymls xxx$ docker-compose --verbose build --no-cache
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.utils.config.find_config_file: Trying paths: [‘/Users/xxx/.docker/config.json’, ‘/Users/xxx/.dockercfg’]
docker.utils.config.find_config_file: Found file at path: /Users/xxx/.docker/config.json
docker.utils.config.find_config_file: Trying paths: [‘/Users/xxx/.docker/config.json’, ‘/Users/xxx/.dockercfg’]
docker.utils.config.find_config_file: Found file at path: /Users/xxx/.docker/config.json
docker.auth.load_config: Found ‘credsStore’ section
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/version HTTP/1.1” 200 861
compose.cli.docker_client.get_client: docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost
compose.cli.docker_client.get_client: Docker version: Platform={‘Name’: ‘Docker Engine - Community’}, Components=[{‘Name’: ‘Engine’, ‘Version’: ‘19.03.12’, ‘Details’: {‘ApiVersion’: ‘1.40’, ‘Arch’: ‘amd64’, ‘BuildTime’: ‘2020-06-22T15:49:27.000000000+00:00’, ‘Experimental’: ‘false’, ‘GitCommit’: ‘48a66213fe’, ‘GoVersion’: ‘go1.13.10’, ‘KernelVersion’: ‘4.19.76-linuxkit’, ‘MinAPIVersion’: ‘1.12’, ‘Os’: ‘linux’}}, {‘Name’: ‘containerd’, ‘Version’: ‘v1.2.13’, ‘Details’: {‘GitCommit’: ‘7ad184331fa3e55e52b890ea95e65ba581ae3429’}}, {‘Name’: ‘runc’, ‘Version’: ‘1.0.0-rc10’, ‘Details’: {‘GitCommit’: ‘dc9208a3303feef5b3839f4323d9beb36df0a9dd’}}, {‘Name’: ‘docker-init’, ‘Version’: ‘0.18.0’, ‘Details’: {‘GitCommit’: ‘fec3683’}}], Version=19.03.12, ApiVersion=1.40, MinAPIVersion=1.12, GitCommit=48a66213fe, GoVersion=go1.13.10, Os=linux, Arch=amd64, KernelVersion=4.19.76-linuxkit, BuildTime=2020-06-22T15:49:27.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network ← (‘ymls_default’)
urllib3.connectionpool._make_request: http://localhost:None “GET /v1.35/networks/ymls_default HTTP/1.1” 404 45
compose.service.build: Building ors-app
compose.cli.verbose_proxy.proxy_callable: docker build ← (path=‘/Users/xxx/Desktop/openrouteservice’, tag=‘openrouteservice/openrouteservice:latest’, rm=True, forcerm=False, pull=False, nocache=True, dockerfile=None, cache_from=None, labels=None, buildargs={‘APP_CONFIG’: ‘./openrouteservice/src/main/resources/app.config.sample’, ‘OSM_FILE’: ‘./openrouteservice/src/main/files/kansas-latest.osm.pbf’}, network_mode=None, target=None, shmsize=None, extra_hosts=None, container_limits={‘memory’: None}, gzip=False, isolation=None, platform=None)
docker.api.build._set_auth_headers: Looking for auth config
docker.api.build._set_auth_headers: Sending auth config ()
urllib3.connectionpool._make_request: http://localhost:None “POST /v1.35/build?t=openrouteservice%2Fopenrouteservice%3Alatest&q=False&nocache=True&rm=True&forcerm=False&pull=False&buildargs=%7B%22APP_CONFIG%22%3A+%22.%2Fopenrouteservice%2Fsrc%2Fmain%2Fresources%2Fapp.config.sample%22%2C+%22OSM_FILE%22%3A+%22.%2Fopenrouteservice%2Fsrc%2Fmain%2Ffiles%2Fkansas-latest.osm.pbf%22%7D HTTP/1.1” 200 None
compose.cli.verbose_proxy.proxy_callable: docker build → <generator object APIClient._stream_helper at 0x10ab4a750>
Step 1/13 : FROM openjdk:11-jdk
—> a16650ebd079
Step 2/13 : ENV MAVEN_OPTS=“-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true”
—> Running in d760d3802bfa
Removing intermediate container d760d3802bfa
—> 148abb4e11da
Step 3/13 : ENV MAVEN_CLI_OPTS=“–batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true”
—> Running in bf52a607c635
Removing intermediate container bf52a607c635
—> 58f4fdaa3109
Step 4/13 : ARG APP_CONFIG=./openrouteservice/src/main/resources/app.config.sample
—> Running in 681034c66b36
Removing intermediate container 681034c66b36
—> ed900ab3dda8
Step 5/13 : ARG OSM_FILE=/Users/xxx/Desktop/openrouteservice/osm_data/kansas-latest.osm.pbf
—> Running in 5646e632b1d3
Removing intermediate container 5646e632b1d3
—> 5eef2492d9f9
Step 6/13 : WORKDIR /ors-core
—> Running in f0d3aebe3246
Removing intermediate container f0d3aebe3246
—> aa86c08ddffa
Step 7/13 : COPY openrouteservice /ors-core/openrouteservice
ERROR: compose.cli.main.main: Service ‘ors-app’ failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder168252557/openrouteservice: no such file or directory

gwfami2 · August 5, 2020, 7:23pm

Update:

I got tired of this and uninstalled docker and openrouteservice then reinstalled everything fresh.

I think I figured out why it was reading the desktop directory. I don’t believe that I had the directory structure setup correctly. Here’s what I currently have.

Now it seems that tomcat doesn’t like the “|” character in the request.

But if I try to remove it, I get this.

Any ideas?

adam · August 6, 2020, 6:18am

Well that definitely looks a lot better in that you get to openrouteservice error responses now

So with the error message - it is much better to use the v2 API as that uses POST methods with the routing parameters in the body, which makes things a lot easier both for the user and for the system. The reason you get the bad request is that special characters in query strings must be encoded (things like pipes, commas and braces) else Tomcat does not handle the requests for security reasons.

There is a page on our GitHub wiki that has examples of what queries should look like in v2, and you can also look at the interactive docs to see what other parameters can be used

gwfami2 · August 6, 2020, 6:33pm

Thanks for all the help.
I’ve been able to use the v2 example for directions to get a good return for the test data.

http://localhost:8080/ors/v2/directions/driving-car?api_key=your-api-key&start=8.681495,49.41461&end=8.687872,49.420318

and matrices too.

curl -X POST
‘http://localhost:8080/ors/v2/matrix/driving-car’
-H ‘Content-Type: application/json; charset=utf-8’
-H ‘Accept: application/json, application/geo+json, application/gpx+xml, img/png; charset=utf-8’
-H ‘Authorization: xxx’
-d ‘{“locations”:[[8.681495,49.41461],[8.687872,49.420318],[8.686441,49.419412],[8.685276,49.422387]]}’

Making progress, but ever so slowly.

Now to see if I can get a map change to work.

gwfami2 · August 6, 2020, 10:16pm

Alright! Was able to download a map and get everything to work.

Thanks for all your help.

gwfami2