Docker - SocketTimeoutException

rki · June 24, 2020, 7:22am

Hello, I have a problem with the current docker installation.
24-Jun-2020 07:09:40.487 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 6607 ms 24 Jun 07:10:17 ERROR [routing.RoutingProfileManager] - java.util.concurrent.ExecutionException: java.lang.RuntimeException: Couldn’t process file data/osm_file.pbf, error: java.net.SocketTimeoutException: Read timed out 24 Jun 07:10:17 ERROR [routing.RoutingProfileManager] - Failed to initialize RoutingProfileManager instance. java.util.concurrent.ExecutionException: java.lang.RuntimeException: Couldn’t process file data/osm_file.pbf, error: java.net.SocketTimeoutException: Read timed out at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_252] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_252] at org.heigit.ors.routing.RoutingProfileManager.initialize(RoutingProfileManager.java:162) ~[classes/:6.1.1] at org.heigit.ors.routing.RoutingProfileManager.getInstance(RoutingProfileManager.java:57) ~[classes/:6.1.1] at org.heigit.ors.servlet.listeners.ORSInitContextListener.lambda$contextInitialized$0(ORSInitContextListener.java:40) ~[classes/:6.1.1] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252] Caused by: java.lang.RuntimeException: Couldn’t process file data/osm_file.pbf, error: java.net.SocketTimeoutException: Read timed out at com.graphhopper.reader.osm.OSMReader.writeOsm2Graph(OSMReader.java:324) ~[graphhopper-reader-osm-v0.13.5.jar:?] at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:176) ~[graphhopper-reader-osm-v0.13.5.jar:?] at com.graphhopper.GraphHopper.importData(GraphHopper.java:735) ~[graphhopper-core-v0.13.5.jar:?] at com.graphhopper.GraphHopper.readData(GraphHopper.java:714) ~[graphhopper-core-v0.13.5.jar:?] at com.graphhopper.GraphHopper.process(GraphHopper.java:701) ~[graphhopper-core-v0.13.5.jar:?] at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:680) ~[graphhopper-core-v0.13.5.jar:?] at org.heigit.ors.routing.graphhopper.extensions.ORSGraphHopper.importOrLoad(ORSGraphHopper.java:131) ~[classes/:6.1.1] at org.heigit.ors.routing.RoutingProfile.initGraphHopper(RoutingProfile.java:181) ~[classes/:6.1.1] at org.heigit.ors.routing.RoutingProfile.(RoutingProfile.java:127) ~[classes/:6.1.1] at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:35) ~[classes/:6.1.1] at org.heigit.ors.routing.RoutingProfileLoader.call(RoutingProfileLoader.java:21) ~[classes/:6.1.1] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_252] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_252] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_252] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_252] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_252] … 1 more Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out at com.graphhopper.reader.dem.AbstractTiffElevationProvider.downloadFile(AbstractTiffElevationProvider.java:159) ~[graphhopper-core-v0.13.5.jar:?] at com.graphhopper.reader.dem.AbstractTiffElevationProvider.getEle(AbstractTiffElevationProvider.java:121) ~[graphhopper-core-v0.13.5.jar:?] at com.graphhopper.reader.dem.MultiSourceElevationProvider.getEle(MultiSourceElevationProvider.java:52) ~[graphhopper-core-v0.13.5.jar:?]

adam · June 24, 2020, 7:38am

The bottom part of the error is showing that it cannot download the elevation file from the source. The first thing to check is that from inside the container there is internet access, and if there is then it is likely that the elevation provider was down for a short while. In which case restarting the docker container should then hopefully work. If there is not internet connection from within the container, then you will need to configure how docker runs based on your system/network setup

rki · June 25, 2020, 8:26am

The Docker Container has internet and dns. Still have the SocketTimeoutException.


root@1bef8faec476:/ors-core# ping google.com
PING google.com (216.58.212.142) 56(84) bytes of data.
64 bytes from ams15s21-in-f142.1e100.net (216.58.212.142): icmp_seq=1 ttl=115 time=16.0 ms
64 bytes from ams15s21-in-f142.1e100.net (216.58.212.142): icmp_seq=2 ttl=115 time=15.9 ms
64 bytes from ams15s21-in-f142.1e100.net (216.58.212.142): icmp_seq=3 ttl=115 time=15.7 ms
64 bytes from ams15s21-in-f142.1e100.net (216.58.212.142): icmp_seq=4 ttl=115 time=15.7 ms
64 bytes from ams15s21-in-f142.1e100.net (216.58.212.142): icmp_seq=5 ttl=115 time=15.7 ms
^C
--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 7ms
rtt min/avg/max/mdev = 15.690/15.813/16.047/0.136 ms
root@1bef8faec476:/ors-core# ping rki-logistics.com
PING rki-logistics.com (217.160.0.119) 56(84) bytes of data.
64 bytes from 217-160-0-119.elastic-ssl.ui-r.com (217.160.0.119): icmp_seq=1 ttl=52 time=10.3 ms
64 bytes from 217-160-0-119.elastic-ssl.ui-r.com (217.160.0.119): icmp_seq=2 ttl=52 time=10.3 ms
64 bytes from 217-160-0-119.elastic-ssl.ui-r.com (217.160.0.119): icmp_seq=3 ttl=52 time=10.2 ms
64 bytes from 217-160-0-119.elastic-ssl.ui-r.com (217.160.0.119): icmp_seq=4 ttl=52 time=10.2 ms
64 bytes from 217-160-0-119.elastic-ssl.ui-r.com (217.160.0.119): icmp_seq=5 ttl=52 time=10.2 ms
^C
--- rki-logistics.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 10ms
rtt min/avg/max/mdev = 10.215/10.259/10.326/0.100 ms
root@1bef8faec476:/ors-core# apt-get update
Hit:1 http://deb.debian.org/debian buster InRelease
Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [204 kB]
Fetched 321 kB in 0s (720 kB/s)   
Reading package lists... Done
root@1bef8faec476:/ors-core#

adam · June 26, 2020, 6:32am

Hmm, strange…

Do you know if your connection is behind a proxy or virus checker? We had an issue once with the elevation tiles where the host organisation for our servers checked all incoming compressed archive files and that was causing some problems as their checker basically didn’t inform the client that the check was happening, so as far as it was concerned it went a minute without getting any response.

If you are running it through your local machine, try visiting srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/tiff/srtm_34_10.zip, and if you see a virus scanning thing (ours was Sophos) then that would be the culprit. You can also try wget inside the container with the same url, and if it shows the HTTP request sent, awating response... message for a while and then very quickly finishes the download, that’s also an indicator of something on the parent network blocking the direct download.

Unfortunately if this is the problem, the only real “solution” is to manually download the elevation files and then run the build process, as the core library we use (GraphHopper) doesn’t allow the user to change the timeout values.

rki · June 26, 2020, 9:45am

Yes, this is the case. Sophos check the files and that takes time. Download the srtm-files via wget and put them into the volume directory elevation-cache is the solution. Now it works. Thank you!

adam · June 26, 2020, 1:36pm

Great, glad it worked

rki · June 26, 2020, 1:48pm

Last question: The OSM data of europe contains finnland. But there are no srtm files to download. Will the graphs process when srtm not there?

rki · June 29, 2020, 9:21am

If I use graphopper direkt from their dockerfile, the files will be downloaded without a problem!
I dont understand why there are problems with the openroute Docker!!!

nils · June 29, 2020, 9:46am

We don’t use GH master/current release, but an older version. No idea what they’re doing differently by now, maybe there’s some switch to the download functionality which is not implemented in our GH version…

About no coverage for SRTM: it falls back to GMTED2010:

github.com

graphhopper/graphhopper/blob/master/docs/core/elevation.md

# Elevation

Per default elevation is disabled. But you can easily enable it e.g. via
`graph.elevation.provider: cgiar`. Or use other possibilities `srtm`, `gmted`
or `multi` (combined cgiar and gmted).

Then GraphHopper will automatically download the necessary data for the area and include elevation 
for all vehicles - making also the distances a bit more precise. 

The default cache directory `/tmp/<provider name>` will be used. For large areas it is highly recommended to 
use a SSD disc, thus you need to specify the cache directory:
`graph.elevation.cache_dir: /myssd/ele_cache/`

## What to download and where to store it? 

All should work automatically but you can tune certain settings like the location where the files are 
downloaded and e.g. if the servers are not reachable, then you set:
`graph.elevation.base_url`

For CGIAR there are two URLs you can use: `http://droppr.org/srtm/v4.1/6_5x5_TIFs` and

This file has been truncated. show original

So, you’d have to download that separately, not sure from where though. If you go into the source code you’ll find it.

rki · June 29, 2020, 10:43am

I let grashopper build the graphs for europe-latest and the downloaded srtm files from grashopper and put it in the elevation-cache directory. But it do not work.