hey how can i add or merge two separate maps like germany and italy osm.pfb files. ?
Hey Daniel,
you could use Osmium Tool - osmcode
I use ubuntu and do the the following:
- after installation of osmium
- Download both pbf files into the same folder
- execute in command line:
osmium merge germany.pbf italy.pbf -o [YOURNEWNAME].pbf
Hope that helps
@DejavuDaniel Here is a shell script to automate this monthly, I use a docker stack with one machine running osmium and a second machine running ORS sharing a volume.
This script includes downloading files, md5 check, sorting and merging with osmium.
It might be a bit overkill because I get most of western europe in mine :
#!/usr/bin/env bash
# Pointer variables sur les bons chemins
osm_file_path=/custom_files/
log_path=/var/log/carto/
# Source des fichiers de cartographie
osm_gfb=https://download.geofabrik.de/europe/
osm_list=(
"france"
"andorra"
"belgium"
"germany/baden-wuerttemberg" "germany/nordrhein-westfalen" "germany/rheinland-pfalz" "germany/saarland"
"italy/nord-ovest"
"luxembourg"
"netherlands"
"spain/aragon" "spain/cataluna" "spain/navarra" "spain/pais-vasco"
"switzerland"
)
if [ "$(date '+\%u')" = "5" ]; then
#Supprimer les fichiers pbf
rm -f ${osm_file_path}*.pbf
rm -f ${osm_file_path}**/*.pbf
# Téléchargement des fichiers pbf
echo "Downloading and processing pbf files"
for osm_file in "${osm_list[@]}"
do
filename="${osm_file_path}${osm_file}"
echo "Processing ${osm_file}"
echo "Downloading from ${osm_gfb}${osm_file}-latest.osm.pbf..." >> ${log_path}${osm_file}.log 2>&1
while true; do
# Téléchargement du fichier
wget --output-document="${filename}.osm.pbf" "${osm_gfb}${osm_file}-latest.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
computed_md5=$(md5sum "${filename}.osm.pbf" | awk '{ print $1 }')
# Téléchargement md5
wget --output-document="${filename}.osm.pbf.md5" "${osm_gfb}${osm_file}-latest.osm.pbf.md5" >> ${log_path}${osm_file}.log 2>&1
expected_md5=$(cat "${filename}.osm.pbf.md5" | awk '{ print $1 }')
# Check md5
if [[ "$computed_md5" == "$expected_md5" ]]; then
echo "File successfully downloaded and verified." >> ${log_path}${osm_file}.log 2>&1
rm "${filename}.osm.pbf.md5"
break
else
echo "MD5 checksum mismatch. Redownloading the file..." >> ${log_path}${osm_file}.log 2>&1
rm "${filename}.osm.pbf.md5" "${filename}.osm.pbf"
fi
done
if [ $? -ne 0 ]; then
echo "Error: Failed to download the file." >> ${log_path}${osm_file}.log 2>&1
exit 1
fi
# Assemblage des fichiers
if [ ! -f "${osm_file_path}osm_file.pbf" ]; then
# Premier fichier : trier puis renommer
echo "sorting file" >> ${log_path}${osm_file}.log 2>&1
osmium sort --strategy="multipass" -v -o "${filename}-sorted.osm.pbf" "${filename}.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
echo "creating file ${osm_file_path}osm_file.pbf" >> ${log_path}${osm_file}.log 2>&1
mv "${filename}-sorted.osm.pbf" "${osm_file_path}osm_file.pbf"
else
# Fichiers subséquents : trier puis merger
echo "sorting file" >> ${log_path}${osm_file}.log 2>&1
osmium sort --strategy="multipass" -v -o "${filename}-sorted.osm.pbf" "${filename}.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
echo "merging file on ${osm_file_path}osm_file.pbf" >> ${log_path}${osm_file}.log 2>&1
osmium merge -v "${osm_file_path}osm_file.pbf" "${filename}-sorted.osm.pbf" -o "${osm_file_path}osm_tmp.pbf" >> ${log_path}${osm_file}.log 2>&1
mv "${osm_file_path}osm_tmp.pbf" "${osm_file_path}osm_file.pbf"
rm -f "${filename}-sorted.osm.pbf"
fi
done
fi
Dear @vchalmel -
this sounds way more sophisticated and I would like to try to implement it myself. Could you elaborate a little more on this.
I am running ORS 7.4. - what are the necessary steps ?
Greetings florian
Do you want some more details on this script specifically or the docker stack ?
I would really like to implement it ot my own server.
I use docker-compose for ors 7.4.
right now my docker-compose.yml looks like this, where I refer to the country_files folder where I manually merge the country files from time to time
version: ‘2.4’
services:
ors-app:
container_name: ors-app-v7
ports:
- “8080:8080”
- “9001:9001”
image: openrouteservice/openrouteservice:v7.2.0
user: “${UID:-0}:${GID:-0}”
build:
context: …/
args:
ORS_CONFIG: ./openrouteservice/src/main/resources/ors-config-sample.json
OSM_FILE: ./openrouteservice/src/main/files/heidelberg.osm.gz
volumes:
- ./graphs:/home/ors/ors-core/data/graphs
- ./elevation_cache:/home/ors/ors-core/data/elevation_cache
- ./logs/ors:/home/ors/ors-core/logs/ors
- ./logs/tomcat:/home/ors/tomcat/logs
- ./conf:/home/ors/ors-conf
- ../../country_files/austriaswitzerland.pbf:/home/ors/ors-core/data/osm_file.pbf
environment:
- BUILD_GRAPHS=False # Forces the container to rebuild the graphs, e.g. when PBF is changed
- "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms3g -Xmx12g"
- "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
Ok, here are the file (its intended to use in a docker swarm so you might have to adjust networking a bit and obviously your volumes if used in a standalone docker server)
Docker compose
version: '3.8'
services:
osmium:
image: <my-docker-repo>:5000/osmium
build:
context: ./osmium/
dockerfile: Dockerfile
deploy:
replicas: 1
resources:
limits:
memory: 64G
reservations:
memory: 32G
restart_policy:
condition: any
delay: 5s
max_attempts: 5
window: 60s
volumes:
- files:/custom_files/
ors-app:
ports:
- 6014:8080
- 9001:9001
image: <my-docker-repo>:5000/custom_ors
build:
context: ./ors/
dockerfile: Dockerfile
deploy:
replicas: 1
placement:
constraints:
- "node.role==manager"
resources:
limits:
memory: 32G
reservations:
memory: 8G
restart_policy:
condition: any
delay: 5s
max_attempts: 5
window: 60s
user: "${ORS_UID:-0}:${ORS_GID:-0}"
volumes:
- graphs:/home/ors/ors-core/data/graphs
- elevation_cache:/home/ors/ors-core/data/elevation_cache
- logs_ors:/home/ors/ors-core/logs/ors
- logs_tomcat:/home/ors/tomcat/logs
- conf:/home/ors/ors-conf
- files:/home/ors/ors-core/data/custom_files
environment:
- "BUILD_GRAPHS=False"
- "JAVA_OPTS=-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms6g -Xmx30g"
- "CATALINA_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
vroom:
image: vroomvrp/vroom-docker:v1.13.0
deploy:
replicas: 1
resources:
limits:
memory: 16G
reservations:
memory: 4G
ports:
- 6013:3000
volumes:
- vroom_conf:/conf
environment:
- VROOM_ROUTER=ors # router to use, osrm, valhalla or ors
depends_on:
- ors-app
links:
- ors-app
volumes:
graphs:
driver: local
driver_opts:
<my opts>
elevation_cache:
driver: local
driver_opts:
<my opts>
logs_ors:
driver: local
driver_opts:
<my opts>
logs_tomcat:
driver: local
driver_opts:
<my opts>
conf:
driver: local
driver_opts:
<my opts>
files:
driver: local
driver_opts:
<my opts>
vroom_conf:
driver: local
driver_opts:
<my opts>
ORS
Dockerfile
FROM openrouteservice/openrouteservice:v7.1.0
USER root
# rechargement mensuel
COPY ["./maj_mensuelle.sh", "/maj_mensuelle.sh"]
RUN chmod +x /maj_mensuelle.sh
RUN touch /mens_cron
RUN echo "#" >> /mens-cron
RUN echo "0 0 1-7 * * /maj_mensuelle.sh" >> /mens-cron
RUN echo "#" >> /mens-cron
RUN chmod 0644 /mens-cron
RUN crontab /mens-cron
# config
RUN cd /home/ors/ors-core/ && mv ./ors-config.json ./ors-config.json.bak
COPY --chown=ors:ors ./ors-config.json /home/ors/ors-core/ors-config.json
# entrypoint personnalisé
COPY --chown=ors:ors ./docker-entrypoint.sh /home/ors/ors-core/docker-entrypoint.sh
RUN chmod +x /home/ors/ors-core/docker-entrypoint.sh
# Start the container
WORKDIR "/home/ors/"
ENTRYPOINT ["/home/ors/ors-core/docker-entrypoint.sh"]
CMD ["/home/ors"]
docker-entrypoint.sh
#!/usr/bin/env bash
echo "Running container as user $(whoami) with id $(id -u)"
if [[ -d /ors-core ]] || [[ -d /ors-conf ]]; then
echo "You're mounting old paths. Remove them and migrate to the new docker setup: https://github.com/GIScience/openrouteservice/blob/master/docker/docker-compose.yml"
echo "Exit setup due to old folders /ors-core or /ors-conf being mounted"
sleep 5
exit 1
fi
ors_base=${1}/
catalina_base=${ors_base}tomcat/
echo "ORS Path: ${ors_base}"
echo "Catalina Path: ${catalina_base}"
ors_data=${ors_base}ors-core/data/
echo "Data Path : ${ors_data}"
ors_graphs=${ors_data}graphs/
echo "Graphs Path : ${ors_graphs}"
tomcat_ors_config=${catalina_base}webapps/ors/WEB-INF/classes/ors-config.json
echo "Tomcat config: ${tomcat_ors_config}"
source_ors_config=${ors_base}ors-core/ors-config.json
echo "Source config: ${source_ors_config}"
public_ors_config_folder=${ors_base}ors-conf/
echo "Public config folder: ${public_ors_config_folder}"
public_ors_config=${public_ors_config_folder}ors-config.json
echo "Public config: ${public_ors_config}"
ors_war_path=${ors_base}ors-core/ors.war
echo "War path: ${ors_war_path}"
log_path=/var/log/carto/
osm_data=${ors_data}custom_files/
if [ -z "${CATALINA_OPTS}" ]; then
export CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
fi
if [ -z "${JAVA_OPTS}" ]; then
export JAVA_OPTS="-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms6g -Xmx30g"
fi
{
echo "CATALINA_BASE=\"${catalina_base}\""
echo "CATALINA_HOME=\"${catalina_base}\""
echo "CATALINA_PID=\"${catalina_base}temp/tomcat.pid\""
echo "CATALINA_OPTS=\"${CATALINA_OPTS}\""
echo "JAVA_OPTS=\"${JAVA_OPTS}\""
} >"${catalina_base}"bin/setenv.sh
echo "Checking pbf file"
cp "${osm_data}osm_file.pbf" "${ors_data}osm_file.pbf"
if [ "$BUILD_GRAPHS" = "True" ]; then
rm -f "${ors_data}osm_file.pbf"
rm -rf ${ors_graphs}*
fi
if [ ! -f "${ors_data}osm_file.pbf" ]; then
cp "${osm_data}osm_file.pbf" "${ors_data}osm_file.pbf"
fi
echo "### openrouteservice configuration ###"
if [ ! -d "${catalina_base}webapps/ors" ]; then
echo "Extract war file to ${catalina_base}webapps/ors"
cp -f "${ors_war_path}" "${catalina_base}"webapps/ors.war
unzip -qq "${catalina_base}"webapps/ors.war -d "${catalina_base}webapps/ors"
fi
if [ ! -f "$public_ors_config" ]; then
echo "No ors-config.json in ors-conf folder. Copying original config from ${source_ors_config}"
mkdir -p "${public_ors_config_folder}"
cp -f "${source_ors_config}" "${public_ors_config}"
fi
echo "Deploy ors with config from ${public_ors_config}"
cp -f "${public_ors_config}" "${tomcat_ors_config}"
# so docker can stop the process gracefully
exec "${catalina_base}"bin/catalina.sh run
maj_mensuelle.sh
#!/usr/bin/env bash
ors_data=/home/ors/ors-core/data/
ors_graphs=${ors_data}graphs/
if [ "$(date '+\%u')" = "6" ]; then
# Suppression du fichier pbf principal
rm -f "${ors_data}osm_file.pbf"
rm -rf ${ors_graphs}*
# Interruption tous process java pour redémarrage
ps ax | grep java | grep -v 'grep' | cut -d '?' -f1 | xargs kill -9
fi
ors-config.json
Nothing fancy, check paths.
{
"ors": {
"info": {
"base_url": "https://openrouteservice.org/",
"support_mail": "support@openrouteservice.org",
"author_tag": "openrouteservice",
"content_licence": "LGPL 3.0"
},
"services": {
"matrix": {
"enabled": true,
"maximum_routes": 25000,
"maximum_routes_flexible": 250,
"maximum_search_radius": 5000,
"maximum_visited_nodes": 100000000,
"allow_resolve_locations": true,
"attribution": "openrouteservice.org, OpenStreetMap contributors"
},
"isochrones": {
"enabled": true,
"maximum_range_distance": [
{
"profiles": "any",
"value": 50000
},
{
"profiles": "driving-car, driving-hgv",
"value": 100000
}
],
"maximum_range_time": [
{
"profiles": "any",
"value": 18000
},
{
"profiles": "driving-car, driving-hgv",
"value": 3600
}
],
"fastisochrones": {
"maximum_range_distance": [
{
"profiles": "any",
"value": 50000
},
{
"profiles": "driving-car, driving-hgv",
"value": 500000
}
],
"maximum_range_time": [
{
"profiles": "any",
"value": 18000
},
{
"profiles": "driving-car, driving-hgv",
"value": 10800
}
],
"profiles": {
"default_params": {
"enabled": false,
"threads": 12,
"weightings": "recommended",
"maxcellnodes": 5000
},
"profile-hgv": {
"enabled": false,
"threads": 12,
"weightings": "recommended, shortest",
"maxcellnodes": 5000
}
}
},
"maximum_intervals": 10,
"maximum_locations": 2,
"allow_compute_area": true
},
"routing": {
"enabled": true,
"mode": "normal",
"routing_description": "This is a routing file from openrouteservice",
"routing_name": "openrouteservice routing",
"sources": [
"/home/ors/ors-core/data/osm_file.pbf"
],
"init_threads": 3,
"attribution": "openrouteservice.org, OpenStreetMap contributors",
"elevation_preprocessed": false,
"profiles": {
"active": [
"car",
"walking"
],
"default_params": {
"encoder_flags_size": 8,
"graphs_root_path": "/home/ors/ors-core/data/graphs",
"elevation_provider": "multi",
"elevation_cache_path": "/home/ors/ors-core/data/elevation_cache",
"elevation_cache_clear": false,
"instructions": true,
"maximum_distance": 100000,
"maximum_distance_dynamic_weights": 100000,
"maximum_distance_avoid_areas": 100000,
"maximum_waypoints": 50,
"maximum_snapping_radius": 400,
"maximum_avoid_polygon_area": 200000000,
"maximum_avoid_polygon_extent": 20000,
"maximum_distance_alternative_routes": 100000,
"maximum_alternative_routes": 3,
"maximum_distance_round_trip_routes": 100000,
"maximum_speed_lower_bound": 80,
"preparation": {
"min_network_size": 200,
"min_one_way_network_size": 200,
"methods": {
"lm": {
"enabled": true,
"threads": 1,
"weightings": "recommended,shortest",
"landmarks": 16
},
"ch": {
"enabled": true,
"threads": 1,
"weightings": "recommended"
}
}
},
"execution": {
"methods": {
"ch": {
"disabling_allowed": true
},
"lm": {
"disabling_allowed": true,
"active_landmarks": 8
}
}
}
},
"profile-car": {
"profiles": "driving-car",
"parameters": {
"encoder_flags_size": 8,
"encoder_options": "turn_costs=true|block_fords=false|use_acceleration=true",
"maximum_distance": 1000000,
"elevation": true,
"maximum_snapping_radius": 350,
"preparation": {
"min_network_size": 200,
"min_one_way_network_size": 200,
"methods": {
"ch": {
"enabled": true,
"threads": 1,
"weightings": "fastest"
},
"lm": {
"enabled": false,
"threads": 1,
"weightings": "fastest,shortest",
"landmarks": 16
},
"core": {
"enabled": true,
"threads": 1,
"weightings": "fastest,shortest",
"landmarks": 64,
"lmsets": "highways;allow_all"
}
}
},
"execution": {
"methods": {
"ch": {
"disabling_allowed": true
},
"lm": {
"disabling_allowed": true,
"active_landmarks": 6
},
"core": {
"disabling_allowed": true,
"active_landmarks": 6
}
}
},
"ext_storages": {
"WayCategory": {},
"HeavyVehicle": {},
"WaySurfaceType": {},
"RoadAccessRestrictions": {
"use_for_warnings": true
}
}
}
},
"profile-hgv": {
"profiles": "driving-hgv",
"parameters": {
"encoder_flags_size": 8,
"encoder_options": "turn_costs=true|block_fords=false|use_acceleration=true",
"maximum_distance": 100000,
"elevation": true,
"preparation": {
"min_network_size": 200,
"min_one_way_network_size": 200,
"methods": {
"ch": {
"enabled": true,
"threads": 1,
"weightings": "recommended"
},
"core": {
"enabled": true,
"threads": 1,
"weightings": "recommended,shortest",
"landmarks": 64,
"lmsets": "highways;allow_all"
}
}
},
"execution": {
"methods": {
"ch": {
"disabling_allowed": true
},
"core": {
"disabling_allowed": true,
"active_landmarks": 6
}
}
},
"ext_storages": {
"WayCategory": {},
"HeavyVehicle": {
"restrictions": true
},
"WaySurfaceType": {}
}
}
},
"profile-bike-regular": {
"profiles": "cycling-regular",
"parameters": {
"encoder_options": "consider_elevation=true|turn_costs=true|block_fords=false",
"elevation": true,
"maximum_distance": 100000,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"HillIndex": {},
"TrailDifficulty": {}
}
}
},
"profile-bike-mountain": {
"profiles": "cycling-mountain",
"parameters": {
"encoder_options": "consider_elevation=true|turn_costs=true|block_fords=false",
"elevation": true,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"HillIndex": {},
"TrailDifficulty": {}
}
}
},
"profile-bike-road": {
"profiles": "cycling-road",
"parameters": {
"encoder_options": "consider_elevation=true|turn_costs=true|block_fords=false",
"elevation": true,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"HillIndex": {},
"TrailDifficulty": {}
}
}
},
"profile-bike-electric": {
"profiles": "cycling-electric",
"parameters": {
"encoder_options": "consider_elevation=true|turn_costs=true|block_fords=false",
"elevation": true,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"HillIndex": {},
"TrailDifficulty": {}
}
}
},
"profile-walking": {
"profiles": "foot-walking",
"parameters": {
"encoder_options": "block_fords=false",
"elevation": true,
"maximum_distance": 10000,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"HillIndex": {},
"TrailDifficulty": {}
},
"preparation": {
"methods": {
"ch": {
"enabled": true,
"threads": 1,
"weightings": "recommended"
}
}
}
}
},
"profile-hiking": {
"profiles": "foot-hiking",
"parameters": {
"encoder_options": "block_fords=false",
"elevation": true,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"HillIndex": {},
"TrailDifficulty": {}
}
}
},
"profile-wheelchair": {
"profiles": "wheelchair",
"parameters": {
"encoder_options": "block_fords=true",
"elevation": true,
"maximum_snapping_radius": 50,
"ext_storages": {
"WayCategory": {},
"WaySurfaceType": {},
"Wheelchair": {
"KerbsOnCrossings": "true"
},
"OsmId": {}
}
}
}
}
}
},
"logging": {
"enabled": true,
"level_file": "DEBUG_LOGGING.json",
"location": "/home/ors/ors-core/logs/ors",
"stdout": true
},
"system_message": [
{
"active": false,
"text": "This message would be sent with every routing bike fastest request",
"condition": {
"request_service": "routing",
"request_profile": "cycling-regular,cycling-mountain,cycling-road,cycling-electric",
"request_preference": "fastest"
}
},
{
"active": false,
"text": "This message would be sent with every request for geojson response",
"condition": {
"api_format": "geojson"
}
},
{
"active": false,
"text": "This message would be sent with every request on API v1 from January 2020 until June 2050",
"condition": {
"api_version": 1,
"time_after": "2020-01-01T00:00:00Z",
"time_before": "2050-06-01T00:00:00Z"
}
},
{
"active": false,
"text": "This message would be sent with every request"
}
]
}
}
Osmium
Dockerfile
FROM ubuntu:20.04
RUN apt-get update --fix-missing && apt-get install -y wget curl osmium-tool cron
RUN mkdir /var/log/carto
RUN mkdir /var/log/carto/germany
RUN mkdir /var/log/carto/spain
RUN mkdir /var/log/carto/italy
# rechargement mensuel
COPY ["./maj_mensuelle.sh","/maj_mensuelle.sh"]
RUN chmod +x /maj_mensuelle.sh
RUN touch /etc/cron.d/mens-cron
RUN echo "#" >> /etc/cron.d/mens-cron
RUN echo "0 0 1-7 * * /maj_mensuelle.sh" >> /etc/cron.d/mens-cron
RUN echo "#" >> /etc/cron.d/mens-cron
RUN chmod 0644 /etc/cron.d/mens-cron
RUN crontab /etc/cron.d/mens-cron
CMD cron && tail -f /dev/null
maj_mensuelle.sh
#!/usr/bin/env bash
# Pointer variables sur les bons chemins
osm_file_path=/custom_files/
log_path=/var/log/carto/
# Source des fichiers de cartographie
osm_gfb=https://download.geofabrik.de/europe/
osm_list=(
"france"
"andorra"
"belgium"
"germany/baden-wuerttemberg" "germany/nordrhein-westfalen" "germany/rheinland-pfalz" "germany/saarland"
"italy/nord-ovest"
"luxembourg"
"netherlands"
"spain/aragon" "spain/cataluna" "spain/navarra" "spain/pais-vasco"
"switzerland"
)
if [ "$(date '+\%u')" = "5" ]; then
#Supprimer les fichiers pbf
rm -f ${osm_file_path}*.pbf
rm -f ${osm_file_path}**/*.pbf
# Téléchargement des fichiers pbf
echo "Downloading and processing pbf files"
for osm_file in "${osm_list[@]}"
do
filename="${osm_file_path}${osm_file}"
echo "Processing ${osm_file}"
echo "Downloading from ${osm_gfb}${osm_file}-latest.osm.pbf..." >> ${log_path}${osm_file}.log 2>&1
while true; do
# Téléchargement du fichier
wget --output-document="${filename}.osm.pbf" "${osm_gfb}${osm_file}-latest.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
computed_md5=$(md5sum "${filename}.osm.pbf" | awk '{ print $1 }')
# Téléchargement md5
wget --output-document="${filename}.osm.pbf.md5" "${osm_gfb}${osm_file}-latest.osm.pbf.md5" >> ${log_path}${osm_file}.log 2>&1
expected_md5=$(cat "${filename}.osm.pbf.md5" | awk '{ print $1 }')
# Check md5
if [[ "$computed_md5" == "$expected_md5" ]]; then
echo "File successfully downloaded and verified." >> ${log_path}${osm_file}.log 2>&1
rm "${filename}.osm.pbf.md5"
break
else
echo "MD5 checksum mismatch. Redownloading the file..." >> ${log_path}${osm_file}.log 2>&1
rm "${filename}.osm.pbf.md5" "${filename}.osm.pbf"
fi
done
if [ $? -ne 0 ]; then
echo "Error: Failed to download the file." >> ${log_path}${osm_file}.log 2>&1
exit 1
fi
# Assemblage des fichiers
if [ ! -f "${osm_file_path}osm_file.pbf" ]; then
# Premier fichier : trier puis renommer
echo "sorting file" >> ${log_path}${osm_file}.log 2>&1
osmium sort --strategy="multipass" -v -o "${filename}-sorted.osm.pbf" "${filename}.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
echo "creating file ${osm_file_path}osm_file.pbf" >> ${log_path}${osm_file}.log 2>&1
mv "${filename}-sorted.osm.pbf" "${osm_file_path}osm_file.pbf"
else
# Fichiers subséquents : trier puis merger
echo "sorting file" >> ${log_path}${osm_file}.log 2>&1
osmium sort --strategy="multipass" -v -o "${filename}-sorted.osm.pbf" "${filename}.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
echo "merging file on ${osm_file_path}osm_file.pbf" >> ${log_path}${osm_file}.log 2>&1
osmium merge -v "${osm_file_path}osm_file.pbf" "${filename}-sorted.osm.pbf" -o "${osm_file_path}osm_tmp.pbf" >> ${log_path}${osm_file}.log 2>&1
mv "${osm_file_path}osm_tmp.pbf" "${osm_file_path}osm_file.pbf"
rm -f "${filename}-sorted.osm.pbf"
fi
done
fi
maj_forcee.sh
(Used to force updates)
#!/usr/bin/env bash
# Pointer variables sur les bons chemins
osm_file_path=/custom_files/
log_path=/var/log/carto/
# Source des fichiers de cartographie
osm_gfb=https://download.geofabrik.de/europe/
osm_list=(
"france"
"andorra"
"belgium"
"germany/baden-wuerttemberg" "germany/nordrhein-westfalen" "germany/rheinland-pfalz" "germany/saarland"
"italy/nord-ovest"
"luxembourg"
"netherlands"
"spain/aragon" "spain/cataluna" "spain/navarra" "spain/pais-vasco"
"switzerland"
)
if [ "True" ]; then
#Supprimer les fichiers pbf
rm -f ${osm_file_path}*.pbf
rm -f ${osm_file_path}**/*.pbf
# Téléchargement des fichiers pbf
echo "Downloading and processing pbf files"
for osm_file in "${osm_list[@]}"
do
filename="${osm_file_path}${osm_file}"
echo "Processing ${osm_file}"
echo "Downloading from ${osm_gfb}${osm_file}-latest.osm.pbf..." >> ${log_path}${osm_file}.log 2>&1
while true; do
# Téléchargement du fichier
wget --output-document="${filename}.osm.pbf" "${osm_gfb}${osm_file}-latest.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
computed_md5=$(md5sum "${filename}.osm.pbf" | awk '{ print $1 }')
# Téléchargement md5
wget --output-document="${filename}.osm.pbf.md5" "${osm_gfb}${osm_file}-latest.osm.pbf.md5" >> ${log_path}${osm_file}.log 2>&1
expected_md5=$(cat "${filename}.osm.pbf.md5" | awk '{ print $1 }')
# Check md5
if [[ "$computed_md5" == "$expected_md5" ]]; then
echo "File successfully downloaded and verified." >> ${log_path}${osm_file}.log 2>&1
rm "${filename}.osm.pbf.md5"
break
else
echo "MD5 checksum mismatch. Redownloading the file..." >> ${log_path}${osm_file}.log 2>&1
rm "${filename}.osm.pbf.md5" "${filename}.osm.pbf"
fi
done
if [ $? -ne 0 ]; then
echo "Error: Failed to download the file." >> ${log_path}${osm_file}.log 2>&1
exit 1
fi
# Assemblage des fichiers
if [ ! -f "${osm_file_path}osm_file.pbf" ]; then
# Premier fichier : trier puis renommer
echo "sorting file" >> ${log_path}${osm_file}.log 2>&1
osmium sort --strategy="multipass" -v -o "${filename}-sorted.osm.pbf" "${filename}.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
echo "creating file ${osm_file_path}osm_file.pbf" >> ${log_path}${osm_file}.log 2>&1
mv "${filename}-sorted.osm.pbf" "${osm_file_path}osm_file.pbf"
else
# Fichiers subséquents : trier puis merger
echo "sorting file" >> ${log_path}${osm_file}.log 2>&1
osmium sort --strategy="multipass" -v -o "${filename}-sorted.osm.pbf" "${filename}.osm.pbf" >> ${log_path}${osm_file}.log 2>&1
echo "merging file on ${osm_file_path}osm_file.pbf" >> ${log_path}${osm_file}.log 2>&1
osmium merge -v "${osm_file_path}osm_file.pbf" "${filename}-sorted.osm.pbf" -o "${osm_file_path}osm_tmp.pbf" >> ${log_path}${osm_file}.log 2>&1
mv "${osm_file_path}osm_tmp.pbf" "${osm_file_path}osm_file.pbf"
rm -f "${filename}-sorted.osm.pbf"
fi
done
fi
Vroom (optional ofc)
vroom-config.yml
cliArgs:
geometry: false # retrieve geometry (-g)
planmode: false # run vroom in plan mode (-c) if set to true
threads: 4 # number of threads to use (-t)
explore: 5 # exploration level to use (0..5) (-x)
limit: '1mb' # max request size
logdir: '/..' # the path for the logs relative to ./src
logsize: '100M' # max log file size for rotation
maxlocations: 1000 # max number of jobs/shipments locations
maxvehicles: 200 # max number of vehicles
override: true # allow cli options override (-c, -g, -t and -x)
path: '' # VROOM path (if not in $PATH)
port: 3000 # expressjs port
router: 'ors' # routing backend (osrm, libosrm or ors)
timeout: 300000 # milli-seconds
baseurl: '/' #base url for api
routingServers:
#osrm:
# car:
# host: '0.0.0.0'
# port: '5000'
# bike:
# host: '0.0.0.0'
# port: '5001'
# foot:
# host: '0.0.0.0'
# port: '5002'
ors:
driving-car:
host: 'ors-app'
port: '8080'
# driving-hgv:
# host: '0.0.0.0'
# port: '8080'
cycling-regular:
host: 'ors-app'
port: '8080'
# cycling-mountain:
# host: '0.0.0.0'
# port: '8080'
# cycling-road:
# host: '0.0.0.0'
# port: '8080'
# cycling-electric:
# host: '0.0.0.0'
# port: '8080'
foot-walking:
host: 'ors-app'
port: '8080'
# foot-hiking:
# host: '0.0.0.0'
# port: '8080'
#valhalla:
# auto:
# host: '0.0.0.0'
# port: '8002'
# bicycle:
# host: '0.0.0.0'
# port: '8002'
# pedestrian:
# host: '0.0.0.0'
# port: '8002'
# motorcycle:
# host: '0.0.0.0'
# port: '8002'
# motor_scooter:
# host: '0.0.0.0'
# port: '8002'
# taxi:
# host: '0.0.0.0'
# port: '8002'
# hov:
# host: '0.0.0.0'
# port: '8002'
# truck:
# host: '0.0.0.0'
# port: '8002'
# bus:
# host: '0.0.0.0'
# port: '8002'
wow - thanks a lot @vchalmel
will need so time to implement this 