So I’ve outlined the context of my question in this short video. All of my data layers are derived from OSM data, and all roads have associated OSMIDs.
First, can one use data more precise/granular than the OpenStreetMap Way IDs (OSMIDs) for the OpenRouteService greenindex weighting csv? Since the GreenIndex weighting csv only takes a list of: <OSM_IDs>,<Weight_Value_Between_0-1>
I quickly realized I need to split hundreds of roadways on OpenStreetMap in order to properly weight the small segments of roads I’m interested in having in my green weighting file. In other words, there are ~1/2 mile long roads spanning many many blocks (identified through one OSM ID). So if I wish to weight just a small segment of that road for bicycle routing, I would need to use the “OSM Split tool” to re-idenitify that segment of road with a unique OSM Way ID. Not sure if this issue has come up before? @nils?
Second, if I want to negatively weight a road, to deter routing, do I just use a negative value from 0 to 1, in that weighting file?
The most comprensive documentation I’ve found on the green index weighting file been from here which states:
factor : Values range from 0 to 1 . 0 equals normal routing. 1 will prefer ways through green areas over a shorter route.
If additional documentation on custom weighting profiles I’d gladly consume it. @nils curious on thoughts on this! Thanks in advance.
Okay, so I’ve realized my approach to weighting the segments is totally wrong. I absolutely should not split the OSM Way IDs on the online public OSM dataset, and then wait for that to refresh to geofabrik, download that and use that in a ORS build for my website.
I have two options I should use to approach this situation
Take the OSM data (the pbf file) and split the ways offline in a program like JOSM, save that, and build the edge graphs.
Or I can not pre-process the pbf file, generate the edge file regularly, and then somehow open that edge file and match the coordinates with my shapefile I wish to apply the weights to. But I think that method will require more programming.
If anyone out there has experience with either of these methods, I’d be curious for input.
As you are actively working on this right now, I want to give you my thoughts although i’m not too familiar with how the backend parses the OSM IDs:
Maybe it’s possible to implement additional logic to the method that reads the ID file so you can further specify what part of the way you want to use:
I imagined something like, instead of only having the way ID
you also have the node IDs of start and end point of the segment you want to use
Or if it’s not working with ids the number of the nodes
159359417,0;4-7;12-13
At the place where the file is read only the specified parts should then be weighted.
I’m pretty sure it’s not working that way, as only the way IDs are stored… but maybe this will help to look from a different angle
Hm, no that won’t be possible (without quite some work).
What @amandus suggests likely won’t work unfortunately, since oe can’t give different properties to segments of OSM features. You’d have to geometrically split up the OSM feature into the same number of features as Dylan’s weighting features.
Also, please be careful with editing OSM live data for your (own more or less) use case. Likely the long-time contributors will revert your changes. So, first thing is: contact your local OSM chapter and discuss. Tbh, I don’t have high hopes here, since it’s veeeery specific to your custom weight geometries. Thing is, we all want to keep the amount of features to a minimum in OSM, so we usually don’t wanna split up road features too much if it’s not for a very good reason. Like nodes for turn restrictions.
What you’ll have to do here is: build your own OSM PBF file. It’s not super straight forward, but it’d be scalable, and you’d never have to manually fix geometries and features, nor would you have to update OSM.
So, let’s start with this: How do you determine the greenness and what are the roads based on that you assign an index? Are they based on OSM?
Likely the long-time contributors will revert your changes. So, first thing is: contact your local OSM chapter and discuss. Tbh, I don’t have high hopes here
Yes, I completely agree, editing the published OSM data for my benefit is not going to happen.
What you’ll have to do here is: build your own OSM PBF file.
Yes, that’s what I was thinking too.
So, let’s start with this: How do you determine the greenness and what are the roads based on that you assign an index? Are they based on OSM?
Yes, all results data is entirely based entirely on OSM data.
‘Greenness’ of my road network is determined from processing OSM road data. Data was re-classified as Low-Stress/High-Stress based upon OSM attribute tags (e.g. speed limit, road type, etc…) Connective Crosswalk Roads were based upon OSM Roads that were within a buffer distance of a Crosswalk POIs data layer. All resulting data share the same lines and vertices as raw OSM data, but the segments are just split at different junctions from the OSM Way IDs as the video illustrates
As I best understand, my current plan to build my own PBF
(which I’d very much welcome feedback):
Open the the PBF in an OfflineDesktop OSM Editor: either JOSM or Merkatoor. 2a. Problem - Neither JOSM or Merkatoor allow for opening up file sizes as large as arizona-latest.osm.pbf… I have to process it in many separate tiles which seems very inefficient, but I don’t know a better way. And then I’d have to merge the results as discussed here using osmium, or osmosis. (Any better ideas?)
Assuming I do get arizona-latest.osm.pbf open in JOSM without having to clip it into many small tiles, I now need to go through and edit the PBF as necessary (i.e. split all of the OSM Way IDs into the shorter segment lengths I wish to have weighted). 3a. Problem - I know JOSM allows me to split ways manually through mouse clicks, but that will take weeks of brute force. Ideally I need to split the ways programmatically via importing my crosswalk road segments, and somehow(?) splitting it based on that shapefile. After doing some google searcher, someone else does discuss programmatically splitting Way IDs here, so maybe that can be done, but not in JOSM.
Assuming I do split the road segments as needed, I now need to assign new Way IDs values to all the new segments I just created, and save that list of IDs in a greenindex.csv file to use as my weighting file for OpenRouteService. I don’t think this step should be too hard.
Proceed with running a local instance of OpenRouteService with my greenindex file and I’m done!
Does this process make sense? I definitely need advice on steps 2 and 3.
I’m not entirely sure whether working with custom PBF files is the preferred solution here. This is because OSM ways are already split internally by openrouteservice into edges which are the segments used for actual routing. So for more granularity you might want to work at the level of edges via so-called flag encoders rather than re-use the GreenIndex. As the cycling profiles use PriorityWeighting, you could deter routing on certain edges by assigning them a lower priority in a customized flag encoder extending CommonBikeFlagEncoder.
Identifying the relevant way segments corresponding to crosswalks programmatically might be tricky though. Do you have some ideas already? We are currently working on a similar problem where we are interested in taking into account traffic lights, see #674. Unfortunately, we don’t have a ready solution yet.
Ok, that’s true, but that approach also needs some sort of OSM data injection. Or @amandus idea with keeping track of segments in the csv. In any case that would need quite a bit of Java dev. Well, a considerable quite a bit I fear.
Altering the PBF is language-independent and framework-independent (could be fairly easy to reuse for other FOSS routers). If you’re already familiar with JS, osmiums node bindings could work for you. I only ever used the Python bindings, and it’s a little awkward but gets the job done.
Still, I don’t know what the “greenness” data is supposed to be. Is it green? Now I heard crosswalks (nothing to do with green). We really need to know to advise on the best strategy. (just read the relevant lines )
Don’t use JOSM for editing that amount of data, you’ll be sick of it after 10 mins. You will have to do it programmatically, every language offer some sort of libraries to parse and create those file. It’s just a Protobuf file, a Google-originated strongly-typed buffer, which is quite popular because of its structure and serialization capabilities. That’ll help you researching.
EDIT: ok, watched the video. Sorry, usually don’t have much patience for videos; I prefer text. Nice one though;) So, as Andrzej said this is going to be a little difficult, since you ideally want to a) either let ORS split ways at crossroads (points) or b) split ways with osmium or related software.
What I’d do for b)
Loop through the nodes of the PBF in read mode, only noting the OSM ID’s which are crosswalks.
Then loop through all ways and
write every feature which does not contain a crossroad unaltered (best to invent new OSM IDs)
if a crosswalk is on a way, its OSM ID is included in the way’s node references, so you take these ways and split them up at the crosswalk and write as well. Also take the IDs for the CSV and write a weight.
That should be it. I’ve done things like that before and can confirm osmiums bindings to write PBF work really well. It’s a steep learning curve, but you’re half-way there.
I don’t think we’re a terribly good fit for this. I’d do it in Python, which wouldn’t really help you understand that much, and I hardly have time. I don’t know him personally, but from his Twitter activity I really like him: Steve Bennett, http://hire.stevebennett.me. He has an impressive portfolio and he might’ve come across a use case like this. At the very least he seems like a very genuine guy, who can point you in the right direction.
)