Locators: Allow to access properties of the intersecting area

[tordans]

I was looking into the new Locators to see if we can migrate some of our custom CSV based code to this new core feature.

I noticed that this would require us to rethink how we access the data compared to what we do now.
It will likely be possible to do all we need, but different and maybe a bit less performant…

So this is meant to share what we are planning to do and start the discussion if it worth adding an additional feature to the locators.

Our goal:
For a given way, know if there is Mapillary coverage of type "panorama" or "regular".

Current approach:

• We iterate over the bike lane segments in osm2pgsql
• We have an external dataset of areas that represent mapillary coverage
  • https://github.com/vizsim/mapillary_coverage/tree/main/output
  • Example

    osm_id,mapillary_coverage
    123,pano
    234,regular
    

• We lookup the ID of the current osm object and add the external data als "pseudo tags" to our ways properties to be used during other LUA processing

This is the code ATM: https://github.com/search?q=repo%3AFixMyBerlin%2Ftilda-geo+mapillary_coverage+path%3A%2F%5Eprocessing%5C%2F%2F&type=code

Reading the CSV takes about 20 Min at the start of the osm2pgsql processing ATM.

processing  | CSV: File cached (2.0s, 1537504 rows) /data/pseudoTagsData/mapillaryCoverageData/mapillary_coverage.csv — Example: {[1978] = {mapillary_coverage = "pano"}}
2025-10-06 09:35:44  Reading input files done in 1205s (20m 5s).

How I would rebuild this using locators:

• Instead of using the CSV, we pre process the polygons to merge all overlapping, buffered Mapillary route lines into as few areas as possible. These are either pano or (fallback) regular for the image type. However, we can have overlapping areas, unless we substract the pano areas in a separate step.
• I import that into a separate DB table
• I then create two locators, one for mapillary_pano, one for mapillary_regular
• And then check against both to learn which it is.

Given the pre processing of the data, I can still use the first_intersecting logic because we made sure beforehand that the pano area "win" for a given area. I we skip the substraction step in the ata preparation we would have to look at all_intersecting so see if one is pano.

Alternative: Access properties of the area via the locator:

I think the approach above should work but one has to think about the properties as separate locators.
An alternative would be…

• We merge all areas (buffered mapillary routes) that overlap and give this area a property "coverage_type=pano|regular"
• We then create one locator mapillary_coverage
• And when one OSM way as a first_intersecting, we access the coverage_type property for that locator result

Since both approaches work in principle the difference will be mainly performance and developer UX, I guess.

[pnorman]

It feels odd to go from a list of OSM IDs to polygons, only to then use it to find if a way of a known OSM ID matches the list.

How does your current approach work with updates to OSM data? How does it work with updates to the coverage data?

[tordans]

It feels odd to go from a list of OSM IDs to polygons, only to then use it to find if a way of a known OSM ID matches the list.

Yes. But we only have the IDs right now because we do extensive pre processing to get them. And loading the CSV takes 20 Min. So I am looking into how we could optimizes and simplify this process…

How does your current approach work with updates to OSM data?

We don't do minutely updates, we create everything from scratch from a fresh OSM PBF once a night.

How does it work with updates to the coverage data?

We have to create the mapillary coverage file manually and try to create an update once every few month.