Hold Onto Your Pearls, Grandma! We’re Switching to OSM!

About Us

• Community Organizational Member of OSMUS since 2024
• Supporter Corporate Member of OSMF since 2024

(Yes, we’re on the Organized Editing wiki page!)

What’s “Authoritative”, Anyway?

It’s all about the source

• What matters:
  • Official
  • Reliable
• What doesn’t matter:
  • Means of access

We want the data to come from the people who are “in charge” of it. Or if we get it from another place, we want it mostly unchanged.

We also want it to be reliable, meaning it’s reasonably up to date, and seems accurate enough that we don’t need to validate it.

If you are on ArcGIS Online, consider the Living Atlas. It’s full of authoritative data, but most of what you’re seeing is just a copy hosted on one of Esri’s servers. It’s no less authoritative for that fact.

And let’s just state: for various Kendall County datasets, you’re looking at the authoritative source. It’s us! So, why keep our stuff in a silo, away from the where the community needs it?

OSM - 2007

Highways and waterways in 2007

Here I want to point out: OSM is full of authoritative data! We talk about it as a crowdsourcing effort, which it is, but there are also a lot of imports.

Roads and boundaries, for instance. These were added from US Census shapefiles back in 2007/2008. Prior to this, most of the US was basically empty.

OSM - 2009

Highways and waterways in 2009

But look at it post-import. Quite a difference!

This means that for a window of time after the import, most of the highways (if not most of the features) in OSM in the US were authoritative federal data.

The fact that these features sit side by side with crowdsourced bench locations doesn’t mean they aren’t authoritative, but it does make it a little harder to tell.

Kendall County

Okay, so what’s our approach to all this as a County?

I was pretty jazzed when I started working at the County. I was on the inside at an authoritative source! Maybe I could facilitate some imports and improve OSM.

I got approval to do so! But it didn’t take long for me to see that for many of the datasets in our “open data catalog”, a lot of it was already in OSM, and it was better.

To no one’s surprise, OSM had superior data in most cases. So we dropped most of our ‘public’ layers in favor of OSM (water features, railways, even centerlines). we reviewed our datasets to determine if we had information that should be on OSM (Fran’s surface type and jurisdiction) and if any datasets weren’t appropriate for OSM - mainly parcels

Make the switch!

For most layers, the answer was clear: using and managing the data directly in OSM didn’t really have a downside, but we would be making our data much more accessible to the broader community. So we decided we would make the switch, and stop maintaining our own copies of a lot of these layers.

Why We Switched

It’s not just out of the goodness of our hearts

• History
• Flexibility
• Collaboration
• OSM and FOSS are cooler

Due Diligence

That’s easy to say, but not a decision to be made lightly. We had to make sure we had tools for keeping an eye on the data.

“Data Gardeners”

• OSMCha
• QGIS project
• MapRoulette
• Tasking Manager

I read the term “data gardener” in an article assessing OSM quality, and I love it. We decided we would be “gardeners” for the data in and around Kendall County.

We’re not gatekeepers, anyone’s welcome to edit. But we make sure that edits happening in our area get checked out. The primary way of doing this is watching a few OSMCha filter lists. We also have a QGIS project where we can see recent edits by user type (a table we manage internally). Similar to the OSMCha watchlist / trusted user, but more flexible.

MapRoulette is good for making “to do” lists out of specific OverPass queries, and the Tasking Manager has, and continues to help manage editing activities across staff.

For specific feature types like address points, we have more focused review tools to maintain a high standard of data.

But is it authoritative?

By engaging directly with OpenStreetMap:

• OSM is the official source, not a mirror of one
• All edits are reviewed by staff
• Quality has improved across the board

Honestly, I feel pretty comfortable claiming authoritative status for the OSM data in Kendall.

The fact is, there is no other official source of some of this data! And because of how accessible OSM is, the data as it stands today has gotten more detailed and accurate than it ever was before!

Okay but we still have Esri stuff

OSM Extract / Sync

flowchart LR

osm[(OSM)] -- osm2pgsql -->
pgsql[(PostGIS)] -- sql, geopandas -->
prtl[(ArcGIS\nEnterprise)]

Light Technical Details

• SQL gets jsonb into tabular format
• Uses changeset timestamp to isolate recent edits
• Compares against Esri layers using osm_id
• Selectively applies adds, updates, deletes
• Updates nightly without issue!

Questions?

Be in Touch!

josh@jcarlson.page

mbriganti@kendallcountyil.gov