Updating the ArcGIS Online Components Database View

Summary

We have an ETL (code) that uses the component_arcgis_online_view Moped database view to push data about project components to an AGOL feature service with point, line, and combined_geometry layers (feature service endpoint). This describes the workflow of adding, updating, or removing fields in this service and the coordination it takes with the DTS Geospatial team.

There are two downstream dependencies of this database view that can break:

AGOL Feature Service (coordinate with the Geospatial team; see below)
PowerBI dataflow (coordinate with the Data Science team; see below)

Code updates

The database view and ETL GraphQL query must be updated with the new or updated field names.

Here is an example pull request: https://github.com/cityofaustin/atd-moped/pull/1357.

Adding a new field

Stage code changes for new field
Submit a service request for the GIS team to add new field
Release or patch to introduce new field into ETL
Tell the Data Science team about this update so they can update the PowerBI dataflow

Renaming an existing field

Stage code change for field rename
Submit a service request for the GIS team to add new column name. We keep the old column name to keep the ETL compatible and working with both the new and old field names
Release or patch to introduce new field into ETL
Request GIS team remove the old field name now that it is unused
Tell the Data Science team about this update so they can update the PowerBI dataflow

Removing an existing field

Stage code change for field removal
Release or patch to remove column from ETL
The ETL will not break with the changes up to this point. The next step can be performed anytime after removing the fields from the code to clean up the AGOL dataset.
Submit a service request for the GIS team to remove the field
Tell the Data Science team about this update so they can update the PowerBI dataflow