Dashboards and Data Expressions

Using and Abusing Arcade in ArcGIS Dashboards

Background

April 2021 - Data Expressions introduced!

Outline

  1. What is a Data Expression?
  2. Essential Parts
  3. Examples

What It Is

A data expression is…

TL;DR - Any Arcade expression that returns a valid FeatureSet.

See the Esri blog for a nice overview.

And What is a FeatureSet?

See the Esri docs for the full spec, but it’s just a JSON object that holds features.

Three attributes are essential, even when empty: fields, features, and geometryType1.

{
  fields: [],
  features: [],
  geometryType: 'some geometry type'
}

Fields

This is the FeatureSet’s schema. It is an array of fields, each with a number of attributes. 1

Only two are truly essential, name and type2.

[
  {
    name: 'some_field',
    type: 'someFieldType'
  },
  {
    name: 'another_field',
    type: 'anotherFieldType'
  }
]

Features

Each individual feature in this array will need a dictionary of attributes corresponding to the fields array.

If working with a spatial FeatureSet, a geometry object is needed as well.

[
  {
    attributes: {
      some_field: 'some value',
      another_field: 1.2
    }
  },
  {
    attributes: {
      some_field: 'another value',
      another_field: null
    },
    geometry: geom // some geometry object
  }
]

Building a Data Expression

Bringing in Data

Before we can do anything, we need to get data into the expression. This is done with the function FeatureSetByPortalItem:

var fs = FeatureSetByPortalItem(
  Portal('your-org-url'),
  'itemid of your service',
  0, // or whatever index your layer / table is at
  ['list', 'of', 'fields'], // or ['*']
  false // set to true to return geometry
)

The Easy Way

Certain Arcade expressions like GroupBy and Distinct will return a FeatureSet for you.

These will also let you use hacky SQL expressions to define ad-hoc fields.

return Distinct(
  fs,
  [
    {
      name: 'objectid',
      expression: 'objectid'
    },
    {
      name: 'new_field',
      expression: "CASE WHEN some_field = 1 THEN 'Yes' ELSE 'No' END"
    }
  ]
)

The Long Way

First, we create a dictionary to hold our FeatureSet.

We can define the fields array here, too.

var out_dict = {
    fields: [
        {name: 'field_A', type: 'esriFieldTypeInteger'},
        {name: 'field_B', type: 'esriFieldTypeString'}
    ],
    features: [],
    geometryType: ''
}

Next, we populate the features array. This is usually done in some kind of loop.

Using the array function Push is handy for this.

for (var f in input_fs){
    Push(
        out_dict['features'],
        { attributes: {
            field_A: 8,
            field_B: 'some string'
        }}
    )
}

Examples

Live Dashboard

Aggregating Data

Getting MW output of coal plants per state.

var fs = FeatureSetByPortalItem(
  Portal("https://arcgis.com"),
  "b063316fac7345dba4bae96eaa813b2f",
  0,
  ["Total_MW", "State", "PrimSource"],
  false
)

return GroupBy(
  fs,
  ["State", "PrimSource"],
  [
    { name: "Total_MW", expression: "Total_MW", statistic: "SUM" },
    { name: "Avg_MW", expression: "Total_MW", statistic: "AVG" },
    { name: "Num_Plants", expression: "1", statistic: "COUNT" }
  ]
)

Simple Buffer

Illinois coal plants, buffered 5 miles.

var plants = FeatureSetByPortalItem(
  Portal("https://www.arcgis.com/"),
  "b063316fac7345dba4bae96eaa813b2f",
  0,
  ["Plant_Name", "Plant_Code"],
  true
)

plants = Filter(plants, "State = 'Illinois' AND PrimSource = 'coal'")

var features = []
var feat

var out_dict = {
  fields: [
    { name: "plant_name", type: "esriFieldTypeString" },
    { name: "plant_code", type: "esriFieldTypeInteger" }
  ],
  geometryType: "esriGeometryPolygon",
  features: []
}

for (var p in plants) {
  var fivemile = Buffer(p, 5, "miles")

  Push(
    out_dict["features"],
    {
      attributes: { plant_name: p["Plant_Name"], plant_code: p["Plant_Code"] },
      geometry: fivemile
    }
  );
}

return FeatureSet(Text(out_dict))

Spatial Join

Population counts within 5 miles of coal plants.

var tracts = FeatureSetByPortalItem(
  Portal("https://www.arcgis.com/"),
  "30338679df5542378ec86997ca447576",
  2,
  ["B01001_001E"],
  false
)

var plants = FeatureSetByPortalItem(
  Portal("https://www.arcgis.com/"),
  "b063316fac7345dba4bae96eaa813b2f",
  0,
  ["Plant_Name", "Plant_Code"],
  true
)

plants = Filter(plants, "State = 'Illinois' AND PrimSource = 'coal'")

var features = []
var feat

var out_dict = {
  fields: [
    { name: "population", type: "esriFieldTypeInteger" },
    { name: "plant_name", type: "esriFieldTypeString" },
    { name: "plant_code", type: "esriFieldTypeInteger" }
  ],
  geometryType: "esriGeometryPolygon",
  features: []
}

for (var p in plants) {
  var fivemile = Buffer(p, 5, "miles")
  var nearby_pop = Intersects(fivemile, tracts)

  Push(
    out_dict["features"],
    {
      attributes:
        {
          population: Sum(nearby_pop, "B01001_001E"),
          plant_name: p["Plant_Name"],
          plant_code: p["Plant_Code"]
        },
      geometry: fivemile
    }
  )
}

return FeatureSet(Text(out_dict))

Calculated Field

Adding Simpson’s Diversity Index to Census Block Groups

var fs = FeatureSetByPortalItem(
  Portal("https://arcgis.com"),
  "2718975e52e24286acf8c3882b7ceb18",
  5,
  [
    "NAME",
    "HISPPOP_CY",
    "NHSPWHT_CY",
    "NHSPBLK_CY",
    "NHSPAI_CY",
    "NHSPASN_CY",
    "NHSPPI_CY",
    "NHSPOTH_CY",
    "NHSPMLT_CY",
    "TOTPOP_CY"
  ],
  false
)

fs = Filter(fs, "ID LIKE '17113%'")

var sql =
`
1 - ((
    (HISPPOP_CY * (HISPPOP_CY - 1)) +
    (NHSPWHT_CY * (NHSPWHT_CY - 1)) +
    (NHSPBLK_CY * (NHSPBLK_CY - 1)) +
    (NHSPAI_CY * (NHSPAI_CY - 1)) +
    (NHSPASN_CY * (NHSPASN_CY - 1)) +
    (NHSPPI_CY * (NHSPPI_CY - 1)) +
    (NHSPOTH_CY * (NHSPOTH_CY - 1)) +
    (NHSPMLT_CY * (NHSPMLT_CY - 1))
)
/
(TOTPOP_CY * (TOTPOP_CY - 1) * 1.1 / 1.1))
`

return GroupBy(
  fs,
  {
    name: "NAME",
    expression: "SUBSTRING(NAME, 0, 10) + '0' + SUBSTRING(NAME, 12)"
  },
  { name: "simpsonDI", expression: sql, statistic: "SUM" }
)

Clocks!

Making a FeatureSet from scratch.

var n = Now();
var thehour = Hour(Now());
var theminute = Minute(Now());

if (thehour > 12) {
  thehour -= 12;
}

var out_dict = {
  fields: [
    { name: "thehour", type: "esriFieldTypeInteger" },
    { name: "theminute", type: "esriFieldTypeInteger" },
    { name: "hourdegrees", type: "esriFieldTypeDouble" },
    { name: "minutedegrees", type: "esriFieldTypeDouble" },
    { name: "ampm", type: "esriFieldTypeString" }
  ],
  geometryType: "",
  features: [
    {
      attributes:
        {
          thehour: thehour,
          theminute: theminute,
          hourdegrees: thehour * 30 + 5 / theminute,
          minutedegrees: theminute * 6,
          ampm: Iif(Hour(Now()) > 12, "PM", "AM")
        }
    }
  ]
}

return FeatureSet(Text(out_dict))

Questions?

Be in Touch!

josh@jcarlson.page

https://jcarlson.page

@jcarlson@mapstodon.space

Home