Obtaining the Tutorial Data

Before beginning the tutorial, you will copy all of the required tutorial data onto your Desktop. Option 1 is best if you are completing this tutorial in one of our short courses or from the GIS/Data Center and Option 2 is best if you are completing the tutorial from your own computer.

OPTION 1: Accessing tutorial data from Fondren Library using the gistrain profile

If you are completing this tutorial from a public computer in Fondren Library and are logged on using the gistrain profile, follow the instructions below:

1. On the Desktop, double-click the Computer icon > gisdata (\\file-rnas.rice.edu) (R:) > Short_Courses > Analyzing_Spatial_Patterns.
2. To create a personal copy of the tutorial data, drag the Patterns folder onto the Desktop.
3. Close all windows.

OPTION 2: Accessing tutorial data online using a personal computer

If you are completing this tutorial from a personal computer, you will need to download the tutorial data online by following the instructions below:

1. Click Patterns.zip above to download the tutorial data.
2. Open the Downloads  folder.
3. Right-click  Patterns.zip and select Extract All....
4. In the 'Extract Compressed (Zipped) Folders' window, accept the default location into the Downloads folder and click Extract.
5. Drag the unzipped  Patterns  folder onto your Desktop.
6. Close all windows.

Spatial Statistics in ArcGIS Pro

Opening an Existing Project

1. On the Desktop, double-click the Patterns folder.

2. Double-click the Patterns.aprx project file to open the project in ArcGIS Pro.

3. In the Catalog pane on the right, expand the Databases folder.
4. Expand the Patterns.gdb geodatabase.
5. Right-click the CensusTracts_MTTW_2014 feature class and select Add To New Map.

This layer provides boundaries for all of the census tracts in Harris County. For more information on census tracts, review the Introduction to Census Geographies course.

1. In the Contents pane on the left, right-click the CensusTracts_MTTW_2014 layer name and select Attribute Table.

MTTW stands for Means of Transportation To Work. The attribute table provides data on the percentage of workers 16 and older who commuted by driving alone, carpooling, or taking public transportation.

1. Close the CensusTracts_MTTW_2014 table view.
2. On the ribbon, click the Analysis tab.
3. On the Analysis tab, within the first Geoprocessing group, click the Tools button.
   
   

Notice that the Geoprocessing pane has opened on the right as a new tab on top of the Catalog pane. Typically, you would use the 'Find Tools' search box at the top of the Geoprocessing pane to search for the name of the tool you'd like to use, but, at times, especially when learning the software, it can be helpful to view the full hierarchy of all the tools available, because you will often discover related and helpful tools that you didn't know existed and wouldn't know to search for. You might also completely forget the name of a tool, but be able to locate it based on the hierarchy. For these reasons, we will be manually navigating the toolboxes throughout this tutorial. The more typical workflow of searching directly for a specific tool is covered at the end of the Introduction to Geoprocessing tutorial.

Spatial Autocorrelation (Moran's I)

1. At the top of the Geoproccessing pane, click the Toolboxes tab.
2. Click the Spatial Statistics Tools toolbox > Analyzing Patterns toolset > Spatial Autocorrelation (Global Moran's I) tool.
3. In the upper right corner of the ‘Spatial Autocorrelation’ tool, hover over the Help ? button to display the tool help in a pop-up window.
   
   

The Spatial Autocorrelation tool evaluates whether data is clustered, dispersed, or randomly distributed based on both feature locations and feature values simultaneously.

Percent Commuting Alone By Car

1. For 'Input Feature Class', use the drop-down menu to select the CensusTracts_MTTW_2014 layer.
2. For 'Input Field', use the drop-down menu to select the Percent_CarTruckOrVan_droveAlone field.
3. Check Generate Report.
4. At the bottom of the Geoprocessing pane, click Run.

Once the tool has finished running, you will see a message at the bottom of the Geoprocessing pane, indicating that the tool has completed with warnings.

1. Hover over the Completed with warnings message to display the tool parameters and messages in a pop-up window.
   
   

Under the lower 'Messages' section, notice that the warning message indicates that the neighborhood search distance used in the analysis was 38893 feet. Since the distance threshold does impact the statistical results, it is important to make note of the threshold. You will learn more about the impact of the distance threshold later on. While the Moran's Index I and z-score are also displayed in the messages, they are easier to interpret in the context of the full report.

1. Within the 'Spatial Autocorrelation Parameters and Messages' pop-up window, click the Report File hyperlink to open the report in your default web browser. The HTML report file can also be accessed directly from your project folder for future reference outside of ArcGIS Pro.
   
   

In the top left, notice that the z-score is 39.43. In the top right, notice that any z-score larger than 2.58 has less than 1% chance of occurring randomly. In the center section is the normal distribution curve and a dotted line illustrating where the z-score for this analysis is located on that curve. It also illustrates that a positive z-score indicates that the data is clustered, while a negative z-score would indicate that the data is dispersed. Below the diagram is a helpful summary sentence, "Given the z-score of 39.43, there is a less than 1% likelihood that this clustered pattern could be the result of random chance." Because the z-score is more than an order of magnitude larger than 2.58, you will notice in the top left that the p-value is actually 0.00, which means there is essentially no chance that this pattern is random.

Simply knowing that the percentages of people who drove alone to work are highly clustered might not provide you with actionable knowledge, but what this result indicates is that there is a strong spatial pattern present, which is worth investing the time to investigate further. If the data was randomly distributed, you could stop here. Because you know your data displays strong clustering, you can now ask more interesting questions. Is it the high or low percentages of commuters driving alone that are clustered? Where are they clustered? You will see find that all of the spatial statistics tools build upon each other to answer these questions and provide additional knowledge.

Percent Commuting By Carpool

As the Geoprocessing pane doesn’t reset after a tool has finished running, it is easy to rerun tools with slightly modified settings. In future versions of ArcGIS Pro, batch processing is also supported, which facilitates multiple runs of the same tool within a single interface.

1. In the Geoprocessing pane, for 'Input Field', use the drop-down menu to select the Percent_CarTruckOrVan_carpooled field.
2. Click Run.
3. Hover over the Completed with warnings message and click the Report File hyperlink.

This time, the z-score is 28.06, which is slightly lower than before, but still indicates that the percent of people who carpool to work is also highly clustered.

Percent Commuting By Public Transportation

1. In the Geoprocessing pane, for 'Input Field', use the drop-down menu to select the Percent_PublicTransportation field.
2. Click Run.
3. Hover over the Completed with warnings message and click the Report File hyperlink.

Again, the z-score indicates that the percent of people who took public transportation to work is highly clustered. The enormous z-score of 55.84 indicates that the public transportation variable is the most highly clustered of the three. Logically, this makes sense, because we would think people living near metro stations or major bus route stops would be more likely to use them and, therefore, people taking public transportation to work would be spatially clustered around those stop locations.

Incremental Spatial Autocorrelation

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Within the Analyzing Patterns toolset, click the Incremental Spatial Autocorrelation tool.
3. In the upper right corner of the ‘Incremental Spatial Autocorrelation’ tool, hover over the Help ? button.
   
   

The Incremental Spatial Autocorrelation tool iterates the Spatial Autocorrelation tool using multiple distance bands and plots the corresponding z-scores at each distance. The result is a chart showing you the peak z-scores at distances where spatial clustering is most pronounced.

Percent Commuting Alone By Car

1. For 'Input Features', use the drop-down menu to select the CensusTracts_MTTW_2014 layer.
2. For 'Input Field', use the drop-down menu to select the Percent_CarTruckOrVan_droveAlone field.
3. For 'Number of Distance Bands', type25.
4. For 'Output Report File', type "ISA_DroveAlone_25.pdf". If you click elsewhere, you will be able to see the full file path and notice that the tool will automatically locate your PDF file within your project folder.
5. Click Run.
6. Hover over the Completed with warnings message and click the Output Report File hyperlink.

Notice that the significance of clustering of the percent of people driving alone to work increases as the distance threshold increases and that there are two peak distances. Also, notice that the data is highly clustered regardless of the distance threshold. On the third page of the PDF, you can see that the peak z-score occurs at a distance of approximately 67,500 feet, or nearly 13 miles.

Percent Commuting By Carpool

1. For 'Input Field', use the drop-down menu to select the Percent_CarTruckOrVan_carpooled field.
2. For 'Number of Distance Bands', type 10.
3. For 'Output Report File', type "ISA_Carpooled_10.pdf".
4. Click Run.
5. Hover over the Completed with warnings message and click the Output Report File hyperlink.

This variable displays a reverse pattern in that the significance of clustering decreases as distance increases. In fact, when the distance is greater than approximately 65,000 feet, the pattern becomes random, rather than clustered, as indicated by the yellow data points on the chart. This is the same distance at which driving alone displayed peak clustering. You could rerun the tool and set a beginning distance and distance increment to determine if there is a peak distance below 40,000 feet.

Percent Commuting By Public Transportation

1. For 'Input Field', use the drop-down menu to select the Percent_PublicTransportation field.
2. For 'Number of Distance Bands', type 10.
3. For 'Output Report File', type "ISA_Transit_10.pdf".
4. Click Run.
5. Hover over the Completed with warnings message and click the Output Report File hyperlink.

This variable displays a similar curve to the driving alone variable with a similar peak distance around 66,000 feet, but the curve is more pronounced.

High/Low Clustering (Getis_Ord General G)

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Within the Analyzing Patterns toolset, click the High/Low Clustering (Getis-Ord General G) tool.
3. In the upper right corner of the ‘High/Low Clustering’ tool, hover over the Help ? button.
   
   asdf

Percent Commuting Alone By Car

1. For 'Input Feature Class', use the drop-down menu to select the CensusTracts_MTTW_2014 layer.
2. For 'Input Field', use the drop-down menu to select the Percent_CarTruckOrVan_droveAlone field.
3. Check Generate Report.
4. Click Run.
5. Hover over the Completed with warnings message and click the Report File hyperlink.

Pay attention to the “General G Summary” table, which shows this is a “low-clusters” scenario.

Percent Commuting By Carpool

1. For 'Input Field', use the drop-down menu to select the Percent_CarTruckOrVan_carpooled field.
2. Check Generate Report.
3. Click Run.
4. Hover over the Completed with warnings message and click the Report File hyperlink.

Pay attention to the “General G Summary” table, which shows this is a “high-clusters” scenario.

Percent Commuting By Public Transportation

1. For 'Input Field', use the drop-down menu to select the Percent_PublicTransportation field.
2. Check Generate Report.
3. Click Run.
4. Hover over the Completed with warnings message and click the Report File hyperlink.

Pay attention to the “General G Summary” table, which shows this is a “high-clusters” scenario.

For the next set of tools, you will run them each twice: once with a polygon dataset and once with a point dataset.

Percent White

Optimized Hot Spot Analysis

1. In the Catalog pane on the right, within the Patterns.gdb geodatabase, right-click the Race_CensusTracts feature class and select Add To New Map.

Since hot spot analysis is based on a distance threshold, it is important to get a good understanding of the spatial autocorrelation of your data as it related to distance threshold. One option is to run the incremental spatial autocorrelation on the same variable first, to help determine an appropriate distance threshold, which can then be entered in the Hot Spot Analysis tool. Another option is to use the Optimized Hot Spot Analysis tool, which will automatically run the Incremental Spatial Autocorrelation tool first in the background to select the distance threshold with the most significant?

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Within the Spatial Statistics toolbox, click the Mapping Clusters toolset > Optimized Hot Spot Analysis tool.
3. In the upper right corner of the ‘Hot Spot Analysis’ tool, hover over the Help ? button.
4. (Help)

asdf

1. For 'Input Features', use the drop-down menu to select the CensusTracts_Race_2014  layer.
2. For 'Output Features', type "Race_OHSA".
3. For 'Analysis Field', use the drop-down menu to select the Percent_White field.
4. Check Generate Report.
5. ClickRun.

Cluster and Outlier Analysis (Anselin Local Morans I)

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Within the Mapping Clusters toolset, click the Cluster and Outlier Analysis (Anselin Local Morans I) tool.
3. For 'Input Feature Class', use the drop-down menu to select the CensusTracts_Race_2014 layer.
4. For 'Input Field', use the drop-down menu to select the Percent_White field.
5. For 'Output Features', rename the feature class from asdf to "Race_COA".

Crime Points

Optimized Hot Spot Analysis (Getis-Ord Gi*)

1. In the Catalog pane on the right, within the Patterns.gdb geodatabase, right-click the Crimes_2014 feature class and select Add To New Map.
2. Right-click the Freeways feature class and click Add To Current Map.

Houston crime data is reported at the block level, not the individual address level to protect confidentiality. Therefore, when the addresses are geocoded, all crimes that happened within a single block are all geocoded to the separately to the same point location in the center of the block, resulting in coincident, or overlapping points for all crimes. In order to run a hot spot analysis on this point data, we, instead, need to have a single point at each location and a value field indicating the number of crimes that occurred in that location. In our case, the address data was all run using a consistant geocoder, so we can be sure that crimes on the same block were located at exactly the same point. If, however, you were aggregating crime data points from a variety of data providers who may have used a variety of geocoding methods or if you created a community website, where residents could manually click on a location to report a crime, you would need a way of standardizing the crime locations, otherwise the number of crimes at each location might simply be 1, which would not provide for as interesting of results. Though not necessary for our dataset, you will test out the integrate tool for messier datasets.

Copy Features

The first step is to use the Collect Events tool to aggregate the data, but it is important to make a copy of the original input data before proceeding, because the Integrate tool modifies the input dataset by shifting, or standardizing, the locations of the input features.

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Click the Data Management Tools toolbox > Features toolset > Copy Features tool
3. For 'Input Features', use the drop-down menu to select the total_Crimes layer.
4. For 'Output Feature Class', rename the output feature class from to total_crimes_original.

Show how to see multiple points at same location and attribute table.

Integrate

Now you are ready to use the integrate tool to standardize the data

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Within the Data Management Tool toolbox, click the Feature Class toolset > Integrate tool.
3. For 'Input Features', use the drop-down menu to select the total crimes layer.

A good rule of thumb for the XY tolerance is to consider the accuracy of your data. In this case, all crimes were reported by blocks and were geocoded at centers of blocks, so we could techically will set XY tolerance to 0 for the aggregation.

Collect Events

1. In the Geoprocessing pane, within the Spatial Statistics Tools toolbox,click the Utilities toolset > Collect Events tool.
2. Click OK.

Notice that the output from Collect Events is rendered with graduated circles reflecting the number of incidents at each point)

• Geoprocessing -> ArcToolbox -> Spatial Statistics Tools -> Mapping Clusters -> Hot Spot Analysis (Getis-Ord Gi*)

• Result:

Note that hot spot doesn’t mean high value. A has a feature with low value, but surrounding neighbors all have high value, which will cause a significantly high mean value than the global mean for this spot. It depends on the scale of the analysis.

Optimized Hot Spot Analysis (Getis-Ord Gi*)

Using Census Block Groups and total_crimes Datasets

Version 10.3 and 10.4 have “optimized hot spot analysis tool”, which automatically aggregates data first and does hot spot analysis for points data. For this tool, we will aggregate data by census block groups.

• At the top of the Geoprocessing pane, click the Back arrow button.
• Within the Spatial Statistics toolbox, click the Mapping Clusters toolset > Optimized Hot Spot Analysis tool.
• In the upper right corner of the ‘Hot Spot Analysis’ tool, hover over the Help ? button.
• (Help)
• For 'Input Features', use the drop-down menu to select the total layer.
• For 'Output Features', type "Race_OHSA".
• For 'Analysis Field', use the drop-down menu to select the Percent_White field.
• Check Generate Report.
• Click Run.

Cluster and Outlier Analysis (Anselin Local Morans I)

1. At the top of the Geoprocessing pane, click the Back arrow button.
2. Within the Mapping Clusters toolset, click the Cluster and Outlier Analysis (Anselin Local Morans I) tool.

Using “AggregatedTotalCrimes” dataset, which was generated during the Hot Spot Analysis.

• Geoprocessing -> ArcToolbox -> Spatial Statistics Tools -> Mapping Clusters -> Cluster and Outlier Analysis (Anselin Local Morans I