What is the Pick tool in ArcGIS's Spatial Analyst commonly used for?
I have recently encountered the Pick tool in the Spatial Analyst of ArcGIS. I understand what the tool does; it creates an output raster using a list of input rasters and a 'position' raster which dictates for each grid cell in the output which input it should receive the value from. I've looked for questions on GIS.SE that mention the Pick tool and there don't seem to be any. Having used GIS for quite some time now, I can't think of a past scenario when this operation would have been useful to me. Even the ArcGIS help documentation, which is normally quite extensive, doesn't reveal a use-case for the tool. I'm very curious whether any of you have ever used this tool and if so what for? Is there some niche area that I am overlooking, e.g. some operation involving multi-spectral images, in which this is a common operation?
Pickis the map algebra analog of a "case" or "switch" statement. Like them it is not indispensable but it can be convenient (and more efficient than deeply nested binary logical operators).
Notable among the uses to which I have put this operation is its ability to implement a cellular automaton.
Another handy use is random selection of rasters. A worked example appears on this site at Creating raster by randomly choosing cell value from multiple overlapping rasters?
most of the tool can be replaced with Map algebra, but the syntax of pick makes it easy to use.
I've used it for mosaicking with multiple masks (you have a set of classification, and you want to combine them based on geographical stratification.)
It is also quite usefull in combination of the local toolset. For example, one of those tools can find the position of the highest value in a set of raster. So if you add one to the result, you can pick the value after the maximum. This is quite usefull for time series analysis.
Chapter 1 Introduction to GIS
A Geographic Information System is a multi-component environment used to create, manage, visualize and analyze data and its spatial counterpart. It’s important to note that most datasets you will encounter in your lifetime can all be assigned a spatial location whether on the earth’s surface or within some arbitrary coordinate system (such as a soccer field or a gridded petri dish). So in essence, any dataset can be represented in a GIS: the question then becomes “does it need to be analyzed in a GIS environment?” The answer to this question depends on the purpose of the analysis. If, for example, we are interested in identifying the ten African countries with the highest conflict index scores for the 1966-78 period, a simple table listing those scores by country is all that is needed.
|MOROCCO||1861||CENTRAL AFRICAN REPUBLIC||618|
Data source: Anselin, L. and John O’Loughlin. 1992. Geography of international conflict and cooperation: spatial dependence and regional context in Africa. In The New Geopolitics, ed. M. Ward, pp. 39-75.
A simple sort on the Conflict column reveals that EGYPT, SUDAN, UGANDA, ZAIRE, TANZANIA, LIBYA, KENYA, SOMALIA, ETHIOPIA, SOUTH AFRICA are the top ten countries.
What if we are interested in knowing whether countries with a high conflict index score are geographically clustered, does the above table provide us with enough information to help answer this question? The answer, of course, is no. We need additional data pertaining to the geographic location and shape of each country. A map of the countries would be helpful.
Figure 1.1: Choropleth representation of African conflict index scores. Countries for which a score was not available are not mapped.
Maps are ubiquitous: available online and in various print medium. But we seldom ask how the boundaries of the map features are encoded in a computing environment? After all, if we expect software to assist us in the analysis, the spatial elements of our data should be readily accessible in a digital form. Spending a few minutes thinking through this question will make you realize that simple tables or spreadsheets are not up to this task. A more complex data storage mechanism is required. This is the core of a GIS environment: a spatial database that facilitates the storage and retrieval of data that define the spatial boundaries, lines or points of the entities we are studying. This may seem trivial, but without a spatial database, most spatial data exploration and analysis would not be possible!
1.1.1 GIS software
Many GIS software applications are available–both commercial and open source. Two popular applications are ArcGIS and QGIS.
A popular commercial GIS software is ArcGIS developed by ESRI (ESRI, pronounced ez-ree),was once a small land-use consulting firm which did not start developing GIS software until the mid 1970s. The ArcGIS desktop environment encompasses a suite of applications which include ArcMap, ArcCatalog, ArcScene and ArcGlobe. ArcGIS comes in three different license levels (basic, standard and advanced) and can be purchased with additional add-on packages. As such, a single license can range from a few thousand dollars to well over ten thousand dollars. In addition to software licensing costs, ArcGIS is only available for Windows operating systems so if your workplace is a Mac only environment, the purchase of a Windows PC would add to the expense.
A very capable open source (free) GIS software is QGIS. It encompasses most of the functionality included in ArcGIS. If you are looking for a GIS application for your Mac or Linux environment, QGIS is a wonderful choice given its multi-platform support. Built into the current versions of QGIS are functions from another open source software: GRASS. GRASS has been around since the 1980’s and has many advanced GIS data manipulation functions however, its use is not as intuitive as that of QGIS or ArcGIS (hence the preferred QGIS alternative).
"Geoprocessing is a GIS operation used to manipulate GIS data. A typical geoprocessing operation takes an input dataset, performs an operation on that dataset, and returns the result of the operation as an output dataset. Common geoprocessing operations include geographic feature overlay, feature selection and analysis, topology processing, raster processing, and data conversion. Geoprocessing allows for definition, management, and analysis of information used to form decisions".
Simply, geoprocessing involves employing an appropriate tool(s) to conduct a spatial analysis to create new information/data.
Below is a video that demonstrates how to use several geoprocessing tools to conduct a spatial analyisis.
A note about projections
Note that the tool has a parameter called Distance Method with two options: Planar and Geodesic.
Planar is set as default for performance, but Geodesic is recommended.
Planar distance is straight line Euclidean distance calculated in a 2D Cartesian coordinate system. Geodesic distance is calculated in a 3D spherical space as the distance across the curved surface of the world. While the tool defaults to Planar for backward compatibility and faster processing, Geodesic is the recommended option as it will always produce a more accurate result. A significant improvement is coming to ArcGIS Pro 2.8 that will further improve the accuracy and performance of geodesic operations.
If you need to use the planar method, use a projected coordinate system that will preserve distances. Generally, UTM projected coordinated system works well for a local region when performing distance-based calculations. Project all the layers into the appropriate projection prior to running the analysis. To learn more about projections, review the following resources.
Input raster representing the true or false result of the desired condition.
It can be of integer or floating point type.
The input whose values will be used as the output cell values if the condition is true.
It can be an integer or a floating point raster, or a constant value.
The input whose values will be used as the output cell values if the condition is false.
It can be an integer or a floating point raster, or a constant value.
A logical expression that determines which of the input cells are to be true or false.
The expression follows the general form of an SQL expression. An example of a where_clause is "VALUE > 100" .
The Kriging tool provides the following functions from which to choose for modeling the empirical semivariogram:
The selected model influences the prediction of the unknown values, particularly when the shape of the curve near the origin differs significantly. The steeper the curve near the origin, the more influence the closest neighbors will have on the prediction. As a result, the output surface will be less smooth. Each model is designed to fit different types of phenomena more accurately.
The diagrams below show two common models and identify how the functions differ:
A spherical model example
This model shows a progressive decrease of spatial autocorrelation (equivalently, an increase of semivariance) until some distance, beyond which autocorrelation is zero. The spherical model is one of the most commonly used models.
An exponential model example
This model is applied when spatial autocorrelation decreases exponentially with increasing distance. Here, the autocorrelation disappears completely only at an infinite distance. The exponential model is also a commonly used model. The choice of which model to use is based on the spatial autocorrelation of the data and on prior knowledge of the phenomenon.
Exponential model example
More of the mathematical models are illustrated below.
Raster Calculator tool dialog box example
The Raster Calculator tool is intended for use in the application only as a GP tool dialog box or in ModelBuilder. It is not intended for use in scripting and is not available in the ArcPy Spatial Analyst module.
The Raster Calculator tool allows you to create and execute a Map Algebra expression that will output a raster.
Use the Layers and variables list to select the datasets and variables to use in the expression. Numerical values and mathematical operators can be added to the expression by clicking the respective buttons in the tool dialog box. A list of commonly used conditional and mathematical tools is provided, allowing you to easily add them to the expression.
Full paths to data or data existing in the specified current workspace environment setting can be entered in quotes ( "" ). Numbers and scalars can be directly entered into an expression.
The operators in the Raster Calculator tool dialog box are:
Multiple geoprocessing tools and operations can be combined in a map algebra expression using standard Python syntax.
When typing tool names, be sure to check the tool name syntax. If the capitalization is incorrect, the expression will be invalid and fail to execute because Python is case sensitive.
An example of the general format of a Map Algebra expression using using geoprocessing tools is:
Con(IsNull("streams"), 0, "streams")
- You do not need to put the output raster name or the = sign in the expression because the output name is specified in the Output raster parameter.
- You do not need to cast input data as a Raster object when using operators.
You can easily clip a raster dataset by setting the extent environment and specifying the input raster name in the expression. When the tool is executed, the resulting raster output will be clipped based on the specified extent.
To create a raster with cells of a constant value, specify the appropriate extent and cell size environment settings and enter the numerical value into the expression.
When multiple operators are used in an expression, they are not necessarily executed in left-to-right order. The operator with the highest precedence value will be executed first. For more information on operator precedence, see operator precedence table. You can use parentheses to control the execution order.
, & , ^ , | ) operators have a higher precedence level than Relational ( < , <= , > , >= , == , != ) operators. Therefore, when Boolean operators are used in the same expression as Relational operators, the Boolean operators will be executed first. To change the order of execution, use parentheses.
When multiple Relational and/or Boolean operators are used consecutively in a single expression, in some cases it may fail to execute. To avoid this potential problem, use appropriate parentheses in the expression so that the execution order of the operators is explicitly defined. For more information, see Complex Statement Rules.
The performance of an operation may be enhanced by the deferred evaluation capabilities of Map Algebra. Deferred evaluation is an optimization technique where individual components of an expression are intelligently processed so as to minimize the creation of intermediate datasets on disk.
Only operators and tools that process on a per-cell basis can take advantage of this capability. Operators and tools that support deferred evaluation are included on the raster calculator tool itself either as a button or in the list of tools provided.
The Raster Calculator tool can be used in ModelBuilder, but keep the following points in mind:
- The syntax of the expression determines how variables are to be specified. If you select a variable from the variable list, it will be automatically added to the expression at the current cursor location. Upon model validation:
- In ModelBuilder, the following variable types are valid inputs for an expression:
- Numeric (Double and Long)
- Data (Raster dataset, raster layer, raster band, .lyr layer file)
- The Raster Calculator tool generally follows the standard connectivity behavior of models in ModelBuilder, with some exceptions resulting from the requirements of formulating a valid Map Algebra expression. These include the following:
- Variables are connected to the Raster Calculator tool when they are selected from the list of variables. All variables are automatically listed in the Variable list inside the tool.
- When a dataset or variable is used in the expression, a link between the variable and the tool will be created. If you remove the variable from the expression, the associated link between the variable and the tool will also be removed.
- If you delete the connection to a variable, the variable is not removed from the expression.
- You should not rename a variable that is connected to the tool, since the variable will not be renamed within the expression. If you do so, the expression will be invalid.
Certain Raster storage environments may apply to this tool when output is in a raster format other than Esri Grid.
- For the Raster Statistics environments, only the Calculate Statistics check box is supported.
- For the Compression environments, only the type of Compression is supported. This environment will only apply when the output would be of integer type. Compression is not supported on floating point rasters, so this environment will only apply when the output would be of integer type.
- The Tile Size environment is only supported for raster output in the following formats: TIFFs, file geodatabases, or an SDE geodatabase.
The Map Algebra expression you want to run.
The expression is composed by specifying the inputs, values, operators, and tools to use. You can type in the expression directly or use the controls to help you create it.
- The Rasters list identifies the datasets available to use in the Map Algebra expression.
- The Tools list provides a selection of commonly used tools to choose from.
The output raster resulting from the Map Algebra expression.
In Python, Map Algebra expressions should be created and executed with the Spatial Analyst module, which is an extension of the ArcPy Python site package.
See Map Algebra in Spatial Analyst to learn about how to perform your analysis in Python.
See Creating output from Map Algebra for information on producing output from Map Algebra expressions in Python.
Frequently Asked Questions (FAQ) about Geographic Information Systems (GIS)
What is GIS?
GIS stands for Geographic Information Systems. The main software package is called ArcGIS and is supplied by the Environmental Systems Research Institute (ESRI). A good place to start learning more about the potential applications of GIS in demography and the social sciences is ESRI’s home page. In addition, you can find a staggering amount of information about GIS on the Internet, including software training and tutorials, data products, statistical techniques, and myriad areas of application (crime, health, planning, etc.).
What are geospatial data?
Geospatial data store the geometric location of a particular feature or object (represented as either a point, a line, or an area), along with attribute information describing what those features represent. Geospatial data may come from a variety of sources—individual, household, or institutional surveys population censuses, land use surveys, human infrastructure databases (streets, buildings, business listings, tax records, etc.), satellite imagery, aerial photographs, and paper maps. It is commonly reported that over 80% of all data are geospatial—that is, these data can be linked to features or objects such as a street address, a census tract, a ZIP code, a school district, a police precinct, a health service area, a county, a state, and/or a country.
What format(s) would my data have to be in for the GIA Core to be able to use them to help me?
Formats readable by spreadsheet or statistical software (and, of course, ArcGIS shapefiles) are the most directly useful however, it may be possible to use other types of files. With ArcGIS, the preferred tabular data format is .dbf. If you have data for specified geographic areas (e.g., counties or census tracts) in Excel, ASCII, or other formats, such as SAS, the data can be easily converted to .dbf format, and can usually be integrated with other geospatial information quite easily.
Which geocoded or geospatial datasets are available to me through the GIA Core?
Census data and boundary files for various geographic levels are the most commonly used data. However, many other datasets are available. Please see the Spatial Data page, which contains a more detailed listing of available resources. Additional datasets are also available from SodaPop, although not all of their datasets have a geographic component. The Geographic Levels and PRI Data Archive page lists available datasets by geographical levels.
What U.S. Census–based geographic boundary files are available from PRI?
PRI currently has files at a variety of geographic levels : nation, region, division, state, county, tract, block group, place, ZIP code, MSA/CMSA, PMSA, urban area, congressional district, American Indian, ANRC, MCD, and block.
How do I go about finding available geospatial data or data on a specific country?
The best source is the DIVA-GIS website. Please contact the GIA team for further assistance.
When does it make sense to combine two geocoded datasets? How can I do this?
You can combine datasets when they contain (a) different areas with records at the same geographic level (e.g., appending records for Alaska and Hawaii to records for the 48 contiguous states to produce a complete dataset of all 50 U.S. states) or (b) additional data for existing records at the same geographic level (e.g., adding environmental data such as average temperature for a county to the census data for a county).
I have a hardcopy map of a specific country (or region). Can I convert this map for use in ArcGIS?
Yes, we can scan maps and images and then convert them into a format that ArcGIS can use. This process may take time, depending on the complexity of the original map and the clarity of the image.
Where can I store the data I extract for use in a GIS project?
You should store data in your personal webspace or space allocated for a project. Or you can store your data on removable media such as a flash drive. Geospatial datasets can be large, so please adopt sound data management practices. Also, it is advisable to back up all project data on a regular basis.
What changes has ESRI made with the release of ArcGIS 10.3?
ESRI has made a number of changes in the new version of ArcGIS Desktop. These changes include new spatial statistics tools, improvements in annotation and labeling, improved raster support, and new geoprocessing tools. A more detailed list of new features in ArcGIS 10.3 is available from ESRI.
What is a project (.mxd) file?
You can think of a project file as a database management tool. GIS users often must work on the same files for long periods of time, and the .mxd file is the management file that remembers which files you use, where they are located, the projection the file is stored in, the colors you have set, etc.
How do I edit a project (.mxd) file? Why would I need to edit a project file?
Once you open your project file, editing is simple. The last saved version of your files will appear on your computer screen, and you can manipulate the existing settings to update the file. (Always remember to save it!) Project files must be edited in order for changes to be reflected in your work. You’ll often work on GIS projects over long periods of time, and as you work, new data, shapefiles, etc. can be added to keep your project current.
What is a shapefile?
Shapefiles are computer files that produce images for users in a GIS environment. There are three basic types of shapefiles:
- Polygons (area-based features such as census tracts and block groups)
- Points (address locations)
- Lines (roads, rivers, etc.)
Several affiliated files must be together for a GIS system to use and present the data. Some common file types are .shp, .shx, .sbn, and .dbf. When you are downloading shapefiles from data sources, there may be other affiliated files make sure you have all files with the same name in the same directory when using them.
How can I address-match data in ArcGIS? What datasets do I need? What are some of the most common problems in address matching?
Address matching in ArcGIS is simple once you have worked with this type of data a few times. To address-match, you need a .dbf file of address information that includes the street address and preferably the ZIP code of your locations of interest. It is possible to address-match with just a street address, but in large urban areas a common street name (e.g., Main Street or 17th) may appear across several ZIP codes, making the address-matching process a more difficult and time-consuming task. As with so many tasks, address-matching accuracy is only as good as the skills of the person doing the work.
How do I prepare spatial data for use with ArcGIS?
If you have data with spatial locators (e.g., census FIPS codes, ZIP codes, street addresses), you can link your data to shapefiles within a GIS environment. You also can join tables in a GIS project.
What is an ArcGIS extension? What extensions are available?
ESRI makes additional ArcGIS functionality available through the use of extensions. Extensions allow you to add specific features to the software as necessary and then unload this functionality when you no longer need it. The full list of available extensions, with descriptions of each, is available here.
What are ArcMap, ArcCatalog, and ArcToolbox?
ArcMap is the application in ArcGIS that you use to create, visualize, and manipulate maps. ArcCatalog is the application in ArcGIS for browsing and organizing data files. ArcToolbox is the application in ArcGIS that contains data management and conversion tools and wizards (e.g., projection tools).
What are some of the functions of the Spatial Analyst extension?
The Spatial Analyst extension can help you find slopes and directions, least-cost paths, and best locations for a new facility based on multiple criteria. This is useful for analyzing raster data as well as feature data.
What are some of the functions of the Geostatistical Analyst extension?
The Geostatistical Analyst extension can help you with the production of statistical surfaces for exploratory spatial data analysis, structural analysis (calculating properties of neighboring areas), and surface prediction and results analysis.
Is there any way to change or expand existing ArcGIS functionality?
ESRI has a number of scripts and other downloads available. Scripts work with the existing software to extend functionality or to automate common tasks. The scripts on ESRI’s site come from ESRI itself and from other ArcGIS users you can even upload your own scripts for other people to use.
What is an .E00 file? How can I convert this for use in ArcGIS?
An .E00 file is an ArcInfo interchange file that contains spatial feature information and attribute data in a fixed-length ASCII format. You use these files for the system-independent exchange of GIS coverages and associated data. To work with data in this format in ArcGIS, you must first convert it using ArcToolbox : In the ArcToolbox window, click the plus signs next to Coverage Tools, Conversion, and To Coverage, respectively. Next, double-click “Import from E00”. Under “Input Interchange File,” navigate to the file want to convert. Specify the output dataset and click OK. The resulting output file(s) will be compatible with ArcGIS. News Honor a Colleague Letter for Suet-ling Pong Steffensmeier named Liberal Arts Research Professor Postdoctoral Training position in Family Demography and Individual Development: NOW ACCEPTING APPLICATIONS Spatial demography publication explores new practices Life as an immigrant inspires College of Education faculty member's book Really Big Data Penn State symposium aims to reconstruct African-American narratives Van Hook Wins Raymond Lombra Award De Jong receives the College of the Liberal Arts' Emeritus Distinction Award Penn State to host national symposium on African-American family issues
- Layer names will be enclosed in double-quote symbols ( " " ).
For example, inraster when selected from the variable list will become "%inraster%" in the expression.