Summary :: Usage :: Parameters :: Outputs :: Warnings :: Limitations :: Code sample

Summary


Finds the nearest upstream site(s). Output is written to a table with optional route line generation.

Usage


Requirement for using this tool:


To use this tool you must have attributed the network with:



This tool creates a new stand alone table recording upstream sites found near each input site. The default behaviour of this tool is to return for each input site a single nearest upstream site. Under the optional processing section of the tool this behaviour can be switched to finding all nearest reachable sites. In this mode once a site is found this blocks access to all sites upstream of the reached site, RivEX continues to search up other tributaries until the source is found or another site (which will block further upstream access).  This tool is ideal in returning distances to upstream barriers. If you wish to visualize the fragmentation of a river network caused by barriers consider using the Create subnetworks from sites tool.


To understand the influence the Return all nearest reachable sites has, review the simple examples given below.


Return all nearest reachable sites unchecked (default)

Return all nearest reachable sites checked on

Example data - Find nearest upstream brown sites for each green site

Scenario for finding nearest upstream sites

Example data - Find all nearest upstream brown sites for each green site

Scenario for finding nearest upstream sites

Output table returns relationships, note brown site 34 not in table as 2 is nearest and blocks upstream searching


Site ID

Near Upstream Site ID

Length (m)

13

2

1068

15

15

2064

Output table returns relationships, note brown site 34 not in table as 2 is nearest and blocks upstream searching


Site ID

Near Upstream Site ID

Length (m)

13

2

1068

15

2

3288

15

15

2064


Build a route to site checked on creates the dotted polylines that can be used in visualization or further spatial analysis

Routes built

Build a route to site checked on creates the dotted polylines that can be used in visualization or further spatial analysis

Routes built


The river network needs to be in a projected coordinated system.


Sites that have no sites upstream are coded with a -1. Therefore all sites are returned in the output as either -1 or with a near site ID. During processing sites are sorted into catchments based upon the catchment ID encoded into the network. Sites are only linked within the same catchment ID, this avoids the tool reporting sites that may exist in another catchment which are linked by a bifurcation.


Before RivEX searches the sites, it checks if all sites have unique ID's and aborts processing if duplicates are found. It will also look for stacked points and warns you if any are found. Stacked points will cause duplication in the output. You might want this if say the site has a date field recording multiple surveys at a single location, you could filter these out by using the tools filter.


If a selection exists on the river network this is cleared.  Input sites must be snapped to the network and have a unique numeric ID (which is not the Object ID field). You can choose to process a subset of sites if they have been selected and you have ticked on the appropriate parameter. If the sites to search is also the upstream sites to find layer then any existing selection is cleared and all sites are processed, this is because they are the same layer.  


Errors that occur which are logged but do not stop processing


Only snapped sites are processed, un-snapped sites are reported in the error log tables (tblFailedNearSitesFrom or tblFailedNearSitesTo) that can be optionally loaded into the map for convenience.


Sites snapped to tributary junctions (nodes) are rejected and reported in the error log tables.


RivEX uses distance from network mouth for determining if a site is upstream and computing the route's length. RivEX can encode into the network illogical scenarios of distance to network mouth caused by illogical connectivity in the river network. This means as RivEX traverses in an upstream direction it encounters sites on segments of river that can have a nearer distance to network mouth. These illogical situations are captured by RivEX and reported in the tblFailedNearSitesFrom error log table.


Optional Processing


  • Ignore self when searching within layer - This option is only valid when the Site to search layer is the same as the Nearest site to find layer, i.e. you are searching WITHIN a single layer. If this is unchecked then the first nearest upstream site found is itself and RivEX returns this, with a zero travel distance.


  • Constrain nearest search to source ID - Constrains the search to return only upstream sites on the same source ID as the site to search.  You can think of this as only searching for sites on the route to source, ignoring any minor tributaries.  This constraint can generate sites that report they have no upstream sites, thus code as -1. This can happen when none of the upstream sites are actually on the source ID.  Why constrain by source ID when searching in a upstream direction?  This essentially stops you associating your site with upstream sites that are on other smaller catchments.


Constrain to source ID

With constrain to source ID checked on, green 15 links to brown 2,

even though brown 15 is nearer, brown 15 is on a different source ID.


  • Return all nearest reachable sites - The default behaviour of this tool is to find the nearest upstream site. Tick this option ON and RivEX will attempt to find all reachable sites.  Once a site is found this blocks access to all sites upstream of the reached site, RivEX continues to search up other tributaries until the source is found or another site (which will block further upstream access).  If no upstream sites are found then the input site records -1 in the output table indicated no sites were found.


  • Build a route to site - This optional task will cause RivEX to build a single polyline from input site to the nearest upstream site, you can use this polyline for simply visualising your output or in further spatial analysis. The name of the output Feature Class is automatically built by RivEX using the input site layers.


  • You can further filter data by the attributes within the upstream site layer to search.  You can build a SQL expression to filter your upstream sites.  For example you may be only interested in upstream sites that occur on Strahler order 3 or greater. If you set this then only sites that fulfil the selection criteria are considered when RivEX traverses the network.  To use this parameter the attributes you want to filter sites by must be encoded into the upstream site layer.  You can use the RivEX tool Transfer network metrics to pass values encoded in the network into your site layer. Creating a SQL expression that does not select any sites will cause RivEX to default to processing all sites.


  • Add an attribute index to output table - Will cause RivEX to add an index to the site ID field in the output table.  An index can significantly speed up query operations such as joins or relates. It does not index the site ID field in the optional route feature class.



Parameters


Name

Help

Data type

River Network

The river network. This needs to be encoded with the RivEX fields catchment ID and Distance to mouth. For best results the network should be within a File GeoDatabase

Feature Layer

Sites to search

A point layer for which you want to find the the sites downstream of, think of this as the FROM layer. This layer must conform to these specifications:


  • Points are snapped to the network, if not, you can use the RivEX snap site tool.
  • Layer must be in the same coordinate system as the river network.
  • Layer must not have a table join.
  • Contain a numeric (integer) field, uniquely identifying the sites.
  • Geometry is a simple point, Multipoint geometry is not allowed.


Note this layer can also be the downstream sites to find layer.


Feature Layer

Site ID

This is a numeric (integer) field that uniquely identifying the sites.


Text, Object ID or FID fields are not accepted.

Field

Process only selected sites

Indicates if only selected sites are to be processed.


  • CHECKED - Only sites that are selected will be processed.
  • UNCHECKED (default) - All sites will be processed, any existing selection will be cleared before processing.


Note if your downstream sites to find layer is also your sites to search layer then if selections exist and you have checked this box, the selection is cleared and all sites are processed. This is because both inputs are the same layer.

Boolean

Nearest sites to find

A point layer with sites snapped to the network. These are the sites to search for in an upstream direction. This layer could also be the sites to search from.

Feature Layer

Nearest Site ID

This is a numeric (integer) field that uniquely identifying the sites.


Text, Object ID or FID fields are not accepted.

Field

Output table name

The output Table name. RivEX auto-populates this field using the input site layer name and attaches "_NearestUSSite" to the end of it. You may need to edit it if the site layer name is not file geodatabase compliant.


This must be a valid file geodatabase table name, so cannot have spaces, unusual characters or start with a number.

You have no control over its location, it will be stored output in ..\RivEX_Workspace\Outputs\fGDB_Sites.gdb

String

Ignore searching when

This option is only valid when the Site to search layer is the same as the Nearest site to find layer, i.e. you are searching WITHIN a single layer.


  • CHECKED (default) - RivEX will ignore the site in the Nearest site to find layer which is the same as the starting site in the Site to search layer.
  • UNCHECKED  - RivEX search will consider all valid sites.


WARNING - If this is unchecked then the first nearest upstream site found is itself and RivEX returns this, with a zero travel distance.


Boolean

Constrain nearest search to source ID

This option will cause RivEX to constrain the search for sites to the source ID that is the same as the site in the Sites to search layer.


  • CHECKED - RivEX will only return sites that are on the same Source ID as the site in the Sites to search layer.
  • UNCHECKED (default) - RivEX will consider all upstream sites.


WARNING - This constraint can cause RivEX to return the default -1 value that indicates there were no upstream sites.  You may have many upstream sites in your data, they were simply not on the same source ID!

Boolean

Return all nearest reachable sites

The default behaviour of this tool is to find the nearest upstream site. Tick this option ON and RivEX will attempt to find all reachable sites.  Once a site is found this blocks access to all sites upstream of the reached site, RivEX continues to search up other tributaries until the source is found or another site (which will block further upstream access).  If no upstream sites are found then the input site records -1 in the output table indicated no sites were found.


  • CHECKED - RivEX will return all reachable sites.
  • UNCHECKED (default) - RivEX will return the nearest site only.


WARNING 1 - If your Nearest site to find layer contains stacked points then even though you might have asked to find a nearest site (this option UNCHECKED) RivEX would return multiple sites as they are all intersecting at the same location (they are stacked on top of each other).


WARNING 2 - If your Nearest site to find layer has sites on one side of a loop and you have asked RivEX to find all reachable sites (this option CHECKED) then its quite possible for RivEX to find sites upstream of a site which you might think should have blocked access to sites upstream of itself. They have been found by RivEX because it has legitimately travelled up the other side of the loop.

Boolean

Build a route

This option will cause RivEX to create a polyline Feature Class and store the route from site to nearest upstream site. This is useful for visualization or further spatial analysis. This optional task does take time so please be patient as RivEX builds the geometries.


  • CHECKED - RivEX will create a polyline for each nearest upstream site found
  • UNCHECKED (default) - RivEX ignores this optional task and builds only the main results table


You have no control over its location, it will be stored in the output output geodatabase

..\RivEX_Workspace\Outputs\fGDB_Sites.gdb with a name of  X_Y_NearestUSRoute where X is site to search and Y is nearest site to find.

Boolean

Filter near sites by attribute

Build a SQL expression to filter your upstream sites.  For example you may be only interested in upstream sites that occur on Strahler order 3 or less.


If you set this then only sites that fulfil the selection criteria are considered when RivEX traverses the network.  


To use this parameter the attributes you want to filter upstream sites by must be encoded into the upstream site layer.  You can use the RivEX tool Transfer network metrics to pass values encoded in the network into your site layer.


Creating a SQL expression that does not select any sites will cause RivEX to default to processing all sites.

SQl expression

Add an attribute index to output

Indicates if RivEX will create an attribute index for the site ID field in the output table


  • CHECKED - RivEX will attempt to add an index to the site ID field in the output table.
  • UNCHECKED (default) - No index will be added to the output table.


If you plan to join or relate data then it is STRONGLY recommended you create attribute indices for the outputs; an index can significantly speed up query operations.

Boolean

Add Error Log Table to Map

If this is ticked (default) the error log table will be added to the map for your convenience if any errors were found.

Boolean

Build a Relate for Error Log

If this tick box is checked then the error log table must be loaded into the map and RivEX builds a standard relate between the Error Log table and the river network layer. You can use the relate to help you jump to and review the error.

Boolean

Outputs


The output table can be given a name but you have no control over where it is stored, this would be in the file geodatabase ..\RivEX_Workspace\Outputs\fGDB_Sites.gdb

Distance values are in the units of the river network. 


The output table contains the following fields:

Field

Description

SiteID

The ID of the site in the Sites to search layer

Site_Cat

The catchment ID the site is within

Site_D2M

The distance the site is from network mouth

NearUSSite_ID

The site ID found for a nearest upstream site in the Nearest site to find layer

NearSite_D2M

The distance the nearest upstream site is from the network mouth

NearSite_Dis

The distance the site is from the nearest upstream site


If you have selected the optional processing task, add attribute index, then the SiteID field in the output dataset will be indexed. An index can significantly speed up query operations such as joins or relates.


If you have checked on the optional parameter Build a route to site, RivEX will create a polyline feature class storing the route from site to nearest upstream site. This output is written to the file geodatabase 

..\RivEX_Workspace\Outputs\fGDB_Sites.gdb using the naming convention of  X_Y_NearestUSRoute where X is site to search and Y is nearest site to find layer names. The Feature Class has the following fields:

Field

Description

Shape_Length

The length of the polyline created, this will be in the units of network (typically metres or feet)

SiteID

The site ID, think of this as the FROM site

NearUSSite_ID

The site ID of the nearest upstream site found, think of this as the TO site


If errors were generated then this tool can create two stand alone tables named tblFailedNearSitesFrom and tblFailedNearSitesTo.  The tables are created in the RivEX workspace folder and stored in the File GeoDatabase found in ..\RivEX_Workspace\ErrorLogs\fGDB_RivEXErrorLogs.gdb


Advice on the structure of the error log table and how it can be used is found within this section of the manual. Take note of the warning if you plan to construct a model using the error log output in subsequent processing.

WarningWarnings

Constraining search to the same source ID or applying a filter can easily introduce a scenario where RivEX fails to return any matched upstream sites when there are clearly lots of sites upstream. In this case you have constrained the data to such an extent that no site passes the logical test and RivEX tags the site as -1 as not having an upstream site.


A limitation of this tool run in a script is that it only accepts feature classes for the site layer. It will fail with inputs as layers with/without selections. If you need to process data with selections you need to use the tool directly from the toolbox or python console, more details here.


With the release of ArcGIS Pro 3.2.1 a switch control appears on parameters that accept tables\feature classes, do not interact with it! More advice here.

Limitations


  1. Sites snapped to tributary junctions (nodes) are rejected and reported in the error log tables.
  2. Sites must be a simple point feature class, multi-point data is not allowed.
  3. Sites are not allowed to have fields that have the same name as RivEX generated fields that occur in the network. Solution is to remove or rename fields in the site layers.
  4. Site layers with table joins are rejected, solution is to remove join before using tool.


Code sample


A minimum code example showing how to call this tool in a python script. This can be run in console or your preferred IDE. If you right click on the tool in the RivEX toolbox and select properties you can view parameter order.


See warning above.


import arcpy

# Import RivEX toolbox
arcpy.ImportToolbox(r"C:\RivEX_ArcPro\RivEX.atbx")

try:
    # Input river Feature Class
    fcRivers = r"C:\Scratch\ORN\data.gdb\ORN"

    # Input search sites Feature Class
    fcSites = r"C:\Scratch\ORN\RivEX_Workspace\Outputs\fGDB_Sampling.gdb\fcSample_10K_spaced"

    # Input upstream sites Feature Class
    fcUSSites = r"C:\Scratch\ORN\RivEX_Workspace\Outputs\fGDB_Sampling.gdb\reg23"

    # Run RivEX tool, find nearest upstream site, with a Shreve order greater than 20, do not constrain to source ID, build route or add attribute index
    # WARNING This tool will fail if river data is a layer with a selection! If you need to pass in selections use tool directly from toolbox    
    res = arcpy.scrFindNearestUSSites_RivEX(fcRivers, fcSites, "SampleID", False, fcUSSites, "SampleID", "tblNearestUS", True, False, True, False, "Shreve > 20", False, False, False)

    # The nearest upstream sites tables, a derived output
    tblNearest = res.getOutput(0)

    # The routes feature class
    fcRoutes = res.getOutput(1)    

    # The error table
    tblErrorFROM = res.getOutput(2)

    # The error table
    tblErrorTO = res.getOutput(3)

    # Count number of sites returned by search, note all sites are returned but a -1 means it failed the search, so need to exclude these
    n = 0
    with arcpy.da.SearchCursor(tblNearest, "SiteID", "NearUSSite_ID <> -1") as cursor:
        for row in cursor:
            n += 1
    print("Number of sites where nearest site is on a channel with Shreve order greater than 20 = " + str(n))
except arcpy.ExecuteError:
    print("FAILED searching!")

Return to top of page