RivEX

Summary :: Usage :: Parameters :: Outputs :: Warnings :: Limitations :: Code sample

Summary

Finds the nearest upstream site(s). Output is written to a table with optional route line generation.

Usage

Requirement for using this tool:

To use this tool you must have attributed the network with:

• Source ID
• Catchment ID
• Distance to Mouth

This tool creates a new stand alone table recording upstream sites found near each input site. The default behaviour of this tool is to return for each input site a single nearest upstream site. Under the optional processing section of the tool this behaviour can be switched to finding all nearest reachable sites. In this mode once a site is found this blocks access to all sites upstream of the reached site, RivEX continues to search up other tributaries until the source is found or another site (which will block further upstream access).  This tool is ideal in returning distances to upstream barriers. If you wish to visualize the fragmentation of a river network caused by barriers consider using the Create subnetworks from sites tool.

To understand the influence the Return all nearest reachable sites has, review the simple examples given below.

The river network needs to be in a projected coordinated system.

Sites that have no sites upstream are coded with a -1. Therefore all sites are returned in the output as either -1 or with a near site ID. During processing sites are sorted into catchments based upon the catchment ID encoded into the network. Sites are only linked within the same catchment ID, this avoids the tool reporting sites that may exist in another catchment which are linked by a bifurcation.

Before RivEX searches the sites, it checks if all sites have unique ID's and aborts processing if duplicates are found. It will also look for stacked points and warns you if any are found. Stacked points will cause duplication in the output. You might want this if say the site has a date field recording multiple surveys at a single location, you could filter these out by using the tools filter.

If a selection exists on the river network this is cleared.  Input sites must be snapped to the network and have a unique numeric ID (which is not the Object ID field). You can choose to process a subset of sites if they have been selected and you have ticked on the appropriate parameter. If the sites to search is also the upstream sites to find layer then any existing selection is cleared and all sites are processed, this is because they are the same layer.  

Errors that occur which are logged but do not stop processing

Only snapped sites are processed, un-snapped sites are reported in the error log tables (tblFailedNearSitesFrom or tblFailedNearSitesTo) that can be optionally loaded into the map for convenience.

Sites snapped to tributary junctions (nodes) are rejected and reported in the error log tables.

RivEX uses distance from network mouth for determining if a site is upstream and computing the route's length. RivEX can encode into the network illogical scenarios of distance to network mouth caused by illogical connectivity in the river network. This means as RivEX traverses in an upstream direction it encounters sites on segments of river that can have a nearer distance to network mouth. These illogical situations are captured by RivEX and reported in the tblFailedNearSitesFrom error log table.

Optional Processing

• Ignore self when searching within layer - This option is only valid when the Site to search layer is the same as the Nearest site to find layer, i.e. you are searching WITHIN a single layer. If this is unchecked then the first nearest upstream site found is itself and RivEX returns this, with a zero travel distance.

• Constrain nearest search to source ID - Constrains the search to return only upstream sites on the same source ID as the site to search.  You can think of this as only searching for sites on the route to source, ignoring any minor tributaries.  This constraint can generate sites that report they have no upstream sites, thus code as -1. This can happen when none of the upstream sites are actually on the source ID.  Why constrain by source ID when searching in a upstream direction?  This essentially stops you associating your site with upstream sites that are on other smaller catchments.

With constrain to source ID checked on, green 15 links to brown 2,

even though brown 15 is nearer, brown 15 is on a different source ID.

• Return all nearest reachable sites - The default behaviour of this tool is to find the nearest upstream site. Tick this option ON and RivEX will attempt to find all reachable sites.  Once a site is found this blocks access to all sites upstream of the reached site, RivEX continues to search up other tributaries until the source is found or another site (which will block further upstream access).  If no upstream sites are found then the input site records -1 in the output table indicated no sites were found.

• Build a route to site - This optional task will cause RivEX to build a single polyline from input site to the nearest upstream site, you can use this polyline for simply visualising your output or in further spatial analysis. The name of the output Feature Class is automatically built by RivEX using the input site layers.

• You can further filter data by the attributes within the upstream site layer to search.  You can build a SQL expression to filter your upstream sites.  For example you may be only interested in upstream sites that occur on Strahler order 3 or greater. If you set this then only sites that fulfil the selection criteria are considered when RivEX traverses the network.  To use this parameter the attributes you want to filter sites by must be encoded into the upstream site layer.  You can use the RivEX tool Transfer network metrics to pass values encoded in the network into your site layer. Creating a SQL expression that does not select any sites will cause RivEX to default to processing all sites.

• Add an attribute index to output table - Will cause RivEX to add an index to the site ID field in the output table.  An index can significantly speed up query operations such as joins or relates. It does not index the site ID field in the optional route feature class.

Parameters

Outputs

The output table can be given a name but you have no control over where it is stored, this would be in the file geodatabase ..\RivEX_Workspace\Outputs\fGDB_Sites.gdb

Distance values are in the units of the river network. 

The output table contains the following fields:

If you have selected the optional processing task, add attribute index, then the SiteID field in the output dataset will be indexed. An index can significantly speed up query operations such as joins or relates.

If you have checked on the optional parameter Build a route to site, RivEX will create a polyline feature class storing the route from site to nearest upstream site. This output is written to the file geodatabase 

..\RivEX_Workspace\Outputs\fGDB_Sites.gdb using the naming convention of  X_Y_NearestUSRoute where X is site to search and Y is nearest site to find layer names. The Feature Class has the following fields:

If errors were generated then this tool can create two stand alone tables named tblFailedNearSitesFrom and tblFailedNearSitesTo.  The tables are created in the RivEX workspace folder and stored in the File GeoDatabase found in ..\RivEX_Workspace\ErrorLogs\fGDB_RivEXErrorLogs.gdb

Advice on the structure of the error log table and how it can be used is found within this section of the manual. Take note of the warning if you plan to construct a model using the error log output in subsequent processing.

Warnings

Constraining search to the same source ID or applying a filter can easily introduce a scenario where RivEX fails to return any matched upstream sites when there are clearly lots of sites upstream. In this case you have constrained the data to such an extent that no site passes the logical test and RivEX tags the site as -1 as not having an upstream site.

A limitation of this tool run in a script is that it only accepts feature classes for the site layer. It will fail with inputs as layers with/without selections. If you need to process data with selections you need to use the tool directly from the toolbox or python console, more details here.

With the release of ArcGIS Pro 3.2.1 a switch control appears on parameters that accept tables\feature classes, do not interact with it! More advice here.

Limitations

1. Sites snapped to tributary junctions (nodes) are rejected and reported in the error log tables.
2. Sites must be a simple point feature class, multi-point data is not allowed.
3. Sites are not allowed to have fields that have the same name as RivEX generated fields that occur in the network. Solution is to remove or rename fields in the site layers.
4. Site layers with table joins are rejected, solution is to remove join before using tool.

Code sample

A minimum code example showing how to call this tool in a python script. This can be run in console or your preferred IDE. If you right click on the tool in the RivEX toolbox and select properties you can view parameter order.

See warning above.

import arcpy

# Import RivEX toolbox
arcpy.ImportToolbox(r"C:\RivEX_ArcPro\RivEX.atbx")

try:
    # Input river Feature Class
    fcRivers = r"C:\Scratch\ORN\data.gdb\ORN"

    # Input search sites Feature Class
    fcSites = r"C:\Scratch\ORN\RivEX_Workspace\Outputs\fGDB_Sampling.gdb\fcSample_10K_spaced"

    # Input upstream sites Feature Class
    fcUSSites = r"C:\Scratch\ORN\RivEX_Workspace\Outputs\fGDB_Sampling.gdb\reg23"

    # Run RivEX tool, find nearest upstream site, with a Shreve order greater than 20, do not constrain to source ID, build route or add attribute index
    # WARNING This tool will fail if river data is a layer with a selection! If you need to pass in selections use tool directly from toolbox    
    res = arcpy.scrFindNearestUSSites_RivEX(fcRivers, fcSites, "SampleID", False, fcUSSites, "SampleID", "tblNearestUS", True, False, True, False, "Shreve > 20", False, False, False)

    # The nearest upstream sites tables, a derived output
    tblNearest = res.getOutput(0)

    # The routes feature class
    fcRoutes = res.getOutput(1)    

    # The error table
    tblErrorFROM = res.getOutput(2)

    # The error table
    tblErrorTO = res.getOutput(3)

    # Count number of sites returned by search, note all sites are returned but a -1 means it failed the search, so need to exclude these
    n = 0
    with arcpy.da.SearchCursor(tblNearest, "SiteID", "NearUSSite_ID <> -1") as cursor:
        for row in cursor:
            n += 1
    print("Number of sites where nearest site is on a channel with Shreve order greater than 20 = " + str(n))
except arcpy.ExecuteError:
    print("FAILED searching!")