Intermediate data and the scratch workspace


When you run a model, output data is created for each process in the model. Some of the data created is of no use after the model is run since it was only created to connect to another process that creates new output. Such data is called intermediate data. To determine if any data variable contains intermediate data, right-click the data variable—if there is a check mark next to Intermediate, then the data is considered intermediate. If there is no check mark, the data is not intermediate and will not be deleted.

Intermediate data will only be deleted when:

Intermediate data will not be deleted if you run the model from the ModelBuilder window. This allows the has-been-run state of the model to be saved between sessions (which means that each time you open a previously run model in ModelBuilder, you will not need to rerun the entire model).

In ModelBuilder, all derived data elements are automatically flagged as intermediate.

If you want to delete intermediate data after running a model in the ModelBuilder window, click the Model menu and click Delete Intermediate Data.

Delete intermediate data

Not all derived data is automatically flagged as intermediate. A data variable that is In/Out cannot be flagged as intermediate, since the input data exists.

Learn more about in/out derived data

NOTE: By default, ModelBuilder assumes that all derived data (with the exception of In/Out derived data) is considered intermediate, even derived data that is the final result of a model, as shown in the figure below.

Model output marked as derived

It's rare that you would want your final output to be deleted, because it means the model has no permanent result. In most cases, these final variables are marked as model parameters, and model parameters cannot be intermediate data.

Learn more about model parameters

If you want to preserve intermediate data, you have three choices:

Auto-generated locations

When you open a tool from ArcToolbox or inside ModelBuilder and provide the necessary input datasets, the location (the workspace pathname followed by the dataset name) of the derived output data is automatically generated. This auto-generated name is constructed using the following logic:

The output data variable will contain the auto-generated name regardless of whether the variable eventually becomes intermediate data, managed data, or a tool parameter.

Learn more about auto-generated names

Setting the scratch workspace

Here are some things you should be aware of regarding writing intermediate scratch data:

NOTE: It is recommended that you use a file geodatabase for your scratch workspace.

The in_memory workspace

You can write intermediate data to your computer's memory rather than to a workspace on disk. Writing intermediate data to memory is extremely fast compared to writing to disk.

NOTE: Only tables and feature classes (points, lines, polygons) can be written to the in_memory workspace. The in_memory workspace does not support extended geodatabase elements such as subtypes, domains, representations, topologies, geometric networks and network datasets. Only simple features and tables can be written.

To write to your computer's memory, use the pathname in_memory, as follows:

          in_memory\results

You can think of in_memory as a special geodatabase workspace. You can use in_memory to write intermediate data.

NOTE: Do not set your scratch workspace environment to the in_memory workspace. Use in_memory only for outputs that you know to be simple features and tables.

The example below shows using in_memory with the Frequency tool.

Using in_memory workspace

Rules about using in_memory workspace are

Managing the in_memory workspace

When you write data to the in_memory workspace, you are consuming your computer's internal memory, and if you write too much data, you will consume the memory an application (such as ArcMap) needs to execute efficiently. Eventually, the application will slow down, causing all tasks (such as refreshing the map) to become very slow.

At some point, the advantages of using the in_memory workspace are offset by the slow-down of the application. When this point is reached depends on many factors, such as the amount of available internal memory, the amount of virtual memory (swap space), the amount of data to be written to the in_memory workspace, and the number of applications open on your computer. In general, you should avoid using in_memory for large datasets and use a workspace on disk instead.

When using the in_memory workspace, you should delete your intermediate data in ModelBuilder when no longer needed so you free up memory.

Learn how to delete intermediate data

You can also use the Delete tool to delete data in the in_memory workspace. You can delete individual tables and feature classes, or you can delete the entire in_memory workspace, which will delete all or its tables and feature classes. An empty in_memory workspace will be recreated after you've deleted it.

In_memory source location

A table or feature class based on data written to the in_memory workspace will have the source location of GPInMemoryWorkspace, as shown in the following illustration from ArcMap:

GPInMemoryWorkspace

The long string of characters enclosed in brackets {} following GPInMemoryWorkspace is a unique identifier created and used by ArcGIS.

See Also