Viewing pipeline files and runs
For an uploaded file pipeline, the Files page contains the list of pipeline files.
For Amazon S3 and Databricks pipelines, you use the pipeline details Overview page to track processed files and pipeline runs. For pipelines that are configured to also redact files, you can configure the redaction for the detected data types. For more information, go to Selecting the handling option for the entity types.
Viewing the list of all files for a pipeline
For uploaded file pipelines, all file management occurs on the Files page. The Files page only contains the file list. When you add a file to the pipeline, it is automatically added to the file list.
For Amazon S3 and Databricks pipelines, the file list is on the Files tab of the Overview page. The list is not populated until you run the pipeline. The list only contains processed files.
Viewing the list of pipeline runs
On the Overview page for an Amazon S3 or Databricks pipeline, the Pipeline Runs tab displays the list of pipeline runs.
For each run, the list includes:
Run identifier
When the run was started
The current status of the pipeline run. The possible statuses are:
Queued - The pipeline run has not started to run yet.
Running - The pipeline run is in progress.
Completed - The pipeline run completed successfully.
Failed - The pipeline run failed.
Viewing the list of pipeline run files
For a pipeline run, to display the list of files that the pipeline run includes, click View Run.
Information in a file list
For each file, the list includes the following information:
File name
For Amazon S3 or Databricks files, the path to the file
The status of the file processing. The possible status are:
Unprocessed - The file is added but a pipeline run to process it has not been started. This only applies to uploaded files that were added since the most recent pipeline run.
Queued - A pipeline run was started but the file is not yet processed.
Running - The file is being processed.
Completed - The file was processed successfully.
Failed - The file could not be processed.
Last updated
