Manage dataset files

Use the Tonic Textual REST API to manage dataset files.

Upload dataset files

post

Upload a file to a dataset for processing.

Path parameters

datasetIdstringRequired

The unique identifier of the dataset

Body

multipart/form-data

documentstringOptional

filestringOptional

File to upload

Responses

200

Returns uploaded file details

application/jsontext/plaintext/json

Response model returned after uploading a file to a dataset.

uploadedFileIdstringOptional

The identifier of the newly uploaded file.

400

Invalid input, unsupported file type, or upload failed

application/jsontext/plaintext/json

409

Duplicate file already uploaded

application/jsontext/plaintext/json

post/api/Dataset/{datasetId}/files/upload

HTTPcURLJavaScriptPython

POST /api/Dataset/{datasetId}/files/upload HTTP/1.1
Content-Type: multipart/form-data
Accept: */*
Content-Length: 33

{
  "document": "text",
  "file": "text"
}

{
  "updatedDataset": {
    "id": "text",
    "name": "text",
    "generatorMetadata": {},
    "outputFormat": "Original",
    "generatorSetup": "{\"NAME_GIVEN\":\"Redaction\", \"NAME_FAMILY\":\"Redaction\"}",
    "labelBlockLists": "{\"NAME_FAMILY\":{\"regexes\":[\".*\\\\s(disease|syndrom|disorder)\"]}}",
    "labelAllowLists": "{\"HEALTHCARE_ID\":{\"regexes\":[\"[a-z]{2}\\\\d{9}\"]}}",
    "tags": [
      "text"
    ],
    "files": [
      {
        "fileId": "text",
        "fileName": "text",
        "fileType": "text",
        "datasetId": "text",
        "numRows": 1,
        "numColumns": 1,
        "piiTypes": [
          "text"
        ],
        "wordCount": 1,
        "redactedWordCount": 1,
        "uploadedTimestamp": {},
        "fileSource": "Local",
        "processingStatus": "text",
        "processingError": "text",
        "mostRecentCompletedJobId": "text",
        "fileParseResultId": "text",
        "filePath": "text",
        "generatedFileStatus": "text"
      }
    ],
    "lastUpdated": {},
    "created": {},
    "creatorUser": {
      "id": "text",
      "userName": "text",
      "firstName": "text",
      "lastName": "text"
    },
    "docXImagePolicy": "Redact",
    "rtfImagePolicy": "Ignore",
    "pdfSignaturePolicy": "Redact",
    "pdfSynthModePolicy": "V1",
    "docXCommentPolicy": "Remove",
    "docXTablePolicy": "Redact",
    "llmClassificationPolicy": "Disabled",
    "llmTableClassificationPolicy": "Disabled",
    "fileSource": "Local",
    "customPiiEntityIds": [
      "text"
    ],
    "operations": [
      "HasAccess"
    ],
    "rescanJobs": [
      {
        "id": "text",
        "status": "text",
        "errorMessages": "text",
        "startTime": {},
        "endTime": {},
        "publishedTime": {},
        "datasetFileId": "text",
        "datasetId": "text",
        "jobType": "DeidentifyFile",
        "ocrServiceProvider": "Azure",
        "modelsInfo": {
          "dateSynthesis": {
            "cpuVariant": "text",
            "modelName": "text",
            "runsOnGpu": true
          },
          "fasttext": {
            "libVersion": "text",
            "model": "text",
            "runsOnGpu": true
          },
          "image": {
            "version": "text"
          },
          "spacy": {
            "libVersion": "text",
            "auxModel": {
              "name": "text",
              "language": "text",
              "runsOnGpu": true,
              "version": "text"
            },
            "multilingualModels": [
              {
                "name": "text",
                "language": "text",
                "runsOnGpu": true,
                "version": "text"
              }
            ]
          },
          "torch": {
            "gpuAvailable": true,
            "libVersion": "text"
          },
          "tonicNer": {
            "enModel": "text",
            "xlmModel": "text"
          },
          "tesseract": {
            "model": {
              "id": "text",
              "version": "text"
            }
          }
        },
        "pdfSynthModePolicy": "V1"
      }
    ],
    "mostRecentExternalFileGenerationJob": {
      "id": "text",
      "status": "text",
      "errorMessages": "text",
      "startTime": {},
      "endTime": {},
      "publishedTime": {},
      "datasetFileId": "text",
      "datasetId": "text",
      "jobType": "DeidentifyFile",
      "ocrServiceProvider": "Azure",
      "modelsInfo": {
        "dateSynthesis": {
          "cpuVariant": "text",
          "modelName": "text",
          "runsOnGpu": true
        },
        "fasttext": {
          "libVersion": "text",
          "model": "text",
          "runsOnGpu": true
        },
        "image": {
          "version": "text"
        },
        "spacy": {
          "libVersion": "text",
          "auxModel": {
            "name": "text",
            "language": "text",
            "runsOnGpu": true,
            "version": "text"
          },
          "multilingualModels": [
            {
              "name": "text",
              "language": "text",
              "runsOnGpu": true,
              "version": "text"
            }
          ]
        },
        "torch": {
          "gpuAvailable": true,
          "libVersion": "text"
        },
        "tonicNer": {
          "enModel": "text",
          "xlmModel": "text"
        },
        "tesseract": {
          "model": {
            "id": "text",
            "version": "text"
          }
        }
      },
      "pdfSynthModePolicy": "V1"
    },
    "fileSourceExternalCredential": {
      "fileSource": "Local",
      "awsCredentialsSource": "UserProvided",
      "credential": {}
    },
    "awsCredentialSource": "text",
    "outputPath": "text",
    "externalFilesInfo": {
      "selectedFiles": [
        "text"
      ],
      "pathPrefixes": [
        "text"
      ],
      "selectedFileExtensions": [
        "text"
      ]
    },
    "ocrServiceProvider": "Azure",
    "fileCount": 1
  },
  "uploadedFileId": "text"
}

Download a dataset file

get

Downloads the specified file from the dataset. The downloaded file is redacted based on the dataset configuration.

Path parameters

datasetIdstringRequired

The unique identifier of the dataset

fileIdstringRequired

The unique identifier of the file

Query parameters

jsonDownloadAsstringOptional

Format for JSON file downloads

Responses

200

Returns the redacted file stream

application/octet-streamtext/plainapplication/jsontext/json

400

Invalid request parameters or unsupported file type for the requested operation

application/octet-streamtext/plainapplication/jsontext/json

404

Dataset or file not found

application/octet-streamtext/plainapplication/jsontext/json

409

File is still processing

application/octet-streamtext/plainapplication/jsontext/json

500

An unexpected error occurred during redaction processing

get/api/Dataset/{datasetId}/files/{fileId}/download

HTTPcURLJavaScriptPython

GET /api/Dataset/{datasetId}/files/{fileId}/download HTTP/1.1
Accept: */*

Download all dataset files

get

Downloads all files from the specified dataset. The downloaded files are redacted based on the dataset configuration.

Path parameters

datasetIdstringRequired

The unique identifier of the dataset

Query parameters

jsonDownloadAsstringOptional

Format for JSON file downloads

Responses

200

Returns ZIP archive of all redacted files

application/octet-streamtext/plainapplication/jsontext/json

400

Invalid request parameters or download option

application/octet-streamtext/plainapplication/jsontext/json

404

Dataset not found

application/octet-streamtext/plainapplication/jsontext/json

500

An unexpected error occurred during processing

get/api/Dataset/{datasetId}/files/download_all

HTTPcURLJavaScriptPython

GET /api/Dataset/{datasetId}/files/download_all HTTP/1.1
Accept: */*