SDMS data becomes stuck in TE_TEMP when attempting to archive a large amount of data - WKB265433

SYMPTOMS

• It has been reported that when NuGenesis SDMS attempts to archive a large amount of data (several thousand files are added to TE_TEMP), the Transport Engine initially handles these as expected but at some point stops processing the files and they remain dormant in the TE_TEMP folder without being archived. The file extensions of all files remain as .servername_port instead of being renamed to .mark_bad or back to .NGD to allow automatic reprocessing

• Stopping and restarting the Transport Engine has no effect because the process thinks the files are already being processed as the file names have been changed from .NGD to .servername_port

• Checking the Transport Engine log file confirms this; it shows there are no files to process despite there being thousands of files in the TE_TEMP folder

ENVIRONMENT

• NuGenesis 9.3

CAUSE

The root cause of this behavior is not currently understood. 

FIX or WORKAROUND

Renaming the affected file extensions back to .NGD forces the Transport Engine to reprocess each file successfully. The processed files are then removed from the TE_TEMP folder as expected.

ADDITIONAL INFORMATION

Defect Report, CRI-6111, "SDMS Data gets stuck in TE_TEMP when a large amount of data is archived", has been created for this issue.