Reveal AI Processing Stages
The steps involved in the Reveal AI ingestion process are set out in the table below to offer a better understanding of the activity taking place and troubleshooting any potential issues with Support.
ID | Display | Explanation |
|---|---|---|
1 | Preparing processing jobs to start | Create jobs in the SQL ProcessingQueue table |
2 | Pre-process | Java processing initialization |
3 | Pre-process - checking source dataset | Status pass Compare documents that are sent to processing vs. existing documents |
4 | Computer-generated content detection | Zero pass Detect computer generated content |
5 | Ingestion | 1st pass Email segmentation Metadata parsing |
6 | Name resolution and threading - entity update | Entity update based on 1st pass processing |
7 | Name resolution and threading - thread creation | Threading creation |
8 | Natural language pipeline | 2nd pass Entity extraction Sentiment detection Text vector generation |
9 | Data finalization - entity update | Entity update based on 2nd pass processing |
10 | Data finalization - network-based merge | Network-based entity merging |
11 | Data finalization - segment count update | Combine Id Segment Counts Thread Properties Relationship Counts Random Id Language |
12 | Data finalization - populating denormal tables | SQL Sync (Populating denormal tables) (2.50) Submit clustering job, create numeric vector and other vectors |
13 | Pattern Detection | Pattern Detection |
14 | Near Duplicate Document Detection | Near Duplicate Document Detection |
15 | Post-process | Java processing steps to stop things such as the JVM monitor (2.80) Submit clustering job, create numeric vector and other vectors |
16 | Warmup | Run Queries to cache server |
17 | Populate copy thread extended | Create thread extended table for exporting purposes later |