SailPoint FAM – SharePoint Data Classification Optimization
If you've been working with SailPoint FAM and noticed your Data Classification task for SharePoint just sitting there — running for hours, not progressing, seemingly stuck — you're not alone. Here's exactly what caused it and how we fixed it.
The Problem
The Data Classification task was:
Running for a long duration
Appearing to be stuck in a hung state
Not progressing efficiently
Not the kind of behavior you want in a production environment.
What Was Actually Causing It?
After investigation, the root cause came down to five configuration problems:
High Batch Size — brsInBatch was set to 1000, pushing large amounts of data at once and overloading the system.
Large Queue Size — batchesCountInQueueLimit at 100 and maxDocumentQueueSize at 1,000,000 were causing memory pressure and queue congestion.
No Thread Configuration — Threads weren't explicitly defined, so the system defaulted to CPU core behavior, leading to uncontrolled CPU usage.
High Timeout Values — timeoutIntervalMinutes was set to 360, meaning failures took extremely long to surface — which is exactly why it looked like a hang.
Deep Hierarchy Traversal — maxDepthForRootQuery at 100 was causing slow, deep crawling of the SharePoint structure.
The Optimization Approach
The fix covered six areas: load control, thread management, queue optimization, timeout handling, SharePoint-specific tuning, and monitoring visibility.
Configuration Changes
Load Control
Parameter | Old Value | New Value | Benefit |
|---|---|---|---|
brsInBatch | 1000 | 200 | Reduces overload |
batchesCountInQueueLimit | 100 | 20 | Controls queue growth |
Reduced batch size ensures smooth and controlled processing instead of overloading the system.
Thread Optimization
Parameter | New Value | Benefit |
|---|---|---|
contentExtractionThreadsCount | 4 | Balanced CPU usage |
indexingThreadsCount | 4 | Efficient parallel processing |
Explicit thread configuration avoids unpredictable CPU utilization.
Queue Optimization
Parameter | Old Value | New Value |
|---|---|---|
maxDocumentQueueSize | 1,000,000 | 200,000 |
maxResourceQueueSize | 10,000 | 5,000 |
maxFileQueueSize | 10,000 | 5,000 |
Reduced queue sizes improve memory usage and processing speed.
SharePoint Optimization
Parameter | Old Value | New Value |
|---|---|---|
maxDepthForRootQuery | 100 | 20 |
Limits deep traversal of the SharePoint hierarchy, improving performance.
Timeout Optimization
Parameter | Old Value | New Value |
|---|---|---|
timeoutIntervalMinutes | 360 | 120 |
MaxTimeoutRetries | 3 | 2 |
Faster detection of issues and quicker recovery.
Engine Performance
Parameter | Old Value | New Value |
|---|---|---|
engineProgressLoopSleep | 5000 | 3000 |
queueLoaderSleepWhenQueueReachedLimit | 5000 | 2000 |
Improves responsiveness and reduces idle time.
Classification Safety
Parameter | New Value |
|---|---|
privacyFileCharacterLimit | 20,000 |
Prevents large files from blocking classification processing.
Monitoring Enhancement
Parameter | Old Value | New Value |
|---|---|---|
enableExtendedStatistics | false | true |
Provides better visibility into task progress and performance.
Results After Optimization
Once these changes were applied, the difference was noticeable:
Improved task stability
Reduced chances of hanging
Better CPU and memory utilization
Faster processing of SharePoint data
Improved monitoring and troubleshooting
Best Practices to Keep in Mind
Start with a small SharePoint scope before scaling
Monitor queues and logs regularly
Tune thread count based on server capacity
Avoid very high batch sizes in production
Conclusion
The problem was solved through optimizing the number of batches, number of queues, thread count, and timeouts. This guarantees that the Data Classification job is executed efficiently without burdening the system, making it fit for production.
