Technical Deep Dive: Thumbnails, Memory Cache, and Evictions

Thumbnails, Memory Cache & Evictions

Overview

RawCull processes Sony ARW (Alpha Raw) image files through two mechanisms:

1. Thumbnail Generation: Creates optimized 2048×1372 thumbnails for the culling UI
2. Embedded Preview Extraction: Extracts full-resolution JPEG previews from ARW metadata for detailed inspection

Both systems integrate with a hierarchical two-tier caching architecture (RAM → Disk) to minimize repeated file processing. The system has been refactored to maximise memory utilisation and minimise unnecessary evictions.

Thumbnail Specifications

Standard Dimensions

All thumbnails are created at a fixed size to ensure consistent performance and caching:

Why 2048×1372?

Original ARW dimensions: 6000×4000 pixels (typical Sony Alpha)
                            ↓
            Downsampled by factor of ~3x
                            ↓
        2048×1372 thumbnails
                            ↓
    Perfect balance:
    - Large enough for detail recognition
    - Small enough for reasonable memory footprint
    - Maintains original aspect ratio

ARW File Format

Structure

Sony ARW files are TIFF-based containers with multiple embedded images:

ARW File (TIFF-based)
├── Index 0: Small thumbnail (≤256×256px)
├── Index 1: Preview JPEG (variable resolution)
├── Index 2: Maker Notes & EXIF Data
└── Index 3+: Raw Sensor Data

Image Discovery

The extraction system uses CGImageSource to enumerate all images:

let imageCount = CGImageSourceGetCount(imageSource)

for index in 0 ..< imageCount {
    let properties = CGImageSourceCopyPropertiesAtIndex(imageSource, index, nil)
    let width = getWidth(from: properties)
    let isJPEG = detectJPEGFormat(properties)
}

JPEG Detection

Identifies JPEG payloads using two markers:

1. JFIF Dictionary: Presence of kCGImagePropertyJFIFDictionary
2. TIFF Compression Tag: Compression value of 6 (TIFF 6.0 JPEG)

let hasJFIF = (properties[kCGImagePropertyJFIFDictionary] as? [CFString: Any]) != nil
let compression = tiffDict?[kCGImagePropertyTIFFCompression] as? Int
let isJPEG = hasJFIF || (compression == 6)

Dimension Extraction

Retrieves image dimensions from multiple sources in priority order:

1. Root Properties: kCGImagePropertyPixelWidth
2. EXIF Dictionary: kCGImagePropertyExifPixelXDimension
3. TIFF Dictionary: kCGImagePropertyTIFFImageWidth
4. Fallback: Return nil if none available

Thumbnail Creation Pipeline

Source File Processing

When a user opens a RawCull project with ARW files:

ARW File (10-30 MB)
    ↓
[RAW Decoder]
    - Load raw sensor data
    - Apply Bayer demosaicing
    - Color correction
    ↓
Full Resolution Image (RGB, 3 bytes/pixel)
    ↓
[Resize Engine]
    - Maintain aspect ratio
    - Bilinear or lanczos filtering
    ↓
2048 × 1372 RGB Thumbnail
    - 16.86 MB uncompressed
    - 6 bytes/pixel (including alpha)

Extraction Process

private nonisolated func extractSonyThumbnail(
    from url: URL,
    maxDimension: CGFloat,  // 2048 for standard size
    qualityCost: Int = 6     // Configurable 4-8 bytes/pixel
) async throws -> CGImage

Phase 1: Image Source Creation

let options = [kCGImageSourceShouldCache: false] as CFDictionary
guard let source = CGImageSourceCreateWithURL(url as CFURL, options) else {
    throw ThumbnailError.invalidSource
}

• Opens ARW file via ImageIO
• kCGImageSourceShouldCache: false prevents intermediate caching

Phase 2: Thumbnail Generation

let thumbOptions: [CFString: Any] = [
    kCGImageSourceCreateThumbnailFromImageAlways: true,
    kCGImageSourceCreateThumbnailWithTransform: true,
    kCGImageSourceThumbnailMaxPixelSize: maxDimension,
    kCGImageSourceShouldCacheImmediately: false
]

guard var image = CGImageSourceCreateThumbnailAtIndex(
    source, 0, thumbOptions as CFDictionary
) else {
    throw ThumbnailError.generationFailed
}

Phase 3: Quality Enhancement (Optional)

If costPerPixel ≠ 6, the image is re-rendered with appropriate interpolation:

let qualityMapping: [Int: CGInterpolationQuality] = [
    4: .low,
    5: .low,
    6: .medium,   // Default, balanced
    7: .high,
    8: .high
]

Phase 4: Return Thread-Safe Image

return image  // CGImage is Sendable, safe for actor boundary

CGImage is returned (not NSImage) because it is Sendable and can cross actor boundaries safely.

Phase 5: Storage (in Actor Context)

let nsImage = NSImage(cgImage: image, size: NSSize(...))
storeInMemoryCache(nsImage, for: url)  // RAM cache immediately

Task.detached(priority: .background) { [cgImage] in
    await self.diskCache.save(cgImage, for: url)
}

System Architecture: Two-Tier Cache

Cache Tiers

┌─────────────────────────────────────────────┐
│          Thumbnail Requested                │
└────────────────┬────────────────────────────┘
                 │
                 ▼
        ┌────────────────────┐
        │  Memory Cache?     │
        │  (NSCache)         │
        └────────┬───────────┘
                 │
       ┌─────────┴──────────┐
       │ HIT (70.2%)        │ MISS (29.8%)
       ▼                    ▼
    Return from       Disk Cache?
    Memory            (FileSystem)
                           │
                    ┌──────┴──────┐
                    │ HIT          │ MISS
                    │ (29.8%)      │
                    ▼              ▼
                 Read from     Decompress
                 Disk, Add     Original ARW,
                 to Memory     Create Thumbnail

    Performance: ~instant    ~instant      ~100-500ms
                 (in-memory)  (disk I/O)    (CPU-bound)

Tier 1: RAM Cache (NSCache)

Managed by SharedMemoryCache actor with dynamic configuration:

let memoryCache = NSCache<NSURL, DiscardableThumbnail>()
memoryCache.totalCostLimit = dynamicLimit  // Based on system RAM
memoryCache.countLimit = 10_000             // High; memory is limiting factor

Characteristics:

• LRU Eviction: Least-recently-used thumbnails removed when cost limit exceeded
• Protocol: Implements NSDiscardableContent for OS-level memory reclamation
• Thread-Safe: Built-in synchronization by NSCache
• Cost-Aware: Respects pixel memory, not item count
• Hit Rate: 70.2% (observed in typical workflows)

Tier 2: Disk Cache

// Location: ~/.RawCull/thumbcache/[projectID]/
// Format: JPEG compressed at 0.7 quality
// Size: 3-5 MB per thumbnail (82-91% compression)

Characteristics:

• Hit Rate: 29.8% (complements memory cache)
• Latency: 50-200 ms (disk I/O + decompression)
• Persistence: Survives app restart
• Automatic Promotion: Disk hits loaded to memory for next access

Disk cache representation formats:

Storage location: ~/.RawCull/thumbcache/[projectID]/

Memory Cache Policy

Cost is calculated per cached image as:

Where:

• Pixel dimensions: Actual pixel size from image.representations or logical image size fallback
• Bytes per pixel: Default is 4 (RGBA: Red, Green, Blue, Alpha), but configured to 6 in this case
• 1.1 multiplier: 10% overhead buffer for NSImage wrapper and caching metadata

With 2048×1372 thumbnail size and 6 bytes/pixel:

Count Limit Calculation

Count limit is set now to fixed 10,000 as cap, but it is controlled by maxium memory allocated for app. Max memory allocated is 10,000 MB (10 GB).

Allocation Strategy

Available System Memory Detection

let physicalMemory = ProcessInfo.processInfo.physicalMemory
let memoryThresholdPercent = 80  // 80% of available RAM
let maxCacheSize = (physicalMemory * memoryThresholdPercent) / 100

// Example Results:
// 8 GB Mac:  6.4 GB available for cache
// 16 GB Mac: 12.8 GB available for cache
// 32 GB Mac: 25.6 GB available for cache

User Configuration

Capacity Planning

Allocated Memory: 10,000 MB
Per-Thumbnail Cost: 19.4 MB

Maximum Thumbnails = 10,000 MB ÷ 19.4 MB per thumbnail
                   = ~515 thumbnails

In Practice (observed): 571 thumbnails
Reason: NSCache's cost calculation accounts for various
        representation formats, slightly improving efficiency

Cost Calculation

// For 2048×1372 thumbnail at 6 bytes/pixel:
Cost = 2048 × 1372 × 6 = 16,860,096 bytes
With 10% overhead: ≈ 19.4 MB per thumbnail

// Cost impacts:
// 4 bytes: Lower quality, more capacity (~645 thumbnails in 10 GB)
// 6 bytes: Balanced quality/capacity (~515 thumbnails)
// 8 bytes: Maximum quality, less capacity (~385 thumbnails)

Eviction Policy

NSCache LRU (Least Recently Used) Strategy

Cache Full → New Item Added
    ↓
[Eviction Engine]
    - Identify least recently used items
    - Remove oldest accessed thumbnails first
    - Continue until space available
    ↓
New Item Inserted

Memory Pressure Monitoring

Background Monitoring Loop (every 100ms):

if usage > 95% of allocation:
    → Aggressive eviction (trim 20%)
    → Log warning

if usage > 80% of allocation:
    → Normal eviction (trim 10% on next cache miss)

if usage < 50% of allocation:
    → No eviction
    → Cache can grow freely

Thresholds by System Configuration

Low Memory Mac (< 8 GB):
    Memory Threshold = 60%
    Default Cache = 3 GB
    Typical Items = ~155 thumbnails

Standard Mac (8-16 GB):
    Memory Threshold = 80%
    Default Cache = 6-10 GB
    Typical Items = ~300-515 thumbnails

High-End Mac (> 16 GB):
    Memory Threshold = 80%
    Default Cache = 12-25 GB
    Typical Items = ~600-1200 thumbnails

Eviction Analysis (Post-Refactor)

Test Parameters

Key insight: The original analysis used 24 MB (from 2048²), but actual thumbnails are rectangular, giving ~19.4 MB per thumbnail.

Phase 1: Initial Thumbnail Scan

Actual cached thumbnails = 571 images at ~19 MB each
                         = ~10,849 MB
                         ≈ 10 GB utilization

Matches allocated memory perfectly!

Phase 2: Interactive Browse (Sequential Access)

Why 231 evictions in Phase 2?

• Capacity: 571 thumbnails
• Browse order: 618 images
• Items browsed beyond capacity: 47
• LRU churn from sequential access: ~184 additional
• Total: 47 + 184 = 231 evictions

Refactored Implementation

Changes Made

1. Removed dimension guessing — Cache relies on NSCache’s actual cost calculations
2. Set countLimit to 10,000 — High enough that memory is the only real constraint
3. Memory threshold increased to 80% — Allows 10 GB allocations on 16 GB+ systems
4. Diagnostic logging — Logs actual cache configuration at startup

Code Changes

File: SharedMemoryCache.swift

// BEFORE: Used estimated 2048×2048, calculated countLimit dynamically
let estimatedCostPerImage = (thumbnailSize * thumbnailSize * costPerPixel * 11) / 10
let countLimit = totalCostLimit / estimatedCostPerImage

// AFTER: Let NSCache calculate actual costs, countLimit is high
let countLimit = 10000  // Very high, memory (totalCostLimit) is real constraint

File: SettingsViewModel.swift

// BEFORE: Memory threshold was 50%
let memoryThresholdPercent = 50  // Restricted 10GB on 16GB Mac

// AFTER: Memory threshold is 80%
let memoryThresholdPercent = 80  // Allows 10GB on 16GB Mac

Embedded Preview Extraction

For detailed inspection, RawCull can extract full-resolution JPEG previews directly from ARW metadata, providing superior quality compared to generated thumbnails.

Selection Strategy

The system selects the widest JPEG from all images embedded in the ARW:

for index in 0 ..< imageCount {
    let properties = CGImageSourceCopyPropertiesAtIndex(imageSource, index, nil)
    if let width = getWidth(from: properties), isJPEG(properties) {
        if width > targetWidth {
            targetIndex = index
            targetWidth = width
        }
    }
}

Sony typically stores higher-quality previews at later indices, so the widest JPEG maximises quality.

Thumbnail vs. Full Preview

Downsampling Decision

let maxPreviewSize: CGFloat = fullSize ? 8640 : 4320

if CGFloat(embeddedJPEGWidth) > maxPreviewSize {
    // Downsample to reasonable size
} else {
    // Use original size (never upscale)
}

• If embedded JPEG is larger than target: downsample to preserve memory
• If embedded JPEG is smaller: preserve original (never upscale)
• fullSize=true: 8640px threshold (professional workflows)
• fullSize=false: 4320px threshold (balanced quality/performance)

Resizing Implementation

private func resizeImage(_ image: CGImage, maxPixelSize: CGFloat) -> CGImage? {
    let scale = min(maxPixelSize / CGFloat(image.width), maxPixelSize / CGFloat(image.height))
    guard scale < 1.0 else { return image }  // Already smaller

    // Draw into new context with .high interpolation
    context.interpolationQuality = .high
    context.draw(image, in: CGRect(x: 0, y: 0, width: newWidth, height: newHeight))
    return context.makeImage()
}

JPEG Export

@concurrent
nonisolated func save(image: CGImage, originalURL: URL) async {
    // Saves alongside original ARW as .jpg at maximum quality (1.0)
    let options: [CFString: Any] = [
        kCGImageDestinationLossyCompressionQuality: 1.0
    ]
}

Thumbnail Generation System: Preload Workflow

Architecture Overview

The ScanAndCreateThumbnails actor manages complete thumbnail lifecycle:

File Processing Pipeline
    ↓
[RAM Cache Check (NSCache)]
    ├─ HIT (70.2%): Return from memory
    └─ MISS (29.8%): Continue
         ↓
    [Disk Cache Check]
    ├─ HIT: Load from disk, promote to RAM
    └─ MISS: Continue
         ↓
    [Extract from ARW]
    ├─ Open ARW file
    ��─ Extract or generate thumbnail
    ├─ Store in RAM Cache
    └─ Save to disk asynchronously

Performance: ~instant (~1ms) → disk (~100ms) → extraction (~200-500ms)

Preload Workflow

func preloadCatalog(at catalogURL: URL, targetSize: Int) async -> Int

Step 1: File Discovery

let urls = await DiscoverFiles().discoverFiles(at: catalogURL, recursive: false)

Step 2: Concurrent Processing with Smart Throttling

let maxConcurrent = ProcessInfo.processInfo.activeProcessorCount * 2

for (index, url) in urls.enumerated() {
    if Task.isCancelled { break }
    if index >= maxConcurrent {
        try? await group.next()  // Sliding window throttle
    }
    group.addTask {
        await self.processSingleFile(url, targetSize: targetSize, itemIndex: index)
    }
}

• Spawns up to 2× processor count tasks
• After reaching limit, waits for one task per new task
• Prevents memory exhaustion on large catalogs (1000+ files)

Cache Lifecycle Management

Initialization

1. Detect system memory
   ↓ 16 GB Mac → threshold = 12.8 GB

2. Load user settings
   ↓ Last setting: 10 GB → use 10 GB

3. Configure NSCache
   ↓ Set totalCostLimit = 10,000,000,000 bytes
   ↓ Set countLimit = 10,000 items (high, not limiting)

4. Initialize background monitoring
   ↓ Start memory pressure checks

5. Log configuration
   ↓ "Cache ready: 10GB, ~515 thumbnails"

Cache Invalidation

Concurrency Model

Actor-Based Architecture

All extraction systems use Swift actors for thread-safe state:

actor ScanAndCreateThumbnails { }
actor ExtractSonyThumbnail { }
actor ExtractEmbeddedPreview { }
actor DiskCacheManager { }

Benefits:

• Serial execution prevents data races
• State mutations are automatically serialized
• No manual locks required
• Safe concurrent calls from multiple views

Isolated State

actor ScanAndCreateThumbnails {
    private var successCount = 0
    private var processingTimes: [TimeInterval] = []
    private var totalFilesToProcess = 0
    private var preloadTask: Task<Int, Never>?
}

Concurrent Extraction Without Isolation Violation

ImageIO operations are nonisolated to avoid blocking the actor:

@concurrent
nonisolated func extractSonyThumbnail(from url: URL, maxDimension: CGFloat) async throws -> CGImage {
    try await Task.detached(priority: .userInitiated) {
        let source = CGImageSourceCreateWithURL(url as CFURL, options)
        // ...
    }.value
}

Cancellation Support

func cancelPreload() {
    preloadTask?.cancel()
    preloadTask = nil
}

Error Handling

Extraction Errors

enum ThumbnailError: Error {
    case invalidSource
    case generationFailed
    case decodingFailed
}

Error Recovery

Batch Processing (non-fatal — continues to next file):

do {
    let cgImage = try await ExtractSonyThumbnail().extractSonyThumbnail(from: url, ...)
    storeInMemoryCache(cgImage, for: url)
} catch {
    Logger.process.warning("Failed to extract \(url.lastPathComponent): \(error)")
}

On-Demand Requests (returns nil; UI shows placeholder):

func thumbnail(for url: URL, targetSize: Int) async -> CGImage? {
    do { return try await resolveImage(for: url, targetSize: targetSize) }
    catch { return nil }
}

Performance Characteristics

Typical Timings (Apple Silicon, 40-50 ARW files, 16 GB Mac)

Memory Usage per Configuration

Quality/Performance Tradeoff

Cost Per Pixel | Memory Per Image | 10 GB Capacity | Quality      | Speed
───────────────────────────────────────────────────────────────────────
4 bytes        | ~15 MB           | ~667           | Good         | Fast
6 bytes        | ~19.4 MB         | ~515           | Excellent    | Balanced
8 bytes        | ~25.8 MB         | ~387           | Outstanding  | Slower

Concurrency Impact

Processor Cores | Max Concurrent Tasks | Benefit
───────────────────────────────────────────────
4-core Mac      | 8 tasks              | 2-3x faster
8-core Mac      | 16 tasks             | 4-6x faster
10-core Mac     | 20 tasks             | 6-8x faster

Monitoring and Diagnostics

Startup Log

[Cache] Initialization Report
────────────────────────────────
System Memory: 16 GB
Memory Threshold: 80% = 12.8 GB
Allocated to Cache: 10 GB
Cost Per Pixel: 6 bytes
Expected Capacity: ~515 thumbnails
Count Limit: 10,000 items (not used as constraint)
LRU Strategy: Enabled
Disk Cache: ~/.RawCull/thumbcache/

✓ Cache initialized and ready

Runtime Statistics

[Cache] Runtime Statistics
──────────────────────────────
Current Usage: 9.87 GB (98.7% of 10 GB)
Thumbnails Cached: 508 items
Memory Hits: 156 | Disk Hits: 68 | Cache Misses: 24 | Evictions: 12
Hit Rate: Memory 70.2% | Disk 29.8%

Configuration Reference

Programmatic Configuration

// File: SharedMemoryCache.swift
let totalCostLimit = 10_000_000_000  // 10 GB in bytes
let costPerPixel = 6                  // bytes per pixel
let countLimit = 10_000               // Very high, not limiting
let memoryThresholdPercent = 80       // 80% of available RAM
let memoryCheckInterval = 0.1         // seconds
let aggressiveEvictionThreshold = 95  // percent of allocation
let normalEvictionThreshold = 80      // percent of allocation

Relevant Source Files

• SharedMemoryCache.swift — Memory cache configuration
• SettingsViewModel.swift — Memory threshold and user settings
• ExtractSonyThumbnail.swift — Quality mapping and thumbnail generation
• ExtractEmbeddedPreview.swift — Preview thresholds (4320/8640 px)
• CacheConfig.swift — Cache limit constants

Best Practices

For Users

1. Match allocation to workflow: 5 GB (8 GB Mac) / 10 GB (16 GB Mac) / 16+ GB (32 GB Mac)
2. Monitor memory usage: leave 2-3 GB free for system and other apps
3. Quality settings: 6 bytes/pixel (default); reduce to 4 for more capacity; increase to 8 for highest quality

For Developers

1. Cache configuration: always query system memory on startup; apply thresholds dynamically
2. Cost calculations: use realistic estimates; account for ~10% overhead
3. Eviction handling: implement LRU consistently; monitor frequency (target < 10 evictions per 100 accesses)
4. Performance profiling: target 70% memory hit rate; profile real-world patterns

Troubleshooting

Data Flow Summary

User initiates bulk thumbnail load
    ↓
[ScanAndCreateThumbnails.preloadCatalog()]
    ├─ Discover files (non-recursive)
    ├─ For each file (concurrency controlled):
    │   ├─ Check RAM cache
    │   │   ✓ HIT (70%): Return immediately
    │   │   ✗ MISS (30%):
    │   ├─ Check disk cache
    │   │   ✓ HIT: Load and promote to RAM
    │   │   ✗ MISS:
    │   ├─ Extract thumbnail:
    │   │   ├─ Open ARW via ImageIO
    │   │   ├─ Generate 2048×1372 thumbnail
    │   │   ├─ Apply quality enhancement (optional)
    │   │   └─ Wrap in NSImage
    │   ├─ Store in RAM (immediate)
    │   └─ Schedule async disk save (background)
    └─ Return success count

On detailed inspection:
    ↓
[JPGPreviewHandler.handle(file)]
    ├─ Check if JPG exists
    │   ✓ YES: Load and display
    │   ✗ NO:
    ├─ Call ExtractEmbeddedPreview
    │   ├─ Find all images in ARW
    │   ├─ Identify widest JPEG
    │   ├─ Decide: downsample or original?
    │   ├─ Decode JPEG
    │   └─ Return CGImage
    └─ Display full preview

Apple Frameworks Used

• ←Previous
• Next→