Enterprise AI Analysis

ICAD-LLM: A One-for-All Paradigm

ICAD-LLM revolutionizes anomaly detection with a 'One-for-All' framework that leverages Large Language Models (LLMs) for in-context learning. Its core innovation, In-Context Anomaly Detection (ICAD), redefines anomalies based on dynamic contextual dissimilarity to a reference set, rather than a static, memorized definition of normality.

This paradigm is realized through three key components: a Modality-Aware Encoder for unified embedding of diverse data, a Prompt-Guided Representation Module utilizing LLMs for context-sensitive representations, and a Contextual Contrastive Learning objective to train for generalized anomaly discrimination.

Unified Data Embedding for Heterogeneous Modalities

The Modality-Aware Encoder is a critical component of ICAD-LLM, addressing REQ1: Feature Alignment. It transforms heterogeneous samples from various modalities (time series, tabular, logs) into a unified embedding space. This allows the model to process disparate inputs within a single, consistent architecture.

• Time Series Data: Utilizes a Convolutional Neural Network (CNN) after instance normalization to align feature dimensions.
• Tabular Data: Employs a two-layer Multilayer Perceptron (MLP) to generate embeddings.
• Log Data: First processed by the LLM's native tokenizer and embedder, then refined by a Transformer encoder.

This ensures that regardless of the original data format, all information is represented in a comparable, unified manner for subsequent processing by the LLM.

LLM-Powered Context-Sensitive Representations

The Prompt-Guided Representation Module is at the core of ICAD-LLM, harnessing pre-trained Large Language Models (LLMs) to extract modality-agnostic representations sensitive to subtle dissimilarities (REQ2).

• Instruction-based Priming: An instruction prompt is prepended to the input sequence, explicitly directing the LLM to focus on assessing contextual dissimilarity, rather than general language understanding.
• Token-anchored Representation Pooling: Special, learnable tokens ([REF_TOK], [TGT_TOK], [NEG_TOK]) are inserted into the input. The LLM is compelled to aggregate and summarize reference, target, and negative sample information into these respective token positions, yielding holistic, context-sensitive representations.

This module ensures that the model can capture the essential characteristics of inputs while being acutely aware of their contextual differences.

Learning General Anomaly Discrimination

The Contextual Contrastive Learning (CCL) objective is designed to fulfill REQ3: Task-Agnostic Discriminative Objective. Unlike traditional AD training methods that rely on task-specific data distributions, CCL explicitly trains the model's universal ability to discriminate dissimilarity.

The training process is structured around sample triplets (Reference Set R, Positive Sample x+, Negative Sample x-) from various modalities:

• Reference Set (R): A set of K normal samples defining the normal context.
• Positive Sample (x+): Another normal sample expected to be similar to R.
• Negative Sample (x-): Can be a simple negative (normal from different dataset) or a hard negative (anomalous from the same dataset).

By minimizing a loss function that maximizes discrepancy for anomalous samples and minimizes it for normal ones, the model learns to produce low discrepancy scores for contextually similar pairs and high scores for dissimilar ones, directly enabling flexible anomaly detection.