Enterprise AI Analysis

Key Areas in Literature

Mixed Integer Linear Programming: MILP is a foundational mathematical modeling technique for complex decision-making, optimizing linear objectives under linear constraints where some or all variables are restricted to integers, effectively modeling discrete choices. Despite its broad applicability, MILP faces limitations in computational complexity for large-scale instances and the requirement for expert formulation.

Artificial Intelligence in Optimization: AI techniques, especially Machine Learning (ML), have effectively addressed optimization problems. ML algorithms learn from data to discern patterns, predict optimal solutions, and guide search algorithms. Learning to Optimize (L2O) is an interdisciplinary field that uses data-trained ML models to augment or replace traditional algorithms. Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) are frequently used AI models in this context, capable of analyzing spatio-temporal dependencies and modeling complex connections.

MILP and AI Applications in Air Transportation and Mobility: AI is crucial for optimizing air transportation. Airlines use AI to analyze large data volumes for optimized schedules, minimizing delays and maximizing aircraft utilization. Within airports, AI algorithms process real-time data for resource allocation. Examples include the Crew Pairing Problem (CPP), where ML models cut CPU time by 74%, and the Crew Rostering Problem (CRP).

Gaps in the Literature: Existing research shows the potential of integrating AI with MILP but has gaps in model formulation, enhancement for specific algorithms, and addressing ethical considerations. There's a need for integrated frameworks suitable for safety-critical systems, robust and scalable AI algorithms that blend adaptive learning with mathematical optimization assurances, and improved model interpretability and explainability in dynamic environments.

Methodology: From MILP to Embeddings

Our computational process converts a MILP instance into a low-dimensional embedding space suitable for Instance Space Analysis (ISA) using the air05 instance from the Mixed Integer Programming Library (MIPLIB).

MILP Instance as a Bipartite Graph: The air05 instance, a set partitioning problem for airline crew scheduling, is transformed into a heterogeneous bipartite graph. Variable nodes represent decision variables, and constraint nodes correspond to linear constraints. Edges indicate non-zero entries in the constraint matrix. Initial features for variable nodes include objective coefficient, lower/upper bounds, and an integrality flag, while constraint nodes feature right-hand-side values and sense. Edge weights are processed via a hyperbolic tangent activation function.

Generating Embeddings with GNNs: We use GNNs as feature extractors to learn high-dimensional embedding vectors for all nodes, capturing topological relationships. Two heterogeneous GNN encoder architectures are evaluated: GCN ("HeteroGraphConv") and GAT ("HeteroGAT"). These GNNs are trained using a self-supervised link prediction task, maximizing similarity of connected pairs and minimizing unconnected pairs via a Binary Cross-Entropy with Logits loss function.

Instance Space Analysis: ISA is adapted to analyze the high-dimensional embedding matrices (Zvar and Zcon) generated by the trained GNNs. Dimensionality reduction techniques like Principal Component Analysis (PCA), Uniform Manifold Approximation and Projection (UMAP), and t-Distributed Stochastic Neighbor Embedding (t-SNE) are used to map embeddings into a 2D space for visualization. K-means clustering, guided by the Elbow and Silhouette methods, is applied to assess cluster congruence.

Results: Instance Profile and GNN Embeddings

Instance Profile and Graph Characteristics: The air05 instance comprises 7,195 integer variables, 426 constraints, and 52,121 non-zero entries in its constraint matrix. This represents a sparsity of 1.70% of potential entries, typical for large-scale combinatorial optimization problems. The non-zero entries are unevenly distributed, highlighting the problem's intricacy and supporting the use of graph-based learning for complex topologies.

GNN Model Training and Validation: Both GCN and GAT models showed a rapid reduction in training loss during the initial epochs, indicating successful learning of graph structure. While both converged, GCN achieved a lower and more stable loss, suggesting a more stable optimization process than GAT, which exhibited higher amplitude oscillations.

Analysis of the Learned Instance Space: PCA revealed limited linear separability for both GCN and GAT embeddings, with clusters significantly overlapping. Non-linear techniques like UMAP and t-SNE were necessary. GCN with UMAP/t-SNE produced well-defined and separate "islands" for variable nodes and distinct clusters for constraint nodes, indicating effective capture of complex non-linear structures. GAT also distinguished variable node clusters (though with a different 'S'-shaped geometry) but struggled significantly to organize the constraint space, leading to disorganized, overlapping clusters.

Discussion: Implications for Air Transportation AI

This study validates bipartite graph embeddings as a viable alternative to manual feature engineering for MILP instances, demonstrated by mapping the air05 instance and aligning with MIPLIB benchmarks. This reduces reliance on domain-specific heuristics and offers a scalable way to analyze heterogeneous instances, though feature augmentation might be needed for full structural expressiveness.

The performance difference between GCN and GAT suggests that simpler models might be more effective for this specific class of sparse bipartite graphs. GCN successfully captured global topology for both variables and constraints, while GAT struggled with constraint space organization. This implies that attention mechanisms might introduce unnecessary complexity for initial structural encoding in these contexts.

The distinct clustering in ISA confirms that learned embeddings capture functional roles, significantly contributing to explainable AI (xAI). This structural awareness is crucial for L2O applications, providing a foundation for training agents in tasks like variable selection for branching or generating warm-start solutions, thereby addressing computational challenges in real-world solvers.

Conclusion: Future Directions

The study successfully demonstrated the integration of AI with MILP for air transportation optimization with explainability. By transforming the air05 crew scheduling problem into a heterogeneous bipartite graph, GNNs were shown to extract structural features, reducing the need for manual feature engineering. The GCN architecture excelled in capturing global topology and producing well-defined, separable clusters for both variables and constraints, while the GAT struggled with constraint space organization.

Visualization confirmed the complexity of the embedding space, with PCA identifying main variance axes but requiring non-linear reductions (UMAP, t-SNE) to reveal the functional roles and distinct cluster structures of decision variables. These findings have significant implications for operational efficiency and sustainability in air mobility, enabling automated feature extraction and improved solver behavior for complex tasks like crew scheduling.

Future research should focus on developing explainable L2O agents that use these structural embeddings for specific solver tasks, such as variable selection or warm-start solution generation. This includes investigating feature augmentation techniques and expanding analysis to a broader range of heterogeneous instances to test generalization capabilities.