### The Core Problem
Abstractive document summarization — generating novel, condensed text from long documents — remains difficult for three key reasons: **document structure blindness** (most models flatten documents into sequences), **multimodal content integration** (tables, figures, and equations are ignored), and **hallucination** (models fabricate facts not present in the source). These issues limit deployment in high-stakes domains like scientific publishing and journalism.
### The Proposed Architecture: MHAN
The authors propose the **Multimodal Hierarchical Attention Network (MHAN)**, built around three integrated components. The **Hierarchical Encoder** operates simultaneously at token, sentence, and paragraph levels, using specialized attention masks and a graph neural network (GRU-based) to model inter-paragraph relationships. This contrasts with prior work that models only one or two granularity levels.
The **Variational Attention Bridge (VAB)** is the model's central innovation. It learns a **Gaussian Mixture Model (GMM)** representation of document semantics from paragraph-level hidden states, sampling latent attention weights that stochastically route information between hierarchy levels. This extends standard VAE approaches (which use simple Gaussian priors) to better capture the multimodal, multi-topic nature of long documents.
The **Multimodal Decoder** generates summaries autoregressively by attending to all three levels of encoder representations simultaneously via a multi-scale attention mechanism. The full training objective combines **negative log-likelihood (NLL)**, **KL divergence** (against a uniform GMM prior), a **semantic consistency loss**, and a **sparsity regularization** term.
### A New Evaluation Metric: SCI
The paper introduces the **Semantic Coherence Index (SCI)** to address limitations of ROUGE, which does not measure factual accuracy. SCI is defined as a product of entity-level overlap, sentence-embedding semantic similarity, and a hallucination penalty. MHAN achieves an average **SCI of 0.847**, versus **0.743** for the ablated baseline — a **14% relative improvement** in factual consistency.
### Experimental Results
MHAN is evaluated on **arXiv** (avg. 4,918 tokens), **PubMed** (3,156 tokens), and **CNN/DailyMail** (781 tokens) — datasets spanning scientific and news domains at varying document lengths. Against baselines including PG-Net, standard Transformer, and Hi-MAP, **MHAN improves ROUGE-L by approximately 2.3% on average**. Ablation studies isolate the contribution of each component: removing the hierarchical encoder causes the largest performance drop (−2.8 ROUGE-L), followed by removal of the variational bridge (−1.3) and cross-modal fusion (−0.9).
### Why It Matters
The combination of structural awareness, probabilistic latent variable modeling, and factual consistency measurement represents a meaningful step toward **reliable, structure-aware summarization** for real-world documents. The SCI metric in particular offers a reusable tool for the broader community to benchmark hallucination reduction. Future work could extend MHAN to truly multimodal inputs (images, tables) beyond the text-structural fusion demonstrated here.