Dongjae Jeon

We trace the diversity loss of arbitrary-order decoding in diffusion LLMs not to the generation order itself, but to low-confidence remasking (LCR), which exponentially suppresses lower-probability tokens as more positions compete. Switching to top-probability position selection (TPP) restores diversity with Pass@k comparable to left-to-right decoding, and Entropy-Guided Initialization (EGI) further improves rollout diversity, solution coverage, and downstream policy optimization.

We propose a truly benign DPO attack that jailbreaks frontier LLMs using only 10 harmless preference pairs that look indistinguishable from ordinary over-refusal reduction data, pairing benign prompts with a helpful answer (preferred) and a refusal (dispreferred). Because DPO directly suppresses refusal behavior, the effect transfers to unseen harmful prompts, reaching up to 81.73% ASR on GPT-4.1-nano for $0.1 while evading the OpenAI Moderation API and frontier-scale LLM auditors.

We diagnose why state-of-the-art VLMs lose a selected visual path when nearby alternatives look locally similar: behavioral interventions, causal masking, and internal attention/representation analyses converge on a single mechanism — nearby distractors pull the model off the true continuation, producing systematic adjacent-path jumps. Standard remedies (model scaling, reasoning, explicit tracing instructions) fail to recover stable path following, and the same failure persists on real metro maps and tangled cables.

Parallel decoding makes diffusion LLM rollouts faster, but confidence-based selection keeps rollouts nearly identical by delaying branching. A single random unmasking step before decoding forces early branching, restoring rollout diversity and improving downstream RL performance without changing the decoder.

We propose DAPD, a training free parallel decoding method for diffusion LLMs that uses self attention to build a dependency graph over masked tokens and unmask an independent set in parallel to reduce joint marginal mismatch. Across LLaDA and Dream, DAPD improves the accuracy steps trade off and yields more globally dispersed, any order unmasking than marginal confidence baselines.

This paper proposes Structured Residual Reconstruction (SRR), which preserves the top singular subspace of activation scaled weights before quantization and uses the remaining rank budget to reconstruct quantization error, guided by a theory based criterion for choosing the preserved rank. It improves post training quantization perplexity and boosts 2 bit QPEFT performance, reporting a 5.9 point average gain on GLUE.

This paper identifies overflow in instruction tuned diffusion LLMs, where increasing the max length paradoxically causes early termination or repetitive outputs because serves as both padding and the stop token. It proposes Rainbow Padding, which replaces repeated padding with a small cyclic set of distinct padding tokens, restoring length robustness and improving reasoning and coding performance with minimal LoRA fine tuning.

This paper proposes A2D, a token-level safety alignment method for diffusion language models that triggers an \texttt{[EOS]} refusal as soon as harmful content arises, making safety robust to any decoding order and any step. On safety benchmarks, A2D drives DIJA prefilling attack success rates from over 80% to near-zero and enables early rejection via thresholded \texttt{[EOS]} probabilities for up to 19.3$\times$ faster safe termination.

This paper introduces the Information Difference Index (IDI), a white box information theoretic metric that quantifies residual forget set information retained in intermediate representations after unlearning, beyond what end task accuracy reveals. Across datasets and architectures, IDI exposes cases where black box evaluations overestimate unlearning and provides a more reliable measure of strong unlearning.

This paper develops an information theoretic foundation for discrete diffusion, deriving I MDSE and I MDCE identities that show common score matching losses (DSE and DCE) yield tight, principled log likelihood estimators rather than loose variational bounds. It also provides practical estimators, including a time free likelihood formula, conditional likelihood for prompt response tasks, and a coupled Monte Carlo likelihood ratio estimator, with experiments validating accuracy and variance stability.

This paper shows that joint quantization plus low rank decomposition quality depends strongly on how the low rank term is initialized, because initialization fixes the roles each component keeps during optimization; it proposes Outlier Driven Low Rank Initialization (ODLRI) that assigns activation sensitive outlier weights to the low rank part so quantization handles the remaining weights more stably. Experiments on Llama2, Llama3, and Mistral show consistent gains in activation aware error, quantization scale, perplexity, and zero shot accuracy in extreme low bit settings.

This paper links diffusion model memorization to high sharpness in the learned log-density landscape and proposes a sharpness-based metric that detects memorization from the earliest sampling steps; it then introduces SAIL, an inference-time initial-noise optimization that steers sampling toward smoother regions to reduce memorization without retraining or prompt edits.

This paper introduces LTF TEST, a long form fairness benchmark that probes demographic bias in essay style generations across 14 topics and 10 demographic axes. Evaluations on multiple LLMs show that bias remains and can surface more clearly in long text than in short form fairness tests, motivating long form bias auditing as a complementary standard.