2026
|
Zhu, Aiqing; Soh, Beatrice W; Pavliotis, Grigorios A; Li, Qianxiao Identifiable learning of dissipative dynamics NATURE COMMUNICATIONS, 17 (1), 2026, DOI: 10.1038/s41467-026-76255-3. Abstract | BibTeX | Endnote @article{WOS:001868019700017,
title = {Identifiable learning of dissipative dynamics},
author = {Aiqing Zhu and Beatrice W Soh and Grigorios A Pavliotis and Qianxiao Li},
doi = {10.1038/s41467-026-76255-3},
times_cited = {0},
year = {2026},
date = {2026-08-01},
journal = {NATURE COMMUNICATIONS},
volume = {17},
number = {1},
publisher = {NATURE PORTFOLIO},
address = {HEIDELBERGER PLATZ 3, BERLIN, 14197, GERMANY},
abstract = {Complex dissipative systems appear across science and engineering, from
polymers and active matter to learning algorithms. These systems operate
far from equilibrium, where energy dissipation and time irreversibility
govern their behavior but are difficult to quantify from data. Here, we
introduce a universal and identifiable neural framework that learns
non-degenerate stochastic dissipative dynamics directly from
trajectories while ensuring interpretability, expressiveness, and
uniqueness. Our method identifies a unique energy landscape, separates
reversible from irreversible motion, and allows direct computation of
the entropy production, providing a principled measure of
irreversibility and deviations from equilibrium. Applications to polymer
stretching in elongational flow and to stochastic gradient Langevin
dynamics reveal characteristic scaling behaviors, including super-linear
scaling of barrier heights and sub-linear scaling of entropy production
rates with the strain rate, and the suppression of irreversibility with
increasing batch size. Our methodology thus provides a data-driven
approach to discovering and interpreting non-equilibrium dynamics.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Complex dissipative systems appear across science and engineering, from
polymers and active matter to learning algorithms. These systems operate
far from equilibrium, where energy dissipation and time irreversibility
govern their behavior but are difficult to quantify from data. Here, we
introduce a universal and identifiable neural framework that learns
non-degenerate stochastic dissipative dynamics directly from
trajectories while ensuring interpretability, expressiveness, and
uniqueness. Our method identifies a unique energy landscape, separates
reversible from irreversible motion, and allows direct computation of
the entropy production, providing a principled measure of
irreversibility and deviations from equilibrium. Applications to polymer
stretching in elongational flow and to stochastic gradient Langevin
dynamics reveal characteristic scaling behaviors, including super-linear
scaling of barrier heights and sub-linear scaling of entropy production
rates with the strain rate, and the suppression of irreversibility with
increasing batch size. Our methodology thus provides a data-driven
approach to discovering and interpreting non-equilibrium dynamics. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AFAiqing Zhu
Beatrice W Soh
Grigorios A Pavliotis
Qianxiao Li
- TIIdentifiable learning of dissipative dynamics
- SONATURE COMMUNICATIONS
- DTArticle
- ABComplex dissipative systems appear across science and engineering, from
polymers and active matter to learning algorithms. These systems operate
far from equilibrium, where energy dissipation and time irreversibility
govern their behavior but are difficult to quantify from data. Here, we
introduce a universal and identifiable neural framework that learns
non-degenerate stochastic dissipative dynamics directly from
trajectories while ensuring interpretability, expressiveness, and
uniqueness. Our method identifies a unique energy landscape, separates
reversible from irreversible motion, and allows direct computation of
the entropy production, providing a principled measure of
irreversibility and deviations from equilibrium. Applications to polymer
stretching in elongational flow and to stochastic gradient Langevin
dynamics reveal characteristic scaling behaviors, including super-linear
scaling of barrier heights and sub-linear scaling of entropy production
rates with the strain rate, and the suppression of irreversibility with
increasing batch size. Our methodology thus provides a data-driven
approach to discovering and interpreting non-equilibrium dynamics. - Z90
- PUNATURE PORTFOLIO
- PAHEIDELBERGER PLATZ 3, BERLIN, 14197, GERMANY
- VL17
- DI10.1038/s41467-026-76255-3
- UTWOS:001868019700017
- ER
- EF
|
Chen, Mengyi; Huang, Pengru; Novoselov, Kostya S; Li, Qianxiao Scalable learning of macroscopic stochastic dynamics PHYSICAL REVIEW MATERIALS, 10 (3), 2026, DOI: 10.1103/mlh4-htxv. Abstract | BibTeX | Endnote @article{WOS:001724571000001,
title = {Scalable learning of macroscopic stochastic dynamics},
author = {Mengyi Chen and Pengru Huang and Kostya S Novoselov and Qianxiao Li},
doi = {10.1103/mlh4-htxv},
times_cited = {0},
issn = {2475-9953},
year = {2026},
date = {2026-03-01},
journal = {PHYSICAL REVIEW MATERIALS},
volume = {10},
number = {3},
publisher = {AMER PHYSICAL SOC},
address = {ONE PHYSICS ELLIPSE, COLLEGE PK, MD 20740-3844 USA},
abstract = {Macroscopic dynamical descriptions of complex physical systems are
crucial for understanding and controlling material behavior. With the
growing availability of data and compute, machine learning has become a
promising alternative to first-principles methods to build accurate
macroscopic models from microscopic trajectory simulations. However, for
spatially extended systems, direct simulations of sufficiently large
microscopic systems that inform macroscopic behavior are prohibitive. In
this work, we propose a framework that learns the macroscopic dynamics
of large stochastic microscopic systems using only small-system
simulations. Our framework employs a partial evolution scheme to
generate training data pairs by evolving large-system snapshots within
local patches. We subsequently derive the closure variables associated
with the macroscopic observables and learn the macroscopic dynamics
using a custom loss. Furthermore, we introduce a hierarchical upsampling
scheme that enables the efficient generation of large-system snapshots
from small-system snapshots. We empirically demonstrate the accuracy and
robustness of our framework through a variety of stochastic spatially
extended systems, including those described by stochastic partial
differential equations, idealized lattice spin systems, and a more
realistic NbMoTa alloy system.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Macroscopic dynamical descriptions of complex physical systems are
crucial for understanding and controlling material behavior. With the
growing availability of data and compute, machine learning has become a
promising alternative to first-principles methods to build accurate
macroscopic models from microscopic trajectory simulations. However, for
spatially extended systems, direct simulations of sufficiently large
microscopic systems that inform macroscopic behavior are prohibitive. In
this work, we propose a framework that learns the macroscopic dynamics
of large stochastic microscopic systems using only small-system
simulations. Our framework employs a partial evolution scheme to
generate training data pairs by evolving large-system snapshots within
local patches. We subsequently derive the closure variables associated
with the macroscopic observables and learn the macroscopic dynamics
using a custom loss. Furthermore, we introduce a hierarchical upsampling
scheme that enables the efficient generation of large-system snapshots
from small-system snapshots. We empirically demonstrate the accuracy and
robustness of our framework through a variety of stochastic spatially
extended systems, including those described by stochastic partial
differential equations, idealized lattice spin systems, and a more
realistic NbMoTa alloy system. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AFMengyi Chen
Pengru Huang
Kostya S Novoselov
Qianxiao Li
- TIScalable learning of macroscopic stochastic dynamics
- SOPHYSICAL REVIEW MATERIALS
- DTArticle
- ABMacroscopic dynamical descriptions of complex physical systems are
crucial for understanding and controlling material behavior. With the
growing availability of data and compute, machine learning has become a
promising alternative to first-principles methods to build accurate
macroscopic models from microscopic trajectory simulations. However, for
spatially extended systems, direct simulations of sufficiently large
microscopic systems that inform macroscopic behavior are prohibitive. In
this work, we propose a framework that learns the macroscopic dynamics
of large stochastic microscopic systems using only small-system
simulations. Our framework employs a partial evolution scheme to
generate training data pairs by evolving large-system snapshots within
local patches. We subsequently derive the closure variables associated
with the macroscopic observables and learn the macroscopic dynamics
using a custom loss. Furthermore, we introduce a hierarchical upsampling
scheme that enables the efficient generation of large-system snapshots
from small-system snapshots. We empirically demonstrate the accuracy and
robustness of our framework through a variety of stochastic spatially
extended systems, including those described by stochastic partial
differential equations, idealized lattice spin systems, and a more
realistic NbMoTa alloy system. - Z90
- PUAMER PHYSICAL SOC
- PAONE PHYSICS ELLIPSE, COLLEGE PK, MD 20740-3844 USA
- SN2475-9953
- VL10
- DI10.1103/mlh4-htxv
- UTWOS:001724571000001
- ER
- EF
|
2025
|
Wu, Shiqi; Meunier, Gerard; Chadebec, Olivier; Li, Qianxiao; Chamoin, Ludovic Learning Dynamics of Nonlinear Field-Circuit Coupled Problems With a
Physics-Data Combined Model INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING, 126 (5), 2025, DOI: 10.1002/nme.70015. Abstract | BibTeX | Endnote @article{WOS:001436955800001,
title = {Learning Dynamics of Nonlinear Field-Circuit Coupled Problems With a
Physics-Data Combined Model},
author = {Shiqi Wu and Gerard Meunier and Olivier Chadebec and Qianxiao Li and Ludovic Chamoin},
doi = {10.1002/nme.70015},
times_cited = {1},
issn = {0029-5981},
year = {2025},
date = {2025-03-01},
journal = {INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING},
volume = {126},
number = {5},
publisher = {WILEY},
address = {111 RIVER ST, HOBOKEN 07030-5774, NJ USA},
abstract = {This work introduces a combined model that integrates a linear
state-space model with a Koopman-type machine-learning model to
efficiently predict the dynamics of nonlinear, high-dimensional, and
field-circuit coupled systems, as encountered in areas such as
electromagnetic compatibility, power electronics, and electric machines.
Using an extended nonintrusive model combination algorithm, the proposed
model achieves high accuracy with an error of approximately 1%,
outperforming baselines: a state-space model and a purely data-driven
model. Moreover, it delivers a computational speed-up of three orders of
magnitude compared with the traditional time-stepping volume integral
method on the same mesh in the online prediction stage, at the cost of a
one-time training effort and previously mentioned error, making it
highly effective for real-time and repeated predictions.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This work introduces a combined model that integrates a linear
state-space model with a Koopman-type machine-learning model to
efficiently predict the dynamics of nonlinear, high-dimensional, and
field-circuit coupled systems, as encountered in areas such as
electromagnetic compatibility, power electronics, and electric machines.
Using an extended nonintrusive model combination algorithm, the proposed
model achieves high accuracy with an error of approximately 1%,
outperforming baselines: a state-space model and a purely data-driven
model. Moreover, it delivers a computational speed-up of three orders of
magnitude compared with the traditional time-stepping volume integral
method on the same mesh in the online prediction stage, at the cost of a
one-time training effort and previously mentioned error, making it
highly effective for real-time and repeated predictions. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AFShiqi Wu
Gerard Meunier
Olivier Chadebec
Qianxiao Li
Ludovic Chamoin
- TILearning Dynamics of Nonlinear Field-Circuit Coupled Problems With a
Physics-Data Combined Model - SOINTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERING
- DTArticle
- ABThis work introduces a combined model that integrates a linear
state-space model with a Koopman-type machine-learning model to
efficiently predict the dynamics of nonlinear, high-dimensional, and
field-circuit coupled systems, as encountered in areas such as
electromagnetic compatibility, power electronics, and electric machines.
Using an extended nonintrusive model combination algorithm, the proposed
model achieves high accuracy with an error of approximately 1%,
outperforming baselines: a state-space model and a purely data-driven
model. Moreover, it delivers a computational speed-up of three orders of
magnitude compared with the traditional time-stepping volume integral
method on the same mesh in the online prediction stage, at the cost of a
one-time training effort and previously mentioned error, making it
highly effective for real-time and repeated predictions. - Z91
- PUWILEY
- PA111 RIVER ST, HOBOKEN 07030-5774, NJ USA
- SN0029-5981
- VL126
- DI10.1002/nme.70015
- UTWOS:001436955800001
- ER
- EF
|
Zhang, Chi; Ren, Lianhai; Cheng, Jingpu; Li, Qianxiao From Weight-Based to State-Based Fine-Tuning: Further Memory Reduction
on LoRA with Parallel Control Singh, A; Fazel, M; Hsu, D; Lacoste-Julien, S; Berkenkamp, F; Maharaj, T; Wagstaff, K; Zhu, J (Ed.): INTERNATIONAL CONFERENCE ON MACHINE LEARNING, pp. 75453-75467, JMLR-JOURNAL MACHINE LEARNING RESEARCH, 1269 LAW ST, SAN DIEGO, CA, UNITED STATES, 2025, (42nd International Conference on Machine Learning-ICML-Annual,
Vancouver, CANADA, JUL 13-19, 2025). Abstract | BibTeX | Endnote @inproceedings{WOS:001693172600013,
title = {From Weight-Based to State-Based Fine-Tuning: Further Memory Reduction
on LoRA with Parallel Control},
author = {Chi Zhang and Lianhai Ren and Jingpu Cheng and Qianxiao Li},
editor = {A Singh and M Fazel and D Hsu and S Lacoste-Julien and F Berkenkamp and T Maharaj and K Wagstaff and J Zhu},
times_cited = {0},
issn = {2640-3498},
year = {2025},
date = {2025-01-01},
booktitle = {INTERNATIONAL CONFERENCE ON MACHINE LEARNING},
volume = {267},
pages = {75453-75467},
publisher = {JMLR-JOURNAL MACHINE LEARNING RESEARCH},
address = {1269 LAW ST, SAN DIEGO, CA, UNITED STATES},
series = {Proceedings of Machine Learning Research},
abstract = {The LoRA method has achieved notable success in reducing GPU memory
usage by applying low-rank updates to weight matrices. Yet, one simple
question remains: can we push this reduction even further? Furthermore,
is it possible to achieve this while reducing computation time and
preserving performance? Answering these questions requires moving beyond
the conventional weight-centric approach. In this paper, we present a
state-based fine-tuning framework that shifts the focus from weight
adaptation to optimizing forward states, with LoRA acting as a special
example. Specifically, state-based tuning introduces parameterized
perturbations to the states within the computational graph, allowing us
to control states across an entire residual block. A key advantage of
this approach is the potential to avoid storing large intermediate
states in models like transformers. Empirical results across multiple
architectures-including ViT, RoBERTa, LLaMA2-7B, and LLaMA3-8B-show that
our method further reduces memory consumption and computation time while
preserving performance. As a result of memory reduction, we explore the
feasibility to train 7B/8B models on consumer-level GPUs like Nvidia
3090, without model quantization. The code is available here.},
note = {42nd International Conference on Machine Learning-ICML-Annual,
Vancouver, CANADA, JUL 13-19, 2025},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The LoRA method has achieved notable success in reducing GPU memory
usage by applying low-rank updates to weight matrices. Yet, one simple
question remains: can we push this reduction even further? Furthermore,
is it possible to achieve this while reducing computation time and
preserving performance? Answering these questions requires moving beyond
the conventional weight-centric approach. In this paper, we present a
state-based fine-tuning framework that shifts the focus from weight
adaptation to optimizing forward states, with LoRA acting as a special
example. Specifically, state-based tuning introduces parameterized
perturbations to the states within the computational graph, allowing us
to control states across an entire residual block. A key advantage of
this approach is the potential to avoid storing large intermediate
states in models like transformers. Empirical results across multiple
architectures-including ViT, RoBERTa, LLaMA2-7B, and LLaMA3-8B-show that
our method further reduces memory consumption and computation time while
preserving performance. As a result of memory reduction, we explore the
feasibility to train 7B/8B models on consumer-level GPUs like Nvidia
3090, without model quantization. The code is available here. - FNClarivate Analytics Web of Science
- VR1.0
- PTMisc
- AFChi Zhang
Lianhai Ren
Jingpu Cheng
Qianxiao Li
- TIFrom Weight-Based to State-Based Fine-Tuning: Further Memory Reduction
on LoRA with Parallel Control - DTInproceedings
- ABThe LoRA method has achieved notable success in reducing GPU memory
usage by applying low-rank updates to weight matrices. Yet, one simple
question remains: can we push this reduction even further? Furthermore,
is it possible to achieve this while reducing computation time and
preserving performance? Answering these questions requires moving beyond
the conventional weight-centric approach. In this paper, we present a
state-based fine-tuning framework that shifts the focus from weight
adaptation to optimizing forward states, with LoRA acting as a special
example. Specifically, state-based tuning introduces parameterized
perturbations to the states within the computational graph, allowing us
to control states across an entire residual block. A key advantage of
this approach is the potential to avoid storing large intermediate
states in models like transformers. Empirical results across multiple
architectures-including ViT, RoBERTa, LLaMA2-7B, and LLaMA3-8B-show that
our method further reduces memory consumption and computation time while
preserving performance. As a result of memory reduction, we explore the
feasibility to train 7B/8B models on consumer-level GPUs like Nvidia
3090, without model quantization. The code is available here. - Z90
- PUJMLR-JOURNAL MACHINE LEARNING RESEARCH
- PA1269 LAW ST, SAN DIEGO, CA, UNITED STATES
- SN2640-3498
- VL267
- BP75453
- EP75467
- UTWOS:001693172600013
- ER
- EF
|
Li, Qianxiao; Lin, Ting; Shen, Zuowei ON THE UNIVERSAL APPROXIMATION PROPERTY OF DEEP FULLY CONVOLUTIONAL
NEURAL NETWORKS SIAM JOURNAL ON MATHEMATICAL ANALYSIS, 57 (5), pp. 5275-5302, 2025, DOI: 10.1137/23M1570119. Abstract | BibTeX | Endnote @article{WOS:001589086500019,
title = {ON THE UNIVERSAL APPROXIMATION PROPERTY OF DEEP FULLY CONVOLUTIONAL
NEURAL NETWORKS},
author = {Qianxiao Li and Ting Lin and Zuowei Shen},
doi = {10.1137/23M1570119},
times_cited = {1},
issn = {0036-1410},
year = {2025},
date = {2025-01-01},
journal = {SIAM JOURNAL ON MATHEMATICAL ANALYSIS},
volume = {57},
number = {5},
pages = {5275-5302},
publisher = {SIAM PUBLICATIONS},
address = {3600 UNIV CITY SCIENCE CENTER, PHILADELPHIA, PA 19104-2688 USA},
abstract = {We study the approximation of shift-invariant or equivariant functions
by deep fully convolutional networks from the dynamical systems
perspective. We prove that deep residual fully convolutional networks
and their continuous-layer counterparts can achieve universal
approximation of these symmetric functions at constant channel width.
Moreover, we show that the same can be achieved by nonresidual variants
with at least two channels in each layer and convolutional kernel size
of at least 2. In addition, we show that these requirements are
necessary in the sense that networks with fewer channels or smaller
kernels fail to be universal approximators.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We study the approximation of shift-invariant or equivariant functions
by deep fully convolutional networks from the dynamical systems
perspective. We prove that deep residual fully convolutional networks
and their continuous-layer counterparts can achieve universal
approximation of these symmetric functions at constant channel width.
Moreover, we show that the same can be achieved by nonresidual variants
with at least two channels in each layer and convolutional kernel size
of at least 2. In addition, we show that these requirements are
necessary in the sense that networks with fewer channels or smaller
kernels fail to be universal approximators. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AFQianxiao Li
Ting Lin
Zuowei Shen
- TION THE UNIVERSAL APPROXIMATION PROPERTY OF DEEP FULLY CONVOLUTIONAL
NEURAL NETWORKS - SOSIAM JOURNAL ON MATHEMATICAL ANALYSIS
- DTArticle
- ABWe study the approximation of shift-invariant or equivariant functions
by deep fully convolutional networks from the dynamical systems
perspective. We prove that deep residual fully convolutional networks
and their continuous-layer counterparts can achieve universal
approximation of these symmetric functions at constant channel width.
Moreover, we show that the same can be achieved by nonresidual variants
with at least two channels in each layer and convolutional kernel size
of at least 2. In addition, we show that these requirements are
necessary in the sense that networks with fewer channels or smaller
kernels fail to be universal approximators. - Z91
- PUSIAM PUBLICATIONS
- PA3600 UNIV CITY SCIENCE CENTER, PHILADELPHIA, PA 19104-2688 USA
- SN0036-1410
- VL57
- BP5275
- EP5302
- DI10.1137/23M1570119
- UTWOS:001589086500019
- ER
- EF
|
