2024
|
Low, Andre K Y; Mekki-Berrada, Flore; Gupta, Abhishek; Ostudin, Aleksandr; Xie, Jiaxun; Vissol-Gaudin, Eleonore; Lim, Yee-Fun; Li, Qianxiao; Ong, Yew Soon; Khan, Saif A; Hippalgaonkar, Kedar Evolution-guided Bayesian optimization for constrained multi-objective optimization in self-driving labs NPJ COMPUTATIONAL MATERIALS, 10 (1), 2024, DOI: 10.1038/s41524-024-01274-x. Abstract | BibTeX | Endnote @article{ISI:001221677000002,
title = {Evolution-guided Bayesian optimization for constrained multi-objective optimization in self-driving labs},
author = {Andre K Y Low and Flore Mekki-Berrada and Abhishek Gupta and Aleksandr Ostudin and Jiaxun Xie and Eleonore Vissol-Gaudin and Yee-Fun Lim and Qianxiao Li and Yew Soon Ong and Saif A Khan and Kedar Hippalgaonkar},
doi = {10.1038/s41524-024-01274-x},
times_cited = {0},
year = {2024},
date = {2024-05-13},
journal = {NPJ COMPUTATIONAL MATERIALS},
volume = {10},
number = {1},
publisher = {NATURE PORTFOLIO},
address = {HEIDELBERGER PLATZ 3, BERLIN, 14197, GERMANY},
abstract = {The development of automated high-throughput experimental platforms has enabled fast sampling of high-dimensional decision spaces. To reach target properties efficiently, these platforms are increasingly paired with intelligent experimental design. However, current optimizers show limitations in maintaining sufficient exploration/exploitation balance for problems dealing with multiple conflicting objectives and complex constraints. Here, we devise an Evolution-Guided Bayesian Optimization (EGBO) algorithm that integrates selection pressure in parallel with a q-Noisy Expected Hypervolume Improvement (qNEHVI) optimizer; this not only solves for the Pareto Front (PF) efficiently but also achieves better coverage of the PF while limiting sampling in the infeasible space. The algorithm is developed together with a custom self-driving lab for seed-mediated silver nanoparticle synthesis, targeting 3 objectives (1) optical properties, (2) fast reaction, and (3) minimal seed usage alongside complex constraints. We demonstrate that, with appropriate constraint handling, EGBO performance improves upon state-of-the-art qNEHVI. Furthermore, across various synthetic multi-objective problems, EGBO shows significative hypervolume improvement, revealing the synergy between selection pressure and the qNEHVI optimizer. We also demonstrate EGBO's good coverage of the PF as well as comparatively better ability to propose feasible solutions. We thus propose EGBO as a general framework for efficiently solving constrained multi-objective problems in high-throughput experimentation platforms.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The development of automated high-throughput experimental platforms has enabled fast sampling of high-dimensional decision spaces. To reach target properties efficiently, these platforms are increasingly paired with intelligent experimental design. However, current optimizers show limitations in maintaining sufficient exploration/exploitation balance for problems dealing with multiple conflicting objectives and complex constraints. Here, we devise an Evolution-Guided Bayesian Optimization (EGBO) algorithm that integrates selection pressure in parallel with a q-Noisy Expected Hypervolume Improvement (qNEHVI) optimizer; this not only solves for the Pareto Front (PF) efficiently but also achieves better coverage of the PF while limiting sampling in the infeasible space. The algorithm is developed together with a custom self-driving lab for seed-mediated silver nanoparticle synthesis, targeting 3 objectives (1) optical properties, (2) fast reaction, and (3) minimal seed usage alongside complex constraints. We demonstrate that, with appropriate constraint handling, EGBO performance improves upon state-of-the-art qNEHVI. Furthermore, across various synthetic multi-objective problems, EGBO shows significative hypervolume improvement, revealing the synergy between selection pressure and the qNEHVI optimizer. We also demonstrate EGBO's good coverage of the PF as well as comparatively better ability to propose feasible solutions. We thus propose EGBO as a general framework for efficiently solving constrained multi-objective problems in high-throughput experimentation platforms. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AULow, AKY
Mekki-Berrada, F
Gupta, A
Ostudin, A
Xie, JX
Vissol-Gaudin, E
Lim, YF
Li, QX
Ong, YS
Khan, SA
Hippalgaonkar, K
- AFAndre K Y Low
Flore Mekki-Berrada
Abhishek Gupta
Aleksandr Ostudin
Jiaxun Xie
Eleonore Vissol-Gaudin
Yee-Fun Lim
Qianxiao Li
Yew Soon Ong
Saif A Khan
Kedar Hippalgaonkar
- TIEvolution-guided Bayesian optimization for constrained multi-objective optimization in self-driving labs
- SONPJ COMPUTATIONAL MATERIALS
- LAEnglish
- DTArticle
- IDGAUSSIAN-PROCESSES
- ABThe development of automated high-throughput experimental platforms has enabled fast sampling of high-dimensional decision spaces. To reach target properties efficiently, these platforms are increasingly paired with intelligent experimental design. However, current optimizers show limitations in maintaining sufficient exploration/exploitation balance for problems dealing with multiple conflicting objectives and complex constraints. Here, we devise an Evolution-Guided Bayesian Optimization (EGBO) algorithm that integrates selection pressure in parallel with a q-Noisy Expected Hypervolume Improvement (qNEHVI) optimizer; this not only solves for the Pareto Front (PF) efficiently but also achieves better coverage of the PF while limiting sampling in the infeasible space. The algorithm is developed together with a custom self-driving lab for seed-mediated silver nanoparticle synthesis, targeting 3 objectives (1) optical properties, (2) fast reaction, and (3) minimal seed usage alongside complex constraints. We demonstrate that, with appropriate constraint handling, EGBO performance improves upon state-of-the-art qNEHVI. Furthermore, across various synthetic multi-objective problems, EGBO shows significative hypervolume improvement, revealing the synergy between selection pressure and the qNEHVI optimizer. We also demonstrate EGBO's good coverage of the PF as well as comparatively better ability to propose feasible solutions. We thus propose EGBO as a general framework for efficiently solving constrained multi-objective problems in high-throughput experimentation platforms.
- C1[Low, Andre K. Y.; Vissol-Gaudin, Eleonore; Hippalgaonkar, Kedar] Nanyang Technol Univ, Sch Mat Sci & Engn, Singapore 639798, Singapore.
[Low, Andre K. Y.; Lim, Yee-Fun; Hippalgaonkar, Kedar] ASTAR, Inst Mat Res & Engn IMRE, 2 Fusionopolis Way,Innovis 08-03, Singapore 138634, Singapore. [Mekki-Berrada, Flore; Ostudin, Aleksandr; Xie, Jiaxun; Khan, Saif A.] Natl Univ Singapore, Dept Chem & Biomol Engn, Singapore 117585, Singapore. [Gupta, Abhishek] Indian Inst Technol Goa, Sch Mech Sci, Ponda 403401, Goa, India. [Lim, Yee-Fun] ASTAR, Inst Sustainabil Chem Energy & Environm ISCE2, 1 Pesek Rd, Singapore 627833, Singapore. [Li, Qianxiao] Natl Univ Singapore, Dept Math, Singapore 119077, Singapore. [Li, Qianxiao] Natl Univ Singapore, Inst Funct Intelligent Mat, Singapore 117544, Singapore. [Ong, Yew Soon] Nanyang Technol Univ, Sch Comp Sci & Engn, Singapore 639798, Singapore - C3Nanyang Technological University; Agency for Science Technology & Research (A*STAR); A*STAR - Institute of Materials Research & Engineering (IMRE); National University of Singapore; Indian Institute of Technology System (IIT System); Indian Institute of Technology (IIT) Goa; Agency for Science Technology & Research (A*STAR); National University of Singapore; National University of Singapore; Institute for Functional Intelligent Materials (I-FIM); Nanyang Technological University
- RPHippalgaonkar, K (corresponding author), Nanyang Technol Univ, Sch Mat Sci & Engn, Singapore 639798, Singapore; Hippalgaonkar, K (corresponding author), ASTAR, Inst Mat Res & Engn IMRE, 2 Fusionopolis Way,Innovis 08-03, Singapore 138634, Singapore; Khan, SA (corresponding author), Natl Univ Singapore, Dept Chem & Biomol Engn, Singapore 117585, Singapore
- FUAME Programmatic Funds by the Agency for Science, Technology and Research [A1898b0043, A20G9b0135]; National Research Foundation (NRF), Singapore [NRF- NRFF13-2021-0005]; The 25th NRF CRP programme [EDUNC-33-18-279-V12]; NRF fellowship [NRF-NRFF13-2021-0011]; Ministry of Education, Singapore
- FXThe authors acknowledge funding from AME Programmatic Funds by the Agency for Science, Technology and Research under Grant No. A1898b0043 and No. A20G9b0135. KH also acknowledges funding from the National Research Foundation (NRF), Singapore under the NRF Fellowship (NRF-NRFF13-2021-0011). SAK and FMB also acknowledge funding from the 25th NRF CRP programme (NRF-CRP25-2020RS-0002). QL also acknowledges support from the NRF fellowship (project No. NRF- NRFF13-2021-0005) and the Ministry of Education, Singapore, under its Research Centre of Excellence award to the Institute for Functional Intelligent Materials (I-FIM, project No. EDUNC-33-18-279-V12).
- NR51
- TC0
- Z90
- U12
- U22
- PUNATURE PORTFOLIO
- PIBERLIN
- PAHEIDELBERGER PLATZ 3, BERLIN, 14197, GERMANY
- J9NPJ COMPUT MATER
- JInpj Comput. Mater.
- PDMAY 13
- PY2024
- VL10
- DI10.1038/s41524-024-01274-x
- PG11
- WCChemistry, Physical; Materials Science, Multidisciplinary
- SCChemistry; Materials Science
- GAQN9T5
- UTWOS:001221677000002
- ER
- EF
|
Wu, Shiqi; Chamoin, Ludovic; Li, Qianxiao Non-intrusive model combination for learning dynamical systems PHYSICA D-NONLINEAR PHENOMENA, 463 , 2024, DOI: 10.1016/j.physd.2024.134152. Abstract | BibTeX | Endnote @article{ISI:001235225600001,
title = {Non-intrusive model combination for learning dynamical systems},
author = {Shiqi Wu and Ludovic Chamoin and Qianxiao Li},
doi = {10.1016/j.physd.2024.134152},
times_cited = {0},
issn = {0167-2789},
year = {2024},
date = {2024-04-27},
journal = {PHYSICA D-NONLINEAR PHENOMENA},
volume = {463},
publisher = {ELSEVIER},
address = {RADARWEG 29, 1043 NX AMSTERDAM, NETHERLANDS},
abstract = {In data -driven modeling of complex dynamic processes, it is often desirable to combine different classes of models to enhance performance. Examples include coupled models of different fidelities, or hybrid models based on physical knowledge and data -driven strategies. A key limitation of the broad adoption of model combination in applications is intrusiveness: training combined models typically requires significant modifications to the learning algorithm implementations, which may often be already well -developed and optimized for individual model spaces. In this work, we propose an iterative, non -intrusive methodology to combine two model spaces to learn dynamics from data. We show that this can be understood, at least in the linear setting, as finding the optimal solution in the direct sum of the two hypothesis spaces, while leveraging only the projection operators in each individual space. Hence, the proposed algorithm can be viewed as iterative projections, for which we can obtain estimates on its convergence properties. To highlight the extensive applicability of our framework, we conduct numerical experiments in various problem settings, with particular emphasis on various hybrid models based on the Koopman operator approach.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In data -driven modeling of complex dynamic processes, it is often desirable to combine different classes of models to enhance performance. Examples include coupled models of different fidelities, or hybrid models based on physical knowledge and data -driven strategies. A key limitation of the broad adoption of model combination in applications is intrusiveness: training combined models typically requires significant modifications to the learning algorithm implementations, which may often be already well -developed and optimized for individual model spaces. In this work, we propose an iterative, non -intrusive methodology to combine two model spaces to learn dynamics from data. We show that this can be understood, at least in the linear setting, as finding the optimal solution in the direct sum of the two hypothesis spaces, while leveraging only the projection operators in each individual space. Hence, the proposed algorithm can be viewed as iterative projections, for which we can obtain estimates on its convergence properties. To highlight the extensive applicability of our framework, we conduct numerical experiments in various problem settings, with particular emphasis on various hybrid models based on the Koopman operator approach. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AUWu, SQ
Chamoin, L
Li, QX
- AFShiqi Wu
Ludovic Chamoin
Qianxiao Li
- TINon-intrusive model combination for learning dynamical systems
- SOPHYSICA D-NONLINEAR PHENOMENA
- LAEnglish
- DTArticle
- DELearning Dynamics; Model Combination; Machine Learning; Koopman Operator; Iterative Projection
- IDKOOPMAN OPERATOR
- ABIn data -driven modeling of complex dynamic processes, it is often desirable to combine different classes of models to enhance performance. Examples include coupled models of different fidelities, or hybrid models based on physical knowledge and data -driven strategies. A key limitation of the broad adoption of model combination in applications is intrusiveness: training combined models typically requires significant modifications to the learning algorithm implementations, which may often be already well -developed and optimized for individual model spaces. In this work, we propose an iterative, non -intrusive methodology to combine two model spaces to learn dynamics from data. We show that this can be understood, at least in the linear setting, as finding the optimal solution in the direct sum of the two hypothesis spaces, while leveraging only the projection operators in each individual space. Hence, the proposed algorithm can be viewed as iterative projections, for which we can obtain estimates on its convergence properties. To highlight the extensive applicability of our framework, we conduct numerical experiments in various problem settings, with particular emphasis on various hybrid models based on the Koopman operator approach.
- C3National University of Singapore; National University of Singapore; Institute for Functional Intelligent Materials (I-FIM); Universite Paris Saclay; Universite Paris Cite; Centre National de la Recherche Scientifique (CNRS); Institut Universitaire de France
- RPLi, QX (corresponding author), Natl Univ Singapore, Dept Math, Singapore 117543, Singapore; Li, QX (corresponding author), Natl Univ Singapore, Inst Funct Intelligent Mat, Singapore 117543, Singapore
- FXThis research is part of the programme DesCartes and is supported by the National Research Foundation, Prime Minister's Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme.
- NR41
- TC0
- Z90
- U12
- U22
- PUELSEVIER
- PIAMSTERDAM
- PARADARWEG 29, 1043 NX AMSTERDAM, NETHERLANDS
- SN0167-2789
- J9PHYSICA D
- JIPhysica D
- PDJUL
- PY2024
- VL463
- DI10.1016/j.physd.2024.134152
- PG15
- WCMathematics, Applied; Physics, Fluids & Plasmas; Physics, Multidisciplinary; Physics, Mathematical
- SCMathematics; Physics
- GASN8P5
- UTWOS:001235225600001
- ER
- EF
|
2023
|
Chen, Xiaoli; Soh, Beatrice W; Ooi, Zi-En; Vissol-Gaudin, Eleonore; Yu, Haijun; Novoselov, Kostya S; Hippalgaonkar, Kedar; Li, Qianxiao Constructing custom thermodynamics using deep learning NATURE COMPUTATIONAL SCIENCE, 4 (1), 2023, DOI: 10.1038/s43588-023-00581-5. Abstract | BibTeX | Endnote @article{ISI:001133735500001,
title = {Constructing custom thermodynamics using deep learning},
author = {Xiaoli Chen and Beatrice W Soh and Zi-En Ooi and Eleonore Vissol-Gaudin and Haijun Yu and Kostya S Novoselov and Kedar Hippalgaonkar and Qianxiao Li},
doi = {10.1038/s43588-023-00581-5},
times_cited = {0},
year = {2023},
date = {2023-12-29},
journal = {NATURE COMPUTATIONAL SCIENCE},
volume = {4},
number = {1},
publisher = {SPRINGERNATURE},
address = {CAMPUS, 4 CRINAN ST, LONDON, N1 9XW, ENGLAND},
abstract = {One of the most exciting applications of artificial intelligence is automated scientific discovery based on previously amassed data, coupled with restrictions provided by known physical principles, including symmetries and conservation laws. Such automated hypothesis creation and verification can assist scientists in studying complex phenomena, where traditional physical intuition may fail. Here we develop a platform based on a generalized Onsager principle to learn macroscopic dynamical descriptions of arbitrary stochastic dissipative systems directly from observations of their microscopic trajectories. Our method simultaneously constructs reduced thermodynamic coordinates and interprets the dynamics on these coordinates. We demonstrate its effectiveness by studying theoretically and validating experimentally the stretching of long polymer chains in an externally applied field. Specifically, we learn three interpretable thermodynamic coordinates and build a dynamical landscape of polymer stretching, including the identification of stable and transition states and the control of the stretching rate. Our general methodology can be used to address a wide range of scientific and technological applications.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
One of the most exciting applications of artificial intelligence is automated scientific discovery based on previously amassed data, coupled with restrictions provided by known physical principles, including symmetries and conservation laws. Such automated hypothesis creation and verification can assist scientists in studying complex phenomena, where traditional physical intuition may fail. Here we develop a platform based on a generalized Onsager principle to learn macroscopic dynamical descriptions of arbitrary stochastic dissipative systems directly from observations of their microscopic trajectories. Our method simultaneously constructs reduced thermodynamic coordinates and interprets the dynamics on these coordinates. We demonstrate its effectiveness by studying theoretically and validating experimentally the stretching of long polymer chains in an externally applied field. Specifically, we learn three interpretable thermodynamic coordinates and build a dynamical landscape of polymer stretching, including the identification of stable and transition states and the control of the stretching rate. Our general methodology can be used to address a wide range of scientific and technological applications. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AUChen, XL
Soh, BW
Ooi, ZE
Vissol-Gaudin, E
Yu, HJ
Novoselov, KS
Hippalgaonkar, K
Li, QX
- AFXiaoli Chen
Beatrice W Soh
Zi-En Ooi
Eleonore Vissol-Gaudin
Haijun Yu
Kostya S Novoselov
Kedar Hippalgaonkar
Qianxiao Li
- TIConstructing custom thermodynamics using deep learning
- SONATURE COMPUTATIONAL SCIENCE
- LAEnglish
- DTArticle
- IDPROFESSIONAL SPORTS; FOOTBALL CLUBS; BROWNIAN DYNAMICS; NEURAL-NETWORKS; IRREVERSIBLE-PROCESSES; FINANCIAL PERFORMANCE; DISCRIMINANT-ANALYSIS; UTILITY MAXIMIZATION; RECIPROCAL RELATIONS; COMPETITIVE BALANCE
- ABOne of the most exciting applications of artificial intelligence is automated scientific discovery based on previously amassed data, coupled with restrictions provided by known physical principles, including symmetries and conservation laws. Such automated hypothesis creation and verification can assist scientists in studying complex phenomena, where traditional physical intuition may fail. Here we develop a platform based on a generalized Onsager principle to learn macroscopic dynamical descriptions of arbitrary stochastic dissipative systems directly from observations of their microscopic trajectories. Our method simultaneously constructs reduced thermodynamic coordinates and interprets the dynamics on these coordinates. We demonstrate its effectiveness by studying theoretically and validating experimentally the stretching of long polymer chains in an externally applied field. Specifically, we learn three interpretable thermodynamic coordinates and build a dynamical landscape of polymer stretching, including the identification of stable and transition states and the control of the stretching rate. Our general methodology can be used to address a wide range of scientific and technological applications.
- C3National University of Singapore; National University of Singapore; Institute for Functional Intelligent Materials (I-FIM); Agency for Science Technology & Research (A*STAR); A*STAR - Institute of Materials Research & Engineering (IMRE); Nanyang Technological University; Chinese Academy of Sciences; Academy of Mathematics & System Sciences, CAS; Chinese Academy of Sciences; Chinese Academy of Sciences; University of Chinese Academy of Sciences, CAS
- RPLi, QX (corresponding author), Natl Univ Singapore, Dept Math, Singapore, Singapore; Novoselov, KS (corresponding author), Natl Univ Singapore, Inst Funct Intelligent Mat, Singapore, Singapore; Hippalgaonkar, K (corresponding author), ASTAR, Inst Mat Res & Engn, Singapore, Singapore; Hippalgaonkar, K (corresponding author), Nanyang Technol Univ, Sch Mat Sci & Engn, Singapore, Singapore
- FXThis research is supported by the Ministry of Education, Singapore, under its Research Centre of Excellence award to the Institute for Functional Intelligent Materials (I-FIM, project no. EDUNC-33-18-279-V12 KSN). K.S.N. is grateful to the Royal Society (UK, grant number RSRP R 190000 KSN) for support. Q.L. acknowledges support from the National Research Foundation, Singapore, under the NRF fellowship (project no. NRF-NRFF13-2021-0005 QL). H.Y. acknowledges support from the National Natural Science Foundation of China under Grant No. 12171467 HY and 12161141017 HY. K.H., B.W.S. and Z.-E.O. acknowledge support from the Accelerated Materials Development for Manufacturing Program at A*STAR via the AME Programmatic Fund by the Agency for Science, Technology and Research under Grant No. A1898b0043 KH. The computational work for this article was partially performed on resources of the National Supercomputing Centre, Singapore. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We thank C. Jie Leong for help with experimental preparation.
- NR140
- TC0
- Z90
- U17
- U27
- PUSPRINGERNATURE
- PILONDON
- PACAMPUS, 4 CRINAN ST, LONDON, N1 9XW, ENGLAND
- J9NAT COMPUT SCI
- JINat. Comput. Sci.
- PDJAN
- PY2024
- VL4
- DI10.1038/s43588-023-00581-5
- PG26
- WCComputer Science, Interdisciplinary Applications; Computer Science, Theory & Methods; Multidisciplinary Sciences
- SCComputer Science; Science & Technology - Other Topics
- GAGC8G9
- UTWOS:001133735500001
- ER
- EF
|
Bao, Chenglong; Li, Qianxiao; Shen, Zuowei; Tai, Cheng; Wu, Lei; Xiang, Xueshuang Approximation Analysis of Convolutional Neural Networks EAST ASIAN JOURNAL ON APPLIED MATHEMATICS, 13 (3), pp. 524-549, 2023, DOI: 10.4208/eajam.2022-270.070123. Abstract | BibTeX | Endnote @article{ISI:001016455600005,
title = {Approximation Analysis of Convolutional Neural Networks},
author = {Chenglong Bao and Qianxiao Li and Zuowei Shen and Cheng Tai and Lei Wu and Xueshuang Xiang},
doi = {10.4208/eajam.2022-270.070123},
times_cited = {2},
issn = {2079-7362},
year = {2023},
date = {2023-08-01},
journal = {EAST ASIAN JOURNAL ON APPLIED MATHEMATICS},
volume = {13},
number = {3},
pages = {524-549},
publisher = {GLOBAL SCIENCE PRESS},
address = {Office B, 9/F, Kings Wing Plaza2, No.1 On Kwan St, Shek Mun, NT, Hong Kong, 00000, PEOPLES R CHINA},
abstract = {In its simplest form, convolution neural networks (CNNs) consist of a fully connected two-layer network g composed with a sequence of convolution layers T. Al-though g is known to have the universal approximation property, it is not known if CNNs, which have the form g degrees T inherit this property, especially when the kernel size in T is small. In this paper, we show that under suitable conditions, CNNs do inherit the universal approximation property and its sample complexity can be characterized. In ad-dition, we discuss concretely how the nonlinearity of T can improve the approximation power. Finally, we show that when the target function class has a certain compositional form, convolutional networks are far more advantageous compared with fully connected networks, in terms of the number of parameters needed to achieve the desired accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In its simplest form, convolution neural networks (CNNs) consist of a fully connected two-layer network g composed with a sequence of convolution layers T. Al-though g is known to have the universal approximation property, it is not known if CNNs, which have the form g degrees T inherit this property, especially when the kernel size in T is small. In this paper, we show that under suitable conditions, CNNs do inherit the universal approximation property and its sample complexity can be characterized. In ad-dition, we discuss concretely how the nonlinearity of T can improve the approximation power. Finally, we show that when the target function class has a certain compositional form, convolutional networks are far more advantageous compared with fully connected networks, in terms of the number of parameters needed to achieve the desired accuracy. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AUBao, CL
Li, QX
Shen, ZW
Tai, C
Wu, L
Xiang, XS
- AFChenglong Bao
Qianxiao Li
Zuowei Shen
Cheng Tai
Lei Wu
Xueshuang Xiang
- TIApproximation Analysis of Convolutional Neural Networks
- SOEAST ASIAN JOURNAL ON APPLIED MATHEMATICS
- LAEnglish
- DTArticle
- DEConvolutional Networks; Approximation; Scaling Analysis; Compositional Functions
- IDDEEP; BOUNDS
- ABIn its simplest form, convolution neural networks (CNNs) consist of a fully connected two-layer network g composed with a sequence of convolution layers T. Al-though g is known to have the universal approximation property, it is not known if CNNs, which have the form g degrees T inherit this property, especially when the kernel size in T is small. In this paper, we show that under suitable conditions, CNNs do inherit the universal approximation property and its sample complexity can be characterized. In ad-dition, we discuss concretely how the nonlinearity of T can improve the approximation power. Finally, we show that when the target function class has a certain compositional form, convolutional networks are far more advantageous compared with fully connected networks, in terms of the number of parameters needed to achieve the desired accuracy.
- C1[Bao, Chenglong] JingZhai Tsinghua Univ, Yau Math Sci Ctr, Beijing 100084, Peoples R China.
[Bao, Chenglong] YanQi Lake Beijing Inst Math Sci & Applicat, 544 HeFangkou Village, Beijing 101408, Peoples R China. [Li, Qianxiao; Shen, Zuowei] Natl Univ Singapore, Dept Math, BLK S17,10 Lower Kent Ridge Rd, Singapore 119076, Singapore. [Li, Qianxiao] Natl Univ Singapore, Inst Funct Intelligent Mat, BLK S9,4 Sci Dr 2, Singapore 117544, Singapore. [Tai, Cheng] Peking Univ, Beijing Inst Big Data Res, 6 Jingyuan, Beijing 100871, Peoples R China. [Wu, Lei] Peking Univ, Sch Math Sci, Beijing 100871, Peoples R China. [Xiang, Xueshuang] Qian Xuesen Lab Space Technol, 104 Youyi Rd, Beijing 100094, Peoples R China - C3Yanqi Lake Beijing Institute of Mathematical Sciences & Applications; National University of Singapore; Institute for Functional Intelligent Materials (I-FIM); National University of Singapore; Peking University; Peking University
- RPBao, CL (corresponding author), JingZhai Tsinghua Univ, Yau Math Sci Ctr, Beijing 100084, Peoples R China; Bao, CL (corresponding author), YanQi Lake Beijing Inst Math Sci & Applicat, 544 HeFangkou Village, Beijing 101408, Peoples R China
- FUNational Key Ramp;D Program of China [2021YFA1001300]; National Natural Science Foundation of China [12271291]; Tsinghua University Initiative Scientific Research Program [NRF-NRFF13-2021- 0005]; National Research Foundation, Singapore, under the NRF fellowship; Distinguished Professorship, National University of Singapore
- FXCL Bao is supported by the National Key R & D Program of China (No. 2021YFA1001300) , by the National Natural Science Foundation of China (No. 12271291) , and the Tsinghua University Initiative Scientific Research Program. QX Li is supported by the National Research Foundation, Singapore, under the NRF fellowship (Project No. NRF-NRFF13-2021- 0005) . ZW Shen is supported by the Distinguished Professorship, National University of Singapore.
- NR41
- TC2
- Z92
- U11
- U24
- PUGLOBAL SCIENCE PRESS
- PIHong Kong
- PAOffice B, 9/F, Kings Wing Plaza2, No.1 On Kwan St, Shek Mun, NT, Hong Kong, 00000, PEOPLES R CHINA
- SN2079-7362
- J9EAST ASIAN J APPL MATH
- JIEast Asian J. Appl. Math.
- PDAUG
- PY2023
- VL13
- BP524
- EP549
- DI10.4208/eajam.2022-270.070123
- PG26
- WCMathematics, Applied
- SCMathematics
- UTWOS:001016455600005
- ER
- EF
|
Siemenn, Alexander E; Ren, Zekun; Li, Qianxiao; Buonassisi, Tonio Fast Bayesian optimization of Needle-in-a-Haystack problems using zooming memory-based initialization (ZoMBI) NPJ COMPUTATIONAL MATERIALS, 9 (1), 2023, DOI: 10.1038/s41524-023-01048-x. Abstract | BibTeX | Endnote @article{ISI:000995481000001,
title = {Fast Bayesian optimization of Needle-in-a-Haystack problems using zooming memory-based initialization (ZoMBI)},
author = {Alexander E Siemenn and Zekun Ren and Qianxiao Li and Tonio Buonassisi},
doi = {10.1038/s41524-023-01048-x},
times_cited = {9},
year = {2023},
date = {2023-05-26},
journal = {NPJ COMPUTATIONAL MATERIALS},
volume = {9},
number = {1},
publisher = {NATURE PORTFOLIO},
address = {HEIDELBERGER PLATZ 3, BERLIN, 14197, GERMANY},
abstract = {Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI, that builds on conventional Bayesian optimization principles to quickly and efficiently optimize Needle-in-a-Haystack problems in both less time and fewer experiments. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering optima in under 100 experiments that are up to 3x more highly optimized than those discovered by similar methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI, that builds on conventional Bayesian optimization principles to quickly and efficiently optimize Needle-in-a-Haystack problems in both less time and fewer experiments. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering optima in under 100 experiments that are up to 3x more highly optimized than those discovered by similar methods. - FNClarivate Analytics Web of Science
- VR1.0
- PTJ
- AUSiemenn, AE
Ren, ZK
Li, QX
Buonassisi, T
- AFAlexander E Siemenn
Zekun Ren
Qianxiao Li
Tonio Buonassisi
- TIFast Bayesian optimization of Needle-in-a-Haystack problems using zooming memory-based initialization (ZoMBI)
- SONPJ COMPUTATIONAL MATERIALS
- LAEnglish
- DTArticle
- IDTHERMOELECTRIC GENERATOR; OUTPUT
- ABNeedle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI, that builds on conventional Bayesian optimization principles to quickly and efficiently optimize Needle-in-a-Haystack problems in both less time and fewer experiments. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering optima in under 100 experiments that are up to 3x more highly optimized than those discovered by similar methods.
- C1[Siemenn, Alexander E.; Buonassisi, Tonio] MIT, Dept Mech Engn, Cambridge, MA 02139 USA.
[Ren, Zekun] Singapore MIT Alliance Res & Technol, Dept Elect & Comp Engn, Singapore, Singapore. [Ren, Zekun] Xinterra, Singapore, Singapore. [Li, Qianxiao] Natl Univ Singapore, Dept Math, Singapore, Singapore. [Li, Qianxiao] Natl Univ Singapore, Inst Funct Intelligent Mat, Singapore, Singapore - C3Massachusetts Institute of Technology (MIT); Singapore-MIT Alliance for Research & Technology Centre (SMART); National University of Singapore; Institute for Functional Intelligent Materials (I-FIM); National University of Singapore
- RPSiemenn, AE (corresponding author), MIT, Dept Mech Engn, Cambridge, MA 02139 USA
- FUU.S. Department of Energy's Office of Energy Efficiency and Renewable Energy (EERE) under the Solar Energy Technology Office (SETO) [DE-EE0009366]; National Research Foundation, Singapore [NRF-NRFF13-2021-0005]; Ministry of Education, Singapore, under its Research Centre of Excellence award [EDUNC-33-18-279-V12]
- FXBasita Das is thanked for help in naming the algorithm. Xiaonan Wang is thanked for initial discussions for this study. John Dagdelen, Hongbin Zhang, and Shyam Dwaraknath are thanked for discussion of and reference to different Needle-in-a-Haystack problems within materials science. We acknowledge the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing HPC resources that have contributed to the research results reported within this paper. A.E.S acknowledges support from the U.S. Department of Energy's Office of Energy Efficiency and Renewable Energy (EERE) under the Solar Energy Technology Office (SETO) award number DE-EE0009366. Q.L. acknowledges support from the National Research Foundation, Singapore (project No. NRF-NRFF13-2021-0005) and the Ministry of Education, Singapore, under its Research Centre of Excellence award to I-FIM (project No. EDUNC-33-18-279-V12).
- NR72
- TC9
- Z99
- U10
- U23
- PUNATURE PORTFOLIO
- PIBERLIN
- PAHEIDELBERGER PLATZ 3, BERLIN, 14197, GERMANY
- J9NPJ COMPUT MATER
- JInpj Comput. Mater.
- PDMAY 26
- PY2023
- VL9
- DI10.1038/s41524-023-01048-x
- PG13
- WCChemistry, Physical; Materials Science, Multidisciplinary
- SCChemistry; Materials Science
- GAH4DG5
- UTWOS:000995481000001
- ER
- EF
|