Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

In this paper, we propose and compare two novel deep generative model-based approaches for the design representation, reconstruction, and generation of porous metamaterials characterized by complex and fully connected solid and pore networks. A highly diverse porous metamaterial database is curated, with each sample represented by solid and pore phase graphs and a voxel image. All metamaterial samples adhere to the requirement of complete connectivity in both pore and solid phases. The first approach employs a dual decoder variational graph autoencoder to generate both solid phase and pore phase graphs. The second approach employs a variational graph autoencoder for reconstructing/generating the nodes in the solid phase and pore phase graphs and a transformer-based large language model (LLM) for reconstructing/generating the connections, i.e., the edges among the nodes. A comparative study was conducted, and we found that both approaches achieved high accuracy in reconstructing node features, while the LLM exhibited superior performance in reconstructing edge features. Reconstruction accuracy is also validated by voxel-to-voxel comparison between the reconstructions and the original images in the test set. Additionally, discussions on the advantages and limitations of using LLMs in metamaterial design generation, along with the rationale behind their utilization, are provided.

References

1.
Zheng
,
X.
,
Lee
,
H.
,
Weisgraber
,
T. H.
,
Shusteff
,
M.
,
DeOtte
,
J.
,
Duoss
,
E. B.
,
Kuntz
,
J. D.
,
Biener
,
M. M.
,
Ge
,
Q.
, and
Jackson
,
J. A.
,
2014
, “
Ultralight, Ultrastiff Mechanical Metamaterials
,”
Science
,
344
(
6190
), pp.
1373
1377
.
2.
Chen
,
H.
, and
Chan
,
C. T.
,
2007
, “
Acoustic Cloaking in Three Dimensions Using Acoustic Metamaterials
,”
Appl. Phys. Lett.
,
91
(
18
), p.
183518
.
3.
Garland
,
A. P.
,
Adstedt
,
K. M.
,
Casias
,
Z. J.
,
White
,
B. C.
,
Mook
,
W. M.
,
Kaehr
,
B.
,
Jared
,
B. H.
,
Lester
,
B. T.
,
Leathe
,
N. S.
, and
Schwaller
,
E.
,
2020
, “
Coulombic Friction in Metamaterials to Dissipate Mechanical Energy
,”
Extreme Mech. Lett.
,
40
, p.
100847
.
4.
Claeys
,
C.
,
de Melo Filho
,
N. G. R.
,
Van Belle
,
L.
,
Deckers
,
E.
, and
Desmet
,
W.
,
2017
, “
Design and Validation of Metamaterials for Multiple Structural Stop Bands in Waveguides
,”
Extreme Mech. Lett.
,
12
, pp.
7
22
.
5.
Qian
,
J.
,
Cheng
,
Y.
,
Zhang
,
A.
,
Zhou
,
Q.
, and
Zhang
,
J.
,
2021
, “
Optimization Design of Metamaterial Vibration Isolator With Honeycomb Structure Based on Multi-fidelity Surrogate Model
,”
Struct. Multidiscipl. Optim.
,
64
(
1
), pp.
423
439
.
6.
Wang
,
Z.
,
Xian
,
W.
,
Baccouche
,
M. R.
,
Lanzerath
,
H.
,
Li
,
Y.
, and
Xu
,
H.
,
2022
, “
Design of Phononic Bandgap Metamaterials Based on Gaussian Mixture Beta Variational Autoencoder and Iterative Model Updating
,”
ASME J. Mech. Des.
,
144
(
4
), p.
041705
.
7.
Wang
,
Z.
,
Xian
,
W.
,
Baccouche
,
M. R.
,
Lanzerath
,
H.
,
Li
,
Y.
, and
Xu
,
H.
,
2021
, “
A Gaussian Mixture Variational Autoencoder-Based Approach for Designing Phononic Bandgap Metamaterials
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Virtual, Online
,
Aug. 17–19
,
American Society of Mechanical Engineers
, DETC2021-67629, V03BT03A002.
8.
Wang
,
Z.
,
Zhuang
,
R.
,
Xian
,
W.
,
Tian
,
J.
,
Li
,
Y.
,
Chen
,
S.
, and
Xu
,
H.
,
2022
, “
Phononic Metamaterial Design Via Transfer Learning-Based Topology Optimization Framework
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
St. Louis, MO
,
Aug. 14–17
,
American Society of Mechanical Engineers
, DETC2022-89932, V03AT03A048.
9.
Gurbuz
,
C.
,
Kronowetter
,
F.
,
Dietz
,
C.
,
Eser
,
M.
,
Schmid
,
J.
, and
Marburg
,
S.
,
2021
, “
Generative Adversarial Networks for the Design of Acoustic Metamaterials
,”
J. Acoust. Soc. Am.
,
149
(
2
), p.
1162
1174
.
10.
Alberdi
,
R.
,
Dingreville
,
R.
,
Robbins
,
J.
,
Walsh
,
T.
,
White
,
B. C.
,
Jared
,
B.
, and
Boyce
,
B. L.
,
2020
, “
Multi-morphology Lattices Lead to Improved Plastic Energy Absorption
,”
Mater. Des.
,
194
, p.
108883
.
11.
Xu
,
H.
, and
Liu
,
Z.
,
2019
, “
Control Variate Multifidelity Estimators for the Variance and Sensitivity Analysis of Mesostructure–Structure Systems
,”
ASCE-ASME J. Risk Uncertain. Eng. Syst. B: Mech. Eng.
,
5
(
2
), p.
020907
.
12.
Liu
,
Z.
,
Xu
,
H.
, and
Zhu
,
P.
,
2020
, “
An Adaptive Multi-fidelity Approach for Design Optimization of Mesostructure-Structure Systems
,”
Struct. Multidiscipl. Optim.
,
62
(
1
), pp.
375
386
.
13.
Zhang
,
Q.
,
Zhang
,
K.
, and
Hu
,
G.
,
2018
, “
Tunable Fluid-Solid Metamaterials for Manipulation of Elastic Wave Propagation in Broad Frequency Range
,”
Appl. Phys. Lett.
,
112
(
22
), p.
221906
.
14.
He
,
Z.-H.
,
Wang
,
Y.-Z.
, and
Wang
,
Y.-S.
,
2021
, “
Active Feedback Control of Sound Radiation in Elastic Wave Metamaterials Immersed in Water With Fluid–Solid Coupling
,”
Acta Mech. Sin.
,
37
(
5
), pp.
803
825
.
15.
Song
,
Y.
, and
Shen
,
Y.
,
2022
, “
Highly Morphing and Reconfigurable Fluid–Solid Interactive Metamaterials for Tunable Ultrasonic Guided Wave Control
,”
Appl. Phys. Lett.
,
121
(
26
), p. 264102.
16.
Gao
,
D.
,
Chen
,
J.
,
Dong
,
Z.
, and
Lin
,
H.
,
2022
, “
Connectivity-Guaranteed Porous Synthesis in Free Form Model by Persistent Homology
,”
Comput. Graphics
,
106
, pp.
33
44
.
17.
Swartz
,
K. E.
,
Tortorelli
,
D. A.
,
White
,
D. A.
, and
James
,
K. A.
,
2022
, “
Manufacturing and Stiffness Constraints for Topology Optimized Periodic Structures
,”
Struct. Multidiscipl. Optim.
,
65
(
4
), p.
129
.
18.
Holdstein
,
Y.
,
Fischer
,
A.
,
Podshivalov
,
L.
, and
Bar-Yoseph
,
P. Z.
,
2009
, “
Volumetric Texture Synthesis of Bone Micro-Structure as a Base for Scaffold Design
,”
2009 IEEE International Conference on Shape Modeling and Applications
,
Beijing, China
,
June 26–28
,
IEEE
, pp.
81
88
.
19.
Men
,
H.
,
Lee
,
K. Y.
,
Freund
,
R. M.
,
Peraire
,
J.
, and
Johnson
,
S. G.
,
2014
, “
Robust Topology Optimization of Three-Dimensional Photonic-Crystal Band-Gap Structures
,”
Opt. Expr.
,
22
(
19
), pp.
22632
22648
.
20.
Kench
,
S.
, and
Cooper
,
S. J.
,
2021
, “Generating 3D Structures From a 2D Slice With GAN-Based Dimensionality Expansion.” arXiv preprint arXiv:2102.07708.
21.
Zheng
,
X.
,
Guo
,
X.
,
Yang
,
Y.
,
Fu
,
Z.
,
Du
,
K.
,
Wang
,
C.
, and
Yi
,
Y.
,
2018
, “
Structure-Dependent Analysis of Nanoporous Metals: Clues From Mechanical, Conduction, and Flow Properties
,”
J. Phys. Chem. C
,
122
(
29
), pp.
16803
16809
.
22.
Xu
,
H.
,
Dikin
,
D. A.
,
Burkhart
,
C.
, and
Chen
,
W.
,
2014
, “
Descriptor-Based Methodology for Statistical Characterization and 3D Reconstruction of Microstructural Materials
,”
Comput. Mater. Sci.
,
85
, pp.
206
216
.
23.
Meyer
,
P. P.
,
Bonatti
,
C.
,
Tancogne-Dejean
,
T.
, and
Mohr
,
D.
,
2022
, “
Graph-Based Metamaterials: Deep Learning of Structure-Property Relations
,”
Mater. Des.
,
223
, p.
111175
.
24.
Makatura
,
L.
,
Wang
,
B.
,
Chen
,
Y.-L.
,
Deng
,
B.
,
Wojtan
,
C.
,
Bickel
,
B.
, and
Matusik
,
W.
,
2023
, “
Procedural Metamaterials: A Unified Procedural Graph for Metamaterial Design
,”
ACM Trans. Graph.
,
42
(
5
), pp.
1
19
.
25.
Yamaguchi
,
K.
,
Yasuda
,
H.
,
Tsujikawa
,
K.
,
Kunimine
,
T.
, and
Yang
,
J.
,
2022
, “
Graph-Theoretic Estimation of Reconfigurability in Origami-Based Metamaterials
,”
Mater. Des.
,
213
, p.
110343
.
26.
Du
,
P.
,
Zebrowski
,
A.
,
Zola
,
J.
,
Ganapathysubramanian
,
B.
, and
Wodo
,
O.
,
2018
, “
Microstructure Design Using Graphs
,”
npj Comput. Mater.
,
4
(
1
), p.
50
.
27.
Guo
,
K.
, and
Buehler
,
M. J.
,
2020
, “
A Semi-Supervised Approach to Architected Materials Design Using Graph Neural Networks
,”
Extreme Mech. Lett.
,
41
, p.
101029
.
28.
Reiser
,
P.
,
Neubert
,
M.
,
Eberhard
,
A.
,
Torresi
,
L.
,
Zhou
,
C.
,
Shao
,
C.
,
Metni
,
H.
,
van Hoesel
,
C.
,
Schopmans
,
H.
, and
Sommer
,
T.
,
2022
, “
Graph Neural Networks for Materials Science and Chemistry
,”
Commun. Mater.
,
3
(
1
), p.
93
.
29.
Nourian
,
N.
,
El-Badry
,
M.
, and
Jamshidi
,
M.
,
2023
, “
Design Optimization of Truss Structures Using a Graph Neural Network-Based Surrogate Model
,”
Algorithms
,
16
(
8
), p.
380
.
30.
Prachaseree
,
P.
, and
Lejeune
,
E.
,
2022
, “
Learning Mechanically Driven Emergent Behavior With Message Passing Neural Networks
,”
Comput. Struct.
,
270
, p.
106825
.
31.
Indurkar
,
P. P.
,
Karlapati
,
S.
,
Shaikeea
,
A. J. D.
, and
Deshpande
,
V. S.
,
2022
, “
Predicting Deformation Mechanisms in Architected Metamaterials Using GNN
,”
arXiv preprint arXiv:2202.09427
.
32.
Maurizi
,
M.
,
Gao
,
C.
, and
Berto
,
F.
,
2022
, “
Predicting Stress, Strain and Deformation Fields in Materials and Structures With Graph Neural Networks
,”
Sci. Rep.
,
12
(
1
), p.
21834
.
33.
Ross
,
E.
, and
Hambleton
,
D.
,
2021
, “
Using Graph Neural Networks to Approximate Mechanical Response on 3D Lattice Structures
,”
Proc. AAG2020-Adv. Archit. Geom.
,
24
, pp.
466
485
.
34.
Wang
,
Z.
,
Bray
,
Z.
,
Naghavi Khanghah
,
K.
, and
Xu
,
H.
,
2024
, “
A Generative Graph Neural Network-Based Framework for Designing Connectivity-Guaranteed Porous Metamaterial Units
,”
ASME 2024 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, (DETC2023-114601)
,
Washington, DC
,
Aug. 25–28
,
American Society of Mechanical Engineers
.
35.
Scarselli
,
F.
,
Gori
,
M.
,
Tsoi
,
A. C.
,
Hagenbuchner
,
M.
, and
Monfardini
,
G.
,
2008
, “
The Graph Neural Network Model
,”
IEEE Trans. Neural Netw.
,
20
(
1
), pp.
61
80
.
36.
Wu
,
Z.
,
Pan
,
S.
,
Chen
,
F.
,
Long
,
G.
,
Zhang
,
C.
, and
Philip
,
S. Y.
,
2020
, “
A Comprehensive Survey on Graph Neural Networks
,”
IEEE Trans. Neural Netw. Learn. Syst.
,
32
(
1
), pp.
4
24
.
37.
Dold
,
D.
, and
van Egmond
,
D. A.
,
2023
, “Differentiable Graph-Structured Models for Inverse Design of Lattice Materials,”
Cell Reports Physical Science
,
4
.
38.
Zhang
,
M.
,
2022
, “Graph Neural Networks: Link Prediction,”
Graph Neural Networks: Foundations, Frontiers, and Applications
,
L.
Wu
,
P.
Cui
,
J.
Pei
, and
L.
Zhao
, eds.,
Springer Nature Singapore
,
Singapore
, pp.
195
223
.
39.
Zhang
,
M.
, and
Chen
,
Y.
,
2018
, “Link Prediction Based on Graph Neural Networks,”
Advances in Neural Information Processing Systems
,
S.
Bengio
,
H.
Wallach
,
H.
Larochelle
,
K.
Grauman
,
N.
Cesa-Bianchi
, and
R.
Garnett
, eds., Curran Associates, Inc., Red Hook, NY, Vol.
31
.
40.
Kipf
,
T. N.
, and
Welling
,
M.
,
2016
, “
Variational Graph Auto-Encoders
,”
arXiv preprint arXiv:1611.07308
.
41.
Guo
,
Z.
,
Wang
,
F.
,
Yao
,
K.
,
Liang
,
J.
, and
Wang
,
Z.
,
2022
, “
Multi-scale Variational Graph Autoencoder for Link Prediction
,”
Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
, Virtual Event, Association for Computing Machinery, pp. 334– 342.
42.
Jin
,
B.
,
Liu
,
G.
,
Han
,
C.
,
Jiang
,
M.
,
Ji
,
H.
, and
Han
,
J.
,
2023
, “
Large Language Models on Graphs: A Comprehensive Survey
,”
arXiv preprint arXiv:2312.02783
.
43.
Vaswani
,
A.
,
Shazeer
,
N.
,
Parmar
,
N.
,
Uszkoreit
,
J.
,
Jones
,
L.
,
Gomez
,
A. N.
,
Kaiser
,
Ł
, and
Polosukhin
,
I.
,
2017
, “Attention Is All You Need,”
Advances in Neural Information Processing Systems
,
I.
Guyon
,
U.
Von Luxburg
,
S.
Bengio
,
H.
Wallach
,
R.
Fergus
,
S.
Vishwanathan
, and
R.
Garnett
, eds., Curran Associates, Inc., Red Hook, NY, Vol.
30
.
44.
Li
,
Y.
,
Li
,
Z.
,
Wang
,
P.
,
Li
,
J.
,
Sun
,
X.
,
Cheng
,
H.
, and
Yu
,
J. X.
,
2023
, “
A Survey of Graph Meets Large Language Model: Progress and Future Directions
,”
arXiv preprint arXiv:2311.12399
.
45.
Xie
,
H.
,
Zheng
,
D.
,
Ma
,
J.
,
Zhang
,
H.
,
Ioannidis
,
V. N.
,
Song
,
X.
,
Ping
,
Q.
,
Wang
,
S.
,
Yang
,
C.
, and
Xu
,
Y.
,
2023
, “Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications,” Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, Association for Computing Machinery, pp.
5270
5281
.
46.
Wen
,
Z.
, and
Fang
,
Y.
,
2023
, “
Prompt Tuning on Graph-Augmented Low-Resource Text Classification
,”
arXiv preprint arXiv:2307.10230
.
47.
Chandra
,
S.
,
Mishra
,
P.
,
Yannakoudakis
,
H.
,
Nimishakavi
,
M.
,
Saeidi
,
M.
, and
Shutova
,
E.
,
2020
, “
Graph-Based Modeling of Online Communities for Fake News Detection
,”
arXiv preprint arXiv:2008.06274
.
48.
Zhao
,
H.
,
Liu
,
S.
,
Ma
,
C.
,
Xu
,
H.
,
Fu
,
J.
,
Deng
,
Z.
,
Kong
,
L.
, and
Liu
,
Q.
,
2023
, “Gimlet: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning,” Advances in Neural Information Processing Systems,
36
, pp.
5850
5887
.
49.
Liu
,
P.
,
Ren
,
Y.
, and
Ren
,
Z.
,
2023
, “GIT-Mol: A Multi-modal Large Language Model for Molecular Science With Graph,”.
50.
Wang
,
T.
,
Roberts
,
A.
,
Hesslow
,
D.
,
Scao
,
T. L.
,
Chung
,
H. W.
,
Beltagy
,
I.
,
Launay
,
J.
, and
Raffel
,
C.
,
2022
, “
What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization?
,”
International Conference on Machine Learning
,
Baltimore, MD
,
PMLR
, Vol. 162, pp.
22964
22984
.
51.
Xian
,
Y.
,
Schiele
,
B.
, and
Akata
,
Z.
,
2017
, “
Zero-Shot Learning-the Good, the Bad and the Ugly
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
4582
4591
.
52.
Li
,
J.
,
Pan
,
K.
,
Ge
,
Z.
,
Gao
,
M.
,
Zhang
,
H.
,
Ji
,
W.
,
Zhang
,
W.
,
Chua
,
T.-S.
,
Tang
,
S.
, and
Zhuang
,
Y.
,
2023
, “
Fine-Tuning Multimodal LLMs to Follow Zero-Shot Demonstrative Instructions
,”
The Twelfth International Conference on Learning Representations
,
Vienna, Austria
,
May 7–11
.
53.
Perozzi
,
B.
,
Fatemi
,
B.
,
Zelle
,
D.
,
Tsitsulin
,
A.
,
Kazemi
,
M.
,
Al-Rfou
,
R.
, and
Halcrow
,
J.
,
2024
, “Let Your Graph Do the Talking: Encoding Structured Data for LLMs,”
arXiv preprint arXiv:2402.05862
.
54.
He
,
X.
,
Bresson
,
X.
,
Laurent
,
T.
,
Perold
,
A.
,
LeCun
,
Y.
, and
Hooi
,
B.
,
2023
, “
Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning
,”
The Twelfth International Conference on Learning Representations
,
Vienna, Austria
,
May 7–11
.
55.
Szabo
,
F.
,
2015
,
The Linear Algebra Survival Guide: Illustrated With Mathematica
,
Academic Press
,
San Diego, CA
.
56.
Otair
,
D. M.
,
2013
, “Approximate k-Nearest Neighbour Based Spatial Clustering Using KD Tree,”
arXiv preprint arXiv:1303.1951
.
57.
Das
,
K. C.
,
Maden
,
A. D.
,
Cangül
,
I. N.
, and
Çevik
,
A. S.
,
2017
, “
On Average Eccentricity of Graphs
,”
Proc. Natl. Acad. Sci., India Sect. A: Phys. Sci.
,
87
(
1
), pp.
23
30
.
58.
Pan
,
S.
,
Hu
,
R.
,
Long
,
G.
,
Jiang
,
J.
,
Yao
,
L.
, and
Zhang
,
C.
,
2018
, “Adversarially Regularized Graph Autoencoder for Graph Embedding,” arXiv preprint
arXiv:1802.04407
. .
59.
Wang
,
C.
,
Pan
,
S.
,
Hu
,
R.
,
Long
,
G.
,
Jiang
,
J.
, and
Zhang
,
C.
,
2019
, “Attributed Graph Clustering: A Deep Attentional Embedding Approach,”
arXiv preprint arXiv:1906.06532
.
60.
Sun
,
D.
,
Li
,
D.
,
Ding
,
Z.
,
Zhang
,
X.
, and
Tang
,
J.
,
2021
, “
Dual-Decoder Graph Autoencoder for Unsupervised Graph Representation Learning
,”
Knowl.-Based Syst.
,
234
, p.
107564
.
61.
Kingma
,
D. P.
, and
Welling
,
M.
,
2013
, “Auto-Encoding Variational Bayes,” arXiv preprint arXiv:1312.6114.
62.
Xu
,
K.
,
Wu
,
L.
,
Wang
,
Z.
,
Feng
,
Y.
,
Witbrock
,
M.
, and
Sheinin
,
V.
,
2018
, “Graph2seq: Graph to Sequence Learning With Attention-Based Neural Networks,”
arXiv preprint arXiv:1804.00823
.
63.
Sarkar
,
S.
, and
Lausen
,
L.
,
2023
, “Testing the Limits of Unified Seq2seq LLM Pretraining on Diverse Table Data Tasks”
64.
Xue
,
L.
,
Barua
,
A.
,
Constant
,
N.
,
Al-Rfou
,
R.
,
Narang
,
S.
,
Kale
,
M.
,
Roberts
,
A.
, and
Raffel
,
C.
,
2022
, “
Byt5: Towards a Token-Free Future With Pre-Trained Byte-to-Byte Models
,”
Trans. Assoc. Comput. Ling.
,
10
, pp.
291
306
.
65.
Pu
,
Y.
,
2022
, “Fine-Tuned LLM as Program Writers.” https://evanthebouncy.github.io/program-synthesis-minimal/generation-with-llm/.
66.
Zhang
,
H.
,
Song
,
H.
,
Li
,
S.
,
Zhou
,
M.
, and
Song
,
D.
,
2023
, “
A Survey of Controllable Text Generation Using Transformer-Based Pre-Trained Language Models
,”
ACM Comput. Surv.
,
56
(
3
), pp.
1
37
.
67.
Raffel
,
C.
,
Shazeer
,
N.
,
Roberts
,
A.
,
Lee
,
K.
,
Narang
,
S.
,
Matena
,
M.
,
Zhou
,
Y.
,
Li
,
W.
, and
Liu
,
P. J.
,
2020
, “
Exploring the Limits of Transfer Learning With a Unified Text-to-Text Transformer
,”
J. Mach. Learn. Res.
,
21
(
140
), pp.
5485
5551
.
68.
Fu
,
Z.
,
Lam
,
W.
,
Yu
,
Q.
,
So
,
A. M.-C.
,
Hu
,
S.
,
Liu
,
Z.
, and
Collier
,
N.
,
2023
, “Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder.”
arXiv preprint arXiv:2304.04052
.
69.
Pourpanah
,
F.
,
Abdar
,
M.
,
Luo
,
Y.
,
Zhou
,
X.
,
Wang
,
R.
,
Lim
,
C. P.
,
Wang
,
X.-Z.
, and
Wu
,
Q. J.
,
2022
, “
A Review of Generalized Zero-Shot Learning Methods
,”
IEEE Trans. Pattern Anal. Mach. Intell
,
45
, (
4
), pp.
4051
4070
. .
70.
Zhang
,
D.
,
Wang
,
J.
, and
Charton
,
F.
,
2024
, “Instruction Diversity Drives Generalization to Unseen Tasks.”
arXiv preprint arXiv:2402.10891
.
71.
Tang
,
J.
,
Yang
,
Y.
,
Wei
,
W.
,
Shi
,
L.
,
Su
,
L.
,
Cheng
,
S.
,
Yin
,
D.
, and
Huang
,
C.
,
2023
, “Graphgpt: Graph Instruction Tuning for Large Language Models.” Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington DC, Association for Computing Machinery, New York, pp.
491
500
.
72.
Shen
,
Z.
,
Zhang
,
M.
,
Zhao
,
H.
,
Yi
,
S.
, and
Li
,
H.
,
2021
, “
Efficient Attention: Attention With Linear Complexities
,”
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
,
Virtual Online
,
Jan. 5–9
, pp.
3531
3539
.
73.
Huggingface
. “google/byt5-base. https://huggingface.co/google/byt5-base.
75.
Hagberg
,
A.
,
Swart
,
P. J.
, and
Schult
,
D. A.
,
2008
,
Exploring Network Structure, Dynamics, and Function Using NetworkX
,
Los Alamos National Laboratory (LANL)
,
Los Alamos, NM
.
76.
Bhattacharjee
,
A.
,
Moraffah
,
R.
,
Garland
,
J.
, and
Liu
,
H.
,
2024
, “Towards LLM-Guided Causal Explainability for Black-Box Text Classifiers.
77.
Lewis
,
M.
,
Liu
,
Y.
,
Goyal
,
N.
,
Ghazvininejad
,
M.
,
Mohamed
,
A.
,
Levy
,
O.
,
Stoyanov
,
V.
, and
Zettlemoyer
,
L.
,
2019
, “Bart: Denoising Sequence-to-Sequence Pre-Training for Natural Language Generation, Translation, and Comprehension,”
arXiv preprint arXiv:1910.13461
.
78.
Renaud
,
O.
, and
Victoria-Feser
,
M.-P.
,
2010
, “
A Robust Coefficient of Determination for Regression
,”
J. Stat. Plan. Inference
,
140
(
7
), pp.
1852
1862
.
79.
Helland
,
I. S.
,
1987
, “
On the Interpretation and Use of R2 in Regression Analysis
,”
Biometrics
,
43
(
1
), pp.
61
69
.
80.
Huggingface
. “Facebook/Bart-Base.” https://huggingface.co/facebook/bart-base.
81.
Ouyang
,
S.
,
Zhang
,
J. M.
,
Harman
,
M.
, and
Wang
,
M.
,
2023
, “LLM is Like a Box of Chocolates: The Non-Determinism of ChatGPT in Code Generation”
arXiv preprint arXiv:2308.02828
.
82.
Pu
,
Y.
,
Ellis
,
K.
,
Kryven
,
M.
,
Tenenbaum
,
J.
, and
Solar-Lezama
,
A.
,
2020
, “
Program Synthesis With Pragmatic Communication
,”
Adv. Neural Inf. Process. Syst.
,
33
, pp.
13249
13259
.
83.
Pu
,
Y.
,
Miranda
,
Z.
,
Solar-Lezama
,
A.
, and
Kaelbling
,
L. P.
,
2018
, Learning to Select Examples for Program Synthesis.
84.
Zhang
,
C.
,
Lu
,
T.
,
Islam
,
M. M.
,
Wang
,
Z.
,
Yu
,
S.
,
Bansal
,
M.
, and
Bertasius
,
G.
,
2023
, “A Simple LLM framework for Long-Range Video Question-Answering.”
arXiv preprint arXiv:2312.17235
.
85.
Raiaan
,
M. A. K.
,
Mukta
,
M. S. H.
,
Fatema
,
K.
,
Fahad
,
N. M.
,
Sakib
,
S.
,
Mim
,
M. M. J.
,
Ahmad
,
J.
,
Ali
,
M. E.
, and
Azam
,
S.
,
2024
, “
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges
,”
IEEE Access
,
12
, pp.
26839
26874
.
86.
Liu
,
X.
,
Yu
,
H.-F.
,
Dhillon
,
I.
, and
Hsieh
,
C.-J.
,
2020
, “
Learning to Encode Position for Transformer With Continuous Dynamical Model
,”
International Conference on Machine Learning
,
Virtual Event
,
July 13–18
,
PMLR
, 119, pp.
6327
6335
.
87.
Hu
,
E. J.
,
Shen
,
Y.
,
Wallis
,
P.
,
Allen-Zhu
,
Z.
,
Li
,
Y.
,
Wang
,
S.
,
Wang
,
L.
, and
Chen
,
W.
,
2021
, “Lora: Low-Rank Adaptation of Large Language Models”
arXiv preprint arXiv:2106.09685
.
88.
Liu
,
H.
,
Tam
,
D.
,
Muqeeth
,
M.
,
Mohta
,
J.
,
Huang
,
T.
,
Bansal
,
M.
, and
Raffel
,
C. A.
,
2022
, “
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper Than In-Context Learning
,”
Adv. Neural Inf. Process. Syst.
,
35
, pp.
1950
1965
.
89.
Mo
,
Y.
,
Yoo
,
J.
, and
Kang
,
S.
,
2023
, “
Parameter-Efficient Fine-Tuning Method for Task-Oriented Dialogue Systems
,”
Mathematics
,
11
(
14
), p.
3048
.
You do not currently have access to this content.