IEEE Circuits and Systems Magazine - Q3 2023 - 33

[33] Y. Y. Siang, M. R. Ahamd, and M. S. Z. Abidin, " Anomaly detection
based on tiny machine learning: A review, " Open Int. J. Inform., vol. 9,
no. 2, pp. 67-78, 2021.
[34] K. Roth et al., " Towards total recall in industrial anomaly detection, "
in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2022,
pp. 14318-14328.
[35] F. Alongi et al., " Tiny neural networks for environmental predictions:
An integrated approach with Miosix, " in Proc. IEEE Int. Conf. Smart
Comput. (SMARTCOMP), 2020, pp. 350-355.
[36] C. Vuppalapati et al., " Automating tiny ML intelligent sensors DevOPS
using Microsoft Azure, " in Proc. IEEE Int. Conf. Big Data, Dec. 2020,
pp. 2375-2384.
[37] C. Vuppalapati et al., " Democratization of AI, albeit constrained IoT
devices & tiny ML, for creating a sustainable food future, " in Proc. 3rd
Int. Conf. Inf. Comput. Technol., Mar. 2020, pp. 525-530.
[38] F. Nakhle and A. L. Harfouche, " Ready, steady, go AI: A practical
tutorial on fundamentals of artificial intelligence and its applications in
phenomics image analysis, " Patterns, vol. 2, no. 9, 2021, Art. no. 100323.
[39] D. J. Curnick et al., " SmallSats: A new technological frontier in ecology
and conservation? " Remote Sens. Ecol. Conservation, vol. 8, no. 2,
pp. 139-150, 2022.
[40] C. Nicolas, B. Naila, and R.-C. Amar, " TinyML smart sensor for energy
saving in Internet of Things precision agriculture platform, " in Proc.
13th Int. Conf. Ubiquitous Future Netw., Jul. 2022, pp. 256-259.
[41] L. Lai, N. Suda, and V. Chandra, " CMSIS-NN: Efficient neural network
kernels for Arm Cortex-M CPUs, " 2018, arXiv:1801.06601.
[42] STMicroelectronics. X-CUBE-AI: AI Expansion Pack for STM32CubeMX.
[Online]. Available: https://www.st.com/en/embedded-software/xcube-ai.html
[43]
MicroTVM: TVM on Bare-Metal. [Online]. Available: https://tvm.
apache.org/docs/topic/microtvm/index.html
[44] E. Liberis and N. D. Lane, " Neural networks on microcontrollers: Saving
memory at inference via operator reordering, " 2019, arXiv:1910.05110.
[45] M. Rusci, A. Capotondi, and L. Benini, " Memory-driven mixed low
precision quantization for enabling deep network inference on microcontrollers, "
in Proc. Mach. Learn. Syst., 2020, pp. 1-10.
[46] A. Capotondi et al., " CMix-NN: Mixed low-precision CNN library
for memoryconstrained edge devices, " IEEE Trans. Circuits Syst. II, Exp.
Briefs, vol. 67, no. 5, pp. 871-875, May 2020.
[47] R. David et al., " TensorFlow lite micro: Embedded machine learning
for TinyML systems, " in Proc. Mach. Learn. Syst., vol. 3, 2021, pp. 800-811.
[48] C. Banbury et al., " MicroNets: Neural network architectures for deploying
TinyML applications on commodity microcontrollers, " in Proc.
Mach. Learn. Syst., vol. 3, 2021, pp. 1-16.
[49] S. Sadiq et al., " TinyOps: ImageNet scale deep learning on microcontrollers, "
in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.
Workshops, Jun. 2022, pp. 2701-2705.
[50] TinyMaix. [Online]. Available: https://github.com/sipeed/TinyMaix
[51] I. Fedorov et al., " UDC: Unified DNAS for compressible TinyML models
for neural processing units, " in Proc. Adv. Neural Inf. Process. Syst.,
2022, pp. 1-16.
[52] H. Cai et al., " TinyTL: Reduce activations, not trainable parameters
for efficient on-device learning, " 2020, arXiv:2007.11622.
[53] H. Ren, D. Anicic, and T. A. Runkler, " TinyOL: TinyML with onlinelearning
on microcontrollers, " in Proc. Int. Joint Conf. Neural Netw., Jul.
2021, pp. 1-8.
[54] S. G. Patil et al., " POET: Training neural networks on tiny devices
with integrated rematerialization and paging, " in Proc. Int. Conf. Mach.
Learn. (PMLR), 2022, pp. 17573-17583.
[55] C. Profentzas, M. Almgren, and O. Landsiedel, " MiniLearn: Ondevice
learning for low-power IoT devices, " in Proc. Int. Conf. Embedded
Wireless Syst. Netw., Linz, Austria, 2022, pp. 1-12.
[56] J. Lin et al., " On-device training under 256 KB memory, " 2022, arXiv:2206.15472.
[57]
S. Han et al., " Learning both weights and connections for efficient
neural network, " in Proc. NeurIPS, 2015, pp. 1-9.
[58] Y. He, X. Zhang, and J. Sun, " Channel pruning for accelerating very
deep neural networks, " in Proc. Int. Conf. Comput. Vis., Oct. 2017, pp.
1398-1406.
[59] J. Lin et al., " Runtime neural pruning, " in Proc. NeurIPS, 2017, pp. 1-11.
[60] Z. Liu et al., " Learning efficient convolutional networks through network
slimming, " in Proc. Int. Conf. Comput. Vis., Oct. 2017, pp. 2755-2763.
[61] Y. He et al., " AMC: AutoML for model compression and acceleration
on mobile devices, " in Proc. Eur. Conf. Comput. Vis., 2018, pp. 815-832.
THIRD QUARTER 2023
[62] Z. Liu et al., " MetaPruning: Meta learning for automatic neural network
channel pruning, " in Proc. Int. Conf. Comput. Vis., Oct./Nov. 2019,
pp. 3295-3304.
[63] S. Han, H. Mao, and W. J. Dally, " Deep compression: Compressing
deep neural networks with pruning, trained quantization and Huffman
coding, " in Proc. Int. Conf. Learning Representations, 2016, pp. 1-14.
[64] C. Zhu et al., " Trained ternary quantization, " in Proc. Int. Conf.
Learning Representations, 2017, pp. 1-10.
[65] M. Rastegari et al., " XNOR-Net: ImageNet classification using binary
convolutional neural networks, " in Proc. Eur. Conf. Comput. Vis.,
2016, pp. 525-542.
[66] S. Zhou et al., " DoReFa-Net: Training low bitwidth convolutional
neural networks with low bitwidth gradients, " 2016, arXiv:1606.06160.
[67] M. Courbariaux and Y. Bengio, " Binarized neural networks: Training
deep neural networks with weights and activations constrained to
+1 or -1, " 2016, arXiv:1602.02830.
[68] J. Choi et al., " PACT: Parameterized clipping activation for quantized
neural networks, " 2018, arXiv:1805.06085.
[69] K. Wang et al., " HAQ: Hardware-aware automated quantization
with mixed precision, " in Proc. Conf. Comput. Vis. Pattern Recognit., Jun.
2019, pp. 8604-8612.
[70] H. F. Langroudi et al., " TENT: Efficient quantization of neural networks
on the tiny edge with tapered fixed point, " 2021, arXiv:2104.02233.
[71] V. Lebedev et al., " Speeding-up convolutional neural networks using
fine-tuned CP-decomposition, " 2014, arXiv:1412.6553.
[72] Y. Gong et al., " Compressing deep convolutional networks using
vector quantization, " 2014, arXiv:1412.6115.
[73] Y.-D. Kim et al.,
" Compression of deep convolutional neural
networks for fast and low power mobile applications, " 2015, arXiv:1511.06530.
[74]
G. Hinton, O. Vinyals, and J. Dean, " Distilling the knowledge in a
neural network, " 2015, arXiv:1503.02531.
[75] W. Park et al., " Relational knowledge distillation, " in Proc. IEEE/CVF
Conf. Comput. Vis. Pattern Recognit., Jun. 2019, pp. 3967-3976.
[76] F. Tung and G. Mori, " Similarity-preserving knowledge distillation, "
in Proc. IEEE/CVF Int. Conf. Comput. Vis., Oct.-Nov. 2019, pp. 1365-1374.
[77] S. I. Mirzadeh et al., " Improved knowledge distillation via teacher assistant, "
in Proc. AAAI Conf. Artif. Intell., 2020, vol. 34, no. 4, pp. 5191-5198.
[78] L. Wang and K.-J. Yoon, " Knowledge distillation and student-teacher
learning for visual intelligence: A review and new outlooks, " IEEE
Trans. Pattern Anal. Mach. Intell., vol. 44, no. 6, pp. 3048-3068, Jun. 2021.
[79] Z. Yang et al., " Focal and global knowledge distillation for detectors, "
in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2022,
pp. 4643-4652.
[80] B. Zhao et al., " Decoupled knowledge distillation, " in Proc. IEEE/
CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2022, pp. 11953-11962.
[81] L. Beyer et al., " Knowledge distillation: A good teacher is patient
and consistent, " in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.,
Jun. 2022, pp. 10925-10934.
[82] B. Zoph and Q. V. Le, " Neural architecture search with reinforcement
learning, " in Proc. Int. Conf. Learning Representations, 2017, pp. 1-16.
[83] B. Zoph et al., " Learning transferable architectures for scalable image
recognition, " in Proc. Conf. Comput. Vis. Pattern Recognit., Jun. 2018,
pp. 8697-8710.
[84] H. Liu, K. Simonyan, and Y. Yang, " DARTS: Differentiable architecture
search, " in Proc. Int. Conf. Learning Representations, 2019, pp. 1-13.
[85] H. Cai, L. Zhu, and S. Han, " ProxylessNAS: Direct neural architecture
search on target task and hardware, " in Proc. Int. Conf. Learning
Representations, 2019, pp. 1-13.
[86] M. Tan et al., " MnasNet: Platform-aware neural architecture search
for mobile, " in Proc. Conf. Comput. Vis. Pattern Recognit., Jun. 2019, pp.
2815-2823.
[87] B. Wu et al., " FBNet: Hardwareaware efficient ConvNet design via
differentiable neural architecture search, " in Proc. Conf. Comput. Vis.
Pattern Recognit., Jun. 2019, pp. 10734-10742.
[88] I. Radosavovic et al., " Designing network design spaces, " 2020,
arXiv:2003.13678.
[89] A. Paszke et al., " PyTorch: An imperative style, high-performance
deep learning library, " in Proc. Adv. Neural Inf. Process. Syst., vol. 32,
2019, pp. 1-12.
[90] M. Abadi et al., " TensorFlow: A system for large-scale machine
learning, " in Proc. Oper. Syst. Des Implementation, 2016, pp. 265-283.
[91] T. Chen et al., " MXNet: A flexible and efficient machine learning
library for heterogeneous distributed systems, " 2015, arXiv:1512.01274.
IEEE CIRCUITS AND SYSTEMS MAGAZINE
33
https://www.st.com/en/embedded-software/x-cube-ai.html https://www.st.com/en/embedded-software/x-cube-ai.html http://www.tvm.apache.org/docs/topic/microtvm/index.html http://www.tvm.apache.org/docs/topic/microtvm/index.html https://www.github.com/sipeed/TinyMaix
IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023
