In-memory batch-normalization for resistive memory based binary neural network hardware

Hyungjun Kim; Yulhwa Kim; Jae Joon Kim

doi:10.1145/3287624.3287718

In-memory batch-normalization for resistive memory based binary neural network hardware

Hyungjun Kim
, Yulhwa Kim
, Jae Joon Kim

Pohang University of Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

22 Scopus citations

Abstract

Binary Neural Network (BNN) has a great potential to be implemented on Resistive memory Crossbar Array (RCA)-based hardware accelerators because it requires only 1-bit precision for weights and activations. While general structures to implement convolution or fully-connected layers in RCA-based BNN hardware were actively studied in previous works, Batch-Normalization (BN) layer, which is another key layer of BNN, has not been discussed in depth yet. In this work, we propose in-memory batch-normalization schemes which integrate BN layers on RCA so that area/energy-efficiency of the BNN accelerators can be maximized. In addition, we also show that sense amp error due to device mismatch can be suppressed using the proposed in-memory BN design.

Original language	English
Title of host publication	ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	645-650
Number of pages	6
ISBN (Electronic)	9781450360074
DOIs	https://doi.org/10.1145/3287624.3287718
State	Published - 21 Jan 2019
Externally published	Yes
Event	24th Asia and South Pacific Design Automation Conference, ASPDAC 2019 - Tokyo, Japan Duration: 21 Jan 2019 → 24 Jan 2019

Publication series

Name	Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC

Conference

Conference	24th Asia and South Pacific Design Automation Conference, ASPDAC 2019
Country/Territory	Japan
City	Tokyo
Period	21/01/19 → 24/01/19

Access to Document

10.1145/3287624.3287718

Cite this

Kim, H., Kim, Y., & Kim, J. J. (2019). In-memory batch-normalization for resistive memory based binary neural network hardware. In ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference (pp. 645-650). (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3287624.3287718

@inproceedings{d32d5b87104742c39f8135b626c911f4,

title = "In-memory batch-normalization for resistive memory based binary neural network hardware",

abstract = "Binary Neural Network (BNN) has a great potential to be implemented on Resistive memory Crossbar Array (RCA)-based hardware accelerators because it requires only 1-bit precision for weights and activations. While general structures to implement convolution or fully-connected layers in RCA-based BNN hardware were actively studied in previous works, Batch-Normalization (BN) layer, which is another key layer of BNN, has not been discussed in depth yet. In this work, we propose in-memory batch-normalization schemes which integrate BN layers on RCA so that area/energy-efficiency of the BNN accelerators can be maximized. In addition, we also show that sense amp error due to device mismatch can be suppressed using the proposed in-memory BN design.",

author = "Hyungjun Kim and Yulhwa Kim and Kim, \{Jae Joon\}",

note = "Publisher Copyright: {\textcopyright} 2019 Association for Computing Machinery.; 24th Asia and South Pacific Design Automation Conference, ASPDAC 2019 ; Conference date: 21-01-2019 Through 24-01-2019",

year = "2019",

month = jan,

day = "21",

doi = "10.1145/3287624.3287718",

language = "English",

series = "Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "645--650",

booktitle = "ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference",

}

Kim, H, Kim, Y & Kim, JJ 2019, In-memory batch-normalization for resistive memory based binary neural network hardware. in ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference. Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC, Institute of Electrical and Electronics Engineers Inc., pp. 645-650, 24th Asia and South Pacific Design Automation Conference, ASPDAC 2019, Tokyo, Japan, 21/01/19. https://doi.org/10.1145/3287624.3287718

In-memory batch-normalization for resistive memory based binary neural network hardware. / Kim, Hyungjun; Kim, Yulhwa; Kim, Jae Joon.
ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference. Institute of Electrical and Electronics Engineers Inc., 2019. p. 645-650 (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - In-memory batch-normalization for resistive memory based binary neural network hardware

AU - Kim, Hyungjun

AU - Kim, Yulhwa

AU - Kim, Jae Joon

PY - 2019/1/21

Y1 - 2019/1/21

N2 - Binary Neural Network (BNN) has a great potential to be implemented on Resistive memory Crossbar Array (RCA)-based hardware accelerators because it requires only 1-bit precision for weights and activations. While general structures to implement convolution or fully-connected layers in RCA-based BNN hardware were actively studied in previous works, Batch-Normalization (BN) layer, which is another key layer of BNN, has not been discussed in depth yet. In this work, we propose in-memory batch-normalization schemes which integrate BN layers on RCA so that area/energy-efficiency of the BNN accelerators can be maximized. In addition, we also show that sense amp error due to device mismatch can be suppressed using the proposed in-memory BN design.

AB - Binary Neural Network (BNN) has a great potential to be implemented on Resistive memory Crossbar Array (RCA)-based hardware accelerators because it requires only 1-bit precision for weights and activations. While general structures to implement convolution or fully-connected layers in RCA-based BNN hardware were actively studied in previous works, Batch-Normalization (BN) layer, which is another key layer of BNN, has not been discussed in depth yet. In this work, we propose in-memory batch-normalization schemes which integrate BN layers on RCA so that area/energy-efficiency of the BNN accelerators can be maximized. In addition, we also show that sense amp error due to device mismatch can be suppressed using the proposed in-memory BN design.

UR - https://www.scopus.com/pages/publications/85061141271

U2 - 10.1145/3287624.3287718

DO - 10.1145/3287624.3287718

M3 - Conference contribution

AN - SCOPUS:85061141271

T3 - Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC

SP - 645

EP - 650

BT - ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 24th Asia and South Pacific Design Automation Conference, ASPDAC 2019

Y2 - 21 January 2019 through 24 January 2019

ER -

Kim H, Kim Y, Kim JJ. In-memory batch-normalization for resistive memory based binary neural network hardware. In ASP-DAC 2019 - 24th Asia and South Pacific Design Automation Conference. Institute of Electrical and Electronics Engineers Inc. 2019. p. 645-650. (Proceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC). doi: 10.1145/3287624.3287718

In-memory batch-normalization for resistive memory based binary neural network hardware

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this