Hardware-Based Noisy Deep Q-Networks Using Low-Frequency Noise of Synaptic Devices for Efficient Exploration

We propose an efficient exploration method using low-frequency noise of synaptic devices applicable to hardware-based deep Q-networks. The proposed method efficiently implements the exploration with a relatively low hardware burden compared with other published studies. A rounded dual channel flash memory cell is used as a synaptic device. The performance evaluation based on a simple Snake game shows that the proposed system achieves performance similar to that using the ϵ-greedy exploration method. Sufficient exploration can be conducted for network training even with a small noise level of the synaptic devices without an additional circuit.