Skip to main navigation Skip to search Skip to main content

A Case for Speculative Address Translation with Rapid Validation for GPUs

  • Junhyeok Park
  • , Osang Kwon
  • , Yongho Lee
  • , Seongwook Kim
  • , Gwangeun Byeon
  • , Jihun Yoon
  • , Prashant J. Nair
  • , Seokin Hong
  • Sungkyunkwan University
  • University of British Columbia

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

A unified address space is vital for heterogeneous systems as it enables efficient data sharing between CPUs and GPUs. However, GPU address translation faces challenges due to high TLB pressure, particularly with irregular and memory-intensive applications. Compared to an ideal scenario, we observe that address translation overheads cause a slowdown of up to 34.5% in modern heterogeneous systems. This paper introduces Avatar, a novel framework to accelerate address translation in GPUs. Avatar comprises two key components: Contiguity-Aware Speculative Translation (CAST) and In-Cache Validation (CAVA) mechanisms. Avatar identifies the potential for predicting virtual-to-physical address mapping by monitoring contiguous pages that lie in both virtual and physical address spaces. Leveraging this insight, CAST speculatively translates virtual addresses into physical addresses. This speculative address translation enables immediate data fetching into GPUs while addressing translation occurs in the background, reducing TLB-miss overhead. Unfortunately, modern GPUs lack support for speculative execution, which limits CAST's performance gain. Data fetched from speculated physical addresses is unusable until validation. CAVA addresses this limitation by quickly validating speculated physical addresses. To this end, CAVA embeds page mapping information into each 32B sector of 128B cache lines. Thus, CAVA enables fetching a sector block from memory for a speculated address and rapidly validating the speculative translation using the embedded mapping information. Our experiments show that Avatar achieves a 90.3% (high) speculation accuracy and improves GPU performance by 37.2% (on average).

Original languageEnglish
Title of host publicationProceedings - 2024 57th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2024
PublisherIEEE Computer Society
Pages278-292
Number of pages15
ISBN (Electronic)9798350350579
DOIs
StatePublished - 2024
Externally publishedYes
Event57th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2024 - Austin, United States
Duration: 2 Nov 2024 → 6 Nov 2024

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
ISSN (Print)1072-4451

Conference

Conference57th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2024
Country/TerritoryUnited States
CityAustin
Period2/11/24 → 6/11/24

Keywords

  • address translation
  • data compression
  • GPU
  • speculation
  • virtual memory

Fingerprint

Dive into the research topics of 'A Case for Speculative Address Translation with Rapid Validation for GPUs'. Together they form a unique fingerprint.

Cite this