(tpu7x-memory)=
# {{ product_tpu7x }} memory hierarchy

How high-bandwidth memory \(HBM\) and host memory are organized on a {{ product_tpu7x }} chip.

<span id="tpu7x-memory__hbm"></span>

(tpu7x-memory__hbm)=
## High-bandwidth memory

Each chip has {{ tpu7x_hbm_capacity }} GiB of HBM, with bandwidth of approximately 7.37 TB/s. The large capacity allows large batch sizes, which can improve throughput. HBM can still be a bottleneck for memory-bound vector operations or inefficient data access patterns.

<span id="tpu7x-memory__chiplets"></span>

(tpu7x-memory__chiplets)=
## Dual-chiplet design

Each chip has two chiplets. Each chiplet is a self-contained unit with one TensorCore, two SparseCores and {{ tpu7x_chiplet_hbm }} GiB of HBM.

<span id="tpu7x-memory__host-memory"></span>

(tpu7x-memory__host-memory)=
## Host memory

The host's main memory connects to the chip with much lower bandwidth than HBM. You can still use it to offload activations or optimizer states and free HBM, which helps with memory pressure in large models.


:::::{container} manifest-nav

% manifest:published-in unfiltered,tpu7x
**Parent topic:**[{{ product_tpu7x }}](../topics/tpu7x-intro.md)
% manifest:end

:::::



:::::{container} manifest-nav

**Related information**  


% manifest:published-in unfiltered,tpu7x
[{{ product_tpu7x }} specifications](../topics/tpu7x-specifications.md)
% manifest:end

% manifest:published-in unfiltered,tpu7x
[{{ product_tpu7x }} supported configurations](../topics/tpu7x-configurations.md)
% manifest:end

% manifest:published-in unfiltered,tpu7x
[{{ product_tpu7x }} VM types](../topics/tpu7x-vm-types.md)
% manifest:end

:::::