Some Q&As about Intel Sapphire Rapids

(source: HC33.slack.com)

[Q1] Is the Dynamic Load Balancer similar to Intel Thread Director, or is it an independently developed system?
	> Dynamic Load Balancer is not related to thread director. It is primarily focused on dynamically managing work offload to acceleration engines - integrated or discrete- thereby balancing the offload management load across CPU cores

[Q2] How will Sapphire Rapids scale down to smaller core counts? Will we see 2-tile or 1-tile variants, or rather different dies (maybe monolithic?) for lower core count SPR products?
	> The 4-tile SPR shown is our largest Sapphire Rapids SoC and best illustrates how the new tiled architecture works. we will reveal more about core counts and how we will scale to lower core counts later. Suffice it to say that the architecture allows the resulting SoC to appear as a completely monolithic CPU to platform & SW.

[Q3] No Sapphire Rapids die shot(s) for this time? For Ice Lake presentation at Hot Chips 2020, an image showing the floorplan of 28C variant is provided in the slides.
	> As for die shots, we did show a die shot in the video but i think you are asking for a die diagram showing core counts, IO counts, etc. - we will be revealing those at a later date.

[Q4] What is the LLC per core? Looks like about 2MB
	> Hello, we'll be revealing the exact sizes for LLC at a later date. private L2$ (MLC) is increased to 2MB per core and is one of the items that is different in the Sapphire Rapids version of the performance core vs the Alder Lake version. Shared LLC per core has increased vs Ice Lake but we are not revealing the exact size just yet.

[Q5] Does EMIB offer uniform D2D latency from one die to every other die including diagonal? (e.g. bottom left die to top right die)?
	> EMIB offers low single digit nanosecond latencies going vertically or horizontally but they are not necessarily exactly the same.  there is no diagonal route here.  the way to picture this is that  you take a large monolithic mesh and then simply draw a vertical line and a horizontal line thru the middle.

[Q6] For a AI workload on CPU, can we achieve performance improvement from AMX without any code changes based on one api?
	> AMX is a set of new instructions so re-compilation of AI related code or libraries will be needed to take full advantage.

[Q7] Any data about the improvement of some end-to-end AI workload? For example, ResNet-50?
	> as stated in the presentation, int8 data types can see up to a 8x acceleration.  That establishes the upper bound for what workloads can see.  we showed a demo in Architecture Day (you can find the video on Youtube) that showed a WL accelerated 7.8x.  other workload performance data will be revealed later

[Q8] How many different memory encryption algorithms are supported? Can they be customer defined?
	> we will be revealing more details about encryption and security at a later date.

[Q9] What is the core micro-arch in Sapphire-Rapids?
	> the Core uarch is code named Golden Cove, what we are referring to as our performance core.  It is the same performance core used in Alder Lake with some specific differences with respect to data center usages & capabilities (such as a larger L2$)

[Q10] On slide 5 you say there is fast VM Migration, is there any hardware that supports that?
	> Yes, we have new HW in the ur re-designed IOMMU, in conjunction with other HW improvements to accelerate VM migration

[Q11] With HBM, do you still support 8 channels of DDR? Also, can you comment on the associativity of the HBM when in caching mode?
	> Yes, we still support the full 8 channels of DDR on the HBM version of Sapphire Rapids. More details such as associativity will be revealed at a late date

[Q12] Can you show me more detail about CLDEMOTE cache management enhancement? What is this? and is it only for Sapphire Rapids, not for AlderLake?
	> CLDEMOTE is a new instruction that basically outs a cache line in the shared cache, accelerating any code that shares data by forcing the line into the correct cache level & state for sharing without needing to wait for snoops.  As far as i know it us not currently planned for Alder Lake but that is a good question for the Alder Lake guys

[Q13] How many PCIe lanes support CXL? Also, do you plan to support Type 3 CXL device on Sapphire Rapids?
	> All PCIe lanes will support CXL.  final lane counts will be revealed later.  any x16 PCIe link will be able to be configured as either CXL or PCIe at boot based on the card that is plugged into it.  Sapphire Rapids will support Type 1 and Type 2 devices but not Type 3 at this time.

	[Q13-1] So Optane 300 will still be on DDR-T rather than CXL?
		> DDR-T2 to be precise. no Optane on CXL in Sapphire Rapids timeframe

	[Q13-2] Would it be possible to use HBM+PMem configuration (w/o DRAM at all)?
		> no, HBM will not support PMEM configs
		> but more details on HBM will be coming at later date

[Q14] How is performance of DRAM when accesses are strided in multiple ways? This access pattern is paramount for stencil computations, common with HPC workloads
	> Memory is interleaved across DRAM to provide best bandwidth.  Depending on the stride, access patterns may vary and overall performance will depend on that - should be nothing new there regarding Sapphire rapids.  What is your specific concern?  Is it accessing memory across different tiles or is it in reference to HBM?

[Q15] In complex stencil computations there are many different read streams when many arrays (2-D or 3D) are employed. It would be beneficial that the strided read patterns are predicted and assisted in this case. The issue is that usually there are multiple strided read streams from DRAM. With HBM the problem is similar. Is the memory interleaving at the cache block size/controller or at a smaller scale (eg 16 or 8 bytes)?
	> understood.  We do have a number of different pre-fetchers, some of which can be tuned to optimize for different strides/access patterns.  What we tuned them on however is the question in this case and whether it targets the specific cases of interest.  Re-tuning is a possibility for specific SKUs & cases.  I would need to dig into this further with my performance team as well as our HPC guys.
	> memory interleaving granularity is 256B

[Q16] Can you comment on the latency cost of having to cross one or more tiles to access memory? I expect the worst case will have two cross-tile roundtrips? Related to the above, are you implementing tile-level NUMA clusters?
	> worst case will be 4 crossings - example: going from upper left to lower right tile, you would first cross down a tile then right a tile and then the return will go up and left - so that is 4 total crossings worst case.  for completely UMA workloads you could incur this 25% of the time.  However, we support both sub-NUMA clustering at the tile level as well as a new UMC clustering at the tile level.  The UMA clustering - called quadrant mode - essentially gets you the majority of the latency benefit of SNC without requiring SW NUMA optimizations by affinitizing home agents with memory controllers on the same tile.  it requires all mem controllers to be populated symmetrically on system and is expected to be our out of the box performant mode on Sapphire

	[Q16-1] thanks. Can you comment on the unloaded latency cost of each crossing? Also, does the BW for the inter-tile link match the on-die mesh BW or is there a potential additional congestion to comprehend?
		> lossless BW across the links. this is one of the key capabilities EMIB delivers. low single digit ns latencies across each EMIB crossing

[Q17] is that 256B interleaving between SPR controllers or at the DDR/HBM channel level?
	> 256B is at the channel level, yes.

[Q18] In yesterday's packaging presentation from your colleagues, we learned that Sapphire Rapids uses 10 EMIB bridges (on the non-HBM variant). Does this mean there is different number of horizontal and vertical bridges?
	> Yes, there are different # of bridges vertical vs horizontal.  basically every row connects to every adjacent row and every column connects to every adjacent column.  If the mesh were perfectly square (# rows = # cols) then there would be same # bridges in each direction but it is not, it is rectangular in each tile, hence the different # of bridges horizontal vs vertical

[Q19] How is deterministic performance maintained when a thread is accessing resources of a neighboring tile, going off-die?
	> first off, the latency for going to a different tile is very low, low single digit ns latencies.  that said, the latencies are higher even if it were a physically monolithic mesh.  we provide a few different mechanisms to help: sub-NUMA clustering & UMA modes help affinitize data with the computation, keeping the latencies local within a tile. we also improved our QoS capabilities to ensure that caches & memory bandwidth can be properly shared

	[Q19-1] It's true that a monolithic die could result in higher static latencies, but tile-stitches are shared resources and it has to be shared across instruction dispatch, io and memory accesses. Doesn't that introduce variability with high utilization? Is there a crediting system to maintain some baseline performance?
		> the tile stitches connect every row to every adjacent row and every column to every adjacent column so it effectively stitches the entire mesh across tiles with 0 loss of bandwidth. the mesh itself is credited and that crediting extends to the inter tile bridges

	[Q19-2] Is clustering effectively pinning to a particular tile?
		> and yes, clustering is effectively pinning to a particular tile

	[Q19-3] the single-digit latency refers to the round-trip latency (request+response), right?
		>low single digit latency refers to a single crossing, so round trip would be twice that - think medium to high single digit latency for round trip

[Q20] What is the frequency at which AMX runs?
	> AMX frequencies will be same as AVX-512.  we are not revealing freq #s at this time

[Q21] Can you talk more about HBM in Sapphire Rapids, specifically size, hierarchy etc.?
	> we're also not revealing HBM size details at this time.  hierarchy is as stated here - flat mode with HBM only or HBM+DDR or using HBM as a DDR backed cache

[Q27] Does Sapphire Rapids have two mirror design dies?
[Q33] A slide made the 4 tiles appear to be mirrored over the horizontal and vertical axes.  Does this require upper left, upper right, lower left, and lower right chiplets and presumably multiple sets of masks?
	> There are 1 logical but 2 physical tiles.  we have a base tile, a mirrored version of it (so we are manufacturing 2 tiles, mirrors).  the upper 2 tiles are the base + mirror and the lower 2 tiles are just a rotation of the upper tiles

[Q34] Has Intel seen demand for a variant of Sapphire Rapids that utilizes one of the new Xe Tiles with HBM2e and a couple CPU Tiles?   I can't help but notice a ~400mm^2 512 EU die could slot into the same space occupied by a SPR CPU Tile.
	> the tiled arch allows for all sorts of interesting SoC designs - there are number of interesting ones we are looking at but there are no production plans just yet

...

[Q ]
	>