File 6202afa5-x86-TSX-cope-with-deprecation-on-WHL-R-CFL-R.patch of Package xen.23582
# Commit ad9f7c3b2e0df38ad6d54f4769d4dccf765fbcee
# Date 2022-02-08 18:00:08 +0000
# Author Andrew Cooper <andrew.cooper3@citrix.com>
# Committer Andrew Cooper <andrew.cooper3@citrix.com>
x86/tsx: Cope with TSX deprecation on WHL-R/CFL-R
The February 2022 microcode is formally de-featuring TSX on the TAA-impacted
client CPUs. The backup TAA mitigation (VERW regaining its flushing side
effect) is being dropped, meaning that `smt=0 spec-ctrl=md-clear` no longer
protects against TAA on these parts.
The new functionality enumerates itself via the RTM_ALWAYS_ABORT CPUID
bit (the same as June 2021), but has its control in MSR_MCU_OPT_CTRL as
opposed to MSR_TSX_FORCE_ABORT.
TSX now defaults to being disabled on ucode load. Furthermore, if SGX is
enabled in the BIOS, TSX is locked and cannot be re-enabled. In this case,
override opt_tsx to 0, so the RTM/HLE CPUID bits get hidden by default.
While updating the command line documentation, take the opportunity to add a
paragraph explaining what TSX being disabled actually means, and how migration
compatibility works.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2242,7 +2242,9 @@ Several microcode updates are relevant:
Introduced MSR_TSX_CTRL on all TSX-enabled MDS_NO parts to date,
CLX/WHL-R/CFL-R, with the controls becoming architectural moving forward
and formally retiring HLE from the architecture. The user can disable TSX
- to mitigate TAA, and elect to hide the HLE/RTM CPUID bits.
+ to mitigate TAA, and elect to hide the HLE/RTM CPUID bits. Also causes
+ VERW to once-again flush the microarchiectural buffers in case a TAA
+ mitigation is wanted along with TSX being enabled.
* June 2021, removing the workaround for March 2019 on client CPUs and
formally de-featured TSX on SKL/KBL/WHL/CFL (Note: SKX still retains the
@@ -2250,19 +2252,32 @@ Several microcode updates are relevant:
PCR3 works fine, and TSX is disabled by default, but the user can re-enable
TSX at their own risk, accepting that the memory order erratum is unfixed.
+ * February 2022, removing the VERW flushing workaround from November 2019 on
+ client CPUs and formally de-featuring TSX on WHL-R/CFL-R (Note: CLX still
+ retains the VERW flushing workaround). TSX defaults to disabled, and is
+ locked off when SGX is enabled in the BIOS. When SGX is not enabled, TSX
+ can be re-enabled at the users own risk, as it reintroduces the TSX Async
+ Abort speculative vulnerability.
+
On systems with the ability to configure TSX, this boolean offers system wide
control of whether TSX is enabled or disabled.
+When TSX is disabled, transactions unconditionally abort. This is compatible
+with the TSX spec, which requires software to have a non-transactional path as
+a fallback. The RTM and HLE CPUID bits are hidden from VMs by default, but
+can be re-enabled if required. This allows VMs which previously saw RTM/HLE
+to be migrated in, although any TSX-enabled software will run with reduced
+performance.
+
+ * When TSX is locked off by firmware, `tsx=` is ignored and treated as
+ `false`.
+
* An explicit `tsx=` choice is honoured, even if it is `true` and would
result in a vulnerable system.
* When no explicit `tsx=` choice is given, parts vulnerable to TAA will be
mitigated by disabling TSX, as this is the lowest overhead option.
- If the use of TSX is important, the more expensive TAA mitigations can be
- opted in to with `smt=0 spec-ctrl=md-clear`, at which point TSX will remain
- active by default.
-
* When no explicit `tsx=` option is given, parts susceptible to the memory
ordering errata default to `true` to enable working TSX. Alternatively,
selecting `tsx=0` will disable TSX and restore PCR3 to a working state.
--- a/xen/include/asm-x86/msr-index.h
+++ b/xen/include/asm-x86/msr-index.h
@@ -71,6 +71,8 @@
#define MSR_MCU_OPT_CTRL 0x00000123
#define MCU_OPT_CTRL_RNGDS_MITG_DIS (_AC(1, ULL) << 0)
+#define MCU_OPT_CTRL_RTM_ALLOW (_AC(1, ULL) << 1)
+#define MCU_OPT_CTRL_RTM_LOCKED (_AC(1, ULL) << 2)
#define MSR_RTIT_OUTPUT_BASE 0x00000560
#define MSR_RTIT_OUTPUT_MASK 0x00000561
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -1185,11 +1185,14 @@ void __init init_speculation_mitigations
* the MDS mitigation of disabling HT and using VERW flushing.
*
* On CPUs which advertise MDS_NO, VERW has no flushing side effect until
- * the TSX_CTRL microcode is loaded, despite the MD_CLEAR CPUID bit being
+ * the TSX_CTRL microcode (Nov 2019), despite the MD_CLEAR CPUID bit being
* advertised, and there isn't a MD_CLEAR_2 flag to use...
*
+ * Furthermore, the VERW flushing side effect is removed again on client
+ * parts with the Feb 2022 microcode.
+ *
* If we're on affected hardware, able to do something about it (which
- * implies that VERW now works), no explicit TSX choice and traditional
+ * implies that VERW might work), no explicit TSX choice and traditional
* MDS mitigations (no-SMT, VERW) not obviosuly in use (someone might
* plausibly value TSX higher than Hyperthreading...), disable TSX to
* mitigate TAA.
--- a/xen/arch/x86/tsx.c
+++ b/xen/arch/x86/tsx.c
@@ -14,6 +14,9 @@
* This is arranged such that the bottom bit encodes whether TSX is actually
* disabled, while identifying various explicit (>=0) and implicit (<0)
* conditions.
+ *
+ * This option only has any effect on systems presenting a mechanism of
+ * controlling TSX behaviour, and where TSX isn't force-disabled by firmware.
*/
int8_t __read_mostly opt_tsx = -1;
int8_t __read_mostly cpu_has_tsx_ctrl = -1;
@@ -54,6 +57,66 @@ void tsx_init(void)
cpu_has_tsx_ctrl = !!(caps & ARCH_CAPS_TSX_CTRL);
has_rtm_always_abort = cpu_has_rtm_always_abort;
+ if ( cpu_has_tsx_ctrl && cpu_has_srbds_ctrl )
+ {
+ /*
+ * On a TAA-vulnerable or later part with at least the May 2020
+ * microcode mitigating SRBDS.
+ */
+ uint64_t val;
+
+ rdmsrl(MSR_MCU_OPT_CTRL, val);
+
+ /*
+ * Probe for the February 2022 microcode which de-features TSX on
+ * TAA-vulnerable client parts - WHL-R/CFL-R.
+ *
+ * RTM_ALWAYS_ABORT (read above) enumerates the new functionality,
+ * but is read as zero if MCU_OPT_CTRL.RTM_ALLOW has been set
+ * before we run. Undo this.
+ */
+ if ( val & MCU_OPT_CTRL_RTM_ALLOW )
+ has_rtm_always_abort = true;
+
+ if ( has_rtm_always_abort )
+ {
+ if ( val & MCU_OPT_CTRL_RTM_LOCKED )
+ {
+ /*
+ * If RTM_LOCKED is set, TSX is disabled because SGX is
+ * enabled, and there is nothing we can do. Override with
+ * tsx=0 so all other logic takes sensible actions.
+ */
+ printk(XENLOG_WARNING "TSX locked by firmware - disabling\n");
+ opt_tsx = 0;
+ }
+ else
+ {
+ /*
+ * Otherwise, set RTM_ALLOW. Not because we necessarily
+ * intend to enable RTM, but it prevents
+ * MSR_TSX_CTRL.RTM_DISABLE from being ignored, thus
+ * allowing the rest of the TSX selection logic to work as
+ * before.
+ */
+ val |= MCU_OPT_CTRL_RTM_ALLOW;
+ }
+
+ set_in_mcu_opt_ctrl(
+ MCU_OPT_CTRL_RTM_LOCKED | MCU_OPT_CTRL_RTM_ALLOW, val);
+
+ /*
+ * If no explicit tsx= option is provided, pick a default.
+ *
+ * With RTM_ALWAYS_ABORT, the default ucode behaviour is to
+ * disable, so match that. This does not override explicit user
+ * choices, or implicit choices as a side effect of spec-ctrl=0.
+ */
+ if ( opt_tsx == -1 )
+ opt_tsx = 0;
+ }
+ }
+
if ( cpu_has_tsx_force_abort )
{
/*
@@ -142,6 +205,19 @@ void tsx_init(void)
*/
if ( cpu_has_tsx_ctrl )
{
+ /*
+ * On a TAA-vulnerable part with at least the November 2019 microcode,
+ * or newer part with TAA fixed.
+ *
+ * Notes:
+ * - With the February 2022 microcode, if SGX has caused TSX to be
+ * locked off, opt_tsx is overridden to 0. TSX_CTRL.RTM_DISABLE is
+ * an ignored bit, but we write it such that it matches the
+ * behaviour enforced by microcode.
+ * - Otherwise, if SGX isn't enabled and TSX is available to be
+ * controlled, we have or will set MSR_MCU_OPT_CTRL.RTM_ALLOW to
+ * let TSX_CTRL.RTM_DISABLE be usable.
+ */
uint32_t hi, lo;
rdmsr(MSR_TSX_CTRL, lo, hi);
