Yes, this doesn’t work. It’s been discussed a lot, like here:
Filter() and filter-like formulas, ModifyRows() and AddOrModifyRow(), nested FormulaMap() will all overwrite CurrentValue with the one from the current context. Currently there’s no way to access CurrentValue from the outer context.
There are workarounds though, and often it’s possible to restructure the formula to get the desired result. Not so easy in your case though, if you want to have a live formula, but it can be implemented with either actions or an extra table.