So it turns out changing the firmware does the trick. Quoted e/os/ works, I didn’t try the older ones.
I knew that phone wasn’t fully updated but for some reason, I was afraid to get it ahead of e/os/ on security patches. So finally I updated it to second to last firmware (March 2024 security patch, J binary) and e/os/ installed and booted without any problems. I couldn’t do it before when phone was on April 2023. firmware, G binary.
For anyone that wants to do it on S20 Ultra, there is nothing to it - update your phone, use official twrp and install e/os/.
Big thanks to @ronnz98 , as always.