so, two things… Money to burn and an idea where a 85mm could be used
You do not actually have to use a 1.4 or 1.2. A lot of portraits are taken with smaller aperture cause you don’t want just the eyeball in focus. Especially when it involves a group. For sure the 85 at the same f4 as your 24-105 wide open is sharper and bit better bokeh… but not a game changer. However for what I’ve heard the sigma 85 dg dn is nice.
I think it will be just fine with what you have and maybe you are even better of using 105 and even a bit more distance.
I really like the 50 I have, mine is nice and sharp even wide open, as is my 24-105. The panny is really good for walk around and portraits. Just get that one first and use it before committing to a 85.
In the past I really liked a 90/2 on Fuji, which is about a 135/2.8 full frame. Sigma does have a 135/1.8 which is regularly on the 2nd hand market. But is a big prime….I hope they will ever make a 135/2.8 like the 100/2.8 macro.