Hi Max,
I mainly shoot our baby right now,
Over the next 12-18 months, you will do different kind of photos than once your kid is 2 years and older. Within the first 12-18 months, the kid is either not moving a lot at all or is not stable enough to walk for itself.
If the kid/baby is not walking, your 50mm lens is perfect for portraits with a little bit environment and (if desired) shallow DOF. That is easy. The difficult part is, once the baby is more active, tries to walk and you always have to have one hand free to support it, hold it or catch it
When my kids have been at that age, I used a lot focal lengths like 28mm, 35mm and 50mm. But the most used FL was the 28mm, because I needed to be very close to the baby to hold it/ catch it and with such a short distance, even 35mm was too short for good images.
The other problem is handholding and using a camera one-handed, because your other hand needs almost always to be free, just in case
I tried normal DSLRs with all kind of different lenses, Fuji X100 (35mm) and Ricoh GR (28mm). By far the most used and most helpful camera was the Ricoh GR with its 28mm. Although it was only APS-C, had slower AF and noise was an issue with low light.
the 18-40mm is just such a dark lens
I agree. And DOF is impossible with it. But DOF is very important with these images, believe me.
I don't really like and need the ultra wide end
Then do not use FL below 28mm
The TTartisan 40mm f2 .... it is to close to my 50mm
Yes and no. Yes, it is too close to the 50mm and no, because you get more of the environment on the image than with a 50mm. But if you have to choose only one FL, I would opt for 28mm as described above.
Think also about what you want to carry around if you walk in the city, shopping with your wife and the baby etc. What else do you have to carry? diapers etc. These are huge bags with "baby stuff" you always have to have with you. Do not underestimate this.
At that time I was not in the mood to carry on top of this additional photo gear with me, except, if it is small and light, like a Ricoh GR or a smartphone. Even a Fuji X100 was too big, because I could not stow it away in a jacket when I needed 2 free hands immediately.
The less you take your camera with you, the less memories you will have. Smartphones are great, but you will regret it later, if you made most images with a smartphone. The image quality even of the best ones can not match APS-C or fullframe sensors. And you can not go back in time anymore to take better images.
Therefore I would think about all options available. Test the different FL with your 18-40 zoom and see how you feel about the results. Not regarding aperture or DOF, only about the FL 24mm/28mm/35mm/40mm
I have had the Sigma 24/3.5 lens. Very small, very light, But because of the FL not the best for close ups with babies IMHO. And 24mm with F3.5, there is no DOF

But each on his own. You have to find the FL, that you like the most.