I'm personally not a fan of Baratza grinders. They're great at grinding, when they work...until they lose calibration or some part wears out. Baratza has outstanding support to help you when that happens, but well...for me, I'd just rather not have to deal with all that. You will definitely get some good advice regarding purchasing one of them, from people here with good experiences using them.
I am a huge fan of dedicated espresso grinders and then something else for "other" brew methods. One major reason I have for saying this is that it takes some practice and skill to dial-in a grind setting for espresso, and then once you're there, you really only want to make minor tweaks as beans age and weather changes in your environment. Going far from the setting requires you to repeat the dial in process...wasting a lot of time and beans.
There are several ways to combat sludge in french press methods...1) better filtration (such as an Espro press); 2) a better grinder (I'm a huge OE Lido fan); 3) using a sieve to remove fines (cumbersome, but definitely worth looking into to see if it's right for you).
I've not owned an Infinity, but my understanding is that's pretty much an entry level flat burr grinder. The Baratza equivalent, which is likely much, much better in the cup is the Encore. The Virtuoso is said to be a big step up. Not sure what I'd buy if I were in the market for a non-espresso grinder, honestly...perhaps a Greville Smart? Currently, I use my OE Lido for press pot, but then, I'm the only one drinking it, and I drink it mostly when I'm not at home, so it's a mobile setup for me. My daily go-to is my espresso bar.