In our days High DPI monitors become more popular. But unfortunately, still there are issues with some applications on such monitors mostly on all operating systems. And Firefox on Linux is not an exclusion.
All what is written bellow mostly relevant to Firefox on Linux, because on Windows and Mac OS Firefox looks good out of the box on high DPI monitors.
Right now (when I’m writing this) in Linux only KDE can scale monitors out of the box.