IIRC those were mapped to specific I/O and IRQ Ports back in the day.

  COM1: I/O port 0x3F8, IRQ 4
  COM2: I/O port 0x2F8, IRQ 3
  COM3: I/O port 0x3E8, IRQ 4
  COM4: I/O port 0x2E8, IRQ 3

I also remember, there was a while where there was no IRQ sharing. So some machines couldn't use all the COM ports available. Or you'd add in like a modem, and would have to disable the onboard COM port if it was using the same IRQ.