I agree with the suggestion to have all four physical servers use your internet sources directly, preferably with default minimum and maximum poll intervals so the software can tune the poll intervals to the conditions as they change.

I also agree with creating a single domain name which has A/AAAA records for all four servers (one or the other, either 4 As or 4 AAAAs, as the clients should not treat the same server via IPv4 and IPv6 as distinct servers unless the network paths are actually distinct, which I doubt they are).

Further, I would suggest the four use each other as additional sources, using the same pool directive as the clients, e.g.

pool ntp.example.com iburst

They will refuse to use their own IP as a source and add the remaining three. Once all servers are referencing each other, I would also suggest enabling orphan mode. Just use “tos orphan 10” in ntp.conf to set the orphan stratum (higher than any normal stratum they operate at, low enough to avoid any downstream clients reaching stratum 15). In case your internet connection is lost, the four servers should then agree on one functioning server the remaining will sync to, providing a single freewheeling clock your entire network will use, rather than 4 different freewheeling clocks which would slowly diverge.