When a port with loopbacked cables to another port of the NIC are added to bond, both ports go down momentarily. When the second port is also enslaved before it can come up, it results in both ports staying down. 
We can see this behaviour when the ports are added to bond via webui or in quick succession in cli. Also when AVX is rebooted, the ports are added to bond by the backend process taken from conf file in seconds of each other resulting in this behaviour. 
Added delay while adding interface to bond to ensure the ports have time to come up.

After adding delay, able to consistently see interfaces coming up after reboot without any issue.

/var/log/messages:
Dec 19 09:30:13 AN kernel: bond6: Adding slave enp175s0f0

Dec 19 09:30:13 AN kernel: i40e 0000:af:00.0 enp175s0f0: already using mac address 00:0d:48:9a:a6:ed

Dec 19 09:30:13 AN kernel: i40e 0000:af:00.0 enp175s0f0: NIC Link is Down

Dec 19 09:30:13 AN kernel: IPv6: ADDRCONF(NETDEV_UP): enp175s0f0: link is not ready

Dec 19 09:30:13 AN kernel: bond6: Enslaving enp175s0f0 as a backup interface with a down link

Dec 19 09:30:13 AN kernel: i40e 0000:af:00.1 enp175s0f1: NIC Link is Down

Dec 19 09:30:15 AN kernel: i40e 0000:af:00.1 enp175s0f1: NIC Link is Up 25 Gbps Full Duplex, Flow Control: None

Dec 19 09:30:15 AN kernel: i40e 0000:af:00.0 enp175s0f0: NIC Link is Up 25 Gbps Full Duplex, Flow Control: None

Dec 19 09:30:15 AN kernel: bond6: link status definitely up for interface enp175s0f0, 25000 Mbps full duplex

Dec 19 09:31:13 AN kernel: bond6: Adding slave enp175s0f1

Dec 19 09:31:13 AN kernel: i40e 0000:af:00.1 enp175s0f1: set new mac address 00:0d:48:9a:a6:ed

Dec 19 09:31:13 AN kernel: i40e 0000:af:00.1 enp175s0f1: NIC Link is Down

Dec 19 09:31:13 AN kernel: IPv6: ADDRCONF(NETDEV_UP): enp175s0f1: link is not ready

Dec 19 09:31:13 AN kernel: bond6: Enslaving enp175s0f1 as a backup interface with a down link

Dec 19 09:31:13 AN kernel: i40e 0000:af:00.0 enp175s0f0: NIC Link is Down

Dec 19 09:31:13 AN kernel: bond6: link status definitely down for interface enp175s0f0, disabling it

Dec 19 09:31:15 AN kernel: i40e 0000:af:00.0 enp175s0f0: NIC Link is Up 25 Gbps Full Duplex, Flow Control: None

Dec 19 09:31:15 AN kernel: bond6: link status definitely up for interface enp175s0f0, 25000 Mbps full duplex

Dec 19 09:31:15 AN kernel: i40e 0000:af:00.1 enp175s0f1: NIC Link is Up 25 Gbps Full Duplex, Flow Control: None

Dec 19 09:31:16 AN kernel: bond6: link status definitely up for interface enp175s0f1, 25000 Mbps full duplex