On macOS Sonoma 14.6.1 (with Darwin kernel 23.6.0), I am booting 5 Hazelcast 3.12 nodes on localhost to emulate a split-brain situation. Every of those nodes listens on a particular port to the communication of different nodes and communicates with different nodes from the configured ports:
| listens on | 5701 | 5702 | 5703 | 5704 | 5705 |
|------------|-------|-------|-------|-------|-------|
| sends from | 33712 | 33721 | 33731 | 33741 | 33751 |
| | 33713 | 33723 | 33732 | 33742 | 33752 |
| | 33714 | 33724 | 33734 | 33743 | 33753 |
| | 33715 | 33725 | 33735 | 33745 | 33754 |
I needed to emulate a cut up mind situation by dropping all TCP packets between nodes 1,2,3 and nodes 4,5. For this I created a pf (Packet filter) guidelines in /and so on/pf.anchors/hazelcast:
block out fast on lo0 proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port 5704
block out fast on lo0 proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port 5704
block out fast on lo0 proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port 5704
block in fast on lo0 proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port 5704
block in fast on lo0 proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port 5704
block in fast on lo0 proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port 5704
block out fast on lo0 proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port 5705
block out fast on lo0 proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port 5705
block out fast on lo0 proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port 5705
block in fast on lo0 proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port 5705
block in fast on lo0 proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port 5705
block in fast on lo0 proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port 5705
block out fast on lo0 proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port 5701
block out fast on lo0 proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port 5702
block out fast on lo0 proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port 5703
block in fast on lo0 proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port 5701
block in fast on lo0 proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port 5702
block in fast on lo0 proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port 5703
block out fast on lo0 proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port 5701
block out fast on lo0 proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port 5702
block out fast on lo0 proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port 5703
block in fast on lo0 proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port 5701
block in fast on lo0 proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port 5702
block in fast on lo0 proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port 5703
On the finish of /and so on/pf.conf, I added:
anchor "hazelcast/*"
load anchor "hazelcast" from "/and so on/pf.anchors/hazelcast"
Then I ran the command:
sudo pfctl -Evf /and so on/pf.conf
Printed:
...
Loading anchor hazelcast from /and so on/pf.anchors/hazelcast
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port = 5701
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port = 5702
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port = 5703
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port = 5701
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port = 5702
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port = 5703
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port = 5704
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port = 5704
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port = 5704
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port = 5705
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port = 5705
block drop out fast on lo0 inet proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port = 5705
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port = 5701
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port = 5702
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33741:33745 to 127.0.0.1 port = 5703
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port = 5701
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port = 5702
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33751:33755 to 127.0.0.1 port = 5703
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port = 5704
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port = 5704
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port = 5704
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33711:33715 to 127.0.0.1 port = 5705
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33721:33725 to 127.0.0.1 port = 5705
block drop in fast on lo0 inet proto tcp from 127.0.0.1 port 33731:33735 to 127.0.0.1 port = 5705
pf enabled
Token : 14399845021355597821
Then I began nodes 1 and 4. Within the log of node 1, I noticed:
Initialized new cluster connection between /127.0.0.1:33715 and /127.0.0.1:5704
What am I doing fallacious?
