Consul Gossip Lab Verify Broken

Bug 1: gossip enabled, same key, encryption disabled

Screenshot of checks and values on node:

You can see that the values are correct, but it’s check is not working.

Steps to Reproduce

# STEP 1: Configure Node01 with key and gossip encryption off
cat << EOF > /etc/consul.d/config.hcl
log_level        = "INFO"
server           = true
bootstrap_expect = 1
ui_config {
  enabled = true
}
datacenter = "consul-cluster"
data_dir   = "/opt/consul/data"

client_addr    = "0.0.0.0"
bind_addr      = "$(hostname -i)"
advertise_addr = "$(hostname -i)"
acl = {
  enabled        = true
  default_policy = "allow"
  down_policy    = "extend-cache"
}

encrypt = "$(consul keygen)"
encrypt_verify_incoming = false
encrypt_verify_outgoing = false
EOF

# STEP 2: configure Node02/03 w same key, encryption off, and restart consul
for NODE in node0{2..3}; do
  ssh $NODE "
hostname -i > ~/HOSTNAME
echo $(grep -oP '(?<=encrypt = ).*' /etc/consul.d/config.hcl) > ~/KEYGEN

cat << EOF > /etc/consul.d/config.hcl
log_level  = "INFO"
server     = false
datacenter = "consul-cluster"
data_dir   = "/opt/consul/data"

client_addr    = "0.0.0.0"
bind_addr      = "HOSTNAME"
advertise_addr = "HOSTNAME"

encrypt = "KEYGEN"
encrypt_verify_incoming = false
encrypt_verify_outgoing = false
EOF
"

ssh $NODE '
sed -i "s/HOSTNAME/$(cat ~/HOSTNAME)/" /etc/consul.d/config.hcl
sed -i "s|KEYGEN|\"$(cat ~/KEYGEN)\"|" /etc/consul.d/config.hcl
'

ssh $NODE systemctl restart consul
done

# STEP 3: run check

The checks for incoming and outgoing are both broken.

Bug 2: outgoing check

Steps to Reproduce

for NODE in node0{2..3}; do
  ssh $NODE '
sed -i '/encrypt_verify_outgoing/s/false/true/' /etc/consul.d/config.hcl
systemctl restart consul
'
done

Bug 3: incoming check

for NODE in node0{2..3}; do
  ssh $NODE '
sed -i '/encrypt_verify_incoming/s/false/true/' /etc/consul.d/config.hcl
systemctl restart consul
'
done

These two are non-bugs. I did not apply changes to node01.

I ran through this again, and I could not reproduce it.

For the steps, I simplified the steps and ran it again, and I could not reproduce it.

for NODE in node0{2..3}; do
    # save gossip key on remote host
    ssh $NODE "
echo $(grep -oP '(?<=encrypt = ).*' /etc/consul.d/config.hcl) > ~/KEYGEN
"
    # generate new configuration file
    ssh $NODE '
cat << EOF > /etc/consul.d/config.hcl
log_level  = "INFO"
server     = false
datacenter = "consul-cluster"
data_dir   = "/opt/consul/data"

client_addr    = "0.0.0.0"
bind_addr      = "$(hostname -i)"
advertise_addr = "$(hostname -i)"

encrypt = "KEYGEN"
encrypt_verify_incoming = false
encrypt_verify_outgoing = false
EOF

sed -i "s|KEYGEN|$(cat ~/KEYGEN)|" /etc/consul.d/config.hcl
'
    # restart remote consul service
    ssh $NODE systemctl restart consul
done