Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
bolha.us is one of the many independent Mastodon servers you can use to participate in the fediverse.
We're a Brazilian IT Community. We love IT/DevOps/Cloud, but we also love to talk about life, the universe, and more. | Nós somos uma comunidade de TI Brasileira, gostamos de Dev/DevOps/Cloud e mais!

Administered by:

Server stats:

252
active users

bolha.us: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.6

#kh8s

1 post1 participant0 posts today
Ariel (🐿 arc)<p>This morning's *absolute* WTF moment:</p><p>Pod network traffic 100% packet loss outbound UNTIL I `tcpdump` it on the node, then it starts working fine.</p><p><a href="https://eigenmagic.net/tags/k8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>k8s</span></a> <a href="https://eigenmagic.net/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kubernetes</span></a> <a href="https://eigenmagic.net/tags/Cilium" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Cilium</span></a> <a href="https://eigenmagic.net/tags/kh8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kh8s</span></a> <a href="https://eigenmagic.net/tags/Networking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Networking</span></a> <a href="https://eigenmagic.net/tags/HomeLab" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HomeLab</span></a></p>
Ariel (🐿 arc)<p>Man Prometheus is a pain to recover once its data store is in any way out of shape. Did NOT help that it was buried inside Kubernetes inside a PVC.</p><p>Thankfully it was only Dev environment today but if this ever pages on Prod we're losing data as it stands.</p><p>I'll write something up for a run book but eesh.</p><p><a href="https://eigenmagic.net/tags/Prometheus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Prometheus</span></a> <a href="https://eigenmagic.net/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kubernetes</span></a> <a href="https://eigenmagic.net/tags/k8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>k8s</span></a> <a href="https://eigenmagic.net/tags/kh8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kh8s</span></a> <a href="https://eigenmagic.net/tags/Observability" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Observability</span></a> <a href="https://eigenmagic.net/tags/o11y" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>o11y</span></a></p>
Ariel (🐿 arc)<p>kubespray, kubeadm init, kind create etc</p><p><a href="https://eigenmagic.net/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kubernetes</span></a> <a href="https://eigenmagic.net/tags/k8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>k8s</span></a> <a href="https://eigenmagic.net/tags/kh8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kh8s</span></a> <a href="https://eigenmagic.net/tags/DevOps" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>DevOps</span></a> <a href="https://eigenmagic.net/tags/PlatformEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PlatformEngineering</span></a></p>
Ariel (🐿 arc)<p>Kubernetes is actually the specification, what you're referring to is Kubernetes' monster</p><p><a href="https://eigenmagic.net/tags/kh8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kh8s</span></a> <a href="https://eigenmagic.net/tags/k8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>k8s</span></a> <a href="https://eigenmagic.net/tags/kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kubernetes</span></a></p>
Ariel (🐿 arc)<p>Aaaaand we have a revelation. Dug through containerd source code and found the check, it's bailing cause loopback provides no IPs on the default interface which is eth0.</p><p>Not really sure how to fix it yet but it's a breakthrough when it was looking pretty stale.</p><p>So....is it even possible to run a pod with ONLY local loopback? No east-west, no access from the host, no vEth pairs just alone.</p><p><a href="https://eigenmagic.net/tags/CNI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>CNI</span></a> <a href="https://eigenmagic.net/tags/Containerd" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Containerd</span></a> <a href="https://eigenmagic.net/tags/kh8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kh8s</span></a> <a href="https://eigenmagic.net/tags/k8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>k8s</span></a> <a href="https://eigenmagic.net/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kubernetes</span></a></p>
Ariel (🐿 arc)<p>I used to think I didn't understand controller manager, now I see it's just a logic rat's nest tied together by their tails.</p><p><a href="https://eigenmagic.net/tags/Kubernetes" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Kubernetes</span></a> <a href="https://eigenmagic.net/tags/k8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>k8s</span></a> <a href="https://eigenmagic.net/tags/kh8s" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>kh8s</span></a></p>
ExploreLive feeds
About