Wednesday, March 17, 2021

Limit cloudflared upstream connections

Just archived my cloudflared patch repo. My patch was a hack to get around a run-away issue: when there are sudden in-rush of requests or network delay, cloudflared will create lots of connection to upstream DNS-over-HTTPS servers. This will trigger the upstream throttling cloudflared and causing it to create even more connections to upstream. The machine will ended up with high CPU usage and no DNS request being resolved.

That is because cloudflared used golang "http.Transport" for the connection without setting a max limit. My hack hard-coded the max number of connection to 2 to avoid the issue. But it is probably inappropriate if cloudflared is used in an enterprise environment.

Luckily someone worked on a fix by adding a command line parameter to specify the max connection. Just add "--max-upstream-conns num_con" as parameter when starting cloudflared.

No comments: