PSA pt 2: better videoconferencing at home on lousy links

Dave Taht <dave.taht@gmail.com> Fri, 24 April 2020 19:04 UTC

Return-Path: <dave.taht@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 86EC53A07AA; Fri, 24 Apr 2020 12:04:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.099
X-Spam-Level:
X-Spam-Status: No, score=-0.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URI_TRY_3LD=1.999] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Uobip1cdyTzZ; Fri, 24 Apr 2020 12:04:25 -0700 (PDT)
Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A52553A07A9; Fri, 24 Apr 2020 12:04:25 -0700 (PDT)
Received: by mail-io1-xd36.google.com with SMTP id e9so11503931iok.9; Fri, 24 Apr 2020 12:04:25 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=23CglMUgirq3NqJeQsKraNJXDD/N1dZ0HXbtUyraNm0=; b=OyidK72nNQsAuoyXDqsVUV2ZGpe0wOzi7/C/EaVKN9muaXYZEG5PHnl8Jonx2xO4O6 n++1Et4zszD/n3Mqw8qG7fBroUFGcjs4iyCkqPB5gfhH/GHcuh+mPtQLR6RpJh3J1f8G dsybAodUxTqU2Vl5g7vfYPEZn1rsVCjJYnDsWAXiHw3QVTL6C49Am8ZO0oOI3romN/Jz 2f2l09WHPIy85bPNFiTayIRJS+cCQ/9cupQg+BfyIXNYjzKOCmqMFt20izhmY+buG9b7 Pah9pUtwWe/O1io/t20DDrvAAiZpA81FXN/jusoW1+589iMKxa0Mwp0oxuO7rYGg0nBk 56og==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=23CglMUgirq3NqJeQsKraNJXDD/N1dZ0HXbtUyraNm0=; b=rsa4VeG9XW7IRzurRt4kxbcms+M2/aB7Dg7FxgQuTDHsxH5WoLNb5rDyhXWx4dZZxK 1AdTqV6zB0QQ01kAR7KBJ8299rQ16fsNkPoYYdob7qcgpqMzi9IJ1QMuQRmxW0QyHgBt oDD/jp4OduD8OuZskqTVGONQ0illMK697MDhZpubiHoq62lzaR9HvjkCxurJIKDlZYdO KQzMCjXXb5cwwp2cBfbcCC9LalCb/m0iepLVGqLJnP1U5l+W2rJYoaRuKCV3tcIH8D78 X8xeaF7WnLQ//61ctZOzPGEJXtFXJskBQdq9EOF0wAvVpSD4bkGZQ3zkHZxPXQybBjbf nRuA==
X-Gm-Message-State: AGi0PuYvpHw491At8U+thr+V8uAI6rTUAHgZQvEQeBghjfMLaeTcpIMp lKTP+3KA3y33PMY1y/d6g764O4hjxDlk1fhMK7VhsHZNhCI=
X-Google-Smtp-Source: APiQypIw8UUW92nA4aGbncCRGW07+DJkDzkQ4jWiUODE3LQsDR33cqEA63GaOiqVSxwlgQz9G1Q9aVSczi1+qj6FWJU=
X-Received: by 2002:a6b:b8d6:: with SMTP id i205mr10382913iof.123.1587755064208; Fri, 24 Apr 2020 12:04:24 -0700 (PDT)
MIME-Version: 1.0
From: Dave Taht <dave.taht@gmail.com>
Date: Fri, 24 Apr 2020 12:04:11 -0700
Message-ID: <CAA93jw6aWm3sxPNnuV9F_1aWuZ5qmto_9u9qFx-OPOXysKLndw@mail.gmail.com>
Subject: PSA pt 2: better videoconferencing at home on lousy links
To: tsvwg IETF list <tsvwg@ietf.org>, IETF Discussion Mailing List <ietf@ietf.org>
Cc: Jim Gettys <jg@freedesktop.org>, bloat <bloat@lists.bufferbloat.net>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/_08ghaFmktiJ53GG2TJ8CuNjjhU>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Apr 2020 19:04:28 -0000

Anyway for those of you trying to share the network with more apps and
people than normal
and have solid, stable videoconferencing no matter the load....

It's usually most important to fix the ISP caused bufferbloat. (
https://gettys.wordpress.com/ ),
and then the wifi.

For major bufferbloat-fixing latency improvements at home, below 400mbit, in the
commercial space, presently the evenroute v3 is benchmarking out as
the least bloated, ever, on
both the wifi and the uplink. They have both fq_codel on the wifi
(rfc8290) and cake on the ISP
down and uplinks. They autoconfig well and their "sag" feature for
detecting upstream
bloat seems to work fairly well, especially on dsl, and they are,
so far as I know, the only ones to get dsl framing more right out of the box.

(I have no financial relationship with evenroute, but they do use all our stuff)

100 people on this one, with a big download from someone else during
the test. near-zero latency impact, totally transparent videoconferencing,
even over wifi: http://www.dslreports.com/speedtest/62398495

Above 400Mbit I generally recommend rolling a custom x86 fw+sqm
solution built around openwrt, ubuntu linux, ipfire, untangle, or
(distant last, but if you like bsd), pfsense.  There's dozens of
others, of course...

free.fr's dsl routers do three tiers of fq_codel at "line rate"
because they wrote their own debloated dsl driver. Sadly, it's the
only debloated dsl driver I know of, and everybody else has to shape.

ubnt's edgerouter's sqm, and pfsense's implementations of fq_codel
both work ok, but require some configuration, eero's SQM
implementation is a little too "automatic" for my taste, and in all
these products don't do dsl compensation right, so you have to shape
much lower than is theoretically possible to get good latency. There
is an out of tree build of "cake" for the edgerouters, which
does do dsl framing compensation right. coping with link compensation
for cable, fiber, or ethernet is just fine on everything, however.

Of course, openwrt, dd-wrt, ipfire, asus-merlin, vyatta, gargoyle, a
couple dozen others, have been shipping a decent fq_codel +
sqm-derived system for the past 5 years or so, if you haven't taken
the time to configure yours, it might contribute to familial harmony
if you do.... if you have an old junked router in your closet worth
reflashing... some spare time...

netduma routers have a fq_codel derived sqm solution but it seems a bit flaky.

On just the wifi front...  All the broadcom wifi products I have tried... suck.
Mt76, ath9k, and and many commercial ath10k based products seem to work ok,
but until last month, openwrt's ath10k implementation sucked (
https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/  )
and the fixes are not in a shipping release. yet.

eero labs wifi has fq_codel and has an "automagic" sqm implementation but,
it's a little too automagic for my taste.

Google wifi does have fq_codel, but not sqm. I don't know what plume
does except that they do all sorts of snoopy things and claim it's great.
Google nest is terrible. The impact of cloudy security cams in general
on yer uplink can be terrible.

ll the cell phone - tether setups I've tried are terrible - 1.6 sec of bloat,
and terrible jitter, regardless.

mikrotik's PCQ is sfq-based and is not the greatest, but if that's all you
have, turn it on.

I have been finding out that while pie is often enabled on docsis 3.1
modems quite a few docsis 3.1 modems are being configured still in a
docsis 3.0 mode. You can find out if you are configured for 3.0 sometimes
by looking at the config screen (sometimes). Even then I'm unsure if
pie is enabled unless my upload speedtest is under 90ms and I see
random looking packet loss. still pie usually >10x better than what
docsis 3.0 used to do, when it is actually on. Regardless I slam
an openwrt box with sqm in front of it, as pie only fixes uplink bloat.

I recently had a report on AT&T fiber's 45Mbit symmetric service - over
480ms of bloat on the uplink: http://www.dslreports.com/speedtest/62767361
(They are turning the sqm-scripts on)

I'm not equipped to give recommendations for better videoconferencing
at home, on other gear. Prioritizing udp not on port 433 almost seems sane,
as does implementing sfq or wred if you have it. Saner solutions wanted,
zoom in particular wants all kinds of ports prioritized.

So far I've seen next to no attempts at diffserv usage either from the
videoconferencing world, in packet captures from zoom and other services.

On the ipv6 front, so far:

zoom does not use ipv6. bluejeans and google hangouts do. jitsi works
with ipv6 also. It's unclear if janus meetecho can be configured correctly
for ipv6 (any tips?). Haven't tried BBB (anyone?), however I'm told their
jitterbuffer was woefully misconfigured in the current release.

anyway...

in the hope that this helps with familial harmony when you try to share
the network at home...

I'd love to know you dslreports results and mitigations and more about
the QOE of
your videoconferences. And I do wish that the unending stream of
other PSAs and corporate blog posts about how the internet is holding
up would be replaced
with useful metrics to those videoconferencing a lot, about loss,
jitter and delay.

and why is rmcat dead?


dave




-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729