AI Foundry lab

The lab machines are wired together: Ethernet runs about 30 times faster than WiFi, and nothing uses it yet

← AI Foundry lab dashboard › Worklog

Covers Wed 7 Oct 2026 (week 41), 15:09–15:31 PDT: aifoundry1, aifoundry2 and aifoundry3 at Studio 45. Written the same afternoon from timed transfers between every pair of machines, run from aifoundry1.

Update, 14 October. Step 1 of To make it usable is done: aifoundry1 and aifoundry2 got fixed private addresses on their wired ports on 11 October and aifoundry3 on 13 October, and each machine then reached the other two over the cable in about 0.5 ms. The link is not yet dependable: all three ports lost it at about 22:35 on 13 October and got it back at 13:56 on 14 October, within seven seconds of each other, which points at the switch or its power, and in between the machines talked over WiFi. Each port has lost its link twice since its machine last started, so the cable and its switch still need a look on site. The measurements below are from 7 October and stand.

The three machines are cabled to one wired network, and it works. Every port came up at 1 Gb/s, every machine reaches the other two, and one TCP stream moves 929 Mb/s (116 MB/s) in every direction: the most a gigabit link carries. WiFi between the same machines moves a median 31 Mb/s (19–45), so the cable is about 30 times faster. A round trip takes 0.26 ms on the cable and about 5.9 ms over WiFi.

929 Mb/sEthernet, any machine to any other (116 MB/s)
31 Mb/sWiFi between the same machines, median (3.9 MB/s)
0.26 msEthernet round trip, against 5.9 ms over WiFi

But nothing uses the cable yet. The wired ports have no IP address. Nothing on that network hands addresses out (no DHCP server), so each machine's NetworkManager asks for 45 seconds, four times, gives up for 5 minutes and starts again. Until each port gets an address, all traffic between the machines, ssh and Tailscale included, still goes over WiFi. These tests used the ports' automatic IPv6 link-local addresses, which exist only while NetworkManager is asking: about 3 minutes in every 8. The fix is a fixed address on each wired port, a one-time step for whoever has root on the machines.

Transfer speed between two machines
Median over the six directions, one TCP stream, in Mb/s. Longer is faster.
Ethernet929 Mb/s
WiFi31 Mb/s
Tailscale (over WiFi)30 Mb/s
02505007501,000
Round trip between two machines
Median over the three pairs, a 64-byte message and its echo, in ms. Shorter is faster.
Ethernet0.26 ms
WiFi5.9 ms
Tailscale (over WiFi)5.2 ms
02468

Every pair, every path

Speed in Mb/s, one TCP stream for 8 seconds, counted by the receiver. Tailscale between these machines connects directly over the WiFi network, so it measures WiFi plus encryption.

From → toEthernetWiFiTailscale
aifoundry1 → aifoundry2928.518.822.2
aifoundry2 → aifoundry1928.522.628.0
aifoundry2 → aifoundry3928.544.839.0
aifoundry3 → aifoundry2928.5*32.931.9
aifoundry3 → aifoundry1928.529.126.6
aifoundry1 → aifoundry3928.4*37.635.1

* The first run in this direction overlapped a NetworkManager restart; the best of its runs is shown (see How it was measured).

Round trip in ms, median (99th percentile in brackets):

PairEthernetWiFiTailscale
aifoundry1 ↔ aifoundry20.26 (0.40)5.3 (48.8)5.2 (18.1)
aifoundry2 ↔ aifoundry30.25 (0.30)5.9 (47.6)5.9 (31.6)
aifoundry3 ↔ aifoundry10.26 (0.44)6.4 (25.4)5.0 (17.3)

What it means

Time to copy a file from one machine to another, at the speeds measured:

FileEthernet (929 Mb/s)WiFi (31 Mb/s)
1 GB (a model checkpoint)9 s4 min
10 GB86 s43 min
100 GB14 min7.2 h

WiFi is also uneven: its slowest round trips took up to 48.8 ms, against 0.44 ms on the cable.

Why 1 Gb/s

Each machine's wired port is an Aquantia AQC107, a 10 Gb/s controller that also runs at 2.5 and 5 Gb/s, and each offers all of those speeds. All three links settled at 1 Gb/s, so the switch (or the cabling) is gigabit. A 10 Gb/s switch would raise the ceiling up to ten times.

To make it usable

  1. A fixed address on each wired port (root, one command per machine) and a name for each, such as aifoundry2-eth. NetworkManager then stops asking for an address every few minutes.
  2. Tailscale should then pick the cable by itself. It offers every address a machine has and uses the fastest path it finds between two machines, so ssh aifoundryN, which goes through Tailscale, would ride the cable. To be checked once the ports have addresses.
  3. Plain ssh to a wired address needs a key. Between the lab machines, ssh works today only through Tailscale; to the wired address it was refused for lack of a key in ~/.ssh/authorized_keys.

How it was measured

The tool, the scripts and every result are in docs/reports/data/2026-10-07-lab-ethernet of the lab repository, with this page as docs/reports/2026-10-07-lab-ethernet.html.