Reviewing Plugable and Asustor 2.5Gbe Dongles on MacOS Monterey 12.5.1
Preface
I’ve always loved MacOS and how efficient and elegant it is compared to Windows. However, since many of my frequently used apps are not MacOS compatible, I had to use my Windows PC. In this scorching summer, I was finally fed up by the room temperature constantly exceeding 87F, so I decided to pull the plug on my PC, which has a 16 core AMD Ryzen 9 5950X CPU, a 3080 Ti GPU, and 64 GB ram, relocated it to another room that no one lives in, set up Wake-on-LAN, and used Parsec to connect to it remotely.
Since then, things have been better. This MacBook Pro 16 2019 that I am using, has an 8 core Intel Core i9, an AMD 5500M GPU, and “only” 16 GB ram.
The whole laptop consumes about 100 watts at peak, where normal operation sits between 40-55 watts, which is lower than my PC at idle. Just by switching to the laptop, my room temperature never exceeded 83F.
But one thing that I missed the most, was my 10Gbe NIC in my PC. Uploading and downloading from the NAS was blazing fast, and I can make backups and snapshots of my PC very frequently, without having to deal with the mere 110 MB/s that a 1Gbe LAN provides. Naturally, I started looking at options that can improve my Mac’s network performance.
I am using a Thunderbolt dock from CalDigit called TS3 Plus. It has a 1Gbe LAN built in and it is very reliable. With that said, being a 1Gbe LAN, it is not bad, but can be certainly better. CalDigit now sells an upgraded version called TS4, which has 2.5Gbe built in. This dock costs $379.99 on B&H, it is rather expensive given I am only paying for the 2.5Gbe upgrade only. Furthermore, TS4 uses Thunderbolt 4, which is not supported on this 2019 MacBook Pro 16 I am using. So, I started the search for usable 2.5Gbe dongles.
The Search has Begun
Doing a quick google search for 2.5Gbe dongles turned up many results. Without wasting too much of your time, I will just summarize it as such that the only viable options are all provided by a Realtek NIC chip called RTL8156. It has two variants, RTL8156 and RTL8156B, the latter is supposed to be a revised version of the original 8156 by fixing some of its problems that plagued the users, such as frequent network drops and poor power management. With this knowledge in mind, I ordered two dongles from Amazon.
Both of these adapters claimed to be compatible with MacOS, without the need to install additional drivers. In the following sections, I will detail my test process and how each of these NICs performed.
The Driver
When plugged in, both of these dongles are easily identified by MacOS without any issue. They all show up in the Network settings as “USB 10/100/1G/2.5G LAN”.
By going into System Information, we can see these NICs are running Apple native drivers “com.apple.driver.usb.cdc.ncm”.
So, the claims that both of these companies make seems to be true that no additional driver is needed, totally plug-and-play.
However, Plugable has mentioned some driver-related issues on their website, note:
There are some minor issues using the adapter in macOS 11.x+. Please see the Get Started information on this product for details.
macOS (11.x+): No driver install available, use built-in NCM driver. Note: There are known issues with incorrect link rate reporting and packet counts on macOS 11.x+ With auto-negotiation, the adapter will get a 2.5Gbps connection, and the device operates normally, but it will appear in macOS that the link rate is only 1000Mbps. Jumbo frames are not currently supported on macOS 11.x+. We are working with Realtek on drivers to resolve these issues.
And this “minor problem” as they called it, isn’t so minor after all. From my initial findings, both RTL8156-based and RTL8156B-based NICs have the same problem. If you go into Network settings and try to change the MTU, it will not have any options for you to choose from, other than the default 1500.
Furthermore, if you use manual settings, the NICs won’t work properly. In the actual testing below, I will be using “Automatically” as the manufacturer suggested.
Testing the Performance
Testing Environment
Point to Point Direct Connection
Host: Windows 10 PC, i9-10850K, MSI Z490-A Pro onboard Realtek RTL8125B-CG 2.5Gbe NIC, connected to QNAP QSW-M2108-2C 2.5Gbe switch, then connected to Brocade ICX 6610 switch via 10Gbe SFP fiber, then to the VM host via 40Gbe QSFP fiber.
Host Validation:
Ran iperf3 from the Host to one of my test VMs with paravirtualized 40Gbe Mellanox ConnectX-3.
iperf3.exe -c [server_IP] -t 30
The result is a very consistent 2.37 Gbits/s
Then, I ran it in reversed mode:
iperf3.exe -c [server_IP] -R -t 30
Total data transferred was 8.29 GB and again, a very consistent 2.37 Gbits/s was observed with 0 Retr. As you can see, this host machine is more than adequate to validate these 2.5Gbe dongles.
The Plugable 2.5Gbe NIC
First, I put Plugable NIC to the test. With no settings modified in the system. Also, I am connecting to my host PC with a 33 ft, or 10m long CAT6 cable. I’ve set the Host to have a IP address of 10.0.2.1 and my Mac with 10.0.2.2.
On the Windows host side:
iperf3.exe -s
On the Mac:
iperf3 -c 10.0.2.1 -t 30
The result is way less performant than Host machine’s onboard NIC, which utilizes PCIE. The Pluggable dongle achieved speed of 2.01 Gbits/sec, with 7.03 GB data transferred, compared to host’s 2.37 Gbps, a theoretical maximum accounting for TCP/IP connection overheads.
Next, we will run it in reversed mode:
iperf3 -c 10.0.2.1 -R -t 30
Things starting to get interesting. In reversed mode, which the Mac is “downloading” from the Host, the Plugable NIC exhibits very inconsistent performance. I ran the test for 5 times using the same command, speed can go as low as 872 Mbits/sec, which is less than 1Gbe, or as high as 1.64 Gbits/sec.
The low:
The high:
Without doing anything else, I immediately ran the original test again, without the -R flag.
As you can see, the Plugable NIC in Tx mode is still able to send data at 2.01 Gbits/sec.
The Asustor 2.5Gbe NIC
With the Windows host untouched, I unplugged the Plugable NIC (no pun intended) and plugged in the Asustor NIC which claims to have RTL8156B installed.
Then again, on the Mac side:
iperf3 -c 10.0.2.1 -t 30
Test result for Tx appears to be the same, 2.02 Gbits/sec and 7.05 GB transferred.
Time for the dreaded reversed mode with -R flag:
iperf3 -c 10.0.2.1 -R -t 30
The same problem persisted. Out of 5 runs, the best one was 1.22 Gbits/sec with 4.18 GB transferred. The worst run was 951 Mbits/sec with 3.32 GB transferred. As you can see, this isn’t what 2.5Gbe is supposed to do.
Worst:
Best:
The Culprit
Then, something got my attention. I’ve been running these dongles through the CalDigit TS3 Plus Thunderbolt 3 Dock using the front USB-C port. By the spec, this Thunderbolt 3 connection can provide up to 40 Gbps connection bandwidth. So, without giving it too much of a thought, I just blindly plugged in these dongles to the front 5 Gbps USB-C port of the Dock since I do not have other equipments installed that will saturate this 40 Gbps link, and these NICs are only rated for 2.5 Gbps, but that proved to be a bad idea in the coming section.
When I have the Plugable 2.5Gbe NIC plugged into the right side Thunderbolt 3 port on the Mac (the left side was occupied by the TS3 dock), the magic happened:
iperf3 -c 10.0.2.1 -R -t 30
The result was 2.37 Gbits/sec, with 8.27 GB data downloaded (to the Mac).
Then I ran the Tx test again:
iperf3 -c 10.0.2.1 -t 30
This time, the Tx speed was also at 2.37 Gbits/sec.
Now, it seems to indicate that the front USB-C port on the TS3 Plus is not able to handle the high data rate for these devices. A brief read on their specs revealed this information:
Interface
…
2 x USB-C
1 x USB-C 3.1 Gen 2 (10Gb/s, 1.5A, 7.5W) Rear
1 x USB-C 3.1 Gen 1 ( 5Gb/s, 1.5A, 7.5W) Front
So, the front USB-C Gen 1 port is “only” rated for 5 Gbps. Although it is perplexing that it can’t even handle a 2.5 Gbps device, let’s run it with the “rear” USB-C Gen 2 port which supposedly can handle 10 Gbps.
The result is crystal clear. Using the rear USB-C Gen 2 port, the Plugable NIC achieved 2.37 Gbits/sec upload, and 2.27 Gbits/sec of download.
Plugable NIC Upload:
Plugable NIC Download:
Now let’s try the Asustor NIC:
It achieved 2.35 Gbits/sec upload, and 2.27 Gbits/sec download. The difference between the two is very narrow.
Asustor NIC Upload:
Asustor NIC Download:
While the download speed is still not up to the full 2.37 Gbits/sec max, it is only less than 10% away from it. I’d be satisfied.
The Conclusion
Each of these NICs I have tested can and will achieve its rated bandwidth, given that you have installed them in the correct ports. In these 30-second short tests, I wasn’t able to stress them enough to have them show any signs of overheating or connection drops. In general, I will say that any RTL8156-based USB-C 2.5Gbe dongle will work under MacOS 12.5.1. Between these two sets of NICs, even though more expensive, I will still choose the Asustor one over the Plugable one because it uses RTL8156B. There are probably cheaper RTL8156B-based dongles out there, but Asustor's name in NAS space is something I can trust.
However, if you are curious as to why the test results above seemed weird, read on.
Extra Reading - How Thunderbolt Works, and Frustration with New USB Naming Schemes
The MacBook Pro 16 (2019) has two Thunderbolt 3 controllers, each gets 40 Gbps of bandwidth, or the equivalent of PCIe 3.0 x4. The CalDigit TS3 Plus dock that I am using, utilizes one of these two controllers to provide up to 40 Gbps of bandwidth to the peripherals connected.
One important detail I have omitted in the above testing section was the display I was using. The Dell U4919DW ultra-wide monitor, which is connected to the TS3 Plus’ rear Thunderbolt passthrough port, has a resolution of 5120x1440, which is like driving 2x 1440p monitors. Using this calculator, I learned that in order to drive this display, I need 13.27 Gbps of DisplayPort bandwidth.
According to this document, quote:
If a DisplayPort display or adapter is plugged in, the Thunderbolt 3-enabled system will detect this, and switch the pins driving the USB-C connector to the DisplayPort alternate mode. Thunderbolt 3 silicon will then act as a router to send raw DisplayPort traffic from the graphics engine within the system out over the USB-C connector pins, and pass that DisplayPort link directly to the display or adapter.
In this mode, a Thunderbolt 3-enabled USB-C port will support a single four-lane (4 x 5.4 Gbps, or HBR2) link of DisplayPort. These four links run across the two pairs of high-speed wires in the USB-C connector and cable. This kind of DisplayPort link can support a single, uncompressed display at 4K resolution at 60 Hz.
For DisplayPort 1.2, the maximum bandwidth on the Thunderbolt interface is about 17 Gbps of data, so for a 20 Gbps Thunderbolt link, a single DisplayPort 1.2 interface can be used, and for a 40 Gbps Thunderbolt link, two DisplayPort interfaces can be connected.
Apparently, two Thunderbolt 3 lanes, or 20 out of the 40 Gbps, has been consumed by this monitor alone. That leaves me with 20 Gbps to work with. Indeed, I have other peripherals connected, such as a Focusrite USB DAC, and a Logitech Unify Receiver for my mouse. However, I don’t believe any of these will consume that much bandwidth, leaving the USB dongles to perform at less than 2.5Gbe in the original testing.
Then I started to dig deeper into CalDigit’s dock, then I found this webpage that explains the bandwidth allocation for the ports.
Below you will find TS3 Plus’ USB and PCIe Interfaces bandwidth allocation and a diagram. This will help you to balance the resources and maximize your devices’ performance.
Group 1: Share the same USB Bus Controller up to 5 Gbps. These ports include the front USB ports and all audio ports (input and output ports on the front as well as the optical S/PDIF port on the back)
Group 2: Share the same USB Bus Controller up to 5 Gbps. These ports include the front SD-Card Reader as well as the 3 USB-A ports on the far right of the back of the TS3 Plus.
Group 3: Share the same USB Bus Controller up to 10 Gbps. These ports include the back 10Gbps USB-C port and the USB-A port to the right of it.
Group 4: PCIe Gigabit Ethernet Controller . This is dedicated to the ethernet port.
As you can see, the front USB-C port’s bandwidth is shared only with the front type A port. However, I did not have anything connected to the type A port while I was doing the testing. With that being said, please note that this port is capable of USB 3.2 5 Gbps, which is now named as USB 3.2 Gen 1x1, or previously known as USB 3.1 Gen 1, or just USB 3.0. Although capable of 5 Gbps, it is definitely an older standard, with a new name attached to it.
Unsurprisingly, the rear USB-C port, is capable of 10 Gbps. While unconfirmed, my best guess it is the USB 3.2 Gen 2x1 standard since it supports both type A and type C devices.
(Source: https://www.kingston.com/en/usb-flash-drives/usb-30 )
In the product listing of the Asustor NIC, it clearly states that this device is USB 3.2 Gen 2, and for the Plugable one, it only states USB 3.0 without further specifying the actual interface substandard. Given how it performed similarly with the Asustor NIC, I believe that this is also a USB 3.2 Gen 2 device as well.
So, the mystery has been solved. Now I know mixing USB 3 devices to different standard ports is a bad idea, but things won’t even need to be this complicated had the USB Implementers Forum (USB-IF) didn’t mess with their naming scheme up so bad in the first place. Furthermore, why a 5 Gbps capable interface can’t drive a 2.5 Gbps device is also absolutely nuts. In the end, I know what works and what doesn’t, so that’s a win for me.