NetworkTables needs to change.

[virtuald]

I agree that NetworkTables has been a problem for awhile, and that the code quality isn't what it should have been -- I should know, I've spent a ton of time in the code, and have found/fixed a number of deadlocks and race conditions in it. However, the ease of use it provides is great, and the tools that have been created around it are quite useful.

I haven't seen the rewritten code yet, but I believe the one doing it is my Python co-collaborator and I have great faith that it'll get done right this time.

I strongly disagree that it should be moved to UDP. There's a reason TCP exists -- namely, it takes care of retransmission/etc for you, so you can depend (ish) that the data you sent will eventually get there (with lots of caveats on this, of course). Coupled with an transmit-on-update-only scheme like NetworkTables uses, this works out great, particularly for large tables or for variables that don't get updated often (which for our usage, is over half of them).

In UDP, if a packet gets dropped, then you have to either implement the retransmission yourself, or you need to ensure that the data gets transmitted often enough that it doesn't matter that you lose the packets.

Off the top of my head, there are two ways you can deal with this in UDP. Send all the data in every packet (which for large tables, would be bad) or ensure that the data gets sent often (which is still bad, doesn't adequately deal with the retransmit problem, and for things that don't get updated often like boolean flags, is annoying to implement). Neither of these sound like attractive options to me.

The internal 100ms (I set it to 50ms on python) wait of NetworkTables is not necessarily a *performance* problem, it's a latency problem. And honestly, that amount of latency doesn't matter 99% of the time if you design your code appropriately. In the cases that it does matter (like the DS packets), then yes, this scheme doesn't work. It would be useful to have a low-latency option built in (maybe by using TCP_NODELAY properly), but I suspect it would be overused by teams.

One other reason that the poll loop exists is that it's very important to decouple network latency from your robot's main loop. Transmitting a packet on the network (even in UDP, particularly on the cRio, not so much on the RoboRIO) can take a non-trivial amount of time compared to the rest of things that you do in your robot's main loop -- particularly if your wireless connection has interference in it -- so sending the packets directly from your robot's main loop is a bad idea. They chose to decouple this with a separate transmit thread and a queue, but it could certainly be implemented more efficiently with an epoll-like event-based mechanism if desired. In fact, many of the performance/deadlock problems NT had initially (and, are still there to some degree) are related to improper locking that led to the main loop waiting for network packets to transmit.