Every year, there are changes in WPILib that only a select few teams get to test out. The solution for teams wanting to test out new changes are to either wait, or to copy-paste a bunch of classes into their code.
Many of these changes are just API changes and don’t rely on stuff that is specific to the control system for that given year. If these new APIs that are added each year could be split into submodules to allow for easy importing, teams that want to test these changes in October can do so by upgrading one of the submodules they import.
The default project could have a dependency that imports all the submodules up to date for a given year, then later in the year teams can update specific submodules by adding an updated dependency for a single submodule (similar to how teams can either depend on old commands or new commands).
I’m not sure if that counts as a build system improvement. More of a request that some of allwpilib is restructured. But it probably involves changing some of the default dependencies if it does actually happen, so there, it’s a build system improvement. I know this is possible with the Java code. I’m unfamiliar with how this would work with C++ code. Maybe it’s similar or the same.
It looks like some of allwpilib was restructured for this year (such as the addition of wpimath), so maybe this is all already possible?
That’s definitely more on the allwpilib structure side to make something like that work.
However, there are issues with trying to do that. Its pretty rare that new functionality is 100% independent. New commands was really the only example in the past few years. All the characterization stuff kind of was, but adding the wpimath dependency for that actually tore through the entire build tree, and couldn’t just have been done independently. For things that actually can be done independently, we can publish them as vendor libraries for previous years temporarily without too much work, but this requires the change to actually be independent, which is not often the case.
The bigger issue is we often times upgrade the native and/or roboRIO compiler during the off-season, which results in us using newer language features. It’s possible this could happen with java too if we ever are able to upgrade the java version. Because of this, it might not be able for us to even build newer features for previous years. In 2019 into 2020 this was the case, as the rio compiler added c++ 17 support, and we heavily used it. In fact for ~4 months we didn’t even have a rio image that worked for wpilib builds.
I can definitely see Glass’s value of camera server with the data for live fields.
Here’s what we’ve wanted, and attempted to do, since 2015. I want to store the robot’s data into a database running on the DS laptop. The database tech doesn’t matter at the moment, and I can finagle whatever jdbc adapter we need. We’ve also figured out some of the bandwidth sharing, latency, and sudden shutdown issues. This data is very valuable to us. Currently we collect it via USB, with many error-prone/forgotten steps in post-processing.
Getting any custom data connection to work on a live field has been the issue since 2015. Suddenly the same connection code we had in 2014 (that won us a WCMP award for a custom heads up display) refused to connect while on the field in 2015. Fast forward a bit.
The really nice thing about NetworkTables’ connection configuration is that it connects regardless of whether the robot is on the field, connected in the classroom via wifi, connected via Ethernet, or connected via USB. It’s flawless. In 2020 I tried to replicate the C++ NT code in Java for just a quick “hello world” test but via one of the allowed ports. It worked in the classroom but not on the field. I had some ideas on what to try at our 2nd event, but then 2020 happened. Maybe the better option is for WPILib to create a way to generate a TCP or UDP connection that takes in the target port for the connection?
We want to stream data once through whatever protocol, then split the data on the driver’s station as we need it: one stream could convert it to NT for Glass if we’re on the field, another would route it to a database, and yet another would sent it to PlotJuggler or an AirTable account if we’re in the classroom. IMO the best way to do this is via a pub/sub or multicasting framework. NT kind of does this - but it seems like the only software that can directly use NT’s data is the first software to connect to NT - such as Shuffleboard in 2018/2019.
One of the big things we did for reliability (and the big reason a raw udp connection wouldn’t work), is when we spin up an NT client, we actually connect to 5 different targets.
In addition, the DS also has a port you can connect to which will give you the IP address of the connected robot. So technically we try 6 targets, and just pick the first one that actually connects.
A way to expose that logic and just return a raw socket in wpiutil might be a decent idea, to make things like that easier. But we’ve found you have to do all of those things to actually be reliable.
There are some tradeoffs here (bandwidth usage/latency), plus the fact nodes can come and go at any time, that make this a difficult nut to crack in the general sense. Pub/sub alone doesn’t solve it–the main area where pub/sub helps is reducing bandwidth usage by only sending each client what that particular client wants (NetworkTables sends everything to everyone). Some COTS pub/sub solutions (e.g. MQTT) are client/server (all messages pass through a central server and updates are sent individually to each client). Other COTS solutions like DDS/RTPS (used for ROS2) have options to use UDP multicast for transport, but multicast is painful on WiFi networks (somewhat less painful for FRC, as the FRC WiFi link only has two nodes in the competition environment, but this is a greater concern for classroom use). Intel has a “Distributed Pub/Sub for IOT” protocol that does true distributed peer-to-peer messaging, but I need to look more into how mature and performant it is (in particular, how does it figure out network latencies to determine what peer is connected to what). Peer-to-peer protocols are also often harder to debug than client-server, and debuggability by inexperienced users is a major concern for FRC.
One temporary solution is that it is relatively straightforward to create a NT “bridge” that would be a client to the robot and be a localhost server that other clients can connect to. This would reduce network bandwidth at the cost of increased latency to the other clients.
That being said, this is an opportune time for input as I’m collecting requirements and doing an analysis of alternatives for the next version of NetworkTables (some key requirements: communicating and timestamping every data change). Thanks for the input.
One thing I would ask that not be done is to couple the coding or the build & deploy process to VS Code. Things are good currently in that we can use other IDEs or editors since Gradle is managing everything and running the builds and deploys. While I understand the need to have a single “supported” IDE, many of us prefer to use other IDEs, often due to comfort levels and familiarity. For example, since I have used IntelliJ IDEA professionally for the past 18 years, and it’s so ingrained in my workflow and muscle memory, I use it for FRC. I prefer it so much that I wrote and maintain an FRC plugin for it. Given there are over 3,400 unique downloads of the plugin, I am not alone in this preference. We also use the IntelliJ IDEA EduTools plugin to teach Java to our students. (It provides in IDE lessons and coding exercises.) So they have familiarity with IntelliJ IDEA from that.
So I would just ask that the heavily lifting be done by Gradle and that VS Code and its FRC plugin be used purely for editing and project/example templates like it is now. Or at the minimum, that everything can be done directly via gradle itself if desired.
In the same vein, better documenting of the gradle command line commands in the documentation as alternatives alongside the VS Code steps would be helpful and appreciated.
How those interact is really not changing at all. We don’t plan on integrating the build setup any more into the IDE. On our side, we also really enjoy the flexibility of being able to deploy without VS Code. That’s not going away anytime soon, so you’re good there.
I have been working on the readmes as I go through rewriting all the plugins, so some of the commands should be documented. For Java specifically, there’s honestly nothing extra we add other then simulateJava, simulateExternalJava and deploy. Any of the other custom commands we add are all for C++ support.
While not specifically VSCode related, we would love for the dashboards (specifically shuffleboard) to be able to accommodate multiple screens. We are trying to revitalize the laptop that came with our original black tote KOP as our driverstation computer with a portable external monitor connected to it. It would be incredible if one instance of shuffleboard could support both the internal screen as well as the external one.
A few years ago, we wanted a camera display on an external monitor. We ended up modifying shuffleboard so it can run two instances. Much easier then trying to add multi window support.
There’s probably a good reason for that check we commented out, but didn’t run into any issues for our use case.
Wow, seriously, one line of code? That certainly is easy. Thank you for sharing. I agree there probably is a reason for throwing the error, but if it worked, it is a potential workaround, and we have a lot of time to test it at the moment.
I think a lot of the reason for only allowing a single instance is because shuffleboard is fairly heavy, and if you have multiple copies it can eat a lot of resources.
For multi window, multi window graphic APIs are a NIGHTMARE, and something we probably won’t look into. A better solution if you want something built in could be to use 2 different dashboards. So on the main screen you could use Shuffleboard, and on the secondary screen you could have a camera stream in Glass (once we finish adding that), and those will not interfere with each other.
Thank you!!! That is incredibly useful to know. That particular computer does not have a lot of memory to spare, so our testing could have gone fine, but then when we transfer to that machine, we could quickly hit roadblocks.
That makes a lot of sense. Thank you for letting us know.
That is what we were considering originally. Glass (or Web Components) seems perfect for this. It is good to know we are on the right track. Thank you every one.
I generally agree with this. I’ve had Windows swap which display head was in the 0-indexed slot, without warning.
If it’s just a web camera that isn’t being processed for data (e.g. vision processing), we have never had a problem directly connecting to the camera’s IP via a web browser while on the field. Just setup a shortcut on the Desktop by dragging the URL from the bar in Chrome to the Desktop, ezpz. This could avoid the case where 2 instances of SB are needed.
Now that we have a good approach for building C++ GUIs, one thing we’re looking at is creating more lightweight standalone applications using the Glass/Sim GUI widgets. For example, creating a standalone C++ version of OutlineViewer was only about 200 lines of code because it uses the same NetworkTables widget as Glass. A standalone application for camera viewing would be similarly very doable after we finish that widget for Glass. An ancillary benefit is that widget improvements in these standalone applications come back to Glass and Sim GUI as well. Code reuse–what a concept! We’re definitely open to more ideas along these lines.
The only real downside of separate dashboard-type applications is that each one gets a full copy of all NetworkTables data (which isn’t that much bandwidth, but is greater than zero). We’re also working on a solution for this.
Technical note re: multiple windows in one application, the graphics library we use for Glass et al (Dear ImGui) doesn’t currently support this. There’s a Dear ImGui development branch with very fancy features along these lines (called “docking”) which allows dragging individual widgets out of the main window (and they magically become separate system windows), but it’s pretty buggy on non-Windows platforms. There’s some patchsets floating around for multiple system windows that look more promising for us to use near-term if we want to explore this route, but I’ve not played with them yet.
Documentation for a formatter has just been added!
While we could’ve provided this information in the robot templates, that would then add an extra online dependency to a project (or we would have to cache this and store it locally). There were also some other issues with adding it to as the default. Documentation that’s pretty much entirely copy-pasta was the simple solution.
What’s the long term outlook for using Gradle, particularly with C++? It seems like the industry is moving towards Bazel and I can’t say I’ve ever seen Gradle used for C++ outside of WPILib except for Android apps. Is keeping up with industry practices a goal? Is this just a “we know it and it works well enough” situation? Wondering what things might look like 3-5 years out.
Gradle has a number of key advantages over other build systems:
It enables us to only have to maintain a single build system for both C++ and Java
Most build systems other than Gradle (and Maven) have very poor Java support
Most C++ build systems have no concept of package management or ability to work with Maven repositories (we need a way to distribute pre-built C++ libraries for multiple targets across multiple host platforms, and Maven has been working well for this)
For these reasons, I’m not really seeing us moving away from Gradle as the officially supported build system in the next 3-5 years unless there’s a massive break on the Gradle side. Bazel is interesting but the maturity and featureset is a little unclear at the moment, especially for Java.
One other thing to add onto what Peter said, is we can handle the deploy entirely inside of Gradle as well. Since its just running Java code, we can do whatever we want. I haven’t done a ton of research into Bazel, but it doesn’t look like integrating the deploy is easy to do. Also, they seem to only provide 64 bit Windows binaries, so we’d have to hope we’d be able to compile 32 bit binaries.
At the high level, we’re relatively happy with Gradle. It has its quirks, but we understand them well. The flaws are half with us, which is why GradleRIO is getting rewritten in Java instead of Groovy, and half are in some weirdness in the old gradle native setup, but we’d have those in any build system. And hopefully someday, we’ll be able to move to the new plugins that are more supported by Gradle. But at least for now until those plugins become powerful enough, there are no known plans to remove the old native plugins.
Now that we have been working with the Command Framework a bit, I would love it if the interface of VScode that remembers your most recent plugin choice for the dialog that pops up when you click the WPILib icon could be mimicked in the most create a new Class/Command dialog. I feel that someone (me) may accidentally add an Old Commands (or Subsytem) Class to their robot that uses the New Commands Framework because they come up first. It would be so nice if we saw the most used Class options come up in order. This is not a necessity by any means, so if a lot of work is involved, it may not be worth it, but it would be sweet as an option.
Here is a screenshot of the dialog to which I am referring.
Hmm. That’s not something built into vscode, but I think it would be pretty easy to reorder the templates with a most recently used algorithm and a local data store. Can you make an issue in the vscode-wpilib repo to remind me to look into it?
On the same coin, I should probably filter so only the ones applicable to the project show up by default. With an override option obviously, but default would probably be helpful.
