Towards Better Block-based Programming

Block-based programming is in a bit of a rough spot in FIRST.

LabVIEW

LabVIEW, once used by a third of all teams, now sees less usage than C++, and only ~181 teams are reported using it. Vendor support, while often present, is second-class in maintenance priorities to Java and C++. But more importantly, it lacks the same volume of libraries and community support that languages that use WPILib enjoy.

This is a really big problem for LabVIEW teams: as swerve is now the dominant competitive drivetrain in FRC, it means there’s far less software support and plug-and-play libraries available for them – your best bet is to look at some other team who has figured it out, and pray you can figure out how to fix it. This has come up in recent discussions on LabVIEW swerve and the appropriateness of suggesting that teams switch off LabVIEW has become a point of contention. Which…isn’t a great look for the current state of the language or its ecosystem in FRC.

The low usage numbers have other effects too in volunteer and mentor availability – I suspect fewer CSAs have ever used LabVIEW than Python even if they have never run RobotPy with a team before. And anecdotally it seems far less people know how to help LabVIEW issues online than in any other language.

Moreover, LabVIEW isn’t a hard requirement for the 2027 RFP either in the same way Java, C++, and Python are, so the long-term viability of the software is in question if NI doesn’t submit a bid. Which, after their recent acquisition by Emerson, seems particularly likely to me.

Blocks

In FTC, we use Blocks. We used to have LabVIEW in the NXT era but I don’t know much about how that went over. It’s backed by a JavaScript API which somehow calls the underlying Java systems of the robot controller Android app. There’s an official tutorial here that guides new teams on how to build simple programs, which I personally think is quite nice.

Like LabVIEW, it’s limited by the lack of version control or collaboration tooling, but I’m personally just not sold on the core execution. Here’s an example program from FIRST’s official guide:

And here’s the roughly equivalent Java code:

// some boilerplate omitted
@Override
public void runOpMode() {
    // Put initialization code here
    // vestigial silliness inherited from a now-illegal control system:
    DcMotor motorTest = hardwareMap.get(DcMotor.class, "motorTest");
    waitForStart();
    double tgtPower = 0;
    while (opModeIsActive()) {
        // Put loop code here
        tgtPower = -gamepad1.left_stick_y;
        motorTest.setPower(tgtPower);
        telemetry.addData("Target Power", tgtPower);
        telemetry.addData("Motor Power", motorTest.getPower());
        telemetry.update();
    }
}

Fundamentally, the Blocks code isn’t less complex than the Java code. Blocks just dresses up the syntax a little bit. And this rubs me the wrong way. It’s like focusing on making punctuation easier without paying attention to grammar, so to speak.

When I was in FLL, if you wanted to move your robot a fixed distance, it was a single move block. The block would take care of steering, counting encoder ticks, and it was really easy to use. I’m not entirely convinced Blocks is that much easier than Java despite the claims.

On the bright side, you can now use Blocks with external Java libraries, so I guess that’s a step above LabVIEW.

So how do we make something better?

I’m not entirely sure.

It should probably build on top of WPILib and be able to use WPILib-targted libraries

…rather than be mutually exclusive to WPILib. If it was easy to add LabVIEW support to something like YAGSL I’d imagine it’d already exist. If you have a system that hopes for a ton of outside development specifically for your block-based language, it’s probably ending up like LabVIEW again.

Have simpler abstractions

I’m not sure how feasible it is but the simplicity that NXT-G and Scratch have appeals to me. I will say that Scratch-likes handle control flow and variables a lot better than NXT-G does.

Make it git-diffable…?

If you encoded every block as a line of text somewhere, you could conceivably have block code that can be visually merged in git. I don’t know how feasible this is though

I don’t really know a good answer here…

…and I’m really curious what y’all think. Please leave your comments!

6 Likes

As we get the new control system that services both FRC and FTC in 2027-2028 for FRC and 2028-2029 for FTC, I hope that we will see both code sets merge closer together in general. I second blocks over LabView.

At FRC level though I think we should be past the block based graphic programming and really emphasizing text based code. That is what most will encounter IRL unless they are working on PLCs (which use ladder logic and do exist still but are going away over time and replaced with microcontrollers running C, Python or something else text based).

In FTC I can see the merit to having middle schoolers learn with block based. FRC is exclusive to highschoolers and we can set the bar higher imo.

2 Likes

One comment: is it worth keeping visual ā€œblockā€ programming alive? Case in point: with the new SpikePrime controller for LEGO, FLL teams can program their robot in Python, and I know that some already are.

9 Likes

If you’re going to use ~ to mean +/- 81 teams then should also say LabVIEW teams ~= C++ teams.

1 Like

Biased but informed personal opinion - Mathworks offers free licenses for Simulink, which would be a good solution.

It’s like Labview, but platform agnostic - at the end of the day, your model generates to C code. This can be integrated into an existing robot project, and mixed/matched with other languages for a ā€œbest of all worldsā€ result.

Team 2040 has been exploring this approach with solid success so far.

Alternatively, Simulink’s code generator could be hooked to just spit out a deployable robot project, preventing the user from needing to write any text-based code at all (a la labview today).

I can see a future where the RFP drives this answer. I can also see a future where the controller just has scratch onboard, and people use that.

5 Likes

I have mixed thoughts. I pretty much saw writing on wall for LabVIEW when switching in 2019. I had struggled to get students into programming with it and the resources available were getting stale. I haven’t had much more success with students with Java, but any amount helps and the community also is a major help.

While I think it should be reasonable for 9-12th graders to pickup coding and textual languages it still isn’t universal particularly in first year. There is some ease to blocks comparatively especially when having supporting materials. Last week a student that definitely would struggle learning Java or python was keeping up with the blocks programming for the XRP sparkfun video. The LEGO (colored blocked) for programming is something that more of our students than I’d like to admit could use, until they learn enough to be confident in python or Java. The XRP blocks ide has a menu selection to display the python code and so the combination would be powerful teaching for new students. Even if the block code wasn’t gitdiffable the underlying python would be.

I don’t know what sort of limitations you’d find for the different parts of code like swerve, pathing, and logging libraries and the more complex block structures they may require, but it may be possible to provide some introductory programming material that is transitioning them towards python.

Just a trailing thought was how LabVIEW gave some opportunities for certificates that might be used outside FRC, and that so it is good to consider opportunities for students that master programming and I see less path forward for block there comparatively to LabVIEW but I could see it as a step some students would pickup better to start.

2 Likes

This is a pretty good question. I’m not sure, and per the 2027 RFP, I’m not sure FIRST is sure either. The exact verbiage is:

Support for a graphical or guided code generation experience is preferred.

Maybe rather than having something like Blocks which tends to have the same level of complexity anyway, we should make it easier to use text-based systems. Kind of like how we used to have GRIP for OpenCV pipelines.

Edited, my bad. For reference, C++ stands at about 216 teams.

Simulink is cool but I feel it leans more towards people who are already knee-deep in engineering and know how control systems are already setup and want to draw block diagrams. Which is still cool, but may or may not fulfill accessibility goals for block-based systems. It’s an interesting point to raise, however – using real industry tools has vocational applications. I’m still mixed on closed source solutions and their maintainability with respect to FRC though so I’m not sure if it makes sense as a primary option.

I think it really comes down to quality of implmentation. XRP’s blocks setup has good blocks that are easy to use. FTC’s implementation scarcely makes it more approachable than Java in my opinion. Something like this that XRP can do would be far more complicated in FTC Blocks, for example:

image

I want Jupyter notebooks! I want students to work in physics models and get working robot code!

2 Likes

Agree! Imo we could be focusing on making textual based languages more accessibly by providing more documentation, walkthrough guides and initial setup.
There are a lot of tools to help on that and it’s a skill that will long live on the kids professional journey.

I’m curious to see what CD thinks about this:

Will Python be used by more teams than LabVIEW during the 2024 season?
  • Yes
  • No
0 voters

The reason I ask is that I see it as a milestone for the lifecycles of both Python and LabVIEW. Python surpassing the market share of another language in its first year of official support would be an amazing feat, but on the other hand, it would be very telling about the state of LabVIEW that a new addition is immediately more popular.

This seems like a better fit for FRC than a full-on block language. I think there’s a revival of RobotBuilder in the works, and I’m interested to see how much it simplifies setting up the structure of your robot code.

1 Like

I dont know how it is these days, but when i was trying to get into programming with java in my senior year, it was pretty tough even with wpilib.

I had 0 prior experience, and my school did not offer a programming course.

I tried using scratch’s blocks to learn concepts but those blocks didnt work in my favor and ended up confusing me more.

Out of high school, i attempted a python course but performed mediocrely. My turning point was html. For some reason, seeing these blocks worked better than scratch. I did better with js and java, but i probably still cant fully do a frc robot yet.

Zero to autonomous seems like a great resource that all teams should be aware of. I would like to see more resources provided for teams underprepared like i was. WPILib’s docs are great, dont get me wrong; it still is a high learning curve for someone who has never touched software before.

This is where an equivalent to the old VEX EasyC program would be handy, methinks.

Program was written in blocks, but you could pull up the text easily enough if it wasn’t displayed by default next to the block code.

Stopped being used because cRIO.

3 Likes

I’ve posted this before in a few places in the past, but it’s worth reposting here. Block based programming is an essential tool to teach programming, especially for groups who are underrepresented in STEM fields, especially computer science.

High school students learning programming do better with block-based languages, and the impact is greatest for female and minority students

It’s for the reasons presented in the paper that I get frustrated when people disparage block/graphical programming languages. They have huge value in lowering the barriers to entry and making programming more accessible. This is absolutely critical in developing the next generation of problem solvers.

21 Likes

I don’t have much experience with block programming, just a bit of messing around with Lego Mindstorms. However, I would agree with the idea that block programming a whole robot is just cumbersome and not at all easier than using a textural language. I do know that every single student we have ever had on the team that has been taught to program with a block system hated it.

However, as to the question of how block programming could be better, it seems to me that the one possibility is to follow the lead of many modern UI systems, where there is a graphical editor, but with some kind of ā€˜code behind’. When done well the graphical and code parts are nicely decoupled so that someone can work on the graphical part without needing to know about the code. One area in FRC coding where this could definitely be applied is putting together commands in a command-based robot. We already have graphical editors for autonomous path following and it would be possible to have an editor that builds on that to allow graphical creation of a whole autonomous routine with multiple paths and ā€˜manipulator’ actions. This would increase the amount of easily accessible software creation that can be used to get students involved.

1 Like

I can agree with this definitely for EV3 custom blocks. Teaching young students how to use that effectively is hard.

When you have a really large complex program one giant graphical diagram is not helpful anymore. Seperated diagrams for each mode is better and that’s the custom blocks sort of for EV3. I still can’t just look at it and know what’s happening inside of that block. I like how in FTC in general even in blocks you have sperate smaller OpModes.

Even the example picture below is not a very complex program with lots of large 3+ case condition blocks. Decision blocks in graphical programming takes up so much more room than in text based. If they could make that system smaller on every iteration I’d be much happier. I shouldn’t need to scroll up and down just to read 4 cases in a decision. When you do have lots of large blocks then so much extra room is needed around everything else in that row and you lose all of that other free space. Now you have to zoom even further out to see it all. Eve. Better is doing tabbed conditions so when you look the screen space is better but you can’t see more than 1 case at a time.

Stuff like that makes it hard to do large projects in any gui based programming tool just due to the limitations of trying to fit it all physically together in some screen space.

FTC blocks is much better than this style of Block that EV3 based around for layout and also introducing more text vs pictures and connection lines

1 Like

I just want to point out that this is very wrong, in my experience. I’m speaking as someone who is currently using LabVIEW to capture and analyze the data from the Teledyne Chroma-D detector in an airborne imaging spectrometer. Fortunately I still had a license from my previous project, which involved using LabVIEW for realtime control of a precision 3D motion platform.

2 Likes

Yes, this is pretty close to what robotbuilder2 (thread, github) is intended to address. I’m on hiatus from the project (will most likely start back up after the 2024 season is over and I’ll have free time again) but there’s a few screenshots and videos showing prototypes for the basic parts of it.

It’s not purely block based like LabVIEW or blockly, but I don’t think that’s a bad thing. Letting users set up logic without needing to know how the command framework works - or even how to write code - is really the key thing. A block-based editor that just mirrors text code is useless when its users may not even know basic concepts like loops and if/else switches. I’m also aiming for robotbuilder2 to show code side-by-side with the GUI editor controls so there’s immediate feedback. The rb2 thread shows that off pretty well, I think, even in the early prototyping phase.

3 Likes

Data analytics sure, but you could do the same with Matlab or Python.

But for stuff like motion control, even the motor controllers support scripting. Maybe saying what they would run into IRL applies to mobile robotics more than all automation. It’s becoming harder to justify a large PLC only system for stuff like AGVs (Line following robots) and AMRs (natural navigation) when you need advanced motion profiling, multiple PID controllers, fast encoder feedback, etc. a Microcontroller can do this stuff normally, PLCs it’s an add on card.

In the last 10+ AGVs I’ve worked on they were all part PLC for safety sensors (OSSD Lidar, estop and 24v circuit control) and then the actual main CPU was a Microcontroller to handle speed control or a smart Motor Controller that could handle scripting. No AMR out there will be PLC only. Natural Navigation, slam, etc are not meant for ladder logic unless it’s a standalone module.

Not all PLCs are capable of what LabView can do. Some don’t even have an Absolute value function (Looking at you ClickPlus from Automation Direct).

IRL every company will have different PLC modules with a different software so learning general ladder logic is useful. But you spend more time learning each software suite than you do needing to learn syntax from PLC to PLC. I have not run into a customer yet with LabView as their ā€œprogramming
languageā€ for this purpose. I see mobile robotics and material handling as the natural extension of what we do in FRC. It’s mobile robots accomplishing a task, whether that’s towing, picking up, etc. it’s material handling. So we should be looking at what the mobile robotics industry is doing as what our students should be learning.

We’re a C++ team. What I say might also apply to Java teams.

It’s incredibly easy to gatekeep in software. There’s a motivation/time/effort barrier to becoming a successful programmer. Frankly, the pandemic gifted me with all 3. I had the time, effort, and motivation to spend a few thousand hours learning. Now, it’s incredibly easy to sip (black, strong, chest-hair-growing) coffee and tell the exasperated and struggling underclassman that their hands are soft and their work-ethic is lacking.

This is a problem.

We’re a student-led no-mentor team. It’s our job to educate the underclassmen to ensure the survival of the team, and it’s hard to integrate all the little details of the WPIib ecosystem into the already-full schedule of the high schooler.

Seriously. In addition to talking about cross-language generalities of your average AP CS course, I find it necessary to go over the following:

  • The compilation process
  • Why your linker error won’t go away
  • How to use the command-line
  • Move/Copy semantics
  • What the heck is Gradle and why is It a pain in the butt to use?
  • Stack/heap, segfaults (and how to debug them)
  • Pointers (and their ā€œsmartā€ children)
  • Git
  • Why you shouldn’t click on every link you find on the internet.
  • Why I’m making you run this script. I promise you I’m not hacking you

So why is this all necessary and why do we still use C++?

  1. When you do something, you should understand what you are doing. There is concern of the maintainability of block-based tools such as Labview, but I think the largest concern should actually be the maintainability of the projects created with these tools if the users don’t actually understand what is going on. There have been many times that I’ve reviewed concerning but functional code that only works on a technicality.
  2. It’s widely used in the industry for embedded development. Actually, C is probably more used, but it’s easy to transition from C++ to C.

IMHO, teams need places where they can get their questions answered quickly. Frankly, I don’t think Chief Delphi is there yet. I generally use Discord to answer my ā€œinternsā€™ā€ questions and diagnose issues. WPILib was made to lower the bar and raise the ceiling for entry into programming. Block-based programming certainly lowers the bar for entry into programming, but it tends to lower the ceiling as well. In my idealistic fantasy world, there is a surplus of mentors who are available 24/7 to answer questions and solve problems to lift teams up to that bar, but sadly IRL is IRL.

On a side note, perhaps LLMs could partially fulfill the role mentioned above.

Oh yeah. Aaron. LMK if your team needs questions answered

3 Likes

I’m not talking post-hoc analytics. I’m talking pulling data off a CameraLink framegrabber, processing the metadata, computing statistics, and displaying it all in realtime. 2048x512 frames, at 168 FPS. I could probably write custom drivers for the framegrabber if I really wanted to, and I might be able to kludge together a fast image display in matplotlib, but the time it would take me would cost a lot more than the ~$5k I paid for LabVIEW and vision.

Similarly, when I needed to build a system to read data from a USB3 camera, compute the centroid of the bright spot, and servo the stages to move the camera to center the spot, LabVIEW was the easy answer. I hacked the business logic together in half a day. Again, I could have gotten to the same result with OpenCV, the V4L Python interface, and the TCP protocol spec for the Newport driver, but that would have been a waste of taxpayer money.

Point is, LabVIEW is alive in well in industry, not least because it handles a lot of the drudgery (hardware interface, GUI housekeeping) for you. To be clear, I also do a ton of programming in C (microcontrollers), C++ (RAD750 etc.), and Python (data analysis, general hacking), but to say that LabVIEW is uncommon in industry, or niche, or on its way out, just isn’t true.

I’ll take your word on the PLC stuff, I’ve never touched one.

1 Like