NI Week Athena Announcement and Q&A Panel

[wilsonmw04]

Originally Posted by Greg McKaskle

I suppose I can respond to a few of the questions and comments.

The RFP description of boot time is in section 6, point 8. Basically, booted and connected in less than 40 seconds, but requests that it be minimized.

Yes the performance is measured. Yes people care about it. But it is hard to have official times when major features are missing. It isn't like you can simply buy one of these are resell it.

As for ftp, it can be enabled, but by default more secure protocols are used instead.

I've already discussed the issues with deployment. A number of performance issues were corrected, tests improved, and now that 2013 SW from NI is officially released, we can verify numbers all over again.

I'd try to give you numbers for LV 2013, but the cRIO that I have in my possession resulted from a team swap and it apparently needs to be cleaned. Right now it looks like a XMas tree ornament when I plug it in. Swarftastic.

Greg McKaskle

Thanks Greg for keeping up with this thread. It's nice to get info from the horse's mouth so to speak. Keep up the good work!

[magnets]

Originally Posted by apalrd

However, we have to wait for the robot to fully boot up (and netcomm/user code to fully initialize), and it takes a while to close everything on the target before downloading, and often fails there (requiring us to reboot and try again). In addition, we see a lot of 'Access Denied' errors when LV does funny things (like LV will disconnect but the cRio will still see it connected), and rebooting to get rid of them and download is also really annoying. So if they allowed it to download when VxWorks was up, and we didn't have to fail and reboot so often, it would be fine (for the actual download, not including the compile).

You probably already know this, but we've found it really helpful to disconnect and connect to the cRIO in the project explorer before we download code. It seems to fix the silly errors that require a reboot.

[Racer26]

Originally Posted by Greg McKaskle

I suppose I can respond to a few of the questions and comments.

The RFP description of boot time is in section 6, point 8. Basically, booted and connected in less than 40 seconds, but requests that it be minimized.

Yes the performance is measured. Yes people care about it. But it is hard to have official times when major features are missing. It isn't like you can simply buy one of these are resell it.

As for ftp, it can be enabled, but by default more secure protocols are used instead.

I've already discussed the issues with deployment. A number of performance issues were corrected, tests improved, and now that 2013 SW from NI is officially released, we can verify numbers all over again.

I'd try to give you numbers for LV 2013, but the cRIO that I have in my possession resulted from a team swap and it apparently needs to be cleaned. Right now it looks like a XMas tree ornament when I plug it in. Swarftastic.

Greg McKaskle

I just don't understand why FIRST is willing to put up with such slow boot times. 40s is an eternity. Most Windows computers boot up in about that much time, and they've got a lot more to do.

My real world experience with the old IFI PIC18F-based RC was that from power application to radio-link established and ready to perform useful work was approximately 3s on average.

Heck, some Windows 8 machines have gotten down into the 10s territory.

I'm sure I'm not the only one that remembers being able to reach down, flip the breaker on my robot, and drive it by the time I could get back to the controls. I don't understand why this isn't being made a priority requirement on a custom-built system.

[Greg McKaskle]

I think it is worthwhile distinguishing between a requirement for a minimally acceptable system and a goal or a metric by which proposals will be judged.

I think everyone wants it to boot faster. But most wifi systems I'm familiar with take 20 or 30 seconds to boot -- smartphones, routers, computers, etc. The more powerful or flexible the system, the longer it takes. Meanwhile, my first Nokia phone booted in a few seconds. I understand why the boot times of the phones differ, and the same technical reasons apply to the FRC controllers.

Greg McKaskle

[Racer26]

Originally Posted by Greg McKaskle

I think it is worthwhile distinguishing between a requirement for a minimally acceptable system and a goal or a metric by which proposals will be judged.

I think everyone wants it to boot faster. But most wifi systems I'm familiar with take 20 or 30 seconds to boot -- smartphones, routers, computers, etc. The more powerful or flexible the system, the longer it takes. Meanwhile, my first Nokia phone booted in a few seconds. I understand why the boot times of the phones differ, and the same technical reasons apply to the FRC controllers.

Greg McKaskle

See my earlier comments about the power being overkill for the application.

Also, if WiFi can't do better than 20 or 30s boot times, which I agree seems to be pretty normal among WiFi devices, then maybe WiFi is the wrong technology.

[magnets]

In my opinion, there's two ways the controller can work. We could have a simple, cheap controller based of a microcontroller (under 50MHz), just like IFI. This could only be programmed in C (or maybe LV), but it probably couldn't run Java. It wouldn't have an operating system, or an FPGA, and encoders/high speed counters would work off of interrupts. We would use a radio like IFI's and we wouldn't be sending the camera image to the driver station. Image processing would be done with a CMUcam through a serial connection, and the controller would cost <$200.

OR, we could go with what NI has given us, a system that's really advanced, really cool, and used in the real world. This system will run a rt os, like vxWorks, or NI's realtime Linux thing (that's really awesome). This controller has an FPGA, more I/O (like USB, ethernet, and CAN), but costs more. The dual core ARM9 SoC with FPGA with 256 MB ram is overkill for most teams, but i expect to see some really cool vision/kinect applications done on the robot. The problem is that this solution is significantly more difficult to implement. NI has only so much money and so many people to make this happen, so while certain distros of embedded linux can boot in <10 seconds, it's not going to happen for us.

Many people say that this trade off is not worth it, but would you really like to go back to the time when only really good teams could use PID loops, or when you had to use look up tables for trig functions, or you needed Kevin Watson's awesome code to make a great robot program? (remember things like this?)
In 05, I could not name a single team that could cap the vision tetra more than 10% of the time. If we had the same challenge again, teams could do it.

As for the compile times, most of the actual compiling/downloading aren't really that bad (except for sometimes in LV, when the no-app thing happens), it's the restarting of the controller. If you want to speed up development, use something that reads constants out of a text file stored on the robot (see the 2013 cheesy poof's code for inspirations).

Also, it's pretty spectacular how easy to use NI's current controller is, and the new one should be the same way. I don't know of any other platform with a dual core processor and an FPGA that's easy for an inexperience programmer to use. FPGA's and embedded systems that run vxWorks are usually way beyond what a kid in high school can program. We also get support from other teams and people like Greg McKaskle to help us work out our problems.

[protoserge]

Is it really that big of a deal that it takes 40 seconds (maximum) to boot? NI has until 2015 to make it faster. How is this system "overpowered"? By leveraging a platform they [NI] are implementing in the MyRIO and the Zynq-powered cRIO, the cost has been decreased from the previous FIRST cRIO and we end up with more capability for the 2015-2020 seasons.

This is 2013. We are doing more with these robots than simply converting joystick inputs into PWM outputs for motor controllers. Teams are integrating computer vision, obstacle recognition, and performing on-the-fly adjustments to their robot control system. If NI allows us even more capability and opens up the FPGA for programming, we could see immensely capable systems integrated into this new footprint without the need for coprocessors.

Only time will tell.

[Racer26]

Except roboRIO isn't used in the real world. Its a custom built solution just for FRC. That argument (kind of) worked with cRIO.

One of the biggest things about the real world is engineering within constraints. That used to be a part of the control system. Back in 2003, we only had 26 bytes of variable space. You had to be creative. I feel like cRIO and roboRIO as FRC control systems give too much power. I agree we needed more than the old IFI system was capable of, I just think cRIO was too big a leap, and roboRIO is that kind of leap again.

C is still the dominant language used in embedded applications, so I don't see that as a limit.

I honestly don't believe that teams being better today has much at all to do with the control system.

WPILib has had a profound impact on making it easier to program an FRC robot, and the sheer quantity of team growth means there are more smart people involved.

I would estimate though, that fewer than 1% of teams competing in 2013 did anything (except stream video to the DS) with the control system that wasn't possible in 2005. And those 1%? They're the ones with the resources to make maximal use of whatever system they're given.

A new control system should target rookie teams with simplicity, while being powerful enough that veterans can do some really cool stuff. Its a delicate balance to strike. I just feel that cRIO and to a greater extent roboRIO err too much on the side of raw power.

That said? I think roboRIO is a dramatic improvement to nearly all of cRIOs shortcomings as an FRC control system. (Footprint, weight, design, etc. Its just better).

[AdamHeard]

Originally Posted by magnets

In my opinion, there's two ways the controller can work. We could have a simple, cheap controller based of a microcontroller (under 50MHz), just like IFI. This could only be programmed in C (or maybe LV), but it probably couldn't run Java. It wouldn't have an operating system, or an FPGA, and encoders/high speed counters would work off of interrupts. We would use a radio like IFI's and we wouldn't be sending the camera image to the driver station. Image processing would be done with a CMUcam through a serial connection, and the controller would cost <$200.

OR, we could go with what NI has given us, a system that's really advanced, really cool, and used in the real world. This system will run a rt os, like vxWorks, or NI's realtime Linux thing (that's really awesome). This controller has an FPGA, more I/O (like USB, ethernet, and CAN), but costs more. The dual core ARM9 SoC with FPGA with 256 MB ram is overkill for most teams, but i expect to see some really cool vision/kinect applications done on the robot. The problem is that this solution is significantly more difficult to implement. NI has only so much money and so many people to make this happen, so while certain distros of embedded linux can boot in <10 seconds, it's not going to happen for us.

Many people say that this trade off is not worth it, but would you really like to go back to the time when only really good teams could use PID loops, or when you had to use look up tables for trig functions, or you needed Kevin Watson's awesome code to make a great robot program? (remember things like this?)
In 05, I could not name a single team that could cap the vision tetra more than 10% of the time. If we had the same challenge again, teams could do it.

As for the compile times, most of the actual compiling/downloading aren't really that bad (except for sometimes in LV, when the no-app thing happens), it's the restarting of the controller. If you want to speed up development, use something that reads constants out of a text file stored on the robot (see the 2013 cheesy poof's code for inspirations).

Also, it's pretty spectacular how easy to use NI's current controller is, and the new one should be the same way. I don't know of any other platform with a dual core processor and an FPGA that's easy for an inexperience programmer to use. FPGA's and embedded systems that run vxWorks are usually way beyond what a kid in high school can program. We also get support from other teams and people like Greg McKaskle to help us work out our problems.

Straw man all the way!

[jhersh]

Originally Posted by Racer26

Except roboRIO isn't used in the real world. Its a custom built solution just for FRC. That argument (kind of) worked with cRIO.

This is not a fair argument. While the packaging for roboRIO is customized for student robotics (not just FIRST) the platform is real-world. The software stack for roboRIO is not designed from the ground up with FIRST in mind, so you can't expect the trade-offs to select for things like 3 second boot times instead of fail-safe network connectivity (for example). While the 2015 control system team will work to optimize things further, there simply aren't enough of us to start over from scratch, optimizing to the utmost for FRC care-abouts. We have to start from the platform we're leveraging.

The only reason this controller is possible is because of how highly leveraged it is. NI invested 60 man-years of effort into building the NI Linux Real-Time platform. As important as FIRST is to us, there's no way I can see that anyone could justify such an investment or anything close solely for a donated / deep-discount product.

[magnets]

Originally Posted by Racer26

Except roboRIO isn't used in the real world.

If you want to be technical, roboRIO is not used in the real world. However, labview, c++, and java all are used in the real world, as is real time linux, ARM processors, FPGA's, and NI's hardware.

[Andy A.]

Originally Posted by Racer26

See my earlier comments about the power being overkill for the application.

Also, if WiFi can't do better than 20 or 30s boot times, which I agree seems to be pretty normal among WiFi devices, then maybe WiFi is the wrong technology.

Could you suggest another wireless technology that could support the bandwidth, field management tasks, documented standards and security requirements while remaining off the shelf and accessible to teams? I honestly can't think of one.

I remember the 900mhz radios from the IFI days. I'll take wifi over that any day of the week.

[apalrd]

Well, Vexnet can connect fast, just saying. And the Vexnet dongle is actually just a USB WiFi dongle repackaged, if you plug it into a computer it might find it as a generic-brand Wifi adapter. I don't have hard numbers, but the user manual suggests 'It usually takes 5 to 10 seconds to successfully establish a link'. I can measure one later if necessary, but this seems about right.

Also, we do a lot of initial development on tether (when we're doing hardcore logic work, not the later stages which are almost all calibration), and booting fast without the radio should be considered too.

I don't believe at all that you can't boot a controller capable of all of FRC's needs roughly as fast as the Vex system, with comparable times for tether and radio operation. You might have to better define what FRC's needs actually are.

[Racer26]

Originally Posted by Andy A.

I remember the 900mhz radios from the IFI days. I'll take wifi over that any day of the week.

An interesting view to be sure:

I'm unsure why you would have it though...

Those 900MHz radios (Rebranded EWave Inc. Screamer422's) were ready to go nearly instantaneously, and capable of 9.6kbps. a little over 1KB/s.

4 years of FRC experience with 2.4/5GHz wifi a/g/n has taught me that its an unreliable standard. Delays to matches are common. Sometimes robots refuse to connect, and they often drop connection mid match, PLUS, being such a widely accepted standard, with a large range of compatible devices, it opens the door to attacks such as what happened at Einstein 2012. They also experience issues because we're using consumer-grade electronics that were never designed for the sort of dynamic loading environment an FRC bot creates. We're using routers that were intended to sit under peoples desks at home and never move.

6 years of FRC experience with the 900MHz serial radio modems taught me that they are essentially bulletproof. I don't think I ever saw one fail in any fashion due to being roughhoused aboard an FRC bot, and I don't remember ever having a radio related match delay. Additionally, the 900MHz band is several orders of magnitude quieter in terms of noise from other consumer electronics, AND has longer range. Its also much tougher to attempt various kinds of attacks on the 900MHz band, as radios are less proliferous.

I certainly won't attempt to say you could shove streaming video over 9.6kbps. I know you can't. There are definitely other technologies that would be better suited than wifi, though.

I would estimate the bandwidth needed for a typical FRC bot carrying streaming video to be somewhere in the range of 2Mbps, allowing for 320x240 streaming video uncompressed, with some overhead for control comms.

[evanperryg]

Originally Posted by apalrd

The Vex guys can boot a Cortex and user code and establish radio link in ~15s. The PIC-based IFI would boot and establish radio link in 5s. Why are we sitting around a minute? And getting worse every year?

Very true. The specs on this thing are pretty impressive, yet (at this point) it is still pathetically slow.

Originally Posted by racer26

The truth of the matter is that we simply don't need that much processing horsepower.

I know I'm not the only one that feels 1 free donated one, and then needing to buy one each subsequent year is NOT a good solution.

You seem to forget that this thing isn't specifically designed for FRC. It was designed for student robotics programs. Some of those programs may require more processing performance than we need.

Why would you need to buy a new cRio every year? That's silly and a waste of money. My team owns 2 cRios, enough for the competition bot and a practice bot. Everything else can probably just use a vex signal splitter or an arduino.

Originally Posted by apalrd

I don't believe at all that you can't boot a controller capable of all of FRC's needs roughly as fast as the Vex system, with comparable times for tether and radio operation. You might have to better define what FRC's needs actually are.

My team knows, dead certain, that you can run an FRC bot as effectively off Vex as off of the cRio. In fact, we have our entire 2012 bot wired into a cortex and it works perfectly.

All in all, I am really excited for the new controller. I love how small it is, in particular. The cRio is really bulky for what we use it for, and this opens up more space for other electrical components. However, if the bootup times are still 40s at release I will be very disappointed. Seeing how great the specs are, I suspect that all they need to do is refine the code. Also, I assume it is too far into development at this point, but 4 Relay outputs doesn't seem like enough.