The CCRE: A comprehensive award-winning robot code framework

[Jaci]

Originally Posted by Colby Skeggs

Once we release v3.0.0, we plan to keep backwards compatibility for a relatively long time, which limits the extent of our possible refactorings.

Why not release as soon as possible? Most of your users will be during build-season start all the way through to a few months after champs due to offseason events. Just my opinion, but as soon as possible seems like the most optimal time to do some major refactoring.

[Cel Skeggs]

Originally Posted by Jaci

Why not release as soon as possible? Most of your users will be during build-season start all the way through to a few months after champs due to offseason events. Just my opinion, but as soon as possible seems like the most optimal time to do some major refactoring.

I'm not quite sure what you're saying - are you suggesting that we should release v3.0.0 as soon as possible? We do plan to - I've spent well over a hundred hours recently working on it. For anyone who wants to try it now, it IS available on the devel-3.x.x branch of our repo, but we won't release it (give it a final version number and mark it as stable) until it's ready.

If you mean in general that we should release all our changes as they happen, again, they are available, but we want to provide assurances to users that what they're doing won't break constantly as we change things. That's why we following Semantic Versioning practices in our code, which means only breaking backwards compatibility in major releases.

[Jaci]

Alrighty, I probably misunderstood something. All good, I look forward to seeing how the code evolves over time

[Cel Skeggs]

Originally Posted by Jaci

Alrighty, I probably misunderstood something. All good, I look forward to seeing how the code evolves over time

Alright! Thanks for your interest - and do feel free to suggest anything you think would make this better.

Aaaand... on a different topic, we're getting a solid amount of attention on GitHub:



If your team ends up using the CCRE during the season or plans to, please let me know about it and let me know anything I can do to help!

Also, I'd love to hear the experiences of anyone who's tried it and decided against it, since we want to make it as useful as possible.

[Cel Skeggs]

At long last! CCRE prerelease 3.0.0-pre1 is out! It's only been 3-4 months.

This is not quite yet the final 3.0.0 release, but it's close, it has a huge list of improvements, and it works!

Full changelog:

Prerelease of CCRE with API 3.0.0-pre1.

Highlights:

* Structure: Removed support for cRIO.
* Structure: Started using Java 8 features everywhere.
* Dataflow: Removed pollability as a dimension of complexity.
* Dataflow: Moved *Mixing methods to instead be default methods on interfaces.
* FRC and roboRIO: Renamed Igneous to FRC.
* Documentation: Added a new Scribble-based documentation system.
* Deployment: Added DeploymentEngine system to manage deployment to robots, which is many times faster than WPILib's deployment tool.

Documentation changes:

* Added a new Scribble-based documentation system.
* Removed examples from last year's FIRSTFare presentation and added this year's examples.
* Continuous integration via Travis CI is now in use, which covers unit testing and checking of certain code maintenance targets, such as keeping license headers up-to-date. This should help future maintainers.
* Javadocumented just about everything important.

Dataflow changes:

* Removed InputPolls. Inputs only now.
* Added DerivedInputs, which replace some functionality of InputPolls.
* Pollable I/O is replaced by interfaces which take polling events and return updating I/O.
* Moved *Mixing methods to instead be default methods on interfaces.
* Added UpdatingInput as a superinterface of the *Input classes.
* send is no longer a fundamental method, but an implementation on top of the new onUpdate.
* Removed `unsend` and replaced it with unsubscription EventOutputs returned by the `send` family of methods.
* Renamed Statuses to Cells and extracted IOs as superinterfaces that merge Inputs and Outputs.
* Converted PIDControl into PIDController and inverted control to make more sense.
* Support for taking derivatives of float channels.
* Added miscellaneous new remixing methods and constant fields.
* Renamed EventStatus.clearListeners to __UNSAFE_clearListeners.

Error handling changes:

* Let setupRobot() and autonomousMain() throw Throwables.
* Added safeSet/safeEvent for top-level handling of errors.
* Removed EventOutputRecoverable and restructured error handling to ensure consistent propagation.

FRC and roboRIO changes:

* Renamed Igneous to FRC.
* Shorted names of everything in FRC.
* Removed the WPILibJ-wrapping style of FRC/Igneous implementation. Direct is the only implementation now.
* Added support for FPGA Counters on the roboRIO.
* Removed WPILib Eclipse plugins as a dependency.
* Added sensorPeriodic as the default polling interval in FRC.
* Renamed IgneousLauncher to FRCImplementation.
* Removed the driver station screen I/O methods.
* Added simpleControl of CAN to FRC and ExtendedMotor.

Deployment and build changes:

* Added DeploymentEngine system to manage deployment to robots, which is many times faster than WPILib's deployment tool.
* Removed dependency on Git in build process; versions are now specified in a source file.

Control interface changes:

* All Joysticks have POV hats now.
* Converted POV hat accesses to angle checks rather than angle and press accessors.
* Added AbstractJoystick to assist conversion of pollable interfaces into updating interfaces.
* Added default ControlBinding.
* Renamed IJoystick to Joystick.

Structural changes:

* Removed support for cRIO.
* Started using Java 8 features everywhere.
* Renamed all of the projects to have shorter and more accurate names.
* Changed expected project structure; added script to autoupgrade projects to v3 structure.
* Removed CCRE Collections.

Testing changes:

* Added FakeTime, for faking time in tests.
* Converted test infrastructure to JUnit.
* Unit tested huge sections of the CCRE that weren't before.

Miscellaneous changes:

* Added an abstraction for Time, for use in time-influenced testing.
* Renamed StorageProvider to Storage.
* Made StorageSegment its own implementation: now final instead of abstract.
* Slimmed down ExpirationTimer.
* Slimmed down StateMachine slightly.
* Added support abstractions for the dataflow event dispatch infrastructure.
* Added utility to rewrite CRLFs to LFs.
* Added miscellaneous support abstractions.
* Rearranged a few packages.
* Renamed just about everything to make meanings clearer.

Networking changes:

* Deabstracted the networking system and integrated traffic metrics.
* Added versioning to the Cluck protocol, which is backwards-compatible with v2.x.x Cluck, but only until the first protocol version increment.
* Added ability to subscribe to IOs.
* Deprecated Cluck UNSUB messages; they are replaced with NACKs.
* Cells are no longer RConfable.
* Removed default PoultryInspector link naming.
* Removed openDataInputStream and openDataOutputStream from ClientSocket.
* Removed minimum logging levels from network logging.
* Removed old Cluck searching methods.

UI changes:

* Resized Emulator components so that they require less screen space.
* Added window title to PoultryInspector.

Removals:

* Removed support for Phidget.
* Removed BooleanFilter.
* Removed StringHolder.
* Removed IgneousCore.
* Removed BootLogger.
* Removed ccre.launcher.Launcher dispatcher.
* Removed RLoad client and server.
* Removed Tokenizer.
* Removed the CCRE_Examples project.
* Removed MultipleSourceBooleanController.
* Removed Utils.currentTimeSeconds.
* Removed cRIO workarounds.
* Removed alternate storage implementations.
* Removed miscellaneous premature optimizations.
* Removed all sorts of deprecated and outdated methods.
* Removed deprecated EventOutput and updateWhen code in InstinctModule.
* Removed ApolloGemini2014 and InfernoDante examples.
* Removed vestigal kinect support, as it was only easily supported with the cRIO.
* Removed other vestigal and deprecated utility methods.

Implementation changes:

* Moved useCustomCompressor into FRC.

Bugfixes:

* Fixed incorrect disabling of digital outputs in emulator.
* Improved all sorts of bounds and argument checks throughout the CCRE.
* Fixed lots of edge cases.
* Fixed encoding incompatibilities and standardized on UTF-8.
* Fixed incorrect error propagation when broadcasting.
* Added missing top-level error handling.
* Fixed some miscellaneous race conditions.
* Fixed incorrect naming of mecanum drive code.

The length of that list should hopefully justify how long this release took to get out the door. Keep in mind that the thing with adding more unit tests took a lot longer than the single changelog line might imply.

Highlights:

• Being a major-number release, we were allowed to break backwards compatibility for the first time since v2.0.0! This was part of the inspiration for many of the changes made.
• We removed support for the cRIO. Getting Java 8 to work on the cRIO would have been more of a challenge than we wanted to take on.
• Thanks to that, the CCRE now takes advantage of a significant chunk of Java 8's featureset! Default interfaces are the biggest advantage, because they allow us to have both a) easily implemented channels AND b) remixing methods on the channels themselves.
• We realized that the InputPoll versus Input dichotomy was not helpful in real programs, and only made things more complicated. So, we got rid of it! Everything's an Input now. To replace the previous ease of defining new InputPolls, there are now DerivedInputs that are almost as easy (for defining InputPolls) and much much easier (for defining Inputs.)
• We renamed a very large number of things in this release, because the old names weren't very helpful. For example: the Igneous subsystem is now called the FRC subsystem.
• We added an entire new style of documentation. Check it out here! This uses Racket's Scribble system, which is good for guidebooks like this. This documentation is not yet complete, but should be a good place to start with the new version.
• We wrote the DeploymentEngine to simplify the deployment of code to FRC robots, and it just so happened that it also sped up everything by a significant margin!
• On a related note, we got rid of our dependency on WPILib's Eclipse plugins, so installation is smoother than ever.

Looking towards the future, we have some more changes planned for the final 3.0.0 release, which should be coming in a few weeks. We do have the opportunity to make more backwards-incompatible changes for that release, but we've made most of them by now, so don't worry too much about that.

We also plan to do things like Eclipse plugins so that you don't have to download the CCRE source to use it!

One quick note on upgrading to CCRE 3: we have a tool that will offer to upgrade your v2.x.x projects for you. Simply import your old project into your new CCRE 3 workspace, and press Build All. You should be asked if you want to upgrade your project to CCRE 3: make sure your old code is backed up, and then press Yes and type in your team number. This won't do everything for you, but will do the easy things like replacing InputPolls with Inputs, replacing Igneous with FRC, and handling most of the other major renamings. You will have to manually move your Mixing invocations to the new system, however.

[Cel Skeggs]

Since we're coming up on build season, here's a clarification about using this under the rules:

Since the CCRE is publicly available, you can use it for FRC robot software legally under the rules (at least as of 2015), if you don't make any changes before Kickoff. If you do make any changes before Kickoff, you must make your changes available publicly, which you could do for example by pushing it to a public repository and either posting the link somewhere or opening a pull request to the main CCRE repository.

We will make sure that 100% of our work is available in our repository at the point of Kickoff. Most of it will be in the latest release - some of it will probably be in the development branch devel-3.x.x - and the tiny remainder may be scattered around feature branches, which you will be able to find on GitHub under the branch listing.

During the build season, we will continue to release all of our changes as normal - we don't use any private versions internally, except, of course, for the time during which each feature is being developed.

If you have any questions or concerns - just ask!

[Cel Skeggs]

More work has been done on the documentation! It's currently up to about 13,000 words across a variety of topics. It'll probably reach around 25,000 words by the time it's done, so don't expect it to be quite complete. It is being updated almost daily with incremental additions.

You can view the new-style documentation here.

If you have any questions, comments, or suggestions about the documentation - including typos - just post them here!

[Cel Skeggs]

Sorry about the wait on 3.0.0. I'm almost done with my college apps, and CCRE v3.0.0 should be out at least a few days before Kickoff.

You can track the progress of the release on this PR.

[Cel Skeggs]

At long last, CCRE v3.0.0 is OUT!

This is a MAJOR RELEASE. It contains all the same changes as v3.0.0-pre1 plus a few extras:

Major changes:
* Rename DriverImpls to Drive and drop "Drive" from method names
* Expand new-style documentation significantly
* Extract setup-mode EventOutputs as CancelOutputs

Minor changes:
* Add single-parameter motor creators to FRC
* Add FloatIO accumulate & friends
* Rename static BooleanOutput.onChange to BooleanOutput.polarize
* Made FloatOutput.filter work consistently with BooleanOutput.filter
* Add waitUntilNot with timeout for symmetry with waitUntil
* Change default value of subscribed FloatInputs to start at NaN
* Collapse various duplicate transformation methods to use each other
* #90: Remove HALControlWord
* Made InstinctMultiModule final
* Clean up miscellaneous code

Bugfixes:
* Fix handling of infinities by motors
* Fix edge case of onChangeBy
* Fix timing issue in PauseTimer unit test
* Fix float inspecificity in testing utilities
* Fix lack of error information during setRoots in Deployment Engine
* Fix occasional error silencing bug in testing utilities
* Fix miscellaneous documentation typos
* Fix CI integration issues

As this is a major release, we now have a backwards compatibility base established: we cannot break backwards compatibility until 4.0.0 - and since 4.0.0 is probably at least a year off, you should be set to trust that the current version is stable and will continue to work.

There are a couple of minor bugs remaining that we're going to fix ASAP and get out as 3.0.1 shortly, but probably nothing major.

And remember these links: the new-style documentation and the Javadoc.

[Cel Skeggs]

3.1.0 will be out sometime next week. 3.0.1 is being skipped because we need to get a release out with the fixes to support the 2016 roboRIO image as soon as possible, and that will require you to upgrade to the 2016 image, so it doesn't qualify as simply a patch release.

Also, this can't happen until after the 2016 NI Update Suite is unlocked and I do enough work to finish adding support, which is why I can't do it now.

Obviously, it won't break any code or API compatibility, just require a newer roboRIO image on the bot.

Also, our CCRE repository now has all of our WIP code as well as all of our releases - simply browse the branches to see all of them, or just wait until they get included in a release.

[Cel Skeggs]

CCRE v3.1.0 is out!

The CCRE now supports the FRC_roboRIO_2016_v19 roboRIO image! It also no longer supports the FRC_roboRIO_2015_v23 roboRIO image, which is why it's a minor release rather than a patch release.

A few other bugs have also been fixed in this release, but no user-visible features.

Release of CCRE with API 3.1.0.

Major changes:
* Add support for 2016 roboRIO image

Minor changes:
* Support new mDNS hostname
* Remove extraneous StandardStreamLogger
* Add support for testing logging messages
* Add testbed sample for Valkyrie

Bugfixes:
* Fix serialization of PoultryInspector components
* Fix infinite loop bug with recursive error reporting in Cluck
* Fix CCRE version number support

The new support might have some bugs - there were a lot of changed JNI bindings! If anything seems to not work, reporting it immediately will get it fixed as fast as possible.

[Cel Skeggs]

CCRE v3.2.0 is out!

This release has two large features (CCRE Eggs and Behavior Arbitrators) and a wide variety of minor features. Also bugfixes.

Release of CCRE with API 3.2.0.

Major changes:
* Implement CCRE Eggs, which package up a code snapshot for later deployment
* Implement Behavior Arbitrators, which manage active subsystem controllers

Minor changes:
* Add static Output.combine methods
* Add negateIf, negatedIf
* Add conditional method for booleans: BooleanInput.select
* Add ExpirationTimer.getRunning method as BooleanIO
* Add ExpirationTimer.runWhen method
* Add onPress, onRelease, onChange with integrated send
* Add absolute value filter
* Add cell() shortcut to Outputs
* Add event and toggle button control bindings
* Add sample of Behavior Arbitrator use
* Provide BooleanIO-based control of InstinctModules
* Remove year-specific message from library version

Bugfixes:
* Fix certain test failures to not appear as errors
* Fix broken FRC inTeleopMode() method
* Fix loading storage segments with escaped names
* Fix version shortening for development versions
* Fix PoultryInspector RConf left click glitch
* Fix POV always-UP glitch that broke ControlBinding
* Fix javac incompatibility on some systems

CCRE Eggs are a way to build (lay) a snapshot of code into a single executable JAR which has the ability to deploy (hatch) that snapshot of code to a robot, even without a development environment installed. This has applications in both saving old code and letting non-programmers deploy new or old versions of code to a robot. CCRE Eggs are still in their first iteration, but they'll get better over the next few releases. For example, we'll add a deployment-time GUI to let you decide where you want to deploy an egg.

Behavior Arbitrators are the other major feature: you can specify a prioritized set of mutually exclusive "Behaviors", and the arbitrator selects the highest-priority behavior that currently wants to run, and allows it to control the actuators associated with that arbitrator.

For example, you could have a "pit safety behavior", "autonomous behavior", "mecanum behavior", "tank drive behavior" and "test mode behavior", and the arbitrator would automatically select the best behavior to apply in the situation, based on the priorities and conditions that you've specified.

Behavior Arbitrators are also in their first released iteration, but they're already practically useful. Behavior Arbitrators are joining Control Bindings, Cluck, Instinct Modules, Control Structures, and RConf as the primary components of the CCRE toolkit.

[Cel Skeggs]

CCRE v3.3.0 is out!

This release is from over a month of active software writing, and adds a very large number of minor features and quite a few major features as well.

Release of CCRE with API 3.3.0.

Major changes:
* Add recording toolkit and Timeline Analyzer
* Add discrete channels
* Rearchitect and expand serial bus handling system
* Add new fleshed-out CAN Talon SRX interface: TalonExtendedMotor
* Add ethernet camera support to PoultryInspector
* Overhaul timing system to use a unified scheduler
* Add StopwatchTimer
* Add support for SD540 speed controller
* Add support for Spark speed controller
* Add support for Victor SP speed controller
* Add support for Talon SRX over PWM

Minor changes:
* Extend Time.* constants to longs to avoid overflow
* Add rumble support to Joysticks
* Support axis inversion in ControlBindings
* Add "OPEN NETWORK" button to PoultryInspector
* Allow PoultryInspector component removal by dropping components back into network palettes
* Add tunable ramping methods
* Add main class selector to SampleRobot
* Add recorder example to Valkyrie code
* Add selectByState to StateMachine
* Add mode tracking via discrete inputs to FRC API
* Add discrete input support to BehaviorArbitrator
* Add Storage support for deletion, existence checking, and file listing
* Deprecate old derivative implementation; add version without the zero-delta bug
* Add modulation (remainder) operation to FloatOperation
* PauseTimers now can take a FloatInput for the delay
* Add support for reading total current of the PDP
* Add Faultable<F> interface
* Add Derived...IO classes to match Derived...Input classes
* Allow Cluck publishing of any IOs, not just Cells
* Improved documentation in various ways

Invisible changes:
* Speed up unit tests
* Overhaul time testing system
* Switch travis CI from trusty-based to container-based
* Add support for reversed InputStreams to Cluck
* Add support for multiple SuperCanvas applications
* Update libwpilibJavaJNI.so to newer, slimmer, custom version
* ExtendedMotors are now enabled as part of simpleControl()
* Miscellaneous code clean-up and refactoring
* Apply author name change

Bugfixes:
* Fix long-standing serialization issue in PoultryInspector
* Fix FRC.counter to not fail most of the time
* Miscellaneous fixes

The recording system is one of the most interesting new features:




This system allows recording precise information about the function of a robot, down to an accuracy of 10 microseconds. In the above diagram, you can see a snippet from an actual robot test run that shows precise information on the timing details of certain periodic events and the exact state of the robot's scheduler. Tools like this allow recording an entire match and looking back later to discover exactly what happened. To use this, look at the ccre.recording package.

We've also introduced a new scheduling system, which collapses all of the different timing threads into a single thread, which increases consistency of timing, simplifies timing code (so fewer bugs), and makes finer-grained CPU load analysis and management possible. You don't need to do anything to work with this - it's automatically used by the latest version of the CCRE.

We added support for SPI and I2C, and expanded support for RS232. We expanded the CAN Talon SRX interface to allow access to almost all of its internals through a CCRE-style reactive interface. We also added support for other PWM speed controllers like the SD540, Spark, Victor SP, and Talon SRX (when used over PWM.)

Discrete channels are a new addition to the core architecture of the CCRE: these are like boolean channels, but with a used-defined set of options, so that it can be used for robot mode access, behavior arbitrators, and more.

The PoultryInspector interface is now slightly more intuitive, and it now supports viewing AXIS M1011/M1013 webcams, including drawing targeting rectangles on top.

This releases includes fixes for all on-field bugs that we know about, and we'll be certain to release any further fixes promptly during the competition season. If you find anything that doesn't work, please contact me immediately so that we can get it fixed quickly.

[Cel Skeggs]

CCRE v3.4.0 is out!

This release is from our competition season, and while it doesn't involve very much in the way of new features, it has a boatload of minor improvements and bugfixes.

Release of CCRE with API 3.4.0.

Major changes:
* Add optional Phase Verifier system

Minor changes:
* Add safe ExtendedMotor control methods
* Add unique identifiers to roboRIOs
* Support programmatic autonomous mode aborting
* Export IO subscribes through Cluck API
* Add 16-bit integer decoders to ByteFiddling
* Scale up PoultryInspector webcam views
* Raise PoultryInspector Webcam highlight components on top of everything else
* Allow DeploymentEngine users to access Emulator API
* Add view toggle button to PoultryInspector channel components
* Change default views on float components to textual
* Add startIfNotAlive to ReporterThread
* Add reset method to PIDController
* Remove CCRE 2->3 updater
* Let RS232IO extend Flushable
* Use events for flow phase cancels in Scheduler
* Improve Deployment Engine API slightly
* Add launcher for CCRE core tests
* Improve sample robot code
* Improve Javadoc slightly

Invisible changes:
* Increase priority of InstinctModule threads
* Streamline roboRIO detection in DeploymentEngine
* Improve TimelineInspector project setup
* Improve use of @Override
* Add phase annotations to a significant portion of the CCRE
* Simplify miscellaneous pieces of code
* Cache byte arrays to avoid GC churn in WebcamReader

Bugfixes:
* Ensure that the user intends to overwrite files in PoultryInspector save dialog
* Fix IO subscriber suffix confusion bug
* Fix Emulator reference issues by offloading classloading from itself
* Fix PauseTimer BooleanInput contract violation
* Fix camera over-refresh bug
* Provide consistent velocities in Emulator
* Remove extraneous logging from Emulator joysticks
* Hide NoRouteToHost log spam
* Increase length of Cluck TX queue
* Ensure that voltage logging includes the entire range
* Fix recorder printing name bug
* Fix miscellaneous Eclipse issue

Are you interested in helping develop the CCRE? Since I'll be graduating from high school, I won't have as much time to dedicate to the CCRE next year, and while we have plenty of talent ourselves to keep it maintained, we're interested in getting contributions from other teams to help us keep it going as long as possible.

If you want to help - or might want to help but aren't sure - drop me a line either here or in the FIRSTCodes Slack!