Custom C++ Toolchain

Wanted to use C++ but chose Java instead because Eclipse is cross-platform?

Using C++ and wishing you could use a newer version of gcc, with performance and optimization improvements? Wanted a native gcc build on linux?

Wishing you could use C++11 features in your robot code? Lambdas, rvalue references, standard library improvements, the whole 9 yards?

After a lot of work by several people, I can say that there are working build scripts for a complete C++ toolchain using GCC 4.7.0. Thanks for the build script go to the following people:

  • Austin Schuh
  • Ross Light
  • Cody Schafer

GCC Build Scripts can be found here:
https://github.com/rbmj/vxworks-gcc-patches

Static-Linked GCC Binaries are in the “Downloads” Section. The binaries I built should run on any modern amd64 linux machine, but I’m not sure, so please test!

WPILib patched for new GCC is here:

The binaries in the downloads section should work on any linux machine.

I don’t have any windows binaries as I haven’t ported the build script to windows, but if anyone wants to help with that, that would be REALLY COOL :slight_smile:

Finally, I have a toolchain file for CMake. This toolchain file is here:
https://github.com/rbmj/cmake_vxworks

You will need to use this toolchain file if you want to compile WPILib. TODO: Give WPILib a find_package/.config file!

An example CMakeLists.txt file for a simple robot project:


cmake_minimum_required(VERSION 2.8)

#project specific EXAMPLE
project(FRC)
set(FRC_SOURCES main.cpp) #just add to this list here
set(FRC_INCLUDES ${WPILIB_INCLUDE_DIR}) #defined by the toolchain
set(FRC_LIBS ${WPILIB_LIBRARY}) #ditto
# below here should stay pretty much the same
add_executable(FRC_UserProgram ${FRC_SOURCES})
set_target_properties(FRC_UserProgram PROPERTIES SUFFIX .out)
target_link_libraries(FRC_UserProgram ${FRC_LIBS})
include_directories(${FRC_INCLUDES})

compile with:


$ cmake /path/to/repo/robot_test -DCMAKE_TOOLCHAIN_FILE=/path/to/toolchain/vxworks_toolchain.cmake 
$ make
$ wput -u FRC_UserProgram.out ftp://10.xx.yy.12/ni-rt/system/ #deploy

CMake is awesome - use it! This toolchain is NOT COMPATIBLE with the ucpp makefiles, so if you want a working executable you MUST use CMake!

TODO:

  • Do more extensive tests, including C++11 features
  • Make the build scripts nicer and less crufty
  • Get a working windows port

I agree!
CMake is awesome… it is used in Open Scene Graph… it’s a great way for the Visual Studio developers to interact with the GCC developers.

The CMake Toolchain file should work with the standard “gccdist” toolchain. If you download this toolchain <1> then the cmake file should work fine (NOTE: It requires munch.sh and strip_syms.sh from the vxworks-gcc-patches repo to be installed in WIND_BASE - maybe should make these a separate package), just set TOOLCHAIN_IS_GCCDIST to true.

The cmake toolchain file would work on windows if it weren’t for the fact that I haven’t bothered to create windows equivalents of munch.sh and strip_syms.sh. I don’t know batch (or maybe it would have to be vbscript…) very well, so I wouldn’t know how to substitute the cut and sed commands.

If I can get the cmake scripts working on windows that opens up the possibility of teams using those dreamspark cards and developing the robot in visual studio, or codeblocks for FOSS purists like myself :smiley:

I don’t know if the generators for VS or C::B support manual target rules though (CMAKE_C_COMPILE_OBJECT et al).

Side note:

I’m thinking in the future this can be four packages:
vxworks-binutils
vxworks-gcc (depends vxworks-binutils)
vxworks-scripts (depends vxworks-binutils)
vxworks-cmake-toolchain (depends vxworks-scripts, vxworks-gcc, vxworks-binutils)

Maybe have a PPA…

So question: Should I keep statically linking the binaries so they work on most linuxes, or should I use “proper” packaging?

Also, if anyone gets a chance to test this with a machine other than my own, that would be cool :slight_smile: maybe get it to work on win32? MSYS seems like the best option…

  1. ftp://ftp.ni.com/pub/devzone/tut/updated_vxworks63gccdist.zip"

Great job! How hard would it be to get gfortran working? Fortran is the primary dependency I’ve been missing to get numpy ported to the cRIO.

AFAIK the only thing you would need to do is change the ./configure line to gcc. If you change the argument in the build script that says “–enable-languages=c,c++” to “–enable-languages=c,c++,fortran” you should get gfortran (I think).

If you’re on windows, you might have to run the script through an MSYS shell. Right now the build script needs a basic set of bash commands, wget, sed, and realpath (IIRC).

OK. I’m looking for some advice:

Currently I have some statically linked binaries up. I’m worried, though, because those also include the system headers, which I don’t have the right to redistribute. Should I take those binaries down and require everyone to use the build script?

I couldn’t find a contact at WindRiver for FRC, so I guess I’ll have to email their generic inquiry account. Maybe this can be easily resolved.

They also do some weird stuff with their include directories, that I’d ideally like to move around to simplify installation of GCC, as right now there’s some hackish stuff I have to do to get GCC to build right :frowning:


Also, I forgot to post instructions for the build script.

You need to call it like this:

/path/to/build/script/build.bash PREFIX DLDIR SRCDIR BUILDDIR

Where:
PREFIX is where to install everything to (e.g. /usr/local)
DLDIR is where to put the downloaded tarballs
SRCDIR is where to extract the sources
BUILDDIR is where to put the build trees.

Note that these should be absolute paths. If you’re in a clean directory, this should be sufficient:

…/build.bash /usr/local pwd/download pwd/src pwd/build

It also needs to be run as root. If you have build problems let me know.

Unfortunately, it isn’t quite this easy. I tried it and got a number of errors in libgfortran that look like they’ll take some time to work through (partly missing things in system headers).

If you try and build libstdc++ without patches you get the same issues. I’m planning on sending a submission to gcc-patches, but in the interim, you can see the kind of simple fixes we had to make.

Check wrn/coreip/ for the include as well. Since GCC doesn’t support setting target-specific include directories without some really bad hacks, I had to put in some temporary stubs to redirect the includes. I’m thinking of restructuring the system headers so one no longer needs this cruft, but I’m pushing the boundaries of fair use as is :frowning:

The other thing that is prone to break is stuff that has to do with the filesystem. Look at the patches for examples of how stuff had to be changed.

Luckily, there were only a few files that screwed up for libstdc++. Hopefully libgfortran is similarly simple. And if you need any help you can post!

OK, finally got around to contacting Wind River. Confirmation Number 319, whatever that means.

Hello,

I am a student on FIRST (For the Inspiration and Recognition of Science and Technology) Robotics Team 612. I am working on developing an alternative toolchain targeting vxWorks in order to allow teams to use a newer version of GCC, primarily in order to have a native linux build for FRC and to get new features (including C++11 support) not present in GCC 3.4.4 that is currently provided to FRC Teams.

Currently, in order to build the necessary runtime libraries, I am using the system headers from [1], which National Instruments distributes but contains (as far as I can tell) mostly Wind River Intellectual Property.

In order for the headers to be accepted, it is necessary to incorporate a few patches. These patches are relatively minor and are only intended to allow the headers to be accepted by the newer version of GCC and increase interoperability and standards compliance. However, the headers contain the following notice:

/*

  • Copyright (c) 1984-2005 Wind River Systems, Inc.
  • The right to copy, distribute or otherwise make use of this software
  • may be licensed only pursuant to the terms of an applicable Wind River
  • license agreement. No license to Wind River intellectual property rights
  • is granted herein. All rights not licensed by Wind River are reserved
  • by Wind River.
    */

Which very clearly states that posession of the file does not confer any Intellectual Property rights to users. Thus, modification and redistribution of such code is illegal. I made my modifications under the Wind River FRC License Agreement (namely, “THIS OBJECT CODE LICENSE AND DISTRIBUTION AGREEMENT”), section 4, and therefore freely assign any copyright to any modifications I made to the system headers to Wind River.

Additionally, in order to share my progress with the community, I wish to redistribute the toolchain I have developed. As a necessary part of its functioning, such a toolchain requires the system headers for the target Operating System (in this case vxWorks). This does not contain any implementation code but is rather only the declarations to allow code to operate inside of vxWorks. In order to share my work with the FIRST community then, it is necessary for me to distribute the vxWorks system headers.

Thus, I request that Wind River release the system headers (and the headers alone - I do not request a license to any implementation code) for vxWorks under an open source license (such as FIRST’s BSD-Style license) in order to facilitate the exchange of knowledge and tools related to vxWorks within the FIRST community. Should Wind River be unwilling to do so, I request that Wind River license allow modification and redistribution of the system headers to vxWorks for non-commercial purposes, such as the FIRST Robotics Competition and related activities by team members and affiliated adults. Again, this would not confer any rights on the actual proprietary implementation of vxWorks, but only the declarations needed to interface with such code.

If you wish to see the patches I have made (they are minimal), I am more than willing to supply them and an explanation as to why they are necessary; they are, after all, your intellectual property under the current license agreement!

Sincerely,

Robert ‘Blair’ Mason, Jr.
Member and Former ‘CEO’
Chantilly Robotics
FIRST Robotics Competition Team 612

[1] ftp://ftp.ni.com/pub/devzone/tut/updated_vxworks63gccdist.zip

OK, so I have a fortran compiler working. The issue: I don’t have a build script yet. I’m not going to make one, as I’m working on transitioning from a build-script system to a package-based system. I think this will be easier for 90% of people, and the 10% who need to compile it themselves can use the command line themselves :-D.

I have a series of patches, which I’ll attach, but they’re based off of the trunk. I’m working with the gcc devs to get them committed, so hopefully by gcc-4.8 vxworks will compile out of the box.

There’s also a series of patches which need to be applied to the C headers. Finally, there’s a patch that needs to be applied while GCC builds and then reverted after the build.

So, some build instructions.

First, install the cross binutils from earlier- those will still work for building gcc.
Then download all of the attachments to this post. Remove the .txt extension from the patch file:


$ mv headers-build.patch{.txt,}

Untar the patches:


$ tar -xf gcc-patches.tgz
$ tar -xf wrs-patches.tgz

Get the latest gcc trunk:


$ svn checkout svn://gcc.gnu.org/svn/gcc/trunk gcc

Get the gccdist and unzip it:


$ wget ftp://ftp.ni.com/pub/devzone/tut/updated_vxworks63gccdist.zip
$ unzip updated_vxworks63gccdist.zip

Apply the permanent patches to gccdist:


$ cat wrs-patches/*.patch | patch -p1 -l -d gccdist/WindRiver/vxworks-6.3/target/h

And apply the temporary gcc build patch:


$ patch -p1 -l -d gccdist/WindRiver/vxworks-6.3/target/h < headers-build.patch

Now switch to the gcc directory.
Since I made changes to configure.ac you’ll need to regenerate that:


$ autoconf

If it barfs at you about not having the right autoconf version, go into config/override.m4 and change the lines that look like this:


dnl Ensure exactly this Autoconf version is used
m4_ifndef([_GCC_AUTOCONF_VERSION],
  [m4_define([_GCC_AUTOCONF_VERSION], [2.64])])

to look like this (note that my autoconf verison is 2.68):


dnl Ensure exactly this Autoconf version is used
m4_ifndef([_GCC_AUTOCONF_VERSION],
  [m4_define([_GCC_AUTOCONF_VERSION], [2.68])])

Then re-run autoconf.

OK, if you’re still with me, now it’s time to run configure. Switch out of the gcc directory (cd …) and make a new directory called build (mkdir build), then switch into the build directory. Here’s how to invoke configure:


# ../gcc/configure \
	    --prefix=/usr \
	    --target=powerpc-wrs-vxworks \
	    --with-gnu-as \
	    --with-gnu-ld \
	    --with-headers="../gccdist/WindRiver/vxworks-6.3/target/h" \
	    --disable-shared \
	    --disable-libssp \
	    --disable-multilib \
	    --with-float=hard \
	    --enable-languages=c,c++,fortran \
            --enable-libstdc++-v3 \
	    --enable-threads=vxworks \
	    --without-gconv \
	    --disable-libgomp \
	    --disable-nls \
	    --disable-libmudflap \
	    --with-cpu-PPC603 \
	    --disable-symvers

You’ll need to have the development libraries for libmpfr, libmpc, and libgmp on your system (libmpfr-dev, libmpc-dev, and libgmp-dev on debian) installed for this to work. Also note the #; configure needs to be run as root here.

Now for the fun part. Hopefully, if you’ve done everything up to this point, you can just say


$ make -j4
/code
and everything will work just fine.  Then just make install as root.

Finally, clean up some of the unnecessary fixes:

patch -p1 -l -R -d /usr/powerpc-wrs-vxworks/sys-include < headers-build.patch



And you should hopefully have a working compiler!

This will become much simpler once I update the build script, but in case any of you want to try this out and understand how to do it, I posted this.

If you have any questions, let me know.

Note that fortran support is untested as I have nothing to test it with.  It would be nice if somebody could test this.

Libgfortran's configure is acting funny too - gotta fix that.<br><br><a class='attachment' href='/uploads/default/original/3X/3/d/3dc92eb316359dae086fd4b330fd32e13d0915db.tgz'>gcc-patches.tgz</a> (10.1 KB)<br><a class='attachment' href='/uploads/default/original/3X/1/1/111aada4d555daab6971cbdca296e038589d1e97.tgz'>wrs-patches.tgz</a> (2.42 KB)<br><a class='attachment' href='/uploads/default/original/3X/9/b/9b52b97d82ae17a1803620f864ef568fb9e6ec64.txt'>headers-build.patch.txt</a> (783 Bytes)<br><br><br><a class='attachment' href='/uploads/default/original/3X/3/d/3dc92eb316359dae086fd4b330fd32e13d0915db.tgz'>gcc-patches.tgz</a> (10.1 KB)<br><a class='attachment' href='/uploads/default/original/3X/1/1/111aada4d555daab6971cbdca296e038589d1e97.tgz'>wrs-patches.tgz</a> (2.42 KB)<br><a class='attachment' href='/uploads/default/original/3X/9/b/9b52b97d82ae17a1803620f864ef568fb9e6ec64.txt'>headers-build.patch.txt</a> (783 Bytes)<br>

I have installed the compiler binaries and wpilib on Xubuntu 12.04 through your .deb files. I then dumped the toolchain file and you sample CMakeLists.txt file into a directory. However, running CMake fails for me.

This appears to be the important part:

powerpc-wrs-vxworks-c++: fatal error: environment variable 'WIND_BASE' not
  defined

What is suppossed to be setting WIND_BASE? (If this isn’t enough, then I can go ahead and post everything I have done related to this project).

Thank you, I’m really excited to get this working.

WIND_BASE doesn’t matter anymore. It just needs to be defined to something. I just say export WIND_BASE=‘asdf’. If you want to be ‘proper’ about it, you can set WIND_BASE to /usr/local/powerpc-wrs-vxworks/wind_base

Okay, then, I guess I got confused because when I tried setting it myself, I got a different error, which I interpreted to be because I had an incorrect setting for that variable :stuck_out_tongue:

I have it working now (I think… I have yet to try it on the robot). However, I did have to make some symbolic links to make it work. In particular, I had to make the following links:

lrwxrwxrwx 1 root root 15 May 26 12:49 /usr/local/powerpc-wrs-vxworks/host -> wind_base/host/
lrwxrwxrwx 1 root root 11 May 26 12:48 /usr/local/powerpc-wrs-vxworks/lib/libWPILib.a -> libwpilib.a
lrwxrwxrwx 1 root root 18 May 26 12:43 /usr/local/powerpc-wrs-vxworks/munch.sh -> wind_base/munch.sh
lrwxrwxrwx 1 root root 23 May 26 12:46 /usr/local/powerpc-wrs-vxworks/strip_syms.sh -> wind_base/strip_syms.sh

Thank you for your help.

This is awesome, and I hope I’ll be programming on Linux come next season.

I’m trying to set up this toolchain on a 64-bit install of Archlinux, and I’m getting some errors after

  • Running the build script for downloading and patching gcc for vxworks
  • Editing the toolchain(changing gccdist to true, setting deploy ip)
  • Creating a CMakeLists.txt file based off your example
  • Running cmake

Maybe this is something of note: while running


cmake /home/azrathud/vxworks-c++-toolchain/project -DCMAKE_TOOLCHAIN_FILE=/home/
azrathud/vxworks-c++-toolchain/cmake_vxworks/vxworks_toolchain.cmake

Cmake outputs that CMAKE_TOOLCHAIN_FILE is not used

I’m trying to compile last season’s code which is in the src directory specified in the CMakeLists.txt

From the errors, I think gcc cannot find the header files in /usr/local/powerpc-wrs-vxworks/sys-include


In file included from /home/azrathud/vxworks-c++-toolchain/project/wpilib/ErrorBase.h:12:0,
                 from /home/azrathud/vxworks-c++-toolchain/project/wpilib/Commands/Command.h:10,
                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.h:4,
                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.cpp:1:
/home/azrathud/vxworks-c++-toolchain/project/wpilib/Error.h:13:21: fatal error: vxWorks.h: No such file or directory
compilation terminated.
make[2]: *** [CMakeFiles/FRC_UserProgram.dir/src/CommandBase.cpp.o] Error 1
make[1]: *** [CMakeFiles/FRC_UserProgram.dir/all] Error 2
make: *** [all] Error 2


I included what I pulled from your WPIlib github repo in the CMakeLists.txt locally, but I looks like from your cmake toolchain file, that I should be placing them in the place under /local … include/WPILib, yes? Also, I added the sys-include to the list of includes, but it didn’t look like it did anything to help.


cmake_minimum_required(VERSION 2.8)

project(FRC)

aux_source_directory(./src FRC_SOURCES) # I'll just run cmake all the time to
        # make sure new files are added

set(FRC_INCLUDES ${WPILIB_INCLUDE_DIR} wpilib ${TOOLCHAIN_PREFIX}/sys-include ) #defined by toolchain
set(FRC_LIBS ${WPILIB_LIBRARY}) # defined by toolchain

add_executable(FRC_UserProgram ${FRC_SOURCES})
set_target_properties(FRC_UserProgram PROPERTIES SUFFIX .out)
target_link_libraries(FRC_UserProgram ${FRC_LIBS})
include_directories(${FRC_INCLUDES})

I think the problem may relate to that I haven’t compiled the wpilib into a shared library, like the toolchain suggests. (I have no ${TOOLCHAIN_PREFIX}/lib/libWPILib.a) How would I go about correctly including the patched WPILib and allow that to be linked correctly, etc?

The internet seems to think that this is a harmless bug output by some versions of cmake that can be fixed by setting CMAKE_TOOLCHAIN_FILE in the file itself.

I’m trying to compile last season’s code which is in the src directory specified in the CMakeLists.txt

From the errors, I think gcc cannot find the header files in /usr/local/powerpc-wrs-vxworks/sys-include


In file included from /home/azrathud/vxworks-c++-toolchain/project/wpilib/ErrorBase.h:12:0,
                 from /home/azrathud/vxworks-c++-toolchain/project/wpilib/Commands/Command.h:10,
                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.h:4,
                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.cpp:1:
/home/azrathud/vxworks-c++-toolchain/project/wpilib/Error.h:13:21: fatal error: vxWorks.h: No such file or directory
compilation terminated.
make[2]: *** [CMakeFiles/FRC_UserProgram.dir/src/CommandBase.cpp.o] Error 1
make[1]: *** [CMakeFiles/FRC_UserProgram.dir/all] Error 2
make: *** [all] Error 2


Yes, that appears to be the case. Depending on how you built gcc, it may or may not have all of the include directories specified. It could also be that CMake is using the wrong gcc, so ask it to be verbose and check that. Finally, find that gcc’s cc1 (probably in /usr/powerpc-wrs-vxworks/bin if memory serves) and run cc1 -v and try and figure out what include directories it has. Check the directories listed.

The final option is that that code worked on windows because windows is a non-case-sensitive filesystem. Check your capitalization (vxWorks.h maybe?)

I included what I pulled from your WPIlib github repo in the CMakeLists.txt locally, but I looks like from your cmake toolchain file, that I should be placing them in the place under /local … include/WPILib, yes? Also, I added the sys-include to the list of includes, but it didn’t look like it did anything to help.


cmake_minimum_required(VERSION 2.8)

project(FRC)

aux_source_directory(./src FRC_SOURCES) # I'll just run cmake all the time to
        # make sure new files are added

set(FRC_INCLUDES ${WPILIB_INCLUDE_DIR} wpilib ${TOOLCHAIN_PREFIX}/sys-include ) #defined by toolchain
set(FRC_LIBS ${WPILIB_LIBRARY}) # defined by toolchain

add_executable(FRC_UserProgram ${FRC_SOURCES})
set_target_properties(FRC_UserProgram PROPERTIES SUFFIX .out)
target_link_libraries(FRC_UserProgram ${FRC_LIBS})
include_directories(${FRC_INCLUDES})


I think the problem may relate to that I haven't compiled the wpilib into a shared library, like the toolchain suggests. (I have no ${TOOLCHAIN_PREFIX}/lib/libWPILib.a) How would I go about correctly including the patched WPILib and allow that to be linked correctly, etc?

If you run cmake on the wpilib directory with a working toolchain file, you’ll get a STATIC library. One can then link to this library with the toolchain WPILIB_LIBRARY - which hopefully at some point in the future will become a .config file.

OK, so a few slight changes to the build process.

headers-build.patch is no longer necessary. Instead, best solution (read: least kludgey) is to manually copy gccdist/WindRiver/vxworks-6.3/target/h to /usr/powerpc-wrs-vxworks/sys-include, and then symlink /usr/powerpc-wrs-vxworks/include to sys-include/wrn/coreip. Yes, this is less kludgey.

Actually all of the patches that are there have been deprecated. The latest GCC incorporates some, and there are others that fix things “better”. These haven’t been incorporated into mainline yet, but the main issue is that I’m going off to school and won’t have internet until late august (no working around this either). So I’m hoping that somebody can make sure that the work is not lost and that these make it in. 1 is approved, but not yet committed, and handles all of the patches to the WRS headers inside of GCC through the fixincludes process. It is a patch series, of which every patch depends on the first in the series. 2 is also approved, but not yet committed, and allows one to force a build of libstdc+±v3 with --enable-libstdc+±v3 passed to configure. A fix for the first gcov issue (obvious) is at 3. A fix for the second gcov issue (approved, not committed) is at 4. With those things done correctly, a complete build works.

The other issue is that some of the targets appear to be mis-configuring themselves, as they are allowing implicit declaration of functions in the configure process, which results in bad detection of the target platform’s available functions. Hopefully somebody can fix this :-D.

I’m looking for someone to take this over, at least until the regular school year starts, at which point I should be able to maintain everything again (at least for a little while).

I’m going to try and get one of the GCC maintainers to adopt this, but if the conversation doesn’t say committed, then it means I wasn’t annoying enough.

NOTE: if you look at a patch message and it says a certain file needs to be regenerated, then you’ll need to do that otherwise things won’t work. The configure files can be regened with autoconf2.64, and the fixincl.x file can be regened by going into the fixincludes directory of gcc and running ./genfixes.

It would be nice if you put the realpath dependency more prominently in the first post. It doesn’t come by default with Ubuntu (the most popular linux distribution, and common among newbies), and the error message when you don’t have it is pretty opaque (It’ll complain about it a few times, but start downloading anyway, only to fatal later.)

I get the following error when running the make command.

couldn't read file "/usr/local/powerpc-wrs-vxworks/host/resource/hutils/tcl/munch.tcl": no such file or directory

(terminal.txt)
The frc_userprogram still appears to build. Is it alright to just ignore this error?

Edit: So I decided to try and deploy the frc_userprogram.out to the robot. I used the wput, reset the robot, and then on the driver station it said no robot code. I checked to make sure the program got on via the web-ftp interface. In addition to that, I also flashed the v43 image… Included is the output from the cmake and make commands.

terminal.txt (1.29 KB)
cmake and make outputs.txt (3.71 KB)


terminal.txt (1.29 KB)
cmake and make outputs.txt (3.71 KB)

@agartner01: That’s due to a bug in the installation, which will be fixed when I come out with a new release (I’m waiting on some patches upstream right now). Read flameout’s post and see if that helps.

Also, If you still have difficulty, please post NetConsole output. That usually helps. But because of some bugs in the cmake files you have to make those symlinks.