Chief Delphi - Custom C++ Toolchain

Page 1 of 3

Show 100 post(s) from this thread on one page

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Programming (http://www.chiefdelphi.com/forums/forumdisplay.php?f=51)

- - Custom C++ Toolchain (http://www.chiefdelphi.com/forums/showthread.php?t=105877)

rbmj	22-04-2012 10:22

Custom C++ Toolchain - GCC 4.7.0!

Wanted to use C++ but chose Java instead because Eclipse is cross-platform?

Using C++ and wishing you could use a newer version of gcc, with performance and optimization improvements? Wanted a native gcc build on linux?

Wishing you could use C++11 features in your robot code? Lambdas, rvalue references, standard library improvements, the whole 9 yards?

After a lot of work by several people, I can say that there are working build scripts for a complete C++ toolchain using GCC 4.7.0. Thanks for the build script go to the following people:
- Austin Schuh
- Ross Light
- Cody Schafer

GCC Build Scripts can be found here:
https://github.com/rbmj/vxworks-gcc-patches

Static-Linked GCC Binaries are in the "Downloads" Section. The binaries I built *should* run on any modern amd64 linux machine, but I'm not sure, so please test!

WPILib patched for new GCC is here:
https://github.com/rbmj/wpilib

The binaries in the downloads section should work on any linux machine.

I don't have any windows binaries as I haven't ported the build script to windows, but if anyone wants to help with that, that would be REALLY COOL :)

Finally, I have a toolchain file for CMake. This toolchain file is here:
https://github.com/rbmj/cmake_vxworks

You will need to use this toolchain file if you want to compile WPILib. TODO: Give WPILib a find_package/.config file!

An example CMakeLists.txt file for a simple robot project:

Code:

cmake_minimum_required(VERSION 2.8)



#project specific EXAMPLE

project(FRC)

set(FRC_SOURCES main.cpp) #just add to this list here

set(FRC_INCLUDES ${WPILIB_INCLUDE_DIR}) #defined by the toolchain

set(FRC_LIBS ${WPILIB_LIBRARY}) #ditto

# below here should stay pretty much the same

add_executable(FRC_UserProgram ${FRC_SOURCES})

set_target_properties(FRC_UserProgram PROPERTIES SUFFIX .out)

target_link_libraries(FRC_UserProgram ${FRC_LIBS})

include_directories(${FRC_INCLUDES})

compile with:

Code:

$ cmake /path/to/repo/robot_test -DCMAKE_TOOLCHAIN_FILE=/path/to/toolchain/vxworks_toolchain.cmake 

$ make

$ wput -u FRC_UserProgram.out ftp://10.xx.yy.12/ni-rt/system/ #deploy

CMake is awesome - use it! This toolchain is NOT COMPATIBLE with the ucpp makefiles, so if you want a working executable you MUST use CMake!

TODO:
- Do more extensive tests, including C++11 features
- Make the build scripts nicer and less crufty
- Get a working windows port

JamesTerm

22-04-2012 20:59

Re: Custom C++ Toolchain - GCC 4.7.0!

Quote:

Originally Posted by rbmj (Post 1161335)

CMake is awesome

I agree!
CMake is awesome... it is used in Open Scene Graph... it's a great way for the Visual Studio developers to interact with the GCC developers.

rbmj	23-04-2012 11:16

Re: Custom C++ Toolchain

The CMake Toolchain file should work with the standard "gccdist" toolchain. If you download this toolchain <1> then the cmake file should work fine (NOTE: It requires munch.sh and strip_syms.sh from the vxworks-gcc-patches repo to be installed in WIND_BASE - maybe should make these a separate package), just set TOOLCHAIN_IS_GCCDIST to true.

The cmake toolchain file would work on windows if it weren't for the fact that I haven't bothered to create windows equivalents of munch.sh and strip_syms.sh. I don't know batch (or maybe it would have to be vbscript...) very well, so I wouldn't know how to substitute the cut and sed commands.

If I can get the cmake scripts working on windows that opens up the possibility of teams using those dreamspark cards and developing the robot in visual studio, or codeblocks for FOSS purists like myself :D

I don't know if the generators for VS or C::B support manual target rules though (CMAKE_C_COMPILE_OBJECT et al).

Side note:

I'm thinking in the future this can be four packages:
vxworks-binutils
vxworks-gcc (depends vxworks-binutils)
vxworks-scripts (depends vxworks-binutils)
vxworks-cmake-toolchain (depends vxworks-scripts, vxworks-gcc, vxworks-binutils)

Maybe have a PPA...

So question: Should I keep statically linking the binaries so they work on most linuxes, or should I use "proper" packaging?

Also, if anyone gets a chance to test this with a machine other than my own, that would be cool :) maybe get it to work on win32? MSYS seems like the best option...

1. ftp://ftp.ni.com/pub/devzone/tut/updated_vxworks63gccdist.zip"

Peter Johnson

23-04-2012 13:51

Re: Custom C++ Toolchain

Great job! How hard would it be to get gfortran working? Fortran is the primary dependency I've been missing to get numpy ported to the cRIO.

rbmj	23-04-2012 14:09

Re: Custom C++ Toolchain

Quote:

Originally Posted by Peter Johnson (Post 1161941)

Great job! How hard would it be to get gfortran working? Fortran is the primary dependency I've been missing to get numpy ported to the cRIO.

AFAIK the only thing you would need to do is change the ./configure line to gcc. If you change the argument in the build script that says "--enable-languages=c,c++" to "--enable-languages=c,c++,fortran" you should get gfortran (I think).

If you're on windows, you might have to run the script through an MSYS shell. Right now the build script needs a basic set of bash commands, wget, sed, and realpath (IIRC).

rbmj	01-05-2012 19:44

Re: Custom C++ Toolchain

OK. I'm looking for some advice:

Currently I have some statically linked binaries up. I'm worried, though, because those also include the system headers, which I don't have the right to redistribute. Should I take those binaries down and require everyone to use the build script?

I couldn't find a contact at WindRiver for FRC, so I guess I'll have to email their generic inquiry account. Maybe this can be easily resolved.

They also do some weird stuff with their include directories, that I'd ideally like to move around to simplify installation of GCC, as right now there's some hackish stuff I have to do to get GCC to build right :-(

--------

Also, I forgot to post instructions for the build script.

You need to call it like this:

/path/to/build/script/build.bash PREFIX DLDIR SRCDIR BUILDDIR

Where:
PREFIX is where to install everything to (e.g. /usr/local)
DLDIR is where to put the downloaded tarballs
SRCDIR is where to extract the sources
BUILDDIR is where to put the build trees.

Note that these should be absolute paths. If you're in a clean directory, this should be sufficient:

# ../build.bash /usr/local `pwd`/download `pwd`/src `pwd`/build

It also needs to be run as root. If you have build problems let me know.

Peter Johnson

01-05-2012 20:34

Re: Custom C++ Toolchain

Quote:

Originally Posted by rbmj (Post 1161949)

Unfortunately, it isn't quite this easy. I tried it and got a number of errors in libgfortran that look like they'll take some time to work through (partly missing things in system headers).

rbmj	01-05-2012 20:57

Re: Custom C++ Toolchain

Quote:

Originally Posted by Peter Johnson (Post 1165480)

Unfortunately, it isn't quite this easy. I tried it and got a number of errors in libgfortran that look like they'll take some time to work through (partly missing things in system headers).

If you try and build libstdc++ without patches you get the same issues. I'm planning on sending a submission to gcc-patches, but in the interim, you can see the kind of simple fixes we had to make.

Check wrn/coreip/ for the include as well. Since GCC doesn't support setting target-specific include directories without some really bad hacks, I had to put in some temporary stubs to redirect the includes. I'm thinking of restructuring the system headers so one no longer needs this cruft, but I'm pushing the boundaries of fair use as is :-(

The other thing that is prone to break is stuff that has to do with the filesystem. Look at the patches for examples of how stuff had to be changed.

Luckily, there were only a few files that screwed up for libstdc++. Hopefully libgfortran is similarly simple. And if you need any help you can post!

rbmj	05-05-2012 22:58

Re: Custom C++ Toolchain

OK, finally got around to contacting Wind River. Confirmation Number 319, whatever that means.

Quote:

Hello,

I am a student on FIRST (For the Inspiration and Recognition of Science and Technology) Robotics Team 612. I am working on developing an alternative toolchain targeting vxWorks in order to allow teams to use a newer version of GCC, primarily in order to have a native linux build for FRC and to get new features (including C++11 support) not present in GCC 3.4.4 that is currently provided to FRC Teams.

Currently, in order to build the necessary runtime libraries, I am using the system headers from [1], which National Instruments distributes but contains (as far as I can tell) mostly Wind River Intellectual Property.

In order for the headers to be accepted, it is necessary to incorporate a few patches. These patches are relatively minor and are only intended to allow the headers to be accepted by the newer version of GCC and increase interoperability and standards compliance. However, the headers contain the following notice:

/*
* Copyright (c) 1984-2005 Wind River Systems, Inc.
*
* The right to copy, distribute or otherwise make use of this software
* may be licensed only pursuant to the terms of an applicable Wind River
* license agreement. No license to Wind River intellectual property rights
* is granted herein. All rights not licensed by Wind River are reserved
* by Wind River.
*/

Which very clearly states that posession of the file does not confer any Intellectual Property rights to users. Thus, modification and redistribution of such code is illegal. I made my modifications under the Wind River FRC License Agreement (namely, "THIS OBJECT CODE LICENSE AND DISTRIBUTION AGREEMENT"), section 4, and therefore freely assign any copyright to any modifications I made to the system headers to Wind River.

Additionally, in order to share my progress with the community, I wish to redistribute the toolchain I have developed. As a necessary part of its functioning, such a toolchain requires the system headers for the target Operating System (in this case vxWorks). This does not contain any implementation code but is rather only the declarations to allow code to operate inside of vxWorks. In order to share my work with the FIRST community then, it is necessary for me to distribute the vxWorks system headers.

Thus, I request that Wind River release the system headers (and the headers alone - I do not request a license to any implementation code) for vxWorks under an open source license (such as FIRST's BSD-Style license) in order to facilitate the exchange of knowledge and tools related to vxWorks within the FIRST community. Should Wind River be unwilling to do so, I request that Wind River license allow modification and redistribution of the system headers to vxWorks for non-commercial purposes, such as the FIRST Robotics Competition and related activities by team members and affiliated adults. Again, this would not confer any rights on the actual proprietary implementation of vxWorks, but only the declarations needed to interface with such code.

If you wish to see the patches I have made (they are minimal), I am more than willing to supply them and an explanation as to why they are necessary; they are, after all, your intellectual property under the current license agreement!

Sincerely,

Robert 'Blair' Mason, Jr.
Member and Former 'CEO'
Chantilly Robotics
FIRST Robotics Competition Team 612

[1] ftp://ftp.ni.com/pub/devzone/tut/upd...s63gccdist.zip

rbmj	22-05-2012 18:46

Re: Custom C++ Toolchain

3 Attachment(s)

Quote:

Originally Posted by Peter Johnson (Post 1161941)

Great job! How hard would it be to get gfortran working? Fortran is the primary dependency I've been missing to get numpy ported to the cRIO.

OK, so I have a fortran compiler working. The issue: I don't have a build script yet. I'm not going to make one, as I'm working on transitioning from a build-script system to a package-based system. I think this will be easier for 90% of people, and the 10% who need to compile it themselves can use the command line themselves :-D.

I have a series of patches, which I'll attach, but they're based off of the trunk. I'm working with the gcc devs to get them committed, so hopefully by gcc-4.8 vxworks will compile out of the box.

There's also a series of patches which need to be applied to the C headers. Finally, there's a patch that needs to be applied while GCC builds and then reverted after the build.

So, some build instructions.

First, install the cross binutils from earlier- those will still work for building gcc.
Then download all of the attachments to this post. Remove the .txt extension from the patch file:

Code:

$ mv headers-build.patch{.txt,}

Untar the patches:

Code:

$ tar -xf gcc-patches.tgz

$ tar -xf wrs-patches.tgz

Get the latest gcc trunk:

Code:

$ svn checkout svn://gcc.gnu.org/svn/gcc/trunk gcc

Get the gccdist and unzip it:

Code:

$ wget ftp://ftp.ni.com/pub/devzone/tut/updated_vxworks63gccdist.zip

$ unzip updated_vxworks63gccdist.zip

Apply the permanent patches to gccdist:

Code:

$ cat wrs-patches/*.patch | patch -p1 -l -d gccdist/WindRiver/vxworks-6.3/target/h

And apply the temporary gcc build patch:

Code:

$ patch -p1 -l -d gccdist/WindRiver/vxworks-6.3/target/h < headers-build.patch

Now switch to the gcc directory.
Since I made changes to configure.ac you'll need to regenerate that:

Code:

$ autoconf

If it barfs at you about not having the right autoconf version, go into config/override.m4 and change the lines that look like this:

Code:

dnl Ensure exactly this Autoconf version is used

m4_ifndef([_GCC_AUTOCONF_VERSION],

  [m4_define([_GCC_AUTOCONF_VERSION], [2.64])])

to look like this (note that my autoconf verison is 2.68):

Code:

dnl Ensure exactly this Autoconf version is used

m4_ifndef([_GCC_AUTOCONF_VERSION],

  [m4_define([_GCC_AUTOCONF_VERSION], [2.68])])

Then re-run autoconf.

OK, if you're still with me, now it's time to run configure. Switch out of the gcc directory (cd ..) and make a new directory called build (mkdir build), then switch into the build directory. Here's how to invoke configure:

Code:

# ../gcc/configure \

            --prefix=/usr \

            --target=powerpc-wrs-vxworks \

            --with-gnu-as \

            --with-gnu-ld \

            --with-headers="../gccdist/WindRiver/vxworks-6.3/target/h" \

            --disable-shared \

            --disable-libssp \

            --disable-multilib \

            --with-float=hard \

            --enable-languages=c,c++,fortran \

            --enable-libstdc++-v3 \

            --enable-threads=vxworks \

            --without-gconv \

            --disable-libgomp \

            --disable-nls \

            --disable-libmudflap \

            --with-cpu-PPC603 \

            --disable-symvers

You'll need to have the development libraries for libmpfr, libmpc, and libgmp on your system (libmpfr-dev, libmpc-dev, and libgmp-dev on debian) installed for this to work. Also note the #; configure needs to be run as root here.

Now for the fun part. Hopefully, if you've done everything up to this point, you can just say
[code]
$ make -j4
[/code
and everything will work just fine. Then just make install as root.

Finally, clean up some of the unnecessary fixes:

Code:

# patch -p1 -l -R -d /usr/powerpc-wrs-vxworks/sys-include < headers-build.patch

And you should hopefully have a working compiler!

This will become much simpler once I update the build script, but in case any of you want to try this out and understand how to do it, I posted this.

If you have any questions, let me know.

Note that fortran support is untested as I have nothing to test it with. It would be nice if somebody could test this.

Libgfortran's configure is acting funny too - gotta fix that.

flameout

26-05-2012 14:01

Re: Custom C++ Toolchain

I have installed the compiler binaries and wpilib on Xubuntu 12.04 through your .deb files. I then dumped the toolchain file and you sample CMakeLists.txt file into a directory. However, running CMake fails for me.

This appears to be the important part:

Code:

powerpc-wrs-vxworks-c++: fatal error: environment variable 'WIND_BASE' not

  defined

What is suppossed to be setting WIND_BASE? (If this isn't enough, then I can go ahead and post everything I have done related to this project).

Thank you, I'm really excited to get this working.

rbmj	26-05-2012 14:32

Re: Custom C++ Toolchain

WIND_BASE doesn't matter anymore. It just needs to be defined to something. I just say export WIND_BASE='asdf'. If you want to be 'proper' about it, you can set WIND_BASE to /usr/local/powerpc-wrs-vxworks/wind_base

flameout

26-05-2012 15:58

Re: Custom C++ Toolchain

Okay, then, I guess I got confused because when I tried setting it myself, I got a different error, which I interpreted to be because I had an incorrect setting for that variable :P

I have it working now (I think... I have yet to try it on the robot). However, I did have to make some symbolic links to make it work. In particular, I had to make the following links:

Code:

lrwxrwxrwx 1 root root 15 May 26 12:49 /usr/local/powerpc-wrs-vxworks/host -> wind_base/host/

lrwxrwxrwx 1 root root 11 May 26 12:48 /usr/local/powerpc-wrs-vxworks/lib/libWPILib.a -> libwpilib.a

lrwxrwxrwx 1 root root 18 May 26 12:43 /usr/local/powerpc-wrs-vxworks/munch.sh -> wind_base/munch.sh

lrwxrwxrwx 1 root root 23 May 26 12:46 /usr/local/powerpc-wrs-vxworks/strip_syms.sh -> wind_base/strip_syms.sh

Thank you for your help.

Azrathud

21-06-2012 16:22

Re: Custom C++ Toolchain

This is awesome, and I hope I'll be programming on Linux come next season.

I'm trying to set up this toolchain on a 64-bit install of Archlinux, and I'm getting some errors after

Running the build script for downloading and patching gcc for vxworks
Editing the toolchain(changing gccdist to true, setting deploy ip)
Creating a CMakeLists.txt file based off your example
Running cmake

Maybe this is something of note: while running

Code:

cmake /home/azrathud/vxworks-c++-toolchain/project -DCMAKE_TOOLCHAIN_FILE=/home/

azrathud/vxworks-c++-toolchain/cmake_vxworks/vxworks_toolchain.cmake

Cmake outputs that CMAKE_TOOLCHAIN_FILE is not used

I'm trying to compile last season's code which is in the src directory specified in the CMakeLists.txt

From the errors, I think gcc cannot find the header files in /usr/local/powerpc-wrs-vxworks/sys-include

Code:

In file included from /home/azrathud/vxworks-c++-toolchain/project/wpilib/ErrorBase.h:12:0,

                 from /home/azrathud/vxworks-c++-toolchain/project/wpilib/Commands/Command.h:10,

                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.h:4,

                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.cpp:1:

/home/azrathud/vxworks-c++-toolchain/project/wpilib/Error.h:13:21: fatal error: vxWorks.h: No such file or directory

compilation terminated.

make[2]: *** [CMakeFiles/FRC_UserProgram.dir/src/CommandBase.cpp.o] Error 1

make[1]: *** [CMakeFiles/FRC_UserProgram.dir/all] Error 2

make: *** [all] Error 2

I included what I pulled from your WPIlib github repo in the CMakeLists.txt locally, but I looks like from your cmake toolchain file, that I should be placing them in the place under /local .. include/WPILib, yes? Also, I added the sys-include to the list of includes, but it didn't look like it did anything to help.

Code:

cmake_minimum_required(VERSION 2.8)



project(FRC)



aux_source_directory(./src FRC_SOURCES) # I'll just run cmake all the time to

        # make sure new files are added



set(FRC_INCLUDES ${WPILIB_INCLUDE_DIR} wpilib ${TOOLCHAIN_PREFIX}/sys-include ) #defined by toolchain

set(FRC_LIBS ${WPILIB_LIBRARY}) # defined by toolchain



add_executable(FRC_UserProgram ${FRC_SOURCES})

set_target_properties(FRC_UserProgram PROPERTIES SUFFIX .out)

target_link_libraries(FRC_UserProgram ${FRC_LIBS})

include_directories(${FRC_INCLUDES})

I think the problem may relate to that I haven't compiled the wpilib into a shared library, like the toolchain suggests. (I have no ${TOOLCHAIN_PREFIX}/lib/libWPILib.a) How would I go about correctly including the patched WPILib and allow that to be linked correctly, etc?

rbmj	27-06-2012 09:14

Re: Custom C++ Toolchain

Quote:

Originally Posted by Azrathud (Post 1174865)

This is awesome, and I hope I'll be programming on Linux come next season.

I'm trying to set up this toolchain on a 64-bit install of Archlinux, and I'm getting some errors after

Running the build script for downloading and patching gcc for vxworks
Editing the toolchain(changing gccdist to true, setting deploy ip)
Creating a CMakeLists.txt file based off your example
Running cmake

Maybe this is something of note: while running

Code:

cmake /home/azrathud/vxworks-c++-toolchain/project -DCMAKE_TOOLCHAIN_FILE=/home/

azrathud/vxworks-c++-toolchain/cmake_vxworks/vxworks_toolchain.cmake

Cmake outputs that CMAKE_TOOLCHAIN_FILE is not used

The internet seems to think that this is a harmless bug output by some versions of cmake that can be fixed by setting CMAKE_TOOLCHAIN_FILE in the file itself.

Quote:

I'm trying to compile last season's code which is in the src directory specified in the CMakeLists.txt

From the errors, I think gcc cannot find the header files in /usr/local/powerpc-wrs-vxworks/sys-include

Code:

In file included from /home/azrathud/vxworks-c++-toolchain/project/wpilib/ErrorBase.h:12:0,

                 from /home/azrathud/vxworks-c++-toolchain/project/wpilib/Commands/Command.h:10,

                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.h:4,

                 from /home/azrathud/vxworks-c++-toolchain/project/src/CommandBase.cpp:1:

/home/azrathud/vxworks-c++-toolchain/project/wpilib/Error.h:13:21: fatal error: vxWorks.h: No such file or directory

compilation terminated.

make[2]: *** [CMakeFiles/FRC_UserProgram.dir/src/CommandBase.cpp.o] Error 1

make[1]: *** [CMakeFiles/FRC_UserProgram.dir/all] Error 2

make: *** [all] Error 2

Yes, that appears to be the case. Depending on *how* you built gcc, it may or may not have all of the include directories specified. It could also be that CMake is using the wrong gcc, so ask it to be verbose and check that. Finally, find that gcc's cc1 (probably in /usr/powerpc-wrs-vxworks/bin if memory serves) and run cc1 -v and try and figure out what include directories it has. Check the directories listed.

The final option is that that code worked on windows because windows is a non-case-sensitive filesystem. Check your capitalization (vxWorks.h maybe?)

Quote:

Code:

cmake_minimum_required(VERSION 2.8)



project(FRC)



aux_source_directory(./src FRC_SOURCES) # I'll just run cmake all the time to

        # make sure new files are added



set(FRC_INCLUDES ${WPILIB_INCLUDE_DIR} wpilib ${TOOLCHAIN_PREFIX}/sys-include ) #defined by toolchain

set(FRC_LIBS ${WPILIB_LIBRARY}) # defined by toolchain



add_executable(FRC_UserProgram ${FRC_SOURCES})

set_target_properties(FRC_UserProgram PROPERTIES SUFFIX .out)

target_link_libraries(FRC_UserProgram ${FRC_LIBS})

include_directories(${FRC_INCLUDES})

If you run cmake on the wpilib directory with a working toolchain file, you'll get a *STATIC* library. One can then link to this library with the toolchain WPILIB_LIBRARY - which hopefully at some point in the future will become a .config file.

All times are GMT -5. The time now is 13:33.

Page 1 of 3

Show 100 post(s) from this thread on one page