How Does Compiling Work?

[masoug]

I have been programming the cRIO for two years, and I still haven't been able to grasp how the compiler makes the .out file and how the cRIO links the program on runtime.

I am currently confused about the whole linking process: why do we need WPILib.a when the FRC_UserProgram.out file links on runtime? And what does FRC_UserProgram.out link to when loaded? Why would I get errors about undefined symbols (for WPILib) on runtime?

I greatly appreciate any help!

Thanks!

[PAR_WIG1350]

a compiler, in simple terms, simply converts code from a high lever, to a low level (machine code, ones and zeros).
other than that, I don't know much about your particular issue, sorry.

[Ether]

Originally Posted by PAR_WIG1350

a compiler, in simple terms, simply converts code from a high lever, to a low level (machine code, ones and zeros).

Ready-to-execute machine code is the result of compiling and linking.

A compiler generally produces object code that needs to be resolved with a linker, not ready-to-execute machine code.

A cross-compiler (like the Windriver installation) produces object code for a target (the cRIO) which differs from the host (Windows PC) on which the compiler is executing.

[mikets]

Don't know the details of the cRIO platform and the Wind River cross compiler but if it were the Windows platform which I know very well, the compiler compiles the source code into machine code for the target platform. If the code calls some sort of library, then the generated code needs to link to the library code. There are two types of linking: static linking and dynamic linking. Static linking incorporates the library code into the main code as if it is part of your program. So all the calls to the library functions are resolved at compile/link time. Dynamic linking means the actual library code is not incorporated into the main code. Instead, the main code is linked to a stub library which get resolved when your program is loaded into memory at which time, all the stub functions got patched to point to the real library code already in memory. I believe the WPI library is part of the cRIO image. If for some reason the stub library you linked to is a different version from the actual library in the cRIO image, you may have unresolved symbols. But you would not kow until runtime when the dynamic linking takes place.

[Ether]

Originally Posted by mikets

Don't know the details of the cRIO platform and the Wind River cross compiler but if it were the Windows platform which I know very well, the compiler compiles the source code into machine code for the target platform. If the code calls some sort of library, then the generated code needs to link to the library code.

There's always a linking step. It may be handled seamlessly so you don't see it.

http://www.cprogramming.com/compilingandlinking.html

[masoug]

Originally Posted by mikets

Don't know the details of the cRIO platform and the Wind River cross compiler but if it were the Windows platform which I know very well, the compiler compiles the source code into machine code for the target platform. If the code calls some sort of library, then the generated code needs to link to the library code. There are two types of linking: static linking and dynamic linking. Static linking incorporates the library code into the main code as if it is part of your program. So all the calls to the library functions are resolved at compile/link time. Dynamic linking means the actual library code is not incorporated into the main code. Instead, the main code is linked to a stub library which get resolved when your program is loaded into memory at which time, all the stub functions got patched to point to the real library code already in memory. I believe the WPI library is part of the cRIO image. If for some reason the stub library you linked to is a different version from the actual library in the cRIO image, you may have unresolved symbols. But you would not kow until runtime when the dynamic linking takes place.

Hmm, stub functions? Like prototypes?

[mikets]

In the Windows platform, when you are calling functions that reside in another DLL, you link to a .lib file. This .lib file contains stub functions that call into an indirect function table. This function table will be resolved (dynamic linked) to the real functions when your program is loaded into memory. This allows multiple programs to link to the same copy of the DLL in memory thus saving memory.

[EdenA]

Originally Posted by mikets

In the Windows platform, when you are calling functions that reside in another DLL, you link to a .lib file. This .lib file contains stub functions that call into an indirect function table. This function table will be resolved (dynamic linked) to the real functions when your program is loaded into memory. This allows multiple programs to link to the same copy of the DLL in memory thus saving memory.

Actually, I believe that the exe (Portable Executable) file contains a table of import entries at a fixed location in virtual memory in the header of the PE where the program calls to for functions in DLLs to be imported. The Windows program loader at runtime then iterates through the import table entires, patching the code at the entries to JMP to the DLL's imported functions in virtual memory (where they are located at a given time). I believe the .lib file's contents will change whether the library is shared or static.

If the library is static, it's code will be stuck in with the rest of the application's which is using it, and the .lib file will probably contain the object file code (an intermediate form of code before machine code, with stuff such as relocation information) of the library.

If the library is shared, the .lib file will probably contain a list of functions which exist in the shared library. The actual code will be separated from the application's which uses it- in a PE file on Windows (.exe, .dll) or .out in vxWorks. This makes it possible to update the library an application is using without recompiling the application, also promoting modularity and code reuse. The purpose of the .lib file for applications on conventional operating systems here is usually to facilitate the process of "static" linking.

However, a lot of this information is irrelevant to the poster's question.

For programming with the vxWorks operating system on the cRIO, it is important to understand in the compile process the distinction between "static" and "dynamic" linking. On vxWorks, .out files (your program) are linked dynamically. What does this mean?

Lets say you have a C/C++ program with a function declared in a header file, and your code calls it, but you haven't compiled the definition of the function (it is unwritten in your project). On a conventional operating system such as Windows or Unix the compiler will resolve links between functions and function calls at compile time- meaning that the application's exe will know for sure where those functions are once it is constructed.

In other words, if you are using a function which is declared to exist but not defined you will get an error at compile time. This is static linking.

On vxWorks, you will not get an error at compile time. You will get an error at runtime, because when vxWorks loads programs it dynamically links all applications calls to functions at runtime (resolves where functions are at the load time, instead of compile time). This is dynamic linking.

Discovering linker errors at runtime can lead to some hard-to-resolve crashes if you aren't debugging an application when it's run by the vxWorks program loader. So, you should usually always run your program via debug kernel task.

The WPI library is not part of the cRIO image. It is a static library, merged in with the rest of your code at compile time. That's why it is so easy to recompile and use your own version of WPI lib.

For a complete view of the traditional compilation process, I would recommend reading Compilers: Principles, Techniques, and Tools (the dragon book).

[masoug]

Okay, I see how this works now...
Thanks!

[jhersh]

The thing that makes the VxWorks (and Linux as well) different from what you see on Windows is that even though WPILib is statically linked with your application, the loader has the chance to resolve any missing symbols at runtime. Because of this, even if the linker knows that it can't find all the WPILib symbols when it's linking (because it's the wrong version of the FPGA for instance) it doesn't know that those symbols won't exist when actually loading it, so you get no error.

On Windows, all symbols you want to link against must exist when the application is linked. That's why on Windows you have a stub library to provide those symbols. This allows the link to know if the symbols you expect to have are there and you will get an error when building your program if they aren't. There are no stub libraries on VxWorks. There's always the case where the stub library and the DLL are inconsistent, and you'll get runtime loader errors in that case on Windows.