Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

[OT] Abusing GNU compiler and linker: Make the linker generate an array of function pointers

by afoken (Chancellor)
on Oct 16, 2019 at 19:48 UTC ( [id://11107560]=perlquestion: print w/replies, xml ) Need Help??

afoken has asked for the wisdom of the Perl Monks concerning the following question:

Hi!

I painted byself into a corner at work. Environment: Atmelstudio 7, GCC 6.3.1 generating ARM code for running on bare metal.

The basic problem

Most of our C modules have some init function, many modules also have a run function. Essentially, the ARM does a poor man's cooperative multitasking, calling all run functions in a fixed order after having called all init functions. So we end up with code very similar to this:

int main(void) { SystemInit(); // from Atmel Foo_Init(); // our code Bar_Init(); // our code Baz_Init(); // our code // many, many more, all hand-written for (;;) { Foo_Run(); // our code Bar_Run(); // our code Baz_Run(); // our code // many, many more, all hand-written } }

Now guess what happens: You forget to add your New_Init() and New_Run() to main(), and suddenly, the system behaves very strange. In fact, this is number one on our debugging check list. Number two is wrong order of the init functions. Also, all that hand-written stuff takes a lot of time when you set up a new project.

Using arrays

So I had the idea to delegate the job to the compiler and the linker. By convention, most init and run functions return void and don't take arguments. I made that a strict requirement, so now I have a common prototype for all init and run functions. I can manually fill two arrays with all of that functions, and reduce main() to iterate over the arrays:

static Func_t initFuncs[] = { Foo_Init, Bar_Init, Baz_Init, ... }; static Func_t runFuncs[] = { Foo_Run, Bar_Run, Baz_Run, ... }; int main(void) { SystemInit(); // from Atmel for (size_t i=0; i<ARRAY_SIZE(initFunc); i++) { initFuncs[i](); } for (;;) { for (size_t i=0; i<ARRAY_SIZE(runFunc); i++) { runFuncs[i](); } } }

Nothing gained so far, we would still have to add New_Init() and New_Run() to main.c. The only difference is that they go into the array initializers instead of main().

Now if I add some small macros to our modules, they take care of handling dependencies. Basically, some auto-generated flag variables and repeatedly calling all init functions until all are done. Init functions return immediately if their dependencies are not yet met. When an init function is done, it replaces itself with the module's run function. So I only have a single function pointer array, initially containing the init functions, later containing the run functions.

Making the computer do it

All I need is an array automatically filled with function pointers to the init functions. Can't be that hard, right? Create a new section ".ourinits" in the linker script for the relocated (initialized) RAM area, create one symbols at start of the section and another one at end of the section, and make the macros generate an array fragment static Func_t XXX_addToList __attribute__((section(".ourinits"))) = XXX_Init;. Well, better don't make it static, because it looks unused to the compiler and will be removed; make it extern so that it reaches the linker.

All of those manual work in main() is gone, main just has to iterate over an external array generated by macros and the compiler, and assembled by the linker.

Except that it does not work. The linker decides to remove my array fragments. Of course it does, because no one references them by name. main() only accesses the array via the start and end symbols. D'oh!

Making the computer do it, take two

__attribute__((constructor))

(That's where I stole a part of my plan.)

Well, not really. Yes, it should generate a list of "constructor" functions, but they are called before main(), the list is in ROM so I can't change it, and I don't want the C runtime to call the init functions once. I want them to be called from main(), repeatedly, and until their dependencies have resolved. Also, I don't want to fight with the starting C runtime and 3rd party code using that mechanism.

Help!

My plan looks so good to us. All that I need is a way to tell the linker to stop cleaning up my precious section. Shouldn't be that hard, right?

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
  • Comment on [OT] Abusing GNU compiler and linker: Make the linker generate an array of function pointers
  • Select or Download Code

Replies are listed 'Best First'.
Re: [OT] Abusing GNU compiler and linker: Make the linker generate an array of function pointers
by afoken (Chancellor) on Oct 16, 2019 at 20:01 UTC

    Using Perlmonks as a rubber duck ... ;-)

    All that I need is a way to tell the linker to stop cleaning up my precious section. Shouldn't be that hard, right?

    It isn't that hard. Just tell the linker to KEEP records in my section. Exactly as it is done by default for constructor functions. D'oh!

    Now my plan works.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re: [OT] Abusing GNU compiler and linker: Make the linker generate an array of function pointers
by Marshall (Canon) on Oct 19, 2019 at 13:56 UTC
    Your question sounds more like a C question than a Perl question.

    Now guess what happens: You forget to add your New_Init() and New_Run() to main(), and suddenly, the system behaves very strange. In fact, it is number one on our debugging check list. Number two is wrong order of the init functions. Also, all that hand-written stuff takes a lot of time when you set up a new project.

    One idea is to have a single config file in some easy to understand format and have Perl write the .c and .h code for this initialization and loop run code from that.

    You have to modify your make file to accommodate that Perl step. In general there is no problem with having a program write another program (Larry Wall says that these are the happiest programs of all).

      Your question sounds more like a C question than a Perl question.

      Yes, that's why I marked my posting as off-topic.

      One idea is to have a single config file in some easy to understand format and have Perl write the .c and .h code for this initialization and loop run code from that.

      Yes, that would be possible.

      I've considered generating the entire main.c from the module sources using a perl script. But we usually put some application-specific code into main that does not fit elsewhere, so a generated main.c is not acceptable, and so I gave up that idea.

      But you are right, I could get away by simply reading all module file names from the AS project file, extracting XXX_Init() functions from the modules, and generate an auxillary file that basically just contains an array of pointers to init functions. That would work without messing with the linker, and I wouldn't even need a config file. We already do something similar to assemble a version number.

      I could even go one step back and assemble two arrays, one for init functions, one for run functions. That way, our software would still have two distinct global phases, one init, one run, and except for making all init and run functions return void and taking no arguments, no changes to existing modules would be needed.

      But one of the reasons to get rid of the two distinct global phases is that some critical modules need the services of other modules while initializing, and those services are available only in the run phase. So we work around by moving part of the initialization to the run function of those modules, and prevent calling most run functions until those critical modules are ready. Manually, of course. -- Boy, that's hard to express in english. In code, it looks about like this:

      int main(void) { SystemInit(); Foo_Init(); Bar_Init(); Baz_Init(); // and so on for(;;) { Foo_Run(); //< module that "Bar" depends on Bar_Run(); //< critical module if (Bar_IsReady()) { Baz_Run(); //< "normal" module // and all other "normal" modules } } }

      Other modules originally had a long-running init function, but no run function. Again, we hacked around by moving parts of the init function to a new run function that will be called only until the long-running init is done:

      int main(void) { SystemInit(); Foo_Init(); Bar_Init(); Baz_Init(); Abc_Init(); // and so on for(;;) { Foo_Run(); //< module that "Bar" depends on Bar_Run(); //< critical module if (!Abc_IsDone()) { Abc_Run(); } if (Bar_IsReady()) { Baz_Run(); //< "normal" module // and all other "normal" modules } } }

      I'm not willing to go back to that situation. The trick of replacing init functions with run functions in the function pointer array automatically resolves that problems. Any module can use any run-time services of any other module it depends on from its init function, and each init function can run as long as required.

      But I will definitly discuss that idea of a perl script instead of the linker for generating the function pointer array at work. Both ways need to touch modules that expect arguments for XXX_Init() and/or XXX_Run(). Both ways need to touch modules that depend on other modules to add the dependency statements. And there is also a minor issue with the code that generates the version number, so we will probably have to scan each module during build anyway. Both could be merged into a single perl script.

      Linker scripts are part of each project, they are in a repository like all other source files, so modifying them is no big deal, and we already modify the linker scripts for other reasons. We also use perl as part of the build process, with all scripts in the repository. Both ways are possible, but both look like voodoo to interns and freshly hired developers that know C only as a subset of desktop C++, and to the old developers whose main experience is with DOS/Win-16 and 8-bit microcontrollers with horrible toolchains. 32-bit ARM and GCC are significantly different, and require a different way of doing things. So maintainability will be part of the discussion.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re: [OT] Abusing GNU compiler and linker: Make the linker generate an array of function pointers
by kikuchiyo (Hermit) on Oct 19, 2019 at 17:54 UTC
      the title of this writeup reminded me of this recent IOCCC winning entry

      *g*

      I've hacked a working prototype module for internal discussion. The header file has about 300 lines of Doxygen documentation and about 30 lines of code, mainly function prototypes and macros. The source file has about 40 lines of code, plus a little bit of doxygen documentation.

      Compared with our other modules, the relation between documentation and code in the source file is normal, may be a little bit short, but the documentation of the header is unusually verbose. Typically, the doxygen lines are twice or three times the number of code lines.

      The prototype is still a little bit rough, it still needs some paranoia checks and some cleanly documented ways for advanced use of the mechanism. That will increase the code size by about 20 lines and the documentation by about 100 lines.

      So yes, some parts of the code may look like coming from the IOCCC, but it's only the code. The documentation explains what happens, why and how, as usual for our projects.


      In fact, I actually use code from an IOCCC winner at work:

      IOCCC winner 1984

      int i;main(){for(;i["]<i;++i){--i;}"];read('-'-'-',i+++"hell\ o, world!\n",'/'/'/'));}read(j,i,p){write(j/p+p,i---j,i/i);}

      It's in a presentation for interns and freshly hired people, on one of the first pages, right after a picture of the HGTTG with the "don't panic" in big friendly letters. That piece of code is a really good example of bad code, on many levels. What follows is an introduction to the way we write code and documentation, why and how we do it, which modules are a little bit special, common errors, trouble with legacy code, and how to debug.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11107560]
Approved by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2024-04-19 06:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found