Generating C++ Modules

How can we add extend the Ravi shell with C or C++ code? In principle, this is very simple: we just want to call some existing code from the shell. In practice, however, we need quite a bit of so-called "wrapper code" or "glue code" which must take care of several points. Let us illustrate this with an example from the next paragraph. Suppose we have written and compiled a function called ilog2,and we want to be able to use this function from the shell, writing, for example ilog2(x+123).The following points have to be solved:

• The name ilog2 is not present in compiled code, once it is loaded. This name somehow must be added to shell's symbol table, together with the address of the entry point.
• Before calling the function ilog2 the shell must convert the argument(s) correctly (to type int, float, or whatever has been declared).
• Arguments given to the shell should be checked for correct number and type, with appropriate error handling in case they are wrong.

The fundamental ambition of RIG is "carefully look at the source code, don't ask the user to repeat information in the source code". This translates into "generate the interface code from the header files of user programs".

A simple example: the function ilog2

We illustrate the preceding discussion with a very simple example: a single function named ilog2 written in C. Our code is composed of a header file essai1.h and an implementation file essai1.cc.

The header file simply reads as follows:

int ilog2(int i);

What the user has to do: module generation

We suppose that the implementation file essai1.cc has been compiled to produce an object file essai1.o. The user now calls RIG to generate the wrapper file modessai1.cc; this file has to be compiled and link-edited with essai1.o.

These steps are carried out using the special command ravitoolin a fairly intuitive way:

	ravitool --generate -o modessai1.cc essai1.h
	ravitool --compile -o modessai1.o modessai1.cc
	ravitool --link -o modessai1.so essai1.o modessai1.o

The first of these commands generates a source file - the "module file" modessai1.cc which is then compiled and link-edited to produce the file modessai1.so. These commands are emitted via the command ravitool which in turn calls the C++ compiler with the appropriate options.

The result of these three commands is the file modessai1.so which is a shared object file that can be dynamically loaded into the Ravi interpreter.

Once this has been done, the module modessai1can be used in the Ravi shell.

Using the module in the shell

The module can now be used in the shell. We show this with C-syntax, and with Scheme syntax.

Loading and Use in the shell (C-syntax)

ilog2          // this proves that ilog2 is not yet defined
load modessai1 // load is (non standard) Scheme function
ilog2         // now defined
ilog2(5341);  // can be used as you expect
ilog2("ab");  // type control

Use in the shell (Scheme-syntax)In Scheme mode, the commands of the preceding paragraph are written as follows:

ilog2          ; proves ilog2 not yet defined
(load "modessai1") ; load is (non standard) Scheme function
ilog2         ; now defined
(ilog2 5341)  ; can be used as you expect
(ilog2 "ab")  ; type control

Generated file

Remember that the source file defines one function with the header:

int ilog2(int i);

The generated file contains a wrapper function named Sc_ilog2(). Looking at the code, you can easily get a feeling how this function works: it fetches the arguments from the stack of Ravi's virtual machine with VS_ElemQ(- GetPar(), the argument is checked to be an integer, the function ilog2is called with the argument value, the result is pushed on the virtual machine stack.

At the end of the generated file, there is an initialisation function that puts the function name and address into Ravi's symbol table:

MakePutCProc("ilog2",Sc_ilog2,1,ScNil);

#include "essai1.h"
static Ravi::Boolean localInitFunc(void);
static ScInitFunc go(localInitFunc);
void Sc_ilog2()
{
  ScVal * FB = & VS_ElemQ(- GetPar());
  int num = 0;
  if(num = 0 , IsFixNum(FB[0]))
    {
      SetResult(ScFixNum(ilog2(GetFixVal(FB[0]))));
      return ;
    }
  Errorf("bad arg [pos %d] for function %s",num,"ilog2");
}
Ravi::Boolean localInitFunc()
{
    MakePutCProc("ilog2",Sc_ilog2,1,ScNil);
  return true;
}

What can be learnt from this example

The above procedure generate-compile-link-load works for C and C++ code, in the case where no particular difficulties arise.

However, when you look carefully at existing C or C++ programs, you will realize that the interface generator's fundamental ambition - to use all information in the source code - is tough to carry through. Even worse: sometimes, important information is lacking in C++ code. And in some cases, we want the interface to do more than just call C-code (for instance with memory management).

As a consequence, some information has to be given by the user for interface generation. These are called "declarations", and usually put into a file with extension ".dcl"

Interfacing a library

Note that the implementation code of the program to be interfaced is not needed for module generation - we only need header files and compiled code, so we can do without the source files (.cc files).

Let's take the following example: I have a library $HOME/Soft/lib/libpyramid.so and the associated header files BinomialDogPyramid.h LaplacianPyramid.h DogPyramid.h in the directory $HOME/Soft/include.

Look carefully at the commands to generate such a module - they resemble closely the first example!

	ravitool --generate -project Imalab -I $HOME/Soft/include -o modpyramid.cc  \
		 BinomialDogPyramid.h LaplacianPyramid.h DogPyramid.h
	ravitool --compile -o modpyramid.o modpyramid.cc
	ravitool --link -L$HOME/Soft/lib -lpyramid -o modpyramid.so modpyramid.o

Comments: The ravitool --generate command has a parameter -project Imalab which specifies that the pyramid code uses data structures from the Imalab project; this introduces a large number of "declarations" that will be explained in the following paragraph on advanced topics. (Through the file Site-Module/Imalab/project.dcl).

For other libraries, of course, you will not need the -project Imalab parameter - but chances are that you need to give some special information that the interface generator cannot deduce frome the header files.

ravitool --generate: Command line parameters

The first step in module generation is automatic program generation using the command ravitool --generate.

General format as Unix command:

ravitool --generate options source_files

This is strictly equivalent to

ravi -mod r-ig options source_files

source-files either is a (possibly empty) list of header files, or a single .ph file. If the list is empty, then the source files must be listed in the dcl-file. .ph files are explained in a later paragraph

Options are the following:

• -o name output file name
• -project Imalab defines project settings
• -u name use declaration file name
• -I dir add dir to header-file search path
• -D name define preprocessor symbol
• -R dir add dir to module path
• -r module require module
• -i name additional include file
• -ia interactive mode (for debugging)
• -cpp-trace n n=0,1,2,4 detailed preprocessor trace (for debugging)
• -nomsg completely silent mode
• -cc name out of date

Preprocessor defaults: When the source file(s) are header files, the preprocessor is active. When the source file is a .ph file, the preprocessor is not active.

The preprocessor symbol RAVI_INTERFACE always is defined during module generation.

The default interface and declarations

In the absence of declarations, RIG generates the default interface for the header files given as arguments. The default interface contains

• a function for each function in the headers.
• a function for each class method in the headers.
• accessor functions for each field of a struct in the headers.

If we want to influence the default behavior, and for many more reasons, we write declarations for the interface generator. Declarations are some kind of pragma. Declarations have the general aspect of data-in-parentheses (they are analyzed as Scheme lists). (Current work: object-attribute-value format).

Declarations can be put in two places: in a special file, the dcl-file (given with a -u argument to the ravitool command), or they may be placed inside C-comments in the source code. Such a C-comment must have the form /*Ravi declare ( declaration ...) */The C-comment may spread over several lines; the list of declarations is read with a single call to the Scheme function read.

Using declarations, you can easily augment the interface, (e.g. to include template classses, or constants), or modify the default settings, (e.g. to exclude an item, or to add methods of a struct),

There is a separate paragraph on each of the declarations for class, struct, template.

General purpose declarations

• (header-files n1 ...) specifies the list of header files to be used for the module. Useful when the list is too long to be included in the Unix command line (I've encountered a case with 52 header files). This declaration may occur only once, replacing any previous header file list.
• (export n1 ...)Only n1 ... are to be included in the interface. Works for: struct, union, enum.
• (no-export n1 ...)Only n1 ... are not included in the interface (all the rest is included). Works for: procedure, struct, union, enum.
• (export-constant n1 ...)Works for: preprocessor constant, C++ constant.
• (scheme-name <scheme-name> <c-name>) Useful when a name is unpractical. Also an ad-hoc means to avoid name clashes.

Class interface

Format: (class id attr ...)
attr ::= attr-name | (attr-name value)

The class declarations specifies diverse informations about the interface for a class using an (attribute value) notation. An attribute name without value specifies the default value (whenever this makes sense).

The class-name id can be written as a symbol, when it is a simple class name, or as a string, when it is a template class. In the latter case, the name of the template must have been declared in a preceding declaration : (template name).

The attributes for the class declaration are:

• export-fields
• export-methods
• abstract
• print print-function
• print-method
• delete delete-function
• delete-method
• reference-count
• reference-count-methods
• dynamic-type
• export-fields
• export-methods

• is-class
• make-constructor

Details about attributes

The default values for class attributes are defined by the internal RIG-variable *class-defaults*. The default setting says that you want to export all methods of the class, and that a class usually is not abstract:

(define *class-defaults* '((export-fields #f)
                           (export-methods #t)
                           (abstract #f) ))

If you want to have get-functions or set-functions for some or all fields of the class, or if you want to exclude certain methods in the interface, you specify this with attributes in the class declaration.

You may also change the default setting for all class definitions in the interface - see the example in the following parapraph about the struct declaration.

export-methods

Useful values for this attribute are:

• export-methods All methods are exported. This is equivalent to: (export-methods #t)
• (export-methods #f) No method is exported. This is the default for structs.
• (export-methods n1 n2 ...) names explicitly the methods to be exported. This also is the only way to selectively exclude methods. (See the example of the fltk interface - use the auxiliary program xref to list all methods).

export-fields

The attribute export-fields is analogous to the attribute export-methods, but it gives you finer control about the generated code: you not only select the fields, but also the kind of function that is generated: read-access, write-access, or both. Accordingly, the values of this attribute may be:

• #t read/write access
• #f no access
• get read
• set write

Useful ways to write this attribute in a class or struct declaration are:

	export-fields  gives default value #t for all fields
	(export-fields #f) #f for all fields
	(export-fields get) get for all fields
	(export-fields nom1 nom2 ...)  export just the named fields
	(export-fields (nom1 get) nom2 ...) export with fine access control

Default value for class fields: #f.

Print functions: attributes print-function, print-method

In the Ravi system, any C-type has a print function, which can be defined via attributes for the C++ class (or for the C-type). If you don't specify one of these attributes, then an object of the class will be printed as a hexadecimal pointer value.

• (print name) gives the name of a function (defined in your code). This function must have the prototype:

  
   	void fn(ScPortOut * port,void * x)
  

  {Remark: this is not checked by RIG}.

• (print-function name) is synonymous with the preceding.
• (print-method method-name) The print function is generated by RIG, calling the indicated method. The method must have the prototype: char* nom-m();

Example of such a generated print function:

(classe toto (print-method info) ...)
void print_toto(ScPortOut * port,void * x)
{
  port -> PrintFormat(((toto *)x) -> info());
}

Struct interface

The struct declaration follows exactly the same syntaxe as the class declaration; however, the default settings are different, given by the internal RIG-variable *struct-defaults*:

(define *struct-defaults* '((export-fields #t)
                            (export-methods #f)
			    (is-class #f)
                            (make-constructor #t)))

(eval (set! *struct-defaults* '((export-fields #t)
                                (export-methods #t)
			        (is-class #t)
                                (make-constructor #t))))

In the default case, access functions are generated for the fields of the struct. In the shell, these functions are named: structname-fieldname set-structname-fieldname!

The attribute is-class says that the struct is to be handled as a class, i.e. that access functions are generated as methods.

This attribute is written as:

	is-class
	(is-class #f)

Default value: #f.

The attribute make-constructor specifies that a constructor without arguments should be generated. This is useful when there is no constructor for this struct.

Callback functions: calling Scheme functions from C code

This is particularly useful for callback functions in graphical interfaces.

#include <PrimaVision/UBitmapABGR.h>
#include <PrimaVision/UBitmapByte.h>
#include <PrimaVision/UBitmapFloat.h>
#include <PrimaVision/UIntArray.h>
/*Ravi declare ((scheme-function (affiche_pos affiche-pos)
                                 (affiche_byte affiche-byte)
		                 bitmap_display
                                 image_data_display
                                 rectangle_display))  */
void affiche_pos(TBitmapABGR *bmp, int pos, char* lab);
void affiche_byte(TBitmapByte *bmp, int pos, char* label, TIntVector* cmp);
void bitmap_display(TBitmap* bmp,int pos);
void bitmap_display(TBitmapFloat* bmp,int pos,float s=0.0);
void rectangle_display(int x1,int y1,int w,int h,int pos,char* color);
void myTestFunc(TBitmapABGR* bmp,int pos)
{
  affiche_pos(bmp,pos,"TEST IMAGE");
}

Template interface

A C++ template is a code pattern (for a class, or for a procedure), and cannot be interfaced directly (maybe a template could be translated into a higher-order Scheme function?). However, there is no problem in interfacing templates instances. All the principles explained so far apply to template instances.

There is just one important question: which instances do we interface? And a less important question: what names do we give to these in Scheme? The rules are as follows:

• for each wanted template instance there is a typedef definition in a header file. The name defined in this definition is used in Scheme (i.e. it will be the name of the class or function).
• A special RIG-declaration specifies the instances to interface:

  
  	(generate-template-class name1 ...)
  

Remarks - current limitations

• template functions are not handled (easy to do)
• There currently is a flag inhibiting template generation, which should be turned on - so you need a declaration like

  
  /*Ravi declare ((generate-template-class myclass)
                  (eval (set! *gen-templates* #t) ) ) */
  

Memory management

C++ syntax

miscellaneous information: names

Commented example

Declarations concerning the preprocessor

The RIG preprocessor usually is active, but one must be aware that it is NOT IDENTICAL with the compiler preprocessor! In particular:

• it expands preprocessor symbols only on demand.
• it does not use the compiler rules for file search.
• preprocessor symbols are not defined by default.
• except the symbol RAVI_INTERFACE. By adding #ifndef RAVI_INTERFACE, you can render parts of header files invisible to RIG. This should be done for function implementations separate from header declarations, i.e. RIG should not find a header and the implementation for the same function. This happens for inline methods, and for template methods.

Declarations concerning the preprocessor:

• (cpp-macro-expand n1 n2 ...) This declaration specifies which preprocessor symbols should be expanded. Limitation for macros: the concatenation operator ## is not implemented.

Declarations for gurus and for debugging

Sometimes we want to influence the interface generator more than the declarations allow - in particular for troubleshooting and debugging. For these cases, the eval declaration is the universal solution: it executes Scheme code directly; of course, one has to know about RIG's internals.

The declaration

(eval e) evaluates the expression e. This allows to set internal variables, to output debug information, etc. If it turns out that a variable often is used this way, then a new kind of declaration is set up - this is the way declarations evolve.

Example:

(eval
   (begin
     (set! *skip-compounds* #t)
     (set! *trace-port* (open-output-file "test.scm"))
     (preprocessor-init #f)
     ))

Explanations:

• The variable *skip-compounds* suppresses syntax analysis within C-compounds; this does no harm inside RIG.
• When the variable *trace-port* is an output-port, the result of syntax analysis (abstract trees) is written into this port; this may be helpful for debugging.

• The function call (preprocessor-init #f)inhibits the preprocessor: all preprocessor commands in the input will be handled as comments.