dynamically calling DLLs

[Steve A.]

Okay guys,
I have spent days trying to figure this one out.
I have been reworking Bxbasic and adding new features that might take it beyond the scope of a toy.
One big hurdle I've been trying to overcome is interpreter "bloat".
You can't just keep adding new features and still keep the runtime engine relatively compact.
That's one major down-side of interpreters, they just keep getting bigger.

I have been experimenting with the idea of dynamically calling functions in DLLs.
I have a simplified version in place, but, it only works with simple data types: int and char, as parameters.

This is not a new idea, it's been done before.
There are commercial products out there that do this.
I'll use LibertyBasic as an example, with: CALLDLL.

Initially, you might think: "okay, just create templates to handle the data types and go from there".
Not that easy. There are an undetermined number of parameters in various combinations of arrangement.

Has anyone here worked on this subject before and come up with a solution?
Your help will be greatly appreciated and I'm sure, useful to others.

[JRS]

Steve,

You might want to have a peek at ScriptBasic and see how it does it. If you want a standalone executable, you have two options. Translate the user PCode into a C wrapper and link it with libscriba.dll for 12KB of interface between the two. The other option is to piggyback the interpreter to the user program. (450KB of run time overhead) Every time scriba starts, it checks to see if there is a user script attached.

ScriptBasic Windows uses an extension module (DYC) to generically call DLLs from the Basic. Traditionally you would create a shared object interface that tightly integrates with the ScriptBasic API for external library interfacing. SB has a simple DECLARE statement to bind it's extension modules with the scripts.

John

[rdc]

Google LoadLibrary and GetProcAddress.

Example:

http://www.daniweb.com/forums/thread109249.html

[Steve A.]

Hey John,

You might want to have a peek at ScriptBasic and see how it does it. If you want a standalone executable, you have two options.

Nope, Bxbasic can do standalone Exe's.
Not the problem.

ScriptBasic Windows uses an extension module (DYC) to generically call DLLs from the Basic. Traditionally you would create a shared object interface that tightly integrates with the ScriptBasic API for external library interfacing. SB has a simple DECLARE statement to bind it's extension modules with the scripts.

I did look at the online ScriptBasic Docs and saw no mention of "dynamically calling DLL's".
I think I even searched the forum for that topic and found nothing.
I must be wrong.

Okay, what you are saying is, SB can take a previously undefined DLL and Function there-in and execute it, passing random parameters?
And, the source code for doing that is available ?

Edit:
I should add, that I did download SB and began combing thru the source files.
I did not find the algorithm I was looking for there.

Any suggestion on what file I might be looking for ?
Thanks

[Steve A.]

Hey Aurel,

What about using static library as option which can be compiled with
interpreter exe and there is no need for external dll-s.
I think (but maby im wrong....) that on this way is execution much faster then with calling dll-s.

Yes, but that's what I'm trying to get away from.
I don't want to add 1000 DLL functions to Bxbasic.

[Steve A.]

Hey Rick,

Google LoadLibrary and GetProcAddress.

That's covered. I already use them.
The problem is an algorithm to plug the call parameters into the correct places for any given function being called.
Example:
let's say I call MyFunc in MyDLL.DLL,
now MyFunc needs one param:

  <call> "MyDLL.DLL", "MyFunc", (int)
-or-
  <call> "MyDLL.DLL", "MyFunc", (char)

It would be a bit tedious, but, I could build a template to handle this.
Now I want two (2) params:

  <call> "MyDLL.DLL", "MyFunc", (int, char)
-or-
  <call> "MyDLL.DLL", "MyFunc", (char, int)

2 params with (only) 2 types of vars equals 4 templates,
3 params with (only) 2 types of vars equals 8 templates,
............<snip>
by the time you get to 10 params you need 1024 templates.
And, there are possibly 5 to 10 different data types.

That's what I'm talking about.

[Steve A.]

Oups i totaly miss the point.

No problem.
It's not an easy problem for me to explain.
Else where, where I've posted, some people told me I was stupid for even wanting to do that.
And, explained in great detail why it was too much of a security risk and a ton of other reasons.
The DLLs and API are there, on every system.
Why not make them available to the user, I say ?

[JRS]

Okay, what you are saying is, SB can take a previously undefined DLL and Function there-in and execute it, passing random parameters?
And, the source code for doing that is available ?

The DYC SB extension module source is available to use in any way you see fit. The dynacall code is the meat of the interface and should be simple to implement in BxBasic.

Here is an example of using the SB DYC extension module to call the Oxygen Basic JIT compiler.

[rdc]

Hey Rick,

Google LoadLibrary and GetProcAddress.

That's covered. I already use them.
The problem is an algorithm to plug the call parameters into the correct places for any given function being called.

You don't need to create templates. All you need to do is to have a declare statement that the user will write for the dll in question including its parameters. You would then parse the declare statement and treat it like a function call except that you would be calling the dll rather than an internal function. It would be up to the user to correctly Declare the DLL. If they don't it would simple generate a runtime error.

[rdc]

Of course you can always just expose the LoadLibrary, et. all. functions in your interpreter and let the user handle the calling in their program. The only thing you need to supply would be the appropriate sized data types. For example, a null terminated string data type would be needed since many string functions use C-strings.

[Steve A.]

You don't need to create templates.

You are correct, I wouldn't, but that has been suggested (else where).

All you need to do is to have a declare statement that the user will write for the dll in question including its parameters. You would then parse the declare statement and treat it like a function call except that you would be calling the dll rather than an internal function. It would be up to the user to correctly Declare the DLL. If they don't it would simple generate a runtime error.

Have you tried this ?
Remember, this is "dynamically calling a DLL function" at Runtime (exclusively).

Okay, right now, I have the ability to call a minimum of DLL/Functions that require only simple parameters configurations.
Like :  (int,char*,char*,int)
is about as complex as it can get. And that is without templates.

Lets say I'm in Bxbasic and I want to display a MessageBox, here is a Bxbasic listing:

1 ...do stuff
2 ...do stuff
3  CALLAPI "user32.dll", "MessageBoxA", (0, "hello world", "Hello", 0)
4 ...
5 ...
<end>

That I can do, right now.
That is simple.
Here is what a portion of the code looks like to make that happen:

Code: [Select]

#define INTEGER               1
#define STRING                2
#define DOUBLE                3

  int params_type[5] = {INTEGER,STRING,STRING,INTEGER};
  int int_params_values[5];
  char **string_params_values;
  void * params[5]; // = {0, 0, 0, 0, 0};

void * (__stdcall *stdcall_func0)(void);
void * (__stdcall *stdcall_func1)(void *);
void * (__stdcall *stdcall_func2)(void *, void *);
void * (__stdcall *stdcall_func3)(void *, void *, void *);
void * (__stdcall *stdcall_func4)(void *, void *, void *, void *);
void * (__stdcall *stdcall_func5)(void *, void *, void *, void *, void *);

void * (__cdecl *ccall_func0)(void);
void * (__cdecl *ccall_func1)(void *);
void * (__cdecl *ccall_func2)(void *, void *);
void * (__cdecl *ccall_func3)(void *, void *, void *);
void * (__cdecl *ccall_func4)(void *, void *, void *, void *);
void * (__cdecl *ccall_func5)(void *, void *, void *, void *, void *);

    char dllname[100];
    char functionname[100];
     int para_meters;
     int stdcall_type = 1;
     int ccall_type = 2;
     int function_type;

void main()
{   /* ignore omissions */

    strcpy(dllname, "user32.dll");
    strcpy(functionname, "MessageBoxA");
    strcpy(text, "helloworld");
    strcpy(caption, "hello");
    para_meters = 4;
    function_type = stdcall_type;
    
    CALL_API();

    return 0;
}
/*----------------------------*/


void CALL_API()
{
    int i = 0;
    HMODULE dll_handle = 0;
    int int_params_used = 0;
    int string_params_used = 0;


    dll_handle = LoadLibrary(dllname);

    if (!dll_handle)
    {
        printf("Could not load %s", dllname);
        return;
    }

    if (para_meters < 0 || para_meters > 5)
    {
        printf("%d parameters not supported", para_meters);
        FreeLibrary(dll_handle);
        return;
    }

    if (para_meters > 0)
    {
        int_params_used = 0;
        string_params_used = 0;

        for (i = 0; i < para_meters; i++)
        {
            if (params_type[i] == INTEGER)
            {
                params[i] = (void*)int_params_values[int_params_used++];
            }
            else if(params_type[i] == STRING)
            {
                params[i] = (void*)string_params_values[string_params_used++];
            }
        }
    }

    switch(para_meters)
    {
        case 0:
            if (function_type == stdcall_type)
            {
                *(void**)&stdcall_func0 = (void*)GetProcAddress(dll_handle, functionname);

                if (!stdcall_func0)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, call the function!
                    stdcall_func0();
                }
            }
            else if (function_type == ccall_type)
            {
                *(void**)&ccall_func0 = (void*)GetProcAddress(dll_handle, functionname);

                if (!ccall_func0)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, call the function!
                    ccall_func0();
                }
            }
            break;
        case 1:
            if (function_type == stdcall_type)
            {
                *(void**)&stdcall_func1 = (void*)GetProcAddress(dll_handle, functionname);

                if (!stdcall_func1)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, so pass one parameter. No big deal...
                    stdcall_func1(params[0]);
                }
            }
            else if (function_type == ccall_type)
            {
                *(void**)&ccall_func1 = (void*)GetProcAddress(dll_handle, functionname);

                if (!ccall_func1)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, so pass one parameter. No big deal...
                    ccall_func1(params[0]);
                }
            }
            break;
        case 2:
//.........<snip>
        default:
            printf("%d parameters not supported", para_meters);
            break;
    }

    // Free library handle in case the system wants to unload the DLL from memory
    FreeLibrary(dll_handle);

}
/*--------- end ----------*/

It's not so simple.

Edit:
I forgot to mention, this code is not entirely correct, (beside the missing portions).
The entire code does work, but, it does have a breaking point.
From what I understand, the fact that it works is purely accidental and not portable.

[Steve A.]

The DYC SB extension module source is available to use in any way you see fit.
The dynacall code is the meat of the interface and should be simple to implement in BxBasic.

I don't see DYC or anything similar in the SB source files.
I do see "dynlolib" and have examined it.
I don't quite see how that works.
Are you able to explain its operation ?

[Steve A.]

Hey Aurel,

Do you want call windows api functions or call external dll.s ?

Both.
Any and all DLLs and API functions. Windows and any others as well.
If you have a DLL, you can get the functions names and the parameter list.
<call> "DLL", "functionName", (parameterList, , , , )

The problem is not in calling the DLL or the FunctionName.
It is in the parameter list.
They are all different.
I want one, (1), a single method for calling all DLL and API functions.

[AIR]

The DYC SB extension module source is available to use in any way you see fit.
The dynacall code is the meat of the interface and should be simple to implement in BxBasic.

I don't see DYC or anything similar in the SB source files.
I do see "dynlolib" and have examined it.
I don't quite see how that works.
Are you able to explain its operation ?

Look in Extensions->DYC->interface.c

It's an enhancement of what was posted Here.

Be advised that this uses ASM in spots....

A.

[Steve A.]

Hey AIR,

Look in Extensions->DYC->interface.c

It's an enhancement of what was posted Here.

Be advised that this uses ASM in spots....

Okay..., the drdobbs article has some very good documentation on the subject.

Note:
file:     SBSource-3.0-RC2.zip does not contain:   ..\extensions\dyc\interface.c

however,
file:    scriba-v2.0b0-source.zip  does contain:     ..\extensions\dyc\interface.c

This is exactly what I need.
Thanks