AllBASIC Forum

BASIC Developer & Support Resources => Interpreters => Topic started by: Steve A. on October 07, 2010, 08:41:07 AM

Title: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 08:41:07 AM
Okay guys,
I have spent days trying to figure this one out.
I have been reworking Bxbasic and adding new features that might take it beyond the scope of a toy.
One big hurdle I've been trying to overcome is interpreter "bloat".
You can't just keep adding new features and still keep the runtime engine relatively compact.
That's one major down-side of interpreters, they just keep getting bigger.

I have been experimenting with the idea of dynamically calling functions in DLLs.
I have a simplified version in place, but, it only works with simple data types: int and char, as parameters.

This is not a new idea, it's been done before.
There are commercial products out there that do this.
I'll use LibertyBasic as an example, with: CALLDLL.

Initially, you might think: "okay, just create templates to handle the data types and go from there".
Not that easy. There are an undetermined number of parameters in various combinations of arrangement.

Has anyone here worked on this subject before and come up with a solution?
Your help will be greatly appreciated and I'm sure, useful to others.
Title: Re: dynamically calling DLLs
Post by: JRS on October 07, 2010, 10:22:27 AM
Steve,

You might want to have a peek at ScriptBasic and see how it does it. If you want a standalone executable, you have two options. Translate the user PCode into a C wrapper and link it with libscriba.dll for 12KB of interface between the two. The other option is to piggyback the interpreter to the user program. (450KB of run time overhead) Every time scriba starts, it checks to see if there is a user script attached.

ScriptBasic Windows uses an extension module (DYC) to generically call DLLs from the Basic. Traditionally you would create a shared object interface that tightly integrates with the ScriptBasic API for external library interfacing. SB has a simple DECLARE statement to bind it's extension modules with the scripts.

John
Title: Re: dynamically calling DLLs
Post by: rdc on October 07, 2010, 11:02:13 AM
Google LoadLibrary and GetProcAddress.

Example:

http://www.daniweb.com/forums/thread109249.html

Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 11:42:30 AM
Hey John,
You might want to have a peek at ScriptBasic and see how it does it. If you want a standalone executable, you have two options.

Nope, Bxbasic can do standalone Exe's.
Not the problem.

Quote
ScriptBasic Windows uses an extension module (DYC) to generically call DLLs from the Basic. Traditionally you would create a shared object interface that tightly integrates with the ScriptBasic API for external library interfacing. SB has a simple DECLARE statement to bind it's extension modules with the scripts.

I did look at the online ScriptBasic Docs and saw no mention of "dynamically calling DLL's".
I think I even searched the forum for that topic and found nothing.
I must be wrong.

Okay, what you are saying is, SB can take a previously undefined DLL and Function there-in and execute it, passing random parameters?
And, the source code for doing that is available ?

Edit:
I should add, that I did download SB and began combing thru the source files.
I did not find the algorithm I was looking for there.

Any suggestion on what file I might be looking for ?
Thanks
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 11:44:56 AM
Hey Aurel,

What about using static library as option which can be compiled with
interpreter exe and there is no need for external dll-s.
I think (but maby im wrong....) that on this way is execution much faster then with calling dll-s.

Yes, but that's what I'm trying to get away from.
I don't want to add 1000 DLL functions to Bxbasic.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 12:07:54 PM
Hey Rick,

Google LoadLibrary and GetProcAddress.

That's covered. I already use them.
The problem is an algorithm to plug the call parameters into the correct places for any given function being called.
Example:
let's say I call MyFunc in MyDLL.DLL,
now MyFunc needs one param:

  <call> "MyDLL.DLL", "MyFunc", (int)
-or-
  <call> "MyDLL.DLL", "MyFunc", (char)

It would be a bit tedious, but, I could build a template to handle this.
Now I want two (2) params:

  <call> "MyDLL.DLL", "MyFunc", (int, char)
-or-
  <call> "MyDLL.DLL", "MyFunc", (char, int)

2 params with (only) 2 types of vars equals 4 templates,
3 params with (only) 2 types of vars equals 8 templates,
............<snip>
by the time you get to 10 params you need 1024 templates.
And, there are possibly 5 to 10 different data types.

That's what I'm talking about.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 12:34:29 PM
Oups i totaly miss the point.

No problem.
It's not an easy problem for me to explain.
Else where, where I've posted, some people told me I was stupid for even wanting to do that.
And, explained in great detail why it was too much of a security risk and a ton of other reasons.
The DLLs and API are there, on every system.
Why not make them available to the user, I say ?
Title: Re: dynamically calling DLLs
Post by: JRS on October 07, 2010, 01:33:38 PM
Quote from: Steve A.
Okay, what you are saying is, SB can take a previously undefined DLL and Function there-in and execute it, passing random parameters?
And, the source code for doing that is available ?

The DYC (http://www.scriptbasic.org/forum/index.php/topic,105.0.html) SB extension module source is available to use in any way you see fit. The dynacall code is the meat of the interface and should be simple to implement in BxBasic.

Here (http://www.allbasic.info/forum/index.php?topic=13.0) is an example of using the SB DYC extension module to call the Oxygen Basic JIT compiler.

Title: Re: dynamically calling DLLs
Post by: rdc on October 07, 2010, 02:00:54 PM
Hey Rick,

Google LoadLibrary and GetProcAddress.

That's covered. I already use them.
The problem is an algorithm to plug the call parameters into the correct places for any given function being called.

You don't need to create templates. All you need to do is to have a declare statement that the user will write for the dll in question including its parameters. You would then parse the declare statement and treat it like a function call except that you would be calling the dll rather than an internal function. It would be up to the user to correctly Declare the DLL. If they don't it would simple generate a runtime error.
Title: Re: dynamically calling DLLs
Post by: rdc on October 07, 2010, 02:27:00 PM
Of course you can always just expose the LoadLibrary, et. all. functions in your interpreter and let the user handle the calling in their program. The only thing you need to supply would be the appropriate sized data types. For example, a null terminated string data type would be needed since many string functions use C-strings.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 02:58:27 PM
You don't need to create templates.

You are correct, I wouldn't, but that has been suggested (else where).

Quote
All you need to do is to have a declare statement that the user will write for the dll in question including its parameters. You would then parse the declare statement and treat it like a function call except that you would be calling the dll rather than an internal function. It would be up to the user to correctly Declare the DLL. If they don't it would simple generate a runtime error.

Have you tried this ?
Remember, this is "dynamically calling a DLL function" at Runtime (exclusively).

Okay, right now, I have the ability to call a minimum of DLL/Functions that require only simple parameters configurations.
Like :  (int,char*,char*,int)
is about as complex as it can get. And that is without templates.

Lets say I'm in Bxbasic and I want to display a MessageBox, here is a Bxbasic listing:

1 ...do stuff
2 ...do stuff
3  CALLAPI "user32.dll", "MessageBoxA", (0, "hello world", "Hello", 0)
4 ...
5 ...
<end>

That I can do, right now.
That is simple.
Here is what a portion of the code looks like to make that happen:

Code: [Select]
#define INTEGER               1
#define STRING                2
#define DOUBLE                3

  int params_type[5] = {INTEGER,STRING,STRING,INTEGER};
  int int_params_values[5];
  char **string_params_values;
  void * params[5]; // = {0, 0, 0, 0, 0};

void * (__stdcall *stdcall_func0)(void);
void * (__stdcall *stdcall_func1)(void *);
void * (__stdcall *stdcall_func2)(void *, void *);
void * (__stdcall *stdcall_func3)(void *, void *, void *);
void * (__stdcall *stdcall_func4)(void *, void *, void *, void *);
void * (__stdcall *stdcall_func5)(void *, void *, void *, void *, void *);

void * (__cdecl *ccall_func0)(void);
void * (__cdecl *ccall_func1)(void *);
void * (__cdecl *ccall_func2)(void *, void *);
void * (__cdecl *ccall_func3)(void *, void *, void *);
void * (__cdecl *ccall_func4)(void *, void *, void *, void *);
void * (__cdecl *ccall_func5)(void *, void *, void *, void *, void *);

    char dllname[100];
    char functionname[100];
     int para_meters;
     int stdcall_type = 1;
     int ccall_type = 2;
     int function_type;

void main()
{   /* ignore omissions */

    strcpy(dllname, "user32.dll");
    strcpy(functionname, "MessageBoxA");
    strcpy(text, "helloworld");
    strcpy(caption, "hello");
    para_meters = 4;
    function_type = stdcall_type;
    
    CALL_API();

    return 0;
}
/*----------------------------*/


void CALL_API()
{
    int i = 0;
    HMODULE dll_handle = 0;
    int int_params_used = 0;
    int string_params_used = 0;


    dll_handle = LoadLibrary(dllname);

    if (!dll_handle)
    {
        printf("Could not load %s", dllname);
        return;
    }

    if (para_meters < 0 || para_meters > 5)
    {
        printf("%d parameters not supported", para_meters);
        FreeLibrary(dll_handle);
        return;
    }

    if (para_meters > 0)
    {
        int_params_used = 0;
        string_params_used = 0;

        for (i = 0; i < para_meters; i++)
        {
            if (params_type[i] == INTEGER)
            {
                params[i] = (void*)int_params_values[int_params_used++];
            }
            else if(params_type[i] == STRING)
            {
                params[i] = (void*)string_params_values[string_params_used++];
            }
        }
    }

    switch(para_meters)
    {
        case 0:
            if (function_type == stdcall_type)
            {
                *(void**)&stdcall_func0 = (void*)GetProcAddress(dll_handle, functionname);

                if (!stdcall_func0)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, call the function!
                    stdcall_func0();
                }
            }
            else if (function_type == ccall_type)
            {
                *(void**)&ccall_func0 = (void*)GetProcAddress(dll_handle, functionname);

                if (!ccall_func0)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, call the function!
                    ccall_func0();
                }
            }
            break;
        case 1:
            if (function_type == stdcall_type)
            {
                *(void**)&stdcall_func1 = (void*)GetProcAddress(dll_handle, functionname);

                if (!stdcall_func1)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, so pass one parameter. No big deal...
                    stdcall_func1(params[0]);
                }
            }
            else if (function_type == ccall_type)
            {
                *(void**)&ccall_func1 = (void*)GetProcAddress(dll_handle, functionname);

                if (!ccall_func1)
                {
                    printf("Could not load function %s in %s", functionname, dllname);
                }
                else
                {   // OK, so pass one parameter. No big deal...
                    ccall_func1(params[0]);
                }
            }
            break;
        case 2:
//.........<snip>
        default:
            printf("%d parameters not supported", para_meters);
            break;
    }

    // Free library handle in case the system wants to unload the DLL from memory
    FreeLibrary(dll_handle);

}
/*--------- end ----------*/

It's not so simple.

Edit:
I forgot to mention, this code is not entirely correct, (beside the missing portions).
The entire code does work, but, it does have a breaking point.
From what I understand, the fact that it works is purely accidental and not portable.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 03:26:31 PM
The DYC SB extension module source is available to use in any way you see fit.
The dynacall code is the meat of the interface and should be simple to implement in BxBasic.

I don't see DYC or anything similar in the SB source files.
I do see "dynlolib" and have examined it.
I don't quite see how that works.
Are you able to explain its operation ?
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 03:34:13 PM
Hey Aurel,

Do you want call windows api functions or call external dll.s ?

Both.
Any and all DLLs and API functions. Windows and any others as well.
If you have a DLL, you can get the functions names and the parameter list.
<call> "DLL", "functionName", (parameterList, , , , )

The problem is not in calling the DLL or the FunctionName.
It is in the parameter list.
They are all different.
I want one, (1), a single method for calling all DLL and API functions.

Title: Re: dynamically calling DLLs
Post by: AIR on October 07, 2010, 05:22:29 PM
The DYC SB extension module source is available to use in any way you see fit.
The dynacall code is the meat of the interface and should be simple to implement in BxBasic.

I don't see DYC or anything similar in the SB source files.
I do see "dynlolib" and have examined it.
I don't quite see how that works.
Are you able to explain its operation ?


Look in Extensions->DYC->interface.c

It's an enhancement of what was posted Here (http://www.drdobbs.com/184416502).

Be advised that this uses ASM in spots....

A.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 07, 2010, 06:30:55 PM
Hey AIR,

Look in Extensions->DYC->interface.c

It's an enhancement of what was posted Here (http://www.drdobbs.com/184416502).

Be advised that this uses ASM in spots....

Okay..., the drdobbs article has some very good documentation on the subject.

Note:
file:     SBSource-3.0-RC2.zip does not contain:   ..\extensions\dyc\interface.c

however,
file:    scriba-v2.0b0-source.zip  does contain:     ..\extensions\dyc\interface.c

This is exactly what I need.
Thanks
Title: Re: dynamically calling DLLs
Post by: JRS on October 07, 2010, 08:56:07 PM
Here is everything you should need to see if this makes sense for BxBasic.

Zip contains the ScriptBasic 2.1 version of the DYC extension module.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 08, 2010, 06:01:51 AM
Here is everything you should need to see if this makes sense for BxBasic.

Thanks John,
I think you've got it.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 09, 2010, 08:02:42 AM
Hey guys,

Update:

okay..., I've been able to determine that at least some of it has to be done in assembly language.
I've looked at the source code that John provided and the sources for "DynaCall", written by Ton Plooy.
It looks like I have two options,
1) I can use inline assembly, embedded in C, or
2) write a module entirely in assembly and compile it into an ".obj" file and "link" it to Bxbasic.

Why Assembly Language, you may ask ?
Well, it has to do with passing parameters to arbitrary functions.
You never know ahead of time how many parameters nor their type(s).
Recall the illustration I made with all the templates.
If you used templates (an obsurd idea) it can be done entirely in C.

You can't simply say: <call>   MessageBoxA(hwnd, "helloworld", "Hello", uType)
                        or: <call>   toupper(mychar)
                        or: <call>   exp(MyDouble)

for calling external functions and expect it to work.

It doesn't.
Remember, we are not talking about a compiler here.
When you compile those statements, the compiler, linker and loader handle all the details for you.
We are talking about calling and executing those DLLs and Functions "on the fly".

The root problem is pushing an arbitrary number of arbitrary types of parameters onto the stack.
C, by itself, has no direct control of the stack.
Their isn't a C method for doing a "push" or "pop".
That is the problem.

The solution is to manually parse the parameters, push them onto the stack and then call the DLL or API function.

I'd like to hear any input, ideas or thoughts you may have on this.
Steve
Title: Re: dynamically calling DLLs
Post by: jcfuller on October 09, 2010, 02:17:57 PM
Why can't you use varargs (...) ? of all dword or long for parameters.

All the protos I use with so/dll in jwasm are all dword

James
Title: Re: dynamically calling DLLs
Post by: Pjot on October 10, 2010, 02:28:29 AM
Hi,

So I missed this discussion...

With GTK-server I have struggled with the exact same problem. There are several external libraries which may help you out:


When I was using these libraries I did not realize the standard Unix 'dlopen' in C can take care of the same functionality. So for Unix, 'dlopen' in <dlfcn.h> is sufficient; in BaCon I am doing the functionality you are looking for, based on this Unix 'dlopen' API. It allows specifying the return type for each function. Furthermore specifying the prototype forces the types for arguments. For example:

Code: [Select]
long (*mysql_init) (long);
void (*mysql_free_result) (long);
char *(*mysql_error) (long);

pointer_mysqlLIB = dlopen("libmysqlclient.so", RTLD_LAZY);

*(long **) (&mysql_init) = dlsym(pointer_mysqlLIBb, "mysql_init");
*(void **) (&mysql_free_result) = dlsym(pointer_mysqlLIB, "mysql_free_result");
*(char ***) (&mysql_error) = dlsym(pointer_mysqlLIB, "mysql_error");

Maybe the Win32 LoadLibrary API call also allows the same trick?

Regards
Peter
Title: Re: dynamically calling DLLs
Post by: JRS on October 10, 2010, 08:16:16 AM
Quote
You can't simply say: <call>   MessageBoxA(hwnd, "helloworld", "Hello", uType)
What you mean by this?
of course that you can i dont understand why you mean that is not posibile?

BxBasic is an interpreter that was written by Steve and this thread is an attempt to help him with calling Windows API functions (DLLs) from a user script/program. Not all Basic languages have external API access support by default at the user level.

Title: Re: dynamically calling DLLs
Post by: Steve A. on October 10, 2010, 04:12:57 PM
Why can't you use varargs (...) ? of all dword or long for parameters.
All the protos I use with so/dll in jwasm are all dword

That sounds interesting. I've never used varargs. It that the same as va_args ?
If so, how do I pass a double ?

From what I read, it looks like you still need to use a template or boiler-plate, (the docs call it a "list"), in order to get the correct var types in the right order.
How would I use that in an exclusively runtime environment ?
By that I mean:
1) I have no idea at compile time what function will be called
2) nor do I know how many parameters or their types will be passed
3) nor their order.

Can you give an example ?

Steve
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 10, 2010, 04:36:44 PM
With GTK-server I have struggled with the exact same problem. There are several external libraries which may help you out:

  • Foreign Function Call Library FFCALL (http://www.haible.de/bruno/packages-ffcall.html)
  • C/Invoke Library CINV (http://www.nongnu.org/cinvoke/)
  • Foreign Function Interface Library FFI (http://sourceware.org/libffi/)
  • DynCall library DYNCALL (http://www.dyncall.org/)

Okay, I'll have to check those out.

Quote
When I was using these libraries I did not realize the standard Unix 'dlopen' in C can take care of the same functionality. So for Unix, 'dlopen' in <dlfcn.h> is sufficient; in BaCon I am doing the functionality you are looking for, based on this Unix 'dlopen' API. It allows specifying the return type for each function. Furthermore specifying the prototype forces the types for arguments. For example:

But, BaCon is a translator, isn't it ?
The BASIC statements ultimately get compiled ?
Or is there an interpreter involved there as well ?

Quote
Code: [Select]
long (*mysql_init) (long);
void (*mysql_free_result) (long);
char *(*mysql_error) (long);

pointer_mysqlLIB = dlopen("libmysqlclient.so", RTLD_LAZY);

*(long **) (&mysql_init) = dlsym(pointer_mysqlLIBb, "mysql_init");
*(void **) (&mysql_free_result) = dlsym(pointer_mysqlLIB, "mysql_free_result");
*(char ***) (&mysql_error) = dlsym(pointer_mysqlLIB, "mysql_error");

Your function pointer setup looks similar to what my (void *) pointer set up is doing:

void * (__stdcall *stdcall_func0)(void);
void * (__stdcall *stdcall_func1)(void *);
void * (__stdcall *stdcall_func2)(void *, void *);

Quote
Maybe the Win32 LoadLibrary API call also allows the same trick?

Well, LoadLibrary does exctly that, it loads a library (a DLL).
Once you have determined the DLL name, you pass that to LoadLibrary as a parameter:
ie:     my_dll_name = user32.dll

 dll_handle = LoadLibrary(my_dll_name);

that returns a handle to the DLL.
That's all.
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 10, 2010, 04:39:01 PM
Quote
You can't simply say: <call>   MessageBoxA(hwnd, "helloworld", "Hello", uType)
What you mean by this?
of course that you can i dont understand why you mean that is not posibile?

Can you give me an example ?

Title: Re: dynamically calling DLLs
Post by: JRS on October 10, 2010, 07:05:35 PM
Quote
But, BaCon is a translator, isn't it ?
The BASIC statements ultimately get compiled ?
Or is there an interpreter involved there as well ?

Steve,

Peter was talking about Gtk-Server not BaCon. Gtk-Server loads Gtk needed DLL/SO dynamically depending on how the user configuration file is defined.

Title: Re: dynamically calling DLLs
Post by: AIR on October 10, 2010, 07:41:59 PM
I think he was speaking "bacon", John.  ;D

dlopen/dlsym is like LoadLibrary/GetProcAddress.  The problem isn't dynamically loading/unloading as you would in a compiled language, it's being able to call dll functions from an interpreted language "on the fly".

Another way of looking at this question/problem is how would one implement "on the fly" loading of arbitrary functions from an unknown (at compile time) library in *userspace* using a compiled language, after the program has been compiled?

If one can solve that, then doing it with an interpreter would be easy, no?

DYC/Dynacall is one way....but it's Windows specific ATM...

A.
Title: Re: dynamically calling DLLs
Post by: JRS on October 10, 2010, 08:00:58 PM
Quote from: AIR
I think he was speaking "bacon", John.

Quote from: Peter
With GTK-server I have struggled with the exact same problem. There are several external libraries which may help you out:

I'm confused then. (wouldn't be the first time)
Title: Re: dynamically calling DLLs
Post by: AIR on October 10, 2010, 08:13:31 PM
read the next paragraph.  :P
Title: Re: dynamically calling DLLs
Post by: Pjot on October 11, 2010, 12:19:48 AM
Quote
But, BaCon is a translator, isn't it ?
The BASIC statements ultimately get compiled ?
Or is there an interpreter involved there as well ?

Ooops you are right there. No interpreter involved indeed - so this was a stupid suggestion, my bad. Too much beer ;)

So then I guess you have to stick with those external libraries I mentioned. For GTK-server I have created a Windows version of FFI using MinGW, you can get it from here (http://www.gtk-server.org/shared/libffi-mingw.zip).

Regards
Peter
Title: Re: dynamically calling DLLs
Post by: jcfuller on October 11, 2010, 03:44:05 AM
Why can't you use varargs (...) ? of all dword or long for parameters.
All the protos I use with so/dll in jwasm are all dword

That sounds interesting. I've never used varargs. It that the same as va_args ?
If so, how do I pass a double ?

From what I read, it looks like you still need to use a template or boiler-plate, (the docs call it a "list"), in order to get the correct var types in the right order.
How would I use that in an exclusively runtime environment ?
By that I mean:
1) I have no idea at compile time what function will be called
2) nor do I know how many parameters or their types will be passed
3) nor their order.

Can you give an example ?

Steve


Sorry Steve I know nothing of interpreters and am still a novice c coder. It just popped into my mind that va_args might be useful when not knowing ahead of time the number of parameters.

James
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 11, 2010, 06:56:00 AM
Sorry Steve I know nothing of interpreters and am still a novice c coder. It just popped into my mind that va_args might be useful when not knowing ahead of time the number of parameters.

Not a problem James.
I had seen something last week, while google-ing this subject, that also mentioned va_args.
I'm open for any and all suggestions, in case I miss something obvious (or hidden).
There's not a lot written, on the web, about this subject.

So far, John, AIR and Peter (Pjot) have given me more information than all last weeks google searches.

Steve
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 11, 2010, 07:03:54 AM
So then I guess you have to stick with those external libraries I mentioned. For GTK-server I have created a Windows version of FFI using MinGW, you can get it from here.

Thanks for the link Peter.
I downloaded it and I'll have a look.

And, thanks a lot for those other 4 links.
Wow, those never came up in any of my searches last week.
From what I read, all 4 seem to be right on target, as well.

Steve
Title: Re: dynamically calling DLLs
Post by: Pjot on October 11, 2010, 10:53:38 PM
Quote
From what I read, all 4 seem to be right on target, as well.

OK. In GTK-server (http://www.gtk-server.org/) I have used all four of them. The source is a little bit messy, but in there you can see how to apply the different API's, if you need an example. From my experience FFI is the most versatile library, and it is almost always available on Linux/Unix and BSD platforms (including MacOSX).

BR,
Peter
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 21, 2010, 10:26:19 AM
Okay, it's time to post some results, here.
First, I want to thank everyone who participated in this discussion, because for me this was a real learning experience.
I took to heart and mind every bit of input everyone had to contribute.
I knew what I wanted to do, I just didn't know how to go about doing it, successfully.
As I had illustrated previously, I did have a working model that just didn't fit most situations and it was mostly a hack anyway.

I downloaded the sources from the links provided by John, AIR and Peter.
I went through the documentation provided and studied the source code for each.
In all cases, a portion of the work of dynamically calling DLL functions at run-time is done using inline assembly language.
That is an important point to make note of.

The reason for that, is that at run-time, when calling a DLL function "on the fly", you have to bypass the regular system call and directly manipulate the STACK.
By that I mean you have to manually "PUSH" the parameters, that the function will be expecting, onto the stack before you call the DLL function.

Now with that said, the DLL tool I found to my likeing was Dyncall.
Not that there was anything wrong with the others, Dyncall just seemed more "turn-key", if you get my meaning.
Now, the documentation (if they had any) for all of these tools was among the worst.
I'm big on "good" documentation.
After exchanging a dozen or so emails with the developers of Dyncall, I finally got it working.
Unfortunately, their lengthy documentation doesn't actually show you how to use it.
It spends more time on describing all the systems they have tested it on and technical info about the build process.

Anyway...,
I have an example here that illustrates, using C source code, just how simple Dyncall makes the process of calling DLL functions:

Code: [Select]
/* This routine dynamically calls MessageBox */
/* (or any API function) at run-time.        */

/* LINK with these LIBs: libdyncall_s.lib, libdynload_s.lib */


#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <dyncall.h>
#include <dynload.h>


  char dllname[32];
  char functionname[32];


int main()
{   int xhwnd = 0, xtype = 1, retVal;
    char text[20] = {"hello world"};
    char capt[10] = {"hello"};
    void* funcptr;
    void* libhandle;

            /* name of DLL to open */
    strcpy(dllname, "user32.dll");

            /* name of function to call */
    strcpy(functionname, "MessageBoxA");

            /* open DLL */
    libhandle = dlLoadLibrary(dllname);
            /* here: check for NULL return: failure */

            /* locate function */
    funcptr = dlFindSymbol(libhandle,functionname);
            /* here: check for NULL return: failure */

            /* create a callvm (to be shared within your app) */
    DCCallVM * pvm = dcNewCallVM(4096);

            /* configure the callvm for call conv..*/
    dcMode(pvm, DC_CALL_C_X86_WIN32_STD);

            /* reset the argument buffer */
    dcReset(pvm);

            /* now load arguments (from left to right order) */
    dcArgInt(pvm, xhwnd);
    dcArgPointer(pvm, text);
    dcArgPointer(pvm, capt);
    dcArgInt(pvm, xtype);

            /* now call by specifying return type */
    retVal = dcCallInt(pvm,funcptr);

            /* free library handle */
    dlFreeLibrary(libhandle);

            /* free VM */
    dcFree(pvm);

    system("pause");

    return 0;
}
/*--------- end main ----------*/

I have attached a Windows executable, called shelldcall.zip for the above code.

Now, that's not the end of it.
I have done a considerable amount of experimentation on my own, as well.
I first experimented using Masm, doing the whole thing in straight assembly language and it worked quite well.
And, much to my surprise, was quite simple to accomplish.

But, since Bxbasic is written in C, using the LccWin32 compiler, I wanted to be able to do this using "inline" assembly language.
For those not familiar with the concept, "inline assembly" is assembly language embeded into the C source code and compiled with the main program.
To illustrate this, I have as an example, the exact same program as the above, written in C, using "inline assembly":

Code: [Select]
#include <windows.h>
#include <stdio.h>
#include <stdlib.h>


   void * (__stdcall *stdcall_func)();

    char dllname[20]="user32.dll";
    char functionname[20]="MessageBoxA";

    HWND hwnd=0;
    UINT utype=1;
    char text[] = "Hello Windows!";
    char strng[] = "Winasm";
    HMODULE hLib=0;


int main()
{
        _asm("pushl  $_dllname");
        _asm("call   %LoadLibraryA");
        _asm("movl   %eax, _hLib");

        _asm("pushl  $_functionname");
        _asm("pushl  %hLib");
        _asm("call   %GetProcAddress");
        _asm("movl   %eax, _stdcall_func");

        _asm("pushl  _utype");
        _asm("pushl  $_strng");
        _asm("pushl  $_text");
        _asm("pushl  _hwnd");
        _asm("call   *_stdcall_func");


    FreeLibrary(hLib);

    system("pause");
    return 0;
}
/*--------- end main ----------*/

Sweet !

I have attached a Windows executable, called shellacall.zip for the above code.


I presently have Bxbasic setup for Dyncall and have been testing it and it works quite well.
Shortly, I will fully develop my own inline version and will replace Dyncall with it.

If anyone has any questions, I will be only too happy to answer them.
I couldn't have done this without your help.

Thanks
Steve
Title: Re: dynamically calling DLLs
Post by: JRS on October 21, 2010, 10:44:23 AM
That's really good news Steve that you got this working to your satisfaction.

I think this will help others as well. (QB64, ...)

Title: Re: dynamically calling DLLs
Post by: Pjot on October 21, 2010, 11:33:17 AM
Quote
Shortly, I will fully develop my own inline version and will replace Dyncall with it.

That's really brave! The inline ASM code look like a complete nightmare to me...  :-\

But good news that it works, and presumably it also will for Unix-based platforms, as Dyncall is a multiplatform library!

Regards
Peter
Title: Re: dynamically calling DLLs
Post by: Steve A. on October 21, 2010, 02:23:38 PM
That's really brave! The inline ASM code look like a complete nightmare to me...  :-\

But good news that it works, and presumably it also will for Unix-based platforms, as Dyncall is a multiplatform library!

Yes, multiplatform is an important consideration in what I do to Bxbasic these days.
As to the ASM code (nightmare)..., yes and no.

I have to admit when I asked Jacob Navia (developer of LccWin32) for assistance in explaining Lcc's assembly code, he tried to discourage me from using it. Not that there is any problem in using inline asm code, it's just that Lcc's inline code is un-documented. Nowhere in the LccWin32 docs does it explain it's usage. Fortunately, John Findlay, (practically famous for his Lcc tutorials) stepped up and explained what I needed to know about Lcc inline.

I'm quite familiar with assembly language. You could say it's my "first language", when it comes to programming languages.
Additionally, Lcc's inline assembly is based on what is called "the AT&T syntax".
Even though the code is being run on an x86 CPU, it's not Intel style code. It's a style of syntax originally developed at AT&T Bell Labs for Unix machines. Hence, most Linux compilers also use AT&T syntax, (multiplatform).