dynamically calling DLLs

[JRS]

Here is everything you should need to see if this makes sense for BxBasic.

Zip contains the ScriptBasic 2.1 version of the DYC extension module.

[Steve A.]

Here is everything you should need to see if this makes sense for BxBasic.

Thanks John,
I think you've got it.

[Steve A.]

Hey guys,

Update:

okay..., I've been able to determine that at least some of it has to be done in assembly language.
I've looked at the source code that John provided and the sources for "DynaCall", written by Ton Plooy.
It looks like I have two options,
1) I can use inline assembly, embedded in C, or
2) write a module entirely in assembly and compile it into an ".obj" file and "link" it to Bxbasic.

Why Assembly Language, you may ask ?
Well, it has to do with passing parameters to arbitrary functions.
You never know ahead of time how many parameters nor their type(s).
Recall the illustration I made with all the templates.
If you used templates (an obsurd idea) it can be done entirely in C.

You can't simply say: <call>   MessageBoxA(hwnd, "helloworld", "Hello", uType)
                        or: <call>   toupper(mychar)
                        or: <call>   exp(MyDouble)

for calling external functions and expect it to work.

It doesn't.
Remember, we are not talking about a compiler here.
When you compile those statements, the compiler, linker and loader handle all the details for you.
We are talking about calling and executing those DLLs and Functions "on the fly".

The root problem is pushing an arbitrary number of arbitrary types of parameters onto the stack.
C, by itself, has no direct control of the stack.
Their isn't a C method for doing a "push" or "pop".
That is the problem.

The solution is to manually parse the parameters, push them onto the stack and then call the DLL or API function.

I'd like to hear any input, ideas or thoughts you may have on this.
Steve

[jcfuller]

Why can't you use varargs (...) ? of all dword or long for parameters.

All the protos I use with so/dll in jwasm are all dword

James

[Pjot]

Hi,

So I missed this discussion...

With GTK-server I have struggled with the exact same problem. There are several external libraries which may help you out:

• Foreign Function Call Library FFCALL
• C/Invoke Library CINV
• Foreign Function Interface Library FFI
• DynCall library DYNCALL

When I was using these libraries I did not realize the standard Unix 'dlopen' in C can take care of the same functionality. So for Unix, 'dlopen' in <dlfcn.h> is sufficient; in BaCon I am doing the functionality you are looking for, based on this Unix 'dlopen' API. It allows specifying the return type for each function. Furthermore specifying the prototype forces the types for arguments. For example:

Code: [Select]

long (*mysql_init) (long);
void (*mysql_free_result) (long);
char *(*mysql_error) (long);

pointer_mysqlLIB = dlopen("libmysqlclient.so", RTLD_LAZY);

*(long **) (&mysql_init) = dlsym(pointer_mysqlLIBb, "mysql_init");
*(void **) (&mysql_free_result) = dlsym(pointer_mysqlLIB, "mysql_free_result");
*(char ***) (&mysql_error) = dlsym(pointer_mysqlLIB, "mysql_error");

Maybe the Win32 LoadLibrary API call also allows the same trick?

Regards
Peter

[JRS]

You can't simply say: <call>   MessageBoxA(hwnd, "helloworld", "Hello", uType)

What you mean by this?
of course that you can i dont understand why you mean that is not posibile?

BxBasic is an interpreter that was written by Steve and this thread is an attempt to help him with calling Windows API functions (DLLs) from a user script/program. Not all Basic languages have external API access support by default at the user level.

[Steve A.]

Why can't you use varargs (...) ? of all dword or long for parameters.
All the protos I use with so/dll in jwasm are all dword

That sounds interesting. I've never used varargs. It that the same as va_args ?
If so, how do I pass a double ?

From what I read, it looks like you still need to use a template or boiler-plate, (the docs call it a "list"), in order to get the correct var types in the right order.
How would I use that in an exclusively runtime environment ?
By that I mean:
1) I have no idea at compile time what function will be called
2) nor do I know how many parameters or their types will be passed
3) nor their order.

Can you give an example ?

Steve

[Steve A.]

With GTK-server I have struggled with the exact same problem. There are several external libraries which may help you out:

• Foreign Function Call Library FFCALL
• C/Invoke Library CINV
• Foreign Function Interface Library FFI
• DynCall library DYNCALL

Okay, I'll have to check those out.

When I was using these libraries I did not realize the standard Unix 'dlopen' in C can take care of the same functionality. So for Unix, 'dlopen' in <dlfcn.h> is sufficient; in BaCon I am doing the functionality you are looking for, based on this Unix 'dlopen' API. It allows specifying the return type for each function. Furthermore specifying the prototype forces the types for arguments. For example:

But, BaCon is a translator, isn't it ?
The BASIC statements ultimately get compiled ?
Or is there an interpreter involved there as well ?

Code: [Select]

long (*mysql_init) (long);
void (*mysql_free_result) (long);
char *(*mysql_error) (long);

pointer_mysqlLIB = dlopen("libmysqlclient.so", RTLD_LAZY);

*(long **) (&mysql_init) = dlsym(pointer_mysqlLIBb, "mysql_init");
*(void **) (&mysql_free_result) = dlsym(pointer_mysqlLIB, "mysql_free_result");
*(char ***) (&mysql_error) = dlsym(pointer_mysqlLIB, "mysql_error");


Your function pointer setup looks similar to what my (void *) pointer set up is doing:

void * (__stdcall *stdcall_func0)(void);
void * (__stdcall *stdcall_func1)(void *);
void * (__stdcall *stdcall_func2)(void *, void *);

Maybe the Win32 LoadLibrary API call also allows the same trick?

Well, LoadLibrary does exctly that, it loads a library (a DLL).
Once you have determined the DLL name, you pass that to LoadLibrary as a parameter:
ie:     my_dll_name = user32.dll

 dll_handle = LoadLibrary(my_dll_name);

that returns a handle to the DLL.
That's all.

[Steve A.]

You can't simply say: <call>   MessageBoxA(hwnd, "helloworld", "Hello", uType)

What you mean by this?
of course that you can i dont understand why you mean that is not posibile?

Can you give me an example ?

[JRS]

But, BaCon is a translator, isn't it ?
The BASIC statements ultimately get compiled ?
Or is there an interpreter involved there as well ?

Steve,

Peter was talking about Gtk-Server not BaCon. Gtk-Server loads Gtk needed DLL/SO dynamically depending on how the user configuration file is defined.

[AIR]

I think he was speaking "bacon", John. 

dlopen/dlsym is like LoadLibrary/GetProcAddress.  The problem isn't dynamically loading/unloading as you would in a compiled language, it's being able to call dll functions from an interpreted language "on the fly".

Another way of looking at this question/problem is how would one implement "on the fly" loading of arbitrary functions from an unknown (at compile time) library in *userspace* using a compiled language, after the program has been compiled?

If one can solve that, then doing it with an interpreter would be easy, no?

DYC/Dynacall is one way....but it's Windows specific ATM...

A.

[JRS]

I think he was speaking "bacon", John.

With GTK-server I have struggled with the exact same problem. There are several external libraries which may help you out:

I'm confused then. (wouldn't be the first time)

[AIR]

read the next paragraph. 

[Pjot]

But, BaCon is a translator, isn't it ?
The BASIC statements ultimately get compiled ?
Or is there an interpreter involved there as well ?

Ooops you are right there. No interpreter involved indeed - so this was a stupid suggestion, my bad. Too much beer

So then I guess you have to stick with those external libraries I mentioned. For GTK-server I have created a Windows version of FFI using MinGW, you can get it from here.

Regards
Peter

[jcfuller]

Why can't you use varargs (...) ? of all dword or long for parameters.
All the protos I use with so/dll in jwasm are all dword

That sounds interesting. I've never used varargs. It that the same as va_args ?
If so, how do I pass a double ?

From what I read, it looks like you still need to use a template or boiler-plate, (the docs call it a "list"), in order to get the correct var types in the right order.
How would I use that in an exclusively runtime environment ?
By that I mean:
1) I have no idea at compile time what function will be called
2) nor do I know how many parameters or their types will be passed
3) nor their order.

Can you give an example ?

Steve

Sorry Steve I know nothing of interpreters and am still a novice c coder. It just popped into my mind that va_args might be useful when not knowing ahead of time the number of parameters.

James