Author Topic: cURL Windows  (Read 666 times)

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
cURL Windows
« on: August 15, 2021, 01:26:33 pm »
I spent more time debugging the URL issue than writing the script.  :(

If I ever find the time, I'm going to replace JOKER with MATCH. A side affect of English not being Peter's native language.

« Last Edit: August 15, 2021, 01:35:16 pm by John »

Offline AIR

  • BASIC Developer
  • Posts: 817
Re: cURL Windows
« Reply #1 on: August 15, 2021, 06:59:52 pm »
You might want to alias it instead so you don't break existing/old code....

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #2 on: August 15, 2021, 08:50:52 pm »
Good point AIR. An alias makes a lot of sense.

It looks like SB can't OPEN a socket connection for a HTTPS URL even using 443 as the port.
« Last Edit: August 15, 2021, 08:55:53 pm by John »

Offline AIR

  • BASIC Developer
  • Posts: 817
Re: cURL Windows
« Reply #3 on: August 15, 2021, 09:54:21 pm »
If I remember, SB sockets don't use ssl, so no TLS support.  That would have to be added for this to work.


Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #4 on: August 15, 2021, 11:21:05 pm »
Thanks AIR.

Your code challenge has created list of issues not working with SB.

  • HTTPS not working with Windows cURL extension
  • SB sockets only support HTTP
  • JOKER needs to retire

Offline AIR

  • BASIC Developer
  • Posts: 817
Re: cURL Windows
« Reply #5 on: August 16, 2021, 12:26:15 pm »
Are you using MinGW to build on Windows (I don't remember)?

If so, I think if you add these two flags to your CFLAGS for the Curl module, it should enable the Windows native ssl support, eliminating the need for OpenSSL:

-DUSE_WINDOWS_SSPI
-DUSE_SCHANNEL

You might also need to change the init call, not sure.  Check the Curl documentation....

AIR.

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #6 on: August 16, 2021, 12:47:47 pm »
I'm using TDM-GCC-32 latest 9.2 I think. I'm using your MAKE (slightly modified) for SB you created. I now create two SB versions. SBC for console and SBW for GUI based programs. Both support Windows styles and themes.

Getting cURL HTTPS to work on Windows has been a challenge.

I will give your suggestions a try and see if I can get it going.


Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #7 on: August 16, 2021, 01:10:45 pm »
Here is the make file I have been using. I just downloaded the cURL source but lost how to get this compiled. Any direction or guidance would be appreciated.

Code: C
  1. all : bin\mod\lib\curl.a bin\mod\dll\curl.dll
  2.  
  3.  
  4. bin\mod\lib\curl.a : bin\mod\obj\curl\s_interface.o
  5.         ar -r ..\..\bin\mod\lib\curl.a ..\..\bin\mod\obj\curl\s_interface.o
  6.  
  7. bin\mod\dll\curl.dll : bin\mod\obj\curl\interface.o
  8.         gcc -Wno-write-strings -static-libgcc -pipe -shared -w -s -O2 -m32 -Wl,--add-stdcall-alias,--output-def,"..\..\bin\mod\dll\curl_imp.def",--out-implib,"..\..\bin\mod\dll\curl_imp.a" -s -o ..\..\bin\mod\dll\curl.dll ..\..\bin\mod\obj\curl\interface.o -lm -lpthread -lcurl
  9.  
  10. bin\mod\obj\curl\interface.o : interface.c
  11.         gcc -w -D_WIN32 -m32 -c -o ..\..\bin\mod\obj\curl\interface.o interface.c
  12.  
  13. bin\mod\obj\curl\s_interface.o : interface.c
  14.         gcc -w -D_WIN32 -DSTATIC_LINK=1 -m32 -c -o ..\..\bin\mod\obj\curl\s_interface.o interface.c
  15.  
  16.  

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #8 on: August 16, 2021, 02:08:24 pm »
I was able to get HTTPS working on Windows using these old files. I have no idea where the source is.  The LIBCURL.DLL is in the bin directory.

Your RSS code challenge on Windows works with these files.

I'm going to download the binary form (DLL) of LIBCURL and compile against it. I think that is how what works was built. What I have must be trying to create a standalone extension module.



« Last Edit: August 16, 2021, 06:00:01 pm by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #9 on: August 16, 2021, 10:30:01 pm »
AIR,

I got everything setup with the latest libcurl libraries and include. This is the errors I'm getting trying to build the extension module. I have seen this before but I forget what you had me do to fix it.

Code: Bash
  1. C:\sbgcc\source\extensions\curl>mingw32-make -B -f Makefile
  2. gcc -w -D_WIN32 -DSTATIC_LINK=1 -m32 -c -o ..\..\bin\mod\obj\curl\s_interface.o interface.c
  3. In file included from interface.c:27:
  4. ../../basext.h:837:41: error: static declaration of '_init' follows non-static declaration
  5.   837 | #define besSUB_PROCESS_START static int _init(){
  6.       |                                         ^~~~~
  7. interface.c:2214:1: note: in expansion of macro 'besSUB_PROCESS_START'
  8.  2214 | besSUB_PROCESS_START
  9.       | ^~~~~~~~~~~~~~~~~~~~
  10. ../../basext.h:978:7: note: previous declaration of '_init' was here
  11.   978 |   int _init(void);\
  12.       |       ^~~~~
  13. interface.c:2212:1: note: in expansion of macro 'besDLL_MAIN'
  14.  2212 | besDLL_MAIN
  15.       | ^~~~~~~~~~~
  16. ../../basext.h:843:42: error: static declaration of '_fini' follows non-static declaration
  17.   843 | #define besSUB_PROCESS_FINISH static int _fini(){
  18.       |                                          ^~~~~
  19. interface.c:2219:1: note: in expansion of macro 'besSUB_PROCESS_FINISH'
  20.  2219 | besSUB_PROCESS_FINISH
  21.       | ^~~~~~~~~~~~~~~~~~~~~
  22. ../../basext.h:979:7: note: previous declaration of '_fini' was here
  23.   979 |   int _fini(void);\
  24.       |       ^~~~~
  25. interface.c:2212:1: note: in expansion of macro 'besDLL_MAIN'
  26.  2212 | besDLL_MAIN
  27.       | ^~~~~~~~~~~
  28. Makefile:14: recipe for target 'bin\mod\obj\curl\s_interface.o' failed
  29. mingw32-make: *** [bin\mod\obj\curl\s_interface.o] Error 1
  30.  
  31. C:\sbgcc\source\extensions\curl>
  32.  
« Last Edit: August 16, 2021, 10:31:32 pm by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #10 on: August 16, 2021, 11:54:46 pm »
I got it to compile remarking out the following.

Code: C
  1. besDLL_MAIN
  2.  
  3. besSUB_PROCESS_START
  4.   INIT_MULTITHREAD
  5.   return 1;
  6. besEND
  7.  
  8. besSUB_PROCESS_FINISH
  9.   FINISH_MULTITHREAD
  10. besEND
  11.  

Compile
Code: Bash
  1. C:\sbgcc\source\extensions\curl>mingw32-make -B -f Makefile
  2. gcc -w -D_WIN32 -DSTATIC_LINK=1 -m32 -c -o ..\..\bin\mod\obj\curl\s_interface.o interface.c
  3. ar -r ..\..\bin\mod\lib\curl.a ..\..\bin\mod\obj\curl\s_interface.o
  4. gcc -w -D_WIN32 -m32 -c -o ..\..\bin\mod\obj\curl\interface.o interface.c
  5. gcc -Wno-write-strings -static-libgcc -pipe -shared -w -s -O2 -m32 -Wl,--add-stdcall-alias,--output-def,"..\..\bin\mod\dll\curl_imp.def",--out-implib,"..\..\bin\mod\dll\curl_imp.a" -s -o ..\..\bin\mod\dll\curl.dll ..\..\bin\mod\obj\curl\interface.o -lm -lpthread -lcurl
  6.  
  7. C:\sbgcc\source\extensions\curl>
  8.  

When I try to run the RSS challenge I get the following extension error I had with the old curl extension module I compiled. This error code isn't defined in the source. The error happens executing the curl::perform() function. Since it works with http URLs it has to be something unique to this version of libcurl.dll and SSL.


(0): error 0x0008113c:Extension specific error: %s


On a positive note the new curl extension module works with http URLs.

These gcc arguments didn't help.

-DUSE_WINDOWS_SSPI
-DUSE_SCHANNEL

I replace the new libcurl.dll that came in the distribution with the one I posted earlier and it works.  Version: 7.50.1.0

The new version is 7.78.0.0 and fails.

I compiled the new curl.dll extension module using the new includes and libcurl.a.

The new curl.exe works with the RSS challenge HTTPS URL.
« Last Edit: August 17, 2021, 02:35:46 am by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #11 on: August 17, 2021, 02:48:28 am »
Success!

Adding the following curl options allowed the new libcurl.dll to work.

Code: Script BASIC
  1. ch = curl::init()
  2. curl::option(ch, "URL", "https://s.ch9.ms/Events/MIX/MIX11/RSS/mp4high")
  3. curl::option(ch, "SSL_VERIFYPEER")
  4. curl::option(ch, "CAINFO","C:\\ScriptBasic\\bin\\curl-ca-bundle.crt")
  5. rss = curl::perform(ch)
  6. curl::finish(ch)
  7.  

This problem of Windows not finding the certificate store may be due to when OpenVPN wiped out my Windows system path and I had to rebuild it from what info I could find on the web. I may be missing that path. (guess) Why the earlier version of the libcurl.dll works is a mystery to me. Outlook keeps asking me to verify my certificate but won't save it.

« Last Edit: August 17, 2021, 04:01:18 am by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #12 on: August 17, 2021, 03:35:19 pm »
I tried the new curl extension module on my older laptop running Windows 10 Pro. I use this laptop for testing and not development. It won't run unless I have he the new lines I added to the rss.sb challenge script. It seems things have changes since 7.5 and a it checks for a certificate before connecting to a HTTPS site.


Offline AIR

  • BASIC Developer
  • Posts: 817
Re: cURL Windows
« Reply #13 on: August 17, 2021, 07:53:25 pm »
Something must be 'broken' on your side.

This works on both my Mac and my Linux server:

Code: C
  1. #include <curl/curl.h>
  2. #include <stdio.h>
  3. // #include <stdlib.h>
  4. #include <string.h>
  5.  
  6. long write_data (char *content, long size, long nmemb, char *stream) {
  7.   strcat(stream,content);
  8.   return size*nmemb;
  9. }
  10.  
  11.  
  12.  
  13. int main(int argc, char **argv) {
  14.     char buffer[128000][10];
  15.     CURL    *curl;
  16.     CURLcode res;
  17.  
  18.     curl_global_init(CURL_GLOBAL_DEFAULT);
  19.     curl = curl_easy_init();
  20.  
  21.     if (curl) {
  22.         curl_easy_setopt(curl, CURLOPT_URL, "http://channel9.msdn.com/Events/MIX/MIX11/RSS/mp4high");
  23.         curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
  24.         curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
  25.         curl_easy_setopt(curl, CURLOPT_WRITEDATA, buffer);
  26.  
  27.         res = curl_easy_perform(curl);
  28.  
  29.         if(res != CURLE_OK) {
  30.             printf("\ncurl_easy_perform() failed: %s\n",curl_easy_strerror(res));
  31.         } else {
  32.             printf("%s\n", *buffer);
  33.         }
  34.  
  35.     }
  36.  
  37.     curl_easy_cleanup(curl);
  38.     curl_global_cleanup();
  39.     return 0;
  40. }

Note that I'm using the 'http' address as specified in the challenge.

"curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);" is what tells cURL to process redirects.

AIR.
« Last Edit: August 18, 2021, 04:33:05 am by AIR »

Offline John

  • Forum Support / SB Dev
  • Posts: 3114
    • ScriptBasic Open Source Project
Re: cURL Windows
« Reply #14 on: August 17, 2021, 09:58:47 pm »
I didn't have any issues on Linux either. Windows cURL is causing the problem. Are you using the latest release of LIBCURL?

The URL in your last example is not the URL in the original post.
« Last Edit: August 17, 2021, 10:13:49 pm by John »