BASIC User Group > Code Challenges

Programatically download the latest 64 bit version of Firefox for Windows.

(1/11) > >>

AIR:
New Day, New Code Challenge.

Download the latest 64 bit version of Firefox for Windows.

Requirements:


* Connect to "https://www.mozilla.org/en-US/firefox/new"
* Parse the html to extract the current version number
* Parse the html to extract the download link to the FULL version (there is a "stub" version, we don't want that)
* Clear the screen
* Show a message with the version of Firefox being downloaded.
* Show the download progress as it's happening
* Show a complete message.
The progress should be displayed in the following format:

--- Quote ---    Downloading Latest Firefox (version number goes here)
    Downloaded 34140000 of 44400072 at 11784kb/s
--- End quote ---

    That is current data downloaded in real time, the total size, and the internet speed each in kb.

    The download progress should display on a single line, updating that line as the download progresses.


I'll be posting a submission using Nim this evening and leave the other languages to whoever wants to submit something....

AIR.

John:
This look like a natural for SB and cURL.

AIR:
NIM Submission, works on macOS and Linux (on a Raspberry Pi)

to compile:  nim c -d:ssl getFirefox.nim


--- Code: ---Firefox Download Challenge (Nim Version) by AIR.

Downloading Latest 64Bit Firefox (63.0) for Windows.

Downloaded 42.343MiB of 42.343MiB

Download Complete.
--- End code ---


--- Code: Text ---import httpclient, htmlparser,xmlparser, xmltree, streams, ospaths, asyncdispatch, strutils const    ClrLine = "\x1b[0K\r"var    totalsize:BiggestInt proc ClrScr() =    stdout.write("\x1b[2J\x1b[H")    proc showProgress(total, progress, speed: BiggestInt) {.async.} =    totalsize = total     stdout.write(ClrLine, "Downloading ", progress.formatSize, " of ", total.formatSize, " at ", (speed div 1000).formatSize(','))     flushFile(stdout) proc download(url,filename:string) {.async.} =    var client = newAsyncHttpClient()     client.onProgressChanged = showProgress     await client.downloadFile(url, filename)      stdout.write(ClrLine, "Downloaded ", totalsize.formatSize, " of ", totalsize.formatSize," ".repeat(20))     flushFile(stdout)     echo "\n\nDownload Complete.\n" proc main() =    var        client = newHttpClient()        src = client.getContent("https://www.mozilla.org/en-US/firefox/new")        c = parseHtml( newStringStream(src) )        url: string        version: string      for html in c.findAll("html"):        version = html.attr("data-latest-firefox")        break     for d in c.findAll("li"):        if d.attr("class") == "os_win64":            for e in d.findAll("a"):                if e.attr("class") == "download-link":                    url = e.attr("href")                    break     ClrScr()     echo "Firefox Download Challenge (Nim Version) by AIR.\n"    echo "Downloading Latest 64Bit Firefox (",version,") for Windows.\n"     waitFor url.download("Firefox Setup " & version & ".exe") main() 

This would have been so much simpler/clearer if Nim supported xpath syntax, but there ya go....


AIR.

John:
Here is my first pass at the challenge. I can download the file fine but the cURL progress meter feature doesn't seem to be working for me yet. I may have to resort to the static INFO funtions to build your download status string.

@AIR - When you said ignore the stub URL, did you mean the download file URL it returns or did I get the right link?

--- Code: Script BASIC ---INCLUDE curl.bas ch = curl::init()curl::option(ch,"URL","https://www.mozilla.org/en-US/firefox/new/")wp = curl::perform(ch) IF wp LIKE "*data-latest-firefox=\"*\" data-esr-versions*" THEN PRINT JOKER(2),"\n" IF wp LIKE """*<div id="other-platforms">*<li class="os_win64">*<a href="*"*""" THEN  curl::option(ch,"URL", JOKER(4))END IF dl_html = curl::perform(ch)IF dl_html LIKE """*href="*"*""" THEN dl_file = JOKER(2) curl::option(ch,"URL", dl_file)curl::option(ch,"FILE","Firefox_Win64.exe")curl::perform(ch) curl::finish(ch) 

jrs@jrs-laptop:~/sb/abcc$ time scriba ffdl.sb
63.0

real   0m2.422s
user   0m1.108s
sys   0m0.291s

jrs@jrs-laptop:~/sb/abcc$ ls -l Firefox_Win64.exe
-rw-r--r-- 1 jrs jrs 44400072 Oct 27 01:02 Firefox_Win64.exe
jrs@jrs-laptop:~/sb/abcc$

AIR:

--- Quote from: John on October 27, 2018, 01:03:53 am ---
@AIR - When you said ignore the stub URL, did you mean the download file URL it returns or did I get the right link?


--- End quote ---

The stub version is a small program you download that will then download and install Firefox.

What we're after is the full installer, which once you have it doesn't require an internet connection for Firefox to be installed on a system.


--- Quote from: John ---I may have to resort to the static INFO funtions to build your download status string.
--- End quote ---

I think you're gonna have to update the Curl module, because according to the old module docs CURLOPT_PROGRESSFUNCTION is not implemented:


--- Quote from: ScriptBasic Documentation ---CURLOPT_PROGRESSFUNCTION is used in conjunction with CURLOPT_PROGRESSDATA to specify a progress function that CURLIB library calls from time to time to allow progress indication for the user. This is not implemented in the ScriptBasic interface.
--- End quote ---

Which means that you won't get real time stats while the download is happening with the Curl module as it currently exists...


AIR.

Navigation

[0] Message Index

[#] Next page

Go to full version