Downloading Files

QB64-PE starting in v3.5.0 includes built-in support for making HTTP requests and receiving their responses. This functionality is implemented in _OPENCLIENT, and the resulting HTTP handle can then be provided the various existing or new stream commands to interact with the response.

HTTP functionality is current unstable, and requires $UNSTABLE:HTTP to be able to use.

Starting the Connection

_OPENCLIENT is used to make an HTTP request, via using the 'HTTP' scheme. The string provided to _OPENCLIENT should be of the form "HTTP:url", where 'url' is the HTTP url you want to make a request too. This can include anything valid in a URL, such as a port, resource path, query string, or fragment. The call to _OPENCLIENT will then return a handle to the HTTP connection which can be provided to all the rest of the commands related to HTTP requests. If the HTTP request cannot be made (for example, the domain is invalid) then zero is returned instead of a valid handle.

Metadata About the HTTP Response

The below commands all give metadata about the HTTP response. They're ordered in terms of usefulness:

• _STATUSCODE can be used to get the HTTP status code given in the HTTP response (Ex. 200, 404, 500, ...)
• LOF will return the 'Content Length' of the HTTP response if it was provided. It gives -1 if it is not known.
• EOF Will return whether the entire HTTP response has been read using GET.
• _CONNECTIONADDRESS$ can be used to get the "Effective URL" that was connected too, after taking redirects into account.
• _CONNECTED can be used to determine whether the HTTP connection to the server is still open.

In general for most HTTP responses you should ignore _CONNECTED and only make use of EOF. EOF will continue to return false until you have read all the data from the HTTP response and the connection has been closed, where-as _CONNECTED will return false even if you still have data to read.

Reading the HTTP Response Content

GET is used to read the data from the HTTP response. It should be used with a variable length string, which will then be resized by GET so that it contains all the data current available for the HTTP response. You can then either parse the data as it comes, or collect it all together to parse later. Data is available for reading with GET until EOF returns true. A _LIMIT or similar delay should be put in place when looping and using GET so that you don't use up too much CPU time. The actual downloading happens on a separate thread in your program, so how often you call GET has no impact on how fast the download is.

Closing The HTTP handle

Like any handle returned from _OPENCLIENT, all HTTP handles need to be closed by using CLOSE when you are finished with them.

Examples

Example 1: This function provides basic download functionality. The downloaded data will returned in a string, and the statuscode is provided so that you can check whether the request was successful.

' Content of the HTTP response is returned. ' The statusCode is also assigned. FUNCTION Download$(url As STRING, statusCode As LONG) DIM h As LONG, content As STRING, s As STRING h = _OPENCLIENT("HTTP:" + url) statusCode = _STATUSCODE(h) WHILE NOT EOF(h) _LIMIT 60 GET #h, , s content = content + s WEND CLOSE #h Download$ = content END FUNCTION

Example 2: This shows how you could modify the basic usage to provide a progress bar. Note that displaying progress is only possible if the server gives a `Content-length` header back in the response, which will be indicating to you via the result of LOF on the handle:

FUNCTION DownloadWithProgress$(url As STRING, statusCode As LONG) DIM h As LONG, content As STRING, s As STRING, length As LONG DIM progress As DOUBLE h = _OPENCLIENT("HTTP:" + url) statusCode = _STATUSCODE(h) length = LOF(h) WHILE NOT EOF(h) _LIMIT 60 GET #h, , s content = content + s ' Display a progress bar if the Content-Length was provided IF length <> -1 THEN progress = CDBL(LEN(content)) / length LOCATE 1, 1 PRINT "["; PRINT STRING$(progress * 78, "*"); PRINT STRING$(78 - progress * 78, " "); PRINT "]"; ELSE LOCATE 1, 1 PRINT "[ Downloading..."; STRING$(80 - 17, " "); "]"; END IF WEND CLOSE #h Download$ = content END FUNCTION

Example 3: This shows processing multiple downloads at the same time, with progress for each (Again keeping in mind that progress only works if LOF gives a valid length):

SUB DownloadMultipleWithProgress() DIM handles(1 To 3) As LONG DIM content(1 To 3) As STRING DIM length(1 to 3) As LONG DIM i As LONG, progress As DOUBLE DIM EndOfFile As LONG, s As STRING handles(1) = _OPENCLIENT("HTTP:https://www.google.com") handles(2) = _OPENCLIENT("HTTP:https://www.google.com") handles(3) = _OPENCLIENT("HTTP:https://www.google.com") FOR i = 1 To UBOUND(handles) length(i) = LOF(handles(i)) NEXT DO _LIMIT 60 EndOfFile = -1 FOR i = 1 To UBOUND(handles) IF handles(i) = 0 THEN _CONTINUE IF EOF(handles(i)) THEN CLOSE #handles(i) handles(i) = 0 _CONTINUE END IF EndOfFile = 0 GET #handles(i), , s content(i) = content(i) + s ' Display a progress bar if the ' Content-Length was provided IF length <> -1 THEN progress = CDBL(LEN(content(i))) / length(i) LOCATE i, 1 PRINT "["; PRINT STRING$(progress * 78, "*"); PRINT STRING$(78 - progress * 78, " "); PRINT "]"; ELSE LOCATE i, 1 PRINT "[ Downloading..."; STRING$(80 - 17, " "); "]"; END IF NEXT LOOP While NOT EndOfFile ' The content() array now contains the results of the downloads END SUB

Example 4: This shows downloading straight to a file by providing the open file number. This approach is much more memory efficient if the HTTP response is large:

' Returns the status code of the HTTP response FUNCTION DownloadToFile&(url As STRING, fileHandle As LONG) DIM h As LONG, content As STRING, s As STRING h = _OPENCLIENT("HTTP:" + url) DownloadToFile& = _STATUSCODE(h) WHILE NOT EOF(h) _LIMIT 60 GET #h, , s PUT #fileHandle, , s WEND CLOSE #h END FUNCTION