Downloading Files: Difference between revisions

From QB64 Phoenix Edition Wiki
Jump to navigation Jump to search
m (Removed protection from "Downloading Files")
No edit summary
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:
:'''HTTP/1.1''' protocol downloads can be done using a '''GET''' request using the following format without HTTP:// in the url:
'''QB64-PE''' starting in '''v3.5.0''' includes built-in support for making HTTP requests and receiving their responses. This functionality is implemented in [[_OPENCLIENT]], and the resulting HTTP handle can then be provided the various existing or new stream commands to interact with the response.
{{TextStart}}CRLF$ = CHR$(13) + CHR$(10)
Request$ = "GET " + File_Path + " HTTP/1.1" + CRLF$ + "Host:" + Web_Address + CRLF$ + CRLF$
{{TextEnd}}
: Two carriage returns end the request to the [[_OPENCLIENT|client]] URL. The header that is returned from the site also ends with two carriage returns when there are no errors. The header will also include the requested file's byte size after "content-length:" as below:
{{TextStart}}HTTP/1.1 200 OK
Server: dbws
Date: Tue, 25 Oct 2011 04:41:03 GMT
Content-Type: text/plain; charset=ascii
Connection: keep-alive
{{text|content-length: 4087|blue}}
x-robots-tag: noindex,nofollow
accept-ranges: bytes
etag: 365n
pragma: public
cache-control: max-age=0 '' ''
{{TextEnd}}
: Each row of text response sent is ended with a carriage return with the end of header having two. After that comes the file data.
: It is recommended that data be loaded immediately by using a [[GET (TCP/IP statement)|GET]] loop so that connections are not dropped during a [[_DELAY|delay]].


{{Text|'''Before QB64-PE v4.0.0 HTTP functionality was unstable and requires [[$UNSTABLE]]:HTTP to be able to use it.'''|red}}


: '''A "raw" Download function that downloads an image directly off of a web page using the image name on that page.'''
 
{| align="right" style="max-width:25%;"
|  __TOC__
|}
== Starting the Connection ==
[[_OPENCLIENT]] is used to make an HTTP request, via using the '''HTTP:''' scheme. The string provided to [[_OPENCLIENT]] should be of the form "'''HTTP:'''url", where '''url''' is the HTTP url you want to make a request too. This can include anything valid in a URL, such as a port, resource path, query string, or fragment. The call to [[_OPENCLIENT]] will then return a handle to the HTTP connection which can be provided to all the rest of the commands related to HTTP requests. If the HTTP request cannot be made (for example, the domain is invalid) then zero is returned instead of a valid handle.
 
 
== Metadata About the HTTP Response ==
The below commands all give metadata about the HTTP response. They're ordered in terms of usefulness:
* [[_STATUSCODE]] can be used to get the HTTP status code given in the HTTP response (Ex. 200, 404, 500, ...)
* [[LOF]] will return the 'Content Length' of the HTTP response if it was provided. It gives -1 if it is not known.
* [[EOF]] Will return whether the entire HTTP response has been read using [[GET (HTTP statement)|GET]].
* [[_CONNECTIONADDRESS$]] can be used to get the "Effective URL" that was connected too, after taking redirects into account.
* [[_CONNECTED]] can be used to determine whether the HTTP connection to the server is still open.
In general for most HTTP responses you should ignore [[_CONNECTED]] and only make use of [[EOF]]. [[EOF]] will continue to return false until you have read all the data from the HTTP response and the connection has been closed, where-as [[_CONNECTED]] will return false even if you still have data to read.
 
 
== Reading the HTTP Response Content ==
[[GET (HTTP statement)|GET]] is used to read the data from the HTTP response. It should be used with a variable length string, which will then be resized by [[GET (HTTP statement)|GET]] so that it contains all the data current available for the HTTP response. You can then either parse the data as it comes, or collect it all together to parse later. Data is available for reading with [[GET (HTTP statement)|GET]] until [[EOF]] returns true. A [[_LIMIT]] or similar delay should be put in place when looping and using [[GET (HTTP statement)|GET]] so that you don't use up too much CPU time. The actual downloading happens on a separate thread in your program, so how often you call [[GET (HTTP statement)|GET]] has no impact on how fast the download is.
 
 
== Closing The HTTP handle ==
Like any handle returned from [[_OPENCLIENT]], all HTTP handles need to be closed by using [[CLOSE]] when you are finished with them.
 
 
{{PageExamples}}
;Example 1:This function provides basic download functionality. The downloaded data will returned in a string, and the statuscode is provided so that you can check whether the request was successful.
{{CodeStart}}
{{CodeStart}}
'' ''
' Content of the HTTP response is returned.
{{Cl|IF...THEN|IF}} Download("www.qb64.net/qb64.png", "qb64logo.png", 10) {{Cl|THEN}} ' timelimit = 10 seconds
' The statusCode is also assigned.
{{Cl|SCREEN}} {{Cl|_LOADIMAGE}}("qb64logo.png",32)
{{Cl|FUNCTION}} Download$(url {{Cl|AS}} {{Cl|STRING}}, statusCode {{Cl|AS}} {{Cl|LONG}})
{{Cl|ELSE}}: {{Cl|PRINT}} "Couldn't download QB64 logo."
    {{Cl|DIM}} h {{Cl|AS}} {{Cl|LONG}}, content {{Cl|AS}} {{Cl|STRING}}, s {{Cl|AS}} {{Cl|STRING}}
{{Cl|END IF}}
{{Cl|SLEEP}}
{{Cl|SYSTEM}}
' ---------- program end -----------


{{Cl|FUNCTION}} Download (url$, file$, timelimit) ' returns -1 if successful, 0 if not
    h = {{Cl|_OPENCLIENT}}("HTTP:" + url)
url2$ = url$
x = {{Cl|INSTR}}(url2$, "/")
{{Cl|IF...THEN|IF}} x {{Cl|THEN}} url2$ = {{Cl|LEFT$}}(url$, x - 1)
client = {{Cl|_OPENCLIENT}}("TCP/IP:80:" + url2$)
{{Cl|IF...THEN|IF}} client = 0 {{Cl|THEN}} {{Cl|EXIT FUNCTION}}
e$ = {{Cl|CHR$}}(13) + {{Cl|CHR$}}(10) ' end of line characters
url3$ = {{Cl|RIGHT$}}(url$, {{Cl|LEN}}(url$) - x + 1)
x$ = "GET " + url3$ + " HTTP/1.1" + e$
x$ = x$ + "Host: " + url2$ + e$ + e$
{{Cl|PUT (TCP/IP statement)|PUT}} #client, , x$
t! = {{Cl|TIMER}} ' start time
{{Cl|DO}}
    {{Cl|_DELAY}} 0.05 ' 50ms delay (20 checks per second)
    {{Cl|GET (TCP/IP statement)|GET}} #client, , a2$
    a$ = a$ + a2$
    i = {{Cl|INSTR}}(a$, "Content-Length:")
    {{Cl|IF...THEN|IF}} i {{Cl|THEN}}
      i2 = {{Cl|INSTR}}(i, a$, e$)
      {{Cl|IF...THEN|IF}} i2 {{Cl|THEN}}
      l = {{Cl|VAL}}({{Cl|MID$}}(a$, i + 15, i2 - i -14))
      i3 = {{Cl|INSTR}}(i2, a$, e$ + e$)
        {{Cl|IF...THEN|IF}} i3 {{Cl|THEN}}
          i3 = i3 + 4 'move i3 to start of data
          {{Cl|IF...THEN|IF}} ({{Cl|LEN}}(a$) - i3 + 1) = l {{Cl|THEN}}
            {{Cl|CLOSE}} client ' CLOSE CLIENT
            d$ = {{Cl|MID$}}(a$, i3, l)
            fh = {{Cl|FREEFILE}}
            {{Cl|OPEN}} file$ {{Cl|FOR}} {{Cl|OUTPUT}} {{Cl|AS}} #fh: {{Cl|CLOSE}} #fh 'Warning! Clears data from existing file
            {{Cl|OPEN}} file$ {{Cl|FOR}} {{Cl|BINARY}} {{Cl|AS}} #fh
            {{Cl|PUT}} #fh, , d$
            {{Cl|CLOSE}} #fh
            Download = -1 'indicates download was successfull
            {{Cl|EXIT FUNCTION}}
          {{Cl|END IF}} ' availabledata = l
        {{Cl|END IF}} ' i3
      {{Cl|END IF}} ' i2
    {{Cl|END IF}} ' i
{{Cl|LOOP}} {{Cl|UNTIL}} {{Cl|TIMER}} > t! + timelimit ' (in seconds)
{{Cl|CLOSE}} client
{{Cl|END FUNCTION}} '' ''
{{CodeEnd}}
{{small|Code by Galleon}}


    statusCode = {{Cl|_STATUSCODE}}(h)


: '''Downloading a [[BINARY]] image file from a download link using [[GET|GET #]] which requires the "content-length" in header.'''
    {{Cl|WHILE}} {{Cl|NOT}} {{Cl|EOF}}(h)
{{CodeStart}} '' ''
        {{Cl|_LIMIT}} 60
{{Cl|DEFINT}} A-Z
        {{Cl|GET (HTTP statement)|GET}} #h, , s
        content = content + s
    {{Cl|WEND}}


CR$ = {{Cl|CHR$}}(13) + {{Cl|CHR$}}(10) 'crlf carriage return line feed characters
    {{Cl|CLOSE}} #h
' Change this to the file's public link
<nowiki>DownFile$ = "http://a5.sphotos.ak.fbcdn.net/hphotos-ak-snc7/293944_10150358253659788_151813304787_8043392_486717139_n.jpg?dl=1"</nowiki>


' Get base URL
    Download$ = content
BaseURL$ = DownFile$
{{Cl|END FUNCTION}}
{{Cl|IF...THEN|IF}} {{Cl|INSTR}}(BaseURL$, "http://") {{Cl|THEN}} BaseURL$ = {{Cl|RIGHT$}}(BaseURL$, {{Cl|LEN}}(BaseURL$) - 7) 'trim http://
{{CodeEnd}}
Path$ = {{Cl|MID$}}(BaseURL$, {{Cl|INSTR}}(BaseURL$, "/")) 'path to file
BaseURL$ = {{Cl|LEFT$}}(BaseURL$, {{Cl|INSTR}}(BaseURL$, "/") - 1) 'site URL


----


' Connect to base URL
;Example 2:This shows how you could modify the basic usage to provide a progress bar. Note that displaying progress is only possible if the server gives a `Content-length` header back in the response, which will be indicating to you via the result of [[LOF]] on the handle:
{{Cl|PRINT}} "Connecting to "; BaseURL$; "...";
{{CodeStart}}
Client& = {{Cl|_OPENCLIENT}}("TCP/IP:80:" + BaseURL$)
{{Cl|FUNCTION}} DownloadWithProgress$(url {{Cl|AS}} {{Cl|STRING}}, statusCode {{Cl|AS}} {{Cl|LONG}})
{{Cl|IF...THEN|IF}} Client& = 0 {{Cl|THEN}} {{Cl|PRINT}} "Failed to connect...": {{Cl|END}}
    {{Cl|DIM}} h {{Cl|AS}} {{Cl|LONG}}, content {{Cl|AS}} {{Cl|STRING}}, s {{Cl|AS}} {{Cl|STRING}}, length {{Cl|AS}} {{Cl|LONG}}
{{Cl|PRINT}} "Done."
    {{Cl|DIM}} progress {{Cl|AS}} {{Cl|DOUBLE}}


' Send download request
    h = {{Cl|_OPENCLIENT}}("HTTP:" + url)
{{Cl|PRINT}} "Sending download request...";
Request$ = "GET " + Path$ + " HTTP/1.1" + CR$ + "Host:" + BaseURL$ + CR$ + CR$
{{Cl|PUT}} #Client&, , Request$
{{Cl|PRINT}} "Done."


' Download the header
    statusCode = {{Cl|_STATUSCODE}}(h)
{{Cl|PRINT}} "Getting HTML header...";
Dat$ = ""
DO
  {{Cl|_LIMIT}} 20
  {{Cl|GET}} #Client&, , gDat$
  Dat$ = Dat$ + gDat$
{{Cl|LOOP}} {{Cl|UNTIL}} {{Cl|INSTR}}(Dat$, CR$ + CR$) ' Loop until 2 CRLFs (end of HTML header) are found
{{Cl|PRINT}} "Done."


' Get file size
    length = {{Cl|LOF}}(h)
FileSizePos = {{Cl|INSTR}}({{Cl|UCASE$}}(Dat$), "CONTENT-LENGTH: ") + 16
FileSizeEnd = {{Cl|INSTR}}(FileSizePos, Dat$, CR$)
FileSize& = {{Cl|VAL}}({{Cl|MID$}}(Dat$, FileSizePos, (FileSizeEnd - FileSizePos) + 1))


{{Cl|PRINT}} "File size:"; FileSize&
    {{Cl|WHILE}} {{Cl|NOT}} {{Cl|EOF}}(h)
{{Cl|PRINT}} "Downloading file...";
        {{Cl|_LIMIT}} 60
        {{Cl|GET (HTTP statement)|GET}} #h, , s
        content = content + s


' Trim off HTML header
        ' Display a progress bar if the Content-Length was provided
EndHeaderPos = {{Cl|INSTR}}(Dat$, CR$ + CR$) + 4
        {{Cl|IF}} length <> -1 {{Cl|THEN}}
Dat$ = {{Cl|RIGHT$}}(Dat$, ({{Cl|LEN}}(Dat$) - EndHeaderPos) + 1)
            progress = {{Cl|CDBL}}({{Cl|LEN}}(content)) / length


' Get the file name tucked at the end of the URL if necessary
            {{Cl|LOCATE}} 1, 1
{{Cl|FOR...NEXT|FOR}} S = {{Cl|LEN}}(DownFile$) {{Cl|TO}} 1 {{Cl|STEP}} -1
            {{Cl|PRINT}} "[";
  {{Cl|IF...THEN|IF}} {{Cl|MID$}}(DownFile$, S, 1) = "/" {{Cl|THEN}}
            {{Cl|PRINT}} {{Cl|STRING$}}(progress * 78, "*");
      OutFile$ = {{Cl|RIGHT$}}(DownFile$, ({{Cl|LEN}}(DownFile$) - S))
            {{Cl|PRINT}} {{Cl|STRING$}}(78 - progress * 78, " ");
      {{Cl|EXIT}} {{Cl|FOR...NEXT|FOR}}
            {{Cl|PRINT}} "]";
  {{Cl|END IF}}
        {{Cl|ELSE}}
{{Cl|NEXT}} S
            {{Cl|LOCATE}} 1, 1
' Remove some kind of tag at the end of the file name in some URLs
            {{Cl|PRINT}} "[ Downloading..."; {{Cl|STRING$}}(80 - 17, " "); "]";
{{Cl|IF...THEN|IF}} {{Cl|INSTR}}(OutFile$, "?") {{Cl|THEN}} OutFile$ = {{Cl|LEFT$}}(OutFile$, {{Cl|INSTR}}(OutFile$, "?") - 1)
        {{Cl|END IF}}
    {{Cl|WEND}}


' Download the rest of the data
    {{Cl|CLOSE}} #h
{{Cl|OPEN}} OutFile$ {{Cl|FOR}} {{Cl|OUTPUT}} {{Cl|AS}} #1: {{Cl|CLOSE}} #1 'Warning! Clears data from an existing file
 
{{Cl|OPEN}} OutFile$ {{Cl|FOR}} {{Cl|BINARY}} {{Cl|AS}} #1 'write data to binary image file
    Download$ = content
DO
{{Cl|END FUNCTION}}
  {{Cl|_LIMIT}} 20
  {{Cl|PUT}} #1, , Dat$
  {{Cl|GET}} #Client&, , Dat$
{{Cl|LOOP}} {{Cl|UNTIL}} {{Cl|LOF}}(1) >= FileSize&
{{Cl|CLOSE}} #1, #Client&
{{Cl|PRINT}} "Done!" '' ''
{{CodeEnd}}
{{CodeEnd}}
{{small|Adapted from code by Jobert14}}
: '''Note:''' Some download links require that a '''tag''' be added after the file name. Remove that tag if it will be used as the file name.


----


: '''Downloading a sequencial text file from a Drop Box download link using HTTP GET and the [[GET (TCP/IP statement)|TCP/IP GET]] statement.'''
;Example 3:This shows processing multiple downloads at the same time, with progress for each (Again keeping in mind that progress only works if [[LOF]] gives a valid length):
{{CodeStart}} '' ''
{{CodeStart}}
CrLf$ = {{Cl|CHR$}}(13) + {{Cl|CHR$}}(10) ' carriage return + line feed ASCII characters
{{Cl|SUB}} DownloadMultipleWithProgress()
    {{Cl|DIM}} handles(1 To 3) {{Cl|AS}} {{Cl|LONG}}
    {{Cl|DIM}} content(1 To 3) {{Cl|AS}} {{Cl|STRING}}
    {{Cl|DIM}} length(1 to 3) {{Cl|AS}} {{Cl|LONG}}
    {{Cl|DIM}} i {{Cl|AS}} {{Cl|LONG}}, progress {{Cl|AS}} {{Cl|DOUBLE}}
    {{Cl|DIM}} EndOfFile {{Cl|AS}} {{Cl|LONG}}, s {{Cl|AS}} {{Cl|STRING}}


Host = {{Cl|_OPENHOST}}("TCP/IP:319")
    handles(1) = {{Cl|_OPENCLIENT}}("HTTP:<nowiki>https://www.google.com</nowiki>")
{{Cl|IF...THEN|IF}} Host {{Cl|THEN}}
    handles(2) = {{Cl|_OPENCLIENT}}("HTTP:<nowiki>https://www.google.com</nowiki>")
  {{Cl|PRINT}} "> Server started succesfully."
    handles(3) = {{Cl|_OPENCLIENT}}("HTTP:<nowiki>https://www.google.com</nowiki>")


  '// Change this to the file's public link
    {{Cl|FOR}} i = 1 To {{Cl|UBOUND}}(handles)
  IP_File$ = "dl.dropbox.com/u/8440706/QB64.INI" 'a Drop Box link
        length(i) = {{Cl|LOF}}(handles(i))
  URL$ = {{Cl|LEFT$}}(IP_File$, {{Cl|INSTR}}(IP_File$, "/") - 1)
     {{Cl|NEXT}}
  Path$ = {{Cl|MID$}}(IP_File$, {{Cl|INSTR}}(IP_File$, "/"))
  Client& = {{Cl|_OPENCLIENT}}("TCP/IP:80:" + URL$)
  {{Cl|IF...THEN|IF}} Client& {{Cl|THEN}}
    Request$ = "GET " + Path$ + " HTTP/1.1" + CrLf$ + "Host:" + URL$ + CrLf$ + CrLf$
    {{Cl|PUT}} #Client&, , Request$
    DO: {{Cl|_LIMIT}} 20 '              load response header
      {{Cl|GET}} #Client&, , Dat$
      Header$ = Header$ + Dat$
    {{Cl|LOOP}} {{Cl|UNTIL}} {{Cl|INSTR}}(Header$, CrLf$ + CrLf$) ' Loop until 2 CRLFs (end of HTML header) are found
     {{Cl|PRINT}} "Header Done."


     ' Get file size from header
     {{Cl|DO}}
    SizePos = {{Cl|INSTR}}({{Cl|UCASE$}}(Header$), "CONTENT-LENGTH:") + 16
        {{Cl|_LIMIT}} 60
    SizeEnd = {{Cl|INSTR}}(SizePos, Header$, CrLf$)
        EndOfFile = -1
    FileSize& = {{Cl|VAL}}({{Cl|MID$}}(Header$, SizePos, (SizeEnd - SizePos) + 1))
    {{Cl|PRINT}} "File size is"; FileSize&; "bytes"
    EndPos = {{Cl|INSTR}}(Header$, CrLf$ + CrLf$) + 4
    Response$ = {{Cl|MID$}}(Header$, EndPos) ' get data after header already downloaded


    start = 1 '// Get file name from original URL path if necessary
        {{Cl|FOR}} i = 1 To {{Cl|UBOUND}}(handles)
    {{Cl|DO...LOOP|DO}} '// Change this to destination local file name and path...
            {{Cl|IF}} handles(i) = 0 {{Cl|THEN}} {{Cl|_CONTINUE}}
      posit = {{Cl|INSTR}}(start, IP_File$, "/")
 
      {{Cl|IF...THEN|IF}} posit {{Cl|THEN}} lastpos = posit: start = posit + 1
            {{Cl|IF}} {{Cl|EOF}}(handles(i)) {{Cl|THEN}}
    {{Cl|LOOP}} {{Cl|UNTIL}} posit = 0
                {{Cl|CLOSE}} #handles(i)
    File$ = {{Cl|MID$}}(IP_File$, lastpos + 1) 'beware of tag suffixes
                handles(i) = 0
    {{Cl|OPEN}} File$ {{Cl|FOR...NEXT|FOR}} {{Cl|BINARY}} {{Cl|AS}} #1
                {{Cl|_CONTINUE}}
    DO: {{Cl|_LIMIT}} 20
            {{Cl|END IF}}
      {{Cl|PUT}} #1, , Response$
 
      {{Cl|GET}} #Client&, , Response$
            EndOfFile = 0
    {{Cl|LOOP}} {{Cl|UNTIL}} {{Cl|LOF}}(1) >= FileSize&
 
    {{Cl|PRINT}} "File download completed!"
            {{Cl|GET (HTTP statement)|GET}} #handles(i), , s
    {{Cl|CLOSE}} #1
            content(i) = content(i) + s
  {{Cl|ELSE}}
 
    {{Cl|PRINT}} "Failed to connect."
            ' Display a progress bar if the
  {{Cl|END IF}}
            ' Content-Length was provided
{{Cl|ELSE}}
            {{Cl|IF}} length <> -1 {{Cl|THEN}}
  {{Cl|PRINT}} "Failed to create server connection..."
                progress = {{Cl|CDBL}}({{Cl|LEN}}(content(i))) / length(i)
{{Cl|END IF}} '' ''
                {{Cl|LOCATE}} i, 1
                {{Cl|PRINT}} "[";
                {{Cl|PRINT}} {{Cl|STRING$}}(progress * 78, "*");
                {{Cl|PRINT}} {{Cl|STRING$}}(78 - progress * 78, " ");
                {{Cl|PRINT}} "]";
            {{Cl|ELSE}}
                {{Cl|LOCATE}} i, 1
                {{Cl|PRINT}} "[ Downloading..."; {{Cl|STRING$}}(80 - 17, " "); "]";
            {{Cl|END IF}}
        {{Cl|NEXT}}
    {{Cl|LOOP}} While {{Cl|NOT}} EndOfFile
 
    ' The content() array now contains the results of the downloads
{{Cl|END SUB}}
{{CodeEnd}}
{{CodeEnd}}
{{small|Code suggested by Matt Kilgore}}


{{PageNavigation}}
----
 
;Example 4:This shows downloading straight to a file by providing the open file number. This approach is much more memory efficient if the HTTP response is large:
{{CodeStart}}
' Returns the status code of the HTTP response
{{Cl|FUNCTION}} DownloadToFile&(url {{Cl|AS}} {{Cl|STRING}}, fileHandle {{Cl|AS}} {{Cl|LONG}})
    {{Cl|DIM}} h {{Cl|AS}} {{Cl|LONG}}, content {{Cl|AS}} {{Cl|STRING}}, s {{Cl|AS}} {{Cl|STRING}}
 
    h = {{Cl|_OPENCLIENT}}("HTTP:" + url)
 
    DownloadToFile& = {{Cl|_STATUSCODE}}(h)
 
    {{Cl|WHILE}} {{Cl|NOT}} {{Cl|EOF}}(h)
        {{Cl|_LIMIT}} 60
        {{Cl|GET (HTTP statement)|GET}} #h, , s
        {{Cl|PUT}} #fileHandle, , s
    {{Cl|WEND}}
 
    {{Cl|CLOSE}} #h
{{Cl|END FUNCTION}}
{{CodeEnd}}
 
 
{{PageReferences}}

Latest revision as of 13:15, 19 November 2024

QB64-PE starting in v3.5.0 includes built-in support for making HTTP requests and receiving their responses. This functionality is implemented in _OPENCLIENT, and the resulting HTTP handle can then be provided the various existing or new stream commands to interact with the response.

Before QB64-PE v4.0.0 HTTP functionality was unstable and requires $UNSTABLE:HTTP to be able to use it.


Starting the Connection

_OPENCLIENT is used to make an HTTP request, via using the HTTP: scheme. The string provided to _OPENCLIENT should be of the form "HTTP:url", where url is the HTTP url you want to make a request too. This can include anything valid in a URL, such as a port, resource path, query string, or fragment. The call to _OPENCLIENT will then return a handle to the HTTP connection which can be provided to all the rest of the commands related to HTTP requests. If the HTTP request cannot be made (for example, the domain is invalid) then zero is returned instead of a valid handle.


Metadata About the HTTP Response

The below commands all give metadata about the HTTP response. They're ordered in terms of usefulness:

  • _STATUSCODE can be used to get the HTTP status code given in the HTTP response (Ex. 200, 404, 500, ...)
  • LOF will return the 'Content Length' of the HTTP response if it was provided. It gives -1 if it is not known.
  • EOF Will return whether the entire HTTP response has been read using GET.
  • _CONNECTIONADDRESS$ can be used to get the "Effective URL" that was connected too, after taking redirects into account.
  • _CONNECTED can be used to determine whether the HTTP connection to the server is still open.

In general for most HTTP responses you should ignore _CONNECTED and only make use of EOF. EOF will continue to return false until you have read all the data from the HTTP response and the connection has been closed, where-as _CONNECTED will return false even if you still have data to read.


Reading the HTTP Response Content

GET is used to read the data from the HTTP response. It should be used with a variable length string, which will then be resized by GET so that it contains all the data current available for the HTTP response. You can then either parse the data as it comes, or collect it all together to parse later. Data is available for reading with GET until EOF returns true. A _LIMIT or similar delay should be put in place when looping and using GET so that you don't use up too much CPU time. The actual downloading happens on a separate thread in your program, so how often you call GET has no impact on how fast the download is.


Closing The HTTP handle

Like any handle returned from _OPENCLIENT, all HTTP handles need to be closed by using CLOSE when you are finished with them.


Examples

Example 1
This function provides basic download functionality. The downloaded data will returned in a string, and the statuscode is provided so that you can check whether the request was successful.
' Content of the HTTP response is returned.
' The statusCode is also assigned.
FUNCTION Download$(url AS STRING, statusCode AS LONG)
    DIM h AS LONG, content AS STRING, s AS STRING

    h = _OPENCLIENT("HTTP:" + url)

    statusCode = _STATUSCODE(h)

    WHILE NOT EOF(h)
        _LIMIT 60
        GET #h, , s
        content = content + s
    WEND

    CLOSE #h

    Download$ = content
END FUNCTION

Example 2
This shows how you could modify the basic usage to provide a progress bar. Note that displaying progress is only possible if the server gives a `Content-length` header back in the response, which will be indicating to you via the result of LOF on the handle:
FUNCTION DownloadWithProgress$(url AS STRING, statusCode AS LONG)
    DIM h AS LONG, content AS STRING, s AS STRING, length AS LONG
    DIM progress AS DOUBLE

    h = _OPENCLIENT("HTTP:" + url)

    statusCode = _STATUSCODE(h)

    length = LOF(h)

    WHILE NOT EOF(h)
        _LIMIT 60
        GET #h, , s
        content = content + s

        ' Display a progress bar if the Content-Length was provided
        IF length <> -1 THEN
            progress = CDBL(LEN(content)) / length

            LOCATE 1, 1
            PRINT "[";
            PRINT STRING$(progress * 78, "*");
            PRINT STRING$(78 - progress * 78, " ");
            PRINT "]";
        ELSE
            LOCATE 1, 1
            PRINT "[ Downloading..."; STRING$(80 - 17, " "); "]";
        END IF
    WEND

    CLOSE #h

    Download$ = content
END FUNCTION

Example 3
This shows processing multiple downloads at the same time, with progress for each (Again keeping in mind that progress only works if LOF gives a valid length):
SUB DownloadMultipleWithProgress()
    DIM handles(1 To 3) AS LONG
    DIM content(1 To 3) AS STRING
    DIM length(1 to 3) AS LONG
    DIM i AS LONG, progress AS DOUBLE
    DIM EndOfFile AS LONG, s AS STRING

    handles(1) = _OPENCLIENT("HTTP:https://www.google.com")
    handles(2) = _OPENCLIENT("HTTP:https://www.google.com")
    handles(3) = _OPENCLIENT("HTTP:https://www.google.com")

    FOR i = 1 To UBOUND(handles)
        length(i) = LOF(handles(i))
    NEXT

    DO
        _LIMIT 60
        EndOfFile = -1

        FOR i = 1 To UBOUND(handles)
            IF handles(i) = 0 THEN _CONTINUE

            IF EOF(handles(i)) THEN
                CLOSE #handles(i)
                handles(i) = 0
                _CONTINUE
            END IF

            EndOfFile = 0

            GET #handles(i), , s
            content(i) = content(i) + s

            ' Display a progress bar if the
            ' Content-Length was provided
            IF length <> -1 THEN
                progress = CDBL(LEN(content(i))) / length(i)
                LOCATE i, 1
                PRINT "[";
                PRINT STRING$(progress * 78, "*");
                PRINT STRING$(78 - progress * 78, " ");
                PRINT "]";
            ELSE
                LOCATE i, 1
                PRINT "[ Downloading..."; STRING$(80 - 17, " "); "]";
            END IF
        NEXT
    LOOP While NOT EndOfFile

    ' The content() array now contains the results of the downloads
END SUB

Example 4
This shows downloading straight to a file by providing the open file number. This approach is much more memory efficient if the HTTP response is large:
' Returns the status code of the HTTP response
FUNCTION DownloadToFile&(url AS STRING, fileHandle AS LONG)
    DIM h AS LONG, content AS STRING, s AS STRING

    h = _OPENCLIENT("HTTP:" + url)

    DownloadToFile& = _STATUSCODE(h)

    WHILE NOT EOF(h)
        _LIMIT 60
        GET #h, , s
        PUT #fileHandle, , s
    WEND

    CLOSE #h
END FUNCTION


QB64 Programming References

Wiki Pages
Main Page with Articles and Tutorials
QB64 specific keywords (alphabetical)
Original QBasic keywords (alphabetical)
QB64 OpenGL keywords (alphabetical)
Keywords by Usage
Got a question about something?
Frequently Asked Questions about QB64
QB64 Phoenix Edition Community Forum
Links to other QBasic Sites:
Pete's QBasic Forum
Pete's QBasic Downloads