Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Grab URL from net to file
#1
I recently posted a question about grabbing a url and wrote the following:

This attempts to grab a url and writes it to a filename:

Code: (Select All)
Rem program to grab url filename.
DefLng A-Z
On Error GoTo Hell
$Unstable:Http

Print "Enter url";: Input url$
Print "Enter filename";: Input Urlfile$
If Len(url$) And Len(Urlfile$) Then
  Z = DownloadToFileX&(url$, Urlfile$)
  If Z = 404 Then
      Print "File not found."
  Else
      If Z >= 200 And Z < 300 Then
        Print "Download success."
      Else
        Print "Download failure."
      End If
  End If
End If
End

Hell:
Z = 404
Resume Next

' get a file from url and download
Function DownloadToFileX& (url As String, Urlfile$)
  Dim bytescopied As Double
  Dim bytestring As String
  Dim clienthandle As Long
  Dim filehandle As Long
  client$ = url
  If Left$(client$, 2) = "//" Then
      client$ = "HTTP:" + client$
  End If
  If Left$(LCase$(client$), 5) = "http:" Or Left$(LCase$(client$), 6) = "https:" Then
      eat$ = ""
  Else
      client$ = "HTTP:" + client$
  End If
  ' check url exists.
  clienthandle = _OpenClient(client$)
  If clienthandle = 0 Then
      DownloadToFileX& = 404
      Print "Cannot open client."
      Exit Function
  End If
  If clienthandle Then
      x = _StatusCode(clienthandle)
      DownloadToFileX& = x
      If x >= 200 And x < 300 Then

        ' prompt to delete output file only if url exists.
        If _FileExists(Urlfile$) Then
            Print "Output file exists."
            Print "Delete file(y/n/q)?";
            Do
              _Limit 50
              X$ = InKey$
              If LCase$(X$) = "y" Then
                  Var$ = Urlfile$ + Chr$(0)
                  Kill Var$
                  Exit Do
              End If
              If LCase$(X$) = "n" Then
                  Exit Do
              End If
              If LCase$(X$) = "q" Then
                  Close #clienthandle
                  DownloadToFileX& = 404
                  Exit Function
              End If
            Loop
        End If

        ' open output file only if url exists.
        filehandle = FreeFile
        Open Urlfile$ For Binary As #filehandle

        ' copy the file.
        bytescopied = 0#
        starttimer = Timer
        While Not EOF(clienthandle)
            _Limit 60
            Get #clienthandle, , bytestring
            Put #filehandle, , bytestring
            bytescopied = bytescopied + Len(bytestring)
            elapsed = Timer - starttimer
            If elapsed < 0 Then elapsed = elapsed + 86400
            If elapsed >= 1 Then
              starttimer = Timer
              _Title "QB64 - Url: " + Str$(bytescopied) + " bytes copied."
            End If
        Wend
        Print "QB64 - Url: " + Str$(bytescopied) + " bytes copied."
      Else
        Print "Cannot open client."
      End If
  End If
  Close #clienthandle, filehandle
End Function
Reply
#2
I wouldn't recommend changing the title of the QB64 program in your download loop like that. Windows is not a huge fan of how QB64 changes the title and excessive title changes can cause issues.
Tread on those who tread on you

Reply
#3
One thing you need to watch out for is corrupting your data file.

Open Urlfile$ For Binary As #filehandle <-- This is where you open your file.

Before that line, add this one: Open Urlfile$ For OUTPUT As #filehandle: CLOSE #filehandle

As it exists, you could have the file already on the drive (lets say at 100,000 bytes of data), and then you choose not to erase it. When you download the new version of the file, it's only 96,000 bytes in size -- so it'll overwrite those first 96,000 bytes and then leave the rest unchanged. You need to blank that file before writing to it, or else you're just asking for trouble in the future with left over data.
Reply
#4
(11-07-2023, 03:12 PM)SMcNeill Wrote: One thing you need to watch out for is corrupting your data file.

      Open Urlfile$ For Binary As #filehandle  <-- This is where you open your file.

Before that line, add this one:        Open Urlfile$ For OUTPUT As #filehandle: CLOSE #filehandle

As it exists, you could have the file already on the drive (lets say at 100,000 bytes of data), and then you choose not to erase it.  When you download the new version of the file, it's only 96,000 bytes in size -- so it'll overwrite those first 96,000 bytes and then leave the rest unchanged.  You need to blank that file before writing to it, or else you're just asking for trouble in the future with left over data.

Oh, yeah. In my opinion, I'd just check if the file exists first then prompt to either kill it and proceed or just not download it. Or, you could even go as far as to make it autoincrement the name so you don't have naming conflicts. Like "file1.jpg(1)" "file1.jpg(2)" and so on and so forth.
Tread on those who tread on you

Reply
#5
Quote:Oh, yeah. In my opinion, I'd just check if the file exists first then prompt to either kill it and proceed or just not download it.
It already does..

btw: If you don't like the timed _Title then just Rem it out..
Reply
#6
(11-08-2023, 12:11 AM)eoredson Wrote:
Quote:Oh, yeah. In my opinion, I'd just check if the file exists first then prompt to either kill it and proceed or just not download it.
It already does..

btw: If you don't like the timed _Title then just Rem it out..

Eric, I was talking to Steve about the killing of the file when he was suggesting how to handle it already existing. As for the "timed _Title", it isn't a matter of me not liking it. I was saying Windows doesn't like having the title changed like that. But, uh, yeah, have fun.
Tread on those who tread on you

Reply




Users browsing this thread: 2 Guest(s)