More info about Random Access files

[PhilOfPerth]

I've been trying to build a small utility to create a Random Access file from an existing text file, and had a few problems getting it to work.
I have an alpha- sorted, single-element text file, and need to do a binary search to find words within it. I have got this to work , but found, after experimenting, that the Len for the file needed to be at least 2 bytes longer than the max data size.  I could find no information about this in the Wiki,  and reckon R/A files deserve their own spot there. Here's a (very simple) prog that demonstrates the problem, with the added 2 bytes.

Code: (Select All)

SW = 1040: SH = 720
Screen _NewImage(SW, SH, 32)
SetFont: F& = _LoadFont("C:\WINDOWS\fonts\courbd.ttf", 20, "monospace"): _Font F&
_ScreenMove (_DesktopWidth - SW) / 2, 90

Print "Aim: to convert a serial text file"
Print "(single element, various lengths, alpha-sorted)"
Print "to Direct access (Random access) file."
Print: Print "Problem: Record length"

'prepare a dummy Sequential list
Restore
Data "ARMY","BREAKFAST","CONCORDE","DANGER","ENERGY","FABCDEFGHIJKLMN"
Dim Wrd$(6): For a = 1 To 6: Read Wrd$(a): Next

'name the R/A file
RandFile$ = "RandFile.dat"

'get longest data length                                    10, length of FLASHPOINT
Input "Max Data length"; MaxLength
RL = MaxLength + 2 '                                        Len needs to be extended by 2 bytes
RecNum = 0

'prepare R/A file
Open RandFile$ For Random As #1 Len = RL

' Read each word and write to random file
For a = 1 To 6: Put #1, a, Wrd$(a): Next
Close

' Verify the random access file
Print: Print "Reading records from random access file:"
Open RandFile$ For Random As #1 Len = RL
For a = 1 To 6
   Get #1, a, wrd$: Print wrd$; "  ";
Next
End
Sleep

[bplus]

Each record for a Random Access file must be exactly the Len spec'd in Open statement

for strings you must use fixed length strings
Dim as String * some number! (not a variable)

Code: (Select All)

SW = 1040: SH = 720
Screen _NewImage(SW, SH, 32)
SetFont: F& = _LoadFont("C:\WINDOWS\fonts\courbd.ttf", 20, "monospace"): _Font F&
_ScreenMove (_DesktopWidth - SW) / 2, 90

Print "Aim: to convert a serial text file"
Print "(single element, various lengths, alpha-sorted)"
Print "to Direct access (Random access) file."

'measure the length of words use length of longest
Restore
Data "ARMY","BREAKFAST","CONCORDE","DANGER","ENERGY","F23456789012345"
Maxlength = 0
Dim Wrd$(6)
For a = 1 To 6:
    Read Wrd$(a):
    If Len(Wrd$(a)) > Maxlength Then Maxlength = Len(Wrd$(a))
Next
Print Maxlength ' = 15

Dim record As String * 15 ' <<< a number not a variable required for fixed strings
' random access needs fixed record lengths must use fixed strings for records

'name the R/A file
RandFile$ = "RandFile.dat"

'prepare R/A file
Open RandFile$ For Random As #1 Len = 15

' Read each record and write to random file
For i = 1 To 6
    record = Wrd$(i)
    Put #1, i, record
Next
Close

' Verify the random access file
Print: Print "Reading records from random access file:"
Open RandFile$ For Random As #1 Len = Maxlength
For i = 1 To 6
    Get #1, i, record: Print record
Next
End

[SMcNeill]

This is because you're printing a variable length string of data.

OPEN "foo.txt" FOR RANDOM AS #1 LEN = 15    <-- this says we're going to put data that is 15 characters *EXACTLY* into that random file
PUT #1, 1,"Apple"   <-- This is *ONLY* 5 characters.
GET #1, 1, a$   <-- But we're supposed to be putting and getting 15 characters....

PRINT a$  <-- What do you expect this to print??

If the above only grabbed 15 characters, it would print "Apple__________" where my underscores are 10 spaces instead.
But that's not what we want, so for variable length strings, it adds a length to the beginning of the string when it prints them. (As INTEGER data.)

PUT #1, 1, "Apple"   <-- This actually puts the integer size + "Apple" and then the rest is nothing.   

By writing the data with size at the beginning of the string, it allows your variable length string to be used and not append those extra spaces to it like it would if you DIM a AS STRING * 15.

QB45 and QB64 has been doing this forever and ever, but you're right; it could probably use a little better documentation on the subject.  When one is working with RANDOM access and putting variable length strings, don't forget to take that integer size into account in size.  It's there, just as you've discovered above. 

[SMcNeill]

' random access needs fixed record lengths must use fixed strings for records

@bplus Read my post above.  This is wrong.  RANDOM works fine with variable length strings.  You just need to account for the CRLF which is attached to them as a size placeholder.

[bplus]

"don't forget to take that CRLF into account in size"

nope not in Random access files!

[Pete]

You can get away with strings of different lengths, but the longest string cannot exceed the specified file length, minus 2-bytes.

Code: (Select All)

Open "crap" For Random As #1 Len = 10

a$ = "4444"

b$ = "666666"

c$ = "88888888"

Put #1, , a$: Put #1, , b$: Put #1, , c$

Close #1

Open "crap" For Random As #1 Len = 10

Get #1, , aa$: Get #1, , bb$: Get #1, , cc$

Close #1

Print aa$, bb$, cc$

Pete

[bplus]

This works fine exactly 15 * 6 for LOF no need for counting CRLF$

Code: (Select All)

' Verify the random access file
Print: Print "Reading records from random access file:"
Open RandFile$ For Random As #1 Len = Maxlength
Print "length of file ="; LOF(1); " this = exactly 15 * 6"

For i = 1 To 6
    Get #1, i, record: Print record
Next
End

[SMcNeill]

Actually, Steve was *WRONG*.   It's been too long since I'd last worked with RANDOM files and variable length strings.  It needs those 2 strings, but it's not a CRLF after the data; it's the size as an integer BEFORE the data:

Code: (Select All)

SW = 1040: SH = 720

Screen _NewImage(SW, SH, 32)

SetFont: F& = _LoadFont("C:\WINDOWS\fonts\courbd.ttf", 20, "monospace"): _Font F&

_ScreenMove (_DesktopWidth - SW) / 2, 90



Print "Aim: to convert a serial text file"

Print "(single element, various lengths, alpha-sorted)"

Print "to Direct access (Random access) file."

Print: Print "Problem: Record length"



'prepare a dummy Sequential list

Restore

Data "ARMY","BREAKFAST","CONCORDE","DANGER","ENERGY","FABCDEFGHIJKLMN"

Dim Wrd$(6): For a = 1 To 6: Read Wrd$(a): Next



'name the R/A file

RandFile$ = "RandFile.dat"



'get longest data length                                    10, length of FLASHPOINT

Input "Max Data length"; MaxLength

RL = MaxLength + 2 '                                        Len needs to be extended by 2 bytes

RecNum = 0



'prepare R/A file

Open RandFile$ For Output As #1: Close

Open RandFile$ For Random As #1 Len = RL



' Read each word and write to random file

For a = 1 To 6: Put #1, a, Wrd$(a): Next

Close



' Verify the random access file

Print: Print "Reading records from random access file:"

Open RandFile$ For Random As #1 Len = RL

For a = 1 To 6

    Get #1, a, wrd$: Print wrd$; "  ";

Next

Sleep

Print

Print

Print "Now to print the actual data in this file, byte by byte."

Close

Open RandFile$ For Binary As #1



Dim a As _Unsigned _Byte 'read it a character at a time

Do Until EOF(1)

    Get #1, , a

    If a > 31 Then

        Print Chr$(a);

    Else

        Print "("; a; ")";

    End If

Loop

And *that's* the secret trick to how you use variable length strings and why they need additional two bytes to them.  I knew it was related to the size of the data, but it wasn't a CRLF like my poor memory wanted to tell me.  As you can see from the actual binary data, it's the size of the string as 2-bytes (INTEGER), and then the string itself.

When dealing with fixed length strings, there's no size appended to the front of the data, so the size is absolute, whereas with variable length strings, the size is +2 as it stores the length of the string before the data.

[bplus]

LOF = 30

Code: (Select All)

Open "crap" For Random As #1 Len = 10
a$ = "4444"
b$ = "666666"
c$ = "88888888"
Put #1, , a$: Put #1, , b$: Put #1, , c$
Close #1
Open "crap" For Random As #1 Len = 10
Get #1, , aa$: Get #1, , bb$: Get #1, , cc$
Close #1
Print aa$, bb$, cc$

Open "crap" For Random As #1 Len = 10
Print "length of file = "; LOF(1); "This = 10 * 3 records!!!!!"
Close #1

[bplus]

To determine number of records divide LOF() by Len = (used in Open statement).