A Question on memory set aside for Zero

[Dimster]

So I was tooling around the other day with an old file that was filled with data. The data is all Dimmed Single Type. I have a routine which helps me pick out individual items of data which had been stored sequentially. The routine I was using to access the various bits of data uses SEEK (which is ideal for Random Access). I was also using a math formula to locate the data which at its heart uses 4 bytes for each data item. This has worked fine until I accessed this old file. The major difference with this old data file and most of the others I access is the value of 0 (zero). So the data in the sequential file looks like this:
56
0
0
0
0
 etc... there are actually over 2500 single type data items with about 40% of the file's data being zeros.

Some where along the line those 0 values were not 4 bytes apart. I'm thinking, either my computer now packs zero differently in memory or something may have changed in QB since I last used that particular file's data. Has anything changed with QB on the memory set aside for Zero?

[Petr]

Couldn't this just be because the Single values used are low? In that case, the resulting 4 bytes would also contain CHR$(0):

Code: (Select All)

Dim a As Single

a = 250

For b = 1 To 4

    Print Asc(MKS$(a), b)

Next

[SMcNeill]

Looks to me like what you're seeing is data that was written using PRINT and LINE INPUT rather than GET/PUT

56 + CRLF = 4 byte
0 + CRLF = 3 bytes... <-- these 0's are throwing your data off.

Sounds like you just need to:

DIM data_array(2500)
OPEN "data_file" FOR INPUT AS #1
DO UNTIL EOF(1)
    count = count + 1
    INPUT #1, data_array(count)
LOOP
CLOSE

---

For deeper analysis, you'd need to supply the actual file for reference, but from your post's example layout, I imagine that's more or less the issue.

[Dimster]

Thanks Petr, I would have thought the 4 bytes would have accounted for any padding after a low number like 56, for example if the sequential numbers were 56 75 then any padding betweer the 6 and 7 would have been accounted for ( ie Seek #1,1 get me the 5 and Seek #1,5 gets me the 7)

Steve, I can't recall what the code was that entered the data in to the sequential file but I almost always would have used Write #??. In your example it makes so much sense that running thru the sequential file with just an Input # does eventually get me to the location where the data is located in the file but I have been using Seek # to find the location and just pick out the data I need rather than constantly running thru it all for one item of data at a time. Random access is what I should have done with this data file.

While I'm not sure of the code, the array was likely MyArray(1 to 50, 0 to 44), which I'm thinking would contain 50 x 45 x 4  (9000) total bytes where everything should be stored?? Is that correct? Or would I need to add a CRLF factor to that math??

[SMcNeill]

It all depends on how the data was written to file.   GET/PUT doesn't have any sort of CRLF usually.   Of course, if you were using GET/PUT with SINGLE values, then they wouldn't be in human readable format.  SINGLE is 4-bytes, but 57 might liook like $#@! for those 4-bytes.  It certainly isn't going to loo like "57".

IF you used WRITE to create the file, then the file would be human readable as you describe, but you won't be able to use SEEK with it effectively.  The issue is the fact that those data are written in base-10, human-language format.

Take the value 12345, for example.  A SINGLE variable type will easily hold that value, but that's going to be 5-bytes in length and not 4.

The only way you could use SEEK with that data  file would to be to convert it first so that each data entry was the same number of bytes.

0057
0000
0123
4567

Something similar to the above would work as it'd be a set 4-bytes for data + CRLF, with the leading 0s working as placeholder for your spacing.

But, from what it looks like you have, and from how you speak of creating the file (using WRITE #), then your only option is just to read that file sequentially to get to the data you want.  Luckily, it doesn't sound as if it's an extremely large file, so I'd either just read it once and store it in an array, or else I'd just SEEK #file_handle, 1 and read it from the start of the file to the point where I get the data I need each time.  Your access times on something small like this isn't going to be that great overall, in the end, I don't think.  It's not like having to sequentially read millions of records to get to that one proper one. 

When you WRITE to a file, you need to INPUT from that file, and if the data is variable length (like it is here), you can't just use SEEK easily with it.

[Dimster]

Thank you Steven and Petr for the help

[Petr]

It occurred to me, when reading it again, you say it's supposed to be stored as a two-dimensional array. So if you use the binary approach:

Code: (Select All)

Dim Arr(1 To 50, 0 To 44) As Single

ff = FreeFile

Open "DataFile" For Binary As ff

Get ff, , Arr()

Close ff



Screen 13

For x = 1 To 50

    For y = 0 To 44

        PSet (x, y) 'just example...

Next y, x

[SMcNeill]

It occurred to me, when reading it again, you say it's supposed to be stored as a two-dimensional array. So if you use the binary approach:

Code: (Select All)

Dim Arr(1 To 50, 0 To 44) As Single

ff = FreeFile

Open "DataFile" For Binary As ff

Get ff, , Arr()

Close ff



Screen 13

For x = 1 To 50

    For y = 0 To 44

        PSet (x, y) 'just example...

Next y, x

That would only work if your data file was in the proper format for that.  This isn't.  The way it has to be read is more like:


Dim Arr(1 To 50, 0 To 44) As Single
ff = FreeFile
Open "DataFile" For INPUT As ff
For x = 1 To 50
For y = 0 To 44
INPUT #ff, arr(x, y)
Next y, x
Close

Then you'll have to file in the array and can access each individual element as wanted.

[Dimster]

Hi Steve, I think I have been misunderstanding the memory storage of 4 bytes per Single Type of variable. You point out

Take the value 12345, for example.  A SINGLE variable type will easily hold that value, but that's going to be 5-bytes in length and not 4.

 
but I have always been of the impression a variable Dimmed Single would always have a max of 7 digits and so the memory of 4 bytes would be enough to hold a 7 digit number. You are pointing out that a Single declared variable is actually 1 byte per digit. That could be exactly why my routine to search the sequential file is running into problems. For the most part, all the values stored in the sequential file are of 4 digits or less.

So to correct my search routine, if the search encounters a value greater than 4 digits, it needs to count the number of digits greater than 4 and add that value to the standard 4 bytes to find the location of the next data item. Does that sound like a logical fix to locating the beginning of the data entries in a sequential file where all data items have been dimmed Single?

[SMcNeill]

@Dimster Kindly read over the code below to get a quick understanding of it, and then run it as well.  The comments inside the code are made to go with the output that the program generates, and I hope they'll help clear up any issues you have about what's going on here, and how you probably need to proceed to do what you're wanting to do.

Code: (Select All)

Screen _NewImage(800, 600, 32)

_ControlChr Off

'Let's go with some DATA to work with, to illustrate what's going on.

example_data:

Data 1,2,3,4,5,6,7,8,9,0,0,0,0,0,0,0,11,12,13,14,15,123,1234,12345,123456,1234567,12345678,123456789,123457890,-1



'and let's make a SINGLE variable to hold that data

Dim MyData As Single



'Now, let's open a file a WRITE that data to file.

Open "temp.txt" For Output As #1

Restore example_data

Do

    Read MyData

    Write #1, MyData

Loop Until MyData = -1

Close



Print "Data written to 'temp.txt'.  Please take a moment and look at that data file and see how it looks."

Print "It should look like the following:"

Open "temp.txt" For Input As #1

For i = 1 To 30

    Input #1, MyData

    Print MyData

Next

Close



Print

Print "Press <ANY KEY> to continue."

Sleep



Cls

Print "Now, as you can see, that data WAS stored in a SINGLE, which is only 4-bytes in size,"

Print "but once we WRITE it to file, it gets printed in a human-readable format, which"

Print "varies in size due to the length of the data.  One can't simply SEEK to any record in"

Print "this file that they want to.  It needs to be read sequentially from start to finish."

Print

Print "IF one wants a RANDOM file so that they can SEEK to any of these particular"

Print "records instantly, then they must make the file without using WRITE and OUTPUT."

Print "Instead, they *must* use RANDOM (or BINARY) and GET/PUT."

Print

Print "Press <ANY KEY> to continue."

Sleep



'let me illustrate below for you:

'we'll use the same data as before, but this time I'm going to write it to a differnt file to compare.

Open "temp_RANDOM.txt" For Random As #1 Len = 4 'only one field, so my records only need to be 4-byte fields

Restore example_data

Do

    Read MyData

    Put #1, , MyData

Loop Until MyData = -1

Close



Cls

Print "Data written to 'temp_RANDOM.txt'.  Please take a moment and look at that data file and see how it looks."

Print "It should look like the following:"

Open "temp_RANDOM.txt" For Input As #1

Line Input #1, file_data$

Print file_data$

Close

Print

Print "Notice how that's not at all human-readable?  That's written in the same 4-byte"

Print "Pattern as a SINGLE is stored in memory."

Print "That data can be SEEKED to, as we know the size -- 4 bytes per record"

Print "OR we can use RANDOM to access it, as we do below for records #3, 7, and 17."

Open "temp_RANDOM.txt" For Random As #1 Len = 4 'only one field, so my records only need to be 4-byte fields

Get #1, 3, MyData: Print MyData

Get #1, 7, MyData: Print MyData

Get #1, 17, MyData: Print MyData

Close

Print

Print "Press <ANY KEY> to continue."

Sleep



Cls

Print "As you can tell by comparing the two data files and their structures,"

Print "the file created with WRITE is nothing like the file created with RANDOM."

Print "It would be more-or-less impossible to use SEEK with the file written with WRITE,"

Print "to just jump back and forth between data fields."

Print

Print "IF one wishes to use SEEK or RANDOM, then the file would need to be created in that manner"

Print "to begin with.  Files written FOR OUTPUT and using WRITE simply don't fit the proper format."

Print "Convert them with a quick and simply utility like the one included below."

Print

'Now, let's convert the first file to suit the second file's formatting style.

Open "temp.txt" For Input As #1 'This is our original file.

Open "temp_CONVERTED" For Random As #2 Len = 4 'the file we're going to convert to.

For i = 1 To 30

    Input #1, MyData 'read the original file which was written FOR OUTPUT

    Put #2, , MyData 'write it in the converted file FOR RANDOM access

Next

Close



'and now, to show off the data from that converted file, let's get the same three records as before

'and print them to the screen.  This is record #3, 7, and 17 in the converted file.

Open "temp_Converted" For Random As #1 Len = 4

Get #1, 3, MyData: Print MyData

Get #1, 7, MyData: Print MyData

Get #1, 17, MyData: Print MyData

Close

Print

Print "See the differences in working with a file written FOR OUTPUT with WRITE,"

Print "and a file written FOR RANDOM, with PUT?"

What you have, is a file opened FOR OUTPUT and created using WRITE to write the data to the file.  SEEK really isn't going to work for you, as you're going to have variable-length data by using WRITE to store the information on the file.

Converting a file of simple values like this from one file type (WRITE format) to another (RANDOM format) is a breeze, and it seems like that's what you really should do here.  Convert it over to the format you want and then use the converted file.  It's the simplest way to do what you're trying to do here, I think.