Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
WordList Load and Search
#2
(02-26-2026, 12:51 AM)SMcNeill Wrote: As we tend to go over this here on the forums repeatedly, I figured I'd share a simple version which works for anything I want to ever toss at it.

Code: (Select All)
Screen _NewImage(1024, 720, 32)
$Color:32

ReDim dict(0) As String 'make a resizeable array to hold our dictionary

t1 = Timer(0.001)
LoadWordList "Collins.txt", dict(), 300000
t2 = Timer(0.001)

Print Using "#.##### seconds to load ###,### words."; t2 - t1, UBound(dict)
For i = 1 To 10
    Print dict(i),
Next
Print
For i = UBound(dict) - 10 To UBound(dict)
    Print dict(i),
Next
Print

'and just to showcase how quick arrays are, let's do a random search of the array from start to bottom
For i = 1 To 1000000 'let's look for 1000000 random words
    word$ = Chr$(65 + Rnd * 26) + Chr$(65 + Rnd * 26) + Chr$(65 + Rnd * 26) + Chr$(65 + Rnd * 26) + Chr$(65 + Rnd * 26) + Chr$(65 + Rnd * 26)
    If FindWord(dict(), word$) Then
        Color Yellow: Print "MATCH:"; word$,: Color White
    Else
        'Print "No match:"; word$,
    End If
Next
Print
Print
Print Using "###.#### seconds to load, parse, search 1,000,000 words, find results, and print them -- TOTAL!!"; Timer(0.001) - t1

Function FindWord (WordList() As String, word As String)
    Dim As Long min, max, p 'min, max, and current search position
    min = 1
    max = UBound(WordList)
    p = (min + max) \ 2
    Do Until p < min _OrElse p > max
        Select Case _StriCmp(word, WordList(p))
            Case -1: max = p - 1
            Case 0: FindWord = p: Exit Function 'we found the word!
            Case 1: min = p + 1
        End Select
        p = (min + max) \ 2
    Loop
End Function

Sub LoadWordList (file$, WordList() As String, Limit As Long) 'this sub loads a list of words for use later
    $Checking:Off
    Dim As String temp, t1
    Dim As Long count, p, p1
    ReDim WordList(Limit) As String 'let's make a nice large array to told the words.  Set the limit you want for yourself
    If _FileExists(file$) Then 'then we have a found word list.  Let's load and parse it
        temp = _ReadFile$(file$)
        p = 1
        Do
            p1 = InStr(p, temp, Chr$(10)) 'look for a chr$(10) end of line marker
            If p1 = 0 Then p1 = InStr(p, temp, Chr$(13)) 'if no chr$(10) then look for a chr$(13) for odd files with it as the CRLF
            If p1 Then 'then we have a delimiter
                t1 = _Trim$(Mid$(temp, p, p1 - p))
                If Right$(t1, 1) = Chr$(13) Then t1 = Left$(t1, Len(t1) - 1) 'if CRLF then strip off chr$(13)
                If t1 <> "" Then 'don't add blank lines to the list
                    count = count + 1
                    WordList(count) = t1
                End If
                p = p1 + 1
            End If
        Loop Until p1 = 0
        If p < Len(temp) Then 'if there's no CRLF for the end of file, we want to last word here
            t1 = Mid$(temp$, p)
            If t1 <> "" Then 'again, don't add if it's a blank line
                count = count + 1
                WordList(count) = Mid$(temp, p)
            End If
        End If
    End If
    ReDim _Preserve WordList(count) As String
    $Checking:On
End Sub

This little routine loads the collins dictionary from below, which has over 279k words or so in it.  Then it makes a series of words out of random letters -- a *MILLION* words, created on the fly, one by one.  Then it searches to determine if the word it hobbled together exists or not.  And then it prints out the words that it created that DO match one in the dictionary.

And best of all?  It times itself as it does all these different things.

And how long does it take to load a 279,000 word dictionary, parse it, search it a million times, and print the results to screen?

About half a second on my laptop.

This is simple.  It's fast and efficient.  Why would anyone need anything else?

Whenever someone needs a way to load a wordlist and search it from now on, just remember to point them to this post here and say, "There you go.  That's all you need."

Nice! and very fast!!! (but)
Can you also show how to call the FindWord function? Probably self-evident to most, but I'm not "most", sorry  Blush
Of all the places on Earth, and all the planets in the Universe, I'd rather live here (Perth, Western Australia.) Big Grin
Please visit my Website at: http://oldendayskids.blogspot.com/
Reply


Messages In This Thread
WordList Load and Search - by SMcNeill - 02-26-2026, 12:51 AM
RE: WordList Load and Search - by PhilOfPerth - 02-26-2026, 05:00 AM
RE: WordList Load and Search - by SMcNeill - 02-26-2026, 05:09 AM
RE: WordList Load and Search - by PhilOfPerth - 02-26-2026, 05:30 AM
RE: WordList Load and Search - by SMcNeill - 02-26-2026, 05:45 AM
RE: WordList Load and Search - by SMcNeill - 02-26-2026, 05:45 AM

Forum Jump:


Users browsing this thread: