Fast QB64 base64 encoder & decoder

[a740g]

RETIRED! See this post.

This is a Base64 encode, decode, and resource loader library based on MODP_B64, a C-based super-fast Base64 encoder and decoder library.

The library has 4 functions:

Code: (Select All)

''' @brief Converts a normal STRING or binary data to a base64 STRING.
''' @param src The normal STRING or binary data to be converted.
''' @return The base64 STRING.
FUNCTION Base64_Encode$ (src AS STRING)
 
''' @brief Converts a base64 STRING to a normal STRING or binary data.
''' @param src The base64 STRING to be converted.
''' @return The normal STRING or binary data.
FUNCTION Base64_Decode$ (src AS STRING)
 
''' @brief Loads a binary file encoded with Bin2Data (CONST).
''' @param src The base64 STRING containing the encoded data.
''' @param ogSize The original size of the data.
''' @param isComp Whether the data is compressed.
''' @return The normal STRING or binary data.
FUNCTION Base64_LoadResourceString$ (src AS STRING, ogSize AS _UNSIGNED LONG, isComp AS _BYTE)
 
''' @brief Loads a binary file encoded with Bin2Data (DATA).
''' Usage:
'''  1. Encode the binary file with Bin2Data.
'''  2. Include the file or its contents.
'''  3. Load the file like so:
'''      Restore label_generated_by_bin2data
'''      Dim buffer As String
'''      buffer = Base64_LoadResourceData
''' @return The normal STRING or binary data.
FUNCTION Base64_LoadResourceData$

It's much faster than my earlier QB64-only implementation which you can find here: Something Tricky (qb64phoenix.com).

Here are some numbers from my Ryzen 5600X box.



I threw in Win32 API-based Base64 encoder & decoder wrappers for the benchmark. However, the library is cross-platform.

The numbers in the image are with C++ optimizations turned on. However, even with optimizations turned off MODP_B64 runs circles around Win32 and my earlier implementation.

The Bin2Data tool lives here: https://qb64phoenix.com/forum/showthread.php?tid=2228

[Dav]

Nice implementation!  And As always your code is polished.

- Dav

[SpriggsySpriggs]

Makes me wonder how fast mine is.

https://qb64forum.alephc.xyz/index.php?t...#msg131690

EDIT: Hmm, very slow, also fails to decode the text from LOREM_IPSUM. Very odd. Never had any problems before and it worked with other Base64 strings that I had in the past. Was also a direct copy of a Rosetta task, so the algorithm might not be good. It fails on the "b4" line of the decode$ function.

2nd EDIT: Nice. If I dig up my Win32 version, it is super fast on my PC.
Samuel's algo:
   

Win32:
   

Code: (Select All)

 
Option _Explicit
 
Const ITERATIONS = 100000
Const LOREM_IPSUM = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."
 
Dim encTxt As String, decTxt As String, i As Long, t As Double
 
Print ITERATIONS; "iterations,"; Len(LOREM_IPSUM); "bytes."
 
Print "Base64 encode..."
 
t = Timer
For i = 1 To ITERATIONS
    encTxt = encodeBase64(LOREM_IPSUM)
Next
Print Using "#####.##### seconds"; Timer - t
 
Print "Base64 decode..."
 
t = Timer
For i = 1 To ITERATIONS
    decTxt = decodeBase64(encTxt)
Next
Print Using "#####.##### seconds"; Timer - t
 
If _StrCmp(decTxt, LOREM_IPSUM) = 0 Then
    Print "Passed"
Else
    Print "Failed"
End If
 
Declare Dynamic Library "Crypt32"
    Function CryptBinaryToString& Alias "CryptBinaryToStringA" (ByVal pbBinary As _Offset, Byval cbBinary As Long, Byval dwFlags As Long, Byval pszString As _Offset, Byval pcchString As _Offset)
    Function CryptStringToBinary& Alias "CryptStringToBinaryA" (ByVal pszString As _Offset, Byval cchString As Long, Byval dwFlags As Long, Byval pbBinary As _Offset, Byval pcbBinary As _Offset, Byval pdwSkip As _Offset, Byval pdwFlags As _Offset)
End Declare
 
Function encodeBase64$ (encode As String)
    Const CRYPT_STRING_NOCRLF = &H40000000
    Const CRYPT_STRING_BASE64 = &H00000001
 
    Dim a As Long
    Dim lengthencode As Long
    Dim encoded As String
    Dim lengthencoded As Long
 
    lengthencode = Len(encode)
 
    a = CryptBinaryToString(_Offset(encode), lengthencode, CRYPT_STRING_BASE64 Or CRYPT_STRING_NOCRLF, 0, _Offset(lengthencoded))
    'Calculate buffer length
    If a <> 0 Then
        encoded = Space$(lengthencoded)
    Else
        encodeBase64 = ""
        Exit Function
    End If
    a = CryptBinaryToString(_Offset(encode), lengthencode, CRYPT_STRING_BASE64 Or CRYPT_STRING_NOCRLF, _Offset(encoded), _Offset(lengthencoded))
    'Acual conversion
    If a <> 0 Then
        encodeBase64 = encoded
    Else
        encodeBase64 = ""
    End If
End Function
 
Function decodeBase64$ (decode As String)
    Const CRYPT_STRING_BASE64_ANY = &H00000006
 
    Dim a As Long
    Dim lengthdecode As Long
    Dim decoded As String
    Dim lengthdecoded As Long
 
    lengthdecode = Len(decode)
    a = CryptStringToBinary(_Offset(decode), lengthdecode, CRYPT_STRING_BASE64_ANY, 0, _Offset(lengthdecoded), 0, 0)
    'Calculate buffer length
    If a <> 0 Then
        decoded = Space$(lengthdecoded)
    Else
        decodeBase64 = ""
        Exit Function
    End If
    a = CryptStringToBinary(_Offset(decode), lengthdecode, CRYPT_STRING_BASE64_ANY, _Offset(decoded), _Offset(lengthdecoded), 0, 0)
    'Actual conversion
    If a <> 0 Then
        decodeBase64 = decoded
    Else
        decodeBase64 = ""
    End If
End Function

[a740g]

The library is updated with the MODP_B64 based encoder/decoder which gives 3x+ faster encode/decode performance.

Full update and download in the first post.

[a740g]

QB64-PE v4.1 has been released. See the announcement here. And with this release, I’ve decided to retire this library.

The built-in functions, _BASE64ENCODE$ and _BASE64DECODE$ simply decimates the competition. So much so, that it puts the Win32 API CryptBinaryToStringA and CryptStringToBinaryA to shame.

Below are some benchmarks. I've included the benchmark code in libbase64.zip\demos\base64\base64_demi.bas. Find the final zip in the first post. Cheers!

Without C++ optimization:


With C++ optimizations:

[Pete]

Hey Sam, @a740g

A quarter of a century ago I wrote my own encryption for office data. It worked great but by today's standards would be considered obsolete. My question is how involved would this be using this new QB64 encryption function to get even something simple like a password encrypted?

So say  want to encrypt my password to this forum:

password$ = "peteistremendousbutsteveisjustamazing"

How would I encrypt that with this new feature so, unlike the example in the wiki, someone can't just decode it with _BASE64DECODE$? I would think there should be a key involved.

Pete

[SMcNeill]

Hey Sam, @a740g

A quarter of a century ago I wrote my own encryption for office data. It worked great but by today's standards would be considered obsolete. My question is how involved would this be using this new QB64 encryption function to get even something simple like a password encrypted?

So say  want to encrypt my password to this forum:

password$ = "peteistremendousbutsteveisjustamazing"

How would I encrypt that with this new feature so, unlike the example in the wiki, someone can't just decode it with _BASE64DECODE$? I would think there should be a key involved.

Pete

https://qb64phoenix.com/qb64wiki/index.php/MD5$
https://qb64phoenix.com/qb64wiki/index.php/ADLER32
https://qb64phoenix.com/qb64wiki/index.php/CRC32

[Pete]

Your dollar sign didn't link. Even at 1/8 Jewish, I can link my dollar signs! I won't bother to quote it.

https://qb64phoenix.com/qb64wiki/index.php/MD5$

Why did it have to be snakes Bit-flipping? And...

Since I have 0 familiarity with any of those newer keywords, none of those tinker toys are going to fit together anytime soon.

Pete

[a740g]

Hey Sam, @a740g

A quarter of a century ago I wrote my own encryption for office data. It worked great but by today's standards would be considered obsolete. My question is how involved would this be using this new QB64 encryption function to get even something simple like a password encrypted?

So say  want to encrypt my password to this forum:

password$ = "peteistremendousbutsteveisjustamazing"

How would I encrypt that with this new feature so, unlike the example in the wiki, someone can't just decode it with _BASE64DECODE$? I would think there should be a key involved.

Pete

Well, you should not use _BASE64ENCODE$ or even _DEFLATE$ for encrypting sensitive data because these are not encryption methods. Anyone can trivially reverse them using _BASE64DECODE$ and _INFLATE$.

_MD5$, _ADLER32, and _CRC32 are not encryption but hashing/checksum functions and the results are not reversible.

As you rightly pointed out, you would need something that takes a key and then encrypts the data using the key. To reverse the process and get back the original data, you would have to use the same key. I think something like AES should do the job quite well.

I'll try to put together something over the weekend.

[SpriggsySpriggs]

Wait, we have hashing functions in QB64 now?