String Compression and Decompression

[John]

Peter on the BaCon forum is experimenting with string compression.

compression and decompression

Here is the Script BASIC solution.

Code: ScriptBasic

1. IMPORT t.bas
2. IMPORT zlib.bas
3.  
4. PRINT FORMAT("War & Peace uncompressed is %~#,###,###~ bytes. Compressed it's %~#,###,###~ bytes.\n", _
5.       FILELEN("warpeace.txt"), LEN(zlib::Compress(t::LoadString("warpeace.txt"))))
6.  

$ time scriba wpc.sb
War & Peace uncompressed is 3,202,941 bytes. Compressed it's 1,195,214 bytes.

real   0m0.385s
user   0m0.377s
sys   0m0.008s
$

[John]

Here is a file compress and uncompress example.

The file being compressed is deleted with the gzip() function, You may want to make a backup before playing with this.

You can use standard Linux utilities to uncompress files compressed with the zlib extension module.

Code: ScriptBasic

1. IMPORT zlib.bas
2.  
3. PRINT FILELEN("warpeace.txt"),"\n"
4. zlib::gzip "warpeace.txt", "warpeace.gz"
5. PRINT FILELEN("warpeace.gz"),"\n"
6. zlib::gunzip("warpeace.gz", "warpeace.txt")
7. PRINT FILELEN("warpeace.txt"),"\n"
8.  

$ time scriba unwpc.sb
3202941
1195226
3202941

real   0m0.450s
user   0m0.433s
sys   0m0.016s
$

[AIR]

Using MBC and libzip(uses zlib):

Code: Text

1. $MODULE ZIP.mbc
2. $EXECON
3.  
4.  
5. dim fname$,ret
6.  
7. if G_argc < 2 then
8.     print "usage: ",command$(0)," archive"
9.     end = 1
10. end if
11.  
12. fname$ = command$(1)
13.  
14. print E"Source file size:\t";lof%(fname$)/1000;"kB"
15.  
16. ret = Compress(fname$,"W&P.zip")
17. if ret then
18.     print E"Compressed file size:\t",ret/1000,"kB"
19. else
20.     end = 1
21. end if
22.  
23. ret = Extract("W&P.zip","UNPACKED-W&P.txt")
24. if ret then
25.     print E"Extracted  file size:\t",ret/1000,"kB"
26. end if

Code: [Select]

[riveraa@MacDev ~/Projects/mbc/compression] $ time ./ziptest "war and peace.txt"
Source file size: 3359kB
Compressed file size: 1220kB
Extracted  file size: 3359kB

real 0m0.417s
user 0m0.404s
sys 0m0.009s
AIR.

[John]

MBC sure looks easier to use than BaCon with its C helper dependencies.

[AIR]

USING GO

Code: Go

1. package main
2.  
3. import (
4.         "fmt"
5.         "log"
6.         "os"
7.  
8.         "github.com/dustin/go-humanize"
9.         "github.com/mholt/archiver"
10. )
11.  
12. func main() {
13.         // List of Files to Zip
14.         files := []string{"warpeace.txt"}
15.         output := "warpeace.zip"
16.         output2 := "warpeace.txt"
17.  
18.         if err := archiver.Archive(files, output); err != nil {
19.                 log.Fatal(err)
20.         }
21.  
22.         if err := archiver.Unarchive(output, "test"); err != nil {
23.                 log.Fatal(err)
24.         }
25.         fmt.Println("Source File Size:", humanize.Bytes(getFileSize("warpeace.txt")))
26.         fmt.Println("Compressed File Size:", humanize.Bytes(getFileSize(output)))
27.         fmt.Println("Extracted File Size:", humanize.Bytes(getFileSize("test/"+output2)))
28.  
29. }
30.  
31. func getFileSize(filename string) uint64 {
32.         file, err := os.Open(filename)
33.         if err != nil {
34.                 // handle the error here
35.                 return 0
36.         }
37.         defer file.Close()
38.  
39.         // get the file size
40.         stat, err := file.Stat()
41.         if err != nil {
42.                 return 0
43.         }
44.         return uint64(stat.Size())
45. }
46.  

OUTPUT:
Source File Size: 3.2 MB
Compressed File Size: 1.2 MB
Extracted File Size: 3.2 MB

real   0m0.248s
user   0m0.224s
sys   0m0.019s

AIR.

[jalih]

Here is 8th version...

Zipping and unzipping file:

Code: [Select]

: app:main
  "warpeace.txt" f:size nip "Source file size is: %d bytes\n" s:strfmt .
  f:zipnew "warpeace.txt" "warpeace.txt" f:zip+
  "warpeace.zip" f:zipsave

  "warpeace.zip" f:size nip "Compressed file size is: %d bytes\n" s:strfmt .

  "warpeace.zip" f:zipopen getcwd true f:unzip drop
  "warpeace.txt" f:size nip "Extracted file size is: %d bytes\n" s:strfmt .
  bye ;

Read text file into buffer, zip it and then unzip data back to buffer:

Code: [Select]

: app:main
  f:zipnew "warpeace.txt" f:slurp
  b:len "Source buffer size is: %d bytes\n" s:strfmt .
  "warpeace.txt" swap f:zip+
  "warpeace.zip" f:zipsave

  "warpeace.zip" f:size nip "Compressed file size is: %d bytes\n" s:strfmt .

  "warpeace.zip" f:zipopen
  "warpeace.txt" f:zip@ nip b:len nip "Extracted buffer size was: %d bytes\n" s:strfmt .
  bye ;


[John]

Your avatar says Mac Daddy but I'm seeing GO Daddy. 

[AIR]

Your avatar says Mac Daddy but I'm seeing GO Daddy. 

I'm coding GO on my MAC....

[AIR]

Jalih, thanks for your example, that's pretty cool!

So 8th automatically pipes the output of a command or file to the next command, right?

At first glance it's kind of cryptic, but when I thought about how you can pipe the output of a command to another command in shell scripting or Powershell, your code made perfect sense.

What does the trailing DOT mean, though?  I do like how concise/compact 8th code is, btw!

Without checking the docs, am I correct in thinking that f: is for file related functions and s: is for strings?

AIR.

[jalih]

So 8th automatically pipes the output of a command or file to the next command, right?

At first glance it's kind of cryptic, but when I thought about how you can pipe the output of a command to another command in shell scripting or Powershell, your code made perfect sense.

What does the trailing DOT mean, though?  I do like how concise/compact 8th code is, btw!

Yes, 8th is a stack based language. You use stack to pass data around. Global variables, word-local and task-local variables are supported but most of the power comes from factoring code into small words, composing them and using stack efficiently.

Dot is a Forth style print word, on my example it just prints the string.

Without checking the docs, am I correct in thinking that f: is for file related functions and s: is for strings?

You are right, those are namespaces and help to write more modular code.