AllBASIC

BASIC User Group => Code Challenges => Topic started by: AIR on November 24, 2018, 04:22:16 pm

Title: JSON Challenge
Post by: AIR on November 24, 2018, 04:22:16 pm
Time for a new challenge:

Using the attached json file, which was pulled from github and shows commit status, show how your language of choice would extract the following fields:


Using Libraries/Built in JSON support is allowed; I thought about restricting this to only "roll your own" implementations, but wanted to make this challenge accessible to a wider audience, since coding a home-made parser might be beyond the reach of most, especially time-wise.

AIR.
Title: Re: JSON Challenge
Post by: John on November 24, 2018, 04:44:19 pm
I'm going to try native SB syntax to create a BASIC only extension json module. Using an external library for something based on a limited definition seems like overkill to me. My SB XML parser was less than 10 lines.
Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 06:12:04 pm
I'm going to try native SB syntax to create a BASIC only extension json module. Using an external library for something based on a limited definition seems like overkill to me. My SB XML parser was less than 10 lines.

Where simple text parsers become not so simple is when you have to handle deeply nested "objects", malformed data sets, and validation of those sets. Also you may need to account for UTF-8.

JSON is not just about brackets; there's a bunch of other things that the official spec supports.

With all that said, go for it! 

AIR.
Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 06:26:00 pm
One of my personal goals with this challenge is to implement an interface that makes it easy to grab the data you need.

For me, it's not only about can the challenge be met, but also about how simple it is for a programmer to use what you've created for other purposes.

With that said, he's what I worked up this evening, using HOTBASIC:

Code: Text
  1. '-------------------------------------------------------
  2. ' DATE: 11/24/2018
  3. ' NAME: parse.bas
  4. ' DESCRIPTION: JSON Parsing demo, using libparson
  5. ' AUTHOR:  Armando I. Rivera (AIR)
  6. ' LANGUAGE:  HOTBASIC! v7.1
  7. ' EDITOR:         AIRSyS-IDE
  8. '-------------------------------------------------------
  9.  
  10. $apptype console: $typecheck on
  11.  
  12. $include "JSON.bas"
  13.  
  14. deflng item, i
  15. dim json as JSON
  16.  
  17. json.init("commits.json")
  18.  
  19. if json.type = JSONArray then
  20.         for i = 0 to json.count-1
  21.                 item = json.object(i)
  22.                 print string$(50,"-")
  23.                 print "Name: "; json.text(item,"commit.author.name")
  24.                 print "Commit Date: "; json.text(item, "commit.author.date")
  25.                 print "SHA Hash: "; json.text(item, "sha")
  26.         next
  27.         print string$(50,"-")
  28. end if
  29.  
  30. end
  31.  

The JSON.bas file consists of:
Code: Text
  1. '-------------------------------------------------------
  2. '   NAME:                       JSON.bas
  3. '   Version:            1.0
  4. '   DATE:                       2018-11-24
  5. '-------------------------------------------------------
  6. ' DEVELOPMENT ENVIRONMENT
  7. '   OpSys:                      Windows
  8. '   Language:   HotBasic
  9. '   Editor:                     AIRSyS-IDE
  10. '   Libraries:          libparson.dll
  11. '-------------------------------------------------------
  12. ' AUTHOR:               Armando I. Rivera (AIR)
  13. '-------------------------------------------------------
  14. ' DESCRIPTION:
  15. '                                               libparson Json parser interface
  16. '
  17. '-------------------------------------------------------
  18. ' Terms of Use/Licence
  19. '                                               MIT
  20. '-------------------------------------------------------
  21.  
  22. declare sub JSONinit (filename as string)
  23. declare function JSONtext(json_object as long,query as string) as string
  24. declare function JSONobject(index as long) as long
  25.  
  26. OBJECT JSON
  27.         root as long
  28.         items as long
  29.         count as long
  30.         type as long
  31.         text as Function
  32.         object as function
  33.         init as Sub
  34. END OBJECT
  35.  
  36. ' JSON VALUE TYPE CODES
  37. CONST    JSONError      =       -1
  38. CONST    JSONNull               =       1
  39. CONST    JSONString     =       2
  40. CONST    JSONNumber     =       3
  41. CONST    JSONObject     =       4
  42. CONST    JSONArray      =       5
  43. CONST    JSONBoolean     =      6
  44.  
  45. begin runonce
  46.         cdecl function json_parse_file lib "libparson.dll"   (filename as string) as long
  47.         cdecl function json_parse_string lib "libparson.dll" (strBuffer as string) as long
  48.         cdecl function json_value_get_type  lib "libparson.dll"   (json_object as long) as long
  49.         cdecl function json_value_get_array  lib "libparson.dll"   (json_object as long) as long
  50.         cdecl function json_array_get_count  lib "libparson.dll"   (json_object as long) as long
  51.         cdecl function json_object_get_string lib "libparson.dll" (json_object as long, keyname as string) as string
  52.         cdecl function json_object_dotget_string lib "libparson.dll"  (json_object as long, keypath as string) as string
  53.         cdecl function json_array_get_object lib "libparson.dll" (json_object as long, index as long) as long
  54.         cdecl function json_object_dotget_value lib "libparson.dll" (json_object as long, keypath as string) as long
  55.         cdecl function json_object_get_value lib "libparson.dll" (json_object as long, keypath as string) as long
  56. end runonce
  57.  
  58. Sub JSONinit(filename as string)
  59.         self.root = json_parse_file(filename)
  60.         self.type = json_value_get_type(self.root)
  61.         if self.type = JSONArray then
  62.                 self.items = json_value_get_array(self.root)
  63.                 self.count = json_array_get_count(self.items)
  64.         end if
  65. End Sub
  66.  
  67. function JSONtext(json_object as long, query as string) as string
  68.         defstr res
  69.        
  70.         if instr(query, ".") then
  71.                 if  json_object_dotget_value(json_object,query) <> 0 then
  72.                         res = json_object_dotget_string(json_object, query)
  73.                 end if
  74. '               print res
  75.         else
  76.                 if  json_object_get_value(json_object,query) <> 0 then
  77.                         res = json_object_get_string(json_object, query)
  78.                 end if
  79.         end if
  80.        
  81.         result = res
  82. end function
  83.  
  84. function JSONobject(index as long) as long
  85.         result =  json_array_get_object(self.items, index)
  86. end function
  87.  
  88.  

Attached you find the output, and the 32bit dll (HOTBASIC is 32bit only)

AIR.
Title: Re: JSON Challenge
Post by: John on November 24, 2018, 06:45:47 pm
Script BASIC

Code: Script BASIC
  1. OPEN "commits.json" FOR INPUT AS #1
  2. fl = FILELEN("commits.json")
  3. json = INPUT(fl,1)
  4. SPLITA json BY _
  5. """  {
  6.    "sha": """ _
  7. TO commits
  8. FOR i = 1 TO UBOUND(commits)
  9.   IF commits[i] LIKE """*committer*"name": "*"*date": "*"*""" THEN
  10.    PRINT JOKER(3),"\n"
  11.    PRINT JOKER(5),"\n"
  12.    PRINT MID(commits[i],2,40),"\n"
  13.    PRINT STRING(40,"="),"\n"
  14.  END IF
  15. NEXT
  16.  


jrs@jrs-laptop:~/sb/abcc/json$ time scriba jsoncc.sb
Linus Torvalds
2018-11-24T20:58:47Z
e195ca6cb6f21633e56322d5aa11ed59cdb22fb2
========================================
Linus Torvalds
2018-11-24T17:42:32Z
d146194f31c96f9b260c5a1cf1592d2e7f82a2e2
========================================
Linus Torvalds
2018-11-24T17:19:38Z
857fa628bbe93017c72ddd0d5304962a2608db07
========================================
Linus Torvalds
2018-11-24T17:11:52Z
abe72ff4134028ff2189d29629c40a40bee0a989
========================================
David S. Miller
2018-11-24T06:35:38Z
07093b76476903f820d83d56c3040e656fb4d9e3
========================================
David S. Miller
2018-11-24T06:34:40Z
3fa528b7682e73e906266bcd43728b8f923bf9b2
========================================
David S. Miller
2018-11-24T06:33:55Z
e7b9fb4f545b1f7885e7c642643828f93d3d79c9
========================================
David S. Miller
2018-11-24T06:31:56Z
ef2a7cf1d8831535b8991459567b385661eb4a36
========================================
David S. Miller
2018-11-24T01:24:24Z
c44c749d3b6fdfca39002e7e48e03fe9f9fe37a3
========================================
David S. Miller
2018-11-24T01:18:15Z
5ed9dc99107144f83b6c1bb52a69b58875baf540
========================================
David S. Miller
2018-11-23T19:59:40Z
18ba58e1c234ea1a2d9835ac8c1735d965ce4640
========================================
David S. Miller
2018-11-23T19:59:40Z
e59ff2c49ae16e1d179de679aca81405829aee6c
========================================
Linus Torvalds
2018-11-23T19:24:55Z
7c98a42618271210c60b79128b220107d35938d9
========================================
Linus Torvalds
2018-11-23T19:20:14Z
3381918fec9278d14f776d1dabd68da85fd6822e
========================================
David S. Miller
2018-11-23T19:20:02Z
484afd1bd3fc6f9f5347289fc8b285aa65f67054
========================================
David S. Miller
2018-11-23T19:18:53Z
605108acfe6233b72e2f803aa1cb59a2af3001ca
========================================
David S. Miller
2018-11-23T19:17:56Z
896585d48e8e9ba44cd1754fbce8537feffcc1a5
========================================
Linus Torvalds
2018-11-23T19:15:27Z
d88783b9c8849d88c3a75b7b9071cba072b47eba
========================================
David S. Miller
2018-11-23T19:08:03Z
5cd8d46ea1562be80063f53c7c6a5f40224de623
========================================
Linus Torvalds
2018-11-23T18:56:16Z
a03bac580ae743d5900af626ac63f7f8cd85def9
========================================
Linus Torvalds
2018-11-23T18:52:57Z
b88af994872418f0a98db6f4a9bae849315a99b0
========================================
Catalin Marinas
2018-11-23T18:44:16Z
4f9f49646a5733c0c2bd49940673dde89a9c5add
========================================
Linus Torvalds
2018-11-23T18:40:19Z
e6005d3c42336074de3745718ac85807dd6e1e6a
========================================
Linus Torvalds
2018-11-23T18:36:02Z
dcd3aa31dcdd6d8eae8d4771c44aeb3b1fec995a
========================================
Linus Torvalds
2018-11-23T18:03:08Z
9b7c880c834c0a1c80a1dc6b8a0b19155361321f
========================================
Catalin Marinas
2018-11-23T17:33:27Z
b5d9a07ef7736b2456b9d3c90568de25e43d8ec3
========================================
Rafael J. Wysocki
2018-11-23T09:32:49Z
1d50088ca3956e5dcd2751a658e7869b9af10bb4
========================================
Rafael J. Wysocki
2018-11-23T09:32:22Z
bec00cb5e97c19d2c8bd10db15d334cb40760000
========================================
Dave Airlie
2018-11-23T01:03:21Z
98c9cdfd34fbb62886e4c5a07e33661aa3352ef5
========================================
David S. Miller
2018-11-22T19:53:26Z
039e70a70c8417b5bf5878a60612ebd2c95f731e
========================================

real   0m0.027s
user   0m0.015s
sys   0m0.011s
jrs@jrs-laptop:~/sb/abcc/json$


Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 07:10:19 pm
Very nice, but you're pulling the wrong SHA hash.
Title: Re: JSON Challenge
Post by: John on November 24, 2018, 08:19:47 pm
Fixed!

I changed my SPLITA pattern which reduced the number of LIKE checks in half making it a bit faster. I think I got the right SHA hash you were looking for.
Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 08:47:33 pm
Yep, that does it all right!
Title: Re: JSON Challenge
Post by: John on November 24, 2018, 08:51:41 pm
Maybe for extra points to make this challenging you could ask for a json generator from a SQLite DB. Then I could show off SB's external pre-processor tricks.
Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 08:53:29 pm
I'm waiting on jalih to chime in.   ;D ;D

Here's a POWERSHELL version:

Code: PowerShell
  1. $jsondata =  Get-Content -Raw -Path .\commits.json | ConvertFrom-Json
  2. $jsondata | % {$_.commit.author.name; $_.commit.author.date; $_.sha; ('-'*40)}

Linus Torvalds
2018-11-24T20:58:47Z
e195ca6cb6f21633e56322d5aa11ed59cdb22fb2
----------------------------------------
Linus Torvalds
2018-11-24T17:42:32Z
d146194f31c96f9b260c5a1cf1592d2e7f82a2e2
----------------------------------------
Linus Torvalds
2018-11-24T17:19:38Z
857fa628bbe93017c72ddd0d5304962a2608db07
----------------------------------------
Linus Torvalds
2018-11-24T17:11:52Z
abe72ff4134028ff2189d29629c40a40bee0a989
----------------------------------------
Andreas Fiedler
2018-11-23T23:16:34Z
07093b76476903f820d83d56c3040e656fb4d9e3
----------------------------------------
Quentin Schulz
2018-11-23T18:01:51Z
3fa528b7682e73e906266bcd43728b8f923bf9b2
----------------------------------------
Fabio Estevam
2018-11-23T17:46:50Z
e7b9fb4f545b1f7885e7c642643828f93d3d79c9
----------------------------------------
Lorenzo Bianconi
2018-11-23T17:28:01Z
ef2a7cf1d8831535b8991459567b385661eb4a36
----------------------------------------
Yangtao Li
2018-11-22T12:34:41Z
c44c749d3b6fdfca39002e7e48e03fe9f9fe37a3
----------------------------------------
Hangbin Liu
2018-11-22T08:15:28Z
5ed9dc99107144f83b6c1bb52a69b58875baf540
----------------------------------------
Jason Wang
2018-11-22T06:36:31Z
18ba58e1c234ea1a2d9835ac8c1735d965ce4640
----------------------------------------
Jason Wang
2018-11-22T06:36:30Z
e59ff2c49ae16e1d179de679aca81405829aee6c
----------------------------------------
Linus Torvalds
2018-11-23T19:24:55Z
7c98a42618271210c60b79128b220107d35938d9
----------------------------------------
Linus Torvalds
2018-11-23T19:20:14Z
3381918fec9278d14f776d1dabd68da85fd6822e
----------------------------------------
Davide Caratti
2018-11-21T17:23:53Z
484afd1bd3fc6f9f5347289fc8b285aa65f67054
----------------------------------------
Paolo Abeni
2018-11-21T17:21:35Z
605108acfe6233b72e2f803aa1cb59a2af3001ca
----------------------------------------
Hangbin Liu
2018-11-21T13:52:33Z
896585d48e8e9ba44cd1754fbce8537feffcc1a5
----------------------------------------
Linus Torvalds
2018-11-23T19:15:27Z
d88783b9c8849d88c3a75b7b9071cba072b47eba
----------------------------------------
Willem de Bruijn
2018-11-20T18:00:18Z
5cd8d46ea1562be80063f53c7c6a5f40224de623
----------------------------------------
Linus Torvalds
2018-11-23T18:56:16Z
a03bac580ae743d5900af626ac63f7f8cd85def9
----------------------------------------
Linus Torvalds
2018-11-23T18:52:57Z
b88af994872418f0a98db6f4a9bae849315a99b0
----------------------------------------
Will Deacon
2018-11-21T15:07:00Z
4f9f49646a5733c0c2bd49940673dde89a9c5add
----------------------------------------
Linus Torvalds
2018-11-23T18:40:19Z
e6005d3c42336074de3745718ac85807dd6e1e6a
----------------------------------------
Linus Torvalds
2018-11-23T18:36:02Z
dcd3aa31dcdd6d8eae8d4771c44aeb3b1fec995a
----------------------------------------
Linus Torvalds
2018-11-23T18:03:08Z
9b7c880c834c0a1c80a1dc6b8a0b19155361321f
----------------------------------------
Sergey Matyukevich
2018-11-16T18:21:30Z
b5d9a07ef7736b2456b9d3c90568de25e43d8ec3
----------------------------------------
Rafael J. Wysocki
2018-11-23T09:32:49Z
1d50088ca3956e5dcd2751a658e7869b9af10bb4
----------------------------------------
Rafael J. Wysocki
2018-11-23T09:32:22Z
bec00cb5e97c19d2c8bd10db15d334cb40760000
----------------------------------------
Dave Airlie
2018-11-23T01:03:20Z
98c9cdfd34fbb62886e4c5a07e33661aa3352ef5
----------------------------------------
David S. Miller
2018-11-22T19:53:26Z
039e70a70c8417b5bf5878a60612ebd2c95f731e
----------------------------------------



AIR.
Title: Re: JSON Challenge
Post by: John on November 24, 2018, 09:33:19 pm
Wow!

A double trailer semi. (PowerHell & JavaShit)  :(
Title: Re: JSON Challenge
Post by: John on November 24, 2018, 10:11:29 pm
SB.js

Script BASIC does JavaShit as well.

Code: Script BASIC
  1. IMPORT js.inc
  2.  
  3. jscode = """
  4. JSON.stringify( { 'foo':1, 'bar':2, 'baz':{ 'quux':3 } }, null, '\t');
  5. """
  6.  
  7. JS::CREATE
  8. PRINT JS::EXEC(jscode),"\n"
  9. JS::DESTROY
  10.  


jrs@jrs-laptop:~/sb/examples/js$ time scriba js_json.sb
{"baz":{"quux":3},"bar":2,"foo":1}

real   0m0.034s
user   0m0.030s
sys   0m0.004s
jrs@jrs-laptop:~/sb/examples/js$
Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 11:05:47 pm
Couple of things regarding the SB submission:

1) You're keying on "committer", which can be different from the person who actually authored the code.

2) If you wanted to filter on say the logon of the person who performed the commit, you would have to alter/add your like statement.

3) If you wanted to add, say, the commit message you'd have to alter/add to your like statement.

While your submission gets the data as I originally set in the challenge, your solution has an issue with flexibility.  If, for example, the source file changes, you have to go back and redo your lookup code.

With my HB submission, altering the code to do steps 2/3 above is as simple as:

Code: Text
  1.                 if json.text(item, "author.login") = "torvalds" then
  2.                         print string$(50,"-")
  3.                         print "Name: "; json.text(item,"commit.author.name")
  4.                         print "Commit Date: "; json.text(item, "commit.author.date")
  5.                         print "SHA Hash: "; json.text(item, "sha")
  6.                         print "Commit Message: "; json.text(item,"commit.message")
  7.                 end if

I didn't have to modify the MODULE code, I just added the filter and the commit message to the main program.

The same thing with PowerShell:
Code: PowerShell
  1. $jsondata =  Get-Content -Raw -Path .\commits.json | ConvertFrom-Json
  2. $jsondata | % {
  3.     if ($_.author.login -eq "torvalds") {
  4.         $_.commit.author.name; $_.commit.author.date; $_.sha; $_.commit.message;('-'*40);
  5.     }
  6. }
  7.  

Note that I'm not putting down your submission, I'm just pointing out some of the issues that can arise when you do a manual text extraction.

AIR.
Title: Re: JSON Challenge
Post by: AIR on November 24, 2018, 11:07:50 pm
SB.js

Script BASIC does JavaShit as well.

Code: Script BASIC
  1. IMPORT js.inc
  2.  
  3. jscode = """
  4. JSON.stringify( { 'foo':1, 'bar':2, 'baz':{ 'quux':3 } }, null, '\t');
  5. """
  6.  
  7. JS::CREATE
  8. PRINT JS::EXEC(jscode),"\n"
  9. JS::DESTROY
  10.  


jrs@jrs-laptop:~/sb/examples/js$ time scriba js_json.sb
{"baz":{"quux":3},"bar":2,"foo":1}

real   0m0.034s
user   0m0.030s
sys   0m0.004s
jrs@jrs-laptop:~/sb/examples/js$


Then you should have no problem using js to properly parse json.

AIR.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 01:32:23 am
Quote from: AIR
Name of Commiter

The challenge is very specific. Read a json file and extract the 3 fields you specified. My script does just that. It is not a generic json library routine. I really don't think maintaining the code isn't very difficult to return the data you're looking for.

Quote
Then you should have no problem using js to properly parse json.

The JavaScript library I'm using for the extension module is no longer maintained and deprecated for a direction I have no interest in. It doesn't work correctly on Windows. It's just my personal JavaScript toy for Linux.
Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 01:45:29 am
My comments are based on what you've stated before about wanting to create a json module for SB.

As a module, your current code won't hold up.

I could have done the same as you and written something that would meet the criteria in just a few lines of code.  But I'm looking to actually re-use what I've coded, and doing it like you've done would not have allowed me to do that without having to change it for each use.

At any rate, you've met the challenge requirements.  If that's enough for you, then that's fine.  But as long as you've known me, I've always tried to push you past the point of "it's good enough".

AIR.
Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 02:08:06 am
PYTHON VERSION

Code: Python
  1. import json
  2.  
  3. datafile = open("commits.json").read()
  4. for item in json.loads(datafile):
  5.     print "-"*40
  6.     print item["commit"]["author"]["name"]
  7.     print item["commit"]["author"]["date"]
  8.     print item["sha"]
  9. print "-"*40
  10.  

AIR.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 02:28:41 am
I'll submit another entry using the jsmn library in SB.
Title: Re: JSON Challenge
Post by: jalih on November 25, 2018, 02:51:14 am
I'm waiting on jalih to chime in.   ;D ;D

Following 8th code should do the trick:

Code: [Select]
"commits.json" f:slurp json> var, data

data @ ( nip "commit" m:@ "author" m:@ nip "name" m:@ . cr "date" m:@ nip . cr "sha" m:@ nip . cr "-" 50 s:* . cr ) a:each drop
Title: Re: JSON Challenge
Post by: jalih on November 25, 2018, 08:48:02 am
Here is a little bit fancier version using simple GUI for displaying data.

Code: [Select]
true app:isgui !

"commits.json" f:slurp json> var, data
a:new var, items
var gui

{
  "kind" : "win",
  "buttons" : 5,
  "title" : "JSON Challenge",
  "wide" : 640,
  "high" : 480,
  "center" : true,
  "children" :
  [
    {
      "kind" : "table",
      "name" : "table",
      "bg" : "lightgreen",
      "hl" : "lightgray",
      "hlc" : "gray",
      "bounds" : "10,10,parent.width-10, top+460",
      "header" : [
                   { "name" : "Name of committer", "width":160 },
                   { "name" : "Date of commit", "width":160 },
                   { "name" : "The SHA HASH for commit", "width":280 }
                 ]
    }
  ]
} var, gui-desc


: app:main
  data @ ( nip "commit" m:@ "author" m:@ nip "name" m:@ swap "date" m:@ nip rot "sha" m:@ nip 3 a:close items @ swap a:push drop ) a:each drop
  gui-desc @ "children" m:@ nip 0 a:@ nip "items" items @ m:! drop
  gui-desc @ g:new gui ! ;
Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 11:52:54 am
Very Nice, Jali!!!!!
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 12:02:51 pm
Why these challenges are so beneficial to all.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 02:16:25 pm
Quote from: AIR
My comments are based on what you've stated before about wanting to create a json module for SB.

My almost instantaneous submission was based on my comment about writing an XML parser in 10 lines.

I feel it would be more beneficial to release your json (jsmn) extension module in a Linux and Windows version.

Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 04:27:47 pm
That's your call, it's not quite finished yet but the core is there.  You could put it out to get feedback, at least.

I actually prefer the way Parson implements the api for json parsing.  It's much easier to port and use, as the HB submission here shows.

Had I found it earlier, I would have based the SB JSON module on it.

AIR.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 05:25:14 pm
Thanks for the update on your current SB json efforts. I'll give Parson a peek before generating a json extension module.

I might use DLLC's FFI (DYC on steroids)  under Windows to prototype the libparson SB interface before doing it in C BASIC as a json extension module.

While on the FFI topic, that would be a nice code challenge. I would love to turn Charles's DLLC FFI into a C extension module that is cross platform.
Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 06:58:54 pm
Code: Script BASIC
  1. INCLUDE json.bas
  2.  
  3. jarray = json::load("commits.json")
  4. for i = 0 to json::count(jarray)-1
  5.     obj = json::object(jarray,i)
  6.     print string(40,"-"),"\n"
  7.     print json::text(obj,"commit.author.name"),"\n"
  8.     print json::text(obj,"commit.author.date"),"\n"
  9.     print json::text(obj,"sha"),"\n"
  10. next
  11. print string(40,"-"),"\n"
  12.  

[riveraa@MacDev ~/tmp/sb/tests] $ scriba parse.bas
----------------------------------------
Linus Torvalds
2018-11-24T20:58:47Z
e195ca6cb6f21633e56322d5aa11ed59cdb22fb2
----------------------------------------
Linus Torvalds
2018-11-24T17:42:32Z
d146194f31c96f9b260c5a1cf1592d2e7f82a2e2
----------------------------------------
Linus Torvalds
2018-11-24T17:19:38Z
857fa628bbe93017c72ddd0d5304962a2608db07
----------------------------------------
Linus Torvalds
2018-11-24T17:11:52Z
abe72ff4134028ff2189d29629c40a40bee0a989
----------------------------------------
Andreas Fiedler
2018-11-23T23:16:34Z
07093b76476903f820d83d56c3040e656fb4d9e3
----------------------------------------
Quentin Schulz
2018-11-23T18:01:51Z
3fa528b7682e73e906266bcd43728b8f923bf9b2
----------------------------------------
Fabio Estevam
2018-11-23T17:46:50Z
e7b9fb4f545b1f7885e7c642643828f93d3d79c9
----------------------------------------
Lorenzo Bianconi
2018-11-23T17:28:01Z
ef2a7cf1d8831535b8991459567b385661eb4a36
----------------------------------------
Yangtao Li
2018-11-22T12:34:41Z
c44c749d3b6fdfca39002e7e48e03fe9f9fe37a3
----------------------------------------
Hangbin Liu
2018-11-22T08:15:28Z
5ed9dc99107144f83b6c1bb52a69b58875baf540
----------------------------------------
Jason Wang
2018-11-22T06:36:31Z
18ba58e1c234ea1a2d9835ac8c1735d965ce4640
----------------------------------------
Jason Wang
2018-11-22T06:36:30Z
e59ff2c49ae16e1d179de679aca81405829aee6c
----------------------------------------
Linus Torvalds
2018-11-23T19:24:55Z
7c98a42618271210c60b79128b220107d35938d9
----------------------------------------
Linus Torvalds
2018-11-23T19:20:14Z
3381918fec9278d14f776d1dabd68da85fd6822e
----------------------------------------
Davide Caratti
2018-11-21T17:23:53Z
484afd1bd3fc6f9f5347289fc8b285aa65f67054
----------------------------------------
Paolo Abeni
2018-11-21T17:21:35Z
605108acfe6233b72e2f803aa1cb59a2af3001ca
----------------------------------------
Hangbin Liu
2018-11-21T13:52:33Z
896585d48e8e9ba44cd1754fbce8537feffcc1a5
----------------------------------------
Linus Torvalds
2018-11-23T19:15:27Z
d88783b9c8849d88c3a75b7b9071cba072b47eba
----------------------------------------
Willem de Bruijn
2018-11-20T18:00:18Z
5cd8d46ea1562be80063f53c7c6a5f40224de623
----------------------------------------
Linus Torvalds
2018-11-23T18:56:16Z
a03bac580ae743d5900af626ac63f7f8cd85def9
----------------------------------------
Linus Torvalds
2018-11-23T18:52:57Z
b88af994872418f0a98db6f4a9bae849315a99b0
----------------------------------------
Will Deacon
2018-11-21T15:07:00Z
4f9f49646a5733c0c2bd49940673dde89a9c5add
----------------------------------------
Linus Torvalds
2018-11-23T18:40:19Z
e6005d3c42336074de3745718ac85807dd6e1e6a
----------------------------------------
Linus Torvalds
2018-11-23T18:36:02Z
dcd3aa31dcdd6d8eae8d4771c44aeb3b1fec995a
----------------------------------------
Linus Torvalds
2018-11-23T18:03:08Z
9b7c880c834c0a1c80a1dc6b8a0b19155361321f
----------------------------------------
Sergey Matyukevich
2018-11-16T18:21:30Z
b5d9a07ef7736b2456b9d3c90568de25e43d8ec3
----------------------------------------
Rafael J. Wysocki
2018-11-23T09:32:49Z
1d50088ca3956e5dcd2751a658e7869b9af10bb4
----------------------------------------
Rafael J. Wysocki
2018-11-23T09:32:22Z
bec00cb5e97c19d2c8bd10db15d334cb40760000
----------------------------------------
Dave Airlie
2018-11-23T01:03:20Z
98c9cdfd34fbb62886e4c5a07e33661aa3352ef5
----------------------------------------
David S. Miller
2018-11-22T19:53:26Z
039e70a70c8417b5bf5878a60612ebd2c95f731e
----------------------------------------


AIR.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 07:04:10 pm
This is why they pay you the BIG bucks.

Thanks for your json extension module contribution!
Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 07:10:31 pm
I threw this together quickly to help get you started....it's in my repository, try it against a few json files before releasing to testing....
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 07:16:50 pm
Okay.

I can't stop staring at your SB json example.

Sweet!
Title: Re: JSON Challenge
Post by: AIR on November 25, 2018, 09:11:05 pm
Thanks.

What I like about the Parson code is that it's free as in free.  MIT license, just need to include attribution and license.  I've been focused on finding stuff like this this is either a single header or a single header/single C file combo.

At some point, you might want to take a look at CLIB (https://github.com/clibs/clib/wiki/Packages), which is an index/repository for stuff like this.  Some of the string stuff there could flesh out your C-BASIC project....I've been trying to find a good hash-table implementation, which C sorely lacks...

AIR.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 09:29:02 pm
Nice set of libraries to build extension modules from. One of the best features of SB is its unlimited expansion ability in a seamless manor.
Title: Re: JSON Challenge
Post by: John on November 25, 2018, 09:55:56 pm
Maybe a good way to get people to try Script BASIC is to have a code challenge to build an extension module from a library of their choice.
Title: Re: JSON Challenge
Post by: AIR on December 13, 2018, 02:50:18 pm
MBC Version:

Code: Text
  1. $MODULE json.inc
  2.  
  3. $EXECON
  4.  
  5. dim item as JSON_Object Ptr
  6. dim json as JOBJECT
  7.  
  8. json.init("commits.json")
  9.  
  10. if json.jsontype = JSONArray then
  11.     for integer i = 0 to json.count-1
  12.         item = json.object(i)
  13.         print string$(50,asc("-"))
  14.         print "Name: ", $json.text(item,"commit.author.name")
  15.         print "Commit Date: ", $json.text(item, "commit.author.date")
  16.         print "SHA Hash: ", $json.text(item, "sha")
  17.     next
  18.     print string$(50,asc("-"))
  19. end if
  20.  
Title: Re: JSON Challenge
Post by: John on December 13, 2018, 03:54:05 pm
Sure looks like a BaCon contender without all the C underware exposed.
Title: Re: JSON Challenge
Post by: John on December 27, 2018, 06:09:08 pm
AIR,

The json extension module doesn't seem to have a way to load a JSON string other than from a file. I'm trying to use it with a web service.
Title: Re: JSON Challenge
Post by: AIR on December 27, 2018, 08:50:47 pm
Duplicate the 'load' function in the interface.c file, and in the duped function replace the json_parse_file call with json_parse_string
Title: Re: JSON Challenge
Post by: John on December 27, 2018, 11:57:52 pm
Thanks AIR!

Works great.

Code: C
  1. /**
  2. =section loadfile
  3. =H json::loadfile(filename)
  4.  
  5. Loads filename, returns array of json objects
  6. */
  7. besFUNCTION(loadfile)
  8.   pModuleObject p;
  9.   char* filename;
  10.  
  11.   JSON_Array *items;
  12.   JSON_Object *obj;
  13.  
  14.   p = (pModuleObject)besMODULEPOINTER;
  15.  
  16.   besARGUMENTS("z")
  17.     &filename
  18.   besARGEND
  19.  
  20.  jRoot = json_parse_file(filename);
  21.  
  22.   switch (json_value_get_type(jRoot)) {
  23.     case JSONArray:
  24.       items = json_value_get_array(jRoot);
  25.       besRETURN_POINTER(items);
  26.       break;
  27.     case JSONObject:
  28.       obj = json_value_get_object(jRoot);
  29.       besRETURN_POINTER(obj);
  30.       break;
  31.   }
  32.  
  33.  
  34. besEND
  35.  
  36.  
  37. /**
  38. =section loadstr
  39. =H json::loadstr(json_str)
  40.  
  41. Loads json string, returns array of json objects
  42. */
  43. besFUNCTION(loadstr)
  44.   pModuleObject p;
  45.   char* json_str;
  46.  
  47.   JSON_Array *items;
  48.   JSON_Object *obj;
  49.  
  50.   p = (pModuleObject)besMODULEPOINTER;
  51.  
  52.   besARGUMENTS("z")
  53.     &json_str
  54.   besARGEND
  55.  
  56.  jRoot = json_parse_string(json_str);
  57.  
  58.   switch (json_value_get_type(jRoot)) {
  59.     case JSONArray:
  60.       items = json_value_get_array(jRoot);
  61.       besRETURN_POINTER(items);
  62.       break;
  63.     case JSONObject:
  64.       obj = json_value_get_object(jRoot);
  65.       besRETURN_POINTER(obj);
  66.       break;
  67.   }
  68.  
  69.  
  70. besEND
  71.  

Code: Script BASIC
  1. ' JSON - LoadStr
  2.  
  3. INCLUDE json.bas
  4.  
  5. json = """
  6. {
  7.    "client1": {
  8.        "name": "Joe Blow",
  9.        "age": 56,
  10.        "address": {
  11.            "city": "Tarrytown",
  12.            "state": "NY",
  13.            "zip": "10891"
  14.        }
  15.    },
  16.    "client2": {
  17.        "name": "John Smith",
  18.        "age": 86,
  19.        "address": {
  20.            "city": "Cupertino",
  21.            "state": "CA",
  22.            "zip": "N\/A"
  23.        }
  24.    }
  25. }
  26. """
  27.  
  28. jObject = json::loadstr(json)
  29. for i = 0 to json::count(jObject)-1
  30.     obj = json::object(jObject,i)
  31.     print string(40,"-"),"\n"
  32.     print json::Get(obj,"name"),"\n"
  33.     print json::Get(obj,"age"),"\n"
  34.     print json::Get(obj,"address.city"),"\n"
  35.     print json::Get(obj,"address.state"),"\n"
  36.     print json::Get(obj,"address.zip"),"\n"
  37. next
  38. print string(40,"-"),"\n"
  39.  


$ scriba json_loadstr.sb
----------------------------------------
Joe Blow
56
Tarrytown
NY
10891
----------------------------------------
John Smith
86
Cupertino
CA
N/A
----------------------------------------
$


Title: Re: JSON Challenge
Post by: John on December 29, 2018, 05:04:48 am
AIR,

Can you please look at your PM's from me about the issues I'm having with the json extension module?

I'm starting to get the feeling it doesn't support arrays.  >:(

Title: Re: JSON Challenge
Post by: John on December 31, 2018, 10:50:09 am
PHP makes json simple with one function each to encode / decode JSON strings to/ from an associative array.

It looks like the parson based JSON extension module is going to need a lot more work to be usable.
Title: Re: JSON Challenge
Post by: John on December 31, 2018, 04:00:08 pm
I think I found another way around having to write my own json ENCODE/DECODE functions in Script BASIC. I discovered that MySQL has a set of functions to work with JSON data. I'm going to give this a try using the Script BASIC MySQL extension module.

Code: MySQL
  1. mysql> SET @data = '{  
  2.    '>     "Person": {    
  3.     '>        "Name": "Homer",
  4.    '>        "Age": 39,
  5.     '>        "Hobbies": ["Eating", "Sleeping"]  
  6.    '>     }
  7.     '>  }';
  8. Query OK, 0 rows affected (0.02 sec)
  9.  
  10. mysql> SELECT JSON_EXTRACT(@data, '$.Person.Name', '$.Person.Age', '$.Person.Hobbies[1]') AS 'Result';
  11. +---------------------------+
  12. | Result                    |
  13. +---------------------------+
  14. | ["Homer", 39, "Sleeping"] |
  15. +---------------------------+
  16. 1 row in set (0.07 sec)
  17.  
  18. mysql>
  19.  
Title: Re: JSON Challenge
Post by: John on December 31, 2018, 04:39:19 pm
I was able to get it to work in Script BASIC using the MySQL extension module.

Code: Script BASIC
  1. IMPORT mysql.bas
  2.  
  3. json = """
  4. {
  5.    "Person": {
  6.       "Name": "Homer",
  7.       "Age": 39,
  8.       "Hobbies": ["Eating", "Sleeping"]
  9.    }
  10. }
  11. """
  12.  
  13. dbh =  mysql::RealConnect("localhost", "user", "password", "database")
  14. mysql::query(dbh,"SET @data = '" & json & "'")
  15. mysql::query(dbh,"SELECT JSON_EXTRACT(@data, '$.Person.Name', '$.Person.Age', '$.Person.Hobbies[1]') AS 'Result'")
  16. ok = mysql::FetchHash(dbh, query_result)
  17. mysql::Close(dbh)
  18.  
  19. PRINT query_result[0], " - ", query_result[1], "\n"
  20.  


$ time scriba sqltest.sb
Result - ["Homer", 39, "Sleeping"]

real   0m0.021s
user   0m0.019s
sys   0m0.001s
$


Using wildcards can return all the data for an object. It looks like you can do a lot with json and MySQL.

Code: Script BASIC
  1. mysql::query(dbh,"SELECT JSON_EXTRACT(@data, '$.Person.*')")
  2.  


$ scriba sqltest.sb
[39, "Homer", ["Eating", "Sleeping"]]
$

Title: Re: JSON Challenge
Post by: John on January 01, 2019, 09:29:12 am
The MySQL json direction is going to work great for my project I'm doing for a client. The concept is to mirror the remote DB via  REST web service. It's too cool that I can have a standard table schema and update it with JSON based data. This eliminates the data conversion that would be needed using a JSON extension module.

BTW:  SQLite also supports a similar JSON interface. This direction my be more appropriate for Windows desktop applications.
Title: Re: JSON Challenge
Post by: AIR on January 01, 2019, 01:44:02 pm
Sorry, John, off doing the family thing for a few days...
Title: Re: JSON Challenge
Post by: John on January 01, 2019, 02:08:48 pm
Nothing to be sorry for. Please enjoy your holiday and time with your family.

I'm happy with the MySQL JSON direction and I'm working on a proof of concept CRUD example.

Title: Re: JSON Challenge - PHP
Post by: John on January 04, 2019, 11:19:33 am
Here is an example of how PHP uses the decode JSON function to create an an object and an associative array.

Code: PHP
  1. <?php
  2.  $json = '
  3. {
  4.  "Person": {
  5.   "Name": "Homer",
  6.   "Age": 39,
  7.   "Hobbies": ["Eating", "Sleeping"]}
  8. }';
  9.  
  10.  var_dump(json_decode($json, true));
  11. ?>
  12.  
  13.  


$ php testjson.php
object(stdClass)#2 (1) {
  ["Person"]=>
  object(stdClass)#1 (3) {
    ["Name"]=>
    string(5) "Homer"
    ["Age"]=>
    int(39)
    ["Hobbies"]=>
    array(2) {
     
  • =>

      string(6) "Eating"
      [1]=>
      string(8) "Sleeping"
    }
  }
}
array(1) {
  ["Person"]=>
  array(3) {
    ["Name"]=>
    string(5) "Homer"
    ["Age"]=>
    int(39)
    ["Hobbies"]=>
    array(2) {
     
  • =>

      string(6) "Eating"
      [1]=>
      string(8) "Sleeping"
    }
  }
}
$

Title: Re: JSON Challenge
Post by: AIR on January 05, 2019, 01:38:15 am
Here's a quick and dirty example using an unreleased version of JADE, along with JUTE (https://github.com/amir-s/jute):

Code: C++
  1. #include <jade.hpp>
  2. #include "jute.h"
  3.  
  4. MAIN
  5.     using namespace jute;
  6.     STRING str(LOADFILE$("test.json"));
  7.     jValue data = parser::parse(str);
  8.     PRINT( data["Person"]["Name"].to_string() );
  9.     PRINT( data["Person"]["Age"].to_string() );
  10.     PRINT( data["Person"]["Hobbies"][0].to_string() );
  11.     PRINT( data["Person"]["Hobbies"][1].to_string() );
  12.     PRINT( "\n", data.to_string());
  13. END
  14.  

$ jade jtest.cc jute.o -ljade -o jtest
$ ./jtest
"Homer"
39
"Eating"
"Sleeping"

{
  "Person": {
    "Name": "Homer",
    "Age": 39,
    "Hobbies": ["Eating", "Sleeping"]
  }
}
Title: Re: JSON Challenge
Post by: John on January 05, 2019, 09:40:01 am
I'm glad you're still on the JADE path. The JUTE json library looks easier to use than the Parson library.

Can SB use C++ libraries to create extension modules?
Title: Re: JSON Challenge
Post by: AIR on February 27, 2019, 01:11:19 am
Since I'm learning GO, here is a GO version of the original challenge:

Code: Go
  1. package main
  2.  
  3. import (
  4.         "encoding/json"
  5.         "fmt"
  6.         "io/ioutil"
  7.         "os"
  8.         "strings"
  9. )
  10.  
  11. func main() {
  12.  
  13.         var jsonData []interface{}
  14.  
  15.         jsonFile, err := os.Open("commits.json")
  16.  
  17.         if err != nil {
  18.                 fmt.Println(err)
  19.                 os.Exit(1)
  20.         }
  21.  
  22.         defer jsonFile.Close()
  23.  
  24.         jsonBytes, _ := ioutil.ReadAll(jsonFile)
  25.  
  26.         if err = json.Unmarshal(jsonBytes, &jsonData); err != nil {
  27.                 fmt.Println(err)
  28.                 os.Exit(1)
  29.         }
  30.  
  31.         for _, item := range jsonData {
  32.                 fmt.Println(strings.Repeat("-", 50))
  33.                 entry := item.(map[string]interface{})
  34.                 commit := entry["commit"].(map[string]interface{})
  35.                 author := commit["author"].(map[string]interface{})
  36.                 fmt.Println("Name:", author["name"])
  37.                 fmt.Println("Commit Date:", author["date"])
  38.                 fmt.Println("SHA Hash:", entry["sha"])
  39.         }
  40.         fmt.Println(strings.Repeat("-", 50))
  41. }
  42.  

GO is a really nice language, though it's primary focus is web / server / commandline oriented.

AIR.
Title: Re: JSON Challenge
Post by: John on February 27, 2019, 08:00:38 am
Thanks AIR for the GO submission!

I was getting worried you retired again.  :)

Actually MySQL turned out to be a great JSON parser for the project I'm working on.
Title: Re: JSON Challenge
Post by: AIR on February 28, 2019, 10:08:33 pm
Go has the built-in capability of mapping json directly to fields in a structure.  Additionally, you can specify that only specific fields are mapped, ignoring all others.

Here is what using this feature looks like:

Code: Go
  1. package main
  2.  
  3. import (
  4.         "encoding/json"
  5.         "fmt"
  6.         "io/ioutil"
  7.         "os"
  8.         "strings"
  9. )
  10.  
  11. type JsonData []struct {
  12.         Sha    string `json:"sha"`
  13.         Commit struct {
  14.                 Author struct {
  15.                         Name string `json:"name"`
  16.                         Date string `json:"date"`
  17.                 } `json:"author"`
  18.         }
  19. }
  20.  
  21. func main() {
  22.         var jsonData JsonData
  23.  
  24.         jsonFile, err := os.Open("commits.json")
  25.  
  26.         if err != nil {
  27.                 fmt.Println(err)
  28.                 os.Exit(1)
  29.         }
  30.  
  31.         defer jsonFile.Close()
  32.  
  33.         jsonBytes, _ := ioutil.ReadAll(jsonFile)
  34.  
  35.         if err = json.Unmarshal(jsonBytes, &jsonData); err != nil {
  36.                 fmt.Println(err)
  37.                 os.Exit(1)
  38.         }
  39.  
  40.         for _, item := range jsonData {
  41.                 fmt.Println(strings.Repeat("-", 50))
  42.                 fmt.Println("Name:", item.Commit.Author.Name)
  43.                 fmt.Println("Commit Date:", item.Commit.Author.Date)
  44.                 fmt.Println("SHA Hash:", item.Sha)
  45.         }
  46.         fmt.Println(strings.Repeat("-", 50))
  47. }
  48.  

I think that, for the most part, someone not familiar with go can look at the code above and have a decent understanding of what it's doing.

The more I study go, the more I like it...

AIR.
Title: Re: JSON Challenge
Post by: John on February 28, 2019, 10:31:01 pm
GO looks pretty bare bones and to do anything you need to include a bunch of libraries. Am I missing something?
Title: Re: JSON Challenge
Post by: AIR on February 28, 2019, 11:01:14 pm
It's a small core with a ton of available PACKAGES (https://golang.org/pkg/#stdlib) in the standard distribution.

I like it's modular nature vs the monolithic approach.  I also like the fact that I can develop on my Mac, and then optionally cross-compile to Linux and Windows from within macOS without having to install additional tool-chains to do so.

Title: Re: JSON Challenge
Post by: John on February 28, 2019, 11:13:35 pm
A nice set of extensions.  The cross platform feature is a big plus.