Author Topic: Database Code Challenge(s)  (Read 23983 times)

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #30 on: November 20, 2018, 01:46:38 PM »
AIR has already offered a promising solution for the back end text DB. Can you share what you have or envisioned for a UI?
« Last Edit: November 20, 2018, 02:26:55 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Re: Database Code Challenge(s)
« Reply #31 on: November 20, 2018, 04:53:25 PM »
Alysson's VB Demo project updates the timestamp of the "record".

Here's how I would do it using Python and yAML (I had to write some custom code so that the output was as close to the original as possible):

Code: Python
  1. #!/usr/bin/env python
  2.  
  3. from datetime import datetime
  4. import air_yaml as yaml
  5.  
  6. with open('testI.yml') as fp:
  7.     record = yaml.load(fp)
  8.  
  9. record["Previous Version"] = record["LastSavedTime"]
  10. record["LastSavedTime"] =  datetime.now().strftime("%d/%m/%Y %H:%M:%S")
  11.  
  12. with open('result.yml', 'wb+') as fp:
  13.     yaml.dump(record, fp, default_flow_style=False, width=79, Dumper=yaml.AIRDumper)

Code: YAML
  1. # ================================================================================
  2. RecordTitle: Mineral Specimen Details
  3. # ================================================================================
  4. Entry No: ARC-99-000021
  5. # ================================================================================
  6. Species:
  7.   Minerals:
  8.  - Zeunerite
  9.   - Rutherfordine
  10.   - Zippeite
  11.   - Chalcopyrite
  12.   - var:
  13.    - Blister Copper
  14.     - Quartz
  15.     - Agate
  16.   - colour:
  17.    - White
  18.     - Opal
  19.     - Transparent
  20.     - var:
  21.      - Common Opal
  22.   - Matrix: Quartz
  23. # ================================================================================
  24. Location:
  25.   Address: 'South Terras Mine
  26.  
  27.     St Stephen in Brannel
  28.  
  29.     St Austell
  30.  
  31.     Cornwall, UK
  32.  
  33.     '
  34.   Grid Reference: SW 935 524
  35.   GPS Coordinates: 50.33444,-4.90194
  36. # ================================================================================
  37. Collection Data:
  38.   collected by: Self collected
  39.   Collection Date: 1979
  40. # ================================================================================
  41. Acquisition Data:
  42.   Acquired Date: 14 Jul 1998
  43.   From: Self collected
  44.   Field Ref No: F20180623-STUM-001
  45.   Temp Ref No: TMP 0 0099
  46. # ================================================================================
  47. Analytical Data:
  48.   Commentary: 4 species
  49.   Zeunerite:
  50.     .comment: Colour and crystal form are typical for this location.
  51.     .date: 14 Aug 2018
  52.     .Report: AN-201808-14-01
  53.     .Ref_Number: ZZ8088
  54.   Zippeite:
  55.     .comment: Colour is diagnostic for this location
  56.     .date: 14 Aug 2018
  57.     .Report: AN-201808-14-02
  58.     .Ref_Number: ZZ9099
  59.   Chalcopyrite:
  60.     .comment: Well characterised at this location. Deep indigo tarnish on old surfaces.
  61.   Rutherfordine:
  62.     .comment: Tentative ID.
  63.     .Reasons: Visual ID only
  64. # ================================================================================
  65. Accession Data:
  66.   Accession No: ZZ9099
  67.   Accession Date: 21 Sept 2000
  68.   Sub-collection: Main
  69. # ================================================================================
  70. Image Data:
  71.   Mugshot: img0001.jpg
  72.   Images:
  73.   - .filename: img0001.jpg
  74.     .Title: Whole Specimen
  75.     .TakenDate: 11 Nov 2018
  76.     .Width: 11.5mm
  77.     .camera: Fuji FinePix L55
  78.   - .filename: img0002.jpg
  79.     .Title: Zeunerite & Chalcopyrite
  80.     .TakenDate: 11 Nov 2018
  81.     .FOV: 1.1mm
  82.     .Camera: CL 1600 USB MicroCam
  83. # ================================================================================
  84. RecordType: 1
  85. # ================================================================================
  86. LastSavedTime: 20/11/2018 19:43:31
  87. # ================================================================================
  88. Previous Version: 13/11/2018 13:18:37
  89. # ================================================================================
  90.  

I still need to fix the "Address" output, but that's pretty much all that's left...

AIR.

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Re: Database Code Challenge(s)
« Reply #32 on: November 20, 2018, 05:34:06 PM »
@Alysson -

I took a deep dive into your code, it's really nice!!

Great Job!

I only wish that VB6 wasn't such a stickler for _stdcall in DLL's, it's the main reason I left it many years ago for Delphi/Lazarus....

AIR.

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #33 on: November 20, 2018, 05:36:21 PM »
Peter Verhas wrote a template like web page generation system where data would replace the tags in the document. I haven't spent enough time looking at the challenge to see if it would apply. Peter called it HEB. (Html Embedded Basic)

Quote from: Peter Verhas
EXTERNAL PREPROCESSOR
This module executes external preprocessors. These preprocessors are standalone executable programs that read the source program and create another file that is read and processed by the ScriptBasic interpreter. If an external preprocessor is used the source file is usually not BASIC but rather some other language, usually a BASIC like language, which is extended some way and the preprocessor creates the pure ScriptBasic conformant BASIC program. The sample preprocessor supplied with ScriptBasic is the HEB (HTML Embedded BASIC) preprocessor that reads HTML embedded BASIC code and creates BASIC program. This HEB source file is a kind of HTML with embedded program fragments, which you may be familiar with in case you program PHP or Microsoft BASIC ASP pages. The HEB preprocessor itself is written in BASIC and is executed by ScriptBasic. Thus when a HEB "language" is executed by ScriptBasic it starts a separate instance of the interpreter and executes the HEB preprocessor on the source file. Of course the HEB preprocessor could be implemented in any language that can be compiled or some way executed on the target machine. Actually the very first version of the HEB preprocessor was written in Perl so when it was first tested the ScriptBasic interpreter started a Perl interpreter before reading the generated BASIC code.

Note that the HEB preprocessor provided in the ScriptBasic package is an example implementation and lacks many features. It can, for example, be fooled by putting a %> characters into a BASIC string constant.
« Last Edit: November 20, 2018, 07:05:17 PM by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #34 on: November 20, 2018, 06:26:22 PM »
Quote from: AIR
I only wish that VB6 wasn't such a stickler for _stdcall in DLL's, it's the main reason I left it many years ago for Delphi/Lazarus....

The SB VB6 connection gives you a choice of how much of VB6 you want to depend on in your app.

Offline AlyssonR

  • Advocate
  • Posts: 131
Re: Database Code Challenge(s)
« Reply #35 on: November 21, 2018, 01:45:26 AM »
@Alysson -

I took a deep dive into your code, it's really nice!!

Great Job!

I only wish that VB6 wasn't such a stickler for _stdcall in DLL's, it's the main reason I left it many years ago for Delphi/Lazarus....

AIR.

Thanks. Coming from you that is a BIG compliment.

I must admit to having considered Lazarus, but my Pascal is almost as rusty as my FORTRAN, and dates back to TurboPascal V2.xx (hell's bells,  I was using it on an 8-bit home micro last time I used Pascal in anger).

Offline AlyssonR

  • Advocate
  • Posts: 131
Re: Database Code Challenge(s)
« Reply #36 on: November 21, 2018, 02:01:29 AM »
AIR has already offered a promising solution for the back end text DB. Can you share what you have or envisioned for a UI?

That is actually a bit of a question.

I intend to use two separate user interfaces - active (admin, data input/update etc.) and passive (catalogue browse)

The catalogue browser will be auto-generated web pages including an off-the shelf search engine. I intend to use a cardfile metaphor for this (old wood, manila widgets, cream laid cards, traditional form font, typewriter data).

I've attached a mock-up that I made some time ago - lacking the quality of the real thing, of course.


The administrative interface is still up in the air, though it is likely to use a current widget set whatever gets used.

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #37 on: November 21, 2018, 10:07:25 AM »
Peter's HEB SB pre processor is sounding better by the post.  8)
« Last Edit: November 21, 2018, 10:14:05 AM by John »

Offline AlyssonR

  • Advocate
  • Posts: 131
Re: Database Code Challenge(s)
« Reply #38 on: November 21, 2018, 02:31:22 PM »
Peter's HEB SB pre processor is sounding better by the post.  8)

Could well be.

Let me get the functionality up and running, and then I can think about front-end.

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #39 on: November 21, 2018, 09:26:57 PM »
I don't want to get you too excited but you could be running those slow processes of yours in SB threads.

If I were to take a stab at this, I would compile a core SB with the interpreter running in a DLL supporting a thread safe environment.  Would be hard to beat.

I'm going to wait till your fully dressed before dancing to this.
« Last Edit: November 21, 2018, 09:38:48 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Re: Database Code Challenge(s)
« Reply #40 on: November 21, 2018, 10:35:49 PM »
Wanted to try something on Windows, so I coded up this HotBasic demo.

It's using libyaml, compiled to a 32bit dll using MinGW (I had to create my own makefile since their autoconf/cmake configs kept looking for MS Visual C and nmake no matter what I told it to do).

The attached file includes the source, a ready to run binary, the yaml file itself and the dll.  I tested under both Win7 and Win10.

Code: Text
  1. $APPTYPE CONSOLE: $TYPECHECK ON
  2. $INCLUDE "yaml.inc"
  3.  
  4. type COLLECTIONDATA
  5.         collected_by as string
  6.         collected_date as string
  7. end type
  8.  
  9. type MINERALS
  10.         name as string*64
  11.         var as string*64
  12.         colors as string*64
  13.         color_var as string*64
  14.         matrix as string*64
  15. end type
  16.  
  17. type IMAGE
  18.         mugshot as string
  19. end type
  20.  
  21. type SPECIES
  22.         minerals as MINERALS
  23. end type
  24.  
  25. type LOCATION
  26.         address as string
  27.         grid as string
  28.         gps as string
  29. end type
  30.  
  31. type ACQUISITION
  32.         date as string
  33.         from as string
  34.         fieldref_no as string
  35.         tempref_no as string
  36. end type
  37.  
  38. type DBINFO
  39.         rec_type as string
  40.         last_save as string
  41.         previous_save as string
  42. end type
  43.  
  44.  
  45. type record
  46.         dummy as string
  47.         title as string
  48.         entry as string
  49.         species as SPECIES
  50.         location as LOCATION
  51.         collection as COLLECTIONDATA
  52.         aquisition as ACQUISITION
  53.         dbinfo as DBINFO
  54. end type
  55.  
  56.  
  57. defstr yamlFile,tk,key,token
  58. deflng parser,ret, indent, child,state=0
  59. dim conf as record
  60.  
  61.  
  62. yamlFile.loadfromfile("testI.yml")
  63.  
  64.  
  65. ret = yaml_parser_initialize(@parser)
  66.  
  67.  
  68. yaml_parser_set_input_string(@parser, yamlFile, yamlFile.len)
  69.  
  70. do
  71.         yaml_parser_scan(@parser, @token)
  72.  
  73.         select case byref(@token)
  74.         '       case YAML_BLOCK_SEQUENCE_START_TOKEN,YAML_BLOCK_MAPPING_START_TOKEN
  75.         '           if child > 0 then
  76.     '             inc indent,2
  77.     '         else
  78.     '             inc child
  79.         '                       ' print "#" + string$(78,"=")
  80.     '         end if
  81.         '       case YAML_BLOCK_END_TOKEN
  82.         '               dec indent,2
  83.         '               dec child
  84.                        
  85.                 case YAML_KEY_TOKEN
  86.                         state = 0
  87.                 case YAML_VALUE_TOKEN
  88.                         state = 1
  89.                 case YAML_SCALAR_TOKEN
  90.                         tk = byref$(byref(@token+4))
  91.  
  92.                         if state = 0 then
  93.                                 key = tk
  94.                         end if
  95.  
  96.                         if state = 1 then
  97.                                 select case key
  98.  
  99.                                         ' ## HEADER INFO ##
  100.                                         case "RecordTitle"
  101.                                                 conf.title = tk
  102.                                         case "Entry No"
  103.                                                 conf.entry = tk
  104.  
  105.                                         ' ## LOCATION INFO ##
  106.                                         case "Address"
  107.                                                 conf.location.address = tk
  108.                                         case "Grid Reference"
  109.                                                 conf.location.grid = tk
  110.                                         case "GPS Coordinates"
  111.                                                 conf.location.gps = tk
  112.  
  113.                                         ' ## COLLECTION INFO ##
  114.                                         case "collected by"
  115.                                                 conf.collection.collected_by = tk
  116.                                         case "Collection Date"
  117.                                                 conf.collection.collected_date = tk
  118.  
  119.                                         ' ## ACQUISITION INFO ##
  120.                                         case "Acquired Date"
  121.                                                 conf.aquisition.date = tk
  122.                                         case "From"
  123.                                                 conf.aquisition.from = tk
  124.                                         case "Field Ref No"
  125.                                                 conf.aquisition.fieldref_no = tk
  126.                                         case "Temp Ref No"
  127.                                                 conf.aquisition.tempref_no = tk
  128.  
  129.                                         ' ## DB SYSTEM INFO ##
  130.                                         case "RecordType"
  131.                                                 conf.dbinfo.rec_type = tk
  132.                                         case "LastSavedTime"
  133.                                                 conf.dbinfo.last_save = tk
  134.                                         case "Previous Version"
  135.                                                 conf.dbinfo.previous_save = tk
  136.                                 '       case else
  137.                                 '               ' print "Unknown key = " + key
  138.                                 end select
  139.  
  140.                                
  141.                                 ' print space$(indent) + key + " = " + tk
  142.                                 ' if indent = 0 then
  143.                                        
  144.                                 ' end if
  145.                         end if
  146.  
  147.                 case else
  148.                         state = 1
  149.         end select
  150.        
  151.  
  152.  
  153. loop until byref(@token) = YAML_STREAM_END_TOKEN
  154.  
  155. 'PAUSE
  156.  
  157. gosub cleanup
  158. gosub print_data
  159.  
  160. END
  161.  
  162. cleanup:
  163.         yaml_token_delete(@token)
  164.         yaml_parser_delete(@parser)
  165. return
  166.  
  167. print_data:
  168.         cls
  169.         print "Title = "; conf.title
  170.         print "Entry# = "; conf.entry
  171.         print "Address = "; conf.location.address
  172.         print "Grid = ";conf.location.grid
  173.         print "GPS = ";conf.location.gps
  174.         print "Collected By = ";conf.collection.collected_by
  175.         print "Collected Date = ";conf.collection.collected_date
  176.         print "Aquisition Date = ";conf.aquisition.date
  177.         print "Aquired From = ";conf.aquisition.from
  178.         print "Field Ref No = ";conf.aquisition.fieldref_no
  179.         print "Temp Ref No = ";conf.aquisition.tempref_no
  180.         print "DB Rec Type = ";conf.dbinfo.rec_type
  181.         print "Last Saved = ";conf.dbinfo.last_save
  182.         print "Previous Save = ";conf.dbinfo.previous_save
  183. return

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #41 on: November 21, 2018, 11:26:33 PM »
One idea is use SB associtive arrays as a form of of readable DB data that has immediate functionality.

Code: ScriptBasic
  1. COLLECTIONDATA{"collected_by"} = "Alysson"
  2. COLLECTIONDATA{"collected_date"} = NOW
  3.  

This embedded in a preproessor HEB like format gives you a DB and web page definition in one file / record.
« Last Edit: November 21, 2018, 11:30:41 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Re: Database Code Challenge(s)
« Reply #42 on: November 21, 2018, 11:32:30 PM »
One idea is use SB associtive arrays as a form of of readable DB data that has immediate functionality.

Code: ScriptBasic
  1. COLLECTIONDATA{"collected_by"} = "Alysson"
  2. COLLECTIONDATA{"collected_date"} = NOW
  3.  

This embedded in a preproessor HEB like format gives you a DB and web page definition in one file / record.

All well and good, but you need to get the data from the file first. So how would you approach that?

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Database Code Challenge(s)
« Reply #43 on: November 21, 2018, 11:38:58 PM »
The associative array definitions is the data along with the HEB definition that would display it (maybe as an include) all as a record.
« Last Edit: November 21, 2018, 11:41:09 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Re: Database Code Challenge(s)
« Reply #44 on: November 21, 2018, 11:41:33 PM »
The associative array definitions is the data along with the HEB definition that would display it all as a record.

So....you're going to type all of that in every time?   ;D

Alysson has the data serialized in a file.  How do you get that into the AA?  And back out in the required format?