Print Page - Database Code Challenge(s)

BASIC User Group => Code Challenges => Topic started by: John on November 16, 2018, 12:27:27 PM

Title: Database Code Challenge(s)
Post by: John on November 16, 2018, 12:27:27 PM

Alysson,

If you have any spare time, it would be great if you can put some effort towards the VB6 OCX form direction. If you don't see it having value then I'l but it on the back burner. Windows isn't my primary development environment.

Thanks!

John

Title: Re: DB Challenge
Post by: AlyssonR on November 17, 2018, 01:48:37 AM

John,

I shall, but once I have some more back-end written.

At the moment I'm developing in VB6 with a view to converting the back-end to SB later - I'll try to get my front end converted to OCX once I get that far.

It'll be a while before I get to that point since I'm actually still working on that custom database project that has been underway for the past *mumbles* years (and am now making admittedly slow headway).

Title: Re: DB Challenge
Post by: John on November 17, 2018, 01:38:47 PM

Please keep us in the loop with your progress.

Title: Re: DB Challenge
Post by: AIR on November 17, 2018, 10:05:36 PM

Hi, Alysson!

Your DB project sounds like it could make an interesting code challenge. Would you be willing to submit a code challenge for this?

AIR.

Title: Re: DB Challenge
Post by: John on November 17, 2018, 10:15:33 PM

+1

AIR,

It's always a treat when you visit the Windows world. Sort of like a full moon. :)

Title: Re: DB Challenge
Post by: AlyssonR on November 18, 2018, 01:44:16 AM

*howls at the full AIR* :)

Okay, I'll pull together a challenge of some sort.

Title: Re: DB Challenge
Post by: John on November 18, 2018, 02:02:03 AM

I knew I could count on you. Can't wait to see where you are with it.

Title: Database Code Challenge(s)
Post by: AlyssonR on November 18, 2018, 02:48:08 AM

Database Code Challenges

I have been asked to come up with a code challenge based on some software that I am writing.

General requirements:

The software is an acquisitions and accessions database for a museum-type collection - thus, there are no physical delete operations on any record - a 'deleted from collection' simply flags the record as hidden. All of the artefact records are, in fact, separate disc directories containing a single text file and any number of media files.

The primary requirement is that the textual data must be kept entirely in human-readable digital format (i.e. text files), thus, software-proofing the data. That includes indices and data (images and other media excepted). The artefact record is, in fact, a single document. Even EXIF-type metadata on media files is maintained in separate text documents alongside the media files.

Software installation is not required - just copy the directory, edit the config.ini file to suit, and run.

It is important that the data files may be printed out as-are, and will look like properly formatted text reports.

Directory structure:

Code: [Select]

DBRoot
 |
 |__ Artefact Record
 |       |
 |       |__ Images
 |       |       |___ *.jpg etc.
 |       |
 |       |__ Documents
 |       |       |___ *.doc, .xls etc.
 |       |
 |       |___artefact.record (text file)
 |
 |___ DBPrimaryIndex.index (text file)
 |___ DBPrimaryIndex.schema (text file)
 |
 |___ DB secondary indices & schemas

For the sake of the code challenges, each program may reside in the same directory as the data file(s).

I will put up a new challenge chunk as each back-end section of the software is completed (I hope).

I should point out that I am using VB6 for developing this software (which means that it'll run under WINE, too).

Challenge 1: Data Records

Next: Challenge 2: Primary Index

Title: Database Code Challenge(s): Challenge 1 - Data Records
Post by: AlyssonR on November 18, 2018, 02:55:08 AM

Challenge 1 - Data Records:

The data records are, apart from the unique 'card number', unspecified in format (schemaless) - the data is entered in such a way that each field is named in the text, and may contain multi-line values, arrays and simple values. Some system data is tucked away at the end of the record document. This is not JSON or XML (which are not pretty enough to just print out as-is)

You need to be able to read the document into the software and to then (after any alterations) to write it back to the file in the same format as if was read.

I have already done this in VB6, and the program tidies up the tabulation of the document at the same time (neat-freak ;))

A typical record looks like this:

Code: [Select]

RecordTitle is              Mineral Specimen Details
                            ========================

Entry No:
                            ARC-99-000021

================================================================================
Species:
  Minerals =                Zeunerite
                            Rutherfordine
                            Zippeite
                            Chalcopyrite
    .var                     Blister Copper
                            Quartz
    .var                     Agate
    .colour                  White
                            Opal
    .var                     Common Opal
    .colour                  Transparent
  Matrix =                  Quartz

================================================================================
Location:
                            South Terras Mine
                            St Stephen in Brannel
                            St Austell
                            Cornwall, UK
  Grid Reference =          SW 935 524
  GPS Coordinates =         50.33444,-4.90194

================================================================================
Collection Data:
  collected by =            Self collected
  Collection Date =         1979

================================================================================
Acquisition Data:
  Acquired Date =           14 Jul 1998
  From =                    Self collected
  Field Ref No =            F20180623-STUM-001
  Temp Ref No =             TMP 0 0099

================================================================================
Analytical Data:
  Commentary =              4 species
  Zeunerite =             { Colour and crystal form are typical for this location.
                          }
    .date                    14 Aug 2018
    .Report                  AN-201808-14-01
    .Ref_Number              ZZ8088
  Zippeite =              { Colour is diagnostic for this location
                          }
    .date                    14 Aug 2018
    .Report                  AN-201808-14-02
    .Ref_Number              ZZ9099
  Chalcopyrite =          { Well characterised at this location.
                            Deep indigo tarnish on old surfaces.
                          }
  Rutherfordine =         { Tentative ID.
                          }
    .Reasons                 Visual ID only

================================================================================
Accession Data:
  Accession No =            ZZ9099
  Accession Date =          21 Sept 2000
  Sub-collection =          Main

Image Data:
      Mugshot =             img0001.jpg
      Images =              img0001.jpg
      .Title                Whole Specimen
      .TakenDate            11 Nov 2018
      .Width                11.5mm
      .camera               Fuji FinePix L55
                            img0002.jpg
      .Title                Zeunerite & Chalcopyrite
      .TakenDate            11 Nov 2018
      .FOV                  1.1mm
      .Camera               CL 1600 USB MicroCam

===== DB System Info ===========================================================
 RecordType is               1 
 LastSavedTime is           13/11/2018 13:18:37 
(Previous Version was       13/11/2018 12:12:47 )
================================================================================

The format is:
Field: Indicates a group of related data
Sub-Field Name = Indicates a named sub-field (or sub-field array)
.Property Indicates a named property of the sub-field
Name = { Indicates the start of a multiline data block
} End of multiline data.

Title: Re: DB Challenge
Post by: AlyssonR on November 18, 2018, 08:28:26 AM

Still working on the functionality at the back end - nothing that actually *looks* like a database yet - just BASIC modules.

The idea of free-form records came from the first version of my minerals database - using IdeaList.

There is little complex searching and nothing as horrendous as deletions or non-sequential records.

Oh yes, and nothing as suspect as compiled stuff yet - just proof of concept examples. One day, I may even do some documentation (laughs up sleeve and tries to hide the growing nose).

Title: Re: DB Challenge
Post by: John on November 18, 2018, 10:09:49 AM

After a quick peek, sure looks like a wiki would be the best tool for the job.

Check out PBWiki as it's based on a text file format.

Title: Re: DB Challenge
Post by: AlyssonR on November 18, 2018, 11:09:16 AM

I considered PBWiki, amongst literally hundreds of other tools of various sorts.

What I need is a little too rigid for a wiki, too flexible (and lightweight) for a standard database, and really not worth the effort with a document automation/workflow system. Nothing *quite* meets the specification - otherwise I'd have resurrected my copy of IdeaList which really is the closest approach.

I've given up looking for something that is (a) close to suitable, (b) too cheap for words and (c) doesn't provide a cliff-like learning curve, and have decided to write it myself. Once the decision was made, deciding on how to do the job took me a couple of months. Each functional module takes me about a week to write and to test thoroughly, alongside an example project (used for testing) - given that the specifications tend to change a bit as I write.

Once the functional code is written, the rest of the work will just be wiring the whole lot up.

The DB itself only requires administration by a single user, though I am planning on delivering lookup via static HTML and Saxon XSLT (read-only service) with an index system being generated by background scripts (I've done this before).

Besides, using an off-the-shelf solution leaves no room for the satisfaction of developing my own application, and this is the first application that I've written in about 30 years - it's all been admin and glue scripts for a loooong time.

Title: Re: DB Challenge
Post by: John on November 18, 2018, 11:57:56 AM

Is this a hobby project you're working on? A good deed for a non-profit museum?

Title: Re: DB Challenge
Post by: AlyssonR on November 18, 2018, 01:44:03 PM

A bit of both, actually.

I don't program commercially (and haven't done so since the early 1980's) - it's usually an incidental aspect of management/support.

I want something for my own collection (which is the original reason I decided to write the thing), and while chatting with the manager of a small museum, discovered that they want something similar to handle their accessions that will (a) run on older hardware, (b) give access to volunteers (and visitors) to look up information about the collection and (c) allow their archivist to get as much information onto the system as possible. Oh and, (d) to be within their budget (roughly £15 and a pub lunch in the town).

They are constantly looking for additional funding, and I offered a good deed with something I wanted to do in any case.

Once it's out there, I'll be offering it for free to anyone who wants it - there are *so many* small museums and private collectors in the UK who want something more than a card index, but less than an all-singing-all-dancing technicolour database. 99% of our museums are privately funded, small and rely on goodwill for most things.

*falls off of soapbox*

Title: Re: DB Challenge
Post by: John on November 18, 2018, 02:15:59 PM

First and foremost don't lock yourself into Windows because that's all the tools your're familiar with. I think AIR got a whiff of you painting yourself into a corner and suggested the code challenge to get it back on the right path.

My submission would be a Script BASIC application server proxy to Apache with MySQL as the DB running on a next to nothing cost wise Linux instance.

Title: Re: DB Challenge
Post by: AlyssonR on November 18, 2018, 03:33:20 PM

Strangely, VB6 software will run well under WINE, but that isn't the point.

Once the project is done in VB (which is my go-to for QUICK), I plan to rework at least the back-end in SB when I have time. Right now, I use Windows on my main box, and the museum is on a mix of Windows XP and 7 - and that is not likely to change for a while. I'm far from painted into a corner here - at least, not once I resurrect my Debian box (and update it a bit).

I particularly want to avoid running *anything* on an SQL database when I can do the job with a lot less logic. Remember that flexibility I specified? It's intrinsic to what I have designed - MySQL et al require a stack of admin to change anything on the DB.

Whilst it's true that what I have written isn't pretty (that comes later), and it isn't terribly fast (but the datasets are only in the thousands of records). It isn't terribly disk efficient (disk space is cheap for this size of DB), nor is it extremely comprehensive and it certainly lacks security, but ....

It meets the cost (time, effort and cash) requirements for the product. It doesn't require much setting up (run it from an external HDD). Adding an additional document base just requires an index schema, an initialisation file and a sample form - a task that most people could manage (yeah, yeah ... better fool and all that).

I am happy with my design, and so is the museum. The only thing that worries me is the guy got a crazed look in his eye and started muttering about pages on an old ViewData terminal they have (like I said - some OLD hardware) - and it's a long time since I wrote an interface to a viewdata service.

Title: Re: DB Challenge
Post by: John on November 18, 2018, 03:49:16 PM

This is sounding more and more like a RetroB challenge if where you are feels that comfortable.

Title: Re: DB Challenge
Post by: AIR on November 18, 2018, 08:48:20 PM

Quote from: John on November 18, 2018, 03:49:16 PM

This is sounding more and more like a RetroB challenge if where you are feels that comfortable.

I disagree.

Most code challenges (RetroB included) tend to settle on a very focused end result. I don't think I've ever seen a challenge issued where the goal is to create a full system/stack that is not limited to a single use case.

My perspective on this particular challenge is that it's an amazing opportunity for a chosen programming language to show what it can do. By that, I mean how seemingly disparate capabilities of a language can be assembled in a way that showcases how well (or bad!!) a given language is at producing the finished product.

One of the things I find intriguing about Alysson's challenge is the requirement that the data be accessible with a simple text editor by anyone in more or less plain English (or your choice of language). That eliminates solutions using *sql based products, xml, json, etc.

So the archiving of the data itself could be viewed as requiring the creation of a configuration file format. Stay with me on this. If you consider something like the old Windows INI file format you'll see what I'm getting at. Personally, I think that something like yAML (https://learnxinyminutes.com/docs/yaml/) would fit the bill since it offers the flexibility that the challenge would require. That's how I'm thinking of approaching it, anyway.

So my process would be:

Settle on a serialization format, that is readable and editable with a simple text editor. But is also easy to work with programmatically.

Work out the FILE structure as Alysson laid out in her challenge post.

Figure out how to present this to an end user so that it makes sense to them.

Figure out how Alysson is going to get the required booze to us in the States and elsewhere.

Sounds like Fun!!!

AIR.

EDIT: Added link to a quick and dirty overview of the yAML format....

Title: Re: DB Challenge
Post by: John on November 18, 2018, 08:58:16 PM

Is this challange restricted Windows and VB6 as the UI?

Title: Re: Database Code Challenge(s)
Post by: AIR on November 18, 2018, 10:16:23 PM

I think the logic to manipulate the data doesn't need to be Windows-centric (it wouldn't be in my case) but the gui is a different matter.

However, if you use a cross-platform toolchain it shouldn't matter much, aside from a few gui tweaks to account for differences in positioning/rendering.

The last time I used VB 4/5/6 was in the late 90's, so I probably wouldn't be using that personally. But I think it's important to consider at some point that the target audience in this case is ultimately Windows-centric.

AIR.

Title: Re: Database Code Challenge(s)
Post by: John on November 18, 2018, 10:38:50 PM

Alysson showed interest in the SB OCX form proof of concept so I'm assuming that direction is still valid. This doesn't limit the challenge to SB as any COM/OLE complaint language should work. One would just need to implement a similar callback environment like SB offers.

Title: Re: Database Code Challenge(s)
Post by: John on November 18, 2018, 11:26:18 PM

PHP would be a good candidate for this challenge. It has an easy to use COM/OLE interface for desktop and something that can go web bound if desired with a wealth of preexisting code.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 19, 2018, 01:04:12 AM

Quick example using Python and yAML to prototype:

Code: Python

#!/usr/bin/env python
 
import yaml
 
with open('testI.yml') as fp:
    record = yaml.load(fp)
 
print "Title: %s" %record["RecordTitle"]
print "Entry Number: %s" %record['Entry No']
print "Minerals: %s\n"  %record["Species"]['Minerals'][:4]
# print "Mineral Colors: %s\n"  %record["Species"]['Minerals']['colour'][:3]
print "Address:\n%s" %record['Location']['Address']
print "Grid: %s" %record['Location']['Grid Reference']
print "GPS: %s\n" %record['Location']['GPS Coordinates']
 
for x in record["Species"]['Minerals'][:4]:
    try:
        print "%s: %s" %(x,record['Analytical Data'][x]['.comment'])
    except:
        continue

[riveraa@MPD ~/Projects/Alysson] $ ./test.py Title: Mineral Specimen Details Entry Number: ARC-99-000021 Minerals: ['Zeunerite', 'Rutherfordine', 'Zippeite', 'Chalcopyrite'] Address: South Terras Mine St Stephen in Brannel St Austell Cornwall, UK Grid: SW 935 524 GPS: 50.33444,-4.90194 Zeunerite: Colour and crystal form are typical for this location. Rutherfordine: Tentative ID. Zippeite: Colour is diagnostic for this location Chalcopyrite: Well characterised at this location. Deep indigo tarnish on old surfaces.

Sample file converted to yml format is attached.

Title: Re: Database Code Challenge(s)
Post by: John on November 19, 2018, 01:44:33 AM

Nice approach and it seems to satisfy the readability requirement well.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 19, 2018, 02:10:31 AM

Apart from using nested stuctures (a VB speciality), you will notice that I don't actually rely on any Windows API calls in the functional module. There is no registry requirement, and by dropping the extracted VB runtime DLLs into the program directory, they don't seem to need registration in the system either (only really applicable to WINE/Linux boxen).

As I said earlier - speed of initial deployment is important right now (I want v1.0 up and in user Beta by New Year), and I will be able to spend time later translating the whole thing into another version of BASIC later. It will also give me a chance to get my head thoroughly around IUP, and perhaps write a translator of VB .frm source into IUP 4 BASIC (have I mentioned how much I detest not being able to do drag & drop form design? Spoiled by VB6, I suppose).

AIR is right that I don't want to be locked into Windows in the long term, but right now it is a convenient platform.
Likewise, I have no desire to be stuck with MySQL (or similar) even though it would be so very simple to implement.
As for PHP .... *shudders* .... one day, maybe.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 19, 2018, 02:14:33 AM

@ AIR -

Nice.

Title: Re: Database Code Challenge(s)
Post by: John on November 19, 2018, 02:21:48 AM

I don't see why the EXTRACT function I did in the LIKE + challenge wouldn't work with this challenge. No callbacks , libraries just load to a string from a text file with one statement and use EXTRACT to grab the data you want then use the SB IUP extension module to display it. You can use LED for the form creation without the need to code the UI.

The REPLACE function would allow you to make changes if you needed programmaticly.

With one code base your application would run native on Windows and Linux.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 19, 2018, 03:48:37 PM

AIR has seen what I'm trying to do - something that is stupidly flexible and easy to configure for a new user.

Essentially, everything has an ini file (or similar) - all in plain English (or whatever language). right now, with VB, I'm stuck with the ASCII character set, but that will change in the future. Whatever happens, I'm actually enjoying writing this software.

Okay, here's the bit I'm currently working on - Index handling.

Like everything else, the primary index is a text file (fixed width record, CrLf termination) and has an external schema.

So - here's a few lines of my test index (yes, it's a part of my mineral collection - makes for excellent test dat for this kind of thing) - there's about 2000 lines of that stuff. The first column is for a "hidden record" flag, and I've trimmed off most of the trailing spaces.

Code: [Select]

 |Unique |     |Accession| |other stuff|           |Collection          |  |Analysis | |Field     |
|number |     |reference|                         |name                |  |reference| |reference |
 ARC00796       M00796                              Minerals                                                                                        
 ARC00797       M00797                              Minerals                            F199709144                                                  
 ARC00798       M00798                              Minerals                                                                                        
 ARC00799       M00799                              Minerals                                                                                        
 ARC00800       M00800                              Minerals                A19990301   F199903261                                                  
 ARC00801       M00801                              Minerals                            F199903261                                                  
 ARC00802       M00802                              Minerals                            F199903261

The first two lines do not appear in the index - they're just for your reference.

And here's the schema for what is actually in there (and I do allow comments in there)

Code: [Select]

Schema:
      Database =       Mineral DB
      Filename =       DBPrimaryIndex.schema
      Target =         DBPrimaryIndex.index
      Size =           148
      ;                Record size of index in bytes - will be padded to fit.
      ;                - the index *is* a random access file, after all.

;
; Type:
;       0      = String
;       1      = Boolean String 'True'/'False'
;       2      = Long Integer
;      98      = Deletion Flag ('D' / 'H' / 'X' / ' ')  (= Deleted / Hidden / Expunged / normal)
;      99      = String / Autonumber
;                       Prefix =          (string)
;                       Body =            (picture)
;                       Suffix =          (string)
;                       Datestamp =        null or (picture)

; Class:
;      [none]  = Default index type (plain secondary index)
;      PK      = Primary Key (auto numbered sequence number)
;      SK      = Secondary Key (usually the fully formatted version of the PK, 
;                which should be an integer
;      FG      = File Group  - by which record cards are arranged on disk
;                              (directory structure)
;      SG      = Secondary File Group - by which records without an FG
;                                       value are grouped on disk.
;


Field:
                     ; DELETION flag (Boolean)
      Name =           Deleted
      LPos =           1
      Width =          1
      Type =           98
      Class =
      Path =           NONE
      XPath =

Field:
                     ; Sequential DB entry number  - a SUBSET of the ARC Number
                     ; SeqNo
      Name =           SeqNo
      LPos =           5
      Width =          6
      Type =           99
      Class =          PK
      Path =           Main.idx
      XPath =

Field:
                     ; Sequential entry number
                     ; ARCNo
      Name =           ARCNo
      LPos =           2
      Width =          15
      Type =           02
      Class =          SK
      Path =           Main.idx
      XPath =
      Prefix =         ARC
      Body =           9999999
      Suffix =         0
      Timestamp =      0
                     ; System timestamp (19 characters for 'Now()' )

Field:
                     ; Accession Number is the specimen reference for
                     ; all accessioned specimens.
                     ; AccNo
      Name =           Accession No.
      LPos =           17
      Width =          12
      Type =           0
      Class =          FG
      Path =           AccNo.idx
      XPath=           Acquiry Reference

Field:
                     ; Field Number or other reference to material that
                     ; hasn't been accessioned as collection specimens.
                     ; AcqRef
      Name =           Acqusition Reference
      LPos =           29
      Width =          24
      Type =           0
      Class =          SG
      Path =           AcqRef.idx
      XPath =

Field:
                     ; Sub Collection Name
                     ; SubColl
      Name =           SubCollection Name
      LPos =           53
      Width =          24
      Type =           0
      Class =
      Path =           NONE
      XPath =

Field:
                     ; Analytical Reference
                     ; Aref
      Name =           Analytical Reference
      LPos =           77
      Width =          12
      Type =           0
      Class =
      Path =           NONE
      XPath =

Field:
                     ; Field Reference
                     ; FRef
      Name =           Field Reference
      LPos =           89
      Width =          12
      Type =           0
      Class =
      Path =           NONE
      XPath =

End:

There will be something similar for the sub-indices (things like dictionary searchable fields etc).

Each schema that gets read into the program joins the schema structure - and, I was surprised when I came up with this, it allows multiple document bases to operate alongside each other natively. With careful arrangement of the fields, it is possible to define a composite field that includes parts of two or more fields concatenated, or even a subset of a single field (SeqNo is the numerical part of ARCNo). As you may have guessed, SeqNo is also the record number in the primary index, and will reference every related item in the system.

(In case you wonder, the short name in the comment is the field mnemonic from the original Access DB)

I have done something similar with the software configuration file - except that I also allow #INCLUDEs in order to allow the stacking of INI parameters to include such things as defaults, new configurations and session information.

You broke the custom config file? Just delete the contents of THAT file, and the configuration reverts to the defaults. There's even a tool for debugging your config ;)

Once I finish the to do list on that chunk of effort, I'll put it up, too.

Title: Re: Database Code Challenge(s)
Post by: John on November 19, 2018, 05:58:45 PM

Quote

right now, with VB, I'm stuck with the ASCII character set,

If you use Krool's common controls OCX, you would have unicode support along with current OS theming.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 20, 2018, 01:33:35 AM

Good point. I had quite forgotten.

Title: Re: Database Code Challenge(s)
Post by: John on November 20, 2018, 01:46:38 PM

AIR has already offered a promising solution for the back end text DB. Can you share what you have or envisioned for a UI?

Title: Re: Database Code Challenge(s)
Post by: AIR on November 20, 2018, 04:53:25 PM

Alysson's VB Demo project updates the timestamp of the "record".

Here's how I would do it using Python and yAML (I had to write some custom code so that the output was as close to the original as possible):

Code: Python

#!/usr/bin/env python
 
from datetime import datetime
import air_yaml as yaml
 
with open('testI.yml') as fp:
    record = yaml.load(fp)
 
record["Previous Version"] = record["LastSavedTime"]
record["LastSavedTime"] =  datetime.now().strftime("%d/%m/%Y %H:%M:%S")
 
with open('result.yml', 'wb+') as fp:
    yaml.dump(record, fp, default_flow_style=False, width=79, Dumper=yaml.AIRDumper)

Code: YAML

# ================================================================================
RecordTitle: Mineral Specimen Details
# ================================================================================
Entry No: ARC-99-000021
# ================================================================================
Species:
  Minerals:
  - Zeunerite
  - Rutherfordine
  - Zippeite
  - Chalcopyrite
  - var:
    - Blister Copper
    - Quartz
    - Agate
  - colour:
    - White
    - Opal
    - Transparent
    - var:
      - Common Opal
  - Matrix: Quartz
# ================================================================================
Location:
  Address: 'South Terras Mine
 
    St Stephen in Brannel
 
    St Austell
 
    Cornwall, UK
 
    '
  Grid Reference: SW 935 524
  GPS Coordinates: 50.33444,-4.90194
# ================================================================================
Collection Data:
  collected by: Self collected
  Collection Date: 1979
# ================================================================================
Acquisition Data:
  Acquired Date: 14 Jul 1998
  From: Self collected
  Field Ref No: F20180623-STUM-001
  Temp Ref No: TMP 0 0099
# ================================================================================
Analytical Data:
  Commentary: 4 species
  Zeunerite:
    .comment: Colour and crystal form are typical for this location.
    .date: 14 Aug 2018
    .Report: AN-201808-14-01
    .Ref_Number: ZZ8088
  Zippeite:
    .comment: Colour is diagnostic for this location
    .date: 14 Aug 2018
    .Report: AN-201808-14-02
    .Ref_Number: ZZ9099
  Chalcopyrite:
    .comment: Well characterised at this location. Deep indigo tarnish on old surfaces.
  Rutherfordine:
    .comment: Tentative ID.
    .Reasons: Visual ID only
# ================================================================================
Accession Data:
  Accession No: ZZ9099
  Accession Date: 21 Sept 2000
  Sub-collection: Main
# ================================================================================
Image Data:
  Mugshot: img0001.jpg
  Images:
  - .filename: img0001.jpg
    .Title: Whole Specimen
    .TakenDate: 11 Nov 2018
    .Width: 11.5mm
    .camera: Fuji FinePix L55
  - .filename: img0002.jpg
    .Title: Zeunerite & Chalcopyrite
    .TakenDate: 11 Nov 2018
    .FOV: 1.1mm
    .Camera: CL 1600 USB MicroCam
# ================================================================================
RecordType: 1
# ================================================================================
LastSavedTime: 20/11/2018 19:43:31
# ================================================================================
Previous Version: 13/11/2018 13:18:37
# ================================================================================
 

I still need to fix the "Address" output, but that's pretty much all that's left...

AIR.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 20, 2018, 05:34:06 PM

@Alysson -

I took a deep dive into your code, it's really nice!!

Great Job!

I only wish that VB6 wasn't such a stickler for _stdcall in DLL's, it's the main reason I left it many years ago for Delphi/Lazarus....

AIR.

Title: Re: Database Code Challenge(s)
Post by: John on November 20, 2018, 05:36:21 PM

Peter Verhas wrote a template like web page generation system where data would replace the tags in the document. I haven't spent enough time looking at the challenge to see if it would apply. Peter called it HEB. (Html Embedded Basic)

Quote from: Peter Verhas

EXTERNAL PREPROCESSOR
This module executes external preprocessors. These preprocessors are standalone executable programs that read the source program and create another file that is read and processed by the ScriptBasic interpreter. If an external preprocessor is used the source file is usually not BASIC but rather some other language, usually a BASIC like language, which is extended some way and the preprocessor creates the pure ScriptBasic conformant BASIC program. The sample preprocessor supplied with ScriptBasic is the HEB (HTML Embedded BASIC) preprocessor that reads HTML embedded BASIC code and creates BASIC program. This HEB source file is a kind of HTML with embedded program fragments, which you may be familiar with in case you program PHP or Microsoft BASIC ASP pages. The HEB preprocessor itself is written in BASIC and is executed by ScriptBasic. Thus when a HEB "language" is executed by ScriptBasic it starts a separate instance of the interpreter and executes the HEB preprocessor on the source file. Of course the HEB preprocessor could be implemented in any language that can be compiled or some way executed on the target machine. Actually the very first version of the HEB preprocessor was written in Perl so when it was first tested the ScriptBasic interpreter started a Perl interpreter before reading the generated BASIC code.

Note that the HEB preprocessor provided in the ScriptBasic package is an example implementation and lacks many features. It can, for example, be fooled by putting a %> characters into a BASIC string constant.

Title: Re: Database Code Challenge(s)
Post by: John on November 20, 2018, 06:26:22 PM

Quote from: AIR

I only wish that VB6 wasn't such a stickler for _stdcall in DLL's, it's the main reason I left it many years ago for Delphi/Lazarus....

The SB VB6 connection gives you a choice of how much of VB6 you want to depend on in your app.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 21, 2018, 01:45:26 AM

Quote from: AIR on November 20, 2018, 05:34:06 PM

@Alysson -

I took a deep dive into your code, it's really nice!!

Great Job!

I only wish that VB6 wasn't such a stickler for _stdcall in DLL's, it's the main reason I left it many years ago for Delphi/Lazarus....

AIR.

Thanks. Coming from you that is a BIG compliment.

I must admit to having considered Lazarus, but my Pascal is almost as rusty as my FORTRAN, and dates back to TurboPascal V2.xx (hell's bells, I was using it on an 8-bit home micro last time I used Pascal in anger).

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 21, 2018, 02:01:29 AM

Quote from: John on November 20, 2018, 01:46:38 PM

AIR has already offered a promising solution for the back end text DB. Can you share what you have or envisioned for a UI?

That is actually a bit of a question.

I intend to use two separate user interfaces - active (admin, data input/update etc.) and passive (catalogue browse)

The catalogue browser will be auto-generated web pages including an off-the shelf search engine. I intend to use a cardfile metaphor for this (old wood, manila widgets, cream laid cards, traditional form font, typewriter data).

I've attached a mock-up that I made some time ago - lacking the quality of the real thing, of course.

The administrative interface is still up in the air, though it is likely to use a current widget set whatever gets used.

Title: Re: Database Code Challenge(s)
Post by: John on November 21, 2018, 10:07:25 AM

Peter's HEB SB pre processor is sounding better by the post. 8)

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 21, 2018, 02:31:22 PM

Quote from: John on November 21, 2018, 10:07:25 AM

Peter's HEB SB pre processor is sounding better by the post. 8)

Could well be.

Let me get the functionality up and running, and then I can think about front-end.

Title: Re: Database Code Challenge(s)
Post by: John on November 21, 2018, 09:26:57 PM

I don't want to get you too excited but you could be running those slow processes of yours in SB threads.

If I were to take a stab at this, I would compile a core SB with the interpreter running in a DLL supporting a thread safe environment. Would be hard to beat.

I'm going to wait till your fully dressed before dancing to this.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 21, 2018, 10:35:49 PM

Wanted to try something on Windows, so I coded up this HotBasic demo.

It's using libyaml, compiled to a 32bit dll using MinGW (I had to create my own makefile since their autoconf/cmake configs kept looking for MS Visual C and nmake no matter what I told it to do).

The attached file includes the source, a ready to run binary, the yaml file itself and the dll. I tested under both Win7 and Win10.

Code: Text

$APPTYPE CONSOLE: $TYPECHECK ON
$INCLUDE "yaml.inc"
 
type COLLECTIONDATA
        collected_by as string
        collected_date as string
end type
 
type MINERALS
        name as string*64
        var as string*64
        colors as string*64
        color_var as string*64
        matrix as string*64
end type
 
type IMAGE
        mugshot as string
end type
 
type SPECIES
        minerals as MINERALS
end type
 
type LOCATION
        address as string
        grid as string
        gps as string
end type
 
type ACQUISITION
        date as string
        from as string
        fieldref_no as string
        tempref_no as string
end type
 
type DBINFO
        rec_type as string
        last_save as string
        previous_save as string
end type
 
 
type record
        dummy as string
        title as string
        entry as string
        species as SPECIES
        location as LOCATION
        collection as COLLECTIONDATA
        aquisition as ACQUISITION
        dbinfo as DBINFO
end type
 
 
defstr yamlFile,tk,key,token
deflng parser,ret, indent, child,state=0
dim conf as record
 
 
yamlFile.loadfromfile("testI.yml")
 
 
ret = yaml_parser_initialize(@parser)
 
 
yaml_parser_set_input_string(@parser, yamlFile, yamlFile.len)
 
do
        yaml_parser_scan(@parser, @token)
 
        select case byref(@token)
        '       case YAML_BLOCK_SEQUENCE_START_TOKEN,YAML_BLOCK_MAPPING_START_TOKEN
        '           if child > 0 then
    '             inc indent,2
    '         else
    '             inc child
        '                       ' print "#" + string$(78,"=")
    '         end if
        '       case YAML_BLOCK_END_TOKEN
        '               dec indent,2
        '               dec child
                        
                case YAML_KEY_TOKEN 
                        state = 0
                case YAML_VALUE_TOKEN
                        state = 1
                case YAML_SCALAR_TOKEN
                        tk = byref$(byref(@token+4))
 
                        if state = 0 then
                                key = tk
                        end if
 
                        if state = 1 then
                                select case key
 
                                        ' ## HEADER INFO ##
                                        case "RecordTitle"
                                                conf.title = tk
                                        case "Entry No"
                                                conf.entry = tk
 
                                        ' ## LOCATION INFO ##
                                        case "Address"
                                                conf.location.address = tk
                                        case "Grid Reference"
                                                conf.location.grid = tk
                                        case "GPS Coordinates"
                                                conf.location.gps = tk
 
                                        ' ## COLLECTION INFO ##
                                        case "collected by"
                                                conf.collection.collected_by = tk
                                        case "Collection Date"
                                                conf.collection.collected_date = tk
 
                                        ' ## ACQUISITION INFO ##
                                        case "Acquired Date"
                                                conf.aquisition.date = tk
                                        case "From"
                                                conf.aquisition.from = tk
                                        case "Field Ref No"
                                                conf.aquisition.fieldref_no = tk
                                        case "Temp Ref No"
                                                conf.aquisition.tempref_no = tk
 
                                        ' ## DB SYSTEM INFO ##
                                        case "RecordType"
                                                conf.dbinfo.rec_type = tk
                                        case "LastSavedTime"
                                                conf.dbinfo.last_save = tk
                                        case "Previous Version"
                                                conf.dbinfo.previous_save = tk
                                '       case else
                                '               ' print "Unknown key = " + key
                                end select
 
                                
                                ' print space$(indent) + key + " = " + tk
                                ' if indent = 0 then
                                        
                                ' end if
                        end if
 
                case else
                        state = 1
        end select
        
 
 
loop until byref(@token) = YAML_STREAM_END_TOKEN
 
'PAUSE
 
gosub cleanup
gosub print_data
 
END
 
cleanup:
        yaml_token_delete(@token)
        yaml_parser_delete(@parser)
return
 
print_data:
        cls
        print "Title = "; conf.title
        print "Entry# = "; conf.entry
        print "Address = "; conf.location.address
        print "Grid = ";conf.location.grid
        print "GPS = ";conf.location.gps
        print "Collected By = ";conf.collection.collected_by
        print "Collected Date = ";conf.collection.collected_date
        print "Aquisition Date = ";conf.aquisition.date
        print "Aquired From = ";conf.aquisition.from
        print "Field Ref No = ";conf.aquisition.fieldref_no
        print "Temp Ref No = ";conf.aquisition.tempref_no
        print "DB Rec Type = ";conf.dbinfo.rec_type
        print "Last Saved = ";conf.dbinfo.last_save
        print "Previous Save = ";conf.dbinfo.previous_save
return

Title: Re: Database Code Challenge(s)
Post by: John on November 21, 2018, 11:26:33 PM

One idea is use SB associtive arrays as a form of of readable DB data that has immediate functionality.

Code: ScriptBasic

COLLECTIONDATA{"collected_by"} = "Alysson"
COLLECTIONDATA{"collected_date"} = NOW
 

This embedded in a preproessor HEB like format gives you a DB and web page definition in one file / record.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 21, 2018, 11:32:30 PM

Quote from: John on November 21, 2018, 11:26:33 PM

One idea is use SB associtive arrays as a form of of readable DB data that has immediate functionality.

Code: ScriptBasic
COLLECTIONDATA{"collected_by"} = "Alysson"
COLLECTIONDATA{"collected_date"} = NOW

This embedded in a preproessor HEB like format gives you a DB and web page definition in one file / record.

All well and good, but you need to get the data from the file first. So how would you approach that?

Title: Re: Database Code Challenge(s)
Post by: John on November 21, 2018, 11:38:58 PM

The associative array definitions is the data along with the HEB definition that would display it (maybe as an include) all as a record.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 21, 2018, 11:41:33 PM

Quote from: John on November 21, 2018, 11:38:58 PM

The associative array definitions is the data along with the HEB definition that would display it all as a record.

So....you're going to type all of that in every time? ;D

Alysson has the data serialized in a file. How do you get that into the AA? And back out in the required format?

Title: Re: Database Code Challenge(s)
Post by: John on November 21, 2018, 11:44:27 PM

I'm going to let Alysson chew on this for awhile before I refine the concept any further.

Required Format?

Title: Re: Database Code Challenge(s)
Post by: jalih on November 22, 2018, 12:25:47 AM

Quote from: AIR on November 21, 2018, 11:32:30 PM

All well and good, but you need to get the data from the file first. So how would you approach that?

Too bad using JSON for data is not allowed. I would have used 8th for this challenge. I think JSON is readable enough and if needed nicely formatted presentation of the data needs just a few lines of code.

In 8th, only few lines of code is needed to read JSON data from file to map. Following code reads data from the file and outputs keys and data from the resulting map:

Code: [Select]

"db.json" f:slurp json>
( swap "%s : %s" s:strfmt . cr ) m:each drop

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 12:32:00 AM

Quote from: John on November 21, 2018, 11:44:27 PM

I'm going to let Alysson chew on this for awhile before I refine the concept any further.

Required Format?

Quote from: AlyssonR

The primary requirement is that the textual data must be kept entirely in human-readable digital format (i.e. text files), thus, software-proofing the data. That includes indices and data (images and other media excepted). The artefact record is, in fact, a single document. Even EXIF-type metadata on media files is maintained in separate text documents alongside the media files.

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 12:37:41 AM

The associative array definitions in a text file aren't readable? The web site definitions and 'meta data' can reside in the included template file. All run by the pre-processor threaded.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 12:42:48 AM

Quote from: John on November 22, 2018, 12:37:41 AM

The associative array definitions in a text file aren't readable?

Have you looked at the sample data file Alysson posted? She also requested that when printed the files should look like a report, no matter what was used to do the printing.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 12:46:05 AM

Quote from: jalih on November 22, 2018, 12:25:47 AM

Too bad using JSON for data is not allowed. I would have used 8th for this challenge. I think JSON is readable enough and if needed nicely formatted presentation of the data needs just a few lines of code.

In 8th, only few lines of code is needed to read JSON data from file to map. Following code reads data from the file and outputs keys and data from the resulting map:
Code: [Select]
"db.json" f:slurp json> ( swap "%s : %s" s:strfmt . cr ) m:each drop

I'm curious to see what you mean. I've attached a json version of the yaml data file I used....

AIR.

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 12:46:28 AM

The question becomes is the goal to maintain a broken design or make it run under today's standards?

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 12:53:20 AM

Quote from: John on November 22, 2018, 12:46:28 AM

The question becomes is the goal to maintain a broken design or make it run under today's standards?

The question is can the code challenge be met as originally presented.

I personally don't have an issue with the design, as you put it. When I saw the format of the data file, it immediately made me think of yaml formatted files. I'd just never seen yaml used to do what Alysson is doing. So where you see something as broken, I see someone pushing the envelop....

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 12:58:04 AM

Quote from: John on November 22, 2018, 12:37:41 AM

The associative array definitions in a text file aren't readable? The web site definitions and 'meta data' can reside in the included template file. All run by the pre-processor threaded.

You added the second sentence while I was posting a reply.

All I can say is: show me....

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 01:05:21 AM

I m still in concept definition mode and coding will come when I see an example of something working and Alysson smiling.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 22, 2018, 02:27:36 AM

That's why I didn't go for associative arrays - the data is intrinsically too flexible for most approaches ... to the point of being positively limp. (I think I may use that expression in the documentation ;D )

What I have actually done is to serialise everything with no expectations as to meaning whatsoever; any meanings are drawn from the index schema and will, at some point, include logic to utilise a field value as a link to a file (probably by using a property such as .link [file path] )

As to background processes, a lot of the routine maintenance tasks will, hopefully, be attended to using SB long before I get around to converting the program logic.

I find myself really mourning the lack of Type (or other data structure) definitions in SB (or have I missed something here??)

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 03:03:35 AM

Quote

I find myself really mourning the lack of Type (or other data structure) definitions in SB (or have I missed something here??)

SB supports LONG, REAL, (binary) STRING, ARRAY and UNDEF. Arrays can be index or associative or a combination of both with no practical limitation. A great way to build complex structures.

You can pass an UNDEF variable as an argument to a function/sub and return a complex array structure in the same variable name.

Title: Re: Database Code Challenge(s)
Post by: jalih on November 22, 2018, 06:47:30 AM

Quote from: AIR on November 22, 2018, 12:46:05 AM

I'm curious to see what you mean. I've attached a json version of the yaml data file I used....

Here is some demo code to read JSON file into map and display the content on screen...

Code: [Select]

var db


: load
  "testI.json" f:slurp json> db ! ;


: number-or-string
  . cr ;


defer: iter


: array
  cr ( nip space space space space dup >kind iter cr ) a:each drop ;


: map
  cr ( swap space space . ": " . dup >kind iter ) m:each drop ;


: iterator
  [ ' number-or-string , ' noop , ' noop , ' number-or-string , ' array , ' map ]  swap
  caseof ;


' iterator w:is iter


: disp
  db @ ( swap . ": " . dup >kind iterator ) m:each drop ;


load disp
bye

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on November 22, 2018, 07:40:08 AM

Quote from: John on November 22, 2018, 03:03:35 AM

Quote
I find myself really mourning the lack of Type (or other data structure) definitions in SB (or have I missed something here??)

SB supports LONG, REAL, (binary) STRING, ARRAY and UNDEF. Arrays can be index or associative or a combination of both with no practical limitation. A great way to build complex structures.

You can pass an UNDEF variable as an argument to a function/sub and return a complex array structure in the same variable name.

I suppose arrays of arrays of arrays of arrays would do it ;)

Use an associative array for the sub property classes ....

I'll sleep on it.

I'll have a play when I have time.

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 09:16:57 AM

Any arrary (matrix) segment can be assigned to a new array or a element of itself. Structures on steroids.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 10:06:19 AM

Quote from: jalih on November 22, 2018, 06:47:30 AM

Quote from: AIR on November 22, 2018, 12:46:05 AM
I'm curious to see what you mean. I've attached a json version of the yaml data file I used....

Here is some demo code to read JSON file into map and display the content on screen...

Thanks, jalih, that's real interesting.

Does 8th support maintaining the order of the data when you output it?

Title: Re: Database Code Challenge(s)
Post by: jalih on November 22, 2018, 11:01:40 AM

Quote from: AIR on November 22, 2018, 10:06:19 AM

Does 8th support maintaining the order of the data when you output it?

8th seems to store the data in different order, so iterating map directly changes order of the data outputted. I guess I would need to iterate the map keys in correct order myself.

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 11:18:14 AM

Maybe a future code challenge could be a JSON parser.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 12:20:15 PM

Quote from: jalih on November 22, 2018, 11:01:40 AM

Quote from: AIR on November 22, 2018, 10:06:19 AM
Does 8th support maintaining the order of the data when you output it?

8th seems to store the data in different order, so iterating map directly changes order of the data outputted. I guess I would need to iterate the map keys in correct order myself.

Most hash/map implementations don't maintain order, for efficiency. In Python, for example, the Dict object doesn't maintain order. But the OrderedDict does while being less efficient. Other languages offer this option, but not all.

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 12:27:35 PM

Associative arrays seems the only way I know of taming these hash lists.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 01:28:10 PM

Quote from: John on November 22, 2018, 12:27:35 PM

Associative arrays seems the only way I know of taming these hash lists.

Same thing, different names. The question is does it maintain order, or do you have to handle that programatically?

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 01:48:51 PM

In SB it's about creating freeform structures that can contain any of the data types SB supports. LBOUND/UBOUND are key functions to keep it all together.

No matter how complex your array structure gets, you can iterate though it using standard indexing methods.

Arrays in ScriptBasic (https://www.scriptbasic.org/forum/index.php/topic,355.msg1639.html#msg1639)

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 02:59:31 PM

Quote from: John on November 22, 2018, 01:48:51 PM

In SB it's about creating freeform structures that can contain any of the data types SB supports. LBOUND/UBOUND are key functions to keep it all together.

No matter how complex your array structure gets, you can iterate though it using standard indexing methods.

So....you have to handle maintaining the original structure programmatically. There's nothing bad about that, why not just say that instead of the marketing pitch?

Anyway, we've derailed the original topic enough...

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 03:07:06 PM

Quote

Anyway, we've derailed the original topic enough

Just chatting at the water cooler waiting for Alysson to return. I see this challenge as extended. How about you getting us going on something else to keep things moving?

There are no sales pitching with open source. It's called advocating for the project.

Title: Re: Database Code Challenge(s)
Post by: AIR on November 22, 2018, 05:33:54 PM

Quote from: John on November 22, 2018, 03:07:06 PM

How about you getting us going on something else to keep things moving?

I think your json suggestion could work. With the stipulation that it be done in native code (meaning no external libs, AND if the language you use already has support for it you don't that capability and instead roll your own). Otherwise it's not much of a challenge.

Title: Re: Database Code Challenge(s)
Post by: John on November 22, 2018, 06:16:14 PM

Sounds great!

I didn't use SB LIKE for the challenge EXTRACT. SB needs an easy to use JSON function.

http://www.json.org

Would rewriting the jsmn library in C BASIC count? :)

Please kick off the new challenge thread when you have time.

Title: Re: Database Code Challenge(s)
Post by: AlyssonR on December 23, 2018, 01:14:26 AM

Hi all,

I've taken a quick peek at the thread - and it looks like everyone's been having fun with it.

I've been on extended domestic maintenance (i.e. fixing stuff around the house AND fixing myself), and will likely only look in on occasion for a while more :( The trouble is that I have far too much on the go right now with harware, software AND hard/software projects simmering as well as trying to keep abreast of everything that needs fixing around the house.

Title: Re: Database Code Challenge(s)
Post by: John on December 23, 2018, 12:22:48 PM

You sound busier than Santa's elves. :)

Title: Re: Database Code Challenge(s)
Post by: AIR on March 04, 2019, 02:24:19 AM

I wanted to give this a try using GO, here is what I have:

Code: Go

package main
 
import (
        "fmt"
        "io/ioutil"
        "log"
        "strings"
 
        "gopkg.in/yaml.v2"
)
 
func getRoot(obj interface{}) map[interface{}]interface{} {
        return obj.(map[interface{}]interface{})
}
func getSection(obj interface{}, field string) map[interface{}]interface{} {
        dict := obj.(map[interface{}]interface{})[field].(map[interface{}]interface{})
        return dict
}
 
func dump(obj map[interface{}]interface{}, indent ...string) {
        for key, val := range obj {
                if key == "Address" {
                        val = strings.ReplaceAll(val.(string), "\n", "\n\t\t   ")
                }
                if indent != nil {
                        fmt.Println(indent[0], "\t", key, "=", val)
                } else {
                        fmt.Println("\t", key, "=", val)
                }
        }
}
 
func getArray(obj interface{}) []interface{} {
        dict := obj.([]interface{})
        return dict
}
 
func dumpArray(obj interface{}) {
        for _, field := range obj.([]interface{}) {
                switch t := field.(type) {
                case string:
                        fmt.Println("\t\t", t)
                case map[interface{}]interface{}:
                        set := []interface{}{t["var"], t["colour"], t["Matrix"]}
                        // x := t["var"]
                        for _, x := range set {
                                if x != nil {
                                        fmt.Println("\t\t", x)
                                }
                        }
                }
        }
}
 
func dumpList(obj interface{}) {
        for _, x := range obj.([]interface{}) {
                dump(x.(map[interface{}]interface{}))
                println()
        }
}
 
type Rec interface{}
 
func main() {
        var rec Rec
        sep := strings.Repeat("-", 40)
        data, err := ioutil.ReadFile("test.yml")
        if err != nil {
                log.Fatal(err)
        }
 
        err = yaml.Unmarshal(data, &rec)
        if err != nil {
                log.Fatal(err)
        }
 
        root := getRoot(rec)
        recordTitle := root["RecordTitle"]
        entryNumber := root["Entry No"]
        recordType := root["RecordType"]
        lastSavedTime := root["LastSavedTime"]
        previousVersion := root["Previous Version"]
        Species := getRoot(root["Species"])
        Minerals := getRoot(Species)["Minerals"]
 
        location := getSection(rec, "Location")
        collectionData := getSection(rec, "Collection Data")
        acquisitionData := getSection(rec, "Acquisition Data")
        analyticalData := getSection(rec, "Analytical Data")
        accessionData := getSection(rec, "Accession Data")
        imageData := getSection(rec, "Image Data")
 
        fmt.Println(sep)
        fmt.Println("RecordTitle =", recordTitle)
        fmt.Println("Entry No =", entryNumber)
        fmt.Println(sep)
 
        fmt.Println("Species\n\tMinerals")
        dumpArray(Minerals)
        fmt.Println(sep)
 
        fmt.Println("Location")
        dump(location)
        fmt.Println(sep)
 
        fmt.Println("Collection Data")
        dump(collectionData)
        fmt.Println(sep)
 
        fmt.Println("Acquisition Data")
        dump(acquisitionData)
        fmt.Println(sep)
 
        fmt.Println("Analytical Data")
        fmt.Println("\tCommentary =", analyticalData["Commentary"])
 
        zeunerite := getRoot(analyticalData["Zeunerite"])
        fmt.Println("\n\tZeunerite")
        dump(zeunerite, "\t")
 
        zippeite := getRoot(analyticalData["Zippeite"])
        fmt.Println("\n\tZippeite")
        dump(zippeite, "\t")
        fmt.Println(sep)
 
        fmt.Println("Accension Data")
        dump(accessionData)
        fmt.Println(sep)
 
        fmt.Println("Image Data")
        fmt.Println("\tMugshot =", imageData["Mugshot"], "\n")
        images := imageData["Images"]
        dumpList(images)
        fmt.Println(sep)
 
        fmt.Println("DB Info")
        fmt.Println("\tRecordType =", recordType)
        fmt.Println("\tLastSavedTime =", lastSavedTime)
        fmt.Println("\tPrevious Version =", previousVersion)
        fmt.Println(sep)
}
 

And here is the output:

---------------------------------------- RecordTitle = Mineral Specimen Details Entry No = ARC-99-000021 ---------------------------------------- Species Minerals Zeunerite Rutherfordine Zippeite Chalcopyrite [Blister Copper Quartz Agate] [White Opal Transparent map[var:[Common Opal]]] Quartz ---------------------------------------- Location Address = South Terras Mine St Stephen in Brannel St Austell Cornwall, UK Grid Reference = SW 935 524 GPS Coordinates = 50.33444,-4.90194 ---------------------------------------- Collection Data collected by = Self collected Collection Date = 1979 ---------------------------------------- Acquisition Data From = Self collected Field Ref No = F20180623-STUM-001 Temp Ref No = TMP 0 0099 Acquired Date = 14 Jul 1998 ---------------------------------------- Analytical Data Commentary = 4 species Zeunerite .Report = AN-201808-14-01 .Ref_Number = ZZ8088 .comment = Colour and crystal form are typical for this location. .date = 14 Aug 2018 Zippeite .comment = Colour is diagnostic for this location .date = 14 Aug 2018 .Report = AN-201808-14-02 .Ref_Number = ZZ9099 ---------------------------------------- Accension Data Accession Date = 21 Sept 2000 Sub-collection = Main Accession No = ZZ9099 ---------------------------------------- Image Data Mugshot = img0001.jpg .filename = img0001.jpg .Title = Whole Specimen .TakenDate = 11 Nov 2018 .Width = 11.5mm .camera = Fuji FinePix L55 .filename = img0002.jpg .Title = Zeunerite & Chalcopyrite .TakenDate = 11 Nov 2018 .FOV = 1.1mm .Camera = CL 1600 USB MicroCam ---------------------------------------- DB Info RecordType = 1 LastSavedTime = 13/11/2018 13:18:37 Previous Version = 13/11/2018 12:12:47 ----------------------------------------

I've attached Linux and Windows binaries (scanned with VirusTotal, clean) cross-compiled on my Mac.

AIR.

Title: Re: Database Code Challenge(s)
Post by: John on March 04, 2019, 07:34:09 AM

GO AIR! :)

AllBASIC Forum

BASIC User Group => Code Challenges => Topic started by: John on November 16, 2018, 12:27:27 PM