Poll

What is your language preference to build Oxygen Basic?

Oxygen Basic
2 (40%)
C/C++/ANSI C
1 (20%)
Assembler
0 (0%)
BCX
2 (40%)
FreeBASIC
0 (0%)
Other
0 (0%)

Total Members Voted: 0

Author Topic: Alternative O2 Builder  (Read 25427 times)

JRS

  • Guest
Alternative O2 Builder
« on: September 23, 2010, 01:53:58 AM »
Charles,

I have been sifting through your source code and from what I see, FreeBASIC is being used as an scripting engine to build a ASM source code program. Instead of using an external assembler, O2 embeds it's generated source as inline ASM statements which FreeBASIC assembles and does any linking that needs to be done.

For grins, lets say I was able to convert the FreeBASIC version of O2 to ScriptBasic and used the gcc's inline ASM feature to create the O2 compiler. Is this theoretically possible?

Wouldn't that offload all that work you have to do to be C friendly? (read C headers, let C be your generic interface helper shell, expand to cross platform, ...)

John

Quote from: Charles - FreeBASIC Forum
Hi everyone!

Thank you very much for all your interesting comments on Oxygen Basic. I was about to announce the project on this forum, but John got in there first :)

I am more than satisfied with FreeBasic as a development language. While I have not made use of its more advanced features, it has proved to be very reliable, and interfaces nicely with assembly code. Thank you Joshy for posting the compatibility fix with FB 0.21

O2 was conceived nearly three years ago as a machine script, in essence machine code with a linker. It was an experiment to see how minimal a compilable language could be. Then came the Assembler shortly followed by a macro language. Eros Olmi (of thinBasic) suggested more people would be able to understand it if I added Basic-like syntax. So I thought okay, I'll have this ready in a few weeks. Little did I know how much work was involved in producing a high level compilable language. A compiler has to spend much of its time doing invisible things like automatic type conversion and operator precedence and one is not fully aware of these complexities until you try to encode them.

Even before adding Basic, I had OOP capability in place. I think retrofitting it to a straight procedural language would have been much more difficult which I think is why many Basics have got stuck in their development.

One very importand and most recent feature in the development of o2 is the ability to read C headers as well as Qbasic headers. This requires O2 to understand quite a large chunk of C itself. So far, this fusion of C and basic has not been too problematic. A Few directives are needed. For instance partial case sensitivity: #case capital which distinguishes fully capitalised keywords.

I use the term JIT because O2 compiles directly to an executable memory image which may be used immediately without first storing it to an EXE file. For programs up to about 40-50k of source code, the compilation time is barely noticeable. This is good for ad-hoc programming where you are not building a static application. The source code is the only item that needs to be stored.

Currently O2 compiled executables depend on Oxygen.dll for the run-time library, but I am working on the 'independent' mode where the run-time library is included. This is of course essential when generating 64bit code with a 32 bit compiler.

I am still devoted to GOTOs. But I have only used them very sparingly 372 times :D. Many of these are error trapping points, and many GOTOs are directed towards a small number of nodes in the program.

It will take about three more months to approach Beta quality releases. Much of the testing at this stage involves exposing Oxygen to as many different situations as possible and testing with various APIs.


Charles

http://oxygenbasic.sourceforge.net
 

« Last Edit: September 23, 2010, 03:35:28 AM by JRS »

cevpegge

  • Guest
Re: Alternative O2 Builder
« Reply #1 on: September 23, 2010, 03:52:26 AM »

John,

Oxygen makes intensive use of FreeBasic Inline Assembler (which is slightly preprocessed GNU GAS using Intel notation). The Oxygen Assembly language is largely compatible with this. We also have a high degree of compatibility between FreeBasic and OxygenBasic language. Then there is the evolving C compatibility. And operating system dependencies are absolutely minimal at this stage so I hope this is sufficient to keep Oxygen's porting options open.

However porting to SB, being a scripting language would pose a few problems like being unable to tightly couple Assembly code with the host script. But theoretically any Turing machine will do.

Looking back over the last 35 years. What has been the most stable language? X86 Assembler!  ;D Who would have thought it! But so many other processors, despite their more elegant architecture, are rapidly heading for extinction, and the main rival is the ARM - whose cores are embedded in the processors of many mobile devices. ARM Assembler is a joy to code. I would love to develop an ARM version of Oxygen should the opportunity arise.

Charles



JRS

  • Guest
Re: Alternative O2 Builder
« Reply #2 on: September 23, 2010, 04:16:15 AM »
Quote
However porting to SB, being a scripting language would pose a few problems like being unable to tightly couple Assembly code with the host script.

<Start Theory>ScriptBasic would only act as an interactive C/ASM source code generating tool. It outputs C / C inline ASM text to be compiled by a C compiler. Once you have a OxygenBasic compiler compiled, SB & gcc would no longer be required.<End Theory>

Maybe I don't fully understand the dependency on FreeBASIC when generic inline ASM is the outputted result. With no real development going on with FreeBASIC in years, it seems a bit risky to bet the farm on it.
« Last Edit: September 23, 2010, 04:19:30 AM by JRS »

cevpegge

  • Guest
Re: Alternative O2 Builder
« Reply #3 on: September 23, 2010, 05:41:43 AM »

Yes I see. I think I would either go for C or self-compile. But the worst that could happen to FB is going into stasis and that largely depends on its major users. The most useful thing about basic is good string manipulation and easy access to assembler. Compilers don't really require more than that.

I spend much of my time refactoring the source code (It can be Draconian sometimes.)  and FB is a comfortable and familiar environment to do this. And FB compiles O2 in less than 2 seconds. It is almost interactive. The only thing I really miss is GOSUB. I often need subroutines that have access to all the parental variables. This would reduce my global list considerably.

Charles

JRS

  • Guest
Re: Alternative O2 Builder
« Reply #4 on: September 23, 2010, 09:57:40 AM »
Your right. C or BCX makes more sense.

Quote from: BCX Docs
$ASM directive

Purpose: The $ASM directive allows inline assembly instructions to be used with BCX. An $ASM directive must be placed before and after the assembly statements.

 Syntax:

 $ASM
  /** Assembly statements go here */
 $ASM

Code: Text
  1.  DIM str1$
  2.  DIM str2$
  3.    
  4.  str1$ = "123456789"
  5.    
  6.  str2$ = AsmLeft$(str1$, 3)
  7.  PRINT str2$
  8.    
  9.  str2$ = AsmMid$(str1$, 4, 3)
  10.  PRINT str2$
  11.    
  12.  str2$ = AsmMid$(str1$, 4)
  13.  PRINT str2$
  14.    
  15.  str2$ = AsmRight$(str1$, 3)
  16.  PRINT str2$
  17.    
  18.    
  19.  FUNCTION SubStr%(SrcStr AS LPSTR, SrcStrLen%, StartPosn%, DestStr AS LPSTR, DestStrLen%)
  20.    $ASM
  21.    nop
  22.    nop
  23.    nop
  24.    
  25.    ;save used registers except for eax:edx
  26.    push ebx
  27.    push ecx
  28.    push edi
  29.    push esi
  30.    
  31.    ;load the 32-bit registers from the stack
  32.    
  33.    mov esi, dword PTR [ebp+8]  ;SrcStr
  34.    mov edx, dword PTR [ebp+12] ;SrcLen
  35.    mov eax, dword PTR [ebp+16] ;StartPosn
  36.    mov edi, dword PTR [ebp+20] ;DestStr
  37.    mov ecx, dword PTR [ebp+24] ;DestLen
  38.    
  39.    ;check for zero-length Destination string
  40.    jecxz NoGood
  41.    
  42.    ;compensate StartPosn for 0 based count
  43.    dec eax
  44.    
  45.    ;check for negative offset of StartPosn
  46.    cmp eax, 0
  47.    jl NoGood
  48.    
  49.    ;point esi to start of SrcStr
  50.    add esi, eax
  51.    
  52.    ;and subtract StartPosn from SrcLen
  53.    sub edx, eax
  54.    
  55.    ;check if offset is past end of SrcStr
  56.    cmp edx, 0
  57.    jle NoGood
  58.    
  59.    ;move substring of esi into edi
  60.    rep movsb
  61.    
  62.    ;return this to indicate function worked
  63.    mov eax, edi
  64.    jmp ReturnAX
  65.    
  66.    NoGood:
  67.    mov eax, -1
  68.    
  69.    ReturnAX:
  70.    pop esi
  71.    pop edi
  72.    pop ecx
  73.    pop ebx
  74.    pop ebp
  75.    ret
  76.    $ASM
  77.    FUNCTION = 0
  78.  END FUNCTION
  79.    
  80.  'Left$ equivalent
  81.  FUNCTION AsmLeft$(fnstr1$, fnint1%)
  82.    DIM RAW startp% = 1
  83.    DIM RAW retstr$ * fnint1% + 256
  84.    SubStr(fnstr1$, LEN(fnstr1$), startp%, retstr$, fnint1%)
  85.    retstr[fnint1] = 0
  86.    FUNCTION = retstr$
  87.  END FUNCTION
  88.    
  89.  'MID$ equivalent
  90.  FUNCTION AsmMid$ OPTIONAL(fnstr1$, fnint1%, fnint2%=0)
  91.    DIM RAW startp% = fnint1%
  92.    DIM RAW lenfnstr1% = LEN(fnstr1$)
  93.    DIM RAW retstr$ * fnint2% + 256
  94.    IF fnint2% = 0 THEN
  95.      fnint2% = lenfnstr1% - startp% + 1
  96.    END IF
  97.    SubStr(fnstr1$, lenfnstr1%, startp%, retstr$, fnint2%)
  98.    retstr[fnint2] = 0
  99.    FUNCTION = retstr$
  100.  END FUNCTION
  101.    
  102.  'RIGHT$ equivalent
  103.  FUNCTION AsmRight$(fnstr1$, fnint1%)
  104.    DIM RAW lenfnstr1% = LEN(fnstr1$)
  105.    DIM RAW startp% = lenfnstr1% - fnint1% + 1
  106.    DIM RAW retstr$ * fnint1% + 256
  107.    SubStr(fnstr1$, lenfnstr1%, startp%, retstr$, fnint1%)
  108.    retstr[fnint1] = 0
  109.    FUNCTION = retstr$
  110.  END FUNCTION
  111.  

Quote
GOSUB ... RETURN statement

Purpose: GOSUB redirects program flow to a label. The flow continues from the label until a RETURN statement is encountered and the flow is returned to the line following the GOSUB Label statement.

 Syntax:

 GOSUB Label

 Label:
  Statements
 RETURN

Code: [Select]
DIM j
 
 CLS
 
 PRINT "Calling Subroutine 9 times ..." : PRINT
 
 FOR j = 1 TO 9
   GOSUB qwerty
 NEXT j
 
 PRINT : PRINT "All Done!"
 
 END                       ' Don't accidently fall into the subroutine
 
 QwErTy:                   ' BCX Subroutine Labels are NOT case sensitive
 
 PRINT " This is instance >>", j, " <<  of the subroutine call."
 
 RETURN                    ' Return from subroutine back to main program
« Last Edit: September 23, 2010, 12:44:55 PM by JRS »

efgee

  • Guest
Re: Alternative O2 Builder
« Reply #5 on: September 23, 2010, 05:11:17 PM »
Charles,

Oxygen makes intensive use of FreeBasic Inline Assembler (which is slightly preprocessed GNU GAS using Intel notation). The Oxygen Assembly language is largely compatible with this. We also have a high degree of compatibility between FreeBasic and OxygenBasic language.

If the Oxygen Assembly language is largely compatible with gas, maybe you could have saved yourself quite some time and utilize the whole "BASIC language" frontend from freeBasic.
Like:

Code: [Select]
fbc -gen oxygen hello.bas

if oxygen is chosen than co2.exe is used instead of gas.exe.
The code to make this possible could have been done in one afternoon...

With that you could have had tons of new users/testers in one night!

efgee

JRS

  • Guest
Re: Alternative O2 Builder
« Reply #6 on: September 23, 2010, 10:26:59 PM »
I'm going to take a guess here but I think Charles has his own ideas how a Basic and a hand coded compiler should work.

I don't see why O2 ASM as defined in FreeBASIC and using the GNU Assembler should not translate well to BCX Inline ASM and compiled with MinGW-gcc under windows for 32 and 64 bit.

Quote
The GNU Assembler (GAS) is the default back-end to the GNU Compiler Collection (GCC) suite. As such, GAS is as portable and retargetable as GCC is. However, GAS uses the AT&T syntax for its instructions, which some users find to be less readable than Intel syntax. As a result, assembly code written inline in a C code file for GCC must also be written in GAS syntax.

GAS is developed specifically to be used as the GCC backend. GCC always feeds GAS syntactically-correct code, so GAS often has minimal error checking.

GAS is available as a part of either the GCC package or the GNU binutils package.
« Last Edit: September 23, 2010, 10:40:47 PM by JRS »

cevpegge

  • Guest
Re: Alternative O2 Builder
« Reply #7 on: September 24, 2010, 12:31:22 AM »

The important thing at present is to work with a single "incarnation" of Oxygen when making a large number of changes. But you have given me ideas about emitting various types of code. With a few adjustments Oxygen could easily emit C or Assembly code, as well as binary. :)

One curious phenomenon: When Oxygen was just an Assembler it was around 150K in size. Adding Basic produced a rapid escalation to around 370k. But while I continue to add new functionality and refactor at the same time, the size has not changed significantly in several months. It seems to have reached a critical mass where only small changes are required to add new functionality. I can always find ways to reduce the code before adding new things. Its a bit like file zipping by hand.

Charles

JRS

  • Guest
Re: Alternative O2 Builder
« Reply #8 on: September 24, 2010, 01:11:46 AM »
Quote
The important thing at present is to work with a single "incarnation" of Oxygen when making a large number of changes.

All I'm saying is that it would seem easier to integrate C libraries (reading headers, inline C in O2Basic, ...) if you used a C compiler to compile your initial compiler. (say that 3 times in a row)  :)

I hate to see you wasting your time when needed functionality already exists when using the correct compiler.



« Last Edit: September 24, 2010, 01:13:42 AM by JRS »

cevpegge

  • Guest
Re: Alternative O2 Builder
« Reply #9 on: September 24, 2010, 02:17:16 AM »
John,

I am considering porting to C but this is not a priority right now. It would disrupt the work flow. Much of my design effort at the moment is in getting the architecture of the program right. This is more or less language-independent. With FreeBasic firmly embedded in my subconsciousness it's one less thing to think about.

C headers are not a problem. Most of the work to read them has been done.

Charles

JRS

  • Guest
Re: Alternative O2 Builder
« Reply #10 on: September 24, 2010, 02:28:59 AM »
I understand. FreeBasic is a vessel that lets you construct an independent compiler in BASIC.

I'll leave you alone for now.  :-X   I'm having too much fun turning the SB webserver into a multi-threaded desktop application server.  ;D

Actually this concept should work just as well using Apache with PHP or Python. Gtk has bindings for both.
« Last Edit: September 24, 2010, 02:55:03 AM by JRS »

cevpegge

  • Guest
Re: Alternative O2 Builder
« Reply #11 on: September 24, 2010, 03:03:56 AM »

Okay John :) What is your time zone by the way. Mine is GMT though I tend to work in 4 hours shifts around the clock.

Charles

JRS

  • Guest
Re: Alternative O2 Builder
« Reply #12 on: September 24, 2010, 03:06:02 AM »
It's 3 am and I just exhausted my last wind so I'm going to bed.




cevpegge

  • Guest
Re: Alternative O2 Builder
« Reply #13 on: September 28, 2010, 05:41:26 AM »
I tried compiling Oxygen with FreeBasic version 0.21 instead of the usual 0.20b. But it was not successful unfortunately. Something fundamental may have changed. Not to mention the indignity of getting locked out of the FB forum about 7 hours ago, and no administrator to be found for assistance, I will make haste and start to plan a C migration. It makes a lot of sense in the long term.

The state-of-the-art parallel processing APIs which make use of the Graphics Hardware cores, OpenCL and Cuda are based on C. There is no escape from C!

Charles
« Last Edit: September 28, 2010, 06:49:38 AM by cevpegge »

JRS

  • Guest
Re: Alternative O2 Builder
« Reply #14 on: September 28, 2010, 07:22:35 AM »
Charles,

Moving to C is a wise and safe choice. The FreeBASIC group can't even get it together to move their own project along. FreeBASIC had it's day but I don't see a future at this time for the language. There also seems to be a bitter and angry side to many of the users and they aren't that friendly to new users of the language that ask stupid questions.