Author Topic: Python 2nd Cut  (Read 7863 times)

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Python 2nd Cut
« on: October 20, 2013, 12:30:06 AM »
EDIT: Latest version will always be located at: https://bitbucket.org/Airr/pybas2nim.git

EDIT:  CHANGES and/or UPDATES WILL BE COMMUNICATED IN BOX BELOW
Quote
{10/22-24 2013}
  • Added DIM support for declaring Variables, Added Sequence support to LET
  • Added support for multiple DIM in one line
  • Added support for Identifier Decorations (Variables and Function names).  So now you can do DIM a$, b!, c# for String, Integer, and Float, respectively
  • Added Nimrod Sequence support, which is essentially a Dynamic Array (auto expands during runtime as needed, with full Garbage Collection so no leaks)
  • Added support for Nimrod IMPORT statement , allowing multiple imports per line (IMPORT StrUtils, ParseUtils, Macros)
  • Added FileIO - Exists, FileRead, and FileWrite
  • SUPPORTED KEYWORDS:
            DIM, LET, PRINT, FOR, TO, STEP, NEXT, IF, THEN, ELSE, END IF, WHILE, WEND, ASC, LEFT, MID,
            RIGHT, UCASE, LCASE, FUNCTION, SUB, RETURN, END SUB, END FUNCTION, AS, IMPORT
            EXISTS, FILEREAD, FILEWRITE

One of the problems any translator is going to face is how to handle inline functions.

By way of example, assigning the result of a call to a function to a variable (simplest example) when you have to also rename the actual function.

Like ASC(), which in Nimrod is ORD()

So, my first Python version is not up to the task, because it doesn't have the ability to continue parsing the line for other keywords, etc.

Other issues include how to handle FOR loops, IF/THEN, and WHILE/WEND constructs, all the while adhering to the Nimrod whitespace requirement.

This version is the start of that.  It properly handles assigning the result of ASC()/ORD() to a variable. It also handles FOR, IF/THEN/ELSE/STEP and WHILE/WEND.  Sorry for the mixed case, but I wanted to be sure that the keywords were in fact case-insensitive.

Given this example program, saved on disk as "test.sb":
Code: [Select]
LET z = "Goodbye."
print "Hello!"
for x = 10 to 0 step -2
    IF x <> 0 THEN
        print x
    else
        print "Done."
    END if
next
let x = 0
while x < 10
    print x
    x &=  1
wend
let b = asc("a")
let c = ucase("uppercase")
print b
print c
print z


This version of the translator produces the following output, with capitalized keywords (to make it easier to see) displayed on screen AND saved to "test.nim":
Code: [Select]
IMPORT StrUtils
VAR z = "Goodbye."
ECHO "Hello!"
FOR x in COUNTDOWN( 10, 0, STEP = 2):
    IF x != 0:
        ECHO x
    ELSE:
        ECHO "Done."


VAR x = 0
WHILE x < 10:
    ECHO x
    x += 1

VAR b = ORD( 'a' )
VAR c = "uppercase".toUpper()
ECHO b
ECHO c
ECHO z

Here's the Python translator code.  You'll need the "pyparsing" module, which can be installed by doing "sudo easy_install pyparsing" in a terminal (on c9.io, "sudo" is not needed [and won't work]:
Code: [Select]
#!/usr/bin/env python

from pyparsing import *

# GLOBAL VARIABLE FOR INDENT LEVEL
indent = ''

# CALLBACK FUNCTIONS FOR PARSER
def do_LET(s,e,t):
    return ' '.join( ('VAR',t[1],t[2],t[3]) )

def do_ASSIGN(s,e,t):
global indent
if t[1] == '&=': t[1] = '+='
return indent + ' '.join( (t[0],t[1],t[2]) )

def do_ASC(s,e,t):
global indent
return ' '.join( ('ORD(',t[2].replace('"',"'"),')') )

def do_PRINT(s,e,t):
global indent
return  indent + ' '.join( ('ECHO', t[1]) )

def do_FUNCSUB(s,l,t):
print t

def do_WHILE(s,l,t):
global indent
tmp = indent + ' '.join( (t[0],t[1],t[2] + t[3]+':') )
indent += ' '*4
return tmp

def do_DEDENT(s,l,t):
global indent
indent = indent[:-4]
return ''

def do_ULCASE(s,l,t):
case = '.toUpper()'
if t[0] == 'LCASE':
case = '.toLower()'
return ''.join( (t[2],case) )

def do_IF(s,l,t):
global indent
if t[2] == '=': t[2] = '=='
if t[2] == '<>': t[2] = '!='
tmp = indent + ' '.join( ( t[0],t[1],t[2] + t[3][:-5] + ':') )
indent += ' '*4
return tmp

def do_ELSE(s,l,t):
global indent
indent = indent[:-4]
tmp = indent + t[0] + ':'
indent += ' '*4
return tmp

def do_FOR(s,l,t):
global indent
line_end ='):'
op = 'COUNTUP('
if t[7]:
if '-' in t[7]:
op = 'COUNTDOWN('
t[7] = t[7].replace('-','')
line_end = ' '.join( (',',t[6], '=', t[8] + line_end) )
tmp = indent + ' '.join( (t[0], t[1], 'in', op, t[3] + ',', t[5] + line_end) )
indent += ' '*4
return tmp

# KEYWORDS
LET = Keyword('LET',  caseless=True)
PRINT = Keyword('PRINT',  caseless=True)
FOR = Keyword('FOR',  caseless=True)
TO = Keyword('TO',  caseless=True)
STEP = Keyword('STEP', caseless=True)
NEXT = Keyword('NEXT',  caseless=True)
IF = Keyword('IF',  caseless=True)
THEN = Keyword('THEN',  caseless=True)
ELSE = Keyword('ELSE',  caseless=True)
ENDIF = Keyword('END IF', caseless=True)
WHILE = Keyword('WHILE',  caseless=True)
WEND = Keyword('WEND',  caseless=True)
END = Keyword('END',  caseless=True)
ASC = Keyword('ASC',  caseless=True)
VAL = Keyword('VAL',  caseless=True)
STR = Keyword('STR',  caseless=True)
MID = Keyword('MID',  caseless=True)
LEFT = Keyword('LEFT',  caseless=True)
UCASE = Keyword('UCASE',  caseless=True)
LCASE = Keyword('LCASE',  caseless=True)
UBOUND = Keyword('UBOUND', caseless=True)
LBOUND = Keyword('LBOUND', caseless=True)

# GRAMMER
IDENT = (Word(alphas, alphanums+'_') | Word(nums))
INTEGER = Word(nums)
PLUS  = Literal( '+' )
MINUS = Literal( '-' )
MULT  = Literal( '*' )
DIV   = Literal( '/' )
ADDOP  = PLUS | MINUS | Literal('&=')
MULTOP = MULT | DIV
EQU = Literal('=')
LT = Literal( '<' )
GT = Literal( '>' )
NEQ = Literal( '<>' )
LTE = Combine(LT, EQU)
GTE = Combine(GT, EQU)
# ENDIF = END+IF | ENDIF #Literal('end if') | Literal('endif')
LPAR,RPAR = ('(',')')
ASSIGNOP = (NEQ | LTE | GTE | EQU | LT | GT )
STRING = dblQuotedString

PROGRAM = Forward()
expression = operatorPrecedence(PROGRAM,
[
(oneOf('* /'), 2, opAssoc.LEFT),
(oneOf('+ -'), 2, opAssoc.LEFT),
])

FORIDENT = ( IDENT | INTEGER | '-'+INTEGER )
FORStmt = FOR + IDENT + '=' + FORIDENT + TO + FORIDENT + Optional(STEP + FORIDENT)

FUNCSUBstmt = IDENT + LPAR + Group(Optional(delimitedList(expression))) + RPAR
PRINTIDENT = ( IDENT | STRING)
PRINTStmt = PRINT + PRINTIDENT
NEXTStmt = NEXT

WHILEStmt = WHILE + IDENT + ASSIGNOP + restOfLine
WENDStmt = WEND + Empty()
IFStmt = IF + IDENT + ASSIGNOP + restOfLine
THENStmt = THEN + Empty()
ELSEStmt = ELSE + Empty()
ENDIFStmt = ENDIF + Empty()
ASCStmt = ASC + LPAR + STRING + RPAR
ULCASEStmt = (UCASE + LPAR + STRING + RPAR) ^ (LCASE + LPAR + STRING + RPAR)
ASSIGNStmt = IDENT + (EQU | ADDOP) + (ASCStmt | ULCASEStmt | IDENT | STRING | INTEGER)
LETStmt = LET + IDENT + EQU + (ASCStmt | ULCASEStmt | IDENT | STRING | INTEGER)

# MASTER COMBINED GRAMMER RULES PASSED TO PARSER
PROGRAM << (PRINTStmt | FORStmt | NEXTStmt | LETStmt | WHILEStmt \
| WENDStmt | IFStmt | ELSEStmt | ENDIFStmt | ASCStmt | FUNCSUBstmt \
| ASSIGNStmt | ULCASEStmt )

# KEYWORD CALLBACK ASSIGNMENTS
LETStmt.setParseAction(do_LET)
ASSIGNStmt.setParseAction(do_ASSIGN)
ASCStmt.setParseAction(do_ASC)
PRINTStmt.setParseAction(do_PRINT)
ULCASEStmt.setParseAction(do_ULCASE)
WHILEStmt.setParseAction(do_WHILE)
WENDStmt.setParseAction(do_DEDENT)
ELSEStmt.setParseAction(do_ELSE)
NEXTStmt.setParseAction(do_DEDENT)
ENDIFStmt.setParseAction(do_DEDENT)
IFStmt.setParseAction(do_IF)
FORStmt.setParseAction(do_FOR)


if __name__ == '__main__':
with open("test.sb") as myfile:
test = "".join(line for line in myfile)

outfile = 'IMPORT StrUtils\n'
retval= PROGRAM.searchString(test)
for x in range( len(retval) ):
outfile += retval[x][0]+'\n'

print outfile

with open('test.nim', 'w') as myFile:
myFile.write(outfile)

Have fun with this; not all of the keywords are active, see if you can add them!

AIR.
« Last Edit: October 24, 2013, 06:25:30 PM by AIR »

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Python 2nd Cut
« Reply #1 on: October 20, 2013, 12:41:33 AM »
Thanks AIR. That's a great start. This gives me something to work with to learn Python and Nimrod code I can use to try and follow along with SB. (shadow BASIC) I really liked the precedence math example that came with the pyparsing library distribution. It seems to be a powerful lib. I'm looking forward to seeing this mature.

« Last Edit: October 21, 2013, 01:03:03 AM by John »

kryton9

  • Guest
Re: Python 2nd Cut
« Reply #2 on: October 20, 2013, 01:05:49 PM »
I can't test any of this now, but appreciate the posts for future reference.

Offline AIR

  • BASIC Developer
  • Posts: 932
  • Coder
Re: Python 2nd Cut
« Reply #3 on: October 23, 2013, 06:08:30 AM »
Updated first post with link to always current version.

Offline John

  • Forum Support / SB Dev
  • Posts: 3597
    • ScriptBasic Open Source Project
Re: Python 2nd Cut
« Reply #4 on: October 23, 2013, 09:08:09 PM »
AIR-BASIC

Code: [Select]
    dim a!
    dim b#
    dim c$
   
    a = 123
    b  = 1.23
    c = "A dynamic string"
   
    print a
    print b
    print c
    let d = @[1, 2, 3, 4, 5, 6]
    let e = "Hello"
    let f = do(blah,foo,bar)

Nimrod translation

Code: [Select]
IMPORT StrUtils
VAR a: int
VAR b: float
VAR c: string
a = 123
b = 1.23
c = "A dynamic string"
ECHO a
ECHO b
ECHO c
VAR d =  @[1, 2, 3, 4, 5, 6]
VAR e = "Hello"
VAR f = do ( blah, foo, bar )