Converting Tokenised BASIC to Plain Text

discuss PC<>Acorn file transfer issues & the use of FDC, XFER, Omniflop/disk etc.
User avatar
BeebMaster
Posts: 2475
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Converting Tokenised BASIC to Plain Text

Postby BeebMaster » Sun Mar 23, 2014 1:59 pm

If I have a BBC BASIC file on my PC, say extracted from a disc image I've made, is there a way to display it in a readable format like that generated by LIST from within BASIC?

Ubuntu, specifically, but I would consider another well-known PC operating system.

It would be nice to drag a BASIC file into a text editor and have an option to display it as a BASIC listing. Or, a nice little util so that I could right-click a file and have a menu option "display as BASIC LIST" would be really good!

Or is there a version of BBC BASIC for Linux/Ubuntu which I've overlooked which would do the trick?

I've searched online for "BBC BASIC plugin for gedit" but that turned out to be very much the long shot I'd anticipated!
Image

User avatar
BeebMaster
Posts: 2475
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby BeebMaster » Sun Mar 23, 2014 2:09 pm

Oooh - I've just discovered Brandy!

I've even downloaded it and installed it and compiled it and made it work already!

Works a treat, although I can't see that there is support for filing system operations to navigate around directories, so I have to use the full pathname in the LOAD statement to get it in, but after that I can LIST and then copy-paste into gedit.
Image

g7jjf
Posts: 354
Joined: Sun Aug 07, 2005 7:29 pm
Location: Notts, England
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby g7jjf » Sun Mar 23, 2014 2:43 pm

There is a Linux version of my DFS Explorer available which will allow you to view a BBC Basic program in LIST format to export elsewhere.

http://www.g7jjf.com/mac_dfs_explorer.htm

Link and requirements towards bottom of page.

Jon

User avatar
jgharston
Posts: 2764
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: Converting Tokenised BASIC to Plain Text

Postby jgharston » Sun Mar 23, 2014 3:54 pm

BeebMaster wrote:Oooh - I've just discovered Brandy!

Works a treat, although I can't see that there is support for filing system operations to navigate around directories, so I have to use the full pathname in the LOAD statement to get it in, but after that I can LIST and then copy-paste into gedit.
Try the Napoleon fork of Brandy. The current version 0.10 has *CD <dir>. I'm currently dragging a comb through the code to get it compiling on different platforms. I've spent this weekend getting the RISC OS build working, the next task is to boot into Linux and get that build working as well.

Hint: on Windows if you SET EDITOR=(path to BBC BASIC to Windows)\bbcwin then Brandy uses the BB4W IDE editor!

On the original question, detokenising a BASIC is simple, you just expand token bytes (&80+) to token strings. It's tokenising that is fiddly outside BASIC. Something like this will do it:

Code: Select all

DIM token$(127)
Set token$() to token strings
in%=OPENIN(file$)
byte%=BGET#in%
line%=BGET#in%
REPEAT
 REPEAT
  line%=256*line%+BGET#in%
  len%=BGET#in%
  byte%=BGET#in%
  IF byte%<128 THEN VDU byte% ELSE PRINT token$(byte%-128);
 UNTIL byte%=13
 PRINT
 line%=BGET#in%
UNTIL line%=255
CLOSE#in%

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
BeebMaster
Posts: 2475
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby BeebMaster » Sun Mar 23, 2014 4:45 pm

g7jjf wrote:There is a Linux version of my DFS Explorer available which will allow you to view a BBC Basic program in LIST format to export elsewhere

Thanks for that, I'll keep it in mind.

I've nearly got Brandy doing what I originally wanted, with this Thunar custom action

Code: Select all

gnome-terminal --tab --maximize --title=%n -e "/usr/bin/brandy -load %n"

I can now right click a tokenised BASIC file (even if it's mounted in ADFSFuse from one of the files on the ADFSFuse mounted CF image!) and get it loaded by Brandy. It doesn't have a start-up list option, but I just have to type LIST when Brandy loads and then I can copy the text to the clipboard and use it at will.
Image

User avatar
BeebMaster
Posts: 2475
Joined: Sun Aug 02, 2009 4:59 pm
Location: Lost in the BeebVault!
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby BeebMaster » Sun Mar 23, 2014 4:52 pm

jgharston wrote:Try the Napoleon fork of Brandy. The current version 0.10 has *CD <dir>. I'm currently dragging a comb through the code to get it compiling on different platforms. I've spent this weekend getting the RISC OS build working, the next task is to boot into Linux and get that build working as well.

That would be good. I've just downloaded and extracted it but I couldn't get it to compile on Ubuntu. It did a few lines' worth of whatever it does but then I got an error:

Code: Select all

make: *** No rule to make target `../src/emulate.h', needed by `../src/mainstate.o'. Stop.


Maybe it's not ready yet or maybe I did something wrong.
Image

User avatar
jgharston
Posts: 2764
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: Converting Tokenised BASIC to Plain Text

Postby jgharston » Sun Mar 23, 2014 7:36 pm

BeebMaster wrote:

Code: Select all

make: *** No rule to make target `../src/emulate.h', needed by `../src/mainstate.o'. Stop.

Maybe it's not ready yet or maybe I did something wrong.
No, that's me not you. I haven't run it through Linux yet, so haven't shaken out the problems.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
sweh
Posts: 1847
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby sweh » Sun Mar 23, 2014 10:35 pm

BeebMaster wrote:If I have a BBC BASIC file on my PC, say extracted from a disc image I've made, is there a way to display it in a readable format like that generated by LIST from within BASIC?

Ubuntu, specifically, but I would consider another well-known PC operating system.

My Utils ( http://sweh.spuddy.org/Beeb/mmb_utils.html ) has a "list" command:

Code: Select all

% beeb list Menu | head
   10 MODE 7
   20 DSK$=""
   30 ON ERROR CHAIN "Menu0"
   40 X=OPENIN"Current"
   50 INPUT#X,DSK$:INPUT#X,TITLE$
   60 CLOSE#X
   70 REM We leave error set so ESCAPE will reload
   80 IF DSK$="" THEN CHAIN "Menu0"
   90 OSCLI "LOAD M."+DSK$+" 7000"
  100 P%=&7000

Doesn't understand newer versions of BASIC though.
Rgds
Stephen

User avatar
jgharston
Posts: 2764
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: Converting Tokenised BASIC to Plain Text

Postby jgharston » Mon Mar 24, 2014 12:48 am

jgharston wrote:
BeebMaster wrote:Maybe it's not ready yet or maybe I did something wrong.
No, that's me not you. I haven't run it through Linux yet, so haven't shaken out the problems.
Napoleon Brandy 1.20/0.10 now successfully compiling on Windows, DOS and RISC OS. Now to fire up Linux and get it compiling there.

Hint: If you SET Brandy$Editor=(path to BB4W)\bbcwin the Brandy EDIT command will use the BBC BASIC for Windows IDE to edit programs. Quite a change from editing in a text editor.

sweh wrote:My Utils ( http://sweh.spuddy.org/Beeb/mmb_utils.html ) has a "list" command. Doesn't understand newer versions of BASIC though.
I don't understand Perl (is that what it is?), but if you add the following to the table in BasUtils.pm:

Code: Select all

  01 => ['CIRCLE', 0x00],    02 => ['ELLIPSE', 0x00],
  03 => ['FILL', 0x00],      04 => ['MOUSE', 0x00],
  05 => ['ORIGIN', 0x00],    06 => ['QUIT', 0x00],
  07 => ['RECTANGLE', 0x00], 08 => ['SWAP', 0x00],
  09 => ['SYS', 0x00],       10 => ['TINT', 0x00],
  11 => ['WAIT', 0x00],      12 => ['INSTALL', 0x00],
  14 => ['PRIVATE', 0x00],   15 => ['BY', 0x00],
  16 => ['EXIT', 0x00]
should do it to understand BB4W BASIC files. ARM BASIC double-tokens are a bit fiddlier. You will need to do:

Code: Select all

IF current_byte=&C6 OR current_byte=&C7 OR current_byte=&C8 THEN
IF next_byte<&8E THEN use current_byte as a single-byte token
ELSE
double-byte token
ENDIF
ENDIF

The full token table is here.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
richardtoohey
Posts: 3378
Joined: Thu Dec 29, 2011 5:13 am
Location: Tauranga, New Zealand

Re: Converting Tokenised BASIC to Plain Text

Postby richardtoohey » Mon Mar 24, 2014 3:12 am

Might be better as a new topic, but what are the goals of Napoleon Brandy?

http://sourceforge.net/projects/napoleonbrandy/

Seems to talk about adding graphics to Brandy BASIC, but more of a Windows project? But given that you are talking about other platforms, maybe not.

Thanks.

User avatar
sweh
Posts: 1847
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby sweh » Mon Mar 24, 2014 12:59 pm

jgharston wrote:The full token table is here.

Hmm, thanks. That table isn't quite clear... looks like there are some major variants:
BASIC 2 (which is what my code handles 'cos I decoded the values from the BASIC2 ROM)
BASIC 4 (BASIC2 plus a few additional commands such as "EDIT")
Basic 4 for Z80 (which is the same as for basic 4 for 6502, except for &CE ?)
ARM Basic
BBC Basic for Windows

Not sure where the "Mac" version comes in; is this a variant of ARM Basic?
Rgds
Stephen

User avatar
jgharston
Posts: 2764
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: Converting Tokenised BASIC to Plain Text

Postby jgharston » Mon Mar 24, 2014 4:07 pm

sweh wrote:Hmm, thanks. That table isn't quite clear...
I thought I'd made it as clear as possible. What's confusing about it?
sweh wrote:looks like there are some major variants:
BASIC 2 (which is what my code handles 'cos I decoded the values from the BASIC2 ROM)
BASIC 4 (BASIC2 plus a few additional commands such as "EDIT")
Basic 4 for Z80 (which is the same as for basic 4 for 6502, except for &CE ?)
ARM Basic
BBC Basic for Windows
The main differences are:
* the &CE token: 6502 BASIC uses it for EDIT, Z80/DOS BASIC uses it for PUT (output to a port, the opposite of GET(nn), input from a port). Normally EDIT would never appear in a program as it's an immediate command.
* Windows BASIC adds tokens by using bytes <32
* ARM BASIC adds tokens by using the immediate command tokens to add new tokens and double-tokens. As an immediate command would normally never appear in a program this works transparently.

An immediate command normally never appears in a program (10 RENUMBER, 20 AUTO ???), and if it does it is never followed by a function token (AUTOLEN, DELETECOS), so the simplest thing to do is to default to detokenising with the full ARM tokens plus the Windows tokens, ie C7 expands to "WHILE" not "DELETE", etc.
With the double-byte tokens expand as a double-byte if the second byte is &8E-&A7, or as a single-byte token if not. That catches cases where an immediate command has been put in a program.

(Some of the immediate commands would actually be useful if they could be used in a program, particularly LIST. Who's never wanted to do ON ERROR LIST ERL:END? Also, I've extended LOAD and SAVE to load/save memory with LOAD "file",address and SAVE "file",start,end)

sweh wrote:Not sure where the "Mac" version comes in; is this a variant of ARM Basic?
Mac as in Apple Mac, so it's a variant of 68000 BASIC.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
sweh
Posts: 1847
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby sweh » Mon Mar 24, 2014 5:18 pm

jgharston wrote:thought I'd made it as clear as possible. What's confusing about it?
...
Mac as in Apple Mac, so it's a variant of 68000 BASIC.

The fact that there's no 68K column, and the words in that box appear to be half-way between two columns so I've no idea where this "floating" box should be...
Rgds
Stephen

User avatar
sweh
Posts: 1847
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: Converting Tokenised BASIC to Plain Text

Postby sweh » Mon Mar 24, 2014 9:57 pm

Well, if I've done it properly...

Code: Select all

% beeb list ../HostFS/jgh/TubeHost  | sed -n 18p     
  180PROCf_init:IFos%=32:LOADATN "SetWindowText",@hwnd%,"Tube Host v"+ver$:tmp$=@tmp$

Clearly LOADATN isn't right. So...

Code: Select all

% beeb list ../HostFS/jgh/TubeHost -tarm | sed -n 18p
  180PROCf_init:IFos%=32:SYS "SetWindowText",@hwnd%,"Tube Host v"+ver$:tmp$=@tmp$

Ah, that's better. We tell it that it's ARM basic and now the 2-byte code is decoded.

(You might recognise the code line :-))

Code: Select all

% beeb list
Syntax: list filename [-o #] [-t variant]
  Known variants (default 'basic2'):
    arm
    b4w
    basic2
    basic4
    z80


I did this with a series of "overlays" which extend (or overwrite) the basic2 bytes.

Code: Select all

# These extensions from http://mdfs.net/Docs/Comp/BBCBasic/Tokens
$basic_tokens{"basic4"}={  206 => ['EDIT',0] };
$extended_tokens{"basic4"}={};

$basic_tokens{"z80"}={  206 => ['PUT',0] };
$extended_tokens{"z80"}={};

$basic_tokens{"arm"}=
{
  127 => ['OTHERWISE',0],   204 => ['ELSE',0],
  201 => ['WHEN',0],        205 => ['ENDIF',0],
  202 => ['OF',0],          206 => ['ENDWHILE',0],
  203 => ['ENDCASE',0]
};
$extended_tokens{"arm"}{198}={ 142 => ['SUM',0], 143 => ['BEAT',0] };
$extended_tokens{"arm"}{199}=
{
  142 => ['APPEND',0],   151 => ['NEW',0],
  143 => ['AUTO',0],     152 => ['OLD',0],
  144 => ['CRUNCH',0],   153 => ['RENUMBER',0],
  145 => ['DELETE',0],   154 => ['SAVE',0],
  146 => ['EDIT',0],     155 => ['TEXTLOAD',0],
  147 => ['HELP',0],     156 => ['TEXTSAVE',0],
  148 => ['LIST',0],     157 => ['TWIN',0],
  149 => ['LOAD',0],     158 => ['TWINO',0],
  150 => ['LVAR',0],     159 => ['INSTALL',0]
};
$extended_tokens{"arm"}{200}=
{
  142 => ['CASE',0],       155 => ['LIBRARY',0],
  143 => ['CIRCLE',0],     156 => ['TINT',0],
  144 => ['FILL',0],       157 => ['ELLIPSE',0],
  145 => ['ORIGIN',0],     158 => ['BEATS',0],
  146 => ['POINT',0],      159 => ['TEMPO',0],
  147 => ['RECTANGLE',0],  160 => ['VOICES',0],
  148 => ['SWAP',0],       161 => ['VOICE',0],
  149 => ['WHILE',0],      162 => ['STEREO',0],
  150 => ['WAIT',0],       163 => ['OVERLAY',0],
  151 => ['MOUSE',0],      164 => ['MANDEL',0],
  152 => ['QUIT',0],       165 => ['PRIVATE',0],
  153 => ['SYS',0],        166 => ['EXIT',0],
  154 => ['INSTALL',0]
};

$basic_tokens{"b4w"}=
{
    1 => ['CIRCLE',0],     198 => ['SUM',0],
    2 => ['ELLIPSE',0],    199 => ['WHILE',0],
    3 => ['FILL',0],       200 => ['CASE',0],
    4 => ['MOUSE',0],      201 => ['WHEN',0],
    5 => ['ORIGIN',0],     202 => ['OF',0],
    6 => ['QUIT',0],       203 => ['ENDCASE',0],
    7 => ['RECTANGLE',0],  204 => ['ELSE',0],
    8 => ['SWAP',0],       205 => ['ENDIF',0],
    9 => ['SYS',0],        206 => ['ENDWHILE',0],
   10 => ['TINT',0],
   11 => ['WAIT',0],
   12 => ['INSTALL',0],
   14 => ['PRIVATE',0],
   15 => ['BY',0],
   16 => ['EXIT',0],
};
$extended_tokens{"b4w"}={};
Rgds
Stephen

User avatar
jgharston
Posts: 2764
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield

Re: Converting Tokenised BASIC to Plain Text

Postby jgharston » Mon Mar 24, 2014 10:32 pm

sweh wrote:Well, if I've done it properly...
...
Ah, that's better. We tell it that it's ARM basic and now the 2-byte code is decoded.
(You might recognise the code line :-))
'Array!
... at line 10 ;)

BTW, RISC OS equivalent is BasConv. Can be run from the Desktop and from the command line.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_


Return to “software & utilities for the pc, mac or unix”

Who is online

Users browsing this forum: No registered users and 2 guests