Matrix Brandy BASIC VI for Linux with SDL: V1.22.4 released

discuss PC<>Acorn file transfer issues & the use of FDC, XFER, Omniflop/disk etc.
Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 12:20 pm

Richard Russell wrote:
Wed Jul 24, 2019 12:01 pm
Soruk wrote:
Wed Jul 24, 2019 11:12 am
Thus, the new code I added for integer multiplication to switch to floats is now used in all cases
You may prefer not to use GCC builtins in Matrix Brandy, although they are widely supported, but for the record there's __builtin_smul_overflow which will multiply two signed 32-bit numbers together and tell you whether there was an overflow. The compiler will "attempt to use hardware instructions to implement" where possible, like conditional jump on overflow etc.
That's interesting, didn't know about them. However, I would need to be careful as I'm not 100% certain the "int" type would still be 32 bits on 64-bit hardware, I suspect it might be 64-bit. Further experimentation on that is needed.

Phlamethrower
Posts: 113
Joined: Fri Nov 24, 2017 1:35 pm
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Phlamethrower » Wed Jul 24, 2019 12:51 pm

Richard Russell wrote:
Wed Jul 24, 2019 10:13 am
Phlamethrower wrote:
Wed Jul 24, 2019 12:02 am
So at some point during the code conversion process the &CC00000 threshold was added, and the logic for coercing the number into an integer was removed.
Can you explain how this threshold works? I don't know about Brandy but there's nothing similar in my BASICs: I allow an integer overflow to occur (trivially detected using the CPU's overflow flag, in assembly language) and deal with it afterwards, so there's no need to 'predict' it.
The ARM can detect overflow, but only for trivial addition/subtraction operations. E.g. there's no overflow detection for multiplication, and if you're doing addition/subtraction with shifted operands then any bits which are shifted out of the high end of the operand won't contribute to the overflow checks. So your choices are to either try and predict the overflow ahead of time, or use 'wider' arithmetic (e.g. 64bit addition) so that you can add a manual overflow check afterward. Each of these options will have its own performance trade-offs.

So in the case of BASIC VI, it looks like they went down the overflow prediction route. But the value they're using to predict the overflow isn't very accurate, and the overflow check is in the wrong place.

This is the code in question, the loop from FRDDC to FRDINTQ.

Line 908: Read a character from AELINE
Lines 909-912: Branch off to the floating-point routine if the character is a decimal point or exponent
Lines 913-915: Branch to the end of the routine if the character isn't a number
Lines 916-917: Multiply IACC by 10 and add the new digit
Lines 918-920: Check for a potential future overflow and branch off to the floating point routine if one is possible. Else go back to line 908 to look for the next digit.

&CC00000 is an approximation of 2^31/10 (&CCCCCCC), chosen because &CC00000 can be used as an immediate value in the CMP instruction. It's checking to see if the next digit will cause an overflow, but it hasn't checked yet to see if there is another digit coming. And of course it's an underestimate of the true value, so even if the check was in the right place it would still generate some false-positives.
Soruk wrote:
Wed Jul 24, 2019 11:12 am
More playing around with integers, I found this one in the legacy code:

Code: Select all

>SYS "Brandy_LegacyIntMaths",1
>PRINT 1073741824 * 2

Number is out of range
>_
I could be wrong, but in RISC OS BASIC I think that addition and subtraction are the only arithmetic operations that have separate float & integer versions. Multiplication always promotes to float (so you'll only get a "number too big" or similar if you try and assign the result to an integer variable), as does division using '/' - unlike DIV and MOD which I believe are integer-only.

The expression parser/evaluator is exposed to assembler code, so it should be possible to write a test program that attempts different calculations and reports back on whether the 'natural type' of the result is integer or float.
Last edited by Phlamethrower on Wed Jul 24, 2019 12:51 pm, edited 1 time in total.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Wed Jul 24, 2019 1:10 pm

Soruk wrote:
Wed Jul 24, 2019 12:20 pm
I'm not 100% certain the "int" type would still be 32 bits on 64-bit hardware, I suspect it might be 64-bit.
That would be extremely unusual. In my experience int is invariably (no more than) 32-bits and long long is invariably (no fewer than) 64-bits (whilst long may sometimes be 32-bits and sometimes 64-bits). Indeed I've always assumed that one reason why x86-64 instructions default to 32-bits is because that's what the int type usually is.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Wed Jul 24, 2019 1:21 pm

Phlamethrower wrote:
Wed Jul 24, 2019 12:51 pm
&CC00000 is an approximation of 2^31/10 (&CCCCCCC), chosen because &CC00000 can be used as an immediate value in the CMP instruction. It's checking to see if the next digit will cause an overflow
Oh, so it's only used in decimal to binary comversion? That's not at all the area of code I thought you were talking about, the issues that Soruk and I have been discussing are concerned with integer arithmetic (e.g. whether it wraps around or not), not base conversion.
Multiplication always promotes to float
Really? I'm extremely surprised by that. I would have expected multiplying two relatively small integers to be a purely integer operation; it is in all my BASICs.

User avatar
Rich Talbot-Watkins
Posts: 1583
Joined: Thu Jan 13, 2005 5:20 pm
Location: Palma, Mallorca
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Rich Talbot-Watkins » Wed Jul 24, 2019 1:32 pm

Richard Russell wrote:
Wed Jul 24, 2019 1:21 pm
Really? I'm extremely surprised by that. I would have expected multiplying two relatively small integers to be a purely integer operation; it is in all my BASICs.
Wasn't there some discussion on that fairly recently? I think the conclusion was that 6502 BBC BASIC will do purely integer multiplication as long as the multiplicands are less than &8000. Not optimal, but fast to check, and probably catches the vast majority of the cases.
Last edited by Rich Talbot-Watkins on Wed Jul 24, 2019 1:34 pm, edited 1 time in total.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 1:36 pm

Richard Russell wrote:
Wed Jul 24, 2019 1:10 pm
Soruk wrote:
Wed Jul 24, 2019 12:20 pm
I'm not 100% certain the "int" type would still be 32 bits on 64-bit hardware, I suspect it might be 64-bit.
That would be extremely unusual. In my experience int is invariably (no more than) 32-bits and long long is invariably (no fewer than) 64-bits (whilst long may sometimes be 32-bits and sometimes 64-bits). Indeed I've always assumed that one reason why x86-64 instructions default to 32-bits is because that's what the int type usually is.
So, I stand corrected, and pleasantly surprised.
On my 32-bit x86 system:

Code: Select all

[soruk@CentOSvm tmp]$ ./intsizes 
    short int: 2
          int: 4
     long int: 4
long long int: 8
       size_t: 4
        void*: 4
[soruk@CentOSvm tmp]$ _
And, on my 64-bit x86-64 system:

Code: Select all

[soruk@Xenon tmp]$ ./intsizes
    short int: 2
          int: 4
     long int: 8
long long int: 8
       size_t: 8
        void*: 8
[soruk@Xenon tmp]$ _
My Raspberry Pi gives the same answers as the 32-bit x86.

Phlamethrower
Posts: 113
Joined: Fri Nov 24, 2017 1:35 pm
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Phlamethrower » Wed Jul 24, 2019 2:09 pm

Richard Russell wrote:
Wed Jul 24, 2019 1:21 pm
Phlamethrower wrote:
Wed Jul 24, 2019 12:51 pm
&CC00000 is an approximation of 2^31/10 (&CCCCCCC), chosen because &CC00000 can be used as an immediate value in the CMP instruction. It's checking to see if the next digit will cause an overflow
Oh, so it's only used in decimal to binary comversion? That's not at all the area of code I thought you were talking about, the issues that Soruk and I have been discussing are concerned with integer arithmetic (e.g. whether it wraps around or not), not base conversion.
Correct - maybe I wasn't clear enough in my first post. My point was that the behaviour of the float & integer arithmetic hasn't changed between BASIC V and BASIC VI (as evidenced by the A%-B% and A-B examples I gave). The different behaviour that Soruk was seeing for "-2147483647 - 2147483647" wasn't caused by a deliberate change that was intended to remove integer wrapping. Instead it's just a bug/quirk in the number parsing code.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 2:24 pm

Phlamethrower wrote:
Wed Jul 24, 2019 2:09 pm
Richard Russell wrote:
Wed Jul 24, 2019 1:21 pm
Phlamethrower wrote:
Wed Jul 24, 2019 12:51 pm
&CC00000 is an approximation of 2^31/10 (&CCCCCCC), chosen because &CC00000 can be used as an immediate value in the CMP instruction. It's checking to see if the next digit will cause an overflow
Oh, so it's only used in decimal to binary comversion? That's not at all the area of code I thought you were talking about, the issues that Soruk and I have been discussing are concerned with integer arithmetic (e.g. whether it wraps around or not), not base conversion.
Correct - maybe I wasn't clear enough in my first post. My point was that the behaviour of the float & integer arithmetic hasn't changed between BASIC V and BASIC VI (as evidenced by the A%-B% and A-B examples I gave). The different behaviour that Soruk was seeing for "-2147483647 - 2147483647" wasn't caused by a deliberate change that was intended to remove integer wrapping. Instead it's just a bug/quirk in the number parsing code.
Hmm. Another quirk. Your A%-B% example earlier, yes, I can replicate that in BASIC VI giving 2. However, after I fixed the integer addition in Brandy where I can give the figures at the immediate mode prompt and it now promotes to float if it can't fit the result in an int, it does the same for integer variable addition - it doesn't wrap, so I get this:

Code: Select all

>SYS "Brandy_LegacyIntMaths",0
>PRINT A%-B%                  
-4.29496729E+09
>SYS "Brandy_LegacyIntMaths",1
>PRINT A%-B%                  
         2
>_
Due to the way Brandy works, I can't make it precisely match BASIC VI's behaviour, either it wraps for immediate integers and integer variables, or it doesn't, I can't separate them into wrapping on integer variables but not immediate integers. I think that's one variation I'm just going to have to live with.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Wed Jul 24, 2019 3:01 pm

Soruk wrote:
Wed Jul 24, 2019 2:24 pm
I can't separate them into wrapping on integer variables but not immediate integers. I think that's one variation I'm just going to have to live with.
In my BASICs an immediate numeric constant is an integer if you don't include a decimal point and a float if you do (assuming the value itself fits in an integer). So 123 is an integer, and ought always to behave the same way as an integer variable containing that value, and 123.0 is a float and ought always to behave the same way as a floating-point variable would (albeit that in my BASICs 'suffixless' variables are variants, not floats, so can contain either).

Phlamethrower
Posts: 113
Joined: Fri Nov 24, 2017 1:35 pm
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Phlamethrower » Wed Jul 24, 2019 3:05 pm

Soruk wrote:
Wed Jul 24, 2019 2:24 pm
Due to the way Brandy works, I can't make it precisely match BASIC VI's behaviour, either it wraps for immediate integers and integer variables, or it doesn't, I can't separate them into wrapping on integer variables but not immediate integers. I think that's one variation I'm just going to have to live with.
Is it possible to make it match the BASIC V behaviour? Because I'm of the opinion that the BASIC VI behaviour is a bug.

(unless there's some other discrepancy between V & VI that can't be explained away by the parser change I've been talking about?)

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 3:09 pm

Phlamethrower wrote:
Wed Jul 24, 2019 3:05 pm
Soruk wrote:
Wed Jul 24, 2019 2:24 pm
Due to the way Brandy works, I can't make it precisely match BASIC VI's behaviour, either it wraps for immediate integers and integer variables, or it doesn't, I can't separate them into wrapping on integer variables but not immediate integers. I think that's one variation I'm just going to have to live with.
Is it possible to make it match the BASIC V behaviour? Because I'm of the opinion that the BASIC VI behaviour is a bug.

(unless there's some other discrepancy between V & VI that can't be explained away by the parser change I've been talking about?)
At run-time, do: SYS "Brandy_LegacyIntMaths",1 - though I struggle to be convinced that getting a correct answer than a wrap-around is a bug!
Last edited by Soruk on Wed Jul 24, 2019 3:10 pm, edited 1 time in total.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 3:12 pm

Richard Russell wrote:
Wed Jul 24, 2019 3:01 pm
Soruk wrote:
Wed Jul 24, 2019 2:24 pm
I can't separate them into wrapping on integer variables but not immediate integers. I think that's one variation I'm just going to have to live with.
In my BASICs an immediate numeric constant is an integer if you don't include a decimal point and a float if you do (assuming the value itself fits in an integer). So 123 is an integer, and ought always to behave the same way as an integer variable containing that value, and 123.0 is a float and ought always to behave the same way as a floating-point variable would (albeit that in my BASICs 'suffixless' variables are variants, not floats, so can contain either).
Similarly in Matrix Brandy, an immediate numeric constant is an integer if it doesn't include a decimal point, unless it overflows a 32-bit int (so 4294967296 would be a float). BASIC VI (and now Matrix, unless you enable LegacyIntMaths) will quite happily allow me to add 2 integer constants, and return a float if is needed. But, BASIC VI (but not Matrix, unless you enable LegacyIntMaths) will wrap when adding two integer variables.
Last edited by Soruk on Wed Jul 24, 2019 3:15 pm, edited 1 time in total.

Phlamethrower
Posts: 113
Joined: Fri Nov 24, 2017 1:35 pm
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Phlamethrower » Wed Jul 24, 2019 3:54 pm

Soruk wrote:
Wed Jul 24, 2019 3:09 pm
though I struggle to be convinced that getting a correct answer than a wrap-around is a bug!
True - without a proper specification for the BBC BASIC language this is all conjecture, and different implementations are free to do as they please.
BASIC VI will quite happily allow me to add 2 integer constants, and return a float if is needed
And what if you want to add 16 integer constants - should that wrap or convert to float?

Code: Select all

PRINT 213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504
PRINT 213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503
213909504 = &CC00000, the magic buggy threshold value that BASIC VI uses. So under BASIC VI the first example will produce a floating point result, and the second example will produce a (wrapped) integer result. Under BASIC V they'll both produce wrapped integer results. BASIC VI behaviour is a bug.
Last edited by Phlamethrower on Wed Jul 24, 2019 3:55 pm, edited 1 time in total.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 3:58 pm

Phlamethrower wrote:
Wed Jul 24, 2019 3:54 pm
Soruk wrote:
Wed Jul 24, 2019 3:09 pm
though I struggle to be convinced that getting a correct answer than a wrap-around is a bug!
True - without a proper specification for the BBC BASIC language this is all conjecture, and different implementations are free to do as they please.
BASIC VI will quite happily allow me to add 2 integer constants, and return a float if is needed
And what if you want to add 16 integer constants - should that wrap or convert to float?

Code: Select all

PRINT 213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504+213909504
PRINT 213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503+213909503
213909504 = &CC00000, the magic buggy threshold value that BASIC VI uses. So under BASIC VI the first example will produce a floating point result, and the second example will produce a (wrapped) integer result. Under BASIC V they'll both produce wrapped integer results. BASIC VI behaviour is a bug.
In Matrix Brandy, by default, neither wrap. Both will wrap if LegacyIntMaths is enabled.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Wed Jul 24, 2019 4:00 pm

Soruk wrote:
Wed Jul 24, 2019 3:12 pm
But, BASIC VI (but not Matrix, unless you enable LegacyIntMaths) will wrap when adding two integer variables.
OK, although I'm now more confused than ever! Have we now established that the only difference in integer behaviour between BASIC V and BASIC VI is in the case of immediate decimal constants, and that they behave exactly the same when the numbers are in integer variables? And, if I've got that right, that the latest Matrix Brandy by default behaves like neither of them when adding or subtracting integers in variables, but more like my BASICs do? Is that a fair summary?
Last edited by Richard Russell on Wed Jul 24, 2019 4:01 pm, edited 1 time in total.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 24, 2019 4:01 pm

Richard Russell wrote:
Wed Jul 24, 2019 4:00 pm
Soruk wrote:
Wed Jul 24, 2019 3:12 pm
But, BASIC VI (but not Matrix, unless you enable LegacyIntMaths) will wrap when adding two integer variables.
OK, although I'm now more confused than ever! Have we now established that the only difference in integer behaviour between BASIC V and BASIC VI is in the case of immediate decimal constants, and that they behave exactly the same when the numbers are in integer variables? And, if I've got that right, that the latest Matrix Brandy by default behaves like neither of them when adding or subtracting integers in variables, but more like my BASICs do? Is that a fair summary?
Seems like that, yes!

User avatar
dhg2
Posts: 148
Joined: Tue Oct 25, 2016 7:37 pm
Contact:

Re: Matrix Brandy BASIC V for Linux with SDL: V1.21.21 released

Post by dhg2 » Thu Jul 25, 2019 12:07 am

Soruk wrote:
Mon Jul 22, 2019 9:28 am
It's been a long time coming, but I've just checked in a commit that implements AUTO.
Thanks very much for this!

By the way, tonight I've come across something that's surprised me and I don't know if it's a bug in brandy, or if I'm just doing something very wrong.
I was bored and I wrote a silly program to pass the time, which is intended to generate random expressions and print out the ones which evaluate to a certain value. The program is causing an error "Expression is too complex to evaluate", but this doesn't actually seem to be true because if I type "PRINT EVAL exp$" at the prompt after the program stops, it evaluates the expression without a problem.

This is the program:
randomexpressions.txt
(1.68 KiB) Downloaded 14 times
Last edited by dhg2 on Thu Jul 25, 2019 12:07 am, edited 1 time in total.
Regards,
- Patrick

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC V for Linux with SDL: V1.21.21 released

Post by Soruk » Thu Jul 25, 2019 8:09 am

dhg2 wrote:
Thu Jul 25, 2019 12:07 am
Soruk wrote:
Mon Jul 22, 2019 9:28 am
It's been a long time coming, but I've just checked in a commit that implements AUTO.
Thanks very much for this!

By the way, tonight I've come across something that's surprised me and I don't know if it's a bug in brandy, or if I'm just doing something very wrong.
I was bored and I wrote a silly program to pass the time, which is intended to generate random expressions and print out the ones which evaluate to a certain value. The program is causing an error "Expression is too complex to evaluate", but this doesn't actually seem to be true because if I type "PRINT EVAL exp$" at the prompt after the program stops, it evaluates the expression without a problem.

This is the program:
randomexpressions.txt
That's an interesting one! I'm away for a long weekend but will take a closer look when I get back.

Edit: Tangentially related, I've commented each error message in errors.c with the error code to aid source tracking more easily - and in the process found three error messages that were out of order with the error codes in errors.h !

Edit 2: Think I fixed it in the latest commit. On an error condition (divide by zero), a stack pointer wasn't being reset.and eventually after 10 errors the stack hit its limit.

A useful hack for debugging both BASIC programs and the Matrix Brandy codebase (but doesn't appear to work in the Windows builds):

Code: Select all

SYS "OS_Write0",<string>,42,42
will send <string> to the controlling terminal on the stderr stream, rather than the Brandy window. (Standard RISC OS SWI, but normally only R0 is significant. R1 and R2 both =42 to redirect output is specific to Matrix Brandy)
Last edited by Soruk on Tue Jul 30, 2019 8:00 am, edited 8 times in total.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 31, 2019 8:50 pm

Recent commits have adjusted the memory layout based on the size parameter - notably, if size <= 64K (HIMEM <= &10000) then PAGE is set to &E00, otherwise it's at &8F00 (as per RISC OS). Brandy isn't using the space below PAGE, however, as its workspaces live in memory not addressable from BASIC so (at least for now) all locations are free to use.
I've also just pushed a further update on the memory model, in that the MODE 7 framebuffer will move to &7C00 if the size <= 31K, otherwise it will remain at &FFFF7C00. The SYS "Brandy_GetVideoDriver" call has been extended, to return this base address in R4.
The NEW command, which can take a size parameter (to request a bigger or smaller memory space) will also be reflected in the above memory locations.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Wed Jul 31, 2019 9:55 pm

Soruk wrote:
Wed Jul 31, 2019 8:50 pm
Recent commits have adjusted the memory layout based on the size parameter - notably, if size <= 64K (HIMEM <= &10000) then PAGE is set to &E00, otherwise it's at &8F00 (as per RISC OS).
My understanding of BBC BASIC (all versions) is that the values of PAGE, LOMEM, HIMEM etc., and the value returned by DIM when allocating memory, need to be 'true' machine addresses. If not, the SYS statement (which may involve passing addresses to, or receiving addresses from, the OS) cannot work. For example if, in BBC BASIC for Windows, I want to read a chunk of data from a file to an allocated block of memory I might do:

Code: Select all

      size% = 1000
      DIM memory% size%
      SYS "ReadFile", handle%, memory%, size%, 0, 0
or similarly in BBC BASIC for SDL 2.0:

Code: Select all

      size% = 1000
      DIM memory%% size%
      SYS "SDL_RWread", handle%%, memory%%, 1, size%
Clearly for the OS to store the data read from the file in BASIC's allocated buffer the value passed as the second parameter of the API function in each case must be a memory address that the OS 'understands', not a 'virtual' address within a sandboxed block of memory private to BBC BASIC.

If Matrix Brandy is returning 'artificial' addresses for PAGE etc., perhaps in an attempt to improve compatibility with Acorn's BASICs, doesn't that prevent SYS being used in this kind of way? Even if Brandy's SYS doesn't currently support calling API functions that receive or return addresses, surely you would not want to prevent it ever doing so?

Right back to the BBC Micro, BASIC programmers have had to get used to the value of PAGE being variable between platforms (e.g. &E00 with CFS and &1900 with DFS, or something completely different on a Second Processor, or something different again in ARM BASIC).

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Wed Jul 31, 2019 10:31 pm

Richard Russell wrote:
Wed Jul 31, 2019 9:55 pm
Soruk wrote:
Wed Jul 31, 2019 8:50 pm
Recent commits have adjusted the memory layout based on the size parameter - notably, if size <= 64K (HIMEM <= &10000) then PAGE is set to &E00, otherwise it's at &8F00 (as per RISC OS).
My understanding of BBC BASIC (all versions) is that the values of PAGE, LOMEM, HIMEM etc., and the value returned by DIM when allocating memory, need to be 'true' machine addresses. If not, the SYS statement (which may involve passing addresses to, or receiving addresses from, the OS) cannot work. For example if, in BBC BASIC for Windows, I want to read a chunk of data from a file to an allocated block of memory I might do:

Code: Select all

      size% = 1000
      DIM memory% size%
      SYS "ReadFile", handle%, memory%, size%, 0, 0
or similarly in BBC BASIC for SDL 2.0:

Code: Select all

      size% = 1000
      DIM memory%% size%
      SYS "SDL_RWread", handle%%, memory%%, 1, size%
Clearly for the OS to store the data read from the file in BASIC's allocated buffer the value passed as the second parameter of the API function in each case must be a memory address that the OS 'understands', not a 'virtual' address within a sandboxed block of memory private to BBC BASIC.

If Matrix Brandy is returning 'artificial' addresses for PAGE etc., perhaps in an attempt to improve compatibility with Acorn's BASICs, doesn't that prevent SYS being used in this kind of way? Even if Brandy's SYS doesn't currently support calling API functions that receive or return addresses, surely you would not want to prevent it ever doing so?

Right back to the BBC Micro, BASIC programmers have had to get used to the value of PAGE being variable between platforms (e.g. &E00 with CFS and &1900 with DFS, or something completely different on a Second Processor, or something different again in ARM BASIC).
Anything DIMmed is in the BASIC space. SYS calls are (currently) only what's provided in the emulation code, and addresses are relative to the base of the BASIC workspace block (set with the -size parameter) appropriate offsetting is done within Brandy so, for example the mmap()ed area of /dev/gpiomem, while allocated an address outside the BASIC workspace area, is readable and writeable. Similarly, memory allocated to the SDL framebuffer is marked as readable and writeable so can be written to and read from BASIC - and in both cases there are SYS calls which do indeed return memory base addresses (and in the case of the screen, size), though these addresses are translated to ones that make sense from within the BASIC environment.

The MODE 7 handling is different, reads and writes to the emulated MODE 7 frame buffer are redirected into the 40x25 char array, and also send a signal to the renderer to re-render the line. Where translations are needed between the physical RAM address and the offset within the BASIC workspace (e.g. SYS returning a pointer to a block like OS_Word 10) these are translated in the SYS call handler for that particular function.

One of your examples uses SYS to read a file? That's not something yet supported in Brandy, and if I do, I'd probably follow the OS_File (and friends) call syntax of RISC OS. Doing it that way has allowed me to do things like take the Snow demo from RISC OS Pico, and run that unmodified. (Indeed, in accordance with the comment at the top of the file, I have decided to include it in the examples directory and will be in the next bundle release.)

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Thu Aug 01, 2019 12:42 am

Soruk wrote:
Wed Jul 31, 2019 10:31 pm
One of your examples uses SYS to read a file? That's not something yet supported in Brandy, and if I do, I'd probably follow the OS_File (and friends) call syntax of RISC OS.
SYS is intended to allow you to call any API function, exported by any library, which in an OS like Windows or Linux could potentially run to millions of functions. The approach you've adopted so far in Matrix Brandy of individually emulating a small set of specific functions isn't scalable.

Perhaps you see Brandy more as an Acorn emulator than a cross-platform implementation of the BBC BASIC language. If so I think that's something of a departure from Dave Daniels' original vision.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Thu Aug 01, 2019 8:57 am

Richard Russell wrote:
Thu Aug 01, 2019 12:42 am
Soruk wrote:
Wed Jul 31, 2019 10:31 pm
One of your examples uses SYS to read a file? That's not something yet supported in Brandy, and if I do, I'd probably follow the OS_File (and friends) call syntax of RISC OS.
SYS is intended to allow you to call any API function, exported by any library, which in an OS like Windows or Linux could potentially run to millions of functions. The approach you've adopted so far in Matrix Brandy of individually emulating a small set of specific functions isn't scalable.

Perhaps you see Brandy more as an Acorn emulator than a cross-platform implementation of the BBC BASIC language. If so I think that's something of a departure from Dave Daniels' original vision.
That may be the case. Previously Brandy (excluding RISC OS builds) had NO support beyond shelling out via *-commands for calling anything in the host OS, it was purely a BASIC interpreter. Maybe dynamic module loading (of modules that weren't linked at build time) is easier in Windows? Maybe my vision of cross-platform is unusual in that I'm wanting a BASIC program in Brandy to be able to work identically across all platforms (yes, I know Raspberry Pi GPIO support isn't supported on any other platform). RISC OS-style graphics is something that existed upstream (though, rather limited), and I've just enhanced that, and that in itself is an emulation of RISC OS's graphics system.

Talking of graphics, there's a BASIC-coded library to support the Tektronics graphics terminal emulation capability of an xterm in Linux/UNIX, which, at the current upstream point only works for text-only builds - effectively exploiting the (equivalent to the) VDU driver of xterm. I've added the functionality and reworked the library so it works on both text and SDL builds (though, it can be argued, why bother, as SDL supercedes this? Because, it's there, and the functionality existed prior to any SDL presence in Brandy).
Last edited by Soruk on Thu Aug 01, 2019 9:03 am, edited 1 time in total.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Thu Aug 01, 2019 11:20 am

Soruk wrote:
Thu Aug 01, 2019 8:57 am
Maybe dynamic module loading (of modules that weren't linked at build time) is easier in Windows?
All modern OSes support dynamic loading of shared libraries ('late binding'). It's generally a two-step process: you first get a handle to the library itself (in Windows that's done with LoadLibrary and in Linux/MacOS/Android/iOS with dlopen), then you find the entry point of the API function (GetProcAddress in Windows, dlsym in the others). The shared library variously has the extension .dll, .so or .dylib depending on platform.
Maybe my vision of cross-platform is unusual in that I'm wanting a BASIC program in Brandy to be able to work identically across all platforms
That's reasonable when it comes to 'internal' functionality of the language, that is functionality which has no dependence on the external environment such as Operating System or hardware. For example it's reasonable to expect number, string or array handling to work 'identically' across all platforms.

But when it comes to 'external' functionality (typically input-output, e.g. graphics or sound or interfacing with other programs and sub-systems) there are two quite different approaches. One is the 'emulation' approach where you choose one particular platform (e.g. RISC OS) and you make your language behave on all platforms as similarly as possible to how it behaves on the emulated platform. The other is the 'native' approach where the language supports as best it can the capabilities of its host. So if the host platform supports, say, accelerated 3D graphics then you make sure that the language provides access to that support, or if it supports playing MP3 files you provide access to that, etc.

Ultimately it comes down to whether you consider BBC BASIC to be a general purpose programming language, like C or Python or Java, or not. My view is absolutely clear on that point: it is. Such a general purpose language must provide access to everything the host is capable of, even if that is at the expense of perfect cross-platform compatibility. My 'BBC BASIC for SDL 2.0' is an attempt at implementing such a language whilst at the same time keeping incompatibilites to the bare minimum through the use of the common SDL 2.0 abstraction layer. It doesn't entirely eliminate them, but it comes close.

From your description you see Matrix Brandy as much more of an emulator, in which you are prepared to sacrifice support for the host's native capabilities to achieve near-perfect cross-platform compatibility. That's your prerogative of course; I don't happen to think it is in the 'spirit' of BBC BASIC (and naturally I'm not keen on RISC OS being the emulated system because I know nothing about it!) but I'm sure there's a niche for it in the BBC BASIC ecosystem.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Thu Aug 01, 2019 1:49 pm

Richard Russell wrote:
Thu Aug 01, 2019 11:20 am
Soruk wrote:
Thu Aug 01, 2019 8:57 am
Maybe dynamic module loading (of modules that weren't linked at build time) is easier in Windows?
All modern OSes support dynamic loading of shared libraries ('late binding'). It's generally a two-step process: you first get a handle to the library itself (in Windows that's done with LoadLibrary and in Linux/MacOS/Android/iOS with dlopen), then you find the entry point of the API function (GetProcAddress in Windows, dlsym in the others). The shared library variously has the extension .dll, .so or .dylib depending on platform.
There is definitely room for this, even without compromising the emulated features, and will definitely be something I will look into.

Certainly the "emulated" OS calls would still be required, as for the environment inside the interpreter they tweak and tune its behaviour (e.g. *FX200 affects whether we respond to ESCAPE), the ColourTrans calls for colour look-ups, and the Brandy SYS calls all get or set other settings specific to Brandy especially the recent work with the integer mathematics, it would be daft to remove them.
Maybe my vision of cross-platform is unusual in that I'm wanting a BASIC program in Brandy to be able to work identically across all platforms
That's reasonable when it comes to 'internal' functionality of the language, that is functionality which has no dependence on the external environment such as Operating System or hardware. For example it's reasonable to expect number, string or array handling to work 'identically' across all platforms.

But when it comes to 'external' functionality (typically input-output, e.g. graphics or sound or interfacing with other programs and sub-systems) there are two quite different approaches. One is the 'emulation' approach where you choose one particular platform (e.g. RISC OS) and you make your language behave on all platforms as similarly as possible to how it behaves on the emulated platform. The other is the 'native' approach where the language supports as best it can the capabilities of its host. So if the host platform supports, say, accelerated 3D graphics then you make sure that the language provides access to that support, or if it supports playing MP3 files you provide access to that, etc.
Some of this would be via SDL, though at this time SDL 1.2 doesn't have accelerated 3D graphics, and Brandy has no support at all for sound. That said, nothing stopping you doing

Code: Select all

OSCLI "mpg321 /path/to/mp3file.mp3 >/dev/null 2>&1 &"
which, if mpg321 is installed on the host, would play that MP3, and would work even on the non-SDL builds.
Ultimately it comes down to whether you consider BBC BASIC to be a general purpose programming language, like C or Python or Java, or not. My view is absolutely clear on that point: it is. Such a general purpose language must provide access to everything the host is capable of, even if that is at the expense of perfect cross-platform compatibility. My 'BBC BASIC for SDL 2.0' is an attempt at implementing such a language whilst at the same time keeping incompatibilites to the bare minimum through the use of the common SDL 2.0 abstraction layer. It doesn't entirely eliminate them, but it comes close.

From your description you see Matrix Brandy as much more of an emulator, in which you are prepared to sacrifice support for the host's native capabilities to achieve near-perfect cross-platform compatibility. That's your prerogative of course; I don't happen to think it is in the 'spirit' of BBC BASIC (and naturally I'm not keen on RISC OS being the emulated system because I know nothing about it!) but I'm sure there's a niche for it in the BBC BASIC ecosystem.
The SDL build, of course provides a canvas on which BASIC can draw on, so bearing in mind BBC BASIC's history (and the initial work done upstream in Brandy 1.20.1) it made sense to continue the path of supporting the BBC Micro and RISC OS drawing functions (though, no sprite functionality yet). The non-SDL builds look back to Brandy's earlier history where it was strictly text-only.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Thu Aug 01, 2019 3:18 pm

Soruk wrote:
Thu Aug 01, 2019 1:49 pm
There is definitely room for this, even without compromising the emulated features, and will definitely be something I will look into.
That's exactly why I wanted to alert you to the potential problem with memory addresses. If you have any ambition to support SYS calling arbitrary functions in shared libraries, a significant proportion of those functions are going to take addresses as parameters (or return them), and they will need to be 'real' machine addresses not 'fake' addresses. No other version of BBC BASIC 'lies' about the addresses of PAGE, HIMEM etc. and programs which make assumptions about their absolute values wouldn't have worked even on the 6502 Second Processor, let alone ARM BASIC or any of my BASICs.
the recent work with the integer mathematics, it would be daft to remove them.
It was your choice to implement the 'integer mathematics' control via a custom SYS. A custom OSCLI would have been an alternative, rather as my BASICs have *FLOAT and *HEX to control various aspects of number handling.
so bearing in mind BBC BASIC's history...
You would expect me to say this, but the "history" you refer to is only half the story: it's the history of Sophie's BASICs! There is a 'parallel' history of my versions of BBC BASIC, starting with the Z80 version in 1982 and working through the MS-DOS, Windows and SDL 2.0 versions which I naturally think are just as deserving of consideration. :wink:

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Thu Aug 01, 2019 4:24 pm

Richard Russell wrote:
Thu Aug 01, 2019 3:18 pm
Soruk wrote:
Thu Aug 01, 2019 1:49 pm
There is definitely room for this, even without compromising the emulated features, and will definitely be something I will look into.
That's exactly why I wanted to alert you to the potential problem with memory addresses. If you have any ambition to support SYS calling arbitrary functions in shared libraries, a significant proportion of those functions are going to take addresses as parameters (or return them), and they will need to be 'real' machine addresses not 'fake' addresses. No other version of BBC BASIC 'lies' about the addresses of PAGE, HIMEM etc. and programs which make assumptions about their absolute values wouldn't have worked even on the 6502 Second Processor, let alone ARM BASIC or any of my BASICs.
This might be quite easy for the 32-bit platforms, but I suspect it'll be a nightmare for 64-bit, unless I shoehorn a 64-bit integer data type in.
so bearing in mind BBC BASIC's history...
You would expect me to say this, but the "history" you refer to is only half the story: it's the history of Sophie's BASICs! There is a 'parallel' history of my versions of BBC BASIC, starting with the Z80 version in 1982 and working through the MS-DOS, Windows and SDL 2.0 versions which I naturally think are just as deserving of consideration. :wink:
Yep, I am certainly aware of that. To be fair, aside from using your DOS version a few times on RM Nimbus machines at school, and more recently tinkering with BBCSDL the only times I've ever really got stuck in programming in BASIC is on Acorn kit (BBC B, Master, Archimedes, RiscPC). So that's the environment path I'm more familiar with.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Thu Aug 01, 2019 5:23 pm

Soruk wrote:
Thu Aug 01, 2019 4:24 pm
This might be quite easy for the 32-bit platforms, but I suspect it'll be a nightmare for 64-bit, unless I shoehorn a 64-bit integer data type in.
That was the solution I adopted, as you know; I don't think there is any realistic alternative (fortunately I'd already added support for 64-bit integers in my 32-bit interpreter, so much of the work had already been done before I moved to 64-bit addressing). You could provide the user with a function which converts 'fake' addresses to 'real' addresses and another that does the reverse; that would cover the simple cases but would still break with more complicated data structures such as pointers to pointers or linked lists and the like.

It's only too common for an API function to take as a parameter a (pointer to a) structure, and within that structure there are one or more pointers, so the structure unavoidably needs to contain 64-bit members in which you can store true memory addresses. No workaround that I know of can resolve that short of having a 64-bit integer data type. At least Brandy (and BASIC VI) already support 64-bit floats so there must be code for handling types of that size (saving them on the stack, passing them as parameters and so on) which ought to help.

Soruk
Posts: 526
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Soruk » Sat Aug 03, 2019 10:44 am

Richard Russell wrote:
Thu Aug 01, 2019 5:23 pm
Soruk wrote:
Thu Aug 01, 2019 4:24 pm
This might be quite easy for the 32-bit platforms, but I suspect it'll be a nightmare for 64-bit, unless I shoehorn a 64-bit integer data type in.
That was the solution I adopted, as you know; I don't think there is any realistic alternative (fortunately I'd already added support for 64-bit integers in my 32-bit interpreter, so much of the work had already been done before I moved to 64-bit addressing). You could provide the user with a function which converts 'fake' addresses to 'real' addresses and another that does the reverse; that would cover the simple cases but would still break with more complicated data structures such as pointers to pointers or linked lists and the like.

It's only too common for an API function to take as a parameter a (pointer to a) structure, and within that structure there are one or more pointers, so the structure unavoidably needs to contain 64-bit members in which you can store true memory addresses. No workaround that I know of can resolve that short of having a 64-bit integer data type. At least Brandy (and BASIC VI) already support 64-bit floats so there must be code for handling types of that size (saving them on the stack, passing them as parameters and so on) which ought to help.
I've got the beginnings of 64-bit ints working (using suffix %%) - does BBCSDL support 64-bit indirections like !addr for 32-bit? It would make sense that, although I can't call Brandy a BBC BASIC implementation, I can say it's an interpreter for the BBC BASIC dialect, to use the same notation yours does.

Edit: You won't see this in the nightlies, I'm doing this work on a branch, and nightly builds mainline.
Last edited by Soruk on Sat Aug 03, 2019 10:45 am, edited 1 time in total.

User avatar
Richard Russell
Posts: 1139
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Matrix Brandy BASIC VI for Linux with SDL: V1.22.0 released

Post by Richard Russell » Sat Aug 03, 2019 11:31 am

Soruk wrote:
Sat Aug 03, 2019 10:44 am
I've got the beginnings of 64-bit ints working (using suffix %%) - does BBCSDL support 64-bit indirections like !addr for 32-bit?
Currently the 32-bit editions don't (and nor does BBC BASIC for Windows); to an extent it is made unnecessary by the existence of structures which allow you to achieve the same effect in a more elegant way:

Code: Select all

      DIM struct{member%%}
      struct.member%% = number
The 64-bit editions do support 64-bit indirection, but you may not like it! The problem is basically the lack of availability of a suitable symbol to use (partly because my BASICs have borrowed all the 'spare' ones for other purposes). After a long debate on the subject at the discussion group, the conclusion we came to was that the only single character operator remaining for this use was ] (it's used to exit the assembler of course, but the syntax guarantees no ambiguity).

So that's what I use, even though it's a bit ugly (various arguments were advanced in mitigation such as its visual similarity to the float indirection operator | and that it's rendered as in MODE 7 which has an indirection-y feel about it!). Like the string indirection operator $ and the float indirection operator | it can only be used in a monadic sense, so ]addr is valid but addr]offset isn't.

Post Reply