Matrix Brandy BASIC VI for console and SDL1.2: V1.22.9 released

[Soruk]

I'm noticing a problem with the latest brandy from github. WAIT with a number parameter doesn't seem to work anymore. Typing, for example, "WAIT 1000" at the prompt just returns instantly.

Whoops. The variable holding the current centisecond timer value was a signed 32-bit int - and is now wrapping. Fixed.

[Soruk]

Hi,

I've put together release V1.22.1 of Matrix Brandy.

Changes include:
- Make the old BASIC I-IV compatible integer mathematics available via a run-time switchable setting.
- Integer addition and subtraction will now return integer if it fits, else a float. (Legacy mode: unchanged, will always return an int, with wraparound).
- Integer multiplication follows new code path at all times, old int-only implementation didn't match any Acorn BBC BASIC version and is thus a bug.
- Fixed stack pointer leak bug.
- Fixed behaviour setting HIMEM immediately after LOADing a file
- SYS "Brandy_GetVideoDriver" extended, returns MODE 7 framebuffer base in R4.
- RISC OS style sound working. MANY thanks to David Hawes.
- Separate out segfault memory error to its own error message.
- Fix pass-back of strings from SYS on 64-bit systems.
- Fix sound with volume = 0 (Thanks David Hawes)
- Sound updates in StringLib family (Thanks David Hawes)
- Bug fix for VAL - it should ONLY handle decimal numbers (not hex or binary)
- Fix flood fill (it just didn't work right), new implementation is however quite memory hungry and recursive.
- Signal handler updated, longjmps now use siglongjmp. check_read() and check_write() no longer required. Note this also allows a program to trample on the BASIC stack.
- Many keyboard updates (Thanks Jonathan Harston)

Downloads including Windows binary are available on the website.

[Soruk]

Subsequent to the V1.22.1 release, I have now merged the 64-bit int capability to the master branch.

In addition, the relative addressing memory model is now gone, PAGE, HIMEM (etc) and indirection operators now use actual memory locations.

[dhg2]

Bug fix for VAL - it should ONLY handle decimal numbers (not hex or binary)

I think that's unfortunate. That was a pretty nice feature that I used quite a bit. Though I can understand why you removed it.

[Soruk]

Bug fix for VAL - it should ONLY handle decimal numbers (not hex or binary)

I think that's unfortunate. That was a pretty nice feature that I used quite a bit. Though I can understand why you removed it.

I could make it switchable, in the same way I've made other features tunable.

[dhg2]

That would be cool, if it's not a hassle for you to implement. Perhaps it would be good as a compile time option.

[Soruk]

That would be cool, if it's not a hassle for you to implement. Perhaps it would be good as a compile time option.

Pull the latest git, and the old behaviour can be enabled with

Code: Select all

SYS "Brandy_ExtendVAL",1

[Richard Russell]

Bug fix for VAL - it should ONLY handle decimal numbers (not hex or binary)

I think that's unfortunate. That was a pretty nice feature that I used quite a bit.

I strongly disagree. It was a bug, and as such broke some of my programs (those that rely on VAL being guaranteed to return zero for any string starting with a character other than 0-9 or +/-). I do not agree with bugs being promoted to 'features' if they are incompatible in this way..

[dhg2]

@Soruk
Thanks!

@Richard
I recognise that it's correct to remove it, and that it's non-standard behaviour that will break a lot of programs written for other versions of BASIC.
In my case I have one or two programs written for brandy that used it, and I might otherwise have needed to update them.
So it's kind of Soruk to keep it as an option for my sake.

While we're at it, I'd like to ask. With this bug removed, what's the correct way to turn a string with a hexadecimal number in it back into an integer? Or a string with a binary number.

Edit: Sorry, I found the answer to this question myself. Using EVAL instead of VAL works.
I feel a bit silly that I didn't think of using EVAL much sooner. Also thanks to Richard for providing more information on this.
I feel bad saying this after you added the option feature for my sake, but now that I'm aware of the proper way of doing things, it seems that it might be best if you removed the option to restore the previous brandy VAL behaviour.

[Soruk]

I feel bad saying this after you added the option feature for my sake, but now that I'm aware of the proper way of doing things, it seems that it might be best if you removed the option to restore the previous brandy VAL behaviour.

I've reverted that change, and pushed that to git.

[Soruk]

Other recent updates include ensuring that PAGE, HIMEM and friends all return positive numbers on 32-bit hardware (as they now return 64-bit ints). Thanks to @jgharston to pointing that one out over on github.

[Soruk]

New changes committed: Initial attempt at console-mode text builds for Windows built in Cygwin.

Good news: It builds, and runs.
Bad news: It echoes back everything typed a second time.

Edit: Fixed.
Edit 2: Windows nightly builds now include the "brandy" SDL version and both "sbrandy" and "tbrandy" text-mode versions.

[dhg2]

Hello,
I've just noticed today that one of my old programs from last year causes an 'Address exception' error on the latest github version of matrix brandy.
Here is the program that causes the error:

jimmywales.txt

(67.05 KiB) Downloaded 36 times

[Richard Russell]

I've just noticed today that one of my old programs from last year causes an 'Address exception' error on the latest github version of matrix brandy.

For that program to run successfully in BBCSDL (on a 64-bit platform) it would be necessary to change pic% to pic%%, p% to p%% and P% to P%% (because they all hold memory pointers). Perhaps the same changes are needed for Matrix Brandy, now it uses true 64-bit addresses.

[dhg2]

Thanks Richard, I actually wasn't aware that Matrix Brandy had been changed to use true 64 bit addresses. (I've been a bit busy lately and haven't been following things as closely. I should have read up more before posting, sorry)
This is a really cool new feature and I'm excited to learn that it's been added to brandy.

Anyway, that has now lead me to discover that if you try to do

Code: Select all

DIM variable%% (number)

it causes a "type mismatch: numeric variable wanted" error. So it seems that DIM needs to be updated to accept 64 bit integer variables.

[Richard Russell]

Other recent updates include ensuring that PAGE, HIMEM and friends all return positive numbers on 32-bit hardware

Is that really what you mean? Suppose that, on a 32-bit platform, HIMEM is &C0000000 (not possible in Windows, normally, but definitely possible in 32-bit Linux or Android). Since it's a 32-bit platform it should be legitimate to assign HIMEM to a 32-bit integer variable:

Code: Select all

      Himem% = HIMEM

which means that HIMEM must return a negative value (-1073741824). If you are now "ensuring that PAGE, HIMEM and friends all return positive numbers" it would imply that in this case HIMEM would return 3221225472 and an attempt to assign it to the variable Himem% would result in a 'Number too big' error, or similar. This surely can't be right.

What BBC BASIC for SDL 2.0 does is to ensure that on a 32-bit platform you can always use 32-bit integer variables to hold addresses (pointers), which is what you would expect and is compatible with other programming languages. That means that PAGE, HIMEM and friends all return signed numbers - as they always have.

[Soruk]

I have family over this weekend but will address @dhg2's issue in the coming days.

What BBC BASIC for SDL 2.0 does is to ensure that on a 32-bit platform you can always use 32-bit integer variables to hold addresses (pointers), which is what you would expect and is compatible with other programming languages. That means that PAGE, HIMEM and friends all return signed numbers - as they always have.

If I read that right, a program written to run on BBCSDL on a 32-bit platform can then fail on a 64-bit platform. That can't be right.... can it?

By making addresses consistently 64-bit clean in Matrix Brandy, admittedly you need to use 64-bit variables but then it shouldn't matter what the host platform is.

Assigning a signed 32-bit value to PAGE and friends and use with indirections on 32-bit hardware are handled gracefully, so no surprises to the programmer.

[Richard Russell]

If I read that right, a program written to run on BBCSDL on a 32-bit platform can then fail on a 64-bit platform. That can't be right.... can it?

If you want a program to run on both 32-bit and 64-bit platforms you must use 64-bit variables for addresses/pointers; they are capable of holding either full 64-bit addresses or (sign-extended) 32-bit addresses. If you want it to run only on 32-bit platforms, you can use either 32-bit variables or 64-bit variables, but using 32-bit variables will hugely improve compatibility with Acorn versions of BBC BASIC.

By making addresses consistently 64-bit clean in Matrix Brandy, admittedly you need to use 64-bit variables but then it shouldn't matter what the host platform is.

No programming language that I'm aware of (with the possible exception of Matrix Brandy, currently) requires you to use 64-bit variables to hold 32-bit addresses.

As far as I'm concerned the overriding consideration is that this code should run successfully on all 8, 16 and 32-bit versions of BBC BASIC:

Code: Select all

      DIM P% size%

If you are saying that might not work in Matrix Brandy, even when running on a 32-bit platform, I am extremely surprised.

[julie_m]

If you want a program to run on both 32-bit and 64-bit platforms you must use 64-bit variables for addresses/pointers; they are capable of holding either full 64-bit addresses or (sign-extended) 32-bit addresses. If you want it to run only on 32-bit platforms, you can use either 32-bit variables or 64-bit variables, but using 32-bit variables will hugely improve compatibility with Acorn versions of BBC BASIC.

If you compile the interpreter for a 32-bit platform, then even when it is running on a 64-bit processor, it will only ever use 32-bit pointers. As far as it's concerned, it has the entire 32-bit addressing space to itself, but a 32-bit userland process cannot see more than 4GB of memory.

If you compile the interpreter for a 64-bit platform, then it will use 64-bit pointers internally. But if you make sure your process never tries to allocate itself any more than 4GB of memory, then and only then can you safely assume the difference between any of your pointers and your own base address will fit into 32 bits.

So you ought to be able safely to use 32-bit pointers as long as (1) you limit your memory usage, and (2) all addresses that BASIC programs wind up seeing are relative and not absolute. BASIC doesn't mind negative addresses anyway -- although it won't let you specify the unsigned equivalent of a number with the high bit set.

For maximum compatibility, the interpreter should work with relative pointers throughout. 32-bit code will be fine on any architecture. The 64-bit version of the code will just work as long as it properly extends the sign of 32-bit pointers, and has allocated itself less than 4GB of memory. Beyond that limit, it will be necessary to use 64-bit variables throughout for pointers.

[Soruk]

So you ought to be able safely to use 32-bit pointers as long as (1) you limit your memory usage, and (2) all addresses that BASIC programs wind up seeing are relative and not absolute. BASIC doesn't mind negative addresses anyway -- although it won't let you specify the unsigned equivalent of a number with the high bit set.

This is what Brandy used to do, however it meant I had to do some rather horrible hacks for SYS to pass back strings safely on 64-bit systems, and should I get round to implementing dlopen() to accessing functionality in other modules relative addressing is a non-starter.

[julie_m]

In which case, the next best thing would be for the 64-bit interpreter to throw an error if you try to assign a pointer to a 32-bit variable while there is more than 4GB allocated (or, user-selectably, at all?); and to bodge the high bits of a pointer if an attempt is being made to write to it from a 32-bit variable.

What you don't want to do is make all variables 64-bit internally, but only expose the bottom 32 bits to BASIC. Comparison operators would work on only the bottom 32 bits, so &00000000000000010011000011101010 would be < &00000000000000000011000011101011. Carries and borrows must propagate all the way into the high bits. As foolproof as this sounds in theory, there are bound to be situations in practice where it will go horribly wrong and no way to hard-specify the behaviour you really want.

[Richard Russell]

So you ought to be able safely to use 32-bit pointers as long as (1) you limit your memory usage, and (2) all addresses that BASIC programs wind up seeing are relative and not absolute.

Which is basically what Matrix Brandy used to do. But, as discussed here before, doing that is not compatible with using the SYS statement to access Operating System API functions, which may require you to pass one or more pointers to, or receive one or more pointers from, the OS. To make that possible, without any caveats, BBC BASIC has to present the programmer with 'absolute' addresses not 'relative' addresses.

For maximum compatibility, the interpreter should work with relative pointers throughout.

For "compatibility", maybe, but not for 'functionality'. For BBC BASIC to be the 'general purpose' programming language that it was designed to be, being able to interface with the OS is essential, hence using absolute addresses (requiring 64-bit integer variables) is also essential. That has been fundamental to the design of BBC BASIC for SDL 2.0.

[Richard Russell]

In which case, the next best thing would be for the 64-bit interpreter to throw an error if you try to assign a pointer to a 32-bit variable while there is more than 4GB allocated

BBC BASIC for SDL 2.0 reports the usual 'Number too big' error if you try to assign a value outside the range -2^31 to +2^31-1 to a 32-bit integer variable. As far as it is concerned, an address/pointer is no different from any other numeric value, so when running on a 64-bit platform it means an address in the bottom 2 Gbytes or in the top 2 Gbytes can be assigned to a 32-bit integer variable, but the rest of the 64-bit address range cannot.

bodge the high bits of a pointer if an attempt is being made to write to it from a 32-bit variable.

"Bodge" seems an odd choice of term. The two common ways of coercing a 32-bit value to 64-bits are 'sign extension' (when bit 31 is copied into bits 32-63) and 'zero extension' (when bits 32-63 are filled with zeroes). Sign extension is what BBCSDL uses because BBC BASIC's 32-bit and 64-bit integers are signed, and it means a program using 64-bit variables for addresses/pointers will run without modification on both 32-bit and 64-bit platforms.

Currently BBC BASIC for Windows, BBC BASIC for SDL 2.0 and Matrix Brandy BASIC all support 64-bit signed integer variables using the %% suffix.

[julie_m]

"Bodge" seems an odd choice of term. The two common ways of coercing a 32-bit value to 64-bits are 'sign extension' (when bit 31 is copied into bits 32-63) and 'zero extension' (when bits 32-63 are filled with zeroes).

I mean bodge as in set them to match the high bits of your own chunk of memory, so you wind up with a correct 64-bit value even though you were only given a 32-bit value. I call it a bodge because it appears to violate the Principle of Least Astonishment, depending which angle you look at it from .....

[Soruk]

Is that really what you mean? Suppose that, on a 32-bit platform, HIMEM is &C0000000 (not possible in Windows, normally, but definitely possible in 32-bit Linux or Android). Since it's a 32-bit platform it should be legitimate to assign HIMEM to a 32-bit integer variable:

Code: Select all

      Himem% = HIMEM

which means that HIMEM must return a negative value (-1073741824). If you are now "ensuring that PAGE, HIMEM and friends all return positive numbers" it would imply that in this case HIMEM would return 3221225472 and an attempt to assign it to the variable Himem% would result in a 'Number too big' error, or similar. This surely can't be right.

Conversely, if addresses are signed at 32-bits then you can end up with the odd situation where HIMEM < PAGE. That can't be right either.

While I don't like the idea, perhaps I can allow putting an unsigned 32-bit value into a signed 32-bit integer variable. It'll work for addressing (see below).

As far as I'm concerned the overriding consideration is that this code should run successfully on all 8, 16 and 32-bit versions of BBC BASIC:

Code: Select all

      DIM P% size%

If you are saying that might not work in Matrix Brandy, even when running on a 32-bit platform, I am extremely surprised.

This does work, giving a signed or unsigned address to indirection on 32-bit works fine.

Code: Select all

Matrix Brandy BASIC VI version 1.22.1 (Linux) 20 Sep 2019

Starting with 524288 bytes free

>PRINT ~PAGE
  B7766008
>*uname -a  
Linux pegasus.matrixnetwork.co.uk 2.6.32-754.14.2.el6.i686 #1 SMP Tue May 14 20:04:54 UTC 2019 i686 i686 i386 GNU/Linux
>p%=PAGE

Number is out of range
>DIM p% 100
>PRINT ~p%
  B7766034
>PRINT (PAGE+&2C), ~(PAGE+&2C)
3077988404  B7766034
>PRINT ?p%
         0
>PRINT p%,~p%
-1216978892           B7766034
>?p%=42
>PRINT ?p%, ?(PAGE+&2C)
        42        42
>_

Edit: In the above, on a 32-bit platform all the memory pseudovariables are now signed by default, so you can always do things like p%=PAGE.

[Soruk]

Thanks Richard, I actually wasn't aware that Matrix Brandy had been changed to use true 64 bit addresses. (I've been a bit busy lately and haven't been following things as closely. I should have read up more before posting, sorry)
This is a really cool new feature and I'm excited to learn that it's been added to brandy.

Anyway, that has now lead me to discover that if you try to do

Code: Select all

DIM variable%% (number)

it causes a "type mismatch: numeric variable wanted" error. So it seems that DIM needs to be updated to accept 64 bit integer variables.

Yup, that was a bug. I've just pushed a commit to address this.

[Richard Russell]

Conversely, if addresses are signed at 32-bits then you can end up with the odd situation where HIMEM < PAGE. That can't be right either.

But that's always been true in BBC BASIC, in principle, since the only variables available to hold addresses/pointers are signed integers. Theoretically even Acorn's 32-bit ARM BASICs could have returned HIMEM less than PAGE if the heap happened to cross the &80000000 address boundary. It's something we have never worried about, so why start worrying now?

While I don't like the idea, perhaps I can allow putting an unsigned 32-bit value into a signed 32-bit integer variable.

32-bits is 32-bits, it's just a set of 0's and 1's, and there's no way of telling whether it represents a signed or unsigned number other than how you have declared the variable in which it is stored. In C pointers are neither signed nor unsigned, they are just pointers!

[Soruk]

Conversely, if addresses are signed at 32-bits then you can end up with the odd situation where HIMEM < PAGE. That can't be right either.

But that's always been true in BBC BASIC, in principle, since the only variables available to hold addresses/pointers are signed integers. Theoretically even Acorn's 32-bit ARM BASICs could have returned HIMEM less than PAGE if the heap happened to cross the &80000000 address boundary. It's something we have never worried about, so why start worrying now?

While I don't like the idea, perhaps I can allow putting an unsigned 32-bit value into a signed 32-bit integer variable.

32-bits is 32-bits, it's just a set of 0's and 1's, and there's no way of telling whether it represents a signed or unsigned number other than how you have declared the variable in which it is stored. In C pointers are neither signed nor unsigned, they are just pointers!

I've just pushed an update that reverses the earlier change, so on 32-bit memory addresses are once again signed, but can be made unsigned if really needed with

Code: Select all

SYS "Brandy_PseudovarsUnsigned",1

[Richard Russell]

the only variables available to hold addresses/pointers are signed integers.

Sorry, that was a silly thing to say because of course BBC BASIC's floating point variables can hold positive 32-bit integers over the entire &00000000 to &FFFFFFFF range. But it isn't usual to use floating-point variables for addresses (not least because of the impact on speed) and, even if you do, the pseudo variables like PAGE and HIMEM (and addresses returned by DIM) are still signed.

[dhg2]

Hello,
I notice that if you try to draw text at the location of the graphics cursor (with VDU 5 and then PRINT), and if the current graphics colour is black, no text is drawn.

Tested with this program:

Code: Select all

MODE 1
PROCtextdraw(100,100,"Matrix Brandy BBC BASIC")
END
DEF PROCtextdraw(x%,y%,s$)
LOCAL xx%,yy%
GCOL 0,3
VDU 5
FOR xx%=-1 TO 1
  MOVE x%+xx%*4,y%
  PRINT ;s$;
NEXT
FOR yy%=-1 TO 1
  MOVE x%,y%+yy%*4
  PRINT ;s$;
NEXT
GCOL 0,0
MOVE x%,y%
PRINT ;s$;
VDU 4
ENDPROC

Screenshots from matrix brandy and RPCemu: