beebjit-0.9.3 released: 1770 support and a few fixes

discuss bbc micro and electron emulators (including mame) here!
Post Reply
User avatar
scarybeasts
Posts: 577
Joined: Tue Feb 06, 2018 7:44 am
Contact:

beebjit-0.9.3 released: 1770 support and a few fixes

Post by scarybeasts » Wed Sep 02, 2020 8:56 pm

Hi,

I've released beebjit-0.9.3.
github: https://github.com/scarybeasts/beebjit
Windows build: https://github.com/scarybeasts/beebjit/ ... _0.9.3.zip

The main focus is on continued accurate execution and decent disc chip emulation / protected disc support:

- 1770 support added!
The command line flag -1770 will replace the 8271 chip with a 1770, and load Acorn DFS v2.26 instead of v0.9.
I believe it's a decent emulation. Protected games load fine (except those not 1770 compatible), include those that require Ctrl-Z-Shift-Break special mode to load. The Watford 1770 ROMs also work and they drive the 1770 quite differently to the Acorn DFS's.
The 1770 based disc copies may be run under emulation and should match real hardware in terms of which titles they will copy, which they won't, and which they will crash on :)
discbeast works under emulation (in fact I use it to tidy up HFEs based on SCP or FDI files).
MFM support isn't fully wired up but this would not be hard.

- Variable length tracks supported in the core.
I found a few varied titles where some protected variants use a protection by "Western Security". This protection takes a different code path for 8271 vs. 1770 based machines!! The 1770 protection path is very advanced and requires precise track lengths. These titles now load under emulation with the 1770.
One example of such a title is this particular Space Mission Mada variant:
viewtopic.php?t=19321#p272935
Some versions of Phantom Combat also use it.

- Full support for testing 40/80 dual discs.
If you want to try loading an 80 track dual 40/80 disc as if you had a 40 track drive, there is -opt disc:drive0-40
If you want to try loading a 40 track dual 40/80 disc as if you had an 80 track drive, there is -opt disc:expand-to-80
By default, a 40 track HFE will behave like you have a 40 track drive and an 80 track HFE will behave like you have an 80 track drive.

- Various bug fixes.
A nasty bug in the 8271 driver was fixed which could return $08 (clock error) on some HFE files.
We found that Colossus Chess has crazy keyboard handling. beebjit now handles it ok.
A nasty regression in debugger breakpoint handling was fixed.
The cursor would jump to the wrong place at the end of a line in MODE7.
Some corner case assert fixes for the video handling / 6845 state.


Cheers
Chris

User avatar
Diminished
Posts: 566
Joined: Fri Dec 08, 2017 9:47 pm
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by Diminished » Wed Sep 02, 2020 9:21 pm

Fine work as always. =D>

User avatar
Diminished
Posts: 566
Joined: Fri Dec 08, 2017 9:47 pm
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by Diminished » Tue Sep 29, 2020 2:32 pm

Here's a wacky patch which implements a dumb idea I had -- "fuzzy breakpoints". This came about as I was thinking about generic ways to force a debugger entry at the end of a game's load and decrypt process.

The basic idea is to keep a running score which is decremented on every 6502 instruction. Rules may be defined which have scores associated with them -- if a rule is hit, a bonus will be added to the running total. Once the running total exceeds a defined threshold, a break into the debugger occurs. The running total is clamped so it cannot go negative (although rules with negative scores may be defined in the interests of discounting false positives).

You can do things like set rules which add small bonuses on repeated low memory writes (say &400-&800), then a larger bonus on a JMP outside the current page, hopefully pushing the running total over the threshold and forcing a break.

There are a lot of improvements that could be made. The rule format is a little bit nasty. I've tested it a little but there are probably still bugs.

I'm not sure how useful it will be, but it's here:
fuzzy-breakpoints-v1-rc4.patch.zip
(10.29 KiB) Downloaded 6 times
Here's the inline help from the debugger:

Code: Select all

FUZZY BREAKPOINTS
When a fuzzy breakpoint rule is hit, the score value specified in the rule will
be added to the current fuzzy breakpoint running total (which starts at zero at
the beginning of execution).

When the running total equals or exceeds a threshold value specified with
"fbpt", a break into the debugger will occur, and the running total will be
reset to zero. The running total is decremented by one every 6502 instruction,
and its value is clamped so it can never become negative. A rule's score may be
positive (adding to the running total, pushing it upwards towards the threshold
specified by the "fbpt" command), or negative (subtracting from the running
total). Negatively-scored rules are useful for discounting situations that would
otherwise result in a false positive.

A fuzzy breakpoint rule is added in the following way:

  fpbr s=<score> [match conditions ...]

Match conditions work as follows. Here, "hN" represents some hex byte:

This rule will trigger when the X register takes value h1:

  x=h1

This rule provides multiple match values, and will trigger if X takes any of the
specified hex values h1, h2 or h3:

  x=h1,h2,h3

Ranges are also recognised; matches occur if X takes a value between h1 and
h2, inclusive:

  x=h1-h2

Ranges and multiple values may also be used together; this specifier will
match if X takes the value h1 or h4, or falls within the range h2 to h3,
inclusive:

  x=h1,h2-h3,h4

Rules may similarly be defined for the Y register:

  y=h1,h2

Or, by using 16-bit values, the accumulator:

  a=hhh1-hhh2,hhh3

Or breakpoints can be defined for the program counter taking a particular value:

  p=addr1

Rules may also match memory reads and writes on specified addresses. This will
match on reads from addr1 or addr2 (in hex):

  r=addr1,addr2

This will match on writes to the region between addr3 and addr4, inclusive:

  w=addr3-addr4

Multiple terms may be specified as part of a rule specification; the rule
will match only if ALL the terms are satisfied; for example:

  fbpr s=1000 a=abcd x=0 y=ff

This rule will only match (adding 1000 to the running total) if A has the
value 'abcd', AND X has the value 0, AND Y has the value 'ff'.

Matches can also be made on the execution of specified opcodes; this specifier
will match on a NOP (opcode &EA):

  o=ea

For opcodes that take operand bytes, the operands must also be specified. This
specifier will match on JMP &1000:

  o=4c:0:10

Commas are allowed here too, so this will match both JMP &1000 and JMP &2000:

  o=4c:0:10,4c:0:20

(Ranges may not be given for opcodes or operands.)

Operands may be wildcarded using '*', so this specifier will match any
"JMP absolute" instruction:

  o=4c:*:*

There is also one special syntax for operands, which will match the value of
either PC low or PC high when the instruction is executed. For example, this
matches a "JMP absolute" whose jump target lies only within the current page:

  o=4c:*:ph

"ph" here specifies the high byte of PC when the instruction is executed. To
match an operand on the low byte of PC instead, use "pl". A rather contrived
example will match on a JMP within the current page anywhere except the
current instruction:

  o=4c:pl:*

"pl" and "ph" may also be inverted by prefixing them with "!". This rule will
add 1000 to the running total if a "JMP absolute" is made anywhere outside of
the currently-executing page:

  fbpr s=1000 o=4c:*:!ph

This currently only works with PC; it is not possible at this time to match
the contents of A, X or Y against instruction operands.

It should of course be noted that fuzzy breakpoints may be used similarly to
traditional non-fuzzy breakpoints simply by setting the score threshold to
a value equal to the value specified in the rule, whereupon the rule will break
immediately.

User avatar
scarybeasts
Posts: 577
Joined: Tue Feb 06, 2018 7:44 am
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by scarybeasts » Tue Oct 13, 2020 7:53 am

Diminished wrote:
Tue Sep 29, 2020 2:32 pm
Here's a wacky patch which implements a dumb idea I had -- "fuzzy breakpoints". This came about as I was thinking about generic ways to force a debugger entry at the end of a game's load and decrypt process.
Sorry for the slow reply -- this is a very interesting idea!

I wonder if it could be worked into a generalized breakpoint facility for even more power?
i.e. we can view breakpoints as:

if (list of conditions) then action

Currently, conditions are rather mundane (PC, register value matches, address range touched).
Actions are even more mundane (break).

So one path could be:
1) Spruce up the actions. I've been meaning to add "print" for a while so that we have memory watchpoints as well as the existing memory breakpoints. A more fun addition would be "change variable". i.e. clear or increment some debugger variable if the condition hits.
2) Spruce up the conditions. Once you've got "change variable" as an action, I think adding "variable comparisons" as a condition goes a long way. i.e. if a variable has gone over some threshold then break, similar to the fuzzy breakpoint idea.


My main reservation about this is: do we want to implement a scripting language of sorts in the built-in debugger? Or would it be cleaner to encourage the scripting to be done externally, along the lines of "cat 'command_file' | beebjit -debug" or "python debug.py | beebjit -debug" ?


Cheers
Chris

User avatar
tricky
Posts: 4785
Joined: Tue Jun 21, 2011 9:25 am
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by tricky » Tue Oct 13, 2020 3:51 pm

How about run command line debugger command(s) and then add new command as required.

User avatar
Diminished
Posts: 566
Joined: Fri Dec 08, 2017 9:47 pm
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by Diminished » Thu Oct 15, 2020 6:27 pm

scarybeasts wrote:
Tue Oct 13, 2020 7:53 am
I wonder if it could be worked into a generalized breakpoint facility for even more power?
i.e. we can view breakpoints as:

if (list of conditions) then action

Currently, conditions are rather mundane (PC, register value matches, address range touched).
Actions are even more mundane (break).

So one path could be:
1) Spruce up the actions. I've been meaning to add "print" for a while so that we have memory watchpoints as well as the existing memory breakpoints. A more fun addition would be "change variable". i.e. clear or increment some debugger variable if the condition hits.
2) Spruce up the conditions. Once you've got "change variable" as an action, I think adding "variable comparisons" as a condition goes a long way. i.e. if a variable has gone over some threshold then break, similar to the fuzzy breakpoint idea.
Yeah, this sounds like a pretty nice way of generalising this stuff. I don't know if just integrating some external scripting language would be the fastest way of getting it done. Not an expert but I think languages like TCL and Lua are intended for integration into other stuff in this way? Of course there are also all sorts of reasons why this might be a bad idea.
My main reservation about this is: do we want to implement a scripting language of sorts in the built-in debugger? Or would it be cleaner to encourage the scripting to be done externally, along the lines of "cat 'command_file' | beebjit -debug" or "python debug.py | beebjit -debug" ?
I think you need the interactivity in there, don't you? Break, inspect variables, all that stuff?

The trouble of course is that writing a nice console debugger with all the user-friendly trimmings is an annoying amount of work. You generally need to drop the Unix console into uncooked mode and implement your own line editor if you want to do things like uparrowing and downarrowing through the command history. Then you have to do it twice because the Windows console works totally differently. You could open another GUI window and do it graphically instead, but then you'll need a font renderer like FreeType or something. Still an annoying amount of work, and still comes with dependencies.

By the way, am I right in thinking that at the moment -- if you modify an opcode in RAM from within the debugger -- it doesn't force a JIT recompilation? I had some strange stuff going on the other day when I was poking around in Citadel's innards. Might have been something I did wrong, though.

User avatar
scarybeasts
Posts: 577
Joined: Tue Feb 06, 2018 7:44 am
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by scarybeasts » Thu Oct 15, 2020 8:46 pm

Diminished wrote:
Thu Oct 15, 2020 6:27 pm
My main reservation about this is: do we want to implement a scripting language of sorts in the built-in debugger? Or would it be cleaner to encourage the scripting to be done externally, along the lines of "cat 'command_file' | beebjit -debug" or "python debug.py | beebjit -debug" ?
I think you need the interactivity in there, don't you? Break, inspect variables, all that stuff?
Hmm yes. Better built-in beebjit debugger it is then!
By the way, am I right in thinking that at the moment -- if you modify an opcode in RAM from within the debugger -- it doesn't force a JIT recompilation? I had some strange stuff going on the other day when I was poking around in Citadel's innards. Might have been something I did wrong, though.
It should force a JIT recompilation upon next execution. The debugger "writem" command writes memory with the "official" bbc_memory_write() function which in turn calls memory_range_invalidate() on the CPU driver.
That said, it could be buggy. I doubt it has seen much testing!
When I'm debugging something, I often use -mode interp. I do mostly trust the JIT engine but the debugger can do strange things like show the same opcode twice when JIT has to drop out to use the interpreter for some complicated situation (hardware register access, self-modifying code, etc.)
Come to think of it, Citadel does use a bunch of self-modifying code.....


Cheers
Chris

User avatar
Diminished
Posts: 566
Joined: Fri Dec 08, 2017 9:47 pm
Contact:

Re: beebjit-0.9.3 released: 1770 support and a few fixes

Post by Diminished » Thu Oct 15, 2020 9:17 pm

I wish I could remember exactly what I did, but I can't.

The emulator hung completely. Had to kill -9 it.

I'll be sure to make a note if I manage to do it again.

Post Reply

Return to “8-bit acorn emulators”