You have hit the limits of representation!
The twos complement in 8 bits can represent values from -128 to +127. It is mathematically the same as adding 256 to a number if it is negative Bit 7 represents -128 as opposed to 128, and also directly indicates the sign: numbers in the range -128 to -1 have bit 7 set, whereas numbers in the range 0 to 127 have bit 7 cleared. (The 6502 keeps track of bit 7 of the last number read, written or calculated in the N flag, bit 7 of the processor status word.) We might be working unsigned, or we might be working with multi-byte quantities where only the leftmost bit of the highest byte indicates the sign. The 6502 does not care much. It just does what it's told, and it's up to the programmer to remember what they meant; for instance is the hex number &E3 the signed decimal number -29, the unsigned decimal number 227 or part of a multi-byte number?
When we take the twos complement, we flip the bits and add 1. Flipping 8 bits is the same as subtracting a number from 255, so we have
TWC(B) = 255 - B + 1
= 256 - B
So when we add A to the twos complement of B, we get A + 256 - B = 256 + A - B. Since the 256's bit (if there even is one; if A was less than B, then 256 + A - B will be less than 256 and so there will be nothing to carry) will be in C, the accumulator will contain just A - B.
The SBC instruction modifies what the ADC instruction does, by flipping the bits of the subtrahend. We then use the carry to add one to the answer -- this is why we use SEC before SBC. If the carry is not set when we subtract the least significant byte of a multi-byte number, that means the low byte of the subtrahend was greater than the low byte of the minuend, therefore we need to borrow one from the next larger byte of the minuend -- so we don't
add one when subtracting the next byte, which means the answer comes out one smaller than it should be.
Now when you add two large positive numbers, you can get a total which is greater than 127, and thus will appear negative if we are treating it as a signed twos complement representation. The 6502 has an extra feature; the overflow flag V, in bit 6 of the processor status word. If you try to add two positive numbers (or subtract a negative number from a positive number) and end up with a negative number, or try to add two negative numbers (or subtract a positive number from a negative number) and end up with a positive number, V will be set. This indicates that the sign bit is wrong -- to represent it correctly there should not be a -128s bit, but a +128s bit and a -256s bit.
I've got this code in BCP, for comparing 16-bit values that might range between -32768 and +32767:
Code: Select all
We subtract the 16-bit value at wkspace,Y from the 16-bit value at wkspace,X and discard all but the high byte of the answer (which happens to contain the sign bit). If V is clear, we jump straight to an RTS and back to the caller. If V is set, we EOR the accumulator with &80, which will flip N, before we RTS. Either way, when we return, N will be set if the value at wkspace,X is smaller than the value at wkspace,Y, or clear if the value at wkspace, X is equal to or greater than the value at wkspace,Y; and this will be so even if the two values are so far apart, the difference exceeds the limits of twos complement representation. For instance, the difference between -20 000 and 20 000 -- both of which fit into the 16-bit signed range -- is 40000, which exceeds the range and looks like -25536. In these circumstances, the processor logic is such as to set the V flag because subtracting something negative from a positive number should never produce a negative result. We pick up on V and change the &9C in A to &1C before we RTS to the caller. Then the next BPL or BMI instruction will see the last number looked at was positive (which is the right side of zero, even though the value in the accumulator is otherwise bogus).