Skip to content

Incorrect results #3

Description

@jaminturner

I think there is a problem with the two's complement implementation in this library. It is correct for positive numbers, but for negative numbers, it is incorrect. The math used to catch and convert negative numbers is:

if x_data & 0x80000 == 0x80000:
    x_data = ~x_data + 1

However, this just gives a negative version of x_data, it doesn't truly take it out of 2s complement form. For example, if the number received from the sensor is "1001 0110 0000 0010 1101", the correct decimal integer of that is -434131. However, the above code gives -614445, which is incorrect.

I think it could be corrected as follows:

if x_data & 0x80000 == 0x80000:
    x_data = x_data - 2**20

Am I wrong/confused on this?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions