Skip to content

Very large errors when quantizing the rotation (with a solution) #31

@f-dy

Description

@f-dy

When quantizing rotations, rounding to nearest is not necessarily optimal.
Consider the following normalized quaternion:

0.000300531 (scalar)
0.995283
-0.0967456
-0.0072609

Quantizing to 8 bits with identity ball map (i.e. dropping the scalar, as done in spz) by rounding each of the remaining three components to nearest gives me the following representation:

254/127.5-1
115/127.5-1
127/127.5-1

Reconstructed normalized quat is (note the large difference on the scalar):

0.0774434 (scalar)
0.992157
-0.0980392
-0.00392151

Which gives me the following reconstruction errors:

  • Geodesic quaternion distance (degrees): 8.85895
  • Rotation angle distance (degrees): 8.84882
  • Axis angle distance (degrees): 0.212142

However, the following quantized representation:

255/127.5-1
115/127.5-1
127/127.5-1

Gives me the following reconstructed normalized quat:

0 (scalar)
0.995221
-0.0975706
-0.00390277

Which has the following errors:

  • Geodesic quaternion distance (degrees): 0.399584
  • Rotation angle distance (degrees): 0.0344378
  • Axis angle distance (degrees): 0.19881

The error is 22 times smaller, just by selecting another rounding direction.

I couldn't find a systematic way of finding the right rounding direction other than trying all 8 possibilities for rounding the three components.

I will submit a PR shortly with code that does that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions