[cairo] Rotations with Quaternions

[c***@aol.com]

Hi Cairo Folks,

Would using quaternions for rotation and shear be workable internally in Cairo? It would make for a simplified function for setting up the rotation and shear of a drawing by requiring pitch, roll and yaw parameters. This is something that is visually easier for a programmer to understand even when drawing in 2d.

This is some test code using quaternions with Cairo and GTK.

https://github.com/cecashon/OrderedSetVelociRaptor/blob/master/Misc/cairo_drawings/ellipse3.c

Eric

[Bryce Harrington]

Post by c***@aol.com
Hi Cairo Folks,
Would using quaternions for rotation and shear be workable internally
in Cairo?
It would make for a simplified function for setting up the rotation
and shear of a drawing by requiring pitch, roll and yaw
parameters. This is something that is visually easier for a programmer
to understand even when drawing in 2d.

Internally Cairo uses matrices to handle transformations, and the
rotation and translation APIs are merely convenience routines on top of
the matrix math. You could add other convenience routines on top to
provide programmers different mechanisms for applying transformations
relatively easily, so long as the operations can be rendered into matrix
operations internally to Cairo.

Post by c***@aol.com
This is some test code using quaternions with Cairo and GTK.
https://github.com/cecashon/OrderedSetVelociRaptor/blob/master/Misc/cairo_drawings/ellipse3.c
Eric

This code is doing all the transformation math user-side, calculating
all the coordinates, and then drawing the lines with cairo directly. It
doesn't use Cairo's matrix transformation support at all.

Part of the trouble is that Cairo's transformation code only uses a
6-element (2x3) matrix. For doing quaternions "properly" you would want
16-element (4x4) matrices, although I wonder if since the end result is
going to be a 2D representation if an 8- 9- element matrix would be
sufficient.

I've got some code I'm working on to expand Cairo's api to handle
projective transformation. This will add px and py, thus allowing
8-element matrix transformations. If you would be interested in doing
the code to provide a quaternion-based user API that represents the
changes as 8-element style matrix operations, perhaps you could build
that as a wrapper around the stuff I'm doing?

Bryce

Post by c***@aol.com
--
cairo mailing list
https://lists.cairographics.org/mailman/listinfo/cairo

[c***@aol.com]

Hi Bryce,

A 3x3 matrix should might work for quaternion rotations. In the sample code this is reduced to a 2x2 matrix, or four variables for each circle rotation, by using initial reference points but that wouldn't work in general to preserve rotations. There would be a need to keep the z component of a plane to make the rotations additive. If you assumed z=0 then a first rotation would work but not a second or it wouldn't work as expected.

This probably is something difficult to add to Cairo. There are a lot of places that the cairo_matrix_t touches in the code. Also I think there would be a need for a new matrix for quaternion rotations and if that is done then you run into further difficulties such as the order of matrix multiplication. A lot of speculation at this point.

I got going on this to figure out 3d rotations using Cairo's api. I could get some special case rotations in 3d but not something that would work for a gyro. Working on putting together a little gyro widget with Cairo and GTK.

I am not familiar with the internals of Cairo. I have looked around a little but I don't have a test setup with the current version. Do you have any suggestions for putting together a test setup? I would be interested in getting a test build set up that I could experiment with. Also to take a look at what you have going with projections.

Eric

[Bryce Harrington]

This post might be inappropriate. Click to display it.

[Bill Spitzak]

Although only a 3x3 matrix is needed to specify an perspective transform,
if you want to use 3D rotations you need to use 4x4 matrix. This is because
the result of a rotation depends on the position of the camera from the
rotation origin and also the projection (ie fov) being used, and describing
that requires more than 9 numbers. I feel supporting 4x4 matrix is not a
good idea and thus supporting perspective using rotation is also not a good
idea.

It looks like a "perspective" operation that takes two numbers which I will
call px and py will work and kind of match cairo's current transformation
concatenation. What this does is premultiply the CTM by the matrix [[1 0 0]
[0 1 0] [px py 1]] (in an arrangement where cairo_matrix_t is described as
[[xx xy x0] [yx yy y0] [0 0 1]].

px is 1/x where x is the location of the vanishing point along the x axis.
py is 1/y where y is the location of the vanishing point along the y axis.

The attached images show a 512x512 image where there first is a translate
by 256,256, then a perspective operation, then a translate by -256,-256
(the effect of these transformations is to put the origin in the middle of
the image). In the first sample the perspective is 1/256,0. In the second
it is 0,1/256. In the third it is 1/256,1/256. If you look carefully you
can see the lines converge in each of them at the center of the edges, ie
256 units away from the origin.




​
If the translation is non-zero then the perspective will produce a w (the
lower-right corner of the matrix) that is not 1.0. I think this needs to be
preserved, rather than attempting to normalize the matrix. This is because
it could be zero, but further transforms can restore a non-zero value. The
normalization can be applied when the matrix is used to make projections.

If a client has figured out a projection matrix it wants, it is possible
but really difficult to decompose this into a perspective and
cairo_matrix_t. So you almost certainly want to add an equivalent of
cairo_transform and cairo_set_matrix that take a 3x3 matrix directly. To
make it easy to call with matrixes computed by modern 3d libraries, it
would be best to also take a 4x4 matrix in *column major order* and ignore
the 3rd row and column. In such a matrix a cairo_matrix_t and perspective
would be the following 16 numbers in this order: [xx yx ? px xy yy ? py ? ?
? ? x0 y0 ? 1], ? are ignored numbers.

Perspective also brings up questions about what to do about the line width,
dash patterns, and the font. IMHO applying perspective to these would break
all kinds of optimizations. The only useful thing to support is perspective
transforms of images. It may also be useful to transform path points by it.
For the same reason rotations don't work, it is useless for the client to
provide a 'z' value, you can only draw paths in the projected plane. To
logically support this you certainly need "line width locking" so changing
the CTM does not change the lines.

Another approach that may be more useful is to simply add three xy->uv
pairs to the source image, as an enhancement of the origin-setting code.
These specify that the location uv in the source will end up at xy in the
CTM. Actually the more I think about it, the better this sounds. This gets
the only useful part of 3D that anybody expects Cairo to support, and
allows a program doing it's own 3D projections to easily texture-map a
triangle that has uv coordinates.

Post by Bryce Harrington

Post by c***@aol.com
Hi Bryce,
A 3x3 matrix should might work for quaternion rotations. In the sample
code this is reduced to a 2x2 matrix, or four variables for each
circle rotation, by using initial reference points but that wouldn't
work in general to preserve rotations. There would be a need to keep
the z component of a plane to make the rotations additive. If you
assumed z=0 then a first rotation would work but not a second or it
wouldn't work as expected.

The code I've been hacking on adds to Cairo a 3x3 matrix with the 9th
element fixed to 1 (so effectively is an 8-element matrix). If that
last element is needed, it shouldn't be too hard to add in subsequently;
I've hidden the matrix definition internally so it'd be no API break to
change, just making sure all the math accounts for the transformation.

Post by c***@aol.com
This probably is something difficult to add to Cairo. There are a lot
of places that the cairo_matrix_t touches in the code. Also I think
there would be a need for a new matrix for quaternion rotations and if
that is done then you run into further difficulties such as the order
of matrix multiplication. A lot of speculation at this point.

Yes, it is proving to be rather difficult. Actually, just expanding the
cairo_matrix_t definition to include projection wasn't too bad, but
keeping the cairo_matrix_t API intact and adding projection as a new
matrix type is rather invasive - everywhere that uses cairo_matrix_t
needs updated to handle a cairo_matrix3_t.
But, I'm working my way through it, and I think once that's in place
then I think the quaternion rotation is doable. Since internally Cairo
assumes a 2D surface (Z=0), perhaps some flattening operation would
allow using 9-elements externally and avoid a lot of logical mess by
using 8 internally.

Post by c***@aol.com
I got going on this to figure out 3d rotations using Cairo's api. I
could get some special case rotations in 3d but not something that
would work for a gyro. Working on putting together a little gyro
widget with Cairo and GTK.
I am not familiar with the internals of Cairo. I have looked around a
little but I don't have a test setup with the current version. Do you
have any suggestions for putting together a test setup? I would be
interested in getting a test build set up that I could experiment
with. Also to take a look at what you have going with projections.

Depends on what operating system you're on.
If you're on Linux it's just `apt-get build-dep libcairo2-dev` or the
equivalent for your distro to install the build dependencies, checkout
the code from git, and build it. See INSTALL and README.
There's more tips on the download page and elsewhere, although can't
https://cairographics.org/download/
https://cairographics.org/end_to_end_build_for_win32/
For experimenting with the matrix code, the good thing is that even just
the image backend will be suitable, so don't have to worry about driver
issues at least. :-)
Bryce
--
cairo mailing list
https://lists.cairographics.org/mailman/listinfo/cairo

[Bill Spitzak]

I had some nice images that made the message too big, but actually arrived
at some conclusions that don't use the images anyway. Basically my thoughts
on this:

1. Although only a 3x3 matrix is needed to specify an perspective
transform, if you want to use 3D rotations you need to use 4x4 matrix. This
is because the result of a rotation depends on the position of the camera
from the rotation origin and also the projection (ie fov) being used, and
describing that requires more than 9 numbers. I feel supporting 4x4 matrix
is not a good idea and thus supporting 3D rotations is not a good idea.

2. It looks like perspective could be supported by premultiplying by the
matrix [[1 0 0] [0 1 0] [px py 1]] (in row-major order such that
cairo_matrix_t is [[xx xy x0] [yx yy y0] [0 0 1]]. px is 1/x where x is the
location of the vanishing point on the x axis, and py is 1/y for the y
axis. I think Cairo needs to track the w and not normalize after each
matrix multiply.

3. It may be useless to specify z in 3D transforms. At first I thought it
was completely useless, but it does appear that multiplying the matrix by
[x y z] produces movement that looks perpendicular to the xy plane where
z=1 is the plane, while z=0 is the distance the camera is from the plane.

4. Perspective produces all kinds of questions about font and line width.
IMHO it should *not* be applied to these, as the result is the same as what
could be achieved by drawing a surface and the projecting that. Also it
would break every stroke and font optimization.

5. I then concluded that a far more useful api is that perspective *only*
changes the mapping of source images and has no effect on anything else.
This limits it to what I think is the only useful part of 3D transforms. A
proposed method that would be useful to a lot of software is to allow four
(I incorrectly said 3) xy->uv pairs to be specified. These say that the
position uv in the source lands at xy in the CTM. This would allow a client
to do it's own 3d transforms and then correctly texture-map a quad by
setting the uv of the corners. You could also do three xy->uvw mappings
which would allow a client to texture-map a triangle.

Post by Bryce Harrington

Post by c***@aol.com
Hi Bryce,
A 3x3 matrix should might work for quaternion rotations. In the sample
code this is reduced to a 2x2 matrix, or four variables for each
circle rotation, by using initial reference points but that wouldn't
work in general to preserve rotations. There would be a need to keep
the z component of a plane to make the rotations additive. If you
assumed z=0 then a first rotation would work but not a second or it
wouldn't work as expected.

The code I've been hacking on adds to Cairo a 3x3 matrix with the 9th
element fixed to 1 (so effectively is an 8-element matrix). If that
last element is needed, it shouldn't be too hard to add in subsequently;
I've hidden the matrix definition internally so it'd be no API break to
change, just making sure all the math accounts for the transformation.

Post by c***@aol.com
This probably is something difficult to add to Cairo. There are a lot
of places that the cairo_matrix_t touches in the code. Also I think
there would be a need for a new matrix for quaternion rotations and if
that is done then you run into further difficulties such as the order
of matrix multiplication. A lot of speculation at this point.

Yes, it is proving to be rather difficult. Actually, just expanding the
cairo_matrix_t definition to include projection wasn't too bad, but
keeping the cairo_matrix_t API intact and adding projection as a new
matrix type is rather invasive - everywhere that uses cairo_matrix_t
needs updated to handle a cairo_matrix3_t.
But, I'm working my way through it, and I think once that's in place
then I think the quaternion rotation is doable. Since internally Cairo
assumes a 2D surface (Z=0), perhaps some flattening operation would
allow using 9-elements externally and avoid a lot of logical mess by
using 8 internally.

Post by c***@aol.com
I got going on this to figure out 3d rotations using Cairo's api. I
could get some special case rotations in 3d but not something that
would work for a gyro. Working on putting together a little gyro
widget with Cairo and GTK.
I am not familiar with the internals of Cairo. I have looked around a
little but I don't have a test setup with the current version. Do you
have any suggestions for putting together a test setup? I would be
interested in getting a test build set up that I could experiment
with. Also to take a look at what you have going with projections.

Depends on what operating system you're on.
If you're on Linux it's just `apt-get build-dep libcairo2-dev` or the
equivalent for your distro to install the build dependencies, checkout
the code from git, and build it. See INSTALL and README.
There's more tips on the download page and elsewhere, although can't
https://cairographics.org/download/
https://cairographics.org/end_to_end_build_for_win32/
For experimenting with the matrix code, the good thing is that even just
the image backend will be suitable, so don't have to worry about driver
issues at least. :-)
Bryce
--
cairo mailing list
https://lists.cairographics.org/mailman/listinfo/cairo

[c***@aol.com]

Thinking about this a little more, you can get 3d quaternion rotations using a cairo matrix with the restriction that you only get one rotation. This is assuming z=0 for the rotation. After one rotation z wouldn't be 0 anymore. That would mean you would have to put your 3d rotation and drawing in a cairo_save(), cairo_restore() pair to use it for several consecutive drawings. It wouldn't be directly compatible with Cairo's other 2d tranforms either. Not too bad though considering you get 3d rotation of a plane without changing anything internally in Cairo.

https://github.com/cecashon/OrderedSetVelociRaptor/blob/master/Misc/cairo_drawings/rotation1.c

Eric

[c***@aol.com]

This post might be inappropriate. Click to display it.

[c***@aol.com]

I added a frame clock to rotation1.c for comparison with the VeloGyro widget. Made a few additional changes so they look the same.

https://github.com/cecashon/OrderedSetVelociRaptor/blob/master/Misc/cairo_drawings/rotation1.c

It looks like everything works fine with cairo's transforms. The quaternion rotations look stable across the reals. The quaternion rotations aren't additive with this setup but that isn't a problem with cairo_save() and cairo_restore(). You can still animate rotations without a problem.

Eric

[c***@aol.com]

Hi Bill,

I took a look at your drawings of the checker pattern. They can be drawn with 3d rotations using quaternions. The rotations use a cairo transform so the drawing doesn't change how fonts are drawn. Nothing is changed internally in Cairo.

If you get a chance could you take a look at the following

https://github.com/cecashon/OrderedSetVelociRaptor/blob/master/Misc/cairo_drawings/rotation2.c

I have had some trouble getting the rotations to follow the "aircraft" spec but it is more a matter of how the axis are labelled for that. The drawing has a few squares that are rotated and their perspective changes along with the rotation. Similar to your checker board drawings.

Appreciate any comments.

Eric

[Bill Spitzak]

That program is only doing orthographic projections. In effect this is a
focal length of infinity and the resulting projections are affine which
Cairo already supports. I think when people talk about 3D projections, they
expect perspective with vanishing points.

Although such a projection from an input plane to an output quad can be
represented by a 3x3 matrix, that does not provide enough information to do
a rotation. If the camera is perpendicular to a plane, you get the same
image if the plane is very close to the camera and it has a wide angle of
view, as if it is far from the camera and the angle of view is narrower,
and you get the same 3x3 matrix. However if you rotate the plane about the
vertical axis by some amount, the resulting image is different for the
close-up wide-angle plane than for the far-away narrow-angle plane (there
will be much more perspective for the close-up one). This shows that a 3x3
matrix does not have enough information to perform a 3d rotation.

Post by c***@aol.com
Hi Bill,
I took a look at your drawings of the checker pattern. They can be drawn
with 3d rotations using quaternions. The rotations use a cairo transform so
the drawing doesn't change how fonts are drawn. Nothing is changed
internally in Cairo.
If you get a chance could you take a look at the following
https://github.com/cecashon/OrderedSetVelociRaptor/blob/
master/Misc/cairo_drawings/rotation2.c
I have had some trouble getting the rotations to follow the "aircraft"
spec but it is more a matter of how the axis are labelled for that. The
drawing has a few squares that are rotated and their perspective changes
along with the rotation. Similar to your checker board drawings.
Appreciate any comments.
Eric

[c***@aol.com]

Thanks Bill,

I understand better what you are saying about the perspective of the plane with vanishing points.

Eric

[c***@aol.com]

These are a couple of 3d drawings using Cairo.

https://github.com/cecashon/OrderedSetVelociRaptor/blob/master/Misc/cairo_drawings/spring1.c

There is a spring that has 3d points and uses a 2x3 rotation matrix to get the 2d points to draw with. It also has perspective by adjusting the x and y values by the z value. Just played around with this a little. A cylinder that uses a 2x3 rotation matrix but doesn't clip on the z-plane. And a cube that uses a 3x3 rotation matrix since it needs to retain the z values for clipping. The cube uses a cairo_scale to give it a distance perspective in the animation. The 3d points are eventually mapped to the 2d screen so you can use cairo 2d transform functions.

Cairo is a 2d library but being able to do a few things in 3d might be helpful. Makes drawing and animation some fun also. I still have a lot to figure out with this though.



Eric