Skip to content

DM-10065: Client side Rotation and Flipping #348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 27, 2017
Merged

DM-10065: Client side Rotation and Flipping #348

merged 10 commits into from
Apr 27, 2017

Conversation

robyww
Copy link
Contributor

@robyww robyww commented Apr 10, 2017

Branch covers the following:

  • Uses html5 canvas to render a rotated or flipped image
  • package.json: added transformation-matrix-js
  • package.json: added point-in-polygon
  • new TileDrawerCanvas replaces TileDrawer
  • thumbnail working
  • new DevicePt coordinate system
  • removed ViewportPt coordinate system
  • drawing layers mostly working
  • select and selection related operations working with rotation
  • select now has a second box that shows when image is rotated, this is used for cropping and image stats
  • rotate dialog changed to use new rotate, flip button uses new flip
  • WCS match working
  • target match working
  • Time series working
  • Compass working, required adding more logic to work with rotate
  • code cleanup of almost all the affected files
  • removed some old unused code.

Testing:
Here are some functions that need special attention.

  • wcs match
  • target match
  • time series viewer
  • image the are both flipped and rotated
  • compass
  • large zoomed images - zoom it the flip and rotate
  • looking at the thumbnail view
  • magnifier (I don’t think it is right)
  • catalog layers - then rotate
  • testing with images that not dss, wise, or m31

Known Issues that will be handle in separate ticket:

@robyww robyww self-assigned this Apr 10, 2017
@robyww robyww requested review from loitly, tgoldina and cwang2016 April 10, 2017 23:07
@robyww
Copy link
Contributor Author

robyww commented Apr 10, 2017

Files to review by reviewer:

@loitly :

  • package.json
  • visualize/iv/PlotTitle.jsx
  • visualize/iv/TileDrawer.jsx
  • visualize/iv/TileDrawerCanvas.jsx
  • visualize/iv/TileDrawHelper.jsx
  • visualize/iv/EventLayer.jsx
  • visualize/iv/ExpandedTools.jsx
  • visualize/iv/ImageViewerDecorate.jsx
  • visualize/ui/ThumbnailView.jsx
  • visualize/reducer/HandlePlotChange.js
  • visualize/reducer/HandlePlotCreation.js

@cwang2016 :

  • visualize/draw/DirectionArrowDrawObj.js
  • visualize/draw/Drawer.js
  • visualize/draw/FootprintObj.js
  • visualize/draw/MarkerFootprintObj.js
  • visualize/draw/PointDataObj.js
  • visualize/draw/SelectBox.js
  • visualize/draw/ShapeDataObj.js
  • drawingLayers/Catalog.js
  • drawingLayers/FootprintTool.js
  • drawingLayers/MarkerTool.js
  • drawingLayers/NorthUpCompass.js
  • drawingLayers/SelectArea.js
  • visualize/CsysConverter.js

@tgoldina :

  • visualize/ChangePrime.js
  • visualize/ImagePlotCntlr.js
  • visualize/ImageViewerLayout.jsx
  • visualize/PlotPostionUtil.js
  • visualize/PlotState.js
  • visualize/Point.js
  • visualize/WebPlot.js
  • visualize/reducer/PlotView.js
  • visualize/task/WcsMatchTask.js
  • visualize/saga/MouseReadoutWatch.js
  • visualize/VisUtil.js

Less Important / Small changes / code cleanup:

  • visualize/region/RegionDescription.js
  • visualize/RelatedDataUtil.js
  • visualize/ui/VisToolbarView.jsx
  • visualize/task/PlotImageTask.js
  • visualize/task/PlotChangeTask.js
  • ui/FitsRotationDialog.jsx
  • api/ApiUtilImage.jsx
  • core/ExternalAccessUtils.js
  • ui/ZoomOptionsPopup.jsx
  • ui/TargetFeedback.jsx
  • ui/TargetPanel.jsx
  • visualize/saga/ImagePlotter.js
  • visualize/ui/VisCtxToolbarView.jsx
  • visualize/task/ImageOverlayTask.js

Removed unused files:

  • ui/ModalDialog.jsx
  • ui/AllDialogsContainer.jsx

@robyww robyww requested review from ejoliet and ymeiymei April 10, 2017 23:17
@ejoliet
Copy link
Contributor

ejoliet commented Apr 11, 2017

I've built and test randomly the tri-view. So far i've found the following:

  • North-up doesn't work if first image is highlighted: i brought 2 images side by side and rotate them to 45 degrees, then apply north-up doesn't change the images

@ejoliet
Copy link
Contributor

ejoliet commented Apr 11, 2017

Another issue is that i've noticed that the footprint rotation is broken.

@robyww
Copy link
Contributor Author

robyww commented Apr 11, 2017

Another issue is that i've noticed that the footprint rotation is broken.

That was noted in known issues at the top of the pull request. There is a list 3 issues we will do in separate tickets.

pt0 = plot.getViewPortCoords(wpt0);
pt = plot.getViewPortCoords(wpt);
pt0 = plot ? plot.getDeviceCoords(wpt0) : wpt0;
pt = plot ? plot.getDeviceCoords(wpt) : wpt;
if (!pt0 || !pt) return;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should here return with 'if (!onlyAddToPath) DrawUtil.stroke(ctx);'?

@ymeiymei
Copy link
Contributor

Found a bug: After rotation, the restore key can't rotate the image back to the default. How to repeat:
External image, wise, target: 9.6 -1.1, cutout 0.15deg, band 1. Rotate the image by 45deg, then restore it to the defaults. Fail.

@robyww
Copy link
Contributor Author

robyww commented Apr 11, 2017

Found a bug: After rotation, the restore key can't rotate the image back to the default.

Good catch. I did not even think of restore.

Now that I think about it that is a little tricky. In the Time Series case I don't think we want to unrotate.

@ymeiymei
Copy link
Contributor

Found another bug: Now the rotation is done by the given angle relative to the original position. For example, rotate 45deg, then rotate 90deg, the result is only rotating 90 deg. Or rotate 45 deg, then another 45 deg, the image has no change for the second rotation. Before the rotation is relative to the current position.

@cwang2016
Copy link
Contributor

It seems that the rotate angle entered into the 'Rotate Image' popup window means the angle relative to the original image. Will that be good to have some number shown on the popup window indicating the current rotate angle, then the user could enter the rotation angle in addition to current rotate angle?

@robyww
Copy link
Contributor Author

robyww commented Apr 11, 2017

Will that be good to have some number shown on the popup window indicating

It is already showing on the title. We could do it in the dialog as well but I think we might want the to be another ticket.

Copy link
Contributor

@loitly loitly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works really well. I did notice a couple of things. It's perfectly fine to merge if they are recorded in another ticket.

Things I noticed from testing:

  • on Safari, if you zoom an image so that it tiles on the server, you will notice 1-2px gap(s) along the borders.

  • bring up 2 same images (IRAS m31);
    click on 'WCS Match';
    move one off screen;
    click on 'Recenter'; only one returns;
    clicked on the other one, image returned, but warning showed up on the other.
    this worked fine in 'dev' branch.

onImageLoad(e) {
if (e) {
e.onload= () => {
this.setState( () => ({imWidth: e.width, imHeight:e.height }));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this right? You are sending a function to setState. Shouldn't it be the returned object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loi and I just talked about this. This is a new way to setState in react.

@robyww
Copy link
Contributor Author

robyww commented Apr 11, 2017

Found another bug: Now the rotation is done by the given angle relative to the original position.

The new way is more consistent with DS9. I think it is the right way to go.

@tgoldina
Copy link
Contributor

tgoldina commented Apr 12, 2017 via email

@robyww
Copy link
Contributor Author

robyww commented Apr 12, 2017

No advantage for what I was doing it. However, for the next major release of react they are encouraging this to be the main way of setting state. There is some performance optimization this helps enable.

@cwang2016
Copy link
Contributor

review completes.

  • Find some issue for region rendering. If a region is defined on image domain, the rotation angle (and flip) should be applied to the shape rendering. I will look into this issue and fix it in another ticket.
  • Could 'rotate angle' be a negative number?

@robyww
Copy link
Contributor Author

robyww commented Apr 12, 2017

Could 'rotate angle' be a negative number?

Yes, I will look into it.

@ymeiymei
Copy link
Contributor

More issues:

  • Not all the footprints can be rotated by the image rotation (WFIRST can).
    *The footprints can not be rotated by hand more than once.
    *The rotation direction now is clockwise but IRSA's convention is counter-clockwise. See detail in IRSA's help:
    "Rotating the image to any angle
    This feature allows you to rotate the image to any angle of your choice, in degrees. It will rotate the image counter-clockwise (to the left) from the current view, not necessarily the original image. For example, entering "45" in the rotation pop-up and hitting "rotate" will rotate the image 45 degrees counter-clockwise relative to its original orientation. Then selecting the icon again, and entering "180" in the pop-up (followed by hitting "rotate") will rotate the image an additional 180 degrees counter-clockwise. To exit the pop-up without making further changes, hit the blue 'x' in the upper right of the pop-up."

*Need further discussion about the rotation be relative to the original or additional to the current position. In other words, we should follow DS9 way or IRSA way (see IRSA help above).

Continue to test ...

@ymeiymei
Copy link
Contributor

Also the rotation angle should be between 0 and 360 degrees. Negative angle is not allowed. Input a negative angle, one should see the error message.

@robyww
Copy link
Contributor Author

robyww commented Apr 12, 2017

Yi wrote:

Not all the footprints can be rotated by the image rotation

Footprints are being handle in another ticket. see: https://jira.lsstcorp.org/browse/DM-10188

rotation direction now is clockwise but IRSA's convention is counter-clockwise

Will fix this one.

@ymeiymei
Copy link
Contributor

ymeiymei commented Apr 12, 2017

The post-rotation cropping frames are confusing: one dashed and one solid. It seems the cropping is done by the dashed one. Do we need the solid frame? See attached image.

I guess the solid frame is for showing the user's mouse dragging?

screen shot 2017-04-12 at 10 18 42 am

@robyww
Copy link
Contributor Author

robyww commented Apr 12, 2017

The post-rotation cropping frames are confusing: one dashed and one solid. It seems the cropping is done by the dashed one. Do we need the solid frame? See attached image.

I guess the solid frame is for showing the user's mouse dragging?

I am open to suggestions. We need to crop on the original image. Therefore when you make a selection now there are some actions that can happen on the selected area like selecting points but crop cropping need to be in original image bounds.

@ymeiymei
Copy link
Contributor

Post rotation flipping issue:

After rotation, the flip is done on the north-south direction (if the image is north up originally ) or the original up-down direction. A bug?

In IRSA, the flip is alway on the Y axis (up and down direction) before or after rotation.

@robyww
Copy link
Contributor Author

robyww commented Apr 12, 2017

After rotation, the flip is done on the north-south direction (if the image is north up originally ) or the original up-down direction. A bug?

This way is more consistent with DS9. I think firefly was doing it wrong before. I am not sure what others do but I think users will expect it to be more like how DS9 does it.

@ymeiymei
Copy link
Contributor

ymeiymei commented Apr 12, 2017

Before when multiple images are locked as a group, rotation is still done for the selected image only.

This version the rotation is done for the whole group which locked. New feature of bug?
How to repeat: Load images, zoom out as a whole group, then rotate: the whole group will be rotated.

@ymeiymei
Copy link
Contributor

ymeiymei commented Apr 12, 2017

After loading some images, right click and select "Reload", this is for reloading the images (IRSA wise way) or going back to search (the current version)? Even it's for going back to search, the search panel's "Cancel" key is not working.

BTW. The catalog "Reload" key on the coverage map reloads the try-view, instead of going back to search. But after loading some images, this coverage map reloading jumps to search.

@robyww
Copy link
Contributor Author

robyww commented Apr 12, 2017

This version the rotation is done for the whole group which locked. New feature of bug?

The way we did it before has been criticized. We only did it that way because it was so CPU on the server. Now it seems to make sense to keep the group lock like everything else.


Yi comments on Trey's above:
Agree. Then we should do group rotation when group locked and do individual rotation when group unlocked. The feature shouldn't depend on other changes, like zoom first or not.

Copy link
Contributor

@tgoldina tgoldina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code assigned to me. Not sure how useful it is, since I have very limited understanding of what it's doing. In general, I would like to see much more comments about the logic. :)

@@ -99,12 +95,14 @@ const ROTATE_START= `${PLOTS_PREFIX}.RotateChangeStart`;
const ROTATE= `${PLOTS_PREFIX}.RotateChange`;
const ROTATE_FAIL= `${PLOTS_PREFIX}.RotateChangeFail`;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see action creators and reducer behavior for these actions commented, but not actions themselves.
Are we going to use ROTATE_CLIENT and FLIP_CLIENT behavior for ROTATE and FLIP later?

Copy link
Contributor Author

@robyww robyww Apr 13, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will remove the old constants. Maybe I should make the client ones the old ones. I will think about it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I will make the just to be ROTATE and FLIP

//-------------------


//-------------------
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a TODO missing between comment lines?

switch (mouseState) {
case DOWN :
dispatchChangeActivePlotView(plotId);
const {scrollX, scrollY}= this.props.plotView;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this.props.plotView can be plotView, since you are passing it as an argument now

var {width:viewPortWidth,height:viewPortHeight}= plot.viewPort.dim;
const {left,top,scrollViewWidth, scrollViewHeight, scrollX,scrollY, viewDim}= getPositionInfo(plotView);

const {plotId, viewDim:{width,height}}= plotView;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice shortcut for width and height initialization

if (!found) {
found= screenAsDevAry.some( (pt) => contains(0,0,viewDim.width, viewDim.height,pt.x,pt.y) ||
intersects(0,0,viewDim.width, viewDim.height,pt.x,pt.y,1,1));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In functions like this, I wish there would be some human readable explanation of code logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will write a description. I had not because it took me awhile to convince myself the concept worked.





Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this file PlotTransformUtils.js - it would better describe what it's for.

Copy link
Contributor Author

@robyww robyww Apr 13, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea. This file was a work-in-progress for a long time. I will give it a better name.

* @return {Number}
*/
getRotationAngle() { return this.rotationAngle; }
// getRotationAngle() { return this.rotationAngle; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How should can we get the rotation angle now? (I know we rely on PlotState a lot in API.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plotView.rotation and plotView.flipY

@@ -22,7 +22,17 @@ var Point = { SPT, IM_PT, IM_WS_PT, VP_PT, PROJ_PT, W_PT, OFFSET_PT};
*
* @prop {Number} x
* @prop {Number} y
* @prop {String} type one of 'ScreenPt', 'ImagePt', 'ImageWorkSpacePt', 'WorldPt', 'ViewPortPt', 'ProjectionPt', 'OffsetPt'
* @prop {String} type one of 'RenderedPt', 'ScreenPt', 'ImagePt', 'ImageWorkSpacePt', 'WorldPt', 'ViewPortPt', 'ProjectionPt', 'OffsetPt'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ViewPortPt completely gone? It is still referenced here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I will clean up the docs

return testPts.every( (p) => pointInPolygon([p.x,p.y], polyPtsAsArray));
}

export function findIntersectionPt(x1, y1, x2, y2, x3, y3, x4, y4) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need jsdoc or comments here to tell which pairs of (x,y) make first and second segments

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will document it.

});
const plot= primePlot(plotView);
const {x,y}= iPt;
if (x >=0 && y >= 0 && x < plot.dataWidth && y < plot.dataHeight) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this check for? To make sure it's a valid image point?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is is support to do the same as line 140. I meant to fix both. It check to see it the image point is on the plot. It should call pointInPlot

@ymeiymei
Copy link
Contributor

I found sometimes the rotation button has no response when I click it. This happened after rotation, zoom, crop, was/target match, etc.

I can't repeat the problem. See the error messages I got in the attached file.

rotationButtonNoResponseErr.txt

@robyww
Copy link
Contributor Author

robyww commented Apr 13, 2017

I see the place in the code but I am not sure how it could get there. If you could figure our how to repeat is would help. I wonder if you have a series of exceptions and it is really the first exception that is creating this problem.

@ymeiymei
Copy link
Contributor

Sorry I tried but I really can't repeat the problem.

I think I am done with this version's testing. Found some minor issues but I rather to double check them in next version. They are hard to repeat also.

robyww added 2 commits April 13, 2017 15:59
 - the html5 canvas will now rotate the image and flip
 - package.json: added transformation-matrix-js
 - package.json: added point-in-polygon
 - more work in understanding how to rotate
 - moving viewport render calculation to TileDrawer, scroll working now
 - thumbnail working
 - new DevicePt coordinate system
 - removed ViewportPt coordinate system
 - drawing layers mostly working
 - select and selection related operations working with rotation
 - select now has a second box that shows when image is rotated, this is
   used for cropping and image stats
 - rotate dialog, rotate north using new rotation
 - wcs match working
 - target match working
 - time series working
 - compass working
  - fixed rotate north not working in certain cases
  - fixed restore not unflipping and unrotating
  - fixed recentering problem when WCS match is on
  - fixed magnifier: now rotated and flipped
  - fixed round off error that made gaps in the image tiles of safari and firefox
  - fixed rotation issues with wcs match when already rotated
  - fixed rotation match checking false positives
  - png download now works
  - flip icon now is on and off
  - rotation angle/direction now consistent with ds9 and irsa standard
  - scrolling speed optimization using lodash throttle
  - added more documentation on a couple of functions
  - rename PlotPostionUtil to PlotTransformUtils
  - actions are now be ROTATE and FLIP
  - code clean up
@ymeiymei
Copy link
Contributor

Completed the test on this new version. This version fixed a lot. My test report:
1)Rotation and flip can be restored.
2)All rotations are relative to the original position (New feature).
3)Flip is on the original Y axis, even after rotation, no matter where North is. (New feature)
4)Rotation direction is fixed to counter-clockwise now.
5)Post rotation cropping: TBD.
6)A bug(?) : When right click an image and select "Reload", the current page should be reloaded (that is the Chrome reload), but instead, the image search panel pops up. Also this search panel can't be cancelled.

robyww and others added 8 commits April 14, 2017 14:07
…e line with center information in footprint drawlayer UI.

DM-10188 Fix vertextDef setting under the drawing layer for mutiple plots
DM-10188 Fix drawing related issues on markers and footprints with new rotation scheme
Dm-10241 Fix the rendering of east arrow in the North/East compass
@robyww robyww merged commit 04299da into dev Apr 27, 2017
@robyww robyww deleted the dm-10065-canvas branch April 27, 2017 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants