Skip to content

IRSA-689: Time Series Viewer Prepare Download doesn’t send to Background Monitor #443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 18, 2017

Conversation

loitly
Copy link
Contributor

@loitly loitly commented Aug 17, 2017

https://caltech-ipac.atlassian.net/browse/IRSA-689

To test:

wise should work. ptf fail for me. I got an unhandled exception.
But, both were added to background monitor as it should be.
@ejoliet, you should have someone look at the exception below. It should not happen.

java.lang.ArrayIndexOutOfBoundsException: 0
	at edu.caltech.ipac.firefly.server.query.lc.PtfFileGroupsProcessor.computeFileGroup(PtfFileGroupsProcessor.java:87)
	at edu.caltech.ipac.firefly.server.query.lc.PtfFileGroupsProcessor.loadData(PtfFileGroupsProcessor.java:41)
	at edu.caltech.ipac.firefly.server.query.FileGroupsProcessor.getData(FileGroupsProcessor.java:48)
	at edu.caltech.ipac.firefly.server.query.FileGroupsProcessor.getData(FileGroupsProcessor.java:33)
	at edu.caltech.ipac.firefly.server.packagedata.PackageMaster$PackagingWorker.work(PackageMaster.java:229)
	at edu.caltech.ipac.firefly.server.query.BackgroundEnv$BackgroundProcessor.run(BackgroundEnv.java:532)
	at java.lang.Thread.run(Thread.java:745)

@loitly loitly requested a review from ejoliet August 17, 2017 04:16
Copy link
Contributor

@ejoliet ejoliet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I've fixed the Exception problem you mentioned, I've tried to download the first page with that fix - see commit pushed in same branch - but i realized that:

  • selecting all means, all table, not only the one i see in this page (there are 6 pages if using the table attache din ticket)
  • the packaging took quite some time (3mins?!) but it says 750Mb (might be random log value?) in the log but actually was little more than 160Mb.

See log here:

08/17 14:54:19 DEBUG   Thread: Packager-bid3006754855_anakin  packagedata.PackageMaster#logPID:106
        Package ID: bid3006754855_anakin
Packager thread: Packager-bid3006754855_anakin is beginning
08/17 14:54:19  INFO   Thread: background-processor-bid3006754855_anakin  packagedata.PackagingController#logStartStatus:172
        PackagingController: Queue Status Report
1 new thread started
------- Started (large): 
  - Total large package (750.0 MB) threads: 1 of 3 allowed
  - Total running threads:                  1 of 5 allowed
  - Longest current wait time:      13 ms
  - Total packaged since beginning: 1
  - Total packaged in background:   0
  - Total immediate packaged:       1
  - Queue high water mark:          1
  - Total large waiting:            0
  - Queue size:                     0
08/17 14:54:19 DEBUG   Thread: background-processor-bid3006754855_anakin  packagedata.PackageMaster#logPID:106
        Package ID: bid3006754855_anakin
package queued in background
Package Report: bid3006754855_anakin, state: WORKING, msg cnt: 0, sizeInBytes: 1231028224
PackagedBundle cnt: 1
            #0 -    files: 587
08/17 14:56:48  INFO   Thread: Packager-bid3006754855_anakin  packagedata.ZipHandler#finishZip:400
        FINISHING ZIP: /hydra/workarea/irsaviewer/stage/bid3006754855_anakin_0.zip
returning url:http://localhost:8080/irsaviewer/servlet/Download?file=%24%7Bstage%7D%2Fbid3006754855_anakin_0.zip&return=PTF_Files.zip&log=true&track=true
08/17 14:57:03  INFO   Thread: Packager-bid3006754855_anakin  packagedata.ZipHandler#finishZip:400
        FINISHING ZIP: /hydra/workarea/irsaviewer/stage/bid3006754855_anakin_1.zip
returning url:http://localhost:8080/irsaviewer/servlet/Download?file=%24%7Bstage%7D%2Fbid3006754855_anakin_1.zip&return=PTF_Files-part2.zip&log=true&track=true
08/17 14:57:03 DEBUG   Thread: Packager-bid3006754855_anakin  packagedata.PackageMaster#logPID:106
        Package ID: bid3006754855_anakin
Packager thread: Packager-bid3006754855_anakin  has completed
08/17 14:57:03  INFO   Thread: Packager-bid3006754855_anakin  packagedata.PackagingController#updateStatistics:322
        Package ID: bid3006754855_anakin
Packaging completed:  zTime= 00:02:43.576, wTime= 00:02:44, size= 1174.0 MB
08/17 14:57:03  INFO   Thread: Packager-bid3006754855_anakin  packagedata.PackagingController#logStartStatus:172
        PackagingController: Queue Status Report
queue empty, 0 new threads started
  - Total large package (750.0 MB) threads: 0 of 3 allowed
  - Total running threads:                  0 of 5 allowed
  - Longest current wait time:      0 ms
  - Total packaged since beginning: 2
  - Total packaged in background:   1
  - Total immediate packaged:       1
  - Queue high water mark:          1
  - Total large waiting:            0
  - Queue size:                     0

Does it make sense? Do you see any more problem? If you feel that this can be improved or any side effect, please, raise separate ticket.

Otherwise, feel free to merge because you solved the problem.
Thanks!

@ejoliet ejoliet merged commit a050a1b into rc Aug 18, 2017
@ejoliet ejoliet deleted the IRSA-689_timeseries_prepare_download branch August 18, 2017 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants