Skip to content

Build worker reboot #204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FeodorFitsner opened this issue Mar 22, 2015 · 12 comments
Closed

Build worker reboot #204

FeodorFitsner opened this issue Mar 22, 2015 · 12 comments
Milestone

Comments

@FeodorFitsner
Copy link
Member

Some Windows software requires server reboot. Build worker service should resume build run after server and build agent restarted. Build session environment variables should be preserved as well.

Test appveyor.yml:

install:
  - set TEST_VAR=Hello, world!
  - echo before reboot
  - ps: Restart-Computer -Force      # here the server goes restarting
  - ps: Start-Sleep -s 10

build_script:
  - echo after reboot
  - echo %TEST_VAR%

test: off
deploy: off
@vors
Copy link

vors commented Apr 8, 2015

I cannot make it work.
Here is the sample:
https://ci.appveyor.com/project/vors/psscriptanalyzer/build/1.0.11

install:
    - set TEST_VAR=Hello, world!
    - echo before reboot
    - ps: Restart-Computer -Force            # here the server goes restarting
    - echo Before sleep
    - ps: |
        Write-Host $env:TEST_VAR
        Write-Host ([datetime]::Now)
        Start-Sleep -s 10
        Write-Host "After sleep"
        Write-Host ([datetime]::Now)

build_script:
    - echo Build
    - ps: |
        Write-Host ([datetime]::Now)
        Write-Host after reboot
        Write-Host $env:TEST_VAR

test: off
deploy: off

According to log reboot happens in just 1 second.

@FeodorFitsner
Copy link
Member Author

Why do you think it wasn't rebooted? ;)

There is no "After sleep" in a build log and reboot happened in 1 minute, not second.

  • Feodor

On Tue, Apr 7, 2015 at 7:39 PM, Sergei Vorobev [email protected]
wrote:

I cannot make it work.
Here is the sample:
https://ci.appveyor.com/project/vors/psscriptanalyzer/build/1.0.11

install:
    - set TEST_VAR=Hello, world!
    - echo before reboot
    - ps: Restart-Computer -Force            # here the server goes restarting
    - echo Before sleep
    - ps: |
        Write-Host $env:TEST_VAR
        Write-Host ([datetime]::Now)
        Start-Sleep -s 10
        Write-Host "After sleep"
        Write-Host ([datetime]::Now)
build_script:
    - echo Build
    - ps: |
        Write-Host ([datetime]::Now)
        Write-Host after reboot
        Write-Host $env:TEST_VAR
test: off
deploy: off

According to log reboot happens in just 1 second.

Reply to this email directly or view it on GitHub:
#204 (comment)

@vors
Copy link

vors commented Apr 8, 2015

Indeed, sorry was tired and didn't recognize it's minutes not seconds.

@dhoer
Copy link

dhoer commented Sep 22, 2016

@FeodorFitsner My build script already scheduled a restart, so I get this exception:

https://ci.appveyor.com/project/dhoer/chef-selenium/build/master-75

Restart-Computer : Failed to restart the computer APPVYR-WIN with the following error message: A system shutdown has already been scheduled

When I only do a sleep, it hangs and eventually fails. I can't disable the reboot in the code either since it is required. Is there anyway that I can wait for the reboot to happen before running my tests given these constraints?

https://github.com/dhoer/chef-selenium/blob/master/appveyor.yml

@FeodorFitsner
Copy link
Member Author

What do you mean by build script already scheduled a restart? Is it some software that was installed and pending restart?

@dhoer
Copy link

dhoer commented Sep 22, 2016

Chef is installing software and schedules a reboot to occur 60 seconds after it completes provisioning. The server needs to reboot in order for paths set to be available and to run a script on startup.

@IlyaFinkelshteyn
Copy link
Contributor

IlyaFinkelshteyn commented Sep 22, 2016

If you feel that you should not wait for Chef to restart, you can run shutdown /a to abort their shutdown before you run Restart-Computer -Force, or (looks safer) try to restart with Restart-Computer -Force -ErrorAction SilentlyContinue and if fail no error will rise and then wait for Chef's restart to happen.

@dhoer
Copy link

dhoer commented Sep 22, 2016

@IlyaFinkelshteyn Thanks for the help. The later still threw an exception and the former hangs and there is no output of shutdown /a or Restart-Computer commands. I see you forked this and tested it. Any other suggestions? This works when running against vagrant box. This is doing an auto-login to user rdp_local on restart. Maybe this interferes with appveyor somehow.

@IlyaFinkelshteyn
Copy link
Contributor

Hi Dennis,

shutdown /a actually does the trick. And even simple Restart-Computer without -force (weird ;)) also does it. Problem is after reboot it stuck with what was set to auto-start.

It was set here https://ci.appveyor.com/project/IlyaFinkelshteyn/chef-selenium/build/master-10#L852

And when I go directly to host machine and connect to console 0 of VM, I see that reboot happened (event viewer messages) and the following picture:

On VM I see command which starts RDP to itself and RDP session to itself (localhost), stuck forever with the following command
C:\java\bin\java.exe" -jar "C:/selenium/server/selenium-server-standalone.jar" -role node -nodeConfig "C:/selenium/config/selenium_node.json" -log "C:/selenium/log/selenium_node.log

Does it work if you run it on your machine locally?

--ilya.

@dhoer
Copy link

dhoer commented Sep 23, 2016

Yes, it works on vagrant locally. We have used this cookbook in prod for the past few years. I do have 11% failure rate in production when spinning up 45 windows node servers - probably due to race condition when registering node to hub - but have never had a failure running locally. The RDP is a work around suggested by AWS to set different screen resolutions in the cloud.

Selenium and its dependencies (drivers and browsers) change often and it would be nice to have integration tests verify all is good.

@dhoer
Copy link

dhoer commented Sep 23, 2016

@IlyaFinkelshteyn BTW, thanks for looking at this. I will dig deeper into why the node is hanging when I have some time.

@IlyaFinkelshteyn
Copy link
Contributor

IlyaFinkelshteyn commented Sep 23, 2016

You are welcome!

I think in Appveyor infrastructure you don’t need to do RDP tricks to adjust screen resolution. Please use this https://github.com/FeodorFitsner/selenium-tests/blob/screen-resolution/appveyor.yml approach.

Than just use Start-process instead of RDP to start Selenium.

--ilya.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants