Skip to content

Unable to use Vagrant driver behind corporate http proxy #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ndobson opened this issue Jun 4, 2014 · 14 comments · Fixed by #162
Closed

Unable to use Vagrant driver behind corporate http proxy #74

ndobson opened this issue Jun 4, 2014 · 14 comments · Fixed by #162
Labels
Type: Bug Doesn't work as expected.
Milestone
1.0

Comments

@ndobson
Copy link

ndobson commented Jun 4, 2014

My http_proxy env vars are set properly but unless I'm missing some Vagrant driver specific config, I'm unable to setup a basic vagrant machine. It fails on getting the machine metadata. Please see the stacktrace below:

[2014-06-03T23:09:44-05:00] INFO: Executing sudo sh /tmp/detect.sh on [email protected]
PLATFORM: el
PLATFORM_VERSION: 6
MACHINE: x86_64
[2014-06-03T23:09:44-05:00] INFO: Completed sh /tmp/detect.sh on [email protected]: exit status 0
[2014-06-03T23:09:44-05:00] DEBUG: Getting metadata for machine jenkins: https://www.opscode.com/chef/metadata?v=&prerelease=false&p=el&pv=6&m=x86_64
[2014-06-03T23:09:44-05:00] DEBUG: Closing SSH session on [email protected]

================================================================================
Error executing action `converge` on resource 'machine_batch[default]'
================================================================================


SocketError
-----------
getaddrinfo: nodename nor servname provided, or not known


Resource Declaration:
---------------------
# In /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/recipe_dsl.rb

 81:             batch = build_resource(:machine_batch, machine_batch_default_name) do
 82:               action resource.action
 83:               machines [ previous, resource ]
 84:             end
 85:             batch.from_recipe self



Compiled Resource:
------------------
# Declared in /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/recipe_dsl.rb:81:in `machine'

machine_batch("default") do
  action [:converge]
  retries 0
  retry_delay 2
  guard_interpreter :default
  driver "vagrant:/Users/nick/machinetest"
  chef_server {:chef_server_url=>"http://127.0.0.1:8889", :options=>{:client_name=>"212406206", :signing_key_filename=>"/Users/nick/archops/.chef/212406206.pem"}}
  machine_options {:vagrant_options=>{"vm.box"=>"opscode-centos-6.4"}}
  cookbook_name "test"
  recipe_name "pipeline"
  machines ["jenkins", "docker"]
end



[2014-06-03T23:09:44-05:00] INFO: Running queued delayed notifications before re-raising exception
[2014-06-03T23:09:44-05:00] DEBUG: Re-raising exception: SocketError - machine_batch[default] (test::pipeline line 81) had an error: SocketError: getaddrinfo: nodename nor servname provided, or not known
/usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:763:in `initialize'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:763:in `open'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:763:in `block in connect'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/timeout.rb:55:in `timeout'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/timeout.rb:100:in `timeout'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:763:in `connect'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:756:in `do_start'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:745:in `start'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/1.9.1/net/http.rb:1285:in `request'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/rest-client-1.6.7/lib/restclient/net_http_ext.rb:51:in `request'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/convergence_strategy/install_cached.rb:124:in `download_metadata_for_platform'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/convergence_strategy/install_cached.rb:82:in `block in download_package_for_platform'
  <internal:prelude>:10:in `synchronize'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/convergence_strategy/install_cached.rb:77:in `download_package_for_platform'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/convergence_strategy/install_cached.rb:46:in `setup_convergence'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef_metal/machine/basic_machine.rb:16:in `setup_convergence'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:50:in `block (2 levels) in <class:MachineBatch>'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:89:in `block (2 levels) in with_ready_machines'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-vagrant-0.4.beta.2/lib/chef_metal_vagrant/vagrant_driver.rb:126:in `block in ready_machines'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-vagrant-0.4.beta.2/lib/chef_metal_vagrant/vagrant_driver.rb:123:in `each_pair'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-vagrant-0.4.beta.2/lib/chef_metal_vagrant/vagrant_driver.rb:123:in `ready_machines'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:82:in `block in with_ready_machines'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/chef_fs/parallelizer.rb:95:in `call'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/chef_fs/parallelizer.rb:95:in `process_input'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/chef_fs/parallelizer.rb:70:in `each'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:101:in `to_a'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:101:in `parallel_do'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:81:in `with_ready_machines'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-metal-0.11.beta.11/lib/chef/provider/machine_batch.rb:48:in `block in <class:MachineBatch>'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/lwrp_base.rb:60:in `instance_eval'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/lwrp_base.rb:60:in `recipe_eval_with_update_check'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider/lwrp_base.rb:45:in `block in action'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/provider.rb:120:in `run_action'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource.rb:637:in `run_action'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:49:in `run_action'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block (2 levels) in converge'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `each'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:81:in `block in converge'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/resource_collection.rb:96:in `execute_each_resource'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/runner.rb:80:in `converge'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:345:in `converge'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:431:in `do_run'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:213:in `block in run'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `fork'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/client.rb:207:in `run'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:217:in `run_chef_client'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:328:in `block in run_application'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `loop'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application/client.rb:317:in `run_application'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/lib/chef/application.rb:67:in `run'
  /usr/local/var/rbenv/versions/1.9.3-p545/lib/ruby/gems/1.9.1/gems/chef-11.12.4/bin/chef-client:26:in `<top (required)>'
  /usr/local/var/rbenv/versions/1.9.3-p545/bin/chef-client:23:in `load'
  /usr/local/var/rbenv/versions/1.9.3-p545/bin/chef-client:23:in `<main>'
@jkeiser
Copy link
Contributor

jkeiser commented Jun 4, 2014

Can you show me more about your http_proxy vars? I'm sure it's options I'm missing.

@ndobson
Copy link
Author

ndobson commented Jun 4, 2014

Sure, these are set in my env:

http_proxy=http://*.*.*.*:3128
ftp_proxy=http://*.*.*.*:3128
https_proxy=http://*.*.*.*:3128
no_proxy=*.ge.com,192.168.1.1,127.0.0.1

And this is in my knife.rb:

http_proxy              ENV['http_proxy']
https_proxy             ENV['https_proxy']
no_proxy                ENV['no_proxy']
bootstrap_proxy         ENV['http_proxy']
bootstrap_no_proxy      "*.ge.com"

I know it's working to some degree because without then no_proxy 127.0.0.1 I'm unable to connect to chef-zero and it bombs out way earlier.
This is the command im running:

chef-client -z -o test::vagrant,test::linux,test::pipeline -l debug

And these are my recipes:

require 'chef_metal_vagrant'
#include_recipe 'ge_proxy'

# Set up a vagrant cluster (place for vms) in ~/machinetest
vagrant_cluster "#{ENV['HOME']}/machinetest"

vagrant_box 'opscode-centos-6.4' do
end

machine 'jenkins' do
  recipe 'test::default'
  tag 'jenkins_master'
  #action [:deestroy, :create]
end

machine 'docker' do
  recipe 'test::default'
  tag 'docker_slave'
  #action [:destroy, :create]
end

@jkeiser jkeiser added the bug label Jun 4, 2014
@jkeiser jkeiser added this to the 1.0 milestone Aug 19, 2014
@johnewart
Copy link
Contributor

@ndobson From the information it looks like you're running chef-zero with vagrant all on the same machine. If that's the case, what are the proxy settings for?

@jkeiser
Copy link
Contributor

jkeiser commented Sep 9, 2014

I am confused by the http_proxy setting with the stars ... the http_proxy variable in chef metal fog appears to be looking for the actual IP of the proxy server. Do you have a variable that indicates that? Setting http_proxy to that in knife.rb might be the right thing.

@ndobson
Copy link
Author

ndobson commented Sep 9, 2014

@jkeiser I substituted the stars for the actual ip address I'm using cause i didn't want to post that here :-)

@jkeiser
Copy link
Contributor

jkeiser commented Sep 9, 2014

Oh! Ha, ok

@jkeiser
Copy link
Contributor

jkeiser commented Sep 9, 2014

I think the issue is the http at the front. For better or worse, the person who added gateway support assumed the scheme would not be on there. Is that a standard format?

You can see the documentation I happened to add just today at https://github.com/opscode/chef-metal/blob/master/lib/chef_metal/transport/ssh.rb . If your format is a standard let's get a patch in there to recognize both!

Regardless, you might want to try to try removing the scheme and the colon slash slash just to see if it works.

ndobson referenced this issue in ndobson/chef-metal Sep 9, 2014
Added proxy support for metadata download
f2f9512
@ndobson ndobson mentioned this issue Sep 9, 2014
Merged
@ndobson
Copy link
Author

ndobson commented Sep 9, 2014

@jkeiser I don't think this issue has to do with ssh gateways, please see my pull request which handles proxying for the metadata download and fixes my issue.

@jkeiser
Copy link
Contributor

jkeiser commented Sep 9, 2014

@ndobson odd--it still looks like https://github.com/opscode/chef-metal/blob/master/lib/chef_metal/transport/ssh.rb#L247-249 won't parse your proxy right though ...

@ndobson
Copy link
Author

ndobson commented Sep 9, 2014

@jkeiser I'm a little confused by your comment, I'm using the vagrant driver connecting to localhost so I don't need an ssh gateway to connect to my cluster nodes. I only need an http_proxy to connect out to the internet.

@jkeiser
Copy link
Contributor

jkeiser commented Sep 9, 2014

ah, so this is about configuring chef so it can download the client?

I'd still like to know if yours is a kind of standard format for proxy so we can support it in gateways, but thanks for sorting me out on your context :)

@johnewart
Copy link
Contributor

You're using Ruby 1.9.3 which doesn't automatically honor the http_proxy environment variable, 2.0+ will automatically use that environment variable when making Net::HTTP calls

So, this fix will make 1.9.3 work - another fix is to use 2.0+

@ndobson
Copy link
Author

ndobson commented Sep 9, 2014

Yeah that is definitely standard format for defining an http_proxy https://docs.getchef.com/config_rb_client.html

@johnewart Ahhhhh yeah that makes sense, I saw in the Net:HTTP doc that it's supposed to automatically use it but I was wondering why that wasn't happening, thanks!

Either way merging this pull request shouldn't break anything, it'll just fix functionality for ruby 1.9.3.

@johnewart
Copy link
Contributor

I agree that it won't break anything, but it potentially means we have proxy support in one place in 1.9 but not in others so it might make sense to throw together a quick wrapper for reuse and be thorough. Also, any idea if this will this support an HTTPS proxy in 1.9?

ndobson referenced this issue in ndobson/chef-metal Sep 9, 2014
Use Chef::HTTP instead of Net::HTTP
ed84166 
Fixes proxy support
@tas50 tas50 added Type: Bug Doesn't work as expected. and removed Bug labels Jul 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Bug Doesn't work as expected.
Milestone
1.0
Development

Successfully merging a pull request may close this issue.

Added proxy support for metadata download ndobson/chef-metal
4 participants
@johnewart @jkeiser @ndobson @tas50