Implement Adam optimizer

[TH3CHARLie]

fixes #288

Summary:
Implement Adam optimizer, follow code style similar to SGD and RMSProp

[TH3CHARLie]

Current test fails at

ClassyVision/test/generic/optim_test_util.py

Lines 80 to 85 in a62699b

	self.assertTrue(
	torch.allclose(
	optim1["state"][id1]["momentum_buffer"],
	optim2["state"][id2]["momentum_buffer"],
	)
	)

because "momentum_buffer" raises KeyError

I'm not pretty sure about this but I think it is due to no momentum in Adam, if so, then maybe the test itself needs modification

[aadcock]

@TH3CHARLie , thanks for submitting the PR! Yeah, I think you are correct, the test likely needs to be updated.

[mannatsingh]

Thanks for working on this TH3CHARLie!

You're right, Adam doesn't have a momentum buffer. Can you add a _check_momentum_buffer to TestOptimizer and return True in the base class? TestAdamOptimizer can return False instead and we can use this to determine if the check should happen or not.

[facebook-github-bot]

@vreis has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@mannatsingh has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@mannatsingh merged this pull request in 3d3fc1d.