Skip to content

Improve and migrate model_surgery guide #2668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 11, 2022
Merged

Conversation

IvyZX
Copy link
Collaborator

@IvyZX IvyZX commented Nov 29, 2022

This PR does the following:

  • Add a bunch more texts explaining why and how to do param and opt state tweaking to readers with little context of Flax terminologies.
  • Explained the FrozenDict a bit more.
  • Make the guide notebook-based, so that it's trivial to open a colab and test the code.

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@gnecula
Copy link
Collaborator

gnecula commented Nov 29, 2022

It is a good idea to transition docs from .rst to Colab+md files. Thanks!

@8bitmp3
Copy link
Collaborator

8bitmp3 commented Nov 30, 2022

It is a good idea to transition docs from .rst to Colab+md files. Thanks!

  • 1 @gnecula We have some docs that use tables with before/after code. Unless we find a Markdown-way of implementing this without any (or too many) (external) Markdown "themes" for Sphinx, we can keep them for now. There may also be a way of doing this in HTML embedded in Markdown 🤔 (we do this in TensorFlow . org)

@IvyZX Thanks for the PR. Will review asap.

@8bitmp3
Copy link
Collaborator

8bitmp3 commented Nov 30, 2022

It is a good idea to transition docs from .rst to Colab+md files. Thanks!

(Note: In some Markdown flavors, code diff highlighting can be done by adding "diff" (similar to adding "python" if you want the code to have python syntax colors). Will investigate if this works in Sphinx for ReadTheDocs. cc @cgarciae You can use it in GitHub comments that support Markdown (during reviews). Reference: https://stackoverflow.com/questions/40883421/diff-syntax-highlighting-in-github-markdown )

@marcvanzee
Copy link
Collaborator

In my opinion, the Github diff highlighting is quite a lot worse than what we have now. We currently have a side-by-side view, which I think is very effective to demonstrate the effect of a change (see e.g. @cgarciae's BatchNorm guide). The diff view with green and red is not so easy to read, I think.

I think these side-by-side views do have a benefit so I would be in favor of keeping them as .rst for now. I do agree that if we don't use these side-by-side views in a doc we should switch to notebooks.

@8bitmp3
Copy link
Collaborator

8bitmp3 commented Nov 30, 2022

side-by-side views do have a benefit so I would be in favor of keeping them as .rst for now.

+1 If there's no Markdown-flavored way of doing this, we can keep those RST docs that use that view.

@IvyZX
Copy link
Collaborator Author

IvyZX commented Nov 30, 2022

I agree that the side-by-side views look pretty convenient and we should keep them as .rst, until we find a good alternative. The doc on this PR doesn't have the side-by-side views so it could be safely converted.

@8bitmp3
Copy link
Collaborator

8bitmp3 commented Dec 1, 2022

After @cgarciae and @levskaya are finished with their reviews, I will add my final suggestions.

@chiamp
Copy link
Collaborator

chiamp commented Dec 2, 2022

I think the FrozenDict explanation is very helpful, thanks for adding it!

@copybara-service copybara-service bot merged commit 12f2b27 into google:main Dec 11, 2022
@IvyZX IvyZX deleted the flax-pen1 branch December 12, 2022 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants