[maskedtensor] Adagrad sparse semantics [3/4]

[george-qi]

Restructured the PR stack from #2042

The new format will be:

PR #1: Overview Tutorial

• Old overview
• Why is MaskedTensor useful
  • Distinguish 0 and NaN gradient
  • Safe softmax tutorial
  • Missing nan ops

PR #2: Sparsity

PR #3: Adagrad sparse semantics, i.e. what does MaskedTensor make easier -- this one

PR #4: Advanced semantics, e.g. NumPy differences and reduction semantics from Old overview

[netlify]

✅ Deploy Preview for pytorch-tutorials-preview ready!

Name	Link
🔨 Latest commit	f5b4021
🔍 Latest deploy log	https://app.netlify.com/sites/pytorch-tutorials-preview/deploys/635c399be9a4b50008dbe61b
😎 Deploy Preview	https://deploy-preview-2052--pytorch-tutorials-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[janeyx99]

Generally, I got the point of the tutorial eventually, so nice! I do believe there are some opportunities to make the tutorial clearer, including:

• highlighting/emphasizing points across the tutorial (see my other comments)
• making the intro less technical --> it may be interesting actually to swap the intro and parts of the conclusion. The conclusion does a better job connecting to the new reader by making the general point that MaskedTensor simplifies code by disentangling the concepts of "sparsity semantics" and "sparsity optimization". The intro currently includes some technical details that do not make it easy for a new reader to immediately know what's up (the one-off implementation details, starting off with an Issue). These details become clearer once someone has gone through the code comparisons, so maybe can go at the end.

[george-qi]

I like the comment on switching the conclusion to the front a lot -- I've implemented that :)

[janeyx99]

Overall good! Just a few more minor clarifications; the general sense is conveyed.

I haven't verified that the links all work (the preview's not done yet), but spot checking a few seem good.