Skip to content

Navigation Menu

Appearance settings

Explore
By company size
By use case
By industry
View all solutions
Topics
- AI
- DevOps
- Security
- Software Development
- View all
Explore
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

ecmwf / WeatherGenerator Public

Notifications You must be signed in to change notification settings
Fork 22
Star 57

Code
Issues 107
Pull requests 9
Actions
Projects 1
Wiki
Security
Insights

Additional navigation options

Code
Issues
Pull requests
Actions
Projects
Wiki
Security
Insights

Introduce masking class and incorporate in TokenizerMasking#383

Merged

clessig merged 29 commits intodevelopecmwf/WeatherGenerator:developfrom

shmh40/dev/masking_classecmwf/WeatherGenerator:shmh40/dev/masking_classCopy head branch name to clipboard

Jun 28, 2025

Conversation Commits29 (29)Checks Files changed

Merged

Introduce masking class and incorporate in TokenizerMasking#383
clessig merged 29 commits intodevelopfrom
shmh40/dev/masking_class

Commits

Commits on Jun 24, 2025

creating masking class and adapting tokenizer_masking to use this class
shmh40
committed
minor changes to masking.py and tokenizer_masking
shmh40
committed
removed old tokenizer_masking
shmh40
committed
include masking_strategy in default_config
shmh40
committed
change ValueError to assert
shmh40
committed
linting formatting changes files
shmh40
committed
further linting of docstrings
shmh40
committed
create mask_source and mask_target in Masker, and update tokenizer_masking to use these, then style improvements
shmh40
committed
linted masking, tokenizer_masking
shmh40
committed
Merge branch 'develop' into shmh40/dev/masking_class
shmh40
authored

Commits on Jun 25, 2025

Merge remote-tracking branch 'origin/develop' into shmh40/dev/masking_class
shmh40
committed
modify masker, rng and perm_sel now part of class, remove extra masking_rate, update comments, remove archived class
shmh40
committed
remove check if all masked, not masked
shmh40
committed
remove self.masking_rate from MultiStreamDS class, and masking args from batchify_source
shmh40
committed
update tokenizer utils with description of idx_ord_lens in comment
shmh40
committed
remove masking args from batchify_, perm_sel removed now internal to Masker class, remove handling special cases of masking (all masked)
shmh40
committed

Commits on Jun 27, 2025

adding masking_strategy: to config
shmh40
committed
Merge remote-tracking branch 'origin/develop' into shmh40/dev/masking_class
shmh40
committed
remove unused mentions of masking_combination
shmh40
committed
removed comment about streams
shmh40
committed
changed assert to check self perm_sel is not None
shmh40
committed
ruff masking, tokenizer_masking
shmh40
committed

Commits on Jun 28, 2025

Ruffed
clessig
committed
Added warning to capture corner case, likely due to incorrect user settings.
clessig
committed
Fixed incorrect call twice
clessig
committed
Fixed missing conditional for logger statement
clessig
committed
Required changes for better handling of rngs
clessig
committed
Improved handling of rngs
clessig
committed
Improved handling of rng
clessig
committed

Footer

© 2025 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.