Support updated Spanish dictionary

[cosmoscalibur]

Used setup in project file codebook.toml

dictionaries = ["es_co"]
use_global = false

Spanish words are flagged as incorrect, but english words are recognized as correct (but not all). Also tested with generic variant "es".

Example spanish text:

Same text translated to english, with setup of dictionary in spanish

[blopker]

Hey! Thanks for the issue. Codebook currently only supports the languages defined in https://github.com/blopker/codebook/blob/main/crates/codebook/src/dictionaries/repo.rs. This can be expanded easily. Sadly, I only know English, so that's where I started. I see if I can find a good Spanish dictionary!

[cosmoscalibur]

@blopker , thanks. I think was supported because Wooorm is mentioned (and this includes spanish but an old version).
I am part of maintainer team of RLA-ES, the spanish hunspell dictionary used in LibreOffice and Firefox.

RLA-ES official repo is https://github.com/sbosio/rla-es, and also maintain the spanish version on https://github.com/titoBouzout/Dictionaries (also compile a large collection of dictionaries, but not normalized in name).

[cosmoscalibur]

I added a PR with spanish dict, but when I try to test compiled codebook-lsp and replace in zed work, is not clear if was downloaded, but spanish words are not recognized.

[blopker]

I've raised an issue with Spellbook: helix-editor/spellbook#6
I'll update here when we can get this fixed 😃

[blopker]

I commented in the PR, but the es dictionary is now working. I'm using the dictionary from wooorm because that seems to work on the current version of Spellbook.

[softkot]

Why you did not include all the dictionaries from woom and streetsidesoftware ?

Are you keep getting PRs like this #36 ?

[blopker]

Hey! Sadly, each dictionary source I've tried has various issues which makes adding them more complicated than just adding a link. I started with the dictionaries I use because of time constraints.

It seems likely now that most of the Wooorm dictionaries will work though, they just aren't updated regularly. Much better than nothing, I suppose. However, the streetside ones are all over the place. I've incorporated a few that I needed and was able to test, but adding all of them was just causing more issues.

PRs are always welcome though!

[blopker]

@cosmoscalibur, I just came across https://github.com/LibreOffice/dictionaries/ it seems well formatted, and I could just get Codebook to use all the dictionaries there. Are the Spanish dictionaries kept sufficiently up to date in that repo?

[cosmoscalibur]

@blopker , LO dictionaries are mainteined by RLA-ES project where I am mainteiner. Spanish dictionary was updated to 2.9 in february and has emoji.

[blopker]

Great, some good news. Now that Spellbook can read Unicode in aff files, using the LO repo as a single source is possible.

However, I tested every dictionary in LO and there were a fair number that aren't UTF8 encoded. Not sure why, but those crashed Spellbook.

I'm in the process of adding the ones that do work. I just need to figure out a system I'm happy with...

[the-mikedavis]

Spellbook technically requires UTF-8 dictionaries because it only (currently) accepts &strs for the content of <dict>.dic and <dict>.aff files and Rust strings are UTF-8 encoded. I haven't looked into where codebook is loading dictionaries but if you are not already, you will need to translate any files which are not already in UTF-8 encoding. I have an example of this in the test suite since some of the legacy test dictionaries and files are not UTF-8 encoded: https://github.com/helix-editor/spellbook/blob/134e5d82a75717b035275b1865fce1fa8c8fe580/tests/legacy.rs#L279-L354

Otherwise any incompatibilities (i.e. dictionaries that Hunspell or Nuspell can use successfully but Spellbook can't) should be considered bugs in Spellbook. Currently the parsing is a little bit pessimistic/rigid and can't handle cases like https://github.com/streetsidesoftware/cspell-dicts/blob/d6d4915439d314c78e194c19e17a307aae9fd4da/dictionaries/en_GB/src/hunspell/en_GB.aff#L36-L38 which I considered to be comments for the sake of simplicity - these cases should be fixed