Hosted models perform better than local BYOM ones

[awaescher]

Hosted models do a better job summarizing pages than local BYOM ones. The UI behaves differently too. I guess this has anything to do how the website content is presented to the model internally. This seems to apply for each slash tool, not just summarization.

See the following screenshots for a comparison of the same question towards the same model - the only difference is whether it's a hosted (left) or local BYOM (right) one:

[awaescher]

I guess version1.74 fixed this

[awaescher]

Awesome to see this is fixed, however I noticed that on some pages the local models cannot summarize the website. My best guess is it depends on the website content length:

Most websites I tried won't work with local models

This StackOverflow site can be summarized ...
✅ https://meta.stackoverflow.com/questions/432632/isnt-this-question-a-bad-review-audit

... while this can't:
❌ https://meta.stackoverflow.com/questions/432657/upcoming-experiment-for-commenting

More examples

✅ https://news.ycombinator.com
✅ https://huggingface.co/enterprise
✅ https://blog.codinghorror.com/the-field-of-dreams-strategy/
❌ https://blog.codinghorror.com/stay-gold-america
❌ https://andrewlock.net/creating-a-source-generator-part-1-creating-an-incremental-source-generator

(by the way, I tried increasing the context length in the BYOM settings without success)

---

The issue title is still a good match because hosted models like llama3.1:8b can summarize all the websites shown above even though it may state rightfully that the context is too large: