Skip to content

feat(generation): add backend support for Groq #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 25, 2025

Conversation

chiruu12
Copy link
Contributor

@chiruu12 chiruu12 commented Mar 18, 2025

Pull Request description

This PR integrates Groq inference capabilities into Rago by implementing a new GroqGen class. This enhancement allows Rago to utilize Groq's OpenAI-compatible API for text generation tasks, As Groq's LPU's are designed to give exeptional results for LLM's making it about 10 times faster than some services and also at lower costs.

Addition of GroqGen Class: Implements the GroqGen class to utilize Groq's OpenAI-compatible API for text generation tasks.

How to test these changes

Tests have been added for the same.

Pull Request checklists

This PR is a:

  • bug-fix
  • new feature
  • maintenance

About this PR:

  • it includes tests.
  • the tests are executed on CI.
  • the tests generate log file(s) (path).
  • pre-commit hooks were executed locally.
  • this PR requires a project documentation update.

Author's checklist:

  • I have reviewed the changes and it contains no misspelling.
  • The code is well commented, especially in the parts that contain more
    complexity.
  • New and old tests passed locally.

Reviewer's Checklist

  • I managed to reproduce the problem locally from the main branch
  • I managed to test the new changes locally
  • I confirm that the issues mentioned were fixed/resolved .

@chiruu12
Copy link
Contributor Author

@xmnlab Please review!!

Copy link

This pull request has been marked as stale because it has been
inactive for more than 5 days. Please update this pull request
or it will be automatically closed in 5 days.

@github-actions github-actions bot added the stale label Mar 24, 2025
Copy link
Member

@xmnlab xmnlab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on that, @chiruu12

@xmnlab xmnlab merged commit 3cda11b into osl-incubator:main Mar 25, 2025
12 checks passed
Copy link

🎉 This PR is included in version 0.14.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants