Skip to content

gptel-context: Allow exclusion of gitignored files #665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
b75a0b8
gptel-rewrite: Exclude gitignored files (first pass)
benthamite Jan 26, 2025
9fbf51a
gptel-context: improve performance by getting list of exclusions once
benthamite Jan 26, 2025
60208d1
gptel-context: improve handling of gitignored files
benthamite Jan 29, 2025
450b1ba
Merge remote-tracking branch 'upstream/master' into exclude-gitignored
benthamite Feb 2, 2025
68fdd0a
Merge branch 'exclude-gitignored'
benthamite Feb 23, 2025
7b2dd6d
Merge remote-tracking branch 'upstream/master' into exclude-gitignored
benthamite Feb 26, 2025
df85051
Merge remote-tracking branch 'upstream/master'
benthamite Feb 26, 2025
c898d46
Merge branch 'master' into exclude-gitignored
benthamite Feb 26, 2025
57c079a
gptel-context: Improve user option docstring
benthamite Feb 26, 2025
2f1f234
gptel-context: Remove caching while preserving performance
benthamite Feb 26, 2025
b69a4c0
README: Document how to exclude gitignored files
benthamite Feb 26, 2025
1b252db
gptel-context: Fix typo
benthamite Feb 26, 2025
7ba8543
Merge branch 'master' into exclude-gitignored
benthamite Mar 10, 2025
a8b67c5
gptel-context: Inform user that git-ignored files can be included
benthamite Mar 12, 2025
834a07e
gptel-context: Replace "unignored" with "git-tracked"
benthamite Mar 12, 2025
ef4a10c
gptel-context: Replace "gitignored" with "git-ignored"
benthamite Mar 12, 2025
20e2ee5
gptel-context: Improve performance of git-ignore exclusion via caching
benthamite Mar 12, 2025
df3a24c
README: Clarify default value of gptel-context-exclude-git-ignored
benthamite Mar 12, 2025
c13ad53
gptel-context: improve git error handling
benthamite Mar 12, 2025
9dfbda9
gptel-context: Restore missing indentation
benthamite Mar 12, 2025
b62d08e
gptel-context: Replace "git-tracked" with "git-unignored" in symbol n…
benthamite Mar 13, 2025
48919bf
gptel-context: Rename gptel-context--skip-file-p
benthamite Mar 13, 2025
b7cd02a
gptel-context: Move relevant defuns to ‘git tracking functions’
benthamite Mar 13, 2025
9cb8cf2
gptel-context: Revise docstrings
benthamite Mar 13, 2025
e2ca876
gptel-context: Check buffer-file-name before passing it as argument
benthamite Mar 13, 2025
9b5a630
gptel-context: Handle possibly nil git-cache argument
benthamite Mar 13, 2025
03cebac
gptel-context: Rename gptel-context--is-git-ignored-p
benthamite Mar 13, 2025
10356a8
gptel-context: Remove needless comment
benthamite Mar 13, 2025
e02f44e
gptel-context: Never exclude git-ignored remote files
benthamite Mar 13, 2025
11e7cf5
gptel-context: Refactor using project.el
benthamite Mar 20, 2025
e9fa7e7
gptel-context: Remove redundancy
benthamite Mar 20, 2025
450cf56
README: Update to reflect project.el refactor
benthamite Mar 20, 2025
1290416
README: Update user option name
benthamite Mar 21, 2025
280a829
gptel-context: pass correct value to gptel-context--get-project-files
benthamite Mar 22, 2025
578fb5f
gptel-context: Simplify gptel-context--message-skipped
benthamite Mar 27, 2025
4b99e1e
gptel-context: Store project files in a local variable
benthamite Apr 7, 2025
c197a74
gptel-context: Reverse hunks with accidentally changed indentation
benthamite Apr 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.org
Original file line number Diff line number Diff line change
Expand Up @@ -1005,6 +1005,7 @@ You can examine the active context from the menu:
And then browse through or remove context from the context buffer:
#+html: <img src="https://github.com/karthink/gptel/assets/8607532/79a5ffe8-3d63-4bf7-9bf6-0457ab61bf2a" align="center" alt="Image showing gptel's context buffer.">

Additionally, you can exclude files or directories in git repositories listed in the =.gitignore= file by setting the user option =gptel-context-exclude-gitignored= to =t=. This can be especially useful when adding entire projects to the context.
*** Handle "reasoning" content

Some LLMs include in their response a "thinking" or "reasoning" block. This text improves the quality of the LLM’s final output, but may not be interesting to you by itself. You can decide how you would like this "reasoning" content to be handled by gptel by setting the user option =gptel-include-reasoning=. You can include it in the LLM response (the default), omit it entirely, include it in the buffer but ignore it on subsequent conversation turns, or redirect it to another buffer. As with most options, you can specify this behvaior from gptel's transient menu globally, buffer-locally or for the next request only.
Expand Down
56 changes: 49 additions & 7 deletions gptel-context.el
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,13 @@ context chunk. This is accessible as, for example:
:group 'gptel
:type 'function)

(defcustom gptel-context-exclude-gitignored t
"Whether to exclude gitignored files from context.
When non-nil, do not add to the context files or directories listed in
the `.gitignore' file of their associated repository."
:group 'gptel
:type 'boolean)

(defun gptel-context-add (&optional arg confirm)
"Add context to gptel in a DWIM fashion.

Expand Down Expand Up @@ -127,8 +134,9 @@ context chunk. This is accessible as, for example:
(mapc action-fn files))))
;; If in an image buffer
((and (derived-mode-p 'image-mode)
(gptel--model-capable-p 'media)
(buffer-file-name))
(gptel--model-capable-p 'media)
(not (gptel-context--skip-file-p (buffer-file-name)))
(buffer-file-name))
(funcall (if (and arg (< (prefix-numeric-value arg) 0))
#'gptel-context-remove
#'gptel-context-add-file)
Expand Down Expand Up @@ -198,9 +206,19 @@ Return PATH if added, nil if ignored."
(defun gptel-context--add-directory (path action)
"Process all files in directory at PATH according to ACTION.
ACTION should be either `add' or `remove'."
(let ((files (directory-files-recursively path "." t)))
(let ((files (directory-files-recursively path "." t))
(git-root (and gptel-context-exclude-gitignored
(executable-find "git")
(locate-dominating-file path ".git")))
unignored-files)
;; Get the list of unignored files once if needed
(when git-root
(setq unignored-files (gptel-context--git-files git-root)))
(mapc (lambda (file)
(unless (file-directory-p file)
(unless (or (file-directory-p file)
(and git-root
(not (member (file-relative-name file git-root)
unignored-files))))
(pcase-exhaustive action
('add
(if (gptel--file-binary-p file)
Expand All @@ -219,14 +237,38 @@ If PATH is a directory, recursively add all files in it.
PATH should be readable as text."
(interactive "fChoose file to add to context: ")
(cond ((file-directory-p path)
(gptel-context--add-directory path 'add))
((gptel--file-binary-p path)
(gptel-context--add-directory path 'add))
((and gptel-context-exclude-gitignored
(gptel-context--skip-file-p path))
(message "Skipping git-ignored file: %s" path))
((gptel--file-binary-p path)
(gptel-context--add-binary-file path))
((gptel-context--add-text-file path))))
(t (gptel-context--add-text-file path))))

;;;###autoload (autoload 'gptel-add-file "gptel-context" "Add files to gptel's context." t)
(defalias 'gptel-add-file #'gptel-context-add-file)

(defun gptel-context--git-files (dir)
"Return a list of unignored files in the Git repo at DIR."
(let ((default-directory dir))
(condition-case err
(process-lines "git" "ls-files" "--cached" "--others" "--exclude-standard")
(error
(message "Error running git ls-files in %s: %S" dir err)
nil))))

(defun gptel-context--skip-file-p (file)
"Return non-nil if FILE should be skipped due to gitignore rules.
This function assumes it will be called with a list of unignored files
available or will make a direct Git call for a single file check."
(when (and gptel-context-exclude-gitignored
(executable-find "git"))
(when-let* ((git-root (locate-dominating-file file ".git"))
(rel-path (file-relative-name file git-root)))
;; Get unignored files directly for this single file
(let ((unignored (gptel-context--git-files git-root)))
(not (member rel-path unignored))))))

(defun gptel-context-remove (&optional context)
"Remove the CONTEXT overlay from the contexts list.

Expand Down