Skip to content

GH-56 Open sqlite db in readonly mode to avoid changes to .shm and .wal files #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 1, 2025

Conversation

zahra-abedi
Copy link
Contributor

@zahra-abedi zahra-abedi commented Apr 29, 2025

Issue #, if available: #56

Description of changes:
When using the library to query the RPM database from rpmdb.sqlite, the library appears to remove the .shm (shared memory) and .wal (write-ahead log) files from the /var/lib/rpm directory. This behavior causes subsequent rpm commands (e.g., rpm -qa) to fail with errors until the rpm command is run with sudo privileges, which recreates the missing files.

This PR opens the db in readonly mode to avoid this issue!

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Verification

Before this change:

$ rpm -qa | wc -l
442
$ sudo ./rpmdb-master | wc -l
444
$ rpm -qa | wc -l
error: sqlite failure: CREATE TABLE IF NOT EXISTS 'Packages' (hnum INTEGER PRIMARY KEY AUTOINCREMENT,blob BLOB NOT NULL): attempt to write a readonly database
error: cannot open Packages index using sqlite - No such file or directory (2)
error: cannot open Packages database in /var/lib/rpm
0

After this change:

$ rpm -qa | wc -l
442
]$ sudo ./rpmdb-sqlitefix | wc -l
444
$ rpm -qa | wc -l
442

@zahra-abedi zahra-abedi force-pushed the fix/sqlite_readonly branch from 96c0705 to 01d56cc Compare April 29, 2025 16:06
Copy link
Owner

@knqyf263 knqyf263 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution.

@@ -36,7 +37,8 @@ func Open(path string) (*SQLite3, error) {
return nil, ErrorInvalidSQLite3
}

db, err := sql.Open("sqlite", path)
// open sqlite3 database in read-only mode
db, err := sql.Open("sqlite", fmt.Sprintf("file:%s?mode=ro&immutable=1&_mutex=no", path))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_mutex is an extended parameter available only in github.com/mattn/go-sqlite3, right? Or is it a standard param?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_mutex is an extended parameter specific to github.com/mattn/go-sqlite3. It is not a standard SQLite parameter. This parameter is used to control the mutex mode for SQLite connections, such as enabling or disabling thread safety. If a different SQLite driver is used, _mutex may not be supported. would you like me to remove it?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to remove it until we need it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it in this force-push.

Here are some more info on this that I found. The library sets the mutex to SQLITE_OPEN_FULLMUTEX by default.
https://www.sqlite.org/c3ref/open.html
Maybe there could be a parameter added to the sqlite3 Open method that defaults to SQLITE_OPEN_FULLMUTEX and makes mutex configurable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@knqyf263 the PR is ready for your review!

Ensure the SQLite database is opened in read-only mode to prevent
modifications to `.shm` and `.wal` files. This change is particularly
important when the code is executed with elevated privileges,
as it avoids unintended changes to the database state.
@zahra-abedi zahra-abedi force-pushed the fix/sqlite_readonly branch from 01d56cc to aacf192 Compare April 30, 2025 15:08
@zahra-abedi zahra-abedi requested a review from knqyf263 April 30, 2025 17:13
Copy link
Owner

@knqyf263 knqyf263 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@knqyf263 knqyf263 merged commit e8ca378 into knqyf263:master May 1, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants