DB caching in internal storage

[georgekav2]

Is your feature request related to a problem? Please describe.
I strongly agree with your position of scope limiting the KeepassDX to file editor status, instead of a do-it-all utility. Better testable code, limited attack surface, and not need to make online and offline variants.

However, I observed a limitation. If we use any LAN-only hosting of database (local sftp, samba, webdav etc). When we are in the same LAN files are opening without any issue. However, in remote operation the app correctly returns IO error, as the File manager is unable to find the LAN resource.

Describe the solution you'd like
There a potential benefit of caching the last opened DB in the internal storage that the app can load if the remote is not accessible.

Describe alternatives you've considered
The only real alternative are either hosting the DB in cloud or syncing via such tools the DB inside the device.

[da2x]

Surely, this is a job for a file manager or file syncing agent, and not KeePassDX. It just introduces more points for potential file conflicts. Syncthing can handle this for you and sync the files back up again when you get back on your LAN. (It can also sync over the internet.)

[georgekav2]

From one point of you yes, from another if you think about it, any remote document API usability is invalidated; the whole list of options we present as storage backend ability you are limiting it them on synching/filesync.

I see your point of conflicts, but dont we need to resolve it also now in case in realtime someone modifies the remote database while we edit it for example? Although we kind of cheat that nwo that we save immediately on change. However the merge in keepass file is a standardized procedure I believe (not trivial thought).

[J-Jamet]

However, I observed a limitation. If we use any LAN-only hosting of database (local sftp, samba, webdav etc). When we are in the same LAN files are opening without any issue. However, in remote operation the app correctly returns IO error, as the File manager is unable to find the LAN resource.

I don't know which file manager you use, but it seems to be unsuitable for the remote operation you are talking about. You should contact the developers of your application so that they can better manage these transfers.

From one point of you yes, from another if you think about it, any remote document API usability is invalidated; the whole list of options we present as storage backend ability you are limiting it them on synching/filesync.

I don't understand this part, what do you mean by invalidated? All the file manager needs to do is use a cache and update the remote data when changes are made to the database file.

I see your point of conflicts, but dont we need to resolve it also now in case in realtime someone modifies the remote database while we edit it for example? Although we kind of cheat that nwo that we save immediately on change. However the merge in keepass file is a standardized procedure I believe (not trivial thought).

The data merge is simply not done yet because I have to study the algorithm but it is planned to do it.

[georgekav2]

I use mixplorer in this scenario, opening a samba hosted db. They will use cache indeed when in editing mode, but not when remote is not accessible in first place. Same will apply to any filemanagers that keep a content link but not sync in background from cloud provider to be able to work offline (like onedrive, google drive etc).

I see your points, you can close the ticket and we keep a record of it that offline cache was discussed and rejected to avoid someone else opening the same feature request.

[J-Jamet]

I don't know what you call offline "cache", actually KeePassDX keeps in temporary memory the database as long as it is open for modifications.
When KeePassDX save the database, the data is sent in the stream provided by the content provider of your file manager and the stream is closed. That's all, nothing more.
If external modifications are made when the stream is open, the user is informed in order to perform actions. (reload the database to retrieve the latest data from the stream or save to overwrite remote data by send data to the stream, merge is under study #840)

KeePassDX only knows URIs (containing metadata and streams of database files), it doesn't know how they are ordered on the disk, and it doesn't care.

If you are talking about a file cache during a data transfer over the network with any protocol in MixPlorer, this is clearly not the job of KeePassDX.
If there is an IO exception, it is because the file manager reports an unmanaged error in the stream during a data transfer (read or write). Maybe there's a transfer on the network not completed, I don't know but as a client of MixPlorer, KeePassDX can only display the error, not correct it.

I hope to be clearer on the technical functioning.

[georgekav2]

Thanks @J-Jamet for the feedback.

I don't know what you call offline "cache", actually KeePassDX keeps in temporary memory the database as long as it is open for modifications.

Yeap this works great. No issue with that. As offline internal cache I meant when you closing the database. If your file manager fails to return the URI content due to IO, then you are out of luck, even if file manager correctly remembers the URIs and the whole process works as expected in online scenarios. If it helps, this feature might looks like the internal cache of keepass2android (even the offline one use it), which is a fallback if document URI fails.

Dont get me wrong, I do not insist on the feature, I replied for adding few more info on the context for future readers.

[J-Jamet]

Dont get me wrong, I do not insist on the feature, I replied for adding few more info on the context for future readers.

Yes, don't worry, it's only a good thing to debate and explain more precisely. ;)

I understand your need to keep an active URI if it is normally linked to a remote file not accessible.
But in this case, we need to create a specific application that can handle this behavior, I know that everyone asks for a suitable file manager, but I haven't had the time to do it yet. The advantage is that the chosen manager has a behavior adapted to our needs, and we don't have to comply with a single behavior.

The comparison with KeePass2Android and often made, I understand that it's easier to have an all-in-one tool. But I created KeePassDX on purpose because I wanted a different kind of development.

Here if I implement a similar caching system, I would have to persistently keep sensitive data in the application while the databases are closed.
Which I don't want. I do my best to delete data when a database is closed, only the URI (which is a symbolic link) remains.
Otherwise I might as well make a password manager that is not linked to a single file, but with internal encryption it would be simpler.

This means that if I have a file manager that only connects to a temporary disk (e.g. a USB key in OTG on the phone), all I have to do is lock the database and disconnect the OTG to have no more trace of sensitive data in the phone.

If there is this cache system, someone else might try to retrieve this data if they have access to the device, even if I deleted the source of the file. And from an architectural point of view, this would complicate the way to manage the data.

It's true that I don't explain all my analyses, but I try to do my best to work in the most modular and secure way possible, and as all the tools are not yet created, it's difficult to grasp.

[georgekav2]

This means that if I have a file manager that only connects to a temporary disk (e.g. a USB key in OTG on the phone), all I have to do is lock the database and disconnect the OTG to have no more trace of sensitive data in the phone.

You have a good use case here indeed, specially in delete context. So we consider the locked database still sensitive material right? Because in the end sync managers (filesync, syncthing, etc) that are not cloud providers will store the synced data on storage accessible to all apps that hold READ_EXTERNAL_STORAGE permission. Letting aside the cache or not point, is it more risky to have an (even forgotten/cached) encrypted database in /data/ when the phone is not rooted?

[J-Jamet]

If the encryption is strong enough, no worries. ;) But basically other applications shouldn't even see this file, unless we have specifically authorized them.
It is a bit the basic principle of configuring firewalls and file permission, we block everything and we authorize each element according to the necessary services.

In /data there is normally no problem but in your opinion, is it riskier to have encrypted data in a folder or no data at all to look at?

Google is more vigilant and stricter about applications that have the READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE permission (with Android 10 new policy).
Here I just want to leave the choice to the users to manage their data themselves.

But the important point for me is not to know if the IO exception was raised but if it's raised in a normal way because the file was deliberately deleted by the user or if it is a bug in the file manager. If the file has been deleted on purpose, the user would like it to be deleted from the cache as well and the cache goal is to keep a copy in case of deletion so it's not ok.

I'm not saying it's not secure for common databases. But it's better if attackers cannot even see any data to decrypt if we have configured our file storage this way.

[georgekav2]

Indeed the deniability of db existence aspect of no cache, even for encrypted database, make sense. Especially if users are not aware that something is cached. I see no automatic way of resolving IO due to remote being not accessible or if file is purposely deleted.

[J-Jamet]

I didn't study the specific cause of the IO exception in MixPlorer, but maybe the remote file directly updates the content provider without even retrieving the content of the file or maybe the URI is completely stopped because the file has changed and a new URI is created. I think it's only a bug at this level.

The solution for me is to implement the cache behavior in your favorite file manager or create a fresh new one.
Most managers simply don't test these cases because they don't even think about it. You should already discuss it to see if there is already a caching system in place and switch the stream content only if the remote file is fully loaded in the same URI.

I haven't completely thought about the workflow of a new file manager application yet but it could be:

• Propose a content provider URI linked to a cache file.
• If the file is updated remotely, download it to another temporary file.
• Once the temporary file is fully downloaded, replace the cache file with the temporary file and indicate the change in the metadata, without changing the address of the URI.
  With this method, there is theoretically no more flow interruption.

[georgekav2]

Mixplorer is not opensource, so I will look elsewhere to implement such cache functionality, they are few interesting filemanager project to begin with and contribute there. Do I need to indicate the change in metadata of the URI, or as long as I keep the temporary cache file in sync with the remote (on first opportunity of connectivity) you will be able to get it?