Skip to content

[BUG] Appending to wrong note #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
3 of 4 tasks
joh6nn opened this issue Oct 18, 2023 · 51 comments
Closed
3 of 4 tasks

[BUG] Appending to wrong note #87

joh6nn opened this issue Oct 18, 2023 · 51 comments
Labels
question Further information is requested
Milestone
3.0

Comments

@joh6nn
Copy link

joh6nn commented Oct 18, 2023

First Steps

  • I've ensured that the issue does not already exist by using the search issues feature
  • I've read the documentation (see the README)
  • I'm on the latest release of obsidian-list-modified
  • I've tried disabling any other plugins that may cause conflicts (recommended)
    • I don't know what other plugins might conflict, but I would be glad to try disabling them

Describe the bug

Links are consistently added to the incorrect note. I've got a vault with the following structure:

|-Entries
| \- [...Several Years of Daily Notes with the folder structure below]
| \- 2023
|    \ - 10 Oct
|        |- [Daily notes for Oct]
|        |- 18.md
|-Notes
|-Templates

Today, I enabled List Modified, with the following settings:

Instead of appending the links to the Notes heading in file Entries/2023/10 Oct/18.md , the links are being appended to Entries/2021/09 Sep/18.md

Expected behaviour

What should happen instead?

links should be appended to the Notes heading in file Entries/2023/10 Oct/18.md

Reproducing

My setup

  • Devices
    • Kubuntu 22.04
    • Android 12
    • Obsidian 14.16 (on both devices)
  • What sync service are you using? (if relevant to bug)
    • SyncThing

image

Steps to reproduce bug

Can you pinpoint when/why this bug happens?

n/a
@franciskafieh
Copy link
Owner

Could you please send me your output file configuration? Basically, just the rest of your settings pane works.

@joh6nn
Copy link
Author

joh6nn commented Oct 19, 2023

image

Ignored Tags is empty.

If you need anything else, just let me know 🙂

@franciskafieh
Copy link
Owner

Would you mind sending me this part?

image

@joh6nn
Copy link
Author

joh6nn commented Oct 19, 2023 •
edited
Loading

Unless I'm missing something, that's in the original screenshot?

Output Format is - ![[link]]
Time Format is unset because I'm not using a time field
Append Space After Headings is off

@franciskafieh
Copy link
Owner

You’re right, my bad.

@franciskafieh franciskafieh reopened this Oct 19, 2023
@franciskafieh
Copy link
Owner

Sorry, misclick on mobile, lol. I’ll take a look in a few hours

@franciskafieh
Copy link
Owner

franciskafieh commented Oct 19, 2023 •
edited
Loading

Alright, last thing: your Daily Notes settings pane, or your Periodic Notes settings if using that plugin

Just want to make sure that's setup properly. If it is, I will need to release a test version to help solve this. It seems to be working fine on my machine.

@joh6nn
Copy link
Author

joh6nn commented Oct 19, 2023

image

@joh6nn
Copy link
Author

joh6nn commented Oct 19, 2023

I do have a bunch of plugins, and I'm more than open to the idea that one or more of them is conflicting, I just don't know which one(s), or how to go about figuring that out (short of very tediously disabling all of them, and then turning them on one at a time. Which I'd love to not have to do)

@franciskafieh
Copy link
Owner

image
I mirrored your daily notes and file name settings here exactly, and everything works fine for me. Could you do the following for me?

  1. open the developer console (ctrl/cmd + shift + i)
  2. type clear() in the prompt area at the bottom
  3. screenshot the output (any errors, warnings, etc, should display)

@joh6nn
Copy link
Author

joh6nn commented Oct 21, 2023

image

I checked to see if there's any additional info if I bump the console output up to Verbose, but there's not, just the Obsidian Git plugin spamming the console with "task complete" messages.

@franciskafieh
Copy link
Owner

I think I’m going to add a debug mode to the plugin. I’ve been meaning to make one for awhile now. I’ll try to get it out tomorrow and we’ll see if it helps. In the mean time, you could try a different vault with just Obsidian List Modified, maybe? If it works there and not in your normal vault, it’s most likely a conflicting plugin

@franciskafieh franciskafieh added the question Further information is requested label Oct 21, 2023
@joh6nn
Copy link
Author

joh6nn commented Oct 23, 2023

Ok, so current situation is super weird:

I tried disabling plugins in this vault in a binary search (ie, I disabled half of them and tested, then disabled half of what was left and tested, and so on), and got down to the only plugin still enabled is List Modified, and I'm still seeing the incorrect behavior.

So I created a new vault, enabled Daily Notes, installed List Modified, and tried to recreate the issue by creating a note for today (2023-10-22), and also creating a note for 2021-09-22. I was able to reproduce the issue twice, but I can't reproduce it consistently. I don't know what was different about the two times I got it to happen.

If you add the debug mode, I'll try to collect more info

@franciskafieh
Copy link
Owner

That’s super odd.. I’ll try to get debug mode out ASAP. I’ve got a pretty bad flu right now though

By any chance, was SyncThing enabled on your second vault, too? I’m wondering if that could be the issue, but honestly, I can’t see how Sync would affect the output file’s location

@joh6nn
Copy link
Author

joh6nn commented Oct 23, 2023

Nope, no SyncThing 🤷

Sorry to hear about the flu, hope you feel better soon!

@franciskafieh
Copy link
Owner

Sorry for the long delay. I'm about to publish it as a pre-release, which means you will need to use the BRAT plugin or manually install it (video here if you need help).

What you're mainly looking for is this part...
image

After you've enabled the verbose mode setting, you should be able to open the dev console as you have done before. Play around with editing files, etc, until you see a message like that or "Log note not found, but should be created automatically. Creating now..."

@franciskafieh
Copy link
Owner

Link to pre-release: https://github.com/franciskafieh/obsidian-list-modified/releases/tag/2.2.1

@joh6nn
Copy link
Author

joh6nn commented Oct 31, 2023

Ok, I made a copy of my vault, and installed 2.2.1 with BRAT:

plugin:obsidian-list-modified:938 OLM settings saved.
plugin:obsidian-list-modified:790 --- Writing tracked files ---
plugin:obsidian-list-modified:744 [Obsidian List Modified] Log note found at: Entries/1999/12/31.md

Then I created a brand new empty vault, installed only BRAT and OLM, configured Daily Note to match my real vault (as noted above), and made a Templates folder, an Entries folder, and a Notes folder. I clicked the Daily Note button in the ribbon to create a daily note from the template, and then created two files in Notes, and OLM correctly added them to today's note.

Then I created Entries/2021/09/31.md. I closed and re-opened the test vault, and created a new file in Notes, and bingo: it added the list to the incorrect note, with all 3 entries (the one from just now, and the 2 from before):

image

@franciskafieh
Copy link
Owner

franciskafieh commented Oct 31, 2023 via email

@joh6nn
Copy link
Author

joh6nn commented Oct 31, 2023

My initial guess here is that the upstream plugin has a bug, because it also incorrectly detected 1999 as the current year, and additionally it also incorrectly detects the current month. I feel like moment.js has enough users that if it was the source of the bug, someone would have noticed 🙂

@joh6nn
Copy link
Author

joh6nn commented Oct 31, 2023

Yup, confirmed: liamcain/obsidian-daily-notes-interface#21

It's been broken for directory structures like YYYY/MM/DDD and similar for a long time, it seems.

@franciskafieh
Copy link
Owner

franciskafieh commented Oct 31, 2023 via email

@joh6nn
Copy link
Author

joh6nn commented Oct 31, 2023

It looks like there's actually a PR that claims to fix it: liamcain/obsidian-daily-notes-interface#25

It's actually fixing a bug in one of the developer's other plugins (the Calendar plugin), but the underlying issue is the same.

I still haven't really wrapped my head around modern JS's functional stuff, but it looks reasonable to me? I think it's dynamically building a RegEx that matches whatever the Daily/Periodic note settings currently are, but I gladly defer to you here

@franciskafieh
Copy link
Owner

franciskafieh commented Oct 31, 2023 via email

@joh6nn
Copy link
Author

joh6nn commented Oct 31, 2023

No worries! You've been super responsive as far as I'm concerned. Also, you're working for free; I'm grateful that you shared your work to begin with! Me putting in the legwork to make a decent bug report is the least I can do 🙂

@franciskafieh
Copy link
Owner

I'll need to work on this tomorrow as adding this fork as a dependency is way harder than it should be. Sorry for the delay

@franciskafieh
Copy link
Owner

try release 2.2.2 for me on BRAT. See if that fixes it

@joh6nn
Copy link
Author

joh6nn commented Nov 2, 2023

Ok, one step forward, one step back:

Recounting all my steps in case it's relevant.

I usually leave Obsidian open in the background, so both my main vault and the from-scratch test vault were still running from the other night. I upgraded OLM in the from-scratch vault, closed it, and reopened it by clicking on the vault icon in the window for my main vault.

As expected, it automatically created a new daily, and OLM correctly identified the Daily Note! But: it incorrectly identified what files to list there: I had only just opened the vault, and not created or modified any other files, so there shouldn't have been any files to list. Instead, it listed all 3 files from the notes folder. Both Obsidian and my file browser show the last modified time as Oct 31 for all 3 files 🤷

image

@franciskafieh
Copy link
Owner

franciskafieh commented Nov 2, 2023 •
edited
Loading

When the console says File created in that screenshot, what file was created? Would you mind sending me your data.json as well? I don't see how this bug could occur, as the logic is pretty straightforward and shouldn't be affected by the daily note location bug

	if (!lastTrackedDate.isSame(moment(), granularity)) {
		displayNotice(
			"New day/week/month detected, writing tracked files and resetting..."
		);
		await saveSettingsAndWriteTrackedFiles();
		
		// this code resets the tracked files in data.json and saves the files
		settings.trackedFiles = [];
		
		// current date as known by moment, should not be affected by file path bug
		settings.lastTrackedDate = moment().format("YYYY-MM-DD");
		await saveSettings();
	}

@franciskafieh
Copy link
Owner

Edit: this reminds me to add some more specific things to the verbose mode

@joh6nn
Copy link
Author

joh6nn commented Nov 2, 2023

The file that was created was the the one with focus in Obsidian: 2023/11 Nov/02.md

Here's my data.json. I glanced at it real quick before uploading, and remembered: I turned off "auto create log note", which is reflected in the json. So either there's a bug in the verbose output and it's writing that to the console when it shouldn't, or there's a bug in the automatic file creation logic and it's running when it shouldn't
data.json

@franciskafieh
Copy link
Owner

Did you edit the data.json at all? It shows no files in the trackedFiles array, so nothing should be written to your daily note... Super odd. Everything is working fine on my end (tracking the right files, note not being created automatically unless told to, etc). If there were problems with this, there would be many other issues open.

This didn't happen before the update, right? I don't see how these changes would affect any of this anyway.

Could you delete the data.json to give OLM a fresh start? Then, enable verbose mode again. Now it shows the names of created files and stuff as well. I'm honestly super confused since usually I know what I'm doing with my own plugin lol

@franciskafieh
Copy link
Owner

Oh wait, the issues with create are my fault. Verbose mode outputs "file created" even if it is not added to data.json (it's filtered out for one reason or another)

As per identifying the right files to modify, I could try adding even more test messages to see why each file is identified. But this shouldn't be necessary, and I don't know why you're getting the wrong list of files when the data.json shows no files at all.

Maybe try deleting it for now (as I said above) and see what happens

@joh6nn
Copy link
Author

joh6nn commented Nov 4, 2023

Sigh. Ok, I apologize, because I feel like I'm taking you down a rabbit hole here:

So to make sure that I hadn't unintentionally messed anything up or introduced any weirdness in the previous round of testing, I deleted my test vault and recreated it.

Like before, I was able to reproduce the issue one time, but I'm not clear on how 🙁. And now I can't reproduce it again.

But while I was trying, I got OLM to insert the list of modified files to the same note twice in a single session. So naturally I wanted to see "what happens when I make a new note now?" and the answer was that it figured out that the list should only be added to the Log Note one time, but it deleted all the content in the note after the note heading 🤷

@franciskafieh
Copy link
Owner

Don't apologize!

and the answer was that it figured out that the list should only be added to the Log Note one time

I think it does this sometimes, but tmk it's a concurrency issue which is pretty difficult to fix without extra reads to the file and complication. As long as it solves itself, it should be fine in my books

but it deleted all the content in the note after the note heading

this shouldn't happen. Do you think what you're describing is related to #81? I need to fix that soon anyway

@joh6nn
Copy link
Author

joh6nn commented Nov 6, 2023

Yeah, that looks like it's either the same issue, or very closely related

@franciskafieh
Copy link
Owner

#81 should be fixed now. try it out and let me know what happens :)

@franciskafieh
Copy link
Owner

Nevermind, one second

@franciskafieh
Copy link
Owner

Should be good now

@mayukhc mayukhc mentioned this issue Nov 23, 2023
Open
@franciskafieh
Copy link
Owner

Any updates? Going to close if this issue is fixed

@joh6nn
Copy link
Author

joh6nn commented Nov 24, 2023

Hey, sorry for the radio silence, I have been crazy busy the last several weeks and haven't had time to give this the attention it deserves. I'm hoping to get to it this weekend

@franciskafieh
Copy link
Owner

No worries

@joh6nn
Copy link
Author

joh6nn commented Dec 2, 2023

so, apologies for the long delay in getting back to this

So, good news, bad news, good news.

Good news: as far as I can tell the initial issue that I opened this ticket for is fixed: in all my testing, OLM now successfully writes to the correct note.

Bad news: I'm still seeing issues with symptoms similar to #81 , where content after the insertion is being overwritten/erased.

Good news: I can consistently reproduce it!

  1. Open the vault, let Obsidian automatically create the daily note
  2. manually create a new note
  3. delete the daily note
  4. close Obsidian, and reopen it, and let it automatically recreate the daily note
  5. manually create a new note
  6. et voila! everything after the inserted content is missing

Also fun: sometimes the inserted content is duplicated. Sometimes it's duplicated in its entirety, and sometimes only one or two lines are duplicated. It doesn't appear to be a rendering issue; opening a different note, and then coming back to the daily note, the duplicated content is still there. I can reproduce this frequently, but haven't figured out specific steps for reproducing it consistently. It's not clear to me whether this is related to the symptoms of #81, or if it's a separate issue

Since the original issue is now solved, do you want to move these other symptoms to a new issue (or issues?)

@joh6nn
Copy link
Author

joh6nn commented Jan 3, 2024

Hey, happy new year! Hope your holidays were good. Just wanted to check back in on this

@franciskafieh
Copy link
Owner

franciskafieh commented Jan 4, 2024 via email

@joh6nn
Copy link
Author

joh6nn commented Jan 4, 2024

No worries, and no rush; enjoy your vacation!

@franciskafieh
Copy link
Owner

Sorry for the wait. After my break I had exams. Everything's done now, so I'm back to working on the plugin.

@joh6nn
Copy link
Author

joh6nn commented Feb 3, 2024

No worries: our lives have to come before our projects! I hope you did well on your exams 🤞

@franciskafieh
Copy link
Owner

#90 would fix this as it would be way more reliable than headings. Just want to see everyone's opinions before making the breaking change

@franciskafieh franciskafieh added this to the 3.0 milestone Jun 25, 2024
@franciskafieh
Copy link
Owner

franciskafieh commented Jun 27, 2024 •
edited
Loading

This has been dead for awhile, but I'm back to working on it. What's super odd is that the local "fork" of the upstream plugin that was causing the bug is exactly the same as the published version. Basically, I forgot to download the "patch" branch instead of the default. And somehow, your issue is still fixed? I'm trying to replicate it right now with the same daily note format as you, and everything seems good. Super odd. I guess I will just leave it as is for now.... I'm stumped

Edit: Talking about the "finding the daily note" bug, not the heading bug, which will definitely be fixed in 3.0. Also, the supposed patch PR won't even build. The contributor messed something up and I haven't had a chance to look into it.

@franciskafieh
Copy link
Owner

Closing this for now. If it continues to happen when 3.0 comes out, please reopen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
3.0
Development

No branches or pull requests
2 participants
@joh6nn @franciskafieh