watcherService CPU/RAM usage

[jabbson]

Issue Type: Performance Issue

watcherService seems to be using way too much CPU and RAM

VS Code version: Code 1.33.0 (0dd516d, 2019-04-04T15:11:32.076Z)
OS version: Darwin x64 18.5.0

System Info

Item	Value
CPUs	Intel(R) Core(TM) i5-4278U CPU @ 2.60GHz (4 x 2600)
GPU Status	2d_canvas: enabled checker_imaging: disabled_off flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on native_gpu_memory_buffers: enabled rasterization: enabled surface_synchronization: enabled_on video_decode: enabled webgl: enabled webgl2: enabled
Load (avg)	8, 8, 8
Memory (System)	8.00GB (0.02GB free)
Process Argv
Screen Reader	no
VM	0%

Process Info

CPU %	Mem MB	   PID	Process
   18	    82	 82473	code main
   13	    49	 82475	   gpu-process
    0	    82	 82476	   window (Untitled-2 — jabbson)
    0	    25	 82479	     extensionHost
  130	  2318	 82480	     watcherService
    0	    57	 82481	   shared-process
    7	    74	 82799	   window (Issue Reporter)
    0	     0	 82800	   /bin/ps -ax -o pid=,ppid=,pcpu=,pmem=,command=

Workspace Info

|  Window (Untitled-2 — jabbson)
|    Folder (jabbson): more than 22115 files
|      File types: localstorage-shm(1731) localstorage-wal(1731) plist(965)
|                  log(931) exp(816) c(713) JPG(684)
|                  localstorage(672) png(619) o(582)
|      Conf files: makefile(44) package.json(24) settings.json(22)
|                  launch.json(8) tslint.json(4) tsconfig.json(2)
|                  tasks.json(2) webpack.config.js(1) gulp.js(1)
|                  jsconfig.json(1);

Extensions (23)

Extension	Author (truncated)	Version
language-x86-64-assembly	13x	2.2.11
vscode-sqlite	ale	0.6.0
code-gnu-global	aus	0.2.2
kivy-vscode	Bat	0.5.0
django-snippets	bib	1.1.1
flask-snippets	cst	0.1.0
vscode-eslint	dba	1.8.2
xml	Dot	2.4.0
gc-excelviewer	Gra	2.1.31
CppSnippets	har	0.0.13
applescript	idl	0.14.2
dotenv	mik	1.0.1
vscode-clang	mit	0.2.3
HTMLHint	mka	0.6.0
vscode-remote-workspace	mkl	0.42.0
language-gettext	mro	0.1.1
python	ms-	2019.3.6352
cpptools	ms-	0.22.1
Go	ms-	0.9.2
printcode	nob	3.0.0
vscode-docker	Pet	0.6.1
mako	tom	0.2.0
sort-lines	Tyr	1.8.0

[jabbson]

This appears to have been related to the source control features. Closing repos made the watcherService to calm down.

[pwaller]

Apologies if I'm hijacking an issue inappropriately, but this does read very much like an issue I'm facing at the moment.

Note: I'm on Ubuntu 18.04, x86_64, installed vscode through a snap. All extensions are disabled.

$ code --version
1.32.3
a3db5be9b5c6ba46bb7555ec5d60178ecc2eaae4
x64

I'm having a significant problem with CPU usage in the watcher service. It's burning all my CPU/battery continuously for tens of minutes after I open the editor.

Some notes:

• I have a directory in my project /outputs/ with >8,000,000 files in it in a complex data structure containing generated data that I don't want to edit with vscode.

• VSCode instructed me to increase the system fs.inotify.max_user_watches if I wanted watching to work, so I did.

• Suddenly, vscode now uses huge amount of CPU (800%) in code .../bootstrap-fork --type=watcherService.

• I'm happy for vscode to ignore this directory. I have the following workspace-specific configuration, which seems to help reduce the CPU usage but does not eliminate it:

  {
          "git.ignoreLimitWarning": true,
          "files.watcherExclude": {
                  "/outputs/**": true
          },
          "search.exclude": {
                  "/outputs/**": true
          },
          "files.exclude": {
                  "/outputs/**": true
          }
  }

• The CPU usage is not reported within vscode's process manager, making it harder to understand what's happening.

• I've tried using the developer tools "performance" and "javascript profiler", but neither of these show any activity occurring.

• This command takes a while to run and produces a large output, I caught it in my process listing, and it might be related.

  $ time git status -z -u | wc -c
  676449428
  
  real	0m11.261s
  user	0m9.100s
  sys	0m2.486s
  

  I have added /outputs/ to my .gitignore which makes this command instant, but does not improve the situation with vscode.

• The CPU usage does eventually cease after awhile, but it is tens of minutes after opening vscode.

• If I close and reopen vscode, the CPU usage reoccurs - which is inconvenient as I do need to restart the editor occasionally due to issues with the language server that I'm using.

• Moving the large directory out of my directory tree does solve the CPU issue. But I don't want to do this.

  • I have tried a workaround using a symbolic link having the physical directory outside of the tree and only a symlink inside the workspace. The workaround does not improve the CPU usage of vscode.

• strace shows the process doing lots of futex/madvise/mmap/munmap syscalls, but no obvious filesystem activity. perf top shows the process is spending lots of time in the nodejs runtime.

• I haven't yet been able to find a way of discovering what the nodejs code is doing within that process. Advice appreciated here so that I can feed back more detailed information.

[Leo1003]

I think the memory issue is related to the bug in Axosoft/nsfw.
Details

[Tyriar]

I'm watching the repo, we should update our fork when nsfw does a release.

[Tyriar]

It was released, we should update https://github.com/Axosoft/nsfw/releases/tag/v1.2.3

[enzious]

Please update the fork :) This is crashing my machine over and over again. I have to monitor memory and restart vscode, otherwise I have to restart my machine. It sneaks up on me, and then it's too late. My uptime has been significantly reduced.

RLS + This Bug = Torture

[Leo1003]

@enzious Currently, not to open two directories in single vscode window can avoid it to use nsfw as its watcher. This is the method I use to prevent this bug before the bug is fixed in vscode.