Out of Memory Error during Analysis Step

[ausetiad]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Using CLI, perform baseline scan and another scan - "./Asa collect -CdlfFkpPsuwh —runid" (following instructions here https://github.com/microsoft/AttackSurfaceAnalyzer/wiki/CLI-Walkthrough)
2. Run "asa export-collect --outputsarif"
3. See error: "Out of memory.
   zsh: abort ./Asa export-collect --outputsarif --lowmemoryusage"

Expected behavior
I expect the analysis to be performed as explained in the walkthroug

Screenshots
If applicable, add screenshots to help explain your problem.

System Configuration (please complete the following information):

• OS: MacOS
• OS Version: MacOS Sonoma 14.3.1
• Application Version: ASA_macos_2.3.308.zip
• CLI or GUI: CLI

Additional Context

[gfs]

Thanks for the report. I'll try to reproduce this and see if there's anything that can be done to reduce memory during the analysis step.

To give me some hints about how to best repro this. Can you share how much memory your system has, how large the total asa.sqlite dbs you've collected are and, if you know, how much memory is being consumed when you hit the out of memory error?

[ausetiad]

@gfs

Machine is a 2023 Mac Studio, Apple M2 Max chip with 32GB of memory.

How do I collect the following information: how large the total asa.sqlite dbs you've collected are and, if you know, how much memory is being consumed when you hit the out of memory error?

[gfs]

@ausetiad

1. Size of databases: Look in the directory you're executing asa in and check the sizes of the asa.sqlite files
2. Memory usage: Open Activity Monitor while executing ASA and watch the memory usage statistic.

You can also try to re-run export collect with --disableimplicitfindings which may reduce memory usage. Based on the output messages I think the out of memory is hit during construction of the sarif output file - removing implicit findings should reduce the size of that file.

[ausetiad]

Sizes of asa.sqlite files:

• asa.sqlite - 14.68 GB
• asa.sqlite_1 - 14.64 GB
• asa.sqlite_2 - 14.65 GB
• asa.sqlite_3 - 14.7 GB
• asa.sqlite_4 - 14.65 GB
• asa.sqlite_5 - 14.62 GB
• asa.sqlite_6 - 14.66 GB

I'm going to try and get the memory usage info while running export collect with --disableimplicitfindings enabled

[gfs]

After further investigation the root cause of this appears to be that every file was reporting a different content hash between collection runs - but with no other changes resulting in a hugely inflated report. I am as of yet unable to repro this on a mac myself.

[gfs]

I was able to find a different replication for the out of memory issue and believe that #731 should help for most cases.