-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[Feature branch] JS: Migrate to shared dataflow library #14412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
asgerf
merged 226 commits into
github:js/shared-dataflow-branch
from
asgerf:js/shared-dataflow
Aug 2, 2024
Merged
Changes from 1 commit
Commits
Show all changes
226 commits
Select commit
Hold shift + click to select a range
60b179b
Shared: add DeduplicatePathGraph
asgerf 51ef0e5
JS: Move TNode into a cached module
asgerf 79e7aae
JS: Add TEarlyStageNode
asgerf b499c60
JS: Add Contents.qll
asgerf 21300ee
JS:Add ConstructorThisArgumentNode
asgerf 01952f1
JS: Add some missing getContainer() predicates
asgerf c839822
JS: Add PostUpdateNode
asgerf 3455463
JS: Add instantiation boilerplate
asgerf 760873c
JS: Basic instantiation of shared library
asgerf f316da7
JS: Add FunctionSelfReferenceNode
asgerf 8dc0800
JS: Add the shared FlowSummaryImpl.qll file
asgerf 60101f5
JS: Instantiate flow summary library
asgerf 32070ab
JS: Implicitly treat array steps as taint steps
asgerf 293899d
JS: Add 'Awaited' token
asgerf 5bccc65
JS: Move SharedFlowStep to AdditionalFlowSteps.qll
asgerf c24a0e0
JS: Move SharedTaintStep to AdditionalTaintSteps.qll
asgerf 1afe06e
JS: Add "additional" and "legacy" steps
asgerf 27c7d50
JS: Do the same for additional taint steps
asgerf 6037ff5
JS: Add LegacyPreUpdateStep
asgerf 3f20d71
JS: Add legacy post-update step
asgerf 46fec8e
JS: Add AdditionalFlowInternal
asgerf a31e251
JS: Add flow summaries for core methods
asgerf 4319b07
JS: Add flow summaries for Arrays
asgerf 5054c43
JS: Add flow summaries/steps for promises and async/await
asgerf f0c2afe
JS: Add flow summaries for maps and sets
asgerf da3a0de
JS: Port String#replace to flow summary
asgerf 0c2e52b
JS: Summary/steps for iterators and generators
asgerf e31ae3a
JS: Model JSON.stringify with "deep" read operators
asgerf 9fef880
JS: Avoid BarrierGuardNode's range from depending on Configuration
asgerf 3ef4786
JS: Collapse some cached stages
asgerf 16df2c3
Create DataFlowImplConsistency.qll
asgerf 7bcf8b8
JS: Capture flow
asgerf 46e4cdc
JS: Disallow consecutive captured contents
asgerf 06fd9c2
JS: Add barrier guard library
asgerf 277292e
JS: Improve performance of barrier guards without pruning
asgerf 1ed3235
JS: use BarrierGuards
asgerf c924b4a
JS: Expose shared API in DataFlow/TaintTracking modules
asgerf 26f7f94
JS: Expose default taint steps/sanitizers
asgerf bc68b6a
JS: Add AdHocWhitelistSanitizer::getABarrierNode()
asgerf aa5a283
JS: Update barriers in TaintedObject
asgerf 449ec72
JS: Port experimental queries
asgerf ccd6d3d
JS: Port example queries
asgerf 17233a6
JS: Port CommandInjection
asgerf fcfab52
JS: Port CodeInjection
asgerf 65e9706
JS: Port TaintedPath
asgerf 547a8a9
JS: Port SqlInjection
asgerf 2818fa6
JS: Updates to shared Xss.qll
asgerf e091fde
JS: Port DomBasedXss
asgerf 46b90e5
JS: Port ReflectedXss
asgerf 5f05232
JS: Port StoredXss
asgerf cf5450d
JS: Port XssThroughDom
asgerf d7b4e0c
JS: Port ExceptionXss
asgerf b221662
JS: Port RequestForgery
asgerf 92816b1
JS: Port ClientSideRequestForgery
asgerf 46fd727
JS: Port ServerSideUrlRedirect
asgerf 81d2721
JS: Port ClientSideUrlRedirect
asgerf f1f4592
JS: Port PrototypePollutingAssignment
asgerf adf7d54
JS: Port PrototypePollutingFunction
asgerf a5c221f
JS: Port PrototypePollutingMergeCall
asgerf b8a6f81
JS: Port CleartextLogging
asgerf 40d68cb
JS: Port CleartextStorage
asgerf ae680e7
JS: Port LoopBoundInjection
asgerf e9189f9
JS: Port LogInjection
asgerf 7a1aead
JS: Port ZipSlip
asgerf 395f523
JS: Port barriers in UrlConcatenation.qll
asgerf 85617c2
JS: Port BrokenCryptoAlgorithm
asgerf 2296a27
JS: Port BuildArtifactLeak
asgerf f14303a
JS: Port ConditionalBypass
asgerf 30f1fbc
JS: Port CorsMisconfigurationForCredentials
asgerf d324e55
JS: Port DeepObjectResourceExhaustion
asgerf abd937a
JS: Port DifferentKindsComparisonBypass
asgerf 8e95a90
JS: Port UntrustedDataToExternalAPI
asgerf 2935aac
JS: Port FileAccessToHttp
asgerf f4d62c3
JS: Port HttpToFileAccess
asgerf 4bac902
JS: Port HardcodedCredentials
asgerf bc88f50
JS: Port HardcodedDataInterpretedAsCode
asgerf 8715c1b
JS: Port HostHeaderPoisoningInEmailGeneration
asgerf 9128722
JS: Port ImproperCodeSanitization
asgerf e3ab5bd
JS: Port IncompleteHtmlAttributeSanitization
asgerf 8c00191
JS: Port IndirectCommandInjection
asgerf 99f63b1
JS: Port InsecureDownload
asgerf cd1a1e2
JS: Port InsecureRandomness
asgerf fd98b25
JS: Port InsecureTemporaryFile
asgerf e1fae3d
JS: Port InsufficientPasswordHash
asgerf 2400af4
JS: Port PostMessageStar
asgerf dcc73a7
JS: Port RegExpInjection
asgerf b9bd052
JS: Port RemotePropertyInjection
asgerf 4af7694
JS: Port ResourceExhaustion
asgerf 06835a8
JS: Port SecondOrderCommandInjection
asgerf d446444
JS: Port ShellCommandInjectionFromEnvironment
asgerf 63343b1
JS: Port StackTraceExposure
asgerf 51624c0
JS: Port TaintedFormatString
asgerf 25962a9
JS: Port TemplateObjectInjection
asgerf 5af608c
JS: Port TypeConfusionThroughParameterTampering
asgerf 32022cc
JS: Port UnsafeCodeConstruction
asgerf 758f424
JS: Port UnsafeDeserialization
asgerf 7f4d42d
JS: Port UnsafeDynamicMethodAccess
asgerf 6e3f4bd
JS: Port UnsafeHtmlConstruction
asgerf d08e450
JS: Port UnsafeJQueryPlugin
asgerf ba9edb4
JS: Port UnsafeShellCommandConstruction
asgerf 8309553
JS: Port UnvalidatedDynamicMethodCall
asgerf 03f8c0f
JS: Port XmlBomb
asgerf c2d170b
JS: Port XpathInjection
asgerf b8847db
JS: Port Xxe
asgerf c55300d
JS: Port PolynomialReDoS
asgerf 43be452
JS: Port meta queries
asgerf d35959a
JS: Add utility for comparing results in tests
asgerf ff08637
JS: Port Arrays test
asgerf 9a15a55
JS: Port SimpleBarrierGuard test
asgerf 1a95961
JS: Port Classes test
asgerf c652470
JS: Do not port CustomLoadStoreStep test
asgerf af05789
JS: Remove noise from data flow test
asgerf 92812ee
JS: Add test for flow summaries
asgerf 09b0ba0
JS: Port Angular2 test
asgerf 466ffdf
JS: Port AsyncTaintTracking test
asgerf 0989227
JS: Port Collections test
asgerf 6600fe9
JS: Port ComposedFunctions test
asgerf a2d4a03
JS: Update framework/data test
asgerf 644f968
JS: Update frameworks/immutable test
asgerf 2eec47b
JS: Update frameworks/Next test
asgerf d205344
JS: Update frameworks/PropertyProjection test
asgerf b934413
JS: Update Redux test
asgerf 3983530
JS: Update Templating/Xss test
asgerf 995df41
JS: Update Vuex test
asgerf 9372f79
JS: Update Generators test
asgerf 50aace3
JS: Add global post-update steps
asgerf 0d10aba
Revert "JS: Add global post-update steps"
asgerf 458f0a0
JS: Port InterProceduralFlow test
asgerf dd8a24c
JS: Port LabelledBarrierGuards test
asgerf 81bd292
JS: Port Promises test
asgerf 98d1bb3
JS: Reorder result sets in a test (trivial change)
asgerf 2364bd8
JS: Fix whitespace in a test (trivial change)
asgerf 771519b
JS: Port Routing test
asgerf e5946bf
JS: Port HeuristicSource test
asgerf 6c9f4a1
JS: Port TaintBarriers test
asgerf b8a0afb
JS: Make overriding ConsistencyChecking.getATestFile() optional
asgerf 32eddd3
JS: Update ReactJS test output
asgerf b304fb4
JS: Reorder result sets in ReactJS test output
asgerf c2f66c0
JS: Update Restify2 test
asgerf 75c915b
JS: Update Spife test
asgerf b5ad366
JS: Block flow into window.location
asgerf 2eff07f
JS: Update TaintTracking test
asgerf 85e8998
JS: Update ImportEquals test
asgerf bab639f
JS: Update ReflectedXssWithCustomSanitizer test
asgerf 9b46c45
JS: Update HeuristicSoruceCodeInjection test
asgerf 98c79e7
JS: Update test output showing lack of global flow (geniune FN)
asgerf 7c5eb89
JS: Add tests for captured 'this' (genuine FN)
asgerf 24bab27
JS: Add TODO for dynamic import step
asgerf 51dec79
JS: Lower access path limit to 2
asgerf d3f5169
JS: Lower field-flow branch limit on Polynomial ReDoS
asgerf e738b5d
JS: Expand callback test case
asgerf 9faf300
JS: Use type-pruning to restrict callback flow
asgerf 5775fe6
JS: Use TAnyType in FlowSummaryPrivate
asgerf 3c7c537
JS: Add content approximation
asgerf a02ab2a
JS: Port heuristic versions of standard queries
asgerf f94aa2c
Update javascript/ql/lib/semmle/javascript/dataflow/internal/DataFlow…
asgerf 28fc8ba
JS: Remove EmptyType
asgerf 76e0445
JS: Be consistent about caching in PreCallGraphStep
asgerf 5aafd33
JS: Rename Arrays2 -> Arrays
asgerf b3fad7a
JS: Rename Iterators2 -> Iterators
asgerf e2f3565
JS: Rename Maps2 -> Maps
asgerf 4334894
JS: Rename Promise2 -> Promise
asgerf 478dd25
JS: Rename Sets2 -> Sets
asgerf 2c1aa08
JS: Rename Strings2 -> Strings
asgerf 13a8e0f
JS: Add failing test for Promise.all()
asgerf 858c79e
JS: Add plain taint step through Promise.all()
asgerf 4043bc1
JS: Explicit mark comment as a TODO
asgerf e66f27c
JS: Move hasWildcardReplaceRegExp to a shared place
asgerf 14e75be
JS: Expand comments and synthetic node name in ForOfLoops
asgerf e640154
JS: Be backwards compatible with AdditionalBarrierGuardNode
asgerf fce2be0
JS: Use BarrierGuardLegacy in TaintedPath
asgerf e0aae53
JS: Remove unnecessary BarrierGuardLegacy class
asgerf b31f20a
JS: Explain why ObjetWrapperFlowLabel is deprecated
asgerf 11983fa
JS: Remove out-commented code
asgerf 0a2050b
JS: Deduplicate predicate in HostHeaderPoisoningQuery
asgerf 406b080
JS: Add comment about allowImplicitRead in PostMessageStar
asgerf ea4bc9c
JS: Comment about manually applying taint steps
asgerf fa8933e
JS: Reduce duplication in UnsafeDynamicMethodAccessQuery
asgerf 5e7d1d5
Merge branch 'main' into js/shared-dataflow-merged
asgerf 97567f4
JS: Update VariableCapture.qll after changes to API
asgerf bb1f729
Update VariableCapture.qll
asgerf e5bc8db
JS: Fix conflicting default for visbleImplInCallContext
asgerf 82abd86
JS: Update uses of AccessPathSyntax
asgerf 8ecdb5c
Update VariableCapture.qll
asgerf ddf6eb3
JS: Quick fix to make DeduplicatePathGraph compile
asgerf eff5f3b
JS: Remove duplicate dependency from qlpack.yml
asgerf 711a08b
JS: Add TODO about switching to the shared library
asgerf c408ab9
Merge branch 'main' into js/shared-dataflow
asgerf 2de9af2
JS: Update to getLocation() in DeduplicatePathGraph
asgerf 19f1462
JS: Update use of Locations
asgerf 5a2260b
JS: Update to match changes to API
asgerf 23d28fc
Shared: add location for 'this' nodes
asgerf 536c115
JS: Fix location override in CaptureNode
asgerf f43a189
JS: Make CaptureNode.toString() more explicit
asgerf 20df5ad
JS: Bugfix in DeduplicatePathGraph
asgerf bd3fccd
JS: Update test output with provenance column
asgerf ecf418b
Merge branch 'main' into js/shared-dataflow
asgerf 102ca77
Switch to getLocation() in DataFlowCall
asgerf 505c532
JS: Implement totalorder()
asgerf 64a9598
JS: Update interface for isUnreachableInCall
asgerf 6c8fb61
Js: Update FlowSummaryImpl.qll to make things compile
asgerf 6e32f27
Rename predicates to be consistent with qlpack
asgerf f0d7c3a
Remove bindingsets
asgerf dd7aff5
Instantiate shared FlowSummary library
asgerf 6b35a76
Migrate to shared FlowSummary library
asgerf 8c4e5e8
Boilerplate implementation of default predicates from FlowSummaryImpl…
asgerf 5811a3c
Port getMadStringFromContentSet -> encodeContent
asgerf b0ea812
Implement encodeReturn
asgerf 6c0c67d
Implement encodeWith/WithoutContent
asgerf 3bebd70
Handle AnyMemberDeep and ArrayElementDeep in encodeContent
asgerf e67e89d
Implement decodeUnknownArgument/ParameterPosition
asgerf fc7c2c5
Remove unused code
asgerf 88edc06
Avoid bad join in compatibleTypesCached
asgerf 53efb58
JS: Update some tests with provenance columns
asgerf af7b4e3
Accept flow difference due to added test cases
asgerf 2473274
JS: Benign test output changes
asgerf e53c0cd
Fix unknown Parameter/Argument decoding
asgerf c52a4b0
JS: Provide RenderSummarizedCallable
asgerf df0488a
Ensure Member tokens from flow summaries are seen in PropertyName
asgerf ee10702
JS: Another provanance test output update
asgerf 90f0e07
JS: Benign update after fixing PropertyName charpred
asgerf c3806a2
JS: Messy test output updates
asgerf 1c730bc
JS: Fix compilation error in DataFlowImplConsistency.qll
asgerf 14fc790
Update DataFlowConsistency.expected
asgerf e5924c1
JS: Another messy test update
asgerf 1d267ef
JS: Fix missing qldoc
asgerf File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you planning to add this step to the new library?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm planning to fix this missing flow by adding general handling of the spread operator, so individual models don't have to specify how to handle spread operators. For example if a model specifies
Argument[1]somewhere, and it applies to call that has a spread argument at position 0 or 1, then it should Just Work. The model shouldn't have to specify something likeArgument[spread]which is what the legacy model currently does.