Add basic QueryEngine scaffolding w/ Encodable hashing
#7347
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This ports most of the current `GeneratorEngine` implementation from the Swift SDK Generator to the SwiftPM code base: https://github.com/apple/swift-sdk-generator/blob/main/Sources/GeneratorEngine/Engine.swift. Since Swift NIO is not supported on Windows, references to `AsyncHTTPClient` have been removed. Additionally, we can't use macros in the SwiftPM code base either, thus the `Query` protocol has to conform to `Encodable` instead. We don't have a consistent hashing implementation for `Encodable` yet, and a temporary stub for it is marked with `fatalError` for now.
MaxDesiatov
added
concurrency
no functional change
|
@swift-ci test |
|
@swift-ci test |
|
@swift-ci test |
|
@swift-ci test windows |
|
@swift-ci test |
|
@swift-ci test windows |
|
@swift-ci test |
|
@swift-ci test windows |
|
@swift-ci clean test windows |
|
@swift-ci test |
|
@swift-ci test windows |
…xd/query-engine-scaffolding # Conflicts: # Package.swift
MaxDesiatov
force-pushed
the
maxd/query-engine-scaffolding
branch
from
8105338 to
c5549ed
Compare
|
@swift-ci test |
|
@swift-ci test |
|
@swift-ci test windows |
xedin
approved these changes
Apr 10, 2024
MaxDesiatov
added a commit
that referenced
this pull request
Apr 11, 2024
Depends on #7347. Previously, `QueryEngine` couldn't cache anything due to a missing hashing implementation that could be consistent across SwiftPM relaunches. Here `HashEncoder` is implemented on top `Codable`. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping. New `QueryEngine` now works for trivial computations, as verified by newly added `QueryEngineTests`.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this pull request
May 15, 2024
…#7347) ### Motivation: Caching of computations in SwiftPM is ad-hoc and mostly relies on in-memory `ThreadSafeKeyValueStore`, which utilizes locking and makes it harder to add `Sendable` conformances on types that have these stores as properties. To allow consistent and persistent caching when SwiftPM processes are relaunched, we can use a SQLite-backed async-first caching engine. ### Modifications: This change ports [most of the current `GeneratorEngine` implementation](https://github.com/apple/swift-sdk-generator/blob/main/Sources/GeneratorEngine/Engine.swift) from the Swift SDK Generator repository to the SwiftPM code base as `QueryEngine`. Since SwiftNIO is not supported on Windows, references to `AsyncHTTPClient` have been removed. Additionally, we can't use macros in the SwiftPM code base either, thus the `Query` protocol has to conform to `Encodable` instead of using macro-generated conformances. We don't have a consistent hashing implementation for `Encodable` yet, and a temporary stub for it is marked with `fatalError` for now. ### Result: NFC, new `QueryEngine` module is not used anywhere yet.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this pull request
May 15, 2024
Depends on swiftlang#7347. Previously, `QueryEngine` couldn't cache anything due to a missing hashing implementation that could be consistent across SwiftPM relaunches. Here `HashEncoder` is implemented on top `Codable`. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping. New `QueryEngine` now works for trivial computations, as verified by newly added `QueryEngineTests`.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this pull request
May 15, 2024
…#7347) ### Motivation: Caching of computations in SwiftPM is ad-hoc and mostly relies on in-memory `ThreadSafeKeyValueStore`, which utilizes locking and makes it harder to add `Sendable` conformances on types that have these stores as properties. To allow consistent and persistent caching when SwiftPM processes are relaunched, we can use a SQLite-backed async-first caching engine. ### Modifications: This change ports [most of the current `GeneratorEngine` implementation](https://github.com/apple/swift-sdk-generator/blob/main/Sources/GeneratorEngine/Engine.swift) from the Swift SDK Generator repository to the SwiftPM code base as `QueryEngine`. Since SwiftNIO is not supported on Windows, references to `AsyncHTTPClient` have been removed. Additionally, we can't use macros in the SwiftPM code base either, thus the `Query` protocol has to conform to `Encodable` instead of using macro-generated conformances. We don't have a consistent hashing implementation for `Encodable` yet, and a temporary stub for it is marked with `fatalError` for now. ### Result: NFC, new `QueryEngine` module is not used anywhere yet.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this pull request
May 15, 2024
Depends on swiftlang#7347. Previously, `QueryEngine` couldn't cache anything due to a missing hashing implementation that could be consistent across SwiftPM relaunches. Here `HashEncoder` is implemented on top `Codable`. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping. New `QueryEngine` now works for trivial computations, as verified by newly added `QueryEngineTests`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation:
Caching of computations in SwiftPM is ad-hoc and mostly relies on in-memory
ThreadSafeKeyValueStore, which utilizes locking and makes it harder to addSendableconformances on types that have these stores as properties. To allow consistent and persistent caching when SwiftPM processes are relaunched, we can use a SQLite-backed async-first caching engine.Modifications:
This change ports most of the current
GeneratorEngineimplementation from the Swift SDK Generator repository to the SwiftPM code base asQueryEngine. Since SwiftNIO is not supported on Windows, references toAsyncHTTPClienthave been removed. Additionally, we can't use macros in the SwiftPM code base either, thus theQueryprotocol has to conform toEncodableinstead of using macro-generated conformances. We don't have a consistent hashing implementation forEncodableyet, and a temporary stub for it is marked withfatalErrorfor now.Result:
NFC, new
QueryEnginemodule is not used anywhere yet.