Release v0.35.0 #10799
Draft
Release v0.35.0 #10799
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
lidel
commented
May 7, 2025
lidel
commented
- Release 0.35 #10760 (comment)
Fixes #10596. The reproviding process can take long. Currently, each CID to be provided is obtained by making a query to the pinner and reading one by one as the CIDs get provided. While this query is ongoing, the pinner holds a Read mutex on the pinset. If a pin-add-request arrives, a goroutine will start waiting for a Write mutex on the pinset. From that point, no new Read mutexes can be taken until the writer can proceed and finishes. However, no one can proceed because the read mutex is still held while the reproviding is ongoing. The fix is mostly in Boxo, where we add a "buffered" provider which reads the cids onto memory so that they can be provided at its own pace without making everyone wait. The consequence is we will need more RAM memory. Rule of thumb is 1GiB extra per 20M cids to be reprovided.
provider: buffer pin providers
* fix typos * fix func name in docstring * use t.TempDir instead of os.MkdirTemp * fix typo * unique option names
* Upgrade to Boxo v0.29.1
* docs(readme): update min. requirements + cleanup Adding note about extra memory requirement due to ba22102 * docs(config): memory cost of buffered provider Co-authored-by: Daniel Norman <[email protected]>
Merge release v0.34.0
Adds `Routing.IgnoreProviders`. This requires initializing a custom providerQueryManager and using it instead of the default created internally in Bitswap. Since the default is created with some internal default configuration options (MaxProviders), this hardcodes it.
Merge release v0.34.1
Support WithIgnoreProviders() in provider query manager
Co-authored-by: Andrew Gillis <[email protected]>
…10754) Co-authored-by: Marcin Rataj <[email protected]>
Replacing hardcoded integer with named default and expose config option for adjusting it, like we do in Rainbow https://github.com/ipfs/kubo/pull/10765/files#r2025455848
…AG Width (#10774) Co-authored-by: Marcin Rataj <[email protected]>
* Upgrade to Boxo v0.30.0
* adjust ipfs stats provide * update boxo dep * bump boxo * fixing tests * docs/chore: mark stat reprovide as experimental * docs: Provider.Strategy explicitly document it is not used - without this legacy users will have it in their config and be very confused --------- Co-authored-by: Marcin Rataj <[email protected]>
* Upgrade to pebble v2.0.3 - Configure latest pebble database format at init - Do not automatically ratchet database format if set in config - Daemon messge about new available pebble format - Document pebble config with formatMajorVersion - Add warning to users running badger, nudging them to switch to flatfs or pebble - docs: explain Pebble's `FormatMajorVersion` - Use pebbleds instead of badgerds in t0060-daemon.sh - Print badgerds warning message to stderr
* Upgrade to pebble v2.0.3 - Configure latest pebble database format at init - Do not automatically ratchet database format if set in config - Daemon messge about new available pebble format - Document pebble config with formatMajorVersion - Add warning to users running badger, nudging them to switch to flatfs or pebble - docs: explain Pebble's `FormatMajorVersion` - Use pebbleds instead of badgerds in t0060-daemon.sh - Print badgerds warning message to stderr
* datastore: metrics optional and off by default When ipfs is initialized, the datastore metrics wrapper is not configured by default as it previously was. To enable datastore metrics during initialization, specifying the appropriate `--profile` option. To enable datastore metrics tracking wrapper, initialize with datastore profile name + "-measure" suffix. For example: ``` ipfs init --profile flatfs-measure ``` Closes #10767 * fix sharness tests for new datastore dafaults * Add sharness test to check metrics added by flatfs-measure profile * Document updated metrics in changelog * update config doc with new profiles * docs(changelog): separate section * initialize non-measure pebbleds with FormatMajorVersion config * docs: fix typos, add docs link --------- Co-authored-by: gammazero <[email protected]> Co-authored-by: Marcin Rataj <[email protected]>
* feat: add Bitswap configuration and related tests * fix: update Bitswap function to use 'provide' parameter for server enablement * docs: update changelog for Bitswap functionality changes * fix: update Bitswap server enablement logic and improve related tests * fix: rename BitswapConfig to Bitswap and update references * docs: config and changelog * fix: `ipfs cat` panic when `Bitswap.Enabled=false` Fixes panic described in: #10782 (comment) --------- Co-authored-by: gystemd <[email protected]> Co-authored-by: gammazero <[email protected]> Co-authored-by: Giulio Piva <[email protected]> Co-authored-by: Marcin Rataj <[email protected]>
* Feat: http retrieval as experimental feature This introduces the http-retrieval capability as an experimental feature. It can be enabled in the configuration `Experimental.HTTPRetrieval.Enabled = true`. Documentation and changelog to be added later. * refactor: HTTPRetrieval.Enabled as Flag * docs(config): HTTPRetrieval section * refactor: reusable MockHTTPContentRouter * feat: HTTPRetrieval.TLSInsecureSkipVerify allows self-signed certificates in tests * feat(config): HTTPRetrieval.MaxBlockSize * test: end-to-end HTTPRetrieval.Enabled this spawns two http services on localhost: 1. HTTP router that returns HTTP provider when /routing/v1/providers/cid i queried 2. HTTP provider that returns a block when /ipfs/cid is queried 3. Configures Kubo to use (1) instead of cid.contact this seems to work (running test with DEBUG=true shows (1) was queried for the test CID and returned multiaddr of (2), but Kubo never requested test CID block from (2) – needs investigation * fix: enable /routing/v1/peers for non-cid.contact we artificially limited every delegated routing endpoint because of cid.contact being limited to one endpoint * feat: Routing.DelegatedRouters make it easy to override the hardcoded implicit HTTP routeur URL without having to set the entire custom Router.Routers and Router.Methods (http_retrieval_client_test.go still needs to be fixed in future commit) * test: flag remaining work * docs: review feedback * refactor: providerQueryMgr with bitswapNetworks this fixes two regressions: (1) introduced in #10717 where we only used bitswapLib2p query manager (this is why E2E did not act on http provider) (2) introduced in #10765 where it was not possible to set binary peerID in IgnoreProviders (we changed to []string) * refactor: Bitswap.Libp2pEnabled replaces Bitswap.Enabled with Bitswap.Libp2pEnabled adds tests that confirm it is possible to disable libp2p bitswap fully and only keep http in client mode also, removes the need for passing empty blockstore in client-only mode * docs: changelog --------- Co-authored-by: Marcin Rataj <[email protected]>
* Add MFS command line options, extend existing mount functions for MFS, set defaults. * Directory listing and file stat. * Add a read-only MFS view. * Add mkdir and interface checks. * Add remove and rename functionality. * Implement all required write interfaces. * Adjust mount functions for other architechtures. * Merge branch 'master' into feat/10710-mfs-fuse-mount * Write a basic read/write test. * Write more basic tests, add a mutex to the file object, fix modtime. * Add a concurrency test, remove mutexes from file and directory structures. * Refactor naming(mfdir -> mfsdir) and add documentation. * Add CID retrieval through ipfs_cid xattr. * Add docs, add xattr listing, fix bugs for mv and stat, refactor. * Add MFS command line options, extend existing mount functions for MFS, set defaults. * docs phrasing * docs: Mounts.MFS * docs: warn about lazy-loaded DAGs * test: TEST_FUSE=1 ./t0030-mount.sh -v --------- Co-authored-by: Guillaume Michel <[email protected]> Co-authored-by: guillaumemichel <[email protected]> Co-authored-by: Marcin Rataj <[email protected]>
* repo: optionally wait to acquire repo lock Retry acquiring repo lock for a specified amount of time. Retry once per second until time is expired ro lock is acquired. Lock wait timeout is specified using the environment variable `IPFS_WAIT_REPO_LOCK` and assigning it a value that is parsable as a golang `time.Duration`. For example: ``` IPFS_WAIT_REPO_LOCK="5s" ``` Closes #10482 * document IPFS_WAIT_REPO_LOCK --------- Co-authored-by: Marcin Rataj <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.