fix(gateway): ipld.ErrNotFound should result in a 404 (#630)

* fix(gateway): ipld.ErrNotFound should result in a 404

In case of blockstore restriction like with --offline or similar restriction, returning a 500 is inappropriate.

It's also going back to a previous gateway behavior (at least in kubo v0.22, possibly later).

* refactor(gateway): match 410 before 404s

just a precaution to ensure 410 takes precedence, until we address
#591

* docs: clarify 404 use case

---------

Co-authored-by: Marcin Rataj <[email protected]>

Author: MichaelMure

CHANGELOG.md

@@ -26,6 +26,7 @@ The following emojis are used to highlight certain changes:
 
 ### Fixed
 
+- `boxo/gateway` now correctly returns 404 Status Not Found instead of 500 when the requested content cannot be found due to offline exchange, gateway running in no-fetch (non-recursive) mode, or a similar restriction that only serves a specific set of CIDs.
 - `bitswap/client` fix memory leak in BlockPresenceManager due to unlimited map growth.
 
 ### Security

gateway/errors.go

@@ -13,6 +13,7 @@ import (
 	"github.com/ipfs/boxo/path"
 	"github.com/ipfs/boxo/path/resolver"
 	"github.com/ipfs/go-cid"
+	ipld "github.com/ipfs/go-ipld-format"
 	"github.com/ipld/go-ipld-prime/datamodel"
 	"github.com/ipld/go-ipld-prime/schema"
 )
@@ -185,10 +186,10 @@ func webError(w http.ResponseWriter, r *http.Request, c *Config, err error, defa
 	switch {
 	case errors.Is(err, &cid.ErrInvalidCid{}):
 		code = http.StatusBadRequest
-	case isErrNotFound(err):
-		code = http.StatusNotFound
 	case isErrContentBlocked(err):
 		code = http.StatusGone
+	case isErrNotFound(err):
+		code = http.StatusNotFound
 	case errors.Is(err, context.DeadlineExceeded):
 		code = http.StatusGatewayTimeout
 	}
@@ -226,6 +227,10 @@ func isErrNotFound(err error) bool {
 		return true
 	}
 
+	if ipld.IsNotFound(err) {
+		return true
+	}
+
 	// Checks if err is of a type that does not implement the .Is interface and
 	// cannot be directly compared to. Therefore, errors.Is cannot be used.
 	for {

gateway/gateway_test.go

@@ -924,7 +924,7 @@ func TestErrorBubblingFromBackend(t *testing.T) {
 		})
 	}
 
-	testError("500 Not Found from IPLD", &ipld.ErrNotFound{}, http.StatusInternalServerError)
+	testError("404 Not Found from IPLD", &ipld.ErrNotFound{}, http.StatusNotFound)
 	testError("404 Not Found from path resolver", &resolver.ErrNoLink{}, http.StatusNotFound)
 	testError("502 Bad Gateway", ErrBadGateway, http.StatusBadGateway)
 	testError("504 Gateway Timeout", ErrGatewayTimeout, http.StatusGatewayTimeout)