refactor(server): change custom advisory and vulnerability data types fr…

[tonaim]

1. Regenerated gRPC code (service.pb.go):
   Updated all getters, raw descriptors, dependency indexes, and field types to use []byte instead of *structpb.Value.
2. Conversion logic (pkg/rpc/convert.go):
   Encoding (ConvertToRPCVulns): JSON-marshal both vuln.Custom and vuln.Vulnerability.Custom into the new byte fields.
   Decoding (ConvertFromRPCVulns): JSON-unmarshal incoming byte slices back into interface{}, falling back to nil on unmarshal errors.
3. Unit tests:
   Updated existing tests to expect raw byte slices for non-empty custom data and nil interfaces for empty data.
   Added new round-trip tests to verify that custom fields serialize → deserialize correctly.

Why This Matters

• Performance & Dependency Reduction
  Removing structpb.Value avoids heavy reflection and descriptor overhead, streamlining serialization on both server and client.

• Data Fidelity
  Callers now get the exact original structure (maps, arrays, strings) without needing to re-interpret a generic Protobuf Value wrapper.

• more cross-language-friendly
  The generated stub contains a plain bytes field, which every language’s core protobuf runtime already knows how to handle. No Struct/Value helpers required.

• Test Consistency
  Ensures empty custom data fields yield a true Go nil interface, matching existing test expectations.

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[deven0t]

do we need this additional unmarshal? cannt we directly assign customAdvisoryData to proto attribute.

[tonaim]

@deven0t , handled it now.

[Copilot]

Pull Request Overview

Refactor serialization of custom advisory and vulnerability data to use raw bytes instead of structpb.Value, updating protobuf definitions, generated code, and conversion logic.

• Changed custom_advisory_data and custom_vuln_data fields in the Vulnerability message from google.protobuf.Value to bytes.
• Regenerated service.pb.go to reflect byte-slice fields, updated getters, raw descriptors, and dependency indexes.
• Updated ConvertToRPCVulns/ConvertFromRPCVulns to marshal custom data as JSON bytes and assign raw bytes on the return path.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
rpc/common/service.proto	Updated custom data fields in Vulnerability to bytes
rpc/common/service.pb.go	Regenerated code for byte-slice fields, getters, and descriptors
pkg/rpc/convert.go	Switched to JSON marshalling and adapted conversion functions

Comments suppressed due to low confidence (1)

pkg/rpc/convert.go:303

• There are no unit tests covering the new JSON serialization logic for custom advisory and vulnerability data. Add tests to verify that data round-trips correctly through ConvertToRPCVulns and ConvertFromRPCVulns.

jsonBytes, _ := json.Marshal(vuln.Custom) // nolint: errcheck

[Copilot]

ConvertFromRPCVulns assigns raw JSON bytes to the Custom field instead of unmarshalling back into a structured object, so callers will receive []byte rather than the original data. Consider using json.Unmarshal to decode vuln.CustomVulnData into an interface{} before assignment.

Suggested change

	vulnCustom = any(vuln.CustomVulnData)
	}
	var advisoryCustom any
	if len(vuln.CustomAdvisoryData) > 0 {
	advisoryCustom = any(vuln.CustomAdvisoryData)
	if err := json.Unmarshal(vuln.CustomVulnData, &vulnCustom); err != nil {
	log.Logger.Warnf("Failed to unmarshal CustomVulnData: %v", err)
	}
	}
	var advisoryCustom any
	if len(vuln.CustomAdvisoryData) > 0 {
	if err := json.Unmarshal(vuln.CustomAdvisoryData, &advisoryCustom); err != nil {
	log.Logger.Warnf("Failed to unmarshal CustomAdvisoryData: %v", err)
	}

[Copilot]

[nitpick] Consider adding a comment above these fields to clarify that they now carry JSON-encoded data as raw bytes, and clients must JSON-decode them to retrieve the original structure.

[knqyf263]

Please update the PR description so we can understand why this change is needed.

[tonaim]

Please update the PR description so we can understand why this change is needed.

Updated the description details.

[knqyf263]

Language Agnostic
Raw JSON bytes are easily consumed by non-Go clients without importing Protobuf Struct libraries.

Are you sure it's language agnostic? We use protobuf anyway.

Removing structpb.Value avoids heavy reflection and descriptor overhead, streamlining serialization on both server and client.

This PR is just migrating to json.Marshal. It should explain why it improves performance.

[tonaim]

Language Agnostic
Raw JSON bytes are easily consumed by non-Go clients without importing Protobuf Struct libraries.

Are you sure it's language agnostic? We use protobuf anyway.

You’re right that the envelope is still a protobuf Vulnerability message, so every consumer will continue to rely on Protobuf‐generated stubs.
The point I was trying to make is that parsing the custom blob itself no longer requires the extra Protobuf helper types (google.protobuf.Struct|Value) on the client side.

Removing structpb.Value avoids heavy reflection and descriptor overhead, streamlining serialization on both server and client.

This PR is just migrating to json.Marshal. It should explain why it improves performance.

So, what was slow with structpb.NewStruct
structpb.NewStruct recursively reflects over every element to wrap it in *structpb.Value allocations.
Reflection in Go is orders of magnitude slower than regular marshaling and adds a lot of heap churn. Found a online article .

To support that having []byte is advantageous, I wrote small benchmarking program.

var oneCustom = map[string]interface{}{
	"id":       "CVE-2025-1234",
	"severity": "HIGH",
	"vendor": map[string]interface{}{
		"name":    "Acme Corp",
		"patched": true,
	},
	"cvss": map[string]interface{}{
		"score": 9.8,
		"vector": map[string]interface{}{
			"attackVector": "NETWORK",
			"scope":        "CHANGED",
		},
	},
}

// Generate **10,000** entries to simulate Trivy scan volume
var dataset = make([]map[string]interface{}, 10000)

func init() {
	for i := 0; i < len(dataset); i++ {
		// You can also clone oneCustom to avoid sharing pointers, but for benchmark it's OK
		dataset[i] = oneCustom
	}
}

// Old method: structpb.NewStruct
func BenchmarkStructPB_Large(b *testing.B) {
	for i := 0; i < b.N; i++ {
		for _, entry := range dataset {
			_, err := structpb.NewStruct(entry)
			if err != nil {
				b.Fatal(err)
			}
		}
	}
}

// New method: json.Marshal
func BenchmarkMarshalBytes_Large(b *testing.B) {
	for i := 0; i < b.N; i++ {
		for _, entry := range dataset {
			_, err := json.Marshal(entry)
			if err != nil {
				b.Fatal(err)
			}
		}
	}
}

go test -bench=. -benchmem -run=^$ benchmark/bench_convert_test.go

BenchmarkStructPB_Large-10         2     71721300 ns/op   36098240 B/op  470122 allocs/op
BenchmarkMarshalBytes_Large-10     3     41519400 ns/op   16429312 B/op  240230 allocs/op

[knqyf263]

Raw JSON bytes are easily consumed by non-Go clients without importing Protobuf Struct libraries.

We still need a protobuf library to parse the HTTP response. Could you please explain in more detail what this means?

The point I was trying to make is that parsing the custom blob itself no longer requires the extra Protobuf helper types (google.protobuf.Struct|Value) on the client side.

If this is what you mean, the above explanatin doesn't seem accurate.

[tonaim]

We still need a protobuf library to parse the HTTP response. Could you please explain in more detail what this means?

You’re absolutely right that the outer envelope remains a normal protobuf Vulnerability message, so any client that speaks gRPC will keep using its protobuf runtime to read the response.

The generated stub contains a plain bytes field, which every language’s core protobuf runtime already knows how to handle. No Struct/Value helpers required.

I am rephrasing the description