Skip to content

[Server] Upgrade to v1beta1 schema. #484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
May 5, 2024
Merged

[Server] Upgrade to v1beta1 schema. #484

Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
e98053e
change location for types package
MUzairS15 Mar 21, 2024
deff7f6
implement Create function for entities
MUzairS15 Mar 21, 2024
1735863
add registrant/host as first class attribute for models
MUzairS15 Mar 21, 2024
02ec85d
complete v1beta1 for model
MUzairS15 Mar 21, 2024
21b9b46
complete v1beta1 for component
MUzairS15 Mar 21, 2024
35d7e3e
update as per latest changes to schema
MUzairS15 Mar 22, 2024
335ca45
reorder
MUzairS15 Mar 25, 2024
2b9672b
implement updateStatus for all entites
MUzairS15 Mar 25, 2024
75a08ac
implement Get function for comp
MUzairS15 Mar 25, 2024
3446a64
implement Get function for relationships
MUzairS15 Mar 25, 2024
c524f5a
implement Get function for relationships
MUzairS15 Mar 25, 2024
0486120
delete v1alpha1
MUzairS15 Mar 25, 2024
ef4981f
add getters to entity filter
MUzairS15 Mar 25, 2024
d56ae45
caategory
MUzairS15 Mar 25, 2024
510b514
finalize changes and package structure:
MUzairS15 Mar 26, 2024
7c35024
move filters to their versioned packages
MUzairS15 Mar 26, 2024
b8c056a
remove unneeded code
MUzairS15 Mar 26, 2024
ddd4ce3
run gofmt
MUzairS15 Mar 26, 2024
b3d02d1
add hostID as param in interface function
MUzairS15 Mar 26, 2024
89313ef
cleanup:
MUzairS15 Mar 26, 2024
1e6a189
fix tags
MUzairS15 Mar 29, 2024
95dbd29
register models in registres table, fix tags and sql
MUzairS15 Mar 31, 2024
2302b3f
update model unique hash generation
MUzairS15 Apr 3, 2024
7a8ff4e
finalize changes
MUzairS15 Apr 7, 2024
3911988
finalize changes
MUzairS15 Apr 30, 2024
8c73a0d
fix merge conflicts
MUzairS15 Apr 30, 2024
e02d271
remove uneeded constructs
MUzairS15 Apr 30, 2024
6e5402c
run gofmt
MUzairS15 Apr 30, 2024
e7a46c6
preload category and initialize slice with cap instead of len
MUzairS15 May 3, 2024
cb90654
rmv logs
MUzairS15 May 3, 2024
2b55870
return comps and rels for model based on filter conds
MUzairS15 May 5, 2024
fb75883
initialize result slice
MUzairS15 May 5, 2024
ab974c8
Merge branch 'master' into MUzairS15/feat/v1beta1
leecalcote May 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions generators/artifacthub/package.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"strings"
"time"

"github.com/layer5io/meshkit/models/meshmodel/core/v1alpha1"
"github.com/layer5io/meshkit/models/meshmodel/core/v1beta1"
"github.com/layer5io/meshkit/utils"
"github.com/layer5io/meshkit/utils/component"
"github.com/layer5io/meshkit/utils/manifests"
Expand Down Expand Up @@ -36,8 +36,8 @@ func (pkg AhPackage) GetVersion() string {
return pkg.Version
}

func (pkg AhPackage) GenerateComponents() ([]v1alpha1.ComponentDefinition, error) {
components := make([]v1alpha1.ComponentDefinition, 0)
func (pkg AhPackage) GenerateComponents() ([]v1beta1.ComponentDefinition, error) {
components := make([]v1beta1.ComponentDefinition, 0)
// TODO: Move this to the configuration
crds, err := manifests.GetCrdsFromHelm(pkg.ChartUrl)
if err != nil {
Expand All @@ -57,7 +57,7 @@ func (pkg AhPackage) GenerateComponents() ([]v1alpha1.ComponentDefinition, error
comp.Model.Metadata["source_uri"] = pkg.ChartUrl
comp.Model.Version = pkg.Version
comp.Model.Name = pkg.Name
comp.Model.Category = v1alpha1.Category{
comp.Model.Category = v1beta1.Category{
Name: "",
}
comp.Model.DisplayName = manifests.FormatToReadableString(comp.Model.Name)
Expand Down
2 changes: 1 addition & 1 deletion generators/artifacthub/package_manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package artifacthub
import (
"fmt"

"github.com/layer5io/meshkit/models"
"github.com/layer5io/meshkit/generators/models"
)

type ArtifactHubPackageManager struct {
Expand Down
2 changes: 1 addition & 1 deletion generators/artifacthub/package_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ func TestGetChartUrl(t *testing.T) {
}
byt, _ := json.MarshalIndent(comp, "", "")

f, err := os.Create(dirName + "/" + comp.Kind + ".json")
f, err := os.Create(dirName + "/" + comp.Component.Kind + ".json")
if err != nil {
t.Errorf("error create file : %v", err)
continue
Expand Down
2 changes: 1 addition & 1 deletion generators/generator.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (

"github.com/layer5io/meshkit/generators/artifacthub"
"github.com/layer5io/meshkit/generators/github"
"github.com/layer5io/meshkit/models"
"github.com/layer5io/meshkit/generators/models"
"github.com/layer5io/meshkit/utils"
)

Expand Down
9 changes: 4 additions & 5 deletions generators/github/git_repo.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,14 @@ package github
import (
"bufio"
"fmt"
"github.com/layer5io/meshkit/generators/models"
"github.com/layer5io/meshkit/utils"
"github.com/layer5io/meshkit/utils/helm"
"github.com/layer5io/meshkit/utils/walker"
"net/url"
"os"
"path/filepath"
"strings"

"github.com/layer5io/meshkit/models"
"github.com/layer5io/meshkit/utils"
"github.com/layer5io/meshkit/utils/helm"
"github.com/layer5io/meshkit/utils/walker"
)

type GitRepo struct {
Expand Down
8 changes: 4 additions & 4 deletions generators/github/package.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"bytes"
"os"

"github.com/layer5io/meshkit/models/meshmodel/core/v1alpha1"
"github.com/layer5io/meshkit/models/meshmodel/core/v1beta1"
"github.com/layer5io/meshkit/utils"
"github.com/layer5io/meshkit/utils/component"
"github.com/layer5io/meshkit/utils/manifests"
Expand All @@ -23,8 +23,8 @@ func (gp GitHubPackage) GetVersion() string {
return gp.version
}

func (gp GitHubPackage) GenerateComponents() ([]v1alpha1.ComponentDefinition, error) {
components := make([]v1alpha1.ComponentDefinition, 0)
func (gp GitHubPackage) GenerateComponents() ([]v1beta1.ComponentDefinition, error) {
components := make([]v1beta1.ComponentDefinition, 0)

data, err := os.ReadFile(gp.filePath)
if err != nil {
Expand All @@ -49,7 +49,7 @@ func (gp GitHubPackage) GenerateComponents() ([]v1alpha1.ComponentDefinition, er
comp.Model.Metadata["source_uri"] = gp.SourceURL
comp.Model.Version = gp.version
comp.Model.Name = gp.Name
comp.Model.Category = v1alpha1.Category{
comp.Model.Category = v1beta1.Category{
Name: "",
}
comp.Model.DisplayName = manifests.FormatToReadableString(comp.Model.Name)
Expand Down
2 changes: 1 addition & 1 deletion generators/github/package_manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package github
import (
"net/url"

"github.com/layer5io/meshkit/models"
"github.com/layer5io/meshkit/generators/models"
"github.com/layer5io/meshkit/utils/walker"
)

Expand Down
4 changes: 2 additions & 2 deletions generators/github/package_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,9 @@ func TestGenerateCompFromGitHub(t *testing.T) {
}
byt, _ := json.MarshalIndent(comp, "", "")

f, err := os.Create(fmt.Sprintf("%s/%s%s", dirName, comp.Kind, ".json"))
f, err := os.Create(fmt.Sprintf("%s/%s%s", dirName, comp.Component.Kind, ".json"))
if err != nil {
t.Errorf("error creating file for %s: %v", comp.Kind, err)
t.Errorf("error creating file for %s: %v", comp.Component.Kind, err)
continue
}
_, _ = f.Write(byt)
Expand Down
2 changes: 1 addition & 1 deletion generators/github/scheme_interface.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package github
import (
"net/url"

"github.com/layer5io/meshkit/models"
"github.com/layer5io/meshkit/generators/models"
)

type DownloaderScheme interface {
Expand Down
2 changes: 1 addition & 1 deletion generators/github/url.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"path/filepath"
"strings"

"github.com/layer5io/meshkit/models"
"github.com/layer5io/meshkit/generators/models"
"github.com/layer5io/meshkit/utils"
"github.com/layer5io/meshkit/utils/helm"
)
Expand Down
4 changes: 2 additions & 2 deletions models/interfaces.go → generators/models/interfaces.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package models

import "github.com/layer5io/meshkit/models/meshmodel/core/v1alpha1"
import "github.com/layer5io/meshkit/models/meshmodel/core/v1beta1"

// anything that can be validated is a Validator
type Validator interface {
Expand All @@ -11,7 +11,7 @@ type Validator interface {
// system's capabilities in Meshery
// A Package should have all the information that we need to generate the components
type Package interface {
GenerateComponents() ([]v1alpha1.ComponentDefinition, error)
GenerateComponents() ([]v1beta1.ComponentDefinition, error)
GetVersion() string
}

Expand Down
2 changes: 2 additions & 0 deletions models/meshmodel/core/policies/data_models.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,5 @@ type NetworkPolicyRegoResponse struct {
ServicePodRelationships []RelationObject `json:"service_pod_relationships,omitempty"`
ServiceDeploymentRelationships []RelationObject `json:"service_deployment_relationships,omitempty"`
}

// Add response struct based on schema for all relationships evaluations. binding, inventory, network...
14 changes: 11 additions & 3 deletions models/meshmodel/core/policies/rego_policy_relationship.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ package policies
import (
"context"
"fmt"
"io/fs"
"path/filepath"

"github.com/layer5io/meshkit/models/meshmodel/core/v1alpha1"
"github.com/layer5io/meshkit/models/meshmodel/registry"
"github.com/layer5io/meshkit/models/meshmodel/registry/v1alpha2"
"github.com/layer5io/meshkit/utils"
"github.com/open-policy-agent/opa/rego"
"github.com/open-policy-agent/opa/storage"
Expand All @@ -26,7 +28,10 @@ func NewRegoInstance(policyDir string, regManager *registry.RegistryManager) (*R
var store storage.Store

ctx := context.Background()
registeredRelationships, _, _ := regManager.GetEntities(&v1alpha1.RelationshipFilter{})
registeredRelationships, _, _, err := regManager.GetEntities(&v1alpha2.RelationshipFilter{})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a concern separate from v1beta1 for Components.

Relationships v1alpha2 was to have been completely previously, right? Should this go forward independently?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, v1alpha2 was completed, but the update was done inplace, i.e. v1alpha1 struct were simply updated in place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that work was done well and was completed, would we expect that the only change necessary here be that of a swap from “1 “to “2”?

if err != nil {
return nil, err
}

if len(registeredRelationships) > 0 {
data := map[string]interface{}{
Expand All @@ -51,7 +56,10 @@ func (r *Rego) RegoPolicyHandler(regoQueryString string, designFile []byte) (int
}
regoEngine, err := rego.New(
rego.Query(regoQueryString),
rego.Load([]string{r.policyDir}, nil),
rego.Load([]string{r.policyDir}, func(abspath string, info fs.FileInfo, depth int) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add comments here and update Meshery Docs.

Is this loading Rego policy from each individual model?

extension := filepath.Ext(abspath)
return extension == "rego"
}),
rego.Store(r.store),
rego.Transaction(r.transaction),
).PrepareForEval(r.ctx)
Expand Down
15 changes: 0 additions & 15 deletions models/meshmodel/core/types/types.go
View file
Open in desktop

This file was deleted.

83 changes: 0 additions & 83 deletions models/meshmodel/core/v1alpha1/category.go
View file
Open in desktop

This file was deleted.

Loading
Loading