Skip to content

fix: improve errors displayed on obs creation failure, do not fail clusterization #390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 10, 2025
+20 −17
Merged

fix: improve errors displayed on obs creation failure, do not fail clusterization #390

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
47cb18e
fix: improve errors displayed on obs creation failure, do not fail cl…
kristina-solovyova Mar 12, 2025
100b584
terraform-docs: automated action
github-actions[bot] Mar 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -459,14 +459,14 @@ proxy_url = VALUE
| <a name="input_deployment_storage_account_name"></a> [deployment\_storage\_account\_name](#input\_deployment\_storage\_account\_name) | Name of exising deployment storage account | `string` | `""` | no |
| <a name="input_enable_application_insights"></a> [enable\_application\_insights](#input\_enable\_application\_insights) | Enable Application Insights. | `bool` | `true` | no |
| <a name="input_function_access_restriction_enabled"></a> [function\_access\_restriction\_enabled](#input\_function\_access\_restriction\_enabled) | Allow public access, Access restrictions apply to inbound access to internal vent | `bool` | `false` | no |
| <a name="input_function_app_dist"></a> [function\_app\_dist](#input\_function\_app\_dist) | Function app code dist | `string` | `"release"` | no |
| <a name="input_function_app_dist"></a> [function\_app\_dist](#input\_function\_app\_dist) | Function app code dist | `string` | `"dev"` | no |
| <a name="input_function_app_identity_name"></a> [function\_app\_identity\_name](#input\_function\_app\_identity\_name) | The user assigned identity name for the function app (if empty - new one is created). | `string` | `""` | no |
| <a name="input_function_app_log_level"></a> [function\_app\_log\_level](#input\_function\_app\_log\_level) | Log level for function app (from -1 to 5). See https://github.com/rs/zerolog#leveled-logging | `number` | `1` | no |
| <a name="input_function_app_storage_account_container_prefix"></a> [function\_app\_storage\_account\_container\_prefix](#input\_function\_app\_storage\_account\_container\_prefix) | Weka storage account container name prefix | `string` | `"weka-tf-functions-deployment-"` | no |
| <a name="input_function_app_storage_account_prefix"></a> [function\_app\_storage\_account\_prefix](#input\_function\_app\_storage\_account\_prefix) | Weka storage account name prefix | `string` | `"weka"` | no |
| <a name="input_function_app_subnet_delegation_cidr"></a> [function\_app\_subnet\_delegation\_cidr](#input\_function\_app\_subnet\_delegation\_cidr) | Subnet delegation enables you to designate a specific subnet for an Azure PaaS service. | `string` | `"10.0.1.0/25"` | no |
| <a name="input_function_app_subnet_delegation_id"></a> [function\_app\_subnet\_delegation\_id](#input\_function\_app\_subnet\_delegation\_id) | Required to specify if subnet\_name were used to specify pre-defined subnets for weka. Function subnet delegation requires an additional subnet, and in the case of pre-defined networking this one also should be pre-defined | `string` | `""` | no |
| <a name="input_function_app_version"></a> [function\_app\_version](#input\_function\_app\_version) | Function app code version (hash) | `string` | `"bef288291e94411aa8c388613e355c4a"` | no |
| <a name="input_function_app_version"></a> [function\_app\_version](#input\_function\_app\_version) | Function app code version (hash) | `string` | `"239d469255a978f32c3795d70ed84706"` | no |
| <a name="input_get_weka_io_token"></a> [get\_weka\_io\_token](#input\_get\_weka\_io\_token) | The token to download the Weka release from get.weka.io. | `string` | `""` | no |
| <a name="input_hotspare"></a> [hotspare](#input\_hotspare) | Number of hotspares to set on weka cluster. Refer to https://docs.weka.io/overview/ssd-capacity-management#hot-spare | `number` | `1` | no |
| <a name="input_install_cluster_dpdk"></a> [install\_cluster\_dpdk](#input\_install\_cluster\_dpdk) | Install weka cluster with DPDK | `bool` | `true` | no |
Expand Down Expand Up @@ -530,10 +530,10 @@ proxy_url = VALUE
| <a name="input_subnet_prefix"></a> [subnet\_prefix](#input\_subnet\_prefix) | Address prefixes to use for the subnet | `string` | `"10.0.2.0/24"` | no |
| <a name="input_subscription_id"></a> [subscription\_id](#input\_subscription\_id) | The subscription id for the deployment. | `string` | n/a | yes |
| <a name="input_tags_map"></a> [tags\_map](#input\_tags\_map) | A map of tags to assign the same metadata to all resources in the environment. Format: key:value. | `map(string)` | `{}` | no |
| <a name="input_tiering_blob_obs_access_key"></a> [tiering\_blob\_obs\_access\_key](#input\_tiering\_blob\_obs\_access\_key) | The access key of the existing Blob object store container. | `string` | `""` | no |
| <a name="input_tiering_blob_obs_access_key"></a> [tiering\_blob\_obs\_access\_key](#input\_tiering\_blob\_obs\_access\_key) | The access key of the existing Blob object store container. If not provided, new obs will be created with given name (tiering\_obs\_name). | `string` | `""` | no |
| <a name="input_tiering_enable_obs_integration"></a> [tiering\_enable\_obs\_integration](#input\_tiering\_enable\_obs\_integration) | Determines whether to enable object stores integration with the Weka cluster. Set true to enable the integration. | `bool` | `false` | no |
| <a name="input_tiering_enable_ssd_percent"></a> [tiering\_enable\_ssd\_percent](#input\_tiering\_enable\_ssd\_percent) | When set\_obs\_integration is true, this variable sets the capacity percentage of the filesystem that resides on SSD. For example, for an SSD with a total capacity of 20GB, and the tiering\_ssd\_percent is set to 20, the total available capacity is 100GB. | `number` | `20` | no |
| <a name="input_tiering_obs_container_name"></a> [tiering\_obs\_container\_name](#input\_tiering\_obs\_container\_name) | Name of existing obs conatiner name | `string` | `""` | no |
| <a name="input_tiering_obs_container_name"></a> [tiering\_obs\_container\_name](#input\_tiering\_obs\_container\_name) | Name of existing obs container name. | `string` | `""` | no |
| <a name="input_tiering_obs_name"></a> [tiering\_obs\_name](#input\_tiering\_obs\_name) | Name of existing obs storage account | `string` | `""` | no |
| <a name="input_tiering_obs_start_demote"></a> [tiering\_obs\_start\_demote](#input\_tiering\_obs\_start\_demote) | Target tiering cue (in seconds) before starting upload data to OBS (turning it into read cache). Default is 10 seconds. | `number` | `10` | no |
| <a name="input_tiering_obs_target_ssd_retention"></a> [tiering\_obs\_target\_ssd\_retention](#input\_tiering\_obs\_target\_ssd\_retention) | Target retention period (in seconds) before tiering to OBS (how long data will stay in SSD). Default is 86400 seconds (24 hours). | `number` | `86400` | no |
Expand Down
21 changes: 12 additions & 9 deletions function-app/code/functions/clusterize/clusterize.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,18 @@ import (
"os"
"strconv"
"strings"
"weka-deployment/common"
"weka-deployment/functions/azure_functions_def"

"github.com/weka/go-cloud-lib/join"
"github.com/weka/go-cloud-lib/utils"

"github.com/lithammer/dedent"

"github.com/weka/go-cloud-lib/clusterize"
cloudCommon "github.com/weka/go-cloud-lib/common"
"github.com/weka/go-cloud-lib/functions_def"
"github.com/weka/go-cloud-lib/join"
"github.com/weka/go-cloud-lib/logging"
"github.com/weka/go-cloud-lib/protocol"
"github.com/weka/go-cloud-lib/utils"

"weka-deployment/common"
"weka-deployment/functions/azure_functions_def"
)

func GetObsScript(obsParams common.AzureObsParams) string {
Expand Down Expand Up @@ -105,13 +104,16 @@ func PrepareWekaObs(ctx context.Context, p *ClusterizationParams) (err error) {
}

if noExistingObs {
common.ReportMsg(ctx, p.Vm.Name, p.StateParams, "debug", "creating OBS as no access key was provided")

p.Obs.AccessKey, err = common.CreateStorageAccount(
ctx, p.SubscriptionId, p.ResourceGroupName, p.Location, p.Obs,
)
if err != nil {
err = fmt.Errorf("failed to create storage account: %w", err)
common.ReportMsg(ctx, p.Vm.Name, p.StateParams, "error", err.Error())
logger.Error().Err(err).Send()
return
return nil
}

if p.Obs.NetworkAccess == "Disabled" && p.CreateBlobPrivateEndpoint {
Expand All @@ -120,9 +122,10 @@ func PrepareWekaObs(ctx context.Context, p *ClusterizationParams) (err error) {

err = common.CreateStorageAccountBlobPrivateEndpoint(ctx, p.SubscriptionId, p.ResourceGroupName, p.Location, p.Obs.Name, endpointName, p.SubnetId, p.PrivateDNSZoneId)
if err != nil {
err = fmt.Errorf("failed to create private endpoint: %w", err)
err = fmt.Errorf("failed to create private endpoint for storage account: %w", err)
common.ReportMsg(ctx, p.Vm.Name, p.StateParams, "error", err.Error())
logger.Error().Err(err).Send()
return
return nil
}
}
}
Expand Down
8 changes: 4 additions & 4 deletions variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,13 +404,13 @@ variable "function_app_storage_account_container_prefix" {
variable "function_app_version" {
type = string
description = "Function app code version (hash)"
default = "bef288291e94411aa8c388613e355c4a"
default = "239d469255a978f32c3795d70ed84706"
}

variable "function_app_dist" {
type = string
description = "Function app code dist"
default = "release"
default = "dev"

validation {
condition = contains(["dev", "release"], var.function_app_dist)
Expand Down Expand Up @@ -500,12 +500,12 @@ variable "tiering_enable_ssd_percent" {
variable "tiering_obs_container_name" {
type = string
default = ""
description = "Name of existing obs conatiner name"
description = "Name of existing obs container name."
}

variable "tiering_blob_obs_access_key" {
type = string
description = "The access key of the existing Blob object store container."
description = "The access key of the existing Blob object store container. If not provided, new obs will be created with given name (tiering_obs_name)."
sensitive = true
default = ""
}
Expand Down