Skip to content

Commit 8340b0a

Browse files
committed
changes
1 parent a34ba75 commit 8340b0a

File tree

4 files changed

+13
-12
lines changed

4 files changed

+13
-12
lines changed

hadoop-tools/hadoop-azure/dev-support/testrun-scripts/config_support.sh renamed to hadoop-tools/hadoop-azure/dev-support/testrun-scripts/configsupport.sh

+8-8
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ if [ ! -f "$FILE" ]; then
2424
exit 1
2525
fi
2626

27-
if [[ "$1" != *.xml ]]; then
27+
if [[ "$FILE" != *.xml ]]; then
2828
echo "The file provided is not an XML file. Exiting...."
2929
exit 1
3030
fi
@@ -63,7 +63,7 @@ do
6363
done
6464

6565
# Mapping for renaming configurations
66-
declare -A rename_configs_map=(
66+
declare -A renameConfigsMap=(
6767
["autothrottling.enable"]="enable.autothrottling" #fs.azure.autothrottling.enable to fs.azure.enable.autothrottling
6868
["rename.dir"]="rename.key" # fs.azure.atomic.rename.dir to fs.azure.atomic.rename.key
6969
["block.blob.buffered.pread.disable"]="buffered.pread.disable" #fs.azure.block.blob.buffered.pread.disable to fs.azure.buffered.pread.disable
@@ -72,14 +72,14 @@ declare -A rename_configs_map=(
7272
)
7373

7474
# Configs not supported in ABFS
75-
unsupported_configs_list=(
75+
unsupportedConfigsList=(
7676
"fs.azure.page.blob.dir"
7777
"fs.azure.block.blob.with.compaction.dir"
7878
"fs.azure.store.blob.md5"
7979
)
8080

8181
# Configurations not required in ABFS Driver and can be removed
82-
obsolete_configs_list=(
82+
obseleteConfigsList=(
8383
"azure.authorization" #fs.azure.authorization, fs.azure.authorization.caching.enable , fs.azure.authorization.caching.maxentries, fs.azure.authorization.cacheentry.expiry.period, fs.azure.authorization.remote.service.urls
8484
"azure.selfthrottling" #fs.azure.selfthrottling.enable, fs.azure.selfthrottling.read.factor, fs.azure.selfthrottling.write.factor
8585
"azure.saskey" #fs.azure.saskey.cacheentry.expiry.period , fs.azure.saskey.usecontainersaskeyforallaccess
@@ -107,7 +107,7 @@ obsolete_configs_list=(
107107
)
108108

109109
# Stop the script if any unsupported config is found
110-
for key in "${unsupported_configs_list[@]}"; do
110+
for key in "${unsupportedConfigsList[@]}"; do
111111
if grep -q "$key" "$OUTPUT_FILE"; then
112112
echo "Remove the following configuration from file and rerun: '$key'"
113113
failure=true
@@ -122,14 +122,14 @@ if [ "$failure" = true ]; then
122122
fi
123123

124124
# Renaming the configs
125-
for old in "${!rename_configs_map[@]}"; do
126-
new="${rename_configs_map[$old]}"
125+
for old in "${!renameConfigsMap[@]}"; do
126+
new="${renameConfigsMap[$old]}"
127127
xmlstarlet ed -L -u "//property/name[contains(., '$old')]" -x "concat(substring-before(., '$old'),
128128
'$new', substring-after(., '$old'))" "$OUTPUT_FILE"
129129
done
130130

131131
# Remove the obsolete configs
132-
for key in "${obsolete_configs_list[@]}"; do
132+
for key in "${obseleteConfigsList[@]}"; do
133133
xmlstarlet ed -L -d "//property[name[contains(text(), '$key')]]" "$OUTPUT_FILE"
134134
done
135135

hadoop-tools/hadoop-azure/src/site/markdown/index.md

+1
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ See also:
3636
* [FNS (non-HNS)](./fns_blob.html)
3737
* [Legacy-Deprecated-WASB](./wasb.html)
3838
* [Testing](./testing_azure.html)
39+
* [WASB Migration Config Support](./wasbToAbfsMigration.html)
3940

4041
## <a name="features"></a> Features of the ABFS connector.
4142

hadoop-tools/hadoop-azure/src/site/markdown/wasbToAbfsMigration.md

+4-3
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
limitations under the License. See accompanying LICENSE file.
1313
-->
1414

15-
# WASB to ABFS Configuration Conversion Script
15+
# WASB to ABFS Driver Configuration Conversion Script
1616

1717
To support customer onboard for migration from WASB to ABFS driver, we've
1818
introduced a script to help with the configuration changes required
@@ -22,9 +22,10 @@ for the same.
2222

2323
ABFS driver has now built support for
2424
FNS accounts (over BlobEndpoint that WASB Driver uses) using the ABFS scheme.
25+
Refer to: [ABFS Driver for Namespace Disabled Accounts](./fns_blob.html) for more details.
2526

2627
The legacy WASB driver has been **deprecated** and is no longer recommended for
27-
use.
28+
use. Refer to: [WASB Deprecation](./wasb.html) for more details.
2829
It's highly recommended for current WASB Driver users to migrate to ABFS driver,
2930
the only Microsoft driver for Azure Storage.
3031

@@ -72,7 +73,7 @@ with the ABFS driver.
7273
Example to run the script:
7374

7475
```shell
75-
./config_support.sh <path-to-xml-file>
76+
./configsupport.sh <path-to-xml-file>
7677
```
7778

7879
For any queries or support, kindly reach out to us at '[email protected]'

hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelegationSAS.java

-1
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,6 @@ public void testReadAndWrite() throws Exception {
218218
@Test
219219
public void checkExceptionForRenameOverwrites() throws Exception {
220220
final AzureBlobFileSystem fs = getFileSystem();
221-
222221
Path src = new Path("a/b/f1.txt");
223222
Path dest = new Path("a/b/f2.txt");
224223
touch(src);

0 commit comments

Comments
 (0)