Skip to content

sed illegal option -- - in build.sh on macOS #140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tomkyle opened this issue Mar 14, 2018 · 2 comments
Closed

sed illegal option -- - in build.sh on macOS #140

tomkyle opened this issue Mar 14, 2018 · 2 comments

Comments

@tomkyle
Copy link
Contributor

tomkyle commented Mar 14, 2018

When running bin/build/build.sh, I get this “illegal option” message during the excution of the apply_pattern function:

sed: illegal option -- -
usage: […]
 [✖] Create '.htaccess'
...
 [✖] Create '.htaccess' fixture

As far as I can see this a sed issue on macos. Workaround: Replace system's sed with a more modern one, gnu-sed. The build.sh then succeeded:

$ brew install gnu-sed --with-default-names
...
🍺  /usr/local/Cellar/gnu-sed/4.4: 10 files ...
$ bin/build/build.sh
 [✔] Clean
 [✔] Create '.htaccess'
 [✔] Create '.htaccess' fixture

Question to sed magicians: Is there a way to avoid such additional installs?

My setup: OSX 10.11.6, homebrew bash4

@tomkyle
Copy link
Contributor Author

tomkyle commented Mar 20, 2018

Caution! The above workaround may break other scripts.
An example on my computer are Frend Weinhaus' imagemagick scripts which produced "doubled" output – each line shows twice – after replacing sed with the above homebrew method. This is because Fred Weinhaus uses sed a lot to produce formatted output (such as help text)

After unlinking the homebrew GNU sed (i.e. restoring the default macOS BSD sed) everything worked fine again. Linking again: Output doubled.

I am not sed expert enough to tell which command flags are responsible for that.

Anyway, I found another solution: tweaking the sed parameters order.

Here the original apply_pattern function:

apply_pattern() {
    sed -e "s/%FilesMatchPattern%/$( \
        cat "${repo_root}/src/files_match_pattern" | \
        sed '/^#/d' | \
        tr -s '[:space:]' '|' | \
        sed 's/|$//' \
    )/g" --in-place "$1"
}

I found if one replaces the --in-place option with shorter -i "", it seems to work fine:

apply_pattern () {
    sed -e "s/%FilesMatchPattern%/$( \
        cat "${repo_root}/src/files_match_pattern" | \
        sed '/^#/d' | \
        tr -s '[:space:]' '|' | \
        sed 's/|$//' \
    )/g" -i "" "$1"
}

Please, could someone with sed experience prove this? Thanks!

@LeoColomb
Copy link
Member

Yup, your solution seems to be good enough! 🎉

LeoColomb pushed a commit that referenced this issue Apr 4, 2018
* Partials configuration

This configuration file defines which .htaccess module partials are enabled or disabled.

* Added requirements

Assoc. arrays require Bash4

* Read new conf file

The default config file can be overridden by script parameter.

* New function insert_if

Wraps the former explicit calls to insert_file and insert_file_comment_out, depending of the partials configuration.

* Modern shebang

The assoc. partials array defined in partials.conf requires Bash4. Mac users will have it installed with homebrew, so we have to specify that interpreter.

* Restored fixtures creation

* Switch back to old shebang

* New conf syntax

Each entry consists of a "keyword" and "filename", separated by space chars.

- Keyword is "title", "enable", "disable", or "omit"
- Filenames refer to module partials

* Conf file for fixtures

The same as for partials.conf, but with modules enabled always.

* Work with new conf files

The create_* functions read their respective conf file line by line, ignoring blank lines and those with leading # sign.

- "title" items are used for header bars
- "enabled" items are, well, enabled,
- "disables" items are commented out,
- "omit" items do not show up in output at all

Bash4 features like associative arrays (and thus, "modern" shebang") are not required any longer.

* Restored output readabilty

* Introduce new variables for output files

* Pass output file with parameter

Preparing the uniting of create_htaccess and create_htaccess_fixture, the output file is passed as second parameter to these functions.

* Removed create_htaccess_fixture

Being nearly identical with create_htaccess, it became surfluous since we started to pass conf and output file as parameters.

* Renamed variables

- partials_*: its not about the partials, it's about htaccess
- fixtures_ *: it's about one single fixture

* Renamed conf file

Once developer maintains his own config file, it should have a self-explanatory name

* Rename fixture conf file

…filenames be self-explanatory

* Move fixture config

Stuff belonging to test domain should go into appropriate directory, hence moved into "test"

* Adapt filename changes

* Tweak clean function

Since output files may now potentially be stored anywhere, clean function should not always remove the whole dist/ folder. – To make sure there's a dist folder in the end, mkdir call now gets the -p option to avoid “File exists" message

* Local variables be declared

* Improved output directory creation

htaccess files are potentially stored anywhere, so mkdir should make sure there is a directory to write in.

* fixture build as command with arguments

reflect changes on ci build commands

* improve output by printing variables

* http -> https is omitted for tests

* cross-origin requests is omitted for tests

* remove unused global var

* Changed parameter order

Output file first, config second. This allows custom output locations from a default config.

* New variable repo_root

This makes it possible to run the script from any custom working directory.

* File check: $PWD vs. repo_root

First seek partials under $PWD, fall back to repo_root

* Error msg when partial file can not be found

* Default output location now $PWD

* Default output location now $PWD

* Introduce the dist folder as look-worthy place

* First draft for a custom-build section

* Fixed wrong variable name

* Improved "custom build" docs

- Tried to improve the rationale introduction to custom builds
- Added jump links/deep links here and there
- Clarified sentences, hopefully

* Added a first test script

This is very basic and tests for the four possible parameter combinations.
Could somebedy help refining the assert_file_contains() function? See comments in source.

* Fixed a bug in configuration file logic

When user passes a custom config file which in fact does _not_ exist, the script used to silently fallback to default configuration. Now it raises an error exit.

* Update title line

* Mock conf for test_build.sh

* Added conf keyword test

The new test builds a htaccess from the new htaccess_test_build.conf file, using mock partials with basic strings. Assertions used:

- Title line present, in uppercase?
- "AAAAAA" present?
- "BBBBBB" commented-out?
- "CCCCCC" not present?

How to avoid the hard-coded strings? Ideas welcome…

* +Output readability

* Fix macOS BSD sed issue, #140

The BSD sed on macOS does not like "--in-place", instead use _-i ""_

* Remove surfluous whitespace

* Source order and readability

* Rename test script

Former test_build.sh tests user-defined output and configuration files, so better rename to test_userbuild

* Integrate test_userbuild.sh

'npm run test' now performs user build test as well

* Return correct exit code

main() now returns exit/return code of create_htaccess. Useful for error exit due to invalid keywords.

* Headline wording

* Amend mock files

Create pair of valid and invalid mock (user) configuration files

* Added test for invalid keywords

* Revert return values thingy, as it leads Travis to fail

When the config parser meets an invalid keyword, the function should not return 1 but rather exit 1 – so's no discussion about how to exit.

* Docblock and whitespace improvements

* Print Summary

* Bullet proofing: temp files and backup

The main faunction used to remove any existing .htaccess before creating the new one. This is risky, if the build process fails somehow.

1. Now, the new .htaccess is built in a temporary directory first, and moved to its target directory on success.

2. Any existing .htaccess will be backuped with ~ suffix, rather than being overwritten.

* Docs draft: update shell output example

* Docs: improved wording

“sentences without attribution are often better”. Yup.

* Removed surfluous function

* Removed recursive flag

…this should protect directories from being deleted. Thx @LeoColomb

* Removed surfluous function

Clean is not used any longer.

* Simplify things

- 'rm -Rf' fails quietly, so no [ -d ... ] check necessary
- Temp file preparations better be handled transparently, leveraging output.

* Removed trap

There's only one file written during script, which is moved to target on success. So the directory is empty anyway, no need to delete something any longer – particularly in the rare case that $temp_directory inbetween has become "/" or sort of…

* Remove temp directory.

A temp *file* is plenty enough.

* Remove printing exit code output after create function

* reduce complexity

remove temp file (sorry) only use a backup file to allow a revert

* lint with shellcheck

* remove node dependency when building

* clean up command and build output

* flat build bin file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants