shebang support

[qoh]

When using deno to write applications or tools, it is desirable to execute those directly as binary [argv], rather than deno path/to/binary.ts [argv].

It could be possible to build custom versions of deno with code embedded, but I won't cover that here (see #986).
In this issue I'm looking at using deno as an executable interpreter with a shebang, such as this file binary containing:

#!/usr/bin/env deno
import { main } from "../src/main";
main();

This currently does not work, for two separate reasons:

• If the executable name is extension-less (just binary rather than binary.ts), deno will not load it.
• Even if it has a valid extension, deno will include the shebang line in the source, causing a syntax error.

Ultimately this means that it doesn't seem like you can do this with deno directly, rather you need something like a wrapper shell script or binary around invoking deno.

Ignoring the shebang line seems entirely reasonable, as that is not valid JavaScript | TypeScript in the first place.

I'm not sure what would be an acceptable solution regarding the extension.
Node just interprets files without an extension as .js files.

[rsp]

I would assume that deno would use TypeScript for extensionless file names.

But I'm not really sure what about extensions in file names with a shebang line.

Relying on the file name in this particular case is tricky because it's often the case that an executable may be symlinked or renamed during installation. It's common in Node today with scripts installed with npm.

I don't want file extensions in my /usr/local/bin but if the file happens to be script.js that needs to be parsed as JS and not TS then script.js cannot be installed to /usr/bin as just "script".

The solutions I see are all ugly - maybe someone else has some better ideas.

1. use a switch in the shebang line

Works well for #!/usr/local/bin/deno --js but for #!/usr/bin/env deno you need hacks like this that I once needed to use feature flags in node:

#!/bin/sh
":" //#; exec /usr/bin/env node --harmony-proxies "$0" "$@"

2. use some magic comment

#!/usr/bin/env deno
// deno-language: js

or something like that. I don't like it really but being optional for cases when there is no file extension that would at least make it possible to write scripts to be installed as system commands.

3. use a special binary name:

#!/usr/bin/env deno
// ... ts code here

and

#!/usr/bin/env denojs
// ... js code here

This is the last one that I thought about and doesn't seem that bad. But it would mean either installing two binaries or one binary and a symlink.

All of the above have some issues, meybe someone has a better idea.

[kitsonk]

All JS currently passes through the TypeScript compiler currently, which is likely to persist, especially if we end up supporting type checking of JS via JSDocs. I do not think treating the vast majority of extensionless files as TypeScript though would create any userland problems, as the compiler is in a very loose mode, and any JavaScript that would be invalid TypeScript is probably a bad idea anyways. It would be best to treat it as TypeScript and then only deal with it if it actually causes problems we can't solve.

[rsp]

@kitsonk I don't think it's that simple. Even this trivial example:

let x = 1;
x = 'a';
console.log(x);

passes as JS but not as TS - both in Deno itself - see:

Update:

I know that in theory TS is syntactically a superset of JS, but even though all JS code can in principle be parsed as TS, it doesn't necessarily mean that it doesn't violate the type system.

To make it work we would need to use any as the type for all variables that are not explicitly typed instead of using the type inference that is used everywhere else, and we would need to do it for all TS code if we want any JS code to be parsed as TS with no type errors.

As for me, I'd rather go the other way and compile the TS code with --strict by default (with a --loose switch in deno instead of non-strict by default) but that's just me and I know it will probably be non-strict by default (with hopefully a short switch like -s for strict parsing) but I hope Deno will not be even less strict than the default non-strict tsc mode, by switching off the type inference.

[kitsonk]

@rsp there are a few issues here that then get all coupled together that should be solved before we try to solve JavaScript support for shebang:

• Supporting ES Modules natively (Native support for loading ES Modules #975)
• Supporting the privileged side determining the media type (Handling of urls which don't have file extension #702)
• Supporting CheckJS (Support CheckJS #976)

(A couple of them are things that have been talked about but I realised didn't have issues)

My opinion is that we could/should implement shebang support for TypeScript only and then deliver the right solution for JavaScript as an incremental change.

[TehShrike]

Perhaps "how should deno handle extensionless files" should be a separate issue from "deno should ignore shebangs"?

[pepa65]

The issue as formulated is about Shebang support. Because scripts need deno run <scriptname> to be executed, this is a problem. a Shebangline #!/usr/bin/env deno run doesn't work, because env will look for a binary called deno run. Most languages accomodate this behaviour, and allow execution without subcommands.
If deno would drop the requirement for subcommand run (like make it a default if not given), then the shebang could just be #!/usr/bin/env deno and it would work. Right now you get error: Found argument '<scriptname>' which wasn't expected, or isn't valid in this context...

[encendre]

The issue as formulated is about Shebang support. Because scripts need deno run <scriptname> to be executed, this is a problem. a Shebangline #!/usr/bin/env deno run doesn't work, because env will look for a binary called deno run. Most languages accomodate this behaviour, and allow execution without subcommands.
If deno would drop the requirement for subcommand run (like make it a default if not given), then the shebang could just be #!/usr/bin/env deno and it would work. Right now you get error: Found argument '<scriptname>' which wasn't expected, or isn't valid in this context...

From man env

   -S/--split-string usage in scripts
       The  -S  option  allows specifying multiple parameters in a script.  Running a script named 1.pl
       containing the following first line:

              #!/usr/bin/env -S perl -w -T
              ...

       Will execute perl -w -T 1.pl .

       Without the '-S' parameter the script will likely fail with:

              /usr/bin/env: 'perl -w -T': No such file or directory

       See the full documentation for more details.

The following works for me

#!/usr/bin/env -S deno run --allow-net

fetch("https://api.ipify.org?format=json")
  .then(v => v.json())
  .then(console.log)

[nayeemrmn]

If deno would drop the requirement for subcommand run (like make it a default if not given), then the shebang could just be #!/usr/bin/env deno and it would work.

You still wouldn't be able to provide runtime flags like permissions :) This is a damning problem with the way shebangs work on Linux which individual CLIs shouldn't cripple themselves to solve, and the env -S workaround above is widely shipped in modern distro versions.

[pepa65]

Yes, that does work in the very latest coreutils (8.30+ has it). So Ubuntu 20.04 is fine, but everyone on an older LTS, of most distros won't have that, the -S flag is new.

[pepa65]

individual CLIs shouldn't cripple themselves to solve

All the older languages/platforms that had to work with env without -S for many years all allow to be called without additional parameters, like: julia script.jl or node script.js.
I appreciate deno wants to do things differently, but differently is not always better or more functional when dealing with different environments and use cases.

[nayeemrmn]

All the older languages/platforms that had to work with env without -S for many years all allow to be called without additional parameters, like: julia script.jl or node script.js.

Wrong, they also have runtime flags which aren't always optional depending on the use case. I don't think having runtime flags is "doing things differently".

[pepa65]

Most binaries have all kinds of runtime flags. But having a subcommand like run is more unusual. Right now you would need a helper script to run a deno script (because env is what enables it to be a stand-alone executable) on slightly older platforms (most of the machines I work with are not yet into the 2020s yet...).

[nayeemrmn]

Most binaries have all kinds of runtime flags. But having a subcommand like run is more unusual.

The point was that the runtime flags pose the same issue, so we may as well have a modern subcommand-based CLI. The subcommand issue has nothing specifically to do with shebang compatibility.

[pepa65]

But runtime flags can be optional, but leaving out the subcommand is now causing an error. That is not the common case for most other scripting engines.
Look, I want to use deno, and I'll probably still use it when this doesn't get fixed, but I don't understand why you would handicap yourselves right out of the gate. What do you win by requiring run??

[nayeemrmn]

But runtime flags can be optional

In the tiny set of cases where no permissions are required? Again, requiring run is of no particular relevance to this issue.

What do you win by requiring run??

You can search for and read through issues related to that if you're curious. In short, the deno script.ts shorthand creates ambiguities because subcommands are valid script names, it would be totally unsound.

env -S exists because it's been accepted that Linux shebangs are the problem and not being able to specify runtime flags is an unacceptable constraint. It wouldn't work for Deno whether run was required or not.

[pepa65]

That's why I am of the opinion that subcommands are not a good idea, because they could be taken as an input filename. I prefer commandline options where there are flags with - or -- and then parameters like filenames. I don't know why one would ever need subcommands, frankly. But I am sure you've had this discussion already, judging by your replies. The issue is closed already anyways...

[nayeemrmn]

I prefer commandline options where there are flags with - or -- and then parameters like filenames. I don't know why one would ever need subcommands, frankly.

Try to describe the myriad of relationships / exclusions / ordering semantics between such flags, and provide good help messages, without subcommands. Keep in mind that Node's CLI doesn't expose such a varied set of functions like repl vs fmt, and that npm is subcommand based.