Skip to content

Parcel 2: multi entry/target builds #3302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
devongovett opened this issue Aug 2, 2019 · 50 comments · Fixed by #5861
Closed

Parcel 2: multi entry/target builds #3302

devongovett opened this issue Aug 2, 2019 · 50 comments · Fixed by #5861

Comments

@devongovett
Copy link
Member

Split out of #2574.

For SSR, it would be useful to support multiple targets with different entries simultaneously.

Three options came up in the previous issue:

Explicit entries per target

const parcel = new Parcel({
  entries: {
    browserEntry_page1: '/path/to/browser/entry/of/page1.js',
    browserEntry_page2: '/path/to/browser/entry/of/page2.js',
    serverEntry: '/path/to/server.js',
  },

  targets: {
    browserEntry_page1: {
      "browsers": ["> 1%", "not dead"]
    },
    browserEntry_page2: {
      "browsers": ["> 1%", "not dead"]
    },
    serverEntry: {
      "node": ["^8.0.0"]
    },
  }
});

An array of parcel options

const parcel = new Parcel([
  {
    entries: ['page1.js', 'page2.js']
    targets: {
      browser: {
        browser: ['>1%', 'not dead']
      }
    }
  },
  {
    entries: ['server.js']
    targets: {
      node: {
        node: ['^8.0.0']
      }
    }
  }
});

Multiple Parcel instances, sharing a worker farm

We were thinking of making the worker farm an option anyway, so this might work for free. By sharing a worker farm instance, multiple Parcel instances could run in parallel.

const workerFarm = new WorkerFarm();

const browserParcel = new Parcel({
  workerFarm,
  entries: ['page1.js', 'page2.js']
  targets: {
    browser: {
      browser: ['>1%', 'not dead']
    }
  }
});

const serverParcel = new Parcel({
  workerFarm,
  entries: ['server.js']
  targets: {
    node: {
      node: ['^8.0.0']
    }
  }
});

Thoughts?

cc. @brillout @padmaia @wbinnssmith

@brillout
Copy link

brillout commented Aug 5, 2019

👍 I'm happy to hear that you are looking into supporting different targets for different entries.

I'm fine with either of the first two proposals. But I do have a slight preference to name entries, e.g. browserEntry_page1. It's slightly easier to reason about names than array indices.

The WorkerFarm proposal sounds very interesting from an SSR perspective. When the user creates a new page pages/newlyCreated.page.js then ssr-coin needs to change the entries on-the-fly. Would I be able to do the following?

const workerFarm = new WorkerFarm();

let serverParcel;
let browserParcel;
ssrCoin.onCreatedOrRemovedPage(({pageServerEntries, pageBrowserEntries}) => {
  // `pageBrowserEntries` would be something like this:
  //   {
  //     first_pageBrowserEntry: '/path/to/my-ssr-app/.build/generated-source-code/first.page.js',
  //     second_pageBrowserEntry: '/path/to/my-ssr-app/.build/generated-source-code/second.page.js',
  //   }
  // `pageServerEntries` would be something like this:
  //   {
  //     first_pageServerEntry: '/path/to/my-ssr-app/pages/first.page.js',
  //     second_pageServerEntry: '/path/to/my-ssr-app/pages/second.page.js',
  //   }
  // (In order to minimize bundle size, `ssr-coin` generates the source code of
  //  the browser entry of each page. This is not necessary for the server;
  //  the page entries for the server are the `pages/*.page.js` files written
  //  by the user.)

  if( browserParcel ) {
    browserParcel.stop();
  }
  if( serverParcel ) {
    serverParcel.stop();
  }

  browserParcel = (
    new Parcel({
      workerFarm,
      entries: {
        ...pageBrowserEntries,
      },
      targets: {
        browser: {
          browser: ['>1%', 'not dead']
        }
    })
  );

  serverParcel = (
    new Parcel({
      workerFarm,
      entries: {
        ...pageServerEntries,
        serverEntry: '/path/to/server/start.js'
      },
      targets: {
        node: {
          node: ['^8.0.0']
        }
      }
    })
  );

  browserParcel.start();
  serverParcel.start();
});

workerFarm.onBuildEnd(assetGraph => {
  // Note that it's convenient to have named the server entry `serverEntry`
  startServer(assetGraph.serverEntry.bundlePath);
});

let serverProcess;
function startServer(serverEntryPath) {
  if( serverProcess ){
    serverProcess.kill();
  }
  serverProcess = fork(serverBuildEntry);
}

It would be super convenient to be able to dynamically change the config of the parcel instances.

I can't wait to use Parcel for ssr-coin :-).

@padmaia
Copy link
Contributor

padmaia commented Aug 5, 2019

One thing to think about is our trend of implementing servers as reporters. An SSR server would need to know about both the client and server build, which seems like it would work best if we went with the first option.

@brillout
Copy link

brillout commented Aug 8, 2019

One thing to think about is our trend of implementing servers as reporters. An SSR server would need to know about both the client and server build, which seems like it would work best if we went with the first option.

With the worker farm proposal, the SSR server would as well know about both the client and server build, correct?

It seems to me that every thing achievable with the first two proposals, can also be achieved with the worker farm proposal. Or am I missing something?

@devongovett
Copy link
Member Author

@padmaia I think an SSR server would typically be wrapping Parcel rather than run as a part of it, which means the wrapper could just run two separate Parcel instances. But if we wanted to support an SSR server running as a Parcel plugin, then yeah, it would need to know about both builds.

@brillout
Copy link

Ok got it.

With the worker farm proposal, the SSR server would as well know about both the client and server build, correct?

If the SSR server is running as a Parcel plugin, then the SSR server wouldn't know aout both builds.

@brillout
Copy link

brillout commented Aug 12, 2019

I think an SSR server would typically be wrapping Parcel rather than run as a part of it

Yes, the thing is that the SSR server should be owned by the user. Currently, and AFAIK, the Parcel dev server is hidden from the user.

What I'm doing is that I expose SSR as a middleware, for example with express:

const express = require('express');
const ssr = require('ssr-coin');

const app = express();
app.use(ssr.express);
// There is also a middleware for Koa and Hapi

I wouldn't know how to provide these middlewares when SSR is run as a Parcel plugin.

If possible, I'd be up to implement a Parcel SSR plugin though.

@Banou26
Copy link
Contributor

Banou26 commented Aug 12, 2019

About the first example, with named entries, one thing that i find annoying is the repetition of configuration for each entry file.
Wouldn't it be better like that ?

const parcel = new Parcel({
  entries: {
    browserEntry: [
      '/path/to/browser/entry/of/page1.js',
      '/path/to/browser/entry/of/page2.js'
    ],
    serverEntry: '/path/to/server.js',
  },

  targets: {
    browserEntry: {
      "browsers": ["> 1%", "not dead"]
    },
    serverEntry: {
      "node": ["^8.0.0"]
    },
  }
});

But yeah, ideally, the worker farm option would be the best thing to have anyways.

@brillout
Copy link

@Banou26

Having named entries is convenient. For example:

workerFarm.onBuildEnd(assetGraph => {
  // Note that it's convenient to have named the server entry `serverEntry`
  startServer(assetGraph.serverEntry.bundlePath);
});

How would you do this if the server entry wouldn't be named serverEntry?

So maybe something like this then:

const parcel = new Parcel({
  entries: {
    browserEntry_page1: '/path/to/browser/entry/of/page1.js',
    browserEntry_page2: '/path/to/browser/entry/of/page2.js',
    serverEntry: '/path/to/server.js',
  },
  targets: [
    {
      entries: [
        'browserEntry_page1',
        'browserEntry_page2',
      ],
      targets: {
        "browsers": ["> 1%", "not dead"]
      }
    },
    {
      entries: [
        'serverEntry',
      ],
      targets: {
        "node": ["^8.0.0"]
      }
    }
  ]
});

Personally, I don't care much. I'm happy as long as I can have different targets for different entries and as long as I don't have to synchronize two concurrent Parcel builds. Named entries are just a slight preference on my side.

@Banou26
Copy link
Contributor

Banou26 commented Aug 12, 2019

Having named entries is convenient. For example:

workerFarm.onBuildEnd(assetGraph => {
  // Note that it's convenient to have named the server entry `serverEntry`
  startServer(assetGraph.serverEntry.bundlePath);
});

I think you are mixing the examples together.

The example i gave is just a modification of the first @devongovett presented to reduce config duplication.

It doesn't involve WorkerFarm, so in the end you'd still have browserEntry as name(if there's such a thing as a name for the BundleGraph) of both /path/to/browser/entry/of/page1.js and /path/to/browser/entry/of/page2.js's bundle graph, and the name serverEntry for /path/to/server.js's bundle graph.

Continuing on the WorkerFarm example you gave, would you really listen for buildEnd events on the WorkerFarm though ?
Wouldn't it make more sense to listen for it on the Parcel instance ?
I thought WorkerFarm just got tasks to run and returned the result to the Parcel instance.

So instead of listening on a buildEnd on the WorkerFarm, you'd listen to it on the Parcel instance, and from there you'd be able to get your 'entry name'.

const workerFarm = new WorkerFarm();

const browserParcel = new Parcel({
  workerFarm,
  entries: ['page1.js', 'page2.js']
  targets: {
    browser: {
      browser: ['>1%', 'not dead']
    }
  }
});

const serverParcel = new Parcel({
  workerFarm,
  entries: ['server.js']
  targets: {
    node: {
      node: ['^8.0.0']
    }
  }
});

browserParcel.on('buildEnd', () => {
  // You know it's the browserParcel entry
})

serverParcel.on('buildEnd', () => {
  // You know it's the serverParcel entry
})

I'm happy as long as I can have different targets for different entries and as long as I don't have to synchronize two concurrent Parcel builds.

I don't think it's possible with this to have your synchronization of two Parcel instances builds to be emitted together.
You'd have to synchronise the builds yourself, but it shouldn't be hard to do.

@Banou26
Copy link
Contributor

Banou26 commented Aug 15, 2019

EDIT: This request was apparently fixed by #3583

It would also be nice to allow for multiple entries using globs via both the CLI and the API, Parcel 1 allowed for it.
Currently, trying to use a glob in either of them throws

× Globs can only be required from a parent file
    at NodeResolver.resolve (c:\dev\parcel\node_modules\@parcel\resolver-default\lib\DefaultResolver.js:114:15)
    at Object.resolve (c:\dev\parcel\node_modules\@parcel\resolver-default\lib\DefaultResolver.js:35:8)
    at ResolverRunner.resolve (c:\dev\parcel\node_modules\@parcel\core\lib\ResolverRunner.js:50:35)
    at async RequestGraph.resolvePath (c:\dev\parcel\node_modules\@parcel\core\lib\RequestGraph.js:323:26)
    at async PromiseQueue._runFn (c:\dev\parcel\node_modules\@parcel\utils\lib\PromiseQueue.js:96:7)
    at async PromiseQueue._next (c:\dev\parcel\node_modules\@parcel\utils\lib\PromiseQueue.js:83:5)

This would be useful for users using tools that use Parcel 2 and wants to perform an action on new files/file deletion on a folder.
Because afaik, right now there's no way to 'automatically' watch for (new/less) entries directly in Parcel 2, it be from the CLI or the API.

If this is not an available option, the only way to implement this efficiently would be to use the WorkerFarm option and watch the globs for file addition/deletion ourselves(which the parcel watcher kinda already do) and spawn/delete Parcel instances everytime there's a change to add/delete entries.

Edit: It would also be nice to have a programmatic API to add/delete entries from a parcel instance possibly running(in watch mode).

This would be really useful in my case, for my testing tool, which heavily uses Parcel.

@brillout
Copy link

brillout commented Sep 1, 2019

I don't think it's possible with this to have your synchronization of two Parcel instances builds to be emitted together.

That's very much what I would want though.

I can already have different targets for different entries by running a different Parcel instance for each target. Other than performance, it seems to me that the whole point of this ticket is to make it easier to have different targets for different entries.

In my experience, the need to synchronize two concurrent builds is the biggest pain when implementing SSR.

Also, I'm not sure what the benefit in the following would be:

const browserParcel = new Parcel({
  entries: browserEntries,
  targets: {
    browser: {
      browser: ['>1%', 'not dead']
    }
  }
});

const serverParcel = new Parcel({
  entries: serverEntries,
  targets: {
    node: {
      node: ['^8.0.0']
    }
  }
});

browserParcel.on('buildEnd', () => {
 // Why should I care *only* about the events of the browser build?
})

serverParcel.on('buildEnd', () => {
 // Why should I care *only* about the events of the server build?
})

I don't see many use cases where someone would be interested in events of only a portion of the build.

It seems to me that the following makes much more sense:

const workerFarm = new WorkerFarm();

const browserParcel = new Parcel({
  workerFarm,
  entries: browserEntries,
  targets: {
    browser: {
      browser: ['>1%', 'not dead']
    }
  }
});

const serverParcel = new Parcel({
  workerFarm,
  entries: serverEntries,
  targets: {
    node: {
      node: ['^8.0.0']
    }
  }
});

// I want to be able to listen to the *global* state events of the build.
workerFarm.on('buildEnd', () => {
  // The server code and the browser code are built
});

// I'm not familiar with workerFarm and I don't know if it makes sense
// to listen to events on workerFarm.
// I don't mind if it's `workerFarm.on('buildEnd'` or `parcel.on('buildEnd'`.
// All I want is to not have to synchronize stuff.

Otherwise I'd have to synchronize the events, for example:

const browserParcel = new Parcel({
  entries: browserEntries,
});

const serverParcel = new Parcel({
  entries: serverEntries,
});

browserParcel.on('buildStart', () => {
  buildState.browser = {
    isBuilding: true,
  };
  onStateChange();
});
browserParcel.on('buildEnd', ({err, assetGraph}) => {
  buildState.browser = {
    isBuilding: false,
    err,
    assetGraph,
  };
  onStateChange();
});

serverParcel.on('buildStart', () => {
  buildState.server = {
    isBuilding: true,
  };
  onStateChange();
});
serverParcel.on('buildEnd', ({err, assetGraph}) => {
  buildState.server = {
    isBuilding: false,
    err,
    assetGraph,
  };
  onStateChange();
});

let neverSucceded = true;
function onStateChange() {
  if( (buildState.browser.isBuilding || buildState.server.isBuilding) ){
    console.log('Building');
    return;
  }

  if( !buildState.browser.err && !buildState.server.err ){
    console.log('Built');
    if( neverSucceded ) {
      startServer();
    } else {
      restartServer();
    }
    neverSucceded = false;
    restartBrowser();
  } else {
    console.error('Build failed');
  }
  if( buildState.browser.err ){
    console.error(buildState.browser.err);
  }
  if( buildState.server.err ){
    console.error(buildState.server.err);
  }
}

This synchronization work is not a show blocker but it is annoying and not particularly user friendly.

I've built an SSR tool on top of webpack. My tool has to synchronize two concurrent webpack builds. The synchronization is a huge pain. In webpack's case the code above is just a tiny tip of the iceberg.

It's not only about SSR tool authors. It's also about developers who want a custom SSR implementation. As a tool author, it's okay to use non-friendly APIs. But, as a developer working for a startup, time is more critical and API user-friendliness substantially more important.

The most complex part of SSR is the building. When people ask whether they should implement SSR themselves, I recommend against it because SSR induces a considerably more complex build.

If Parcel makes SSR easy to implement then many companies would be able to implement SSR themselves which would be wonderful.

@devongovett
Copy link
Member Author

The core infrastructure for this is implemented in #3583. That PR supports targets per entry as resolved from package.json, but not in the Parcel API yet. I think this ticket is now a matter of deciding on an API for passing entries and targets into Parcel and then connecting that to the Target resolution infrastructure.

@brillout
Copy link

I think I found an API that would suit everyone:

const parcel = new Parcel({
  entries: [
    {
      entry: '/path/to/page1.js',
      target: 'universal',
    },
    {
      entry: '/path/to/page2.js',
      target: 'universal',
    },
    {
      entry: '/path/to/server.js',
      target: 'node',
    },
  ],

  targets: {
    // *********************
    // ** Parcel defaults **
    // *********************

    // Following the zero-config approach, Parcel provides defaults `browser`,
    // `browserModern` and `node`.
    // These defaults can be overwritten in `package.json#targets`
    // or with the Parcel programmatic API.

    browser: {
      browsers: ["> 0.25%", "not dead"],
    },
    browserModern: {
      browsers: ["last 2 version"],
    },
    node: {
      nodejs: "^8.0.0",
    },

    // ********************
    // ** Custom targets **
    // ********************

    // Custom targets defined by the Parcel user.

    // We use an array for multi-bundle targeting.
    // This generates two bundles: one that works in all browsers and
    // another one that works only in modern browsers.
    browserOldAndNew: [
      'browser',
      'browserModern',
    ],

    // This generates ONE bundle.
    // (I know that Parcel doesn't support isomorphic builds; this
    // is just to showcase the API.)
    isomorphic: {
      browsers: ["> 0.25%", "not dead"],
      nodejs: "^8.0.0",
    },

    // This generates TWO bundles.
    // (Such `universal` target is what SSR tools usually do.)
    universal: [
      'browser',
      'node',
    ],
  },
});

I purposely didn't name the entries here: since an entry is always a file on the disk, we can simply take the absolute path of the entry file as name. For example:

const serverPath = '/path/to/server.js';

const parcel = new Parcel({
  entries: [{ entry: serverPath, target: 'node'}],
});

parcel.on('buildEnd', assetGraph => {
  reloadServer(assetGraph[serverPath].bundlePath);
});

I'd be happy to research sensible defaults for node, browser, and browserModern, if you want.

The only thing missing here is the ability to dynamically add/remove entries, which is crucial for SSR. But this could be solved by #3699 - [RFC] New plugin type entry-finder.

@jamiekyle-eb
Copy link
Contributor

I think it's worthwhile to consider having this supported in package.json.

{
  "source": "src/index.html", // default source

  "browser": "dist/browser/index.html",
  "ssr": "dist/ssr/index.mjs",

  "targets": {
    "browser": {
      "node": ">=8.0.0"
    },
    "ssr" {
      "source": "src/ssr.tsx", // overwrite source
      "node": ">=12.0.0"
    }
  },

  // Alternatively:
  "source": {
    "browser": "src/index.html",
    "ssr": "src/ssr.tsx"
  },

  // Alternatively-er:
  "sources": {
    "browser": "src/index.html",
    "ssr": "src/ssr.tsx"
  }
}

@Banou26
Copy link
Contributor

Banou26 commented Oct 30, 2019

@brillout I'm sorry but that's a big no for me, this is just unnecessarily complex,

const parcel = new Parcel({
  entries: [
    {
      entry: '/path/to/page1.js',
      target: ['browser', 'node'],
    },
    {
      entry: '/path/to/page2.js',
      target: ['browser', 'node'],
    },
    {
      entry: '/path/to/server.js',
      target: 'node',
    },
  ],

  targets: {
    browser: {
      browsers: ["> 0.25%", "not dead"],
    },
    node: {
      nodejs: "^8.0.0",
    }
  }
})

Would serve the same purpose, while being more explicit and straightforward.

I don't like the fact that you can reference targets, from targets, this makes you have to look back everytime you see a definition that use a reference.
Configurations should be explicit and simple, sure, if you have a lot of different needs, it'll get complex, but for a simple configuration like this, it shouldn't be and look complex for nothing.

Edit 1: BTW, about your isomorphic target, if parcel supports ESM outputs and the top level await proposal(which is getting shipped in v8 and webpack as an experiment), we could have multi-target bundles since node will, at its next version, support ESM modules as a stable feature

Edit 2: While we're at it, we could even remove some config duplication by allowing entries as array, but then i'm not sure that entries/entry/target should be the name of the properties

const parcel = new Parcel({
  entries: [
    {
      entry: ['/path/to/page1.js', '/path/to/page2.js'],
      target: ['browser', 'node'],
    },
    {
      entry: '/path/to/server.js',
      target: 'node',
    },
  ],

  targets: {
    browser: {
      browsers: ["> 0.25%", "not dead"],
    },
    node: {
      nodejs: "^8.0.0",
    }
  }
})

@brillout
Copy link

brillout commented Oct 30, 2019

@Banou26 I like your version, it's simpler 👍

About isomorphic, neat — that's super interesting. (If you're curious; SSR will still need two different bundles: the browser bundle needs all dependencies to be included, whereas for Node.js it's important to not bundle dependencies.)


@jamiekyle-eb

AFAICT, package.json#source only makes sense from a perspective of building an NPM package.

From an SSR perspective, package.json#browser and package.json#main don't make much sense. And, from an SPA perspective, these fields don't make much sense either. (It actually took me a while to understand package.json#source because I wasn't using Parcel to build an NPM package.)

Maybe Parcel should support both: package.json#source for people who want to build an NPM package and package.json#entries for people who want to build an SPA or an SSR app.

(EDIT: Typo - I meant package.json#main, not package.json#dist.)

@brillout
Copy link

@Banou26

Edit 2

I find the following confusing:

{
  "entries": [
    {
      "entry": ["/path/to/page1.js", "/path/to/page2.js"],
      "target": ["browser", "node"]
    }
  ]
}

Does this mean that there will be a single bundle with 2 entry points? Or that there will be 2 bundles with each having a single entry point?

I know the answer but I'm not sure a Parcel beginner would.

@brillout
Copy link

Actually, package.json#entries could be merged into package.json#source. Resulting in the following.

SSR:

const parcel = new Parcel({
  source: [ // Note how it's `source` not `entries`
    {
      entry: '/path/to/page1.js',
      target: ['browser', 'node'],
    },
    {
      entry: '/path/to/page2.js',
      target: ['browser', 'node'],
    },
    {
      entry: '/path/to/server.js',
      target: 'node',
    },
  ],

  targets: {
    browser: {
      engines: {
        browsers: ["> 0.25%", "not dead"],
      }
    },
    node: {
      engines: {
        nodejs: "^8.0.0",
      }
    }
  }
})

NPM package:

//package.json

// In a zero-config way:

{
  "source": "src/index.js",
  "main": "lib/index.js" // The default target of `main` is `node`
}
//package.json

// The default target can be overridden by explicitly
// setting the target:
{
  "main": "lib/index.js",
  "source": [
    {
      "entry": "./src/index.js",
      "target": "nodeModern",
      // `output` needs to be explicitly set equal to `main`.
      // (Because defining `package.json#source` as an Array
      //  is an escape hatch to Parcel's zero-config.)
      "output": "lib/index.js"
    },
  ],
  "targets": {
    "nodeModern": {
      "engines": {
        "nodejs": ">=13.x"
      }
    }
  }
}

SPA:

//package.json

{
  "source": "src/index.html" // The default target is `browser`
                              // when the entry is an `.html`
  // `package.json#main` has no semantics in the context of an SPA
}

@jamiekyle-eb
Copy link
Contributor

AFAICT, package.json#source only makes sense from a perspective of building an NPM package.

I don't agree with that, I think it's perfectly acceptable to do things like:

{
  "source": "src/index.html",
  "main": "dist/index.html"
}

I agree that package.json#main is kinda a shitty target name, but I also expect package entry names to become more abstract over time.

I think this is a perfectly reasonable setup for an app that supports multiple source entry points which split out into different targets.

{
  "source": "src/index.html",
  "dist:legacy": "dist/legacy/index.html",
  "dist:modern": "dist/modern/index.html",
  "dist:ssr": "dist/ssr/ssr.js",
  "targets": {
    "dist:legacy": {
      "browsers": [">0.5%", "not dead"]
    },
    "dist:modern": {
      "browsers": [">3%", "not dead"]
    },
    "dist:ssr": {
      "node": ">=10",
      "source": "src/ssr.js"
    }
  }
}

I want to stick to Parcel using package.json as much as possible, even going as far as not allowing this stuff to be configured programmatically with a new Parcel({ .... }) type API. I would rather require a package.json be on disk.

@rtsao
Copy link

rtsao commented Oct 30, 2019

Have the existing browserlist or engines fields been considered for environment configuration of the targets?

For an app, the package.json might look like:

{
  "browserslist": {
    "legacy": [">0.5%", "not dead"],
    "modern": [">3%", "not dead"]
  },
  "engines": {
    "node": "12.13.0"
  },
  "volta": {
    "node": "12.13.0",
    "npm": "6.12.1"
  }
}

A couple points on this:

  1. It seems to me the SSR bundle output should always just match the minimum node semver specified in the engines field. I'm trying to think of reason why it shouldn't, but can't think of one.
  2. Re-using the existing browserlist field means linters and other tools (in addition to Parcel) can read this config.

I think enforcing static package.json target configuration is a worthwhile goal, but I would hope this could be done in a generic way rather than being tied to Parcel specifically.

Using the above approach, the environment configuration for various targets is somewhat decoupled from bundler-specific implementation details on how those targets actually get compiled (e.g. entries, plugins, etc.)

I think unique, named ids for browser targets is a reasonable interface (and one that already exists with browserlist)

@jamiekyle-eb
Copy link
Contributor

Have the existing browserlist or engines fields been considered for environment configuration of the targets?

@rtsao Yeah, browserslist and engines.* is used as the default for some targets. targets[key].browsers/node is a way to override it on a target-by-target basis.

@jamiekyle-eb
Copy link
Contributor

There are a lot of assumptions being made in this thread that go against what Parcel is trying to achieve.

  • You should be able to compile N targets at once, regardless if they are 3 different versions being compiled for Node, 6 different versions compiled for the browser, and 19 versions being compiled for electron. We should not be prescriptive about what people are allowed to compile, that's just shooting ourselves in the foot needlessly.
  • We do not want tools to "wrap" Parcel in such a way that includes implicit configuration either through a Node API or some sort of runtime manipulation of Parcel. Tools like Jest or Next.js doing this today is a huge source of problems users have with build tools. The only way to configure Parcel should be explicitly through the local file system. So we should focus on making sure that covers everyone's needs.
  • We want to make use of existing "standards" (even if those are just community norms) as much as possible, adding as little "new api" to the ecosystem as we can. package.json#targets was the most self-contained we could be able the API we needed to add. We needed an easy way to discover all the "entry fields" to a package.json (main/module/browser/ssr/etc), and we needed an easy way to specify the differences between those entry fields so that we didn't have "magic names". package.json#browserslist accepting an object is interesting because we didn't know that existed. But engines.node and other similar fields do not support it, so it's still necessary.
  • package.json#source is not something we came up with and it does not support an object so we should be careful introducing something other tools might not expect

@jamiekyle-eb
Copy link
Contributor

We don't intend to use the main, module, or browser fields for anything other than libraries.

I disagree with this. I think main, module, and browser are all fine to use for applications.

@jamiekyle-eb
Copy link
Contributor

Leaving us with:

{
  "source": {
    "spa": {
      "entry": "browser/index.html",
      "target": "browserModern"
    },
    "server": {
      "entry": "server/start.js",
      "target": "node"
    }
  },
  "spa": "dist/browser/index.html",
  "server": "dist/server/start.js",
  "browserslist": {
    "browserModern": [">= 1%"]
  },
  "engines": {
    "node": [">=8.x"]
  }
}

All that accomplishes is flipping package.json#target upside down so it's inside package.json#source and making it impossible to support multiple versions of node. Not to mention that package.json#source is not something Parcel invented and that version of it would almost certainly not be accepted by the community.

Here is the equivalent code with package.json#targets:

{
  "spa": "dist/browser/index.html",
  "server": "dist/server/start.js",
  "browserslist": [">= 1%"],
  "engines": {
    "node": [">=8.x"]
  },
  "targets": { 
    "spa": {
      "source":  "browser/index.html",
      "node": false,
      "browser": true
    },
    "server": {
      "source": "server/start.js",
      "node": true,
      "browser": false
    }
  }
}
  • Want to know the entry points? -> Object.keys(pkg.targets)
  • Want to know which environments a target supports? -> pkg.targets[name]
  • Want to add another target for node/browser/electron/whatever? -> Just add another one to the list.

You could even simplify it by making use of existing patterns in the ecosystem:

{
  "main": "dist/server/start.js",
  "browser": "dist/browser/index.html",
  "source": "server/start.js",
  "browserslist": [">= 1%"],
  "engines": {
    "node": [">=8.x"]
  },
  "targets": { 
    "browser": {
      "source":  "browser/index.html",
    }
  }
}

@devongovett
Copy link
Member Author

I disagree with this. I think main, module, and browser are all fine to use for applications.

Those fields have specified/implicit meaning to other tools:

  • main and browser are CommonJS modules
  • module is an ES6 module

All three are JS library targets because they are used to resolve require/import by Node or other bundlers. There is no precedence of them being used for applications, or anything other than JavaScript.

@jamiekyle-eb
Copy link
Contributor

jamiekyle-eb commented Nov 6, 2019

There is no precedence of them being used for applications, or anything other than JavaScript.

I've seen them used for CSS before. I'm not going to try to find a quote now, but the npm team has said they intend it to be used for other languages. There are also several bundlers that can handle arbitrary languages as the package.json#main/etc as long as they are set up to do so.

One of the things I want to be careful with is taking Parcel "defaults" and making them "rules". We might treat package.json#module one way by default because of how most of the ecosystem uses it, but every part of that should be overwriteable by package.json#target.module. It shouldn't require you to create a new target name and/or not allow you to overwrite any of Parcel's default behavior around it.

@devongovett
Copy link
Member Author

It is all possible to override, I'm just not sure we should encourage it:

{
  "main": "dist/index.html",
  "targets": {
    "main": {
      "context": "browser",
      "isLibrary": false,
      "includeNodeModules": true,
      "engines": {
        "browsers": ["> 1%"]
      }
    }
  }
}

@Banou26
Copy link
Contributor

Banou26 commented Nov 7, 2019

I don't understand why should we be able to have a main/module/ect if the package isn't a library, like, what's the use of it, you won't be able to run anything from it, you'll have to start something from the scripts/from your terminal anyways, so it's no different than having an entries property, it's just overall less explicit...
The only use case for apps to use main is to have a dist path, i didn't mean to not use them, i just didn't like seeing multiple user-defined properties like spa and server for applications.

If we have non JS scripts as main/module/browser for libraries, does it mean that now, we'll start seeing NPM packages that are bundler specific ? I think we need to think about stuff that this would affect, problems like #3477 (comment) could start showing up.

{
  "main": "dist/index.html",
  "targets": {
    "main": {
      "context": "browser",
      "isLibrary": false,
      "includeNodeModules": true,
      "engines": {
        "browsers": ["> 1%"]
      }
    }
  }
}

What's targets.main.context for ?
And how does it work if we wanna have multiple browser targets for one entry ? There we can only define 1 browser target and 1 node target per entry.

@brillout your argument about not needing multiple node targets is based on the assumption that you have control over your Node version.

Not all hosts gives you the latest versions of node.
And it cost nothing to support it anyways, if we support multiple browser targets why not multiple node targets, why not multiple electron targets, why not multiple any JS runtime targets ?

For now my best take in all of this is the configuration I talked about in #3302 (comment)

@brillout
Copy link

brillout commented Nov 9, 2019

In the context of a library, the shared configs package.json#main, package.json#browser and package.json#module have clear semantics and are already used by a multitude of tools. These shared configs are a standard and it makes sense for Parcel to use them.

But, in the context of an application, there is no such standard and there are no shared configs package.json#xyz. It doesn't add any value to do this:

// SPA + Node.js server

{
  "spa": "dist/browser/index.html",
  "server": "dist/server/start.js",
  "targets": {
    "spa": {
      "source":  "browser/index.html",
      "browser": true,
      "node": false
    },
    "server": {
      "source": "server/start.js",
      "browser": false,
      "node": true
    }
  }
}

Since there are no shared configs, why don't we just skip them and do something like this:

// SPA + Node.js server

{
  "source": [
    {
      "entry": "browser/index.html",
      "outFile": "dist/browser/index.html"
      "target": "browser"
    },
    {
      "entry": "server/start.js",
      "outFile": "dist/server/start.js"
      "target": "node"
    }
  ]
}

This package.json#source array could also be used for libraries to gain further control:

// Library

{
  "source": [
    {
      "entry": "src/index.js",
      "outFile": "dist/index.js",
      "target": {
        "context": "node",
        "isLibrary": true,
        "includeNodeModules": false,
        // The zero-config setup would build for `node>=8.x`.
        // Thanks to the `package.json#source` array we can override this.
        "engines": {
          "node": ">=13.x"
        }
      }
      // ...
    }
  ],
  "main": "dist/index.js"
}

The idea here is to go with a dual interface strategy:

  • In a zero-config setup, Parcel reads shared configs.
  • In a full-control setup, Parcel ONLY reads the package.jon#source array.

A nice thing about the pacakge.json#source array is that the programmatic API could use the exact same interface:

// An SSR tool

// How would you achieve something like this with the
// current v2 design? AFAICT it would end up super ugly.

const parcel = new Parcel({
  source: [
    // The user defines a `pages/index.html` as an
    // HTML document wrapper for all pages.
    {
      "entry": "pages/index.html",
      "target": "browser",
      "outDir": "dist/browser/"
    },

    // We build the server.
    {
       "entry": "server/index.js",
       "target": "node",
       "outDir": "dist/nodejs/",
    },

    // We build the landing page for the browser (for hydration).
    {
      "entry": "pages/landing/index.page.js",
      "target": "browser",
      "outDir": "dist/browser/"
    },
    // We build the landing page for Node.js (for server-side HTML rendering).
    {
      "entry": "pages/landing/index.page.js",
      "target": "node",
      "outDir": "dist/nodejs/"
    },

    // Other pages
    {
      "entry": "pages/about/index.page.js",
      "target": "browser",
      "outDir": "dist/browser/"
    },
    {
      "entry": "pages/about/index.page.js",
      "target": "node",
      "outDir": "dist/nodejs/"
    },

    // ...
  ],
});

Since shared configs don't make sense for the programmatic API, only the package.json#source array interface is allowed.


The case for package.json#staticDir and package.json#server

Shared configs enable beautiful zero-config setups, as we can seen in the context of libraries.

If Parcel introduces the new shared configs package.json#staticDir and package.json#server, then we can introduce beautiful zero-config setups for apps:

// SPA

{
  "source": "src/index.html",
  "staticDir": "dist/"
  // (Parcel knows about `staticDir`: The default target is
  // `package.json#browserlist` or `browser: [">= 0.25%"]`)
}
// Node.js server

{
  "source": "src/server.js",
  "server": "dist/server.js"
  // (Parcel knows about `server`: The default target is
  // `package.json#engines.node` or `node: ">=8.x"`)
}
// SPA + Node.js server

{
  "source": {
    "staticDir": "browser/index.html",
    "server": "server/index.js"
  },
  "staticDir": "dist/browser/",
  "server": "dist/server/index.js"
}

These are as beautiful as our zero-config setup for libraries 😍.

Also, another immediate benefit is that package.json#staticDir and package.json#server can be used by a Parcel dev server middleware. (Node.js apps and SSR require a dev middleware. Such middleware needs to know where the static directory is and where the server entry is.)

Other tools would eventually use these new fields as well.

If the user wants full control, he can use the package.json#source array instead.


Why not package.json#targets

package.json#source is basically the same than package.json#targets but flipped upside down, as Jamie says.

AFAIK, we all agree on this:

// Node.js library

{
  "source": "src/index.js",
  "main": "dist/index.js"
}

So, instead of introducing a second field package.json#targets, we extend our existing package.json#source field.

In the end, we have a single package.json#source field that is Parcel specific and fully controlled by Parcel. All other package.json fields are shared configs.

I can't think of anything simpler than that.

@brillout
Copy link

brillout commented Nov 9, 2019

@Banou26 @jamiekyle-eb Multi Node.js targeting could be achieved with a package.json#source array:

{
  "source": [
    {
      "entry": "src/index.js",
      "outFile": "dist/modern.js",
      "target": {
        "engines": {
          "node": ">=13.x"
        }
      }
    },
    {
      "entry": "src/index.js",
      "outFile": "dist/index.js",
      "target": {
        "engines": {
          "node": ">=8.x"
        }
      }
    }
  ]
}

This is verbose because the goal of the package.json#source array is not to be beautiful but to give full control to the Parcel user. See my previous post for beautiful zero-config setups.

@jamiekyle-eb

I think main, module, and browser are all fine to use for applications.

We are all seem to disagree with you. Would you mind elaborating? Personally, I can't fathom what your motivation is.

That's how I see it:

  • These shared configs are useless in the context of apps. I'm not aware of any tool that would benefit from Parcel using these shared configs.
  • It's confusing to overlap semantics. package.json#main = entry point of Node.js library — that's clear. What you want is: package.json#main = abstract entry point of whatever package.json represents. But what does this mean for an SPA? For an MPA? For a full-stack app? In the context of an app, the notion of "entry point" is confusing. It doesn't surprise me that @Banou26 has such strong feeling towards your propsal.

@parcel-bundler parcel-bundler deleted a comment from jamiekyle-eb Nov 15, 2019
@devongovett
Copy link
Member Author

We aren't going to change the source and targets configuration. They have been well considered and are quite flexible to the needs of various tools even outside of Parcel.

Seems like the best solution to this is to make two packages, e.g. in a monorepo: a server package and a client package. Each package.json should define the targets it wants. Then, you can point parcel at both folders with package.json and it will build them both together.

In many cases, this won't be necessary at all: you'll have the same entrypoint for both client and server and can simply have a single package with two targets.

@brillout
Copy link

@jamiekyle-eb I'm sorry if my reply came across as offensive. That wasn't my intention. I'm sorry. The only thing I care about is the beauty of Parcel and its zero-config philosophy which I agree so much with. (I actually started to build a zero-config bundler before Devon created Parcel but I never came to finish it.)

@devongovett

Seems like the best solution to this is to make two packages

Neat idea. And, from an SSR perspective, it should work. I'll try that for @parcel-ssr.

Although, how would you define multi targets for **/*.pages.js then? Like the following?

// pages/package.json

{
  "source": "**/*.page.js"
  // `dist/browser/` is the static directory to be served
  "pages-browser": "dist/browser/",
  // `dist/nodejs/` is read by the SSR plugin
  "pages-nodejs": "dist/nodejs/",
  "targets": {
    "pages-browser": {
      "context": "browser",
    },
    "pages-nodejs": {
      "context": "node"
    }
  }
}

When doing SSR, the pages need two targets.

I find it weird and counter-intuitive to use artificial package.json fields package.json#pages-browser and package.json#pages-nodejs. Artificial in the sense that they don't have any semantics outside of my app.

AFAIK, most package.json fields have well-defined and universal semantics: if I look at the package.json of a random project I just stumbled upon, I expect to know the meaning of most pacakge.json fields. If I then see package.json#pages-browser, my first thought would be "Why is pages-browser defined in package.json? That's a Parcel specific config; why isn't it defined in .parcelrc?".

I'm wondering if it wouldn't be prettier to have something like this:

// pages/package.json

{
  "source": "**/*.page.js"
  "targets": [
    {
      "context": "browser",
      "outDir": "dist/browser/"
    },
    {
      "context": "node"
      "outDir": "dist/nodejs/"
    }
  ]
}

I'm only speaking of my personal and limited perspective, I surely am missing many aspects and use cases. And I know that you don't want to change the source and targets configuration — I'm just wondering.

And, any thoughts on introducing new shared configs package.json#staticDir and package.json#server? I still believe that it could be lovely to have zero-config setups like this 😊

// browser/package.json

{
  "source": "index.html",
  "staticDir": "dist/"
  // (Parcel knows about `staticDir`: The default target is
  // `package.json#browserlist` or `browser: [">= 0.25%"]`)
}
// server/package.json

{
  "source": "start.js",
  "server": "dist/start.js"
  // (Parcel knows about `server`: The default target is
  // `package.json#engines.node` or `node: ">=8.x"`)
}

I'm currently experimenting and implementing a deploy NPM package to auto-deploy Node.js servers. My deploy NPM package would love to share the package.json#server config with Parcel 😏.

@devongovett
Copy link
Member Author

Although, how would you define multi targets for **/*.pages.js then? Like the following?

yes

I find it weird and counter-intuitive to use artificial package.json fields package.json#pages-browser and package.json#pages-nodejs. Artificial in the sense that they don't have any semantics outside of my app.

Yeah, the targets field is defining them. You define the meaning of the package entry in targets.

If I then see package.json#pages-browser, my first thought would be "Why is pages-browser defined in package.json? That's a Parcel specific config; why isn't it defined in .parcelrc?".

It's not necessarily Parcel specific. Other tools can also read the metadata for the custom targets and decide to use their entry points. We've already been discussing support for this in Node. targets essentially lets you define your own custom entry points with enough metadata for other tools to consume them.

@brillout
Copy link

We've already been discussing support for this in Node.

Interesting. Was the discussion public; can I read about it? I searched but couldn't find any public discussion.

Other tools can also read the metadata for the custom targets and decide to use their entry points

It seems that targets only says something about the environment:

  • Node.js or browser
  • CJS or ES Modules
  • Etc.

It however doesn't say what the target means.

But I guess a tool could infer the meaning.

{
  "custom-target": "dist/server.js",
  "targets": {
    "custom-target": {
      "context": "node"
    }
  },
  "dependencies": {
    "nodemon": "^1.2.3"
  }
}

Since nodemon is a server tool, it can infer that a "context": "node" target of a package.json that contains the dependency package.json#dependencies.nodemon is most likely going to be the entry point of a server. That way, nodemon would guess that package.json#custom-target is the server entry file to execute and to auto-restart.

For what it's worth, I still find that a package.json#server and package.json#staticDir would be prettier.

To me, a beautiful design is mostly about 1. simplicity and 2. clear communication.

I'd say that

  • package.json#main = entry point of Node.js library.
  • package.json#browser = entry point of browser library.
  • package.json#server = entry point of server.
  • package.json#staticDir = entry point of frontend.

is both crystal clear as well as super simple. I can't think of any argument against package.json#serer and package.json#staticDir 😇.

We could have package.json#server and package.json#staticDir in addition to package.json#targets.

Just my 2 cents.

Thanks for the reply and I'll experiment all that with parcel-ssr.

Super eager to do SSR with Parcel. Thanks for having created Parcel, it's beautiful.

@devongovett
Copy link
Member Author

devongovett commented Nov 16, 2019

You can build that yourself though!

{
  "server": "dist/server.js",
  "targets": {
    "server": {
      "context": "node",
      "engines": {
        "node": "12.x"
      }
    }
  }
}

@brillout
Copy link

You can build that yourself though!

I know and, as a tool author, this is perfectly fine. But, as a Parcel end-user, I'm a little bit saddened that Parcel doesn't introduce new semantics for package.json#server for a better zero-config setup.

Btw., what is the zero-config story going to look like for an SPA? Like the following?

{
  "source": "src/index.html",

  // We cannot use `package.json#browser` since it's already used by libraries.
  "spa": "dist/index.html",

  "targets": {
    "context": "browser"
  }
}

Hm... that still leaves me wondering why not this:

{
  "source": "src/index.html",
  "staticDir": "dist/"
  // Parcel knows that `package.json#staticDir` denotes the directory
  // holding all static assets for the browser; the default target is
  // `package.json#browserlist || [">= 0.25%"]`
}

Sorry to be pushy, it's just difficult to abandon beauty 😊.

I have some thoughts about Parcel's programmatic API that I'll write down in this ticket in the next coming days.

@Banou26
Copy link
Contributor

Banou26 commented Nov 18, 2019

To build an SPA with parcel, you can just do this

{
  targets: {
    // or however the target is configured
    spa: {
      browsers: ["> 1%", "not dead"]
    }
  },
  scripts: {
    spa: "parcel --target=spa src/index.js"
  }
}

What's wrong with this way ?

I mean, an SPA is like whatever else browser script, there's nothing that change.

@brillout
Copy link

About the programmatic API: it would be absolutely wonderful if Parcel supports dynamic configurations. It seems that Parcel v2 watches package.json and .parcelrc. Maybe Parcel could also "watch" the programmatic API configuration. For example:

const parcel = new Parcel();

parcel.config = {
  source: [
     {entry: '/path/to/pages/landing/Landing.page.js'},
     {entry: '/path/to/pages/about/About.page.js'},
  ],
};

await parcel.build();

// We later add a new entry:
parcel.config.source.push({entry: '/path/to/pages/contact/Contact.page.js'});
// Parcel automatically rebuilds.

That would enormously help tool authors. My current SSR tool (Goldpage), which is implemented on top of Webpack, needs to manually stop and restart Webpack and that's a huge pain. Would be lovely if I can simply dynamically change the config instead.

AFAICT, watching a JS object should be possible by using a recursive proxied object. I can implement a POC, if you guys are interested.

@Banou26
Maybe we can get rid of the Parcel CLI: all options are defined in package.json and .parcelrc. Parcel would force users to do local installs such as

"scripts": {
  "build": "parcel"
},
"dependencies": {
  "parcel": "2.0.0"
}

@jamiekyle-eb

We do not want tools to "wrap" Parcel in such a way that includes implicit configuration either through a Node API or some sort of runtime manipulation of Parcel. Tools like Jest or Next.js doing this today is a huge source of problems users have with build tools.

I agree and I'm actually trying to do something like Next.js but as a Parcel plugin: github.com/brillout/parcel-ssr. But for this to work it seems that Parcel will need to implement changes (parcel/issues/created_by/brillout) and I'm not sure if Parcel is willing to implement these changes I need. If not then there is no other choice for me (and for any other SSR tool author) than to use Parcel with a programmatic API.

@cdll
Copy link

cdll commented Dec 9, 2019

i wonder what's the final MPA entries format? i could not start my parcel tasks with programmatic config below:

// import {
  // Parcel
  // ,ConfigProvider
// } from '@parcel/core'
const Parcel = require('@parcel/core').default

console.info(Parcel)

const pkgConfig = require('../package.json')
console.info(pkgConfig.source)
const entries = [
  ...require('glob').sync('./src/**/*.pug')
]
console.info(JSON.stringify(entries, null, 4))

const browsers = [
  "> 8%",
  "not ie < 10",
  "iOS > 8",
  "Android >= 4.2"
]
const parcel = new Parcel({
  entries
  ,targets: {
    index: {
      browsers
    }
    ,browser: browsers
  }
  ,buildStart (eve) {
    console.info(101, eve)
  }
  ,buildProgress (eve) {
    console.info(302, eve)
  }
  ,buildSuccess (res) {
    console.info(200, res)
  }
  ,buildEnd (res) {
    console.warn(502, res)
  }
  ,watch: true
  ,serve: true
  ,cacheDir: '.cache'
  ,logLevel: 3
  ,target: 'browser'
  // ConfigProvider
  // ,config: {
  // }
})

console.info(parcel)
parcel.run()

but only got [nodemon] clean exit - waiting for changes before restart msg while MPA not bundled~
with [email protected]

@Banou26
Copy link
Contributor

Banou26 commented Dec 9, 2019

@ricardobeat
Copy link

ricardobeat commented Jan 5, 2020

What's the status of multi-entry builds from package.json for SSR in Parcel 2, current alpha? I tried all possible combinations from this thread and elsewhere and could not try it out :(

@Banou26
Copy link
Contributor

Banou26 commented Jan 6, 2020

@krisnye
Copy link
Contributor

krisnye commented Feb 25, 2021

@ricardobeat This merge now makes any combination of multiple entries and multiple targets possible:

#5861

Just add a source to a target and that target will only be built from the specified source instead of the package.json#source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment