Skip to content

Decouple Turbo tasks from package.json scripts And allow more automation #1495

Closed
@zaripych

Description

@zaripych

Describe the feature you'd like to request

I've been thinking about this for some time now. The idea keeps nagging me. Imagine - I have a dts task that I can run for every package in my monorepo, but that task doesn't have any value for the end user (read developer in a team using the monorepo) by itself - meaning user will never run that task manually via turbo run or pnpm run. The task is needed to run another dts-bundle task - essentially dts produces an intermediate artefact that allows dts-bundle that depend on it to succeed without errors. Now the problem is that dts-bundle needs to run dts for all dependencies.

This seem to work nicely:

  "pipeline": {
    "dts": {
      "inputs": [...],
      "outputs": [".tsc-out"]
    },
    "dts-bundle": {
      "dependsOn": ["^dts"],
      "inputs": [...],
      "outputs": ["dist/**/*.d.ts"]
    },
  }

But little issues creep up - mostly scalability:

  • Every package has to have that dts task declared in package.json scripts - it probably wouldn't make sense to have to copy-paste it everywhere since the task is basically single script that can work in every package sub-folder and produces output in that package sub-folder, but has no input other than source code and doesn't need any configuration on per package basis
  • The dts task must be run for every dependency of a package where we about to run dts-bundle, but a developer might forget to add a script to package.json for one of those dependencies - when adding that dts script they might also make a mistake and pass wrong parameters or things like that
  • When a developer decides that they want to start using dts-bundle for their package they have to go and check every dependency and whether that dependency has dts task declared. Hopefully it's very easy change and a quick PR away, but those packages could be owned by other teams.

In addition to that it's also possible that inputs of some other build task in one package is different to another package's inputs. But the fact that those differences should be specified in the root of the monorepo in a single file is kind of limiting. If we look at monorepos with huge number of packages I feel single turbo.json might become a bottleneck (with some merge issues maybe even?). Sometimes a script can determine those inputs/outputs programmatically and can operate in the entire monorepo for any sub-package. E.g. I imagine a script could determine all the dependent *.ts files imported and build a graph of files that it depends on (inputs) - and that would probably be more efficient than using globs like src/**/*.ts that could hit hundreds of files.

Another script could determine which dependencies need to have dts task run and which don't. There is generally a lot of things that can and should be automated and turbo doesn't provide too much here. The CLI could be considered as API and turbo.json can be generated - but it makes it harder to use hybrid approach - use ability of the turbo to cache and also allow devs to use turbo.json without having to deal with a script.

Describe the solution you'd like

  1. Allow specifying commands right in the turbo.json, example:
  "pipeline": {
    "dts": {
      "command": "node ./monorepo-scripts/generateDts.js",
      // ^-- no need to copy-pasta in every package
      "inputs": [...],
      "outputs": [".tsc-out"]
    },
  }

turbo would assume the command exists in every package as if it was added to every package's scripts config.

  1. Allow passing location of a custom turbo.dts.json file OR preferably - allow it to be passed as stdin. If location of the turbo.json implies root of the monorepo we could also add extra parameter or assume it's cwd. This will make turbo super powerful tool that would allow caching any task - whether it can be described by turbo.json manually or generated by a script.

Both of the above seem like a simple change on the surface. But there must be some caching implications. E.g. we would need to store hash of the stdin config somewhere which could serve as a cache key for the rest of the data associated with the run ...

Describe alternatives you've considered

Generating turbo.json is possible, but writing scripts to every package.json - seem like a less desired path as it's often modified manually.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions