Skip to content

uv init: Project layouts, defaults and terminology #8178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MikeHart85 opened this issue Oct 14, 2024 · 25 comments
Open

uv init: Project layouts, defaults and terminology #8178

MikeHart85 opened this issue Oct 14, 2024 · 25 comments
Assignees
Labels
needs-decision Undecided if this should be done

Comments

@MikeHart85
Copy link

Thank you for working on this wonderful tool.

I'd like to suggest several changes to uv init arguments and defaults, with the following goals:

  • Use consistent and established terminology
  • Options to support common usage patterns, so seasoned Python developers can have things their way
  • Defaults which guide Python newcomers towards best practices, hint at available features

Much of this revolves around separating the concept of project directory structure ("layout" in Python) from project purpose and contents ("library", "application", etc), as well as disambiguating the overloaded term "package". Currently these terms are conflated and inconsistent in uv.

Apologies in advance for the massive wall of text.

Current behavior

"Application" (default):

$ uv init [--app] project-name
$ tree project-name/
project-name/
├── hello.py
├── pyproject.toml  # only [project] (builds don't work due to misnamed hello.py)
└── README.md

"Library":

$ uv init --lib project-name
project-name/
├── pyproject.toml  # [project], [build-system]
├── README.md
└── src
    └── project_name
        ├── __init__.py  # contains "hello" code
        └── py.typed

"Application Package":

$ uv init [--app] --package project-name  # --app appears to do nothing here, in spite of docs suggesting it
project-name/
├── pyproject.toml  # [project], [project.scripts], [build-system]
├── README.md
└── src
    └── project_name
        └── __init__.py  # contains "hello" code

Suggested behavior

Src layout (default):

$ uv init [--layout=src] project-name
project-name/
├── pyproject.toml  # [project], [project.scripts], [build-system]
├── README.md
└── src
    └── project_name
        └── __init__.py  # empty
        └── example.py   # def say_hello(): print("...")

Flat layout:

$ uv init --layout=flat project-name
project-name/
├── pyproject.toml  # [project], [project.scripts], [build-system]
├── README.md
└── project_name
    └── __init__.py  # empty
    └── example.py   # def say_hello(): print("...")

Single module layout:

$ uv init --layout=single project-name
project-name/
├── pyproject.toml  # [project], [project.scripts], [build-system]
├── README.md
└── project_name.py  # def say_hello(): print("...") 
                     # NOTE: file name MUST be project_name.py (it replaces the package)

Bare / no layout:

$ uv init --layout=<bare|none> project-name  # maybe pick one rather than allowing either one
project-name/
├── pyproject.toml  # [project]
└── README.md
  • Add --no-entrypoint to suppress [project.scripts]
    • Consider adding --entrypoint=name to customize key name under [project.scripts], defaulting to project-name
  • Add --build-system=<hatchling|...|none>, with hatchling being default, and none to suppress [build-system]
    • Consider adding --no-build-system, synonymous with --build-system=none, for symmetry
    • Remove --no-package
  • Add --typed to create a py.typed
  • Remove --package
  • Remove --app, or:
    • Make synonymous with --entrypoint=project-name
    • Consider generating example code to be more "app-like"
    • Consider __main__.py [1] instead of example.py, but that may unnecessarily confuse newcomers
  • Remove --lib, or:
    • Make synonymous with --no-entrypoint
    • Consider generating example code to be more "lib-like"
    • Consider implying --typed (IMO explicit uv init --lib --typed would be better)

I would just remove --lib and --app to keep things simple.

Reasoning for suggestions

  • "Entry point" is the established term for shortcuts to functions that ultimately become commands when installed [2]
  • "Layout" is the established term for project directory structure [3], [4]
    • Src layout should be default
      • It is the most robust, avoids import conflicts, most suitable for serious projects
      • Prevents needing to restructure when project outgrows other layouts
      • Uv eliminates its main downside (very easy to get up-to-date REPL):
      • uv run python
      • >>> import project_name
    • Flat layout is also quite popular, and currently not available in uv init
    • Single module (current --app, default) layout is the most limited
      • It should most certainly not be the default
      • Has issues and limitations which will confuse newcomers
      • Should really only be used for single file projects, if at all
      • Currently broken for builds due to example file always being hello.py
      • Changing file to project_name.py allows builds to work
    • --layout=none for custom layouts, notebook projects, etc
  • Project layout is not related to whether it is an "app" or "lib"
    • Any layout can be an application or a library
    • Project contents and usage determine whether it is an application, or library, or both
    • Not as clear cut in Python as "executable" vs "library" projects in compiled languages, shouldn't attempt to force that pattern since it doesn't fit
    • "App" and "Application" terms should probably be reserved for possible integration with tools like PyInstaller, cxfreeze, etc (IE, generating a self-contained binary executable for distribution)
    • A library isn't required to have PEP 561 compliant py.typed
  • Current use of --package conflates it with build-system/distribution, src layout, and absence of py.typed
    • A "package" in Python is simply a directory with an __init__.py [5]
    • Ideally, the term "package" should not be used for anything else, to avoid confusion
    • A package doesn't have to be distributable / have a build-system
    • Single module (IE, non-package) distributions are a thing (hence --layout=single with build-system)
      • A prominent example would be six, which is a single module library: PyPI GitHub
  • __init__.py should not contain general purpose code
    • __init__.py is a special purpose file, and putting arbitrary code there can easily have unintended side effects
    • Its meant for initializing packages, making symbols available at package level, __all__, etc [6]
    • Putting the hello world example there can mislead newcomers into thinking this is where all their code belongs
  • Don't see a reason not to include entry point and build system by default in all layouts which support it
    • Can easily be removed / ignored / switched off if not needed
    • Default presence shows newcomers what is possible / where it belongs
    • Reduces friction when a project graduates from hobby to distribution
  • Layouts are mutually exclusive, hence grouping them under one argument with a value
    • Less room for confusion than having multiple differently named options, which may or may not be mixable
  • All three layouts (src, flat, single) work with uv run project-name and uv build as shown
    • Only uv init needs to change, to support generating them

Context

Discussion in the latter floated using "distribution" (ala PDM) instead of "package" (ala Poetry) in some contexts, but it seems "package" won out. I would argue "distribution" would have been the correct choice. As mentioned, "package" has a very specific meaning in Python, not strictly related to whether the project is built for distribution.

Thanks for considering.

@Weilet
Copy link

Weilet commented Oct 24, 2024

I think uv init --layout=<bare|none> is very useful for me when it comes to creating a Django app. I am searching for a way to initialize the project with only a pyproject.toml, so that I can use django-admin to manage my project without too much unnecessary stuff.

@zanieb zanieb self-assigned this Oct 24, 2024
@zanieb zanieb added the needs-decision Undecided if this should be done label Oct 24, 2024
@zanieb
Copy link
Member

zanieb commented Oct 24, 2024

Thanks for your thoughts! I'll own responding to this and seeing what we can integrate.

@drcongo
Copy link

drcongo commented Nov 2, 2024

I've only ended up in this thread because I was confused about the default behaviour here - to me it feels like uv init with no options should do the equivalent of uv init --layout=<bare|none> mentioned above and create nothing but a pyproject.toml. I can't imagine ever needing uv to create the .gitignore, .git/ and hello.py so for that to be the default seems kinda wild.

Feel free to ignore me though, I realise I'm a data point of one and maybe those do somehow make sense to other devs.

@thcrt
Copy link

thcrt commented Nov 6, 2024

I think both the options available and their documentation are somewhat confusing at present. There are multiple different decisions in determining how a project should be structured, but those decisions seem to be split between uv init's sub-optimal presets and further configuration options in pyproject.toml that need to be set manually.

Often, I might want to create a standalone application, such as a Flask webapp. This doesn't need to expose a Python API or be imported into other projects, so I would assume I don't need a build system to turn my code into a package. I'll probably just write a Dockerfile that will produce a container image with my dependencies installed, ready to run my code directly.

In terms of layout, I probably want a 'flat layout', where my code is in ./myapp. This is what Flask, for instance, recommends. That's not available from uv init -- I have to choose between:

  • a 'Packaged Application'
    This configures a build system I don't need and nests all my code in ./src/myapp.
  • a 'Library'
    This has all the downsides of a 'Packaged Application', and also doesn't specify a command-line entry point.
  • an 'Application'
    This puts my code in ., the project root, which will get cluttered quickly if I want to build anything bigger than a single-file script.

None of these do what I want! No matter what I choose, I'll have to move things around and change pyproject.toml to account for the changes.

This is addressed in #6460, which seems to be resolved by #6585, adding 'Virtual Projects', which don't get built or installed as packages. But the only place this is documented is under the setting that enables it, where I can read a brief summary of what a virtual project is, but not why I might want to use one. #6585 states that a project will be treated as 'virtual' merely by the lack of a [build-system] in pyproject.toml, but this isn't documented there. The documentation on uv init doesn't mention 'virtual projects' at all.

A brief aside regarding 'virtual projects' and uv init: it seems like uv init --virtual creates a virtual project, but the --virtual flag can't be used with --package (for a 'Packaged Application') or with --lib (for a 'Library'). And uv init --virtual produces exactly the same result as uv init without the --virtual flag, not even adding package = false to pyproject.toml. So it's not clear to me what the flag is supposed to do.

I love uv as a whole, but this is extremely confusing to me. Maybe I'm mistaken on some of this, and I'm doing something wrong, in which case please feel free to correct me.

@junapur
Copy link

junapur commented Nov 19, 2024

I agree. As someone migrating from Poetry to uv, uv's project layout options were a lot harder for me to understand.

@jromal
Copy link

jromal commented Dec 8, 2024

I've only ended up in this thread because I was confused about the default behaviour here - to me it feels like uv init with no options should do the equivalent of uv init --layout=<bare|none> mentioned above and create nothing but a pyproject.toml. I can't imagine ever needing uv to create the .gitignore, .git/ and hello.py so for that to be the default seems kinda wild.

I can follow the logic for not wanting a "hello.py", but I can fully understand that an example (in the terms mentioned by the request) is a good thing for most of the people. I cannot support not creating the ".git" and ".gitignore" files. All amateurs (which I count myself) should have it, but many do not know about it. Nothing serious should be done without them.

What we need is a revamp on the "layout" structure. Which is confusing. I had to test all the options to understant what is meant. With the "--layout" option is clear and does what people do. .

@clbarnes
Copy link

I can't imagine ever needing uv to create the .gitignore, .git/ and hello.py so for that to be the default seems kinda wild.

Counterpoint: I can't imagine ever needing to start a python project without a .gitignore file, and it's annoying to go and fetch the exact same file every time, when uv aims to be a one-stop shop. Of course, a minority of people don't use git for version control, although a host of other dev tools respect .gitignore when it comes to ignoring files to search/ index/ include so arguably it's useful to have even if you're not using git.

I think that hello.py, while deleted/ renamed immediately by developers who start new projects regularly, acts as a useful marker for where your functional code should go. Between script, app, and library projects, flat or src layouts, multi-package projects etc., that may not be clear to someone less experienced, or even an experienced developer who hasn't followed the breakneck pace of the evolution of python's best practices in the last few years.

@drcongo
Copy link

drcongo commented Jan 22, 2025

@clbarnes Not everyone is starting a new project, nor running uv where their .gitignore, .git repo or README lives - think mono-repos, projects where uv is inside docker, projects where you're in a git submodule etc. - in some of these cases, creating a .git repo is a positively hostile thing to do as you can suddenly end up with nested repositories being pushed and then someone (me) has to untangle it.

@allenc97
Copy link

Lots of insightful thoughts. I would also argue for project layout to be separated from rather the project is intended to be an application or a python package. In fact, it is not uncommon for python applications to also use a src layout. This is especially useful with containers as mounting all source code files is just a singular mount, or for standardized use in CI/CD.

@Bunker-D
Copy link

Bunker-D commented Jan 28, 2025

I second the idea of a more customized option for uv init. I would also like the option to setup a personal default configuration.

Additionally, it would be neat to be able to customize the starting .gitignore (at least, setting up a personal default .gitignore).

Typically, I would systematically exclude .*_cache (.pytest_cache and .ruff_cache) and .venv, and I'm pretty sure developers' needs would change based on their favorite tools.

@Bunker-D
Copy link

Also, while I don't disagree with @clbarnes 's justification of hello.py (making me neutral about the creation of hello.py), I think src/<…>/__init__.py should not be initialized with code inside it. My big issue with this behavior is that this dummy code not visible at first sight when looking at the project. Worse than having to clean it up, devs might miss it,

zanieb added a commit that referenced this issue Feb 5, 2025
People are looking for a less opinionated version of `uv init`. The goal
here is to create a `pyproject.toml` and nothing else. With the `--lib`
or `--package` flags, we'll still configure a build backend but we won't
create the source tree. This disables things like the default
`description`, author behavior, and VCS.

See

- #8178
- #7181
- #6750
@chrish42
Copy link

Also coming here because I love the uv experience overall... except for its package layout creation options, which I found really confusing. I wanted to create a src-layout for a library that was going to be packaged and released eventually, and by far most of my time struggling with uv was spent trying to figure out how to get that config.

The --library, etc. options are not great currently, because they bundle too many things together. It's totally fine to have defaults for new users (and please do that). But if I want a src-layout for my project (whatever it is), it would be so much easier to just ask explicitly ("better than implicit") for it by tacking on a --layout=src option to my uv init invocation.

Also, I was kind of expecting this choice of layout to be also reflected in the section for uv of the pyproject.toml file, at least, so all uv invocations know about it. Right now, I have a makefile whose main function is to tack on uv run --project src to all the commands I need to run during development.

For me, this is the only thing making me hesitate to recommend uv more broadly to my team.

@zanieb
Copy link
Member

zanieb commented Feb 11, 2025

The --library, etc. options are not great currently, because they bundle too many things together. It's totally fine to have defaults for new users (and please do that). But if I want a src-layout for my project (whatever it is), it would be so much easier to just ask explicitly ("better than implicit") for it by tacking on a --layout=src option to my uv init invocation.

What things did you not want?

@zanieb
Copy link
Member

zanieb commented Feb 11, 2025

Right now, I have a makefile whose main function is to tack on uv run --project src to all the commands I need to run during development.

I'm a little confused by this, you want to run with your working directory as src/?

@hunter-gatherer8
Copy link

hunter-gatherer8 commented Feb 20, 2025

How about allowing users to provide their own configuration template in some uv config directory? Or even several templates one could choose with something like uv init --template=my-named-preset?

@zanieb
Copy link
Member

zanieb commented Feb 20, 2025

@hunter-gatherer8 please see #9754

@ivan-kleshnin
Copy link

ivan-kleshnin commented Feb 27, 2025

Can someone take a little time to elaborate the reasoning behind "src" layout for non-Pythonistas 🙏

With "flat" layout:

project_name/
  project_name/

we already have a folder-level isolation. It does look "nested" to a TypeScript-first version of me 😄 I get that the project can have multiple packages and those packages will, supposedly, have common files under project_name/src. But is there any benefit for simple (and probably more frequent) one-package-per-project cases?

Quoting the article, that everyone refers to:
https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/

The src layout requires installation of the project to be able to run its code, and the flat layout does not.

I don't get this point. With Poetry I had to install my library package, and it definitely has the "flat" layout.

The src layout helps prevent accidental usage of the in-development copy of the code.

This point is more clear. But it's a "protection by convention", not the strongest argument. We can still accidentally put a non-public data or code in src. So, again, I would like to learn pros/cons of each layout better, maybe someone can suggest another article or video?

@astrojuanlu
Copy link

astrojuanlu commented Feb 27, 2025

Can someone take a little time to elaborate the reasoning behind "src" layout for non-Pythonistas 🙏

import package_name may pick up the code from your source checkout and not the installed one depending on your current working directory. src prevents this

@konstin
Copy link
Member

konstin commented Feb 27, 2025

https://hynek.me/articles/testing-packaging/ and https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure have some discussion of the topic.

@clbarnes
Copy link

clbarnes commented Feb 28, 2025

Can someone take a little time to elaborate the reasoning behind "src" layout for non-Pythonistas 🙏

Python was originally designed as a scripting language, more akin to bash or perl than Java. As such, it's pretty liberal about sweeping local directories for python code to import and execute. However, there are a few different ways of executing python code and that module discovery happens differently in different execution contexts. Some tools designed with that style in mind (e.g. testing utilities) also do things under the hood to make things "easier", which actually makes module discovery even more complicated.

As python has been used for bigger, more complicated projects, this module discovery complexity has been found to add little utility and a lot of headaches. Isolating your code in a src folder prevents it from being auto-discovered in most cases, which improves consistency, puts the developer in control of what is imported and how, and makes your development environment behave more similarly to your deployment environment.

Python as a language has changed to support bigger, more complicated projects (e.g. typing), as has the tooling around it (e.g. uv!).

Also it helps separate the actual module from general project-related cruft like CI configuration, scripts, infrastructure setup, docs etc..

@matthias-busch
Copy link

Coming from using poetry I also find the uv init options a bit confusing and convoluted.

I personally would prefer the pretty simple/bareback approach of poetry's poetry new some-name for a project skeleton that looks like this:

❯ tree bar-delete-later
bar-delete-later
├── README.md
├── bar_delete_later
│   └── __init__.py
├── pyproject.toml
└── tests
    └── __init__.py

and poetry new --src some-name resulting in:

❯ tree foobar-delete-later
foobar-delete-later
├── README.md
├── pyproject.toml
├── src
│   └── foobar_delete_later
│       └── __init__.py
└── tests
    └── __init__.py

This might be a simple case of personal preference and I totally (and welcome) uv's intent to being as user-friendly as possible but I personally think pruning back on the options and just giving options for the src/flat layout would suffice and more straightforward.

I would also love to have the tests folder generated by default for both approaches.

@jlcheng
Copy link

jlcheng commented Mar 7, 2025

I am grateful that this conversation is happening here. As someone who recently needed to setup a Python project, I ran into the flat vs src decision myself. In the end, I decided that I would use the src layout as my default, as well as making the choice to use uv as my preferred tool. But the src-vs-flat choice in Python will probably never go away and I think it won't be easy for uv to, for the lack of a better phrase, "make the right choice." There won't be a perfect answer.

Having said that, I think simply discussing this matter in the uv documentation will go a long way to helping the community, especially people new to Python project layouts like me. Today, the uv documentation (https://docs.astral.sh/uv/concepts/projects/init/) discusses the difference between "Application", "Packaged Application", and "Library". This page could be a good place to discuss why the uv team chose Application as the default and what the implications are (e.g., perhaps referencing https://hynek.me/articles/testing-packaging/).

This is akin to the practice of keeping architectural decision records (https://adr.github.io/). The goal is not to justify that the current decision to make Application the default is the right one--people will always debate such a decision. The goal is to give users context about how the decision was made at the time, which empowers users to better evaluate the alternatives given their own situations and future evolutions in the Python community.

Thanks again for considering this topic. And many thanks to @MikeHart85 for his thoughtful and intelligent proposal.

@zanieb
Copy link
Member

zanieb commented Mar 7, 2025

This page could be a good place to discuss why the uv team chose Application as the default and what the implications are (e.g., perhaps referencing https://hynek.me/articles/testing-packaging/).

I think this is summarized in #3957 (comment) — we don't want this to be the default longterm.

We intend to revisit this entire interface once our build backend is ready.

@carlosferreyra
Copy link

Thank you for pointing this proposal!
I've been following this thread, but I'm a noob in the open source community, so a little shy to give any opinion.

Proposal

following with @zanieb ideas and the philosophy of uv, for becoming the cargo of python (or npm/bunof python):
Having uv as a tool for scaffolding an initial template is a must.

What it goes in my mind, is that you can set a basic "PEP compliant version of project initialization" for uv init (specially when it comes matching pyproject specs., and use uvx for anything that has to do with custom templates (including those that work with cookiecutter or copier. This follows 3 goals:

  • keeps a clean workflow for the uv client, knowing that creating a project is a one time action during the lifetime of the project itself.
  • Maintain strong compliance with PEP without sacrificing out-of-the-box features from uv.
  • Make a equivalent use of what it wasnpx create-react-app|npx create-expo-app and would be uvx create-{some-python-template}

Also, you could use the proposal as a way to migrate from current out the box options out there in the python community, to a more uv compliant version of it. Benefits:

  • Decrease of python community friction, when migrating to the uv ecosystem.
  • Leave uvx as an entrypoint for extended scaffolding models (i.e, even one for cloud-based or AI projects), and even using the community popular choices as a possible builtin-template for uv.

An example of this was the uvx migrate-to-uv script which allows to migrate from poetry to uv, but this very concept could also be applied for scaffolding templates from scratch.

Possible Drawbacks

  • Dealing with a new set of standards when designing a new template through uvx and have the community to adhere to those standards
  • Increased complexity in maintaining two separate but related commands (uv init and uvx).
  • Potential confusion for new users about when to use uv init versus uvx.
  • The need for significant effort to define and maintain the "PEP compliant" template for uv init, ensuring it remains up-to-date with evolving Python standards.
  • Reliance on the community to create and maintain high-quality templates for uvx. If the community uptake is slow, the uvx ecosystem might lack useful options.
  • Overlap in functionality with existing tools like cookiecutter or copier might lead to fragmentation or resistance from users already comfortable with those tools.

Basic Structure

The uv init purpose should be to set up the least amount of settings for the project, since even when you need scaffolding, you also need to keep the option for a full customization from scratch. having said that, I agree with the initial proposal of the issue, but with a small caveat: all templates, no matter if flat,lib, or package should have the /src folder, since its a good widely accepted standard for the dev community, and also separates the project configuration from any project implementation (even for frontend devs).

Easter Egg

the team could even create a dedicated package for using the uvx client, to build templates (uvx start-project --with cookiecutter|copier|[uv])... and those templates can be selected by the uv team to comply with the uv ecosystem.

@tmct
Copy link

tmct commented Apr 10, 2025

uv is a great tool for managing virtual environments, even outside of packages, so I'm wondering: should there be a template for e.g. --venv, which just gives you all you need to start managing a virtual environment?

.git
.gitignore
pyproject.toml  # (with no [build-system])
.python-version

Currently, uv init --no-readme --no-package gets you close to this but creates an undesired "main.py", and uv init --bare is good but misses .python-version, and the git files. So - potentially a --no-main option, or allowing --vcs git etc to work with --bare would achieve this - but perhaps this use-case is worthy of its own template?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-decision Undecided if this should be done
Projects
None yet
Development

No branches or pull requests