Contributing to Plone core: skip buildout, use magic

[ksuess]

buildout.coredev can do without buildout.

@mauritsvanrees and @jensens created a Makefile support of a dev setup with straight forward package installation (uv) instead of buildout. This is awesome.

Fix plone/buildout.coredev#1014

---

📚 Documentation preview 📚: https://plone6--1936.org.readthedocs.build/contributing/core/index.html#install-plone-core-for-development

[stevepiercy]

I'm not clear what the intent here is. I've heard arguments to both remove and keep buildout.

Personally, I think both buildout and pip should be removed as methods for contributing and installation, and move to uv and Cookieplone. It's absurd to have three methods to get started with Plone. There should be one and only one method.

Given the lack of accurate documentation for both buildout and pip methods, I'm fine with letting them wither and die.

[ksuess]

it’s draft. Versuch dich in Kontemplation.

[jensens]

the mx* method in fact uses uv

[stevepiercy]

The release managers ought to be involved in the switch from buildout to make. Has there been a discussion, decision, or announcement to switch?

This page was updated in #1671, taking over a month of review, discussion, and revisions. All the participants in that PR should also be involved in the discussion to switch. I'll request a review from them all.

I can't remember exactly, but there were some arguments made for keeping buildout. However, maybe that is not in the context of developing Plone core?

[davisagli]

This part is fine, but sections later in the file assume that Plone was installed using buildout. (For example, the "Edit packages" section shows how to check out a new package from source by editing buildout.local.cfg.) We can't update only the installation instructions without also updating the other instructions that assume installation was done using buildout.

[mauritsvanrees]

@jensens see my comment #1936 (comment). Do we need to checkout plone/documentation?

Initially this was done in 2021 in this PR by Jens to ease contributions to the documentation. It can be useful to have the documentation right there in the coredev buildout.

But for me it would be fine to remove it from there. The .git directory of the docs is 434 MB. Even with a git clone depth of 1 (by CI on Jenkins or on GitHub Actions), it is still 11 MB that we pull in each time, even when it has no influence on the tests.

Also, I can never remember which directory contains the Plone documentation and which the few pages of the more internal buildout.coredev documentation. Locally I have this:

• docs (coredev docs)
• documentation/docs (docs.plone.org, where I never liked that this ends up in a sub directory)
• src/docs (same as the second one, but this is probably from a time when I did not have the new Makefile and related files configured correctly yet)

[mauritsvanrees]

Actually, when you run buildout, the documentation gets cloned to documentation/docs, but with make sources it ends up in src/docs. It seems to be configured fine in mxsources.ini:

[settings]
docs-directory = documentation
...
[docs]
...
target = ${settings:docs-directory}

It seems the target is ignored. Seems an error in mxdev.
I will open an issue there. And I am making a PR for buildout.coredev to remove the clone of the documentation.

[mauritsvanrees]

This looks good. Thank you!

I have been busy making changes to the Makefile part in plone/buildout.coredev#1020, which is ready for review. That is for Plone 6.2, but most of it can and should be copied to 6.1 as well. We can make improvement to the docs once that has landed (and has been battle tested).

But for now this docs PR is fine. Can be merged.

[stevepiercy]

It can be useful to have the documentation right there in the coredev buildout.

It's OK to keep it, too. But if we keep it, we need to either update the command in the documentation to run all tests to .venv/bin/pytest --ignore=src/docs, create a Make command that does that instead such as make test-all, or create some other workaround.

I'll go along with what y'all think is best.

[davisagli]

Nice work, thank you!

[davisagli]

Why Classic UI specifically? I often install the Volto distribution if I'm working on backend changes that I want to test with Volto.

[ksuess]

Yes, of course, this should not recommend one of the distributions.

[stevepiercy]

@ksuess @davisagli the instructions on this page do not cover starting the Volto frontend and don't include how to start to develop in Volto. In fact, the second sentence of this page states the following.

Although Plone core includes Volto—the React based, default frontend for Plone 6—this guide does not apply to Volto and its packages. To contribute to Volto, see Contributing to Volto.

Unless and until the instructions on this page actually support Volto development, then we should use the previous version of this sentence, telling developers to select Classic UI.

In my testing, I couldn't even visit the Volto frontend because it doesn't get started. Maybe there is a way to do that, and if there is, then let's get it into this page.

[davisagli]

@stevepiercy When I'm working on something that involves changes in both Volto and in the backend, I start Volto from the volto repository (as covered in https://plone6--1936.org.readthedocs.build/volto/contributing/developing-core.html#start-plone) and then I start the backend from the buildout.coredev repository (instead of running make backend-docker-start from the volto repository, which doesn't give the option to check out backend packages) and then create a site using the Volto distribution.

I'm totally okay with leaving that as out of scope for this PR (let's get this merged), but I think we should explain it somewhere (maybe it belongs under "Contributing to Volto", but it's really about contributing to the backend and then testing those changes with Volto before they're released)

[davisagli]

(Maybe we should add a make frontend-docker-start command to buildout.coredev to parallel make backend-docker-start in the Volto repository, in case you just want to get the most recent Volto release running quickly for testing without cloning it separately)

[stevepiercy]

@stevepiercy When I'm working on something that involves changes in both Volto and in the backend, I start Volto from the volto repository (as covered in https://plone6--1936.org.readthedocs.build/volto/contributing/developing-core.html#start-plone) and then I start the backend from the buildout.coredev repository (instead of running make backend-docker-start from the volto repository, which doesn't give the option to check out backend packages) and then create a site using the Volto distribution.

Are you saying that you have two clones, one each for volto and buildout.coredev as siblings or something?

I'm totally okay with leaving that as out of scope for this PR (let's get this merged),

Yeah, I think it is out of scope here.

but I think we should explain it somewhere (maybe it belongs under "Contributing to Volto", but it's really about contributing to the backend and then testing those changes with Volto before they're released)

Are you saying, for example, adding or editing a plone.restapi endpoint in your buildout.coredev checkout, then running tests in your volto checkout? If so, we don't have that in the docs yet. It never occurred to me to have the two checkouts talk to each other. I guess as long as the backend runs on the same host and port that the volto checkout requires, it's all good, right?

[davisagli]

@stevepiercy Yeah, that's right, there's no actual configuration to make the two repositories work with each other, it's just based on the convention of running the backend on port 8080, and Volto expecting one there by default.

[davisagli]

@stevepiercy Okay to merge from your perspective?

[stevepiercy]

@davisagli see my comment at #1936 (comment). I'll try again later today to see if that is still the case, blocking a merge, unless someone else can verify first.

[davisagli]

@stevepiercy We removed the documentation from being included in plone/buildout.coredev#1022, so I don't think there's more work needed in relation to that comment.

[stevepiercy]

@davisagli @ksuess see comment at #1936 (comment). I reverted that change.

I also ran through the page to ensure it works as described and intended, and the plone/documentation repo is no longer checked out, so .venv/bin/pytest works.

=== 2 failed, 871 passed, 6 skipped, 4200352 warnings in 624.19s (0:10:24) ===

So the documentation is correct, but there are some broken tests.

Merging as soon as it's all green.

I'm also happy if someone wants to expand this page to include contributing to Volto.