Skip to content

/documentation/specs/*.md formatting/linting/cleanup #11611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion documentation/specs/event-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
[EventSource](https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.tracing.eventsource?view=netframework-4.8) is the tool that allows Event Tracing for Windows (ETW) used in MSBuild. Among its useful features, functions with names ending in "start" and "stop" correlate between calls such that it can automatically record how long the event between the two calls took. It also provides an easy way to cheaply opt in or out, log auxiliary messages in addition to time, and add progress updates in the middle of an event as needed.

## EventSource in MSBuild

EventSource is primarily used to profile code. For MSBuild specifically, a major goal is to reduce the time it takes to run, as measured (among other metrics) by the Regression Prevention System (RPS), i.e., running specific scenarios. To find which code segments were likely candidates for improvement, EventSources were added around a mix of code segments. Larger segments that encompass several steps within a build occur nearly every time MSBuild is run and take a long time. They generally run relatively few times. Smaller methods with well-defined purposes may occur numerous times. Profiling both types of events provides both broad strokes to identify large code segments that underperform and, more specifically, which parts of them. Profiled functions include:

| Event | Description |
Expand All @@ -20,7 +21,7 @@ EventSource is primarily used to profile code. For MSBuild specifically, a major
| ExecuteTaskYield | Requests to yield the node, often while the task completes other work. |
| ExpandGlob | Identifies a list of files that correspond to an item, potentially with a wildcard. |
| GenerateResourceOverall | Uses resource APIs to transform resource files into strongly-typed resource classes. |
| LoadDocument | Loads an XMLDocumentWithLocation from a path.
| LoadDocument | Loads an XMLDocumentWithLocation from a path. |
| MSBuildExe | Executes MSBuild from the command line. |
| MSBuildServerBuild | Executes a build from the MSBuildServer node. |
| PacketReadSize | Reports the size of a packet sent between nodes. Note that this does not include time information. |
Expand Down
2 changes: 1 addition & 1 deletion documentation/specs/low-priority-switch.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ Visual Studio, on the other hand, should always run at normal priority. This ens
4. Any reused nodes are at the priority they themselves specify. Normal priority nodes are actually at normal priority, and low priority nodes are actually at BelowNormal priority.
5. All nodes are at the priority they should be when being used to build even if a normal priority process had connected to them as normal priority worker nodes, and they are now executing a low priority build.


## Non-goals

Perfect parity between windows and mac or linux. Windows permits processes to raise their own priority or that of another process, whereas other operating systems do not. This is very efficient, so we should use it. As we expect this feature to be used in Visual Studio, we anticipate it being less used on mac and linux, hence not being as high priority to make it just as efficient.
Expand All @@ -27,6 +26,7 @@ Perfect parity between windows and mac or linux. Windows permits processes to ra
Each node (including worker nodes) initially takes its priority from its parent process. Since we now need the priority to align with what it is passed instead of its parent, attempt to adjust priority afterwards if necessary as part of node startup.

BuildManager.cs remembers the priority of the previous build it had executed. If that was set to a value that differs from the priority of the current build:

1. On windows or when decreasing the priority: lowers the priority of all connected nodes
2. On linux and mac when increasing the priority: disconnects from all nodes.

Expand Down
73 changes: 41 additions & 32 deletions documentation/specs/project-cache.md

Large diffs are not rendered by default.

11 changes: 8 additions & 3 deletions documentation/specs/question.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,19 @@ Question switch ask if the next build is up-to-date. It will start a build, but
[Fast Up-To-Date Check](https://github.com/dotnet/project-system/blob/cd275918ef9f181f6efab96715a91db7aabec832/docs/up-to-date-check.md) is a system that is implemented by the Project System, that decides, if it needs to run MSBuild. MSBuild takes a non-trival amount of time to load, evaluate, and run through each target and task. Fast Up-To-Date is faster, but can be less accurate, suitable for an IDE and a human interface. It is not accurate enough for a CI.

## Usage

Question mode is designed to be used on the command line. Run your normal build, then run again with /question.
```

```cmd
msbuild /p:Configuration=Debug Project1.csproj /bl:build.binlog
msbuild /p:Configuration=Debug Project1.csproj /bl:incremental.binlog /question
```

If there are no errors, then your build is up-to-date.
If there are errors, then investigate the error. See common errors below. Keep both logs to help with your investigation.

## Custom Tasks

Task author can implement the optional `IIncrementalTask` interface that will expose `FailIfNotIncremental`. `FailIfNotIncremental` is true when /question switch is used. The custom task will need to decide how it want to handle their behavior. For example. If there is already a message describing why the task cannot be skipped, then simply convert the message to a error. Remember to return false to stop the build. For the best reproducibility, do not modify any files on disk.

```C#
Expand All @@ -32,6 +36,7 @@ else
```

## Shipping Tasks

When question switch is used, it will modify the shipping task with these behavior. Note: this is still experimental and can change.

`Exec`
Expand Down Expand Up @@ -73,11 +78,11 @@ Error when SkipUnchangedFiles is true.
`ZipDirectory`
Error if the destination zip file doesn't exists.


## Common Error

- **Typographical error**. Spelling, casing, or incorrect path. Check if the target inputs and outputs real files.
- Inputs and Outputs are sometimes used for Cross Product. Try to move all to Outputs. If not possible, use Returns instead of Inputs.
- **Double Checks**. Since target and task could be incremental, if both are implemented, then it can lead task skipping but not the task. For example, a Target has inputs A and outputs B. If A is newer, than B, then the target will start. If the task compares the content of A and B and deems nothing has changed, then B is not updated. If such case, this leads to target rerunning.
- **Exec Task** are not Skipable, thus they should be wrapped with Target Inputs and Outputs or other systems. For backwards compatibility, Question will not issue an error.
- **FileWritten**. The common clean system will remove files that aren't in the FileWritten itemgroup. Sometimes task output won't be add to FileWritten itemgroup.
- **Build, then Build**. Sometimes, a 2nd build will break up to date. Question after the 2nd build.
- **Build, then Build**. Sometimes, a 2nd build will break up to date. Question after the 2nd build.
2 changes: 2 additions & 0 deletions documentation/specs/rar-core-scenarios.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@ effect. Be it eliminating allocations, simplifying tight loops, reordering cases
address the elephant in the room: the file I/O resulting from scanning of assemblies, checking their timestamps, and reading/writing on-disk caches.

For regular project references the system works as about as efficient as possible.

- In a cold scenario, where there is no state in memory or on disk, the referenced assembly file has to be scanned for its name and dependencies.
- In a warm scenario, where there is no state in memory but a disk cache exists, the assembly name and dependencies are read from the cache, together with the
corresponding timestamp which is compared to the current timestamp of the assembly file. If they match the cached data is used.
Expand Down Expand Up @@ -188,6 +189,7 @@ enlistment - the system may prime by building the full solution and then the dev
cache and get sub-optimal first-time build performance.

Saving of the per-project disk cache may be further optimized by

- Keeping the timestamp of the cache file in memory and skipping the save if the relevant cache items haven't become dirty (i.e. the dependencies have not changed)
*and* the timestamp of the cache file hasn't changed since the last save. In hot inner loop scenarios this would reduce the save to a timestamp check.
- Saving the file asynchronously, i.e. not blocking the build on completing the save operation.
Expand Down
7 changes: 4 additions & 3 deletions documentation/specs/remote-host-object.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ A remote host object must be registered in the [Running Object Table (ROT)](http

[The registration of interfaces](https://docs.microsoft.com/en-us/dotnet/framework/interop/how-to-register-primary-interop-assemblies) is the only thing interop with COM that need extra care. There are 3 interfaces involved in out-of-proc tasks work: `IVsMSBuildTaskFileManager`, `IPersistFileCheckSum` and `ITaskHost`. `IVsMSBuildTaskFileManager` and `IPersistFileCheckSum` are registered globally in Windows registry by VS existing setup. `ITaskHost` is also configured in VS using registration-free. So the only work is to configure it using registration-free in **MSBuild**. That results the change in msbuild.exe.manifest file and the change to generate tlb file for ITaskHost.

## Annotated additions to the msbuild.exe.manifest file.
```
## Annotated additions to the msbuild.exe.manifest file

```xml
<file name="Microsoft.Build.Framework.tlb"> -- Location of the tlb, it should be in the same directory as msbuild.exe
<typelib
tlbid="{D8A9BA71-4724-481D-9CA7-0DA23A1D615C}" -- matches what is embedded in the tlb with ITaskHost
Expand All @@ -24,7 +25,7 @@ A remote host object must be registered in the [Running Object Table (ROT)](http

If is part of the work for [allowing out-of-proc tasks to access unsaved changes](https://github.com/dotnet/project-system/issues/4406)

## More reference:
## More reference

[RegFree COM Walkthrough](https://msdn.microsoft.com/library/ms973913.aspx)

Expand Down
33 changes: 20 additions & 13 deletions documentation/specs/sdk-resolvers-algorithm.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,26 @@
## SDK Resolution Algorithm
# SDK Resolution Algorithm

In 17.3 under ChangeWave 17.4 the sdk resolution algorithm is changed.

### Reason for change
## Reason for change

Previously (before ChangeWave 17.4) all SDK resolvers were loaded and then ordered by priority. The resolvers are tried one after one until one of them succeeds. In order to decrease the number of assemblies to be load we change the behavior in 17.3 under ChangeWave 17.4.

### New SDK Resolution Algorithm
## New SDK Resolution Algorithm

Under ChangeWave 17.4 all the resolvers divides into two groups:

- Specific resolvers, i.e. resolvers with specified sdk name pattern `ResolvableSdkPattern`
- General resolvers, i.e. resolvers without specified sdk name pattern `ResolvableSdkPattern`

The resolving algorithm works in two passes.
- On the first pass all the specific resolvers that match the given sdk name would be loaded, ordered by priority and tried one after one.
The resolving algorithm works in two passes.

- On the first pass all the specific resolvers that match the given sdk name would be loaded, ordered by priority and tried one after one.
- If the sdk is not found, on the second pass all general resolvers would be loaded, ordered by priority and tried one after one.

By default the resolvers are general. To make all the resolvers from some dll specific, in the corresponding manifest (xml file) one need to specify the `ResolvableSdkPattern` using C# regex format:
```

```xml
<SdkResolver>
<Path>MySdkResolver.dll</Path>
<ResolvableSdkPattern>MySdk.*</ResolvableSdkPattern>
Expand All @@ -23,27 +29,28 @@ By default the resolvers are general. To make all the resolvers from some dll sp

Note, that the manifest file, if exists, from ChangeWave 17.4 would have preference over the dll.
The sdk discovery works according to the following algorithm:
- First try locate the manifest file and use it.
- If it is not found, we try to locate the dll in the resolver's folder.

- First try locate the manifest file and use it.
- If it is not found, we try to locate the dll in the resolver's folder.
Both xml and dll name should match the following name pattern `...\SdkResolvers\(ResolverName)\(ResolverName).(xml/dll)`.

### Failed SDK Resolution
## Failed SDK Resolution

> 🚧 Note
>
> This page is a work in progress.

SDK resolvers previously attempted to continue when one critically fails (throws an unhandled exception). This lead to misleading error messages such as:

```
```text
warning MSB4242: The SDK resolver "Microsoft.DotNet.MSBuildWorkloadSdkResolver" failed to run. 's' is an invalid start of a property name. Expected a '"'. LineNumber: 14 | BytePositionInLine: 8.
error MSB4236: The SDK 'Microsoft.NET.SDK.WorkloadAutoImportPropsLocator' specified could not be found. [C:\foo\bar.csproj]
```

`MSB4236` is a red herring while `MSB4242` is the real error despite being logged as a warning. Because of this, SDK resolvers now fail the build _immediately_ upon unhandled exceptions. These exceptions are propogated as `SdkResolverException`s, and `MSB4242` has been promoted to an error code. The new error message appears like so:

```
C:\src\temp\8-18>"C:\foo\dotnet-sdk-6.0.100-preview.7.21379.14-win-x64\dotnet.exe" build
```text
C:\src\temp\8-18>"C:\foo\dotnet-sdk-6.0.100-preview.7.21379.14-win-x64\dotnet.exe" build
Microsoft (R) Build Engine version 17.0.0-dev-21420-01+5df152759 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

Expand All @@ -56,4 +63,4 @@ C:\foo\bar.csproj : error MSB4242: SDK Resolver Failure: "The SDK resolver "Micr
1 Error(s)

Time Elapsed 00:00:00.15
```
```
Loading