/api/-/query response size #454

amvanbaren · 2022-05-18T18:40:24Z

Fixes #379

Testing steps

Open this PR in Gitpod.
Let Gitpod initialize. This can take 5 - 10 minutes, because it will download 485 vscode.bat extensions and then publish them to the local instance.
Get https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/v4/-/query?extensionId=vscode.bat&includeAllVersions=false. The response should be ~80 KB. The first object in the extensions array is the latest version. All other objects are version links.
Get https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/v4/-/query?extensionId=vscode.bat&includeAllVersions=true. The response should be ~800 KB. The extensions array contains all versions. Only the last object is a version link for the latest version.
Get https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/v3/-/query?extensionId=vscode.bat&includeAllVersions=true. The response should be ~800 KB.
Get https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/v2/-/query?extensionId=vscode.bat&includeAllVersions=true. The response should be ~650 KB.
For comparison, get https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/-/query?extensionId=vscode.bat&includeAllVersions=true. The response should be ~30 MB.

ccorn · 2022-05-19T20:19:13Z

I can confirm that this works in Gitpod as described.

Response sizes:

32487536 verquery.json
  665435 verquery.v2.json

That is a reduction to about 2% of the original response size. Nice work!

To compare the output files in the first detailed version entry, I ran the following command (may need recent bash or zsh):

diff -u <(jq -S '.extensions[0]' verquery.json) \
        <(jq -S '.extensions[0].versions[0]' verquery.v2.json)

Accordingly, v2 removes the following entries from the detailed .versions[] entries:

allVersions
downloadCount
name
namespace
namespaceAccess
namespaceUrl
preview
reviewCount
reviewsUrl
unrelatedPublisher

Congratulations for getting rid of those n**2 allVersions entries.

According to

jq -S '.extensions[0]|del(.versions,.versionLinks)' verquery.v2.json

the following entries have been added to the level above .versions[]:

downloadCount
name
namespace
namespaceUrl
preview
reviewCount
reviewsUrl

Remarks and questions

The hierarchy-level-changing move of the mentioned tags makes sense if and only if the mentioned entries are not version-specific. (Are they?)
For example, this model implies that an extension cannot be renamed or moved to another namespace in a new version. (Can it?)
What about the following removed entries?
```
namespaceAccess
unrelatedPublisher
```
Why keep the toplevel .extensions[]?
By now, it seems to be bound to have no less and no more than one entry.
It would be nice to have a version of this PR that only deletes the allVersions entries and preserves the existing response structure in all other respects.
- This way, clients can be adapted easily by just tweaking the query URL.
- There would be no need to check if perhaps the change breaks the existing data model.
- The little redundancy left would be eliminated by compression.
- The main goal, making the queries scale well with version count, would still be achieved.

amvanbaren · 2022-05-20T10:38:34Z

The hierarchy-level-changing move of the mentioned tags makes sense if and only if the mentioned entries are not version-specific. (Are they?)

Yes, the changes reflect the database model. The data hierarchy is: Namespace > Extension > ExtensionVersion.

downloadCount: total extension downloads.
name: extension name.
namespace: namespace name.
namespaceUrl: url pointing to the namespace.
preview: whether the extension is in preview mode.
reviewCount: total extension reviews.
reviewsUrl: url pointing to extension reviews.

For example, this model implies that an extension cannot be renamed or moved to another namespace in a new version. (Can it?)

Currently it can't. However, issue #427 wants to add this ability.

What about the following removed entries?
namespaceAccess
unrelatedPublisher

These entries have been deprecated for quite a while now. I made the bold move to exclude them completely.

Why keep the toplevel .extensions[]?
By now, it seems to be bound to have no less and no more than one entry.

It seems that way, because this query is by extensionId (format: {namespaceName}.{extensionName}), which limits the extensions[] to 1. It's also possible to query by namespaceName or namespaceUuid to get all extensions in a namespace, or to query by extensionName to get all extensions matching extensionName across all namespaces.

It would be nice to have a version of this PR that only deletes the allVersions entries and preserves the existing response structure in all other respects.

I'll add a /api/v3/-/query endpoint that only deletes the allVersions entries.

amvanbaren · 2022-05-20T12:40:13Z

I've added the /api/v3/-/query endpoint. The response size is ~800 KB, so the allVersions entry contributes the most to the response size.

ccorn · 2022-05-20T17:52:53Z

preview: whether the extension is in preview mode.

But that one is version-specific. E.g.

Query URL	preview
https://open-vsx.org/api/vscode/bat/1.62.3	false
https://open-vsx.org/api/vscode/bat/1.64.0-next.d9fa2b12136	true

ccorn · 2022-05-20T18:13:04Z

For example, this model implies that an extension cannot be renamed or moved to another namespace in a new version. (Can it?)

Currently it can't. However, issue #427 wants to add this ability.

Maybe it is better to err on the side of caution then.

ccorn · 2022-05-20T18:14:45Z

I've added the /api/v3/-/query endpoint. The response size is ~800 KB, so the allVersions entry contributes the most to the response size.

Thanks! I will try it. Awesome progress!

ccorn · 2022-05-20T18:25:39Z

Maybe it is better to err on the side of caution then.

In fact, the code mentions namespaceUuid and extensionUuid, so I suppose that the actual immutable identifiers are UUIDs, not names. Therefore, at least in principle, names could be variable.

ccorn · 2022-05-21T02:14:28Z

I have tried this PR at 5189e62 now.

As before, I downloaded the responses and, out of curiosity, piped them through gzip -c | wc -c and zstd -c | wc -c to see how compression affects the transfer size.

Response                size        gzip        zstd
verquery.json       32487533     3183772       34584
verquery.v2.json      665432       26802       20876
verquery.v3.json      825209       28031       21052

With v2 and v3, the responses are short enough that gzip is much more effective.
zstd is the winner as expected, but that's a sideshow.

v3 still has the structure of v2 with a sub-array versions and a map versionLinks replacing the common allVersions.
Apart from that, no tags have been moved up the hierarchy or removed, so the following comparison command shows absolutely no difference:

diff -u <(jq -S '.extensions|del(.[].allVersions)' verquery.json) \
        <(jq -S '.extensions[0].versions' verquery.v3.json)

which should reduce the effort needed for adaptation to the new API.

Conclusion

I can confirm that v3 works as advertised.

Remark

Personally, I think that the versionLinks map is unnecessary:

All the information available at the links in its values is already present in the versions array.
Users of the query API certainly would know the /api/${namespace}/${name}/${version} endpoint anyway.

So I'd remove the versionLinks (or provide a single string entry in each versions[] entry named versionLink). That leaves only the versions sub-array. Which can be flattened away as is the case now. Then the only difference to the current state of affairs would be that the uncalled-for .allVersions entries would no longer be present.

That means that current clients not interested in the version links just have to change their /api/ to /api/v4/ (or whatever version number emerges) and nothing else. As a result, server load would soon drop by orders of magnitude. Wouldn't that be great?

amvanbaren · 2022-05-23T08:01:25Z

preview: whether the extension is in preview mode.

But that one is version-specific. E.g.
Query URL preview
https://open-vsx.org/api/vscode/bat/1.62.3 false
https://open-vsx.org/api/vscode/bat/1.64.0-next.d9fa2b12136 true

It currently is, because there was a mix-up between preview and pre-release #386. The current production version still runs the old code where the preview property actually indicates whether a version is a pre-release or not.

In fact, the code mentions namespaceUuid and extensionUuid, so I suppose that the actual immutable identifiers are UUIDs, not names. Therefore, at least in principle, names could be variable.

Yes, the actual identifiers are UUIDs. Names could be variable, but an extension name must be unique within a namespace. Also a lot of the API endpoints take namespaceName and extensionName as parameters. How this change would impact the server needs to be investigated further.

ccorn · 2022-05-26T05:41:17Z

I have tried this PR at ab6a02a in Gitpod.

Indeed, v4 is precisely the original response without all the .allVersions erntries, as I have wished for.

Response sizes with ìncludeAllversions:

Response             size    gzip   zstd
verquery.json    32487536 3183746  34586
verquery.v2.json   665435   26815  20696
verquery.v3.json   825212   27741  20974
verquery.v4.json   766596   21605  16309

The v4 response is a bit longer than v2 when uncompressed, but shorter when compressed.

Besides, I notice that the original response has allVersions URLs ending with

/api/${namespace}/${name}/universal/${version}

whereas v2 and v3 use only

/${namespace}/${name}/${version}

which results in broken links, presumably because the /api prefix is missing.

Response sizes without ìncludeAllversions:

Response           size  gzip  zstd
ver1query.json    67056  6843  6836
ver1query.v2.json 60211  6751  6768
ver1query.v3.json 60268  6754  6797
ver1query.v4.json  1652   723   736

In fact, v4 has no allVersions (nor versionLinks) map even in unversioned queries without includeAllVersions. Conceptually that is OK with me, but this raises the question how to get the version links map if one is not interested in all the details associated with includeAllVersions.

It would probably be easiest if queries or API lookups

without an extension's version specified (example: /api/${namespace}/${name}, counterexample: /api/${namespace}/${name}/${version}) and
meant to yield only one version per extension (that is, without includeAllVersions)

(and only those queries) had one allVersions map included as before. That is, mimic the current behavior, except that allVersions is NOT included if includeAllVersions is true or if the query already specifies a version.

That way,

The scenarios where allVersions might actually be useful still have it available,
results of queries with specified extension version would not have allVersions and thus be long-term cacheable (unless the version is an alias like latest),
the complexity explosion from the combination of includeAllVersions and allVersions would be prevented.

ccorn · 2022-05-26T07:07:43Z

What about making the query parameter includeAllVersions ternary?

`includeAllVersions`	Meaning
`false`	no `allVersions`, metadata for one version
`true`	no `allVersions`, metadata for all versions
`links` or unspecified	`allVersions` and metadata for one version

That would be more flexible because clients can get cross-reference links in addition to metadata for one specific version, as is the case now.
Yet it would still prevent complexity explosion.

Edit: The low-budget binary version of this proposal would be to define the semantics of false as those for links, i.e. allVersions is included if and only if includeAllVersions is false.

The only drawback would be that there is no long-term caching available: The ternary approach means that a concrete version (unlike latest and such) with an explicit includeAllVersions=false could be delivered with longer-term caching headers.

amvanbaren · 2022-05-26T08:08:36Z

In fact, v4 has no allVersions (nor versionLinks) map even in unversioned queries without includeAllVersions. Conceptually that is OK with me, but this raises the question how to get the version links map if one is not interested in all the details associated with includeAllVersions.

I've added a url field to the extension object. Clients can search the extensions array to find the url (versionLink) by a combination of namespaceName, extensionName and version. Version links are minimal extension objects. Only namespace, name, version and url are specified.

Calling https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/v4/-/query?extensionId=vscode.bat&includeAllVersions=false shows the difference between a 'full' extension object (first object) and a 'minimal' extension object (all other objects).

ccorn · 2022-05-26T08:35:00Z

Version links are minimal extension objects. Only namespace, name, version and url are specified.

Interesting approach. Going to take a look. However, we would need the engines sub-objects for the task of finding a compatible version as well. Current Theia download:plugins logic also uses the files.download object of the selected version, but your idea is probably that that information can be retrieved separately.

We see here that this approach makes assumptions about what meta-information the client must scan before it can select one version. My guess is that these assumptions are not going to be satisfactory for long.

amvanbaren · 2022-05-26T10:40:55Z

The download:plugins logic calls:
140 const extension = await client.getLatestCompatibleExtensionVersion(id);
and getLatestCompatibleExtensionVersion calls:
119 const extensions = await this.getAllVersions(id);
this.getAllVersions(id); calls the /api/-/query?extensionId=<EXT_ID>&includeAllVersions=true
The API call returns 'full' extension objects for all versions. The download:plugins logic doesn't use the allVersions map.

includeAllVersions does not refer to the allVersions map. Instead it is to specify what to include in the response when extensionVersion is undefined. To give you an overview:

if extensionVersion is defined, then ignore the includeAllVersions parameter.
if includeAllVersions=true and extensionVersion is undefined, then for each extension include all versions in the response.
if includeAllVersions=false or undefined and extensionVersion is undefined, then for each extension only include the latest version in the response.

ccorn · 2022-05-26T10:49:24Z

Oh, I see: ìncludeAllVersions=false in v4 now returns several entries in the extensions array, with all but the first entry minimalized.

ìncludeAllVersions=true returns all entries as before, augmented with url entries.

Neat! Some bugs I noticed:

The new v4 url field seems to be missing an /api (or api/v4).

v4 adds a spurious entry stub:

$ diff -u <(jq -S 'del(.extensions[].allVersions)' verquery.json) \
          <(jq -S 'del(.extensions[].url)' verquery.v4.json)
--- /dev/fd/63  2022-05-26 12:10:14.145286993 +0200
+++ /dev/fd/62  2022-05-26 12:10:14.145286993 +0200
@@ -24764,6 +24764,11 @@
       "verified": false,
       "version": "1.44.2",
       "versionAlias": []
+    },
+    {
+      "name": "bat",
+      "namespace": "vscode",
+      "version": "latest"
     }
   ]
 }

Response sizes with and without includeAllVersions:

Response             size    gzip   zstd
verquery.json    32487536 3183855  34527
verquery.v2.json   667379   26817  20970
verquery.v3.json   827156   27971  21145
verquery.v4.json   816598   23999  18589
ver1query.json      67056    6843   6837
ver1query.v2.json   62155    6716   6748
ver1query.v3.json   62212    6716   6779
ver1query.v4.json   85436    6366   5750

Somehow I like this concept, although I probably should not. It is better structured in that it does not pretend that allVersions were a property of any specific version, yet it is compatible with existing usage (apart from allVersions of course), as long as only one extension ID is queried.

However, It seems to be more cumbersome for queries whose response can cover more than one extension ID. For those, a ternary includeAllVersions, as described earlier, might be useful. Thus, an explicit false would reduce the extensions array to one entry per extension ID again.

ccorn · 2022-05-26T11:11:51Z

v4 adds a spurious entry stub

No, that is intended I guess, (It also has a url, but my diff command removes that for purposes of comparing). So it's just another minimized entry.

Edit: No, wait. This happens in the response for includeAllVersions=true. That one should not have minimized entries at all. So yes, that's a bug.

amvanbaren · 2022-05-26T11:34:42Z

Maybe the 'minimized' entries add to the confusion. I mean when includeAllVersions=false the response still contains multiple extension entries: one 'full' object and 'minimized' objects for all other versions.

Instead only setting the allVersions map for the first version for each extension (${namespace}.${name}) solves the issue and is closer to the original implementation. It would require minimal changes for clients to adopt.
If extensionVersion is defined the allVersions map is set on the matching version. In other cases the allVersions map is only set on the latest version.

ccorn · 2022-05-26T11:44:29Z

I suppose that allVersionsshould never be needed when includeAllVersions=true. (The url entries are better suited for that case.) So we do not need to make incongruent element layouts for those responses. The only question where allVersions might be wanted is for queries without includeAllVersions. The ternary approach would allow clients to be explicit about their expectations.

amvanbaren · 2022-05-26T13:52:33Z

I suppose that allVersionsshould never be needed when includeAllVersions=true. (The url entries are better suited for that case.) So we do not need to make incongruent element layouts for those responses. The only question where allVersions might be wanted is for queries without includeAllVersions. The ternary approach would allow clients to be explicit about their expectations.

I've added extensionVersion to the table to see how it interacts with includeAllVersions

includeAllVersions	extensionVersion	Meaning
false	specified	no allVersions, metadata for specified version
false	unspecified	no allVersions, metadata for latest version
true	specified	ignore includeAllVersions, i.e. includeAllVersions is unspecified
true	unspecified	no allVersions, metadata for all versions
links or unspecified	specified	allVersions and metadata for specified version
links or unspecified	unspecified	allVersions and metadata for latest version

Edit: I've added the /api/v5/-/query endpoint:
https://8080-<GITPOD_WORKSPACE_ID>.gitpod.io/api/v5/-/query?extensionId=vscode.bat&includeAllVersions=false&extensionVersion=1.46.1
By changing or omitting the includeAllVersions and extensionVersion parameters you can test the responses.

ccorn · 2022-05-26T17:37:45Z

Thanks! I will try v5 within the next two hours.

ccorn · 2022-05-27T10:04:53Z

This took some time because I tried lots of combinations and wrote a bash script to automate most of those tests.

Summary

I have tried 3388130. It works as advertised, reduces response size (in some cases drastically), is both reasonably backwards-compatible and (with the ternary includeAllVersions) more flexible than the original API.

I look forward to see the v5 API in production.

More detailed v5 notes

allVersions is not deleted, but emptied. I suppose that this is a good idea, as it might prevent unnecessary failures in structure-checking clients.

To give you an idea of what I have checked, the v5-related output of my test script is given below. Response filenames contain the values of extensionId, extensionVersion, and includeAllVersions, if specified, separated by underscores.

Response                                      size    gzip   zstd
verquery.vscode.bat_true.json             32487533 3183822  34929
verquery.v5.vscode.bat_true.json            826637   23332  18566
verquery.vscode.bat.json                     67056    6830   6837
verquery.vscode.bat_false.json               67056    6830   6837
verquery.v5.vscode.bat.json                  62305    6310   5575
verquery.v5.vscode.bat_links.json            62305    6310   5575
verquery.v5.vscode.bat_false.json             1778     723    750
verquery.vscode.bat_1.53.2.json              66918    6797   6806
verquery.vscode.bat_1.53.2_true.json         66918    6797   6806
verquery.vscode.bat_1.53.2_false.json        66918    6797   6806
verquery.v5.vscode.bat_1.53.2.json           62150    6274   5548
verquery.v5.vscode.bat_1.53.2_true.json      62150    6274   5548
verquery.v5.vscode.bat_1.53.2_links.json     62150    6274   5548
verquery.v5.vscode.bat_1.53.2_false.json      1623     681    711

Check verquery.v5.vscode.bat_1.53.2.json = verquery.v5.vscode.bat_1.53.2_links.json
Check verquery.v5.vscode.bat_1.53.2_true.json = verquery.v5.vscode.bat_1.53.2_links.json
Check verquery.v5.vscode.bat.json = verquery.v5.vscode.bat_links.json

latest = 1.64.0-next.d9fa2b12136
Check version in verquery.v5.vscode.bat_1.53.2_false.json = 1.53.2: true
Check version in verquery.v5.vscode.bat_false.json = 1.64.0-next.d9fa2b12136: true
Check version in verquery.v5.vscode.bat_1.53.2_links.json = 1.53.2: true
Check version in verquery.v5.vscode.bat_links.json = 1.64.0-next.d9fa2b12136: true
Check version in verquery.v5.vscode.bat_1.53.2.json = 1.53.2: true
Check version in verquery.v5.vscode.bat.json = 1.64.0-next.d9fa2b12136: true

vercount = 485
Check version count in verquery.v5.vscode.bat_1.53.2_false.json == 1: true
Check version count in verquery.v5.vscode.bat_false.json == 1: true
Check version count in verquery.v5.vscode.bat_1.53.2_true.json == 1: true
Check version count in verquery.v5.vscode.bat_true.json == 485: true
Check version count in verquery.v5.vscode.bat_1.53.2_links.json == 1: true
Check version count in verquery.v5.vscode.bat_links.json == 1: true
Check version count in verquery.v5.vscode.bat_1.53.2.json == 1: true
Check version count in verquery.v5.vscode.bat.json == 1: true

Check url presence in verquery.v5.vscode.bat_1.53.2_false.json: true
Check url presence in verquery.v5.vscode.bat_false.json: true
Check url presence in verquery.v5.vscode.bat_1.53.2_true.json: true
Check url presence in verquery.v5.vscode.bat_true.json: true
Check url presence in verquery.v5.vscode.bat_1.53.2_links.json: true
Check url presence in verquery.v5.vscode.bat_links.json: true
Check url presence in verquery.v5.vscode.bat_1.53.2.json: true
Check url presence in verquery.v5.vscode.bat.json: true

Check allVersions empty in verquery.v5.vscode.bat_1.53.2_false.json: true
Check allVersions empty in verquery.v5.vscode.bat_false.json: true
Check allVersions nonempty in verquery.v5.vscode.bat_1.53.2_true.json: true
Check allVersions empty in verquery.v5.vscode.bat_true.json: true
Check allVersions nonempty in verquery.v5.vscode.bat_1.53.2_links.json: true
Check allVersions nonempty in verquery.v5.vscode.bat_links.json: true
Check allVersions nonempty in verquery.v5.vscode.bat_1.53.2.json: true
Check allVersions nonempty in verquery.v5.vscode.bat.json: true

Compare other metadata in verquery.v5.vscode.bat_1.53.2_false.json with original
Compare other metadata in verquery.v5.vscode.bat_false.json with original
Compare other metadata in verquery.v5.vscode.bat_1.53.2_true.json with original
Compare other metadata in verquery.v5.vscode.bat_true.json with original
Compare other metadata in verquery.v5.vscode.bat_1.53.2_links.json with original
Compare other metadata in verquery.v5.vscode.bat_links.json with original
Compare other metadata in verquery.v5.vscode.bat_1.53.2.json with original
Compare other metadata in verquery.v5.vscode.bat.json with original

The script found no v5-related errors (but one v4-related error mentioned earlier: one spurious added minimized entry in verquery.v4.vscode.bat_true.json).

In particular, the semantics related to .allVersions is precisely as detailed in the six-case table combining extensionVersion state with the ternary includeAllVersions.

Unrelated notes

For includeAllVersions=trueand unspecified extensionVersion, all API versions so far, including the original, return the latest version not at position [0]. In my tests, those occur in position [2].

While not relevant for this PR, this means that the logic in theia download:plugins for determining the latest compatible version might get fooled: There the idea is to pick the first listed version that has a compatible engines entry.

Therefore I would like to know what the actual sorting order is.

amvanbaren · 2022-05-30T09:05:14Z

For includeAllVersions=trueand unspecified extensionVersion, all API versions so far, including the original, return the latest version not at position [0]. In my tests, those occur in position [2].
Therefore I would like to know what the actual sorting order is.

The SemanticVersion class is used to parse and compare versions.
The parsing logic is in the constructor:

openvsx/server/src/main/java/org/eclipse/openvsx/util/SemanticVersion.java

Lines 24 to 42 in 72706d1

    
           public SemanticVersion(String s) { 
        
               this.original = s; 
        
               int suffixIndex = s.length(); 
        
               for (int i = 0; i < s.length(); i++) { 
        
                   char c = s.charAt(i); 
        
                   if (!Character.isDigit(c) && c != '.') { 
        
                       suffixIndex = i; 
        
                       break; 
        
                   } 
        
               } 
        
               var main = s.substring(0, suffixIndex); 
        
               var split = main.split("\\."); 
        
               for (int i = 0; i < split.length; i++) { 
        
                   parts.add(split[i]); 
        
               } 
        
               if (suffixIndex < s.length()) { 
        
                   parts.add(s.substring(suffixIndex)); 
        
               } 
        
           }

And here's the comparison logic:

openvsx/server/src/main/java/org/eclipse/openvsx/util/SemanticVersion.java

Lines 86 to 106 in 72706d1

    
           @Override 
        
           public int compareTo(SemanticVersion other) { 
        
                  int minSize = Math.min(this.parts.size(), other.parts.size()); 
        
                  for (int i = 0; i < minSize; i++) { 
        
                      String left = this.parts.get(i); 
        
                      String right = other.parts.get(i); 
        
                      boolean leftIsNumber = isNumber(left); 
        
                      boolean rightIsNumber = isNumber(right); 
        
                      if (!leftIsNumber && !rightIsNumber) 
        
                          // Regard versions as equal in terms of sorting if they differ only in their suffix 
        
                          return 0; 
        
                      int compare; 
        
                      if (leftIsNumber && rightIsNumber) 
        
                          compare = Integer.compare(Integer.parseInt(left), Integer.parseInt(right)); 
        
                      else 
        
                          compare = left.compareTo(right); 
        
                      if (compare != 0) 
        
                          return compare; 
        
                  } 
        
           	return -Integer.compare(this.parts.size(), other.parts.size()); 
        
           }

The unit test describes how SemanticVersion is supposed to work:

openvsx/server/src/test/java/org/eclipse/openvsx/util/SemanticVersionTest.java

Lines 18 to 30 in 72706d1

    
           @Test 
        
           public void testCompare() { 
        
               assertThat(new SemanticVersion("2.0.0").compareTo(new SemanticVersion("1.2.3"))) 
        
                       .isEqualTo(1); 
        
               assertThat(new SemanticVersion("1.2.3").compareTo(new SemanticVersion("1.2.4"))) 
        
                       .isEqualTo(-1); 
        
               assertThat(new SemanticVersion("1.2.3-next.bc11e2c5").compareTo(new SemanticVersion("1.2.3-next.6aa3b0d6"))) 
        
                       .isEqualTo(0); 
        
               assertThat(new SemanticVersion("1.2.3").compareTo(new SemanticVersion("1.2.3-next.bc11e2c5"))) 
        
                       .isEqualTo(1); 
        
               assertThat(new SemanticVersion("10.0").compareTo(new SemanticVersion("9.0"))) 
        
                       .isEqualTo(1); 
        
           }

You can see on line 24 that versions 1.2.3-next.bc11e2c5 and 1.2.3-next.6aa3b0d6 must be equal according to the SemanticVersion logic. This is incorrect, build hashes should also be compared to each other. I've removed the SemanticVersion class and added the semver4j library. It's used in the server code to parse and compare semvers based on the NPM convention.

allVersions is not deleted, but emptied. I suppose that this is a good idea, as it might prevent unnecessary failures in structure-checking clients.

That was a bug. allVersions is no longer added to the response if it's empty.

@ccorn Thanks for taking the time to test and review the changes. It really helps a lot.

ccorn · 2022-05-31T22:46:42Z

Thanks! I have updated the test script according to your bug fixes, so now the version of the first result of includeAllVersions=true responses is checked as well, and the size check of allVersions has been updated to include a presence/absence check.

Except for that one v4 version count error, all my tests pass.
I have attached the test log output test-eclipse-openvsx-pull-454-c015d32-out.txt.

ccorn · 2022-06-01T00:25:48Z

While the test results agree with your specifications, current open-vsx.org behaves differently:

Browsing https://open-vsx.org/api/-/query?extensionId=vscode.bat, I get in .extension[0]:

version: "1.62.3"
versionAlias: ["latest"]

But if I replace open-vsx.org with the gitpod instance for this PR, I get a -next version (all API versions, including the one without /v). Should not "latest" point to a non-next version?

With includeAllVersions=true, current open-vsx.org returns the greatest -next version first, then, at an appropriately farther position, the above stable version tagged with "latest" makes its appearance. With this PR however, "latest" always points to the first entry, usually a -next version.

Should that be so?

ccorn · 2022-06-01T00:48:36Z

For the version sorting, I suppose that the least-surprising approach is (still) a lexicographic one, but with the version suffix (an unordered hash value) replaced with the publishing timestamp (which actually indicates an order). That is, begin with what SemanticVersion does and in case of equality (or EquivalentTo in semver4j) compare the timestamps.

It would be easiest if most of the original server code remained in effect, with just the includeAllVersions handling refined according to the case table and .allVersions included only if includeAllVersions is (effectively) links.

amvanbaren · 2022-06-01T14:48:21Z

Should not "latest" point to a non-next version?

latest is now the absolute latest version, pre-release or not.
The change was made here: #419 as a response to #410 (comment).
These changes are not yet running in production.

For the version sorting, I suppose that the least-surprising approach is (still) a lexicographic one, but with the version suffix (an unordered hash value) replaced with the publishing timestamp (which actually indicates an order). That is, begin with what SemanticVersion does and in case of equality (or EquivalentTo in semver4j) compare the timestamps.

Ok, that's how the previous sort worked. I'll revert the changes.

openvsx/server/src/main/java/org/eclipse/openvsx/LocalRegistryService.java

Lines 389 to 397 in 72706d1

    
           private Comparator<ExtensionVersionDTO> getExtensionVersionComparator() { 
        
               // comparators combine comparator ExtensionVersion.SORT_COMPARATOR and order by of RepositoryService.findActiveExtensions 
        
               var versionComparator = Comparator.<ExtensionVersionDTO, SemanticVersion>comparing(ExtensionVersionDTO::getSemanticVersion) 
        
                       .thenComparing(ExtensionVersionDTO::getTimestamp) 
        
                       .reversed(); 
        
               return Comparator.<ExtensionVersionDTO, String>comparing(ev -> ev.getExtension().getName()) 
        
                       .thenComparing(versionComparator); 
        
           }

ccorn · 2022-06-01T15:12:42Z

Ok, that's how the previous sort worked. I'll revert the changes.

I see. But how can the "latest" entry show up at [2] then (in 3388130)?
If that is just a side effect of, say, parallelism in filling the gitpod instance, that could be ignored (though I'd still like to see the original timestamps in effect). But double-checking would be good because correct sorting and correct labelling of versions seems crucially important to me.

ccorn · 2022-06-07T13:53:46Z

At 4942915, I get a 500 Internal Server Error.

java.lang.IllegalArgumentException: Argument is not an array
	at java.base/java.lang.reflect.Array.getLength(Native Method)
	at org.hibernate.internal.util.collections.ArrayHelper.toList(ArrayHelper.java:122)
	at org.eclipse.openvsx.repositories.ExtensionVersionJooqRepository.toExtensionVersion(ExtensionVersionJooqRepository.java:179)
	at org.jooq.impl.ResultImpl.map(ResultImpl.java:1318)
	at org.eclipse.openvsx.repositories.ExtensionVersionJooqRepository.fetch(ExtensionVersionJooqRepository.java:162)
	at org.eclipse.openvsx.repositories.ExtensionVersionJooqRepository.findAllActiveByExtensionNameAndNamespaceName(ExtensionVersionJooqRepository.java:91)
	at org.eclipse.openvsx.repositories.RepositoryService.findActiveExtensionVersionsByExtensionName(RepositoryService.java:276)
	at org.eclipse.openvsx.LocalRegistryService.queryV5(LocalRegistryService.java:682)
	at org.eclipse.openvsx.LocalRegistryService$$FastClassBySpringCGLIB$$c162f6fb.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687)
	at org.eclipse.openvsx.LocalRegistryService$$EnhancerBySpringCGLIB$$8d073861.queryV5(<generated>)
	at org.eclipse.openvsx.RegistryAPI.getQueryV5(RegistryAPI.java:514)
	at jdk.internal.reflect.GeneratedMethodAccessor160.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:893)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:807)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1061)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:961)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:626)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:106)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:218)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:178)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:115)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:141)
	at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)

amvanbaren · 2022-06-07T19:31:15Z

I see. But how can the "latest" entry show up at [2] then (in 3388130)?
If that is just a side effect of, say, parallelism in filling the gitpod instance, that could be ignored (though I'd still like to see the original timestamps in effect). But double-checking would be good because correct sorting and correct labelling of versions seems crucially important to me.

Different sort comparators were used in VersionUtil, ExtensionVersion and LocalRegistryService. I've refactored the code, so only ExtensionVersion.SORT_COMPARATOR is used to sort extension versions. Now [0] is the latest.

ccorn · 2022-06-07T20:07:33Z

Different sort comparators were used in VersionUtil, ExtensionVersion and LocalRegistryService. I've refactored the code, so only ExtensionVersion.SORT_COMPARATOR is used to sort extension versions. Now [0] is the latest.

Indeed it works. According to the test script output applied to 6c698ec, all v5 tests pass, and that includes checking [0] == latest. (In fact, all my tests pass, even those unrelated to v5, except the one v4 list output size test, as noted before.) Cursory manual inspection confirms that the version sorting goes indeed by version (without hash part), then by timestamp.

This v5 looks like a suitable and desirable change.

Added /api/v2/-/query endpoint Deleted DTOs Use the same ExtensionVersion Comparator everywhere Added unit tests Add performance test for `/api/v2/-/query` endpoint.

amvanbaren · 2022-12-14T20:19:15Z

Performance on staging looks good. LGTM!

RegistryAPI

RegistryAPIGetNamespaceSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4606   KO=394   )
> min response time                                    140 (OK=140    KO=143   )
> max response time                                   1137 (OK=1137   KO=466   )
> mean response time                                   155 (OK=155    KO=157   )
> std deviation                                         45 (OK=46     KO=24    )
> response time 50th percentile                        148 (OK=148    KO=152   )
> response time 75th percentile                        154 (OK=154    KO=161   )
> response time 95th percentile                        171 (OK=170    KO=178   )
> response time 99th percentile                        398 (OK=447    KO=201   )
> mean requests/sec                                 31.847 (OK=29.338 KO=2.51  )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4601 ( 92%)
> 800 ms < t < 1200 ms                                   5 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               394 (  8%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 429                       394 (100,0%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4626   KO=374   )
> min response time                                    140 (OK=140    KO=145   )
> max response time                                   1148 (OK=1148   KO=474   )
> mean response time                                   158 (OK=158    KO=161   )
> std deviation                                         48 (OK=49     KO=29    )
> response time 50th percentile                        151 (OK=151    KO=157   )
> response time 75th percentile                        158 (OK=158    KO=163   )
> response time 95th percentile                        175 (OK=175    KO=177   )
> response time 99th percentile                        453 (OK=454    KO=214   )
> mean requests/sec                                  31.25 (OK=28.913 KO=2.337 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4621 ( 92%)
> 800 ms < t < 1200 ms                                   5 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               374 (  7%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 429                       374 (100,0%)
================================================================================

RegistryAPIGetExtensionSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=2555   KO=2445  )
> min response time                                    142 (OK=155    KO=142   )
> max response time                                   1442 (OK=1442   KO=490   )
> mean response time                                   170 (OK=183    KO=156   )
> std deviation                                         55 (OK=70     KO=25    )
> response time 50th percentile                        163 (OK=171    KO=152   )
> response time 75th percentile                        172 (OK=180    KO=158   )
> response time 95th percentile                        198 (OK=215    KO=171   )
> response time 99th percentile                        403 (OK=474    KO=186   )
> mean requests/sec                                  29.07 (OK=14.855 KO=14.215)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2550 ( 51%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            5 (  0%)
> failed                                              2445 ( 49%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   2439 (99,75%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f      6 ( 0,25%)
ound 404
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=2741   KO=2259  )
> min response time                                    144 (OK=158    KO=144   )
> max response time                                   3330 (OK=3330   KO=647   )
> mean response time                                   186 (OK=207    KO=160   )
> std deviation                                        173 (OK=231    KO=26    )
> response time 50th percentile                        170 (OK=178    KO=156   )
> response time 75th percentile                        180 (OK=187    KO=163   )
> response time 95th percentile                        204 (OK=215    KO=177   )
> response time 99th percentile                        493 (OK=652    KO=198   )
> mean requests/sec                                 26.455 (OK=14.503 KO=11.952)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2718 ( 54%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                           23 (  0%)
> failed                                              2259 ( 45%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   2252 (99,69%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f      7 ( 0,31%)
ound 404
================================================================================

RegistryAPIGetExtensionTargetPlatformSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4829   KO=171   )
> min response time                                    141 (OK=141    KO=143   )
> max response time                                   2177 (OK=813    KO=2177  )
> mean response time                                   157 (OK=156    KO=167   )
> std deviation                                         48 (OK=40     KO=154   )
> response time 50th percentile                        150 (OK=150    KO=153   )
> response time 75th percentile                        154 (OK=154    KO=160   )
> response time 95th percentile                        178 (OK=179    KO=170   )
> response time 99th percentile                        442 (OK=443    KO=182   )
> mean requests/sec                                 31.447 (OK=30.371 KO=1.075 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4824 ( 96%)
> 800 ms < t < 1200 ms                                   5 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               171 (  3%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    157 (91,81%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     14 ( 8,19%)
ound 404
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4833   KO=167   )
> min response time                                    141 (OK=141    KO=145   )
> max response time                                    894 (OK=894    KO=193   )
> mean response time                                   159 (OK=159    KO=157   )
> std deviation                                         42 (OK=42     KO=9     )
> response time 50th percentile                        152 (OK=152    KO=155   )
> response time 75th percentile                        158 (OK=158    KO=162   )
> response time 95th percentile                        183 (OK=184    KO=174   )
> response time 99th percentile                        445 (OK=449    KO=184   )
> mean requests/sec                                 31.056 (OK=30.019 KO=1.037 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4828 ( 97%)
> 800 ms < t < 1200 ms                                   5 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               167 (  3%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    153 (91,62%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     14 ( 8,38%)
ound 404
================================================================================

RegistryAPIGetExtensionVersionSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=2459   KO=2541  )
> min response time                                    143 (OK=144    KO=143   )
> max response time                                   1817 (OK=1817   KO=602   )
> mean response time                                   164 (OK=172    KO=157   )
> std deviation                                         61 (OK=81     KO=30    )
> response time 50th percentile                        154 (OK=158    KO=152   )
> response time 75th percentile                        164 (OK=170    KO=158   )
> response time 95th percentile                        183 (OK=196    KO=172   )
> response time 99th percentile                        454 (OK=480    KO=190   )
> mean requests/sec                                  29.94 (OK=14.725 KO=15.216)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2450 ( 49%)
> 800 ms < t < 1200 ms                                   3 (  0%)
> t > 1200 ms                                            6 (  0%)
> failed                                              2541 ( 51%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   2521 (99,21%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     20 ( 0,79%)
ound 404
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=2522   KO=2478  )
> min response time                                    143 (OK=144    KO=143   )
> max response time                                   3179 (OK=3179   KO=636   )
> mean response time                                   169 (OK=179    KO=159   )
> std deviation                                        119 (OK=164    KO=30    )
> response time 50th percentile                        157 (OK=160    KO=155   )
> response time 75th percentile                        167 (OK=175    KO=162   )
> response time 95th percentile                        189 (OK=197    KO=176   )
> response time 99th percentile                        463 (OK=486    KO=202   )
> mean requests/sec                                 27.933 (OK=14.089 KO=13.844)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2508 ( 50%)
> 800 ms < t < 1200 ms                                   5 (  0%)
> t > 1200 ms                                            9 (  0%)
> failed                                              2478 ( 50%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   2459 (99,23%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     19 ( 0,77%)
ound 404
================================================================================

RegistryAPIGetExtensionVersionTargetPlatformSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4664   KO=336   )
> min response time                                    141 (OK=141    KO=144   )
> max response time                                    752 (OK=752    KO=189   )
> mean response time                                   158 (OK=159    KO=154   )
> std deviation                                         42 (OK=43     KO=8     )
> response time 50th percentile                        150 (OK=150    KO=152   )
> response time 75th percentile                        157 (OK=157    KO=158   )
> response time 95th percentile                        181 (OK=182    KO=168   )
> response time 99th percentile                        448 (OK=450    KO=181   )
> mean requests/sec                                  31.25 (OK=29.15  KO=2.1   )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4664 ( 93%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               336 (  7%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    301 (89,58%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     35 (10,42%)
ound 404
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4669   KO=331   )
> min response time                                    141 (OK=141    KO=145   )
> max response time                                    790 (OK=790    KO=468   )
> mean response time                                   159 (OK=158    KO=162   )
> std deviation                                         40 (OK=40     KO=31    )
> response time 50th percentile                        150 (OK=150    KO=157   )
> response time 75th percentile                        158 (OK=157    KO=164   )
> response time 95th percentile                        189 (OK=189    KO=181   )
> response time 99th percentile                        446 (OK=446    KO=215   )
> mean requests/sec                                 31.056 (OK=29     KO=2.056 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4669 ( 93%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               331 (  7%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    294 (88,82%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     37 (11,18%)
ound 404
================================================================================

RegistryAPIGetFileSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       8757 (OK=3757   KO=5000  )
> min response time                                     78 (OK=141    KO=78    )
> max response time                                   3256 (OK=3256   KO=2162  )
> mean response time                                   134 (OK=176    KO=102   )
> std deviation                                        108 (OK=144    KO=49    )
> response time 50th percentile                        147 (OK=163    KO=83    )
> response time 75th percentile                        162 (OK=172    KO=108   )
> response time 95th percentile                        182 (OK=196    KO=161   )
> response time 99th percentile                        293 (OK=467    KO=177   )
> mean requests/sec                                 36.794 (OK=15.786 KO=21.008)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          3743 ( 43%)
> 800 ms < t < 1200 ms                                   4 (  0%)
> t > 1200 ms                                           10 (  0%)
> failed                                              5000 ( 57%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   3757 (75,14%)
ound 403
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   1038 (20,76%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    205 ( 4,10%)
ound 404
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       8734 (OK=3734   KO=5000  )
> min response time                                     79 (OK=141    KO=79    )
> max response time                                   3196 (OK=3196   KO=3181  )
> mean response time                                   133 (OK=173    KO=103   )
> std deviation                                         81 (OK=88     KO=60    )
> response time 50th percentile                        148 (OK=166    KO=83    )
> response time 75th percentile                        165 (OK=176    KO=145   )
> response time 95th percentile                        185 (OK=195    KO=163   )
> response time 99th percentile                        225 (OK=462    KO=182   )
> mean requests/sec                                 37.166 (OK=15.889 KO=21.277)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          3727 ( 43%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            7 (  0%)
> failed                                              5000 ( 57%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   3734 (74,68%)
ound 403
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   1069 (21,38%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    197 ( 3,94%)
ound 404
================================================================================

RegistryAPIGetFileTargetPlatformSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       7938 (OK=2938   KO=5000  )
> min response time                                     78 (OK=141    KO=78    )
> max response time                                  15325 (OK=750    KO=15325 )
> mean response time                                   137 (OK=169    KO=119   )
> std deviation                                        179 (OK=43     KO=221   )
> response time 50th percentile                        149 (OK=162    KO=84    )
> response time 75th percentile                        162 (OK=170    KO=153   )
> response time 95th percentile                        183 (OK=198    KO=170   )
> response time 99th percentile                        304 (OK=421    KO=280   )
> mean requests/sec                                 34.364 (OK=12.719 KO=21.645)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2938 ( 37%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              5000 ( 63%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   2938 (58,76%)
ound 403
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   1150 (23,00%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    912 (18,24%)
ound 404
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       7891 (OK=2891   KO=5000  )
> min response time                                     79 (OK=140    KO=79    )
> max response time                                   3190 (OK=3189   KO=3190  )
> mean response time                                   145 (OK=180    KO=124   )
> std deviation                                        163 (OK=184    KO=145   )
> response time 50th percentile                        150 (OK=165    KO=85    )
> response time 75th percentile                        166 (OK=174    KO=155   )
> response time 95th percentile                        186 (OK=193    KO=176   )
> response time 99th percentile                        387 (OK=476    KO=307   )
> mean requests/sec                                 33.867 (OK=12.408 KO=21.459)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2879 ( 36%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                           12 (  0%)
> failed                                              5000 ( 63%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   2891 (57,82%)
ound 403
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   1209 (24,18%)
ound 429
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    900 (18,00%)
ound 404
================================================================================

RegistryAPIGetQuerySimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=5000   KO=0     )
> min response time                                    141 (OK=141    KO=-     )
> max response time                                   3932 (OK=3932   KO=-     )
> mean response time                                   162 (OK=162    KO=-     )
> std deviation                                        122 (OK=122    KO=-     )
> response time 50th percentile                        149 (OK=149    KO=-     )
> response time 75th percentile                        154 (OK=154    KO=-     )
> response time 95th percentile                        170 (OK=170    KO=-     )
> response time 99th percentile                        456 (OK=456    KO=-     )
> mean requests/sec                                  29.94 (OK=29.94  KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4992 (100%)
> 800 ms < t < 1200 ms                                   1 (  0%)
> t > 1200 ms                                            7 (  0%)
> failed                                                 0 (  0%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=5000   KO=0     )
> min response time                                    140 (OK=140    KO=-     )
> max response time                                   2550 (OK=2550   KO=-     )
> mean response time                                   172 (OK=172    KO=-     )
> std deviation                                         98 (OK=98     KO=-     )
> response time 50th percentile                        149 (OK=149    KO=-     )
> response time 75th percentile                        154 (OK=154    KO=-     )
> response time 95th percentile                        303 (OK=303    KO=-     )
> response time 99th percentile                        591 (OK=591    KO=-     )
> mean requests/sec                                 28.902 (OK=28.902 KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4993 (100%)
> 800 ms < t < 1200 ms                                   2 (  0%)
> t > 1200 ms                                            5 (  0%)
> failed                                                 0 (  0%)
================================================================================

RegistryAPIGetQueryV2Simulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=4996   KO=4     )
> min response time                                    140 (OK=140    KO=146   )
> max response time                                   1065 (OK=1065   KO=167   )
> mean response time                                   157 (OK=157    KO=157   )
> std deviation                                         47 (OK=47     KO=9     )
> response time 50th percentile                        147 (OK=147    KO=157   )
> response time 75th percentile                        152 (OK=152    KO=165   )
> response time 95th percentile                        168 (OK=169    KO=167   )
> response time 99th percentile                        444 (OK=444    KO=167   )
> mean requests/sec                                 31.447 (OK=31.421 KO=0.025 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4995 (100%)
> 800 ms < t < 1200 ms                                   1 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                                 4 (  0%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 429                         4 (100,0%)
================================================================================


null

RegistryAPISearchSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=3068   KO=1932  )
> min response time                                    143 (OK=145    KO=143   )
> max response time                                   3286 (OK=3286   KO=614   )
> mean response time                                   292 (OK=379    KO=155   )
> std deviation                                        387 (OK=473    KO=27    )
> response time 50th percentile                        162 (OK=186    KO=151   )
> response time 75th percentile                        200 (OK=277    KO=157   )
> response time 95th percentile                       1031 (OK=1430   KO=168   )
> response time 99th percentile                       2222 (OK=2358   KO=185   )
> mean requests/sec                                 17.007 (OK=10.435 KO=6.571 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2694 ( 54%)
> 800 ms < t < 1200 ms                                 175 (  4%)
> t > 1200 ms                                          199 (  4%)
> failed                                              1932 ( 39%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   1932 (100,0%)
ound 429
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=3242   KO=1758  )
> min response time                                    143 (OK=146    KO=143   )
> max response time                                   5210 (OK=5210   KO=627   )
> mean response time                                   340 (OK=438    KO=159   )
> std deviation                                        511 (OK=612    KO=35    )
> response time 50th percentile                        167 (OK=192    KO=153   )
> response time 75th percentile                        216 (OK=303    KO=160   )
> response time 95th percentile                       1327 (OK=1850   KO=174   )
> response time 99th percentile                       2742 (OK=3064   KO=199   )
> mean requests/sec                                 14.577 (OK=9.452  KO=5.125 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          2775 ( 56%)
> 800 ms < t < 1200 ms                                 174 (  3%)
> t > 1200 ms                                          293 (  6%)
> failed                                              1758 ( 35%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   1758 (100,0%)
ound 429
================================================================================

RegistryAPIVerifyTokenSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=0      KO=5000  )
> min response time                                    144 (OK=-      KO=144   )
> max response time                                    805 (OK=-      KO=805   )
> mean response time                                   159 (OK=-      KO=159   )
> std deviation                                         38 (OK=-      KO=38    )
> response time 50th percentile                        153 (OK=-      KO=153   )
> response time 75th percentile                        159 (OK=-      KO=159   )
> response time 95th percentile                        171 (OK=-      KO=171   )
> response time 99th percentile                        364 (OK=-      KO=364   )
> mean requests/sec                                  31.25 (OK=-      KO=31.25 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                             0 (  0%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              5000 (100%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   4788 (95,76%)
ound 400
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f    212 ( 4,24%)
ound 429
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=0      KO=5000  )
> min response time                                    144 (OK=-      KO=144   )
> max response time                                    788 (OK=-      KO=788   )
> mean response time                                   164 (OK=-      KO=164   )
> std deviation                                         41 (OK=-      KO=41    )
> response time 50th percentile                        158 (OK=-      KO=158   )
> response time 75th percentile                        165 (OK=-      KO=165   )
> response time 95th percentile                        180 (OK=-      KO=180   )
> response time 99th percentile                        464 (OK=-      KO=464   )
> mean requests/sec                                  29.94 (OK=-      KO=29.94 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                             0 (  0%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              5000 (100%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f   4952 (99,04%)
ound 400
> status.find.in(200,201,202,203,204,205,206,207,208,209,304), f     48 ( 0,96%)
ound 429
================================================================================

VSCodeAPI

VSCodeAdapterExtensionQuerySimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=5000   KO=0     )
> min response time                                    181 (OK=181    KO=-     )
> max response time                                   1881 (OK=1881   KO=-     )
> mean response time                                   418 (OK=418    KO=-     )
> std deviation                                        136 (OK=136    KO=-     )
> response time 50th percentile                        347 (OK=347    KO=-     )
> response time 75th percentile                        478 (OK=478    KO=-     )
> response time 95th percentile                        628 (OK=628    KO=-     )
> response time 99th percentile                       1013 (OK=1013   KO=-     )
> mean requests/sec                                  11.71 (OK=11.71  KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4901 ( 98%)
> 800 ms < t < 1200 ms                                  73 (  1%)
> t > 1200 ms                                           26 (  1%)
> failed                                                 0 (  0%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       5000 (OK=5000   KO=0     )
> min response time                                    182 (OK=182    KO=-     )
> max response time                                   3805 (OK=3805   KO=-     )
> mean response time                                   476 (OK=476    KO=-     )
> std deviation                                        319 (OK=319    KO=-     )
> response time 50th percentile                        465 (OK=465    KO=-     )
> response time 75th percentile                        492 (OK=492    KO=-     )
> response time 95th percentile                        774 (OK=774    KO=-     )
> response time 99th percentile                       1705 (OK=1705   KO=-     )
> mean requests/sec                                 10.163 (OK=10.163 KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4783 ( 96%)
> 800 ms < t < 1200 ms                                 123 (  2%)
> t > 1200 ms                                           94 (  2%)
> failed                                                 0 (  0%)
================================================================================

VSCodeAdapterGetAssetSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       8828 (OK=3828   KO=5000  )
> min response time                                     78 (OK=141    KO=78    )
> max response time                                    816 (OK=816    KO=809   )
> mean response time                                   130 (OK=168    KO=101   )
> std deviation                                         51 (OK=41     KO=37    )
> response time 50th percentile                        150 (OK=162    KO=84    )
> response time 75th percentile                        161 (OK=169    KO=90    )
> response time 95th percentile                        177 (OK=187    KO=159   )
> response time 99th percentile                        212 (OK=460    KO=171   )
> mean requests/sec                                 37.726 (OK=16.359 KO=21.368)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          3825 ( 43%)
> 800 ms < t < 1200 ms                                   3 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              5000 ( 57%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 403                      3828 (76,56%)
> status.find.is(200), but actually found 429                       762 (15,24%)
> status.find.is(200), but actually found 404                       410 ( 8,20%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       8941 (OK=3941   KO=5000  )
> min response time                                     79 (OK=141    KO=79    )
> max response time                                    809 (OK=809    KO=712   )
> mean response time                                   134 (OK=176    KO=100   )
> std deviation                                         58 (OK=52     KO=38    )
> response time 50th percentile                        152 (OK=167    KO=84    )
> response time 75th percentile                        166 (OK=175    KO=90    )
> response time 95th percentile                        184 (OK=193    KO=163   )
> response time 99th percentile                        379 (OK=481    KO=178   )
> mean requests/sec                                 36.946 (OK=16.285 KO=20.661)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          3940 ( 44%)
> 800 ms < t < 1200 ms                                   1 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              5000 ( 56%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 403                      3941 (78,82%)
> status.find.is(200), but actually found 429                       632 (12,64%)
> status.find.is(200), but actually found 404                       427 ( 8,54%)
================================================================================

VSCodeAdapterItemSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       9589 (OK=9178   KO=411   )
> min response time                                    143 (OK=144    KO=143   )
> max response time                                    760 (OK=760    KO=179   )
> mean response time                                   160 (OK=160    KO=152   )
> std deviation                                         39 (OK=39     KO=6     )
> response time 50th percentile                        153 (OK=153    KO=150   )
> response time 75th percentile                        160 (OK=160    KO=154   )
> response time 95th percentile                        173 (OK=174    KO=164   )
> response time 99th percentile                        452 (OK=453    KO=175   )
> mean requests/sec                                 30.833 (OK=29.511 KO=1.322 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          9178 ( 96%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                               411 (  4%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 429                       400 (97,32%)
> status.find.is(200), but actually found 404                        11 ( 2,68%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       9804 (OK=9608   KO=196   )
> min response time                                    144 (OK=144    KO=144   )
> max response time                                   3173 (OK=3173   KO=179   )
> mean response time                                   165 (OK=165    KO=153   )
> std deviation                                         83 (OK=84     KO=7     )
> response time 50th percentile                        156 (OK=156    KO=152   )
> response time 75th percentile                        163 (OK=163    KO=156   )
> response time 95th percentile                        179 (OK=179    KO=167   )
> response time 99th percentile                        463 (OK=464    KO=171   )
> mean requests/sec                                 29.982 (OK=29.382 KO=0.599 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          9599 ( 98%)
> 800 ms < t < 1200 ms                                   2 (  0%)
> t > 1200 ms                                            7 (  0%)
> failed                                               196 (  2%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 429                       182 (92,86%)
> status.find.is(200), but actually found 404                        14 ( 7,14%)
================================================================================

VSCodeAdapterUnpkgSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       7199 (OK=4403   KO=2796  )
> min response time                                     78 (OK=140    KO=78    )
> max response time                                    777 (OK=777    KO=464   )
> mean response time                                   134 (OK=156    KO=100   )
> std deviation                                         48 (OK=41     KO=34    )
> response time 50th percentile                        146 (OK=150    KO=84    )
> response time 75th percentile                        153 (OK=157    KO=90    )
> response time 95th percentile                        168 (OK=172    KO=158   )
> response time 99th percentile                        209 (OK=448    KO=172   )
> mean requests/sec                                 35.995 (OK=22.015 KO=13.98 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4403 ( 61%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              2796 ( 39%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 403                      2199 (78,65%)
> status.find.is(200), but actually found 429                       565 (20,21%)
> status.find.is(200), but actually found 404                        32 ( 1,14%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       7192 (OK=4422   KO=2770  )
> min response time                                     79 (OK=140    KO=79    )
> max response time                                   3471 (OK=3471   KO=3178  )
> mean response time                                   142 (OK=166    KO=104   )
> std deviation                                        130 (OK=137    KO=108   )
> response time 50th percentile                        149 (OK=153    KO=84    )
> response time 75th percentile                        157 (OK=161    KO=94    )
> response time 95th percentile                        175 (OK=179    KO=163   )
> response time 99th percentile                        415 (OK=468    KO=184   )
> mean requests/sec                                 34.411 (OK=21.158 KO=13.254)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          4408 ( 61%)
> 800 ms < t < 1200 ms                                   5 (  0%)
> t > 1200 ms                                            9 (  0%)
> failed                                              2770 ( 39%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 403                      2192 (79,13%)
> status.find.is(200), but actually found 429                       545 (19,68%)
> status.find.is(200), but actually found 404                        33 ( 1,19%)
================================================================================

VSCodeAdapterVspackageSimulation

f53c16d

0364a2e


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       8237 (OK=3237   KO=5000  )
> min response time                                     79 (OK=148    KO=79    )
> max response time                                    773 (OK=773    KO=517   )
> mean response time                                   131 (OK=166    KO=109   )
> std deviation                                         49 (OK=44     KO=38    )
> response time 50th percentile                        150 (OK=159    KO=85    )
> response time 75th percentile                        158 (OK=166    KO=148   )
> response time 95th percentile                        173 (OK=184    KO=160   )
> response time 99th percentile                        202 (OK=464    KO=172   )
> mean requests/sec                                 37.271 (OK=14.647 KO=22.624)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          3237 ( 39%)
> 800 ms < t < 1200 ms                                   0 (  0%)
> t > 1200 ms                                            0 (  0%)
> failed                                              5000 ( 61%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 403                      3237 (64,74%)
> status.find.is(200), but actually found 429                      1740 (34,80%)
> status.find.is(200), but actually found 404                        23 ( 0,46%)
================================================================================


================================================================================
---- Global Information --------------------------------------------------------
> request count                                       8347 (OK=3347   KO=5000  )
> min response time                                     78 (OK=149    KO=78    )
> max response time                                   3665 (OK=3665   KO=642   )
> mean response time                                   137 (OK=179    KO=108   )
> std deviation                                        127 (OK=186    KO=41    )
> response time 50th percentile                        152 (OK=161    KO=84    )
> response time 75th percentile                        160 (OK=168    KO=149   )
> response time 95th percentile                        175 (OK=182    KO=161   )
> response time 99th percentile                        209 (OK=481    KO=175   )
> mean requests/sec                                 35.978 (OK=14.427 KO=21.552)
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          3332 ( 40%)
> 800 ms < t < 1200 ms                                   1 (  0%)
> t > 1200 ms                                           14 (  0%)
> failed                                              5000 ( 60%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 403                      3347 (66,94%)
> status.find.is(200), but actually found 429                      1625 (32,50%)
> status.find.is(200), but actually found 404                        28 ( 0,56%)
================================================================================

ccorn · 2023-03-16T12:24:40Z

I have adapted the test script to the now-live v2 API.

The test output looks good, except that there are constantly differing downloadCount entries between ~~v2 and the original~~ responses with implicit and explicit includeAllVersions defaults (how?), which cause three comparisons to fail:

$ bash test-eclipse-openvsx-api-v2.sh open-vsx.org

+ curl -LsS --compressed 'https://open-vsx.org/api/-/query?extensionId=vscode.bat&extensionVersion=1.53.2&includeAllVersions=false'
+ curl -LsS --compressed 'https://open-vsx.org/api/-/query?extensionId=vscode.bat&includeAllVersions=false'
+ curl -LsS --compressed 'https://open-vsx.org/api/-/query?extensionId=vscode.bat&extensionVersion=1.53.2&includeAllVersions=true'
+ curl -LsS --compressed 'https://open-vsx.org/api/-/query?extensionId=vscode.bat&includeAllVersions=true'
+ curl -LsS --compressed 'https://open-vsx.org/api/-/query?extensionId=vscode.bat&extensionVersion=1.53.2'
+ curl -LsS --compressed 'https://open-vsx.org/api/-/query?extensionId=vscode.bat'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&extensionVersion=1.53.2&includeAllVersions=false'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&includeAllVersions=false'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&extensionVersion=1.53.2&includeAllVersions=true'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&includeAllVersions=true'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&extensionVersion=1.53.2&includeAllVersions=links'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&includeAllVersions=links'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat&extensionVersion=1.53.2'
+ curl -LsS --compressed 'https://open-vsx.org/api/v2/-/query?extensionId=vscode.bat'

Response                                      size    gzip   zstd
verquery.vscode.bat_1.53.2_false.json        42097    6654   6643
verquery.vscode.bat_false.json               42289    6748   6765
verquery.vscode.bat_1.53.2_true.json         42097    6654   6643
verquery.vscode.bat_true.json             20601744 3144785  36174
verquery.vscode.bat_1.53.2.json              42097    6654   6643
verquery.vscode.bat.json                     42289    6748   6765
verquery.v2.vscode.bat_1.53.2_false.json      1378     661    691
verquery.v2.vscode.bat_false.json             1570     742    796
verquery.v2.vscode.bat_1.53.2_true.json      42158    6213   5501
verquery.v2.vscode.bat_true.json            738051   27534  23607
verquery.v2.vscode.bat_1.53.2_links.json     42158    6213   5501
verquery.v2.vscode.bat_links.json            42350    6298   5635
verquery.v2.vscode.bat_1.53.2.json           42158    6213   5501
verquery.v2.vscode.bat.json                  42350    6298   5635

Check verquery.vscode.bat_1.53.2.json = verquery.vscode.bat_1.53.2_false.json
Check verquery.vscode.bat_1.53.2_true.json = verquery.vscode.bat_1.53.2_false.json
Check verquery.vscode.bat.json = verquery.vscode.bat_false.json
verquery.vscode.bat.json verquery.vscode.bat_false.json sind verschieden: Byte 41616, Zeile 1
Check verquery.v2.vscode.bat_1.53.2.json = verquery.v2.vscode.bat_1.53.2_links.json
Check verquery.v2.vscode.bat_1.53.2_true.json = verquery.v2.vscode.bat_1.53.2_links.json
Check verquery.v2.vscode.bat.json = verquery.v2.vscode.bat_links.json
verquery.v2.vscode.bat.json verquery.v2.vscode.bat_links.json sind verschieden: Byte 41616, Zeile 1

latest = 1.70.2
Check version in verquery.vscode.bat_1.53.2_false.json = 1.53.2: true
Check version in verquery.vscode.bat_false.json = 1.70.2: true
Check version in verquery.vscode.bat_1.53.2_true.json = 1.53.2: true
Check version in verquery.vscode.bat_true.json = 1.70.2: true
Check version in verquery.vscode.bat_1.53.2.json = 1.53.2: true
Check version in verquery.vscode.bat.json = 1.70.2: true
Check version in verquery.v2.vscode.bat_1.53.2_false.json = 1.53.2: true
Check version in verquery.v2.vscode.bat_false.json = 1.70.2: true
Check version in verquery.v2.vscode.bat_1.53.2_true.json = 1.53.2: true
Check version in verquery.v2.vscode.bat_true.json = 1.70.2: true
Check version in verquery.v2.vscode.bat_1.53.2_links.json = 1.53.2: true
Check version in verquery.v2.vscode.bat_links.json = 1.70.2: true
Check version in verquery.v2.vscode.bat_1.53.2.json = 1.53.2: true
Check version in verquery.v2.vscode.bat.json = 1.70.2: true

vercount = 488
Check version count in verquery.vscode.bat_1.53.2_false.json == 1: true
Check version count in verquery.vscode.bat_false.json == 1: true
Check version count in verquery.vscode.bat_1.53.2_true.json == 1: true
Check version count in verquery.vscode.bat_true.json == 488: true
Check version count in verquery.vscode.bat_1.53.2.json == 1: true
Check version count in verquery.vscode.bat.json == 1: true
Check version count in verquery.v2.vscode.bat_1.53.2_false.json == 1: true
Check version count in verquery.v2.vscode.bat_false.json == 1: true
Check version count in verquery.v2.vscode.bat_1.53.2_true.json == 1: true
Check version count in verquery.v2.vscode.bat_true.json == 488: true
Check version count in verquery.v2.vscode.bat_1.53.2_links.json == 1: true
Check version count in verquery.v2.vscode.bat_links.json == 1: true
Check version count in verquery.v2.vscode.bat_1.53.2.json == 1: true
Check version count in verquery.v2.vscode.bat.json == 1: true

Check url presence in verquery.v2.vscode.bat_1.53.2_false.json: true
Check url presence in verquery.v2.vscode.bat_false.json: true
Check url presence in verquery.v2.vscode.bat_1.53.2_true.json: true
Check url presence in verquery.v2.vscode.bat_true.json: true
Check url presence in verquery.v2.vscode.bat_1.53.2_links.json: true
Check url presence in verquery.v2.vscode.bat_links.json: true
Check url presence in verquery.v2.vscode.bat_1.53.2.json: true
Check url presence in verquery.v2.vscode.bat.json: true

Check allVersions not present in verquery.v2.vscode.bat_1.53.2_false.json: true
Check allVersions not present in verquery.v2.vscode.bat_false.json: true
Check allVersions present and nonempty in verquery.v2.vscode.bat_1.53.2_true.json: true
Check allVersions not present in verquery.v2.vscode.bat_true.json: true
Check allVersions present and nonempty in verquery.v2.vscode.bat_1.53.2_links.json: true
Check allVersions present and nonempty in verquery.v2.vscode.bat_links.json: true
Check allVersions present and nonempty in verquery.v2.vscode.bat_1.53.2.json: true
Check allVersions present and nonempty in verquery.v2.vscode.bat.json: true

Compare other metadata in verquery.v2.vscode.bat_1.53.2_false.json with original
Compare other metadata in verquery.v2.vscode.bat_false.json with original
Compare other metadata in verquery.v2.vscode.bat_1.53.2_true.json with original
Compare other metadata in verquery.v2.vscode.bat_true.json with original
Compare other metadata in verquery.v2.vscode.bat_1.53.2_links.json with original
Compare other metadata in verquery.v2.vscode.bat_links.json with original
--- /dev/fd/63	2023-03-16 12:59:35.652886050 +0100
+++ /dev/fd/62	2023-03-16 12:59:35.652886050 +0100
@@ -6,7 +6,7 @@
     "dependencies": [],
     "description": "Provides snippets, syntax highlighting, bracket matching and folding in Windows batch files.",
     "displayName": "Windows Bat Language Basics (built-in)",
-    "downloadCount": 147073,
+    "downloadCount": 147071,
     "engines": {
       "vscode": "^1.52.0"
     },
Compare other metadata in verquery.v2.vscode.bat_1.53.2.json with original
Compare other metadata in verquery.v2.vscode.bat.json with original

3 error(s)

amvanbaren force-pushed the feature/issue-379-query-refactor branch from c015d32 to 99de410 Compare June 1, 2022 14:55

amvanbaren mentioned this pull request Jun 14, 2022

[OpenVSX] Use improved /api/-/query endpoint eclipse-theia/theia#11291

Closed

amvanbaren force-pushed the feature/issue-379-query-refactor branch from 6c698ec to 7d58e65 Compare June 30, 2022 12:48

amvanbaren force-pushed the feature/issue-379-query-refactor branch from 7d58e65 to e380e8a Compare October 19, 2022 12:24

amvanbaren force-pushed the feature/issue-379-query-refactor branch 2 times, most recently from 8d4b565 to 2662e17 Compare December 9, 2022 15:56

Reduce server bandwidth usage

f53c16d

Added /api/v2/-/query endpoint Deleted DTOs Use the same ExtensionVersion Comparator everywhere Added unit tests Add performance test for `/api/v2/-/query` endpoint.

amvanbaren force-pushed the feature/issue-379-query-refactor branch from 2662e17 to f53c16d Compare December 14, 2022 17:04

amvanbaren merged commit f302432 into eclipse:master Dec 14, 2022

/api/-/query response size #454

/api/-/query response size #454

Uh oh!

Conversation

amvanbaren commented May 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing steps

Uh oh!

ccorn commented May 19, 2022

Uh oh!

amvanbaren commented May 20, 2022

Uh oh!

amvanbaren commented May 20, 2022

Uh oh!

ccorn commented May 20, 2022

Uh oh!

ccorn commented May 20, 2022

Uh oh!

ccorn commented May 20, 2022

Uh oh!

ccorn commented May 20, 2022

Uh oh!

ccorn commented May 21, 2022

Uh oh!

amvanbaren commented May 23, 2022

Uh oh!

ccorn commented May 26, 2022

Uh oh!

ccorn commented May 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amvanbaren commented May 26, 2022

Uh oh!

ccorn commented May 26, 2022

Uh oh!

amvanbaren commented May 26, 2022

Uh oh!

ccorn commented May 26, 2022

Uh oh!

ccorn commented May 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amvanbaren commented May 26, 2022

Uh oh!

ccorn commented May 26, 2022

Uh oh!

amvanbaren commented May 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ccorn commented May 26, 2022

Uh oh!

ccorn commented May 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amvanbaren commented May 30, 2022

Uh oh!

ccorn commented May 31, 2022

Uh oh!

ccorn commented Jun 1, 2022

Uh oh!

ccorn commented Jun 1, 2022

Uh oh!

amvanbaren commented Jun 1, 2022

Uh oh!

ccorn commented Jun 1, 2022

Uh oh!

ccorn commented Jun 7, 2022

Uh oh!

amvanbaren commented Jun 7, 2022

Uh oh!

ccorn commented Jun 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amvanbaren commented Dec 14, 2022

RegistryAPI

VSCodeAPI

Uh oh!

ccorn commented Mar 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amvanbaren commented May 18, 2022 •

edited

Loading

ccorn commented May 26, 2022 •

edited

Loading

ccorn commented May 26, 2022 •

edited

Loading

amvanbaren commented May 26, 2022 •

edited

Loading

ccorn commented May 27, 2022 •

edited

Loading

ccorn commented Jun 7, 2022 •

edited

Loading

ccorn commented Mar 16, 2023 •

edited

Loading