Skip to content

[MPQEditor] Create or edit mixed precision quantization json file. #1505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/MPQEditor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# MixedPrecisionQantizationEditor (MPQEditor for short)

## About

MPQEditor povides easy way of creation/editing MPQ json files, suitable for
Quantization step in CfgEditor. Also it gives the ability to select the most
sensitive to quantization layers using `visq` data.

## How to use

Currently MPQEditor is integrated into EXPLORER view by context menu item
"Create MPQ json".

In order to use it:
1) call context menu for a `.circle` model in EXPLORER view
2) call "Create MPQ json"
3) give a name to MPQ json
4) edit MPQ json using MPQEditor
5) all layers from 'layers' section in json are shown in grid-table, so
that their options for quantization can be edited using their respective
dropdowns. Layers which don't belong to 'layers' section will be quantized
using default parameters.
6) to make 'model-graph-view' more informative the user may use 'visq'
information, it shows which layers are more sensitive to quantization than
others

## Screen layout
Layout is divided into three main parts:
1) Default Quantization Parameters

<img src="images/MpqDefaultSection.png" width=500>

2) 'Layers' section

<img src="images/MpqLayersSectionGen.png" width=500>

3) Model graph view

<img src="images/MpqModelGraphView.png" width=500>

### Layers section
<img src="images/MpqLayersSection.png" width=500>

1. `Add more layers`
- use it to add more layers for editing their quantization properties
2. `VISQ control`
- info
- actual path
- 'load visq' button
- 'clear visq' button
3. `Model graph is shown/hidden`
- use this to enable selection on `Model graph view`
4. `Individual layer`
- it consists of quantization properties set for each layer
5. `Quantization properties of layer`
- quantization and granularity fields for each layer
6. `Restore layer`
- use it to restore the layer to default state


For the whole workflow see:

<img src="images/MpqWorkflow.gif" width=1000/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets' make the image a bit smaller and text bigger

  • reduce empty space region
  • remove OUTPUT window

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok



Binary file added docs/images/MpqDefaultSection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/MpqLayersSection.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/MpqLayersSectionGen.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/MpqModelGraphView.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/MpqWorkflow.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 7 additions & 1 deletion media/CircleGraph/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ const viewMode = {
viewer: 0, // default circle viewer
selector: 1, // circle partition editor node selector
visq: 2, // quantization error viewer
visqselector: 3, //quantization error viewer which is able to select nodes
// refer https://github.com/Samsung/ONE-vscode/issues/1350
};

Expand Down Expand Up @@ -97,6 +98,8 @@ host.BrowserHost = class {
this._mode = viewMode.selector;
} else if (__viewMode === "visq") {
this._mode = viewMode.visq;
} else if (__viewMode === "visqselector") {
this._mode = viewMode.visqselector;
}
}

Expand Down Expand Up @@ -167,6 +170,9 @@ host.BrowserHost = class {
case "visq":
this._msgVisq(message);
break;
case "scrollToSelected":
this._view.setScrollToSelected(message.value);
break;
}
});

Expand Down Expand Up @@ -263,7 +269,7 @@ host.BrowserHost = class {
});

this._view.show("welcome spinner");
if (this._mode === viewMode.visq) {
if (this._mode === viewMode.visq || this._mode === viewMode.visqselector) {
// request visq data prior to model
// model is inside visq data
vscode.postMessage({ command: "visq" });
Expand Down
29 changes: 23 additions & 6 deletions media/CircleGraph/view.js
Original file line number Diff line number Diff line change
Expand Up @@ -542,13 +542,21 @@ view.View = class {
}
}

setScrollToSelected(value) {
this._scrollToSelected = value;
}

/**
* @brief toggleSelect will select or toggle select with CtrlKey down
* @param viewNode view.Node instance
* @note works on host mode is viewMode.selector
*/
toggleSelect(viewNode) {
if (viewNode && this._host._mode === viewMode.selector) {
if (
viewNode &&
(this._host._mode === viewMode.selector ||
this._host._mode === viewMode.visqselector)
) {
if (this._keyCtrl) {
// toggle selection
let index = this._selectionNodes.indexOf(viewNode);
Expand Down Expand Up @@ -585,7 +593,10 @@ view.View = class {

clearSelection() {
this._clearSelection();
if (host._mode === viewMode.selector) {
if (
host._mode === viewMode.selector ||
this._host._mode === viewMode.visqselector
) {
this._host.onView("selection");
}
}
Expand Down Expand Up @@ -1098,7 +1109,10 @@ view.View = class {
}

applyStyleSheetVisq(element) {
if (this._host._mode === viewMode.visq) {
if (
this._host._mode === viewMode.visq ||
this._host._mode === viewMode.visqselector
) {
let rules = [];
for (const styleSheet of this._host.document.styleSheets) {
if (styleSheet.title === "visq_style") {
Expand Down Expand Up @@ -1503,7 +1517,7 @@ view.Node = class extends grapher.Node {

_visq(node) {
const host = this.context.view._host;
if (host._mode !== viewMode.visq) {
if (host._mode !== viewMode.visq && host._mode !== viewMode.visqselector) {
return;
}

Expand Down Expand Up @@ -1535,7 +1549,7 @@ view.Node = class extends grapher.Node {
}
}
let visqSuffix = undefined;
if (host._mode === viewMode.visq) {
if (host._mode === viewMode.visq || host._mode === viewMode.visqselector) {
if (node.visq_index) {
let qstyle = `node-item-type-visq-${node.visq_index}`;
styles.push(qstyle);
Expand Down Expand Up @@ -1571,7 +1585,10 @@ view.Node = class extends grapher.Node {

if (host._mode === viewMode.viewer || host._mode === viewMode.visq) {
title.on("click", () => this.context.view.showNodeProperties(node, null));
} else if (host._mode === viewMode.selector) {
} else if (
host._mode === viewMode.selector ||
host._mode === viewMode.visqselector
) {
// toggle select with click
title.on("click", () => {
this.context.view.toggleSelect(this);
Expand Down
87 changes: 87 additions & 0 deletions media/MPQEditor/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
<!DOCTYPE html>
<html lang="en">
<!--
Copyright (c) 2023 Samsung Electronics Co., Ltd. All Rights Reserved

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<head>
<meta charset="utf-8">
<meta
http-equiv="Content-Security-Policy"
content="img-src vscode;
style-src 'unsafe-inline' ${webview.cspSource};
script-src 'nonce-${nonce}';">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Mixed Precision Quantization editor</title>
<link rel="stylesheet" href="${cssUri}">
<link rel="stylesheet" href="${codiconUri}">
<script nonce="${nonce}" type="module" src="${toolkitUri}"></script>
<script nonce="${nonce}" type="module" src="${scriptUri}"></script>
</head>
<body>
<div class="maintitle">
<span>Edit mixed precision quantization json file</span>
</div>
<div>
<div>
<div>
<vscode-radio-group id="DefaultDtype">
<label slot="label">Quantized Data Type</label>
<vscode-radio value="uint8">uint8</vscode-radio>
<vscode-radio value="int16">int16</vscode-radio>
</vscode-radio-group>
</div>
<div>
<vscode-radio-group id="DefaultGranularity">
<label slot="label">Granularity</label>
<vscode-radio value="layer">layer</vscode-radio>
<vscode-radio value="channel">channel</vscode-radio>
</vscode-radio-group>
</div>
</div>
</div>
<div>
<vscode-divider></vscode-divider>
</div>
<div>
<div class="leftbtns">
<vscode-button id="AddSpecificLayer" appearance="icon">
<span slot="start" class="codicon codicon-add"></span>
</vscode-button>
</div>
<div class="rightbtns">
<vscode-text-field id="VisqInputPath" startIcon="true" placeholder="visq path" readonly="true">
<span slot="start" class="codicon codicon-question" style="cursor: pointer">
<span class="help">
visq.json file with layer-wise quantization errors
</span>
</span>
<span id="visq-file" slot="end" class="codicon codicon-search" style="cursor: pointer"></span>
<span id="visq-delete" slot="end" class="codicon codicon-chrome-close" style="cursor: pointer"></span>
</vscode-text-field>
<vscode-checkbox id="circle-graph">Model Graph</vscode-checkbox>
</div>
</div>
<div>
<vscode-data-grid id="LayersTable" class="layers" generate-header="default" aria-label="Default Header" grid-template-columns="1fr 110px 120px 40px">
<vscode-data-grid-row row-type="header">
<vscode-data-grid-cell cell-type="columnheader" grid-column="1">Name</vscode-data-grid-cell>
<vscode-data-grid-cell cell-type="columnheader" grid-column="2">Quantization</vscode-data-grid-cell>
<vscode-data-grid-cell cell-type="columnheader" grid-column="3">Granularity</vscode-data-grid-cell>
<vscode-data-grid-cell cell-type="columnheader" grid-column="4">&nbsp;</vscode-data-grid-cell>
</vscode-data-grid-row>
</vscode-data-grid>
</div>
</body>
</html>
Loading