Skip to content

Rebase o19s/evaluation_lab to opensearch_projects/main #523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 192 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
192 commits
Select commit Hold shift + click to select a range
72b85cb
Mockup for new search-relevance experience (#489)
fen-qin Mar 20, 2025
4e3be12
Component to enable search-relevance and opensearch API calls (#490)
fen-qin Mar 21, 2025
d2b8810
Add DEVELOPER_GUIDE.md for router setup + fix missing files (#493)
fen-qin Apr 3, 2025
85a49a3
add get query set by id to router
fen-qin Apr 8, 2025
8638501
search config list testing
wrigleyDan Apr 8, 2025
cb22386
swap Experimentation pages out for Tester page
wrigleyDan Apr 8, 2025
5efc408
fix Tester page errors
wrigleyDan Apr 8, 2025
44c6691
fix brocken API call to list search configs
wrigleyDan Apr 9, 2025
be4a944
add table view for retrieved search configs to tester page
wrigleyDan Apr 9, 2025
f5f0f8a
WIP Query Set Listing
alexeyrodriguez Apr 9, 2025
b9ef369
Add query set size
alexeyrodriguez Apr 9, 2025
b645571
Navigation to basic query set visualization
alexeyrodriguez Apr 9, 2025
91ce2c4
First stab at Query Set Creation
alexeyrodriguez Apr 10, 2025
e57373f
search configuration table overview as component in single page app
wrigleyDan Apr 10, 2025
f12ab5e
integrate search config table in experiment SPA
wrigleyDan Apr 10, 2025
ffb84f9
commit local changes to prepare for query set merge
wrigleyDan Apr 10, 2025
bd3fb88
resolve merge conflicts
wrigleyDan Apr 10, 2025
cf1860c
add missing route to resolve 404 when retrieving search configs
wrigleyDan Apr 10, 2025
6f73728
add TODO for table column extension once the backend is ready.
wrigleyDan Apr 10, 2025
ea409b1
add frontend component for search config creation.
wrigleyDan Apr 10, 2025
709204f
Merge pull request #1 from o19s/search_config_testing
wrigleyDan Apr 11, 2025
164c275
adding routes, adjusting form to search config inputs
wrigleyDan Apr 11, 2025
2fab513
switch from URL parameters to JSON body
wrigleyDan Apr 11, 2025
b8383c6
rename pairwise experiment occurrences to result list comparison expe…
wrigleyDan Apr 14, 2025
879d983
change custom experiment to result list comparison experiment, show s…
wrigleyDan Apr 14, 2025
8c17306
First version of visual comparison component.
alexeyrodriguez Mar 25, 2025
4ad69e9
Remove hard coded text
alexeyrodriguez Apr 14, 2025
cc46c59
fill dropdown boxes for query set and search configs with existing ob…
wrigleyDan Apr 14, 2025
fb3c305
add routes for experiments and integrate them into result list compar…
wrigleyDan Apr 14, 2025
caae74e
Merge pull request #2 from o19s/compare-prototype
alexeyrodriguez Apr 14, 2025
9114491
bugfixing, remove logging
wrigleyDan Apr 15, 2025
f8c94f2
Merge pull request #3 from o19s/create-search-config
wrigleyDan Apr 15, 2025
161d84b
Merge pull request #4 from o19s/create-result-list-comparison-experiment
wrigleyDan Apr 15, 2025
325b924
Switch to endpoints provided by plugin in the branch sample-api. Also…
alexeyrodriguez Apr 14, 2025
accf64a
Merge pull request #5 from o19s/switch-to-sample-api
alexeyrodriguez Apr 15, 2025
6653a3a
UI changes on experiment page and query_sets and search_configuration…
fen-qin Apr 15, 2025
883aaa4
Merge pull request #6 from fen-qin/fen_dev
alexeyrodriguez Apr 16, 2025
85376b9
Tabs and subtab navigation happens through routes. Also added entity …
alexeyrodriguez Apr 16, 2025
f20c842
Merge pull request #7 from o19s/view-and-routing
alexeyrodriguez Apr 16, 2025
e8385e0
Restore experiment create
alexeyrodriguez Apr 16, 2025
63375f2
add experiment list view, change existing EvaluationResults view whic…
fen-qin Apr 16, 2025
a43347a
add indexes and k selections for experiment_create_view
fen-qin Apr 16, 2025
aa53a4b
move index to be part of search configuration
fen-qin Apr 16, 2025
dece717
Merge pull request #8 from fen-qin/fen_dev
alexeyrodriguez Apr 17, 2025
cec9d20
First working experiment detailed view and some fixes on listing pages.
alexeyrodriguez Apr 16, 2025
490b1f3
Merge pull request #9 from o19s/querysets-view
alexeyrodriguez Apr 17, 2025
beb10fa
First working version of retrieving attributes from objects
alexeyrodriguez Apr 17, 2025
ee4c013
Merge pull request #10 from o19s/querysets-view
alexeyrodriguez Apr 17, 2025
3cbbd42
fix code editor to not compact json
fen-qin Apr 17, 2025
484401e
fix delete modal typo
fen-qin Apr 17, 2025
851b7b0
Merge pull request #11 from fen-qin/fen_dev
wrigleyDan Apr 17, 2025
04fd102
add the missing s to pptss
wrigleyDan Apr 21, 2025
e1c43b5
Merge pull request #13 from o19s/pptss-fix
alexeyrodriguez Apr 22, 2025
ffd6070
add search configuration validation panel to create view
fen-qin Apr 18, 2025
1da2630
Metric values are shown with two digits
alexeyrodriguez Apr 22, 2025
571a841
Grammar
alexeyrodriguez Apr 22, 2025
63de66c
Format dates using OpenSearch Dashboards configuration
alexeyrodriguez Apr 22, 2025
d4e6d4d
Remove unnecessary history parameter
alexeyrodriguez Apr 22, 2025
ebdc6c2
Merge pull request #14 from o19s/ui-corrections
alexeyrodriguez Apr 22, 2025
01bd3d2
Merge pull request #12 from fen-qin/fen_dev
fen-qin Apr 22, 2025
901b79d
Venn diagram of result summary, brittle layout at the moment
alexeyrodriguez Apr 23, 2025
278a2c8
Reverse the column order in the first results list
alexeyrodriguez Apr 23, 2025
bbf644e
Make sure rank circles do not shrink
alexeyrodriguez Apr 23, 2025
52d0668
Show images on top of the hover pane
alexeyrodriguez Apr 23, 2025
719a0ff
Display query in visual comparison
alexeyrodriguez Apr 23, 2025
c9e8ff2
Merge pull request #16 from o19s/ui-corrections
alexeyrodriguez Apr 23, 2025
a524954
Display Search Configuration names as results and show the number of …
alexeyrodriguez Apr 24, 2025
d0213ed
Make dashed lines solid and thicker; and make improved/dropped colors…
alexeyrodriguez Apr 24, 2025
8a8db6f
Make field selection control be next to label
alexeyrodriguez Apr 24, 2025
5ced136
Remove unnecessary comment
alexeyrodriguez Apr 24, 2025
14d9cc9
Merge pull request #17 from o19s/improved-view-experiment
alexeyrodriguez Apr 24, 2025
35a88d3
Prepare the visual comparison component for side-by-side visualization
alexeyrodriguez Apr 24, 2025
2ccfe10
Make tooltip positioning more flexible to enable side by side layout
alexeyrodriguez Apr 24, 2025
340045f
Remove dead code
alexeyrodriguez Apr 25, 2025
68fb143
Modularization of hover pane
alexeyrodriguez Apr 25, 2025
f2cf754
Removing further dead code
alexeyrodriguez Apr 25, 2025
68c86ce
Further refactoring
alexeyrodriguez Apr 25, 2025
f385d05
Result list modularization
alexeyrodriguez Apr 25, 2025
ddcd09d
Modularization of the connection lines component
alexeyrodriguez Apr 25, 2025
59697ae
Modularized the result columns
alexeyrodriguez Apr 25, 2025
e6c323c
Moved result column to its own component
alexeyrodriguez Apr 25, 2025
e9c6b14
Further layout for eyeballing component
alexeyrodriguez Apr 25, 2025
bb61d4b
Merge pull request #18 from o19s/side-layout-eyeball
alexeyrodriguez Apr 25, 2025
aecf874
Navigate to list experiments view after experiment creation
alexeyrodriguez Apr 25, 2025
32348b9
Fix errors in Search Configuration validation
alexeyrodriguez Apr 25, 2025
080ca18
Quick fix formatting of validation pane
alexeyrodriguez Apr 25, 2025
21c4ae9
Merge pull request #19 from o19s/ui-flow-fixes
alexeyrodriguez Apr 25, 2025
d008dcb
Make eyeball tool resizable to improve usability
alexeyrodriguez Apr 28, 2025
39a7902
Merge pull request #20 from o19s/resizable-experiment-view
alexeyrodriguez Apr 28, 2025
0a6a197
Hot fix incorrect error reporting in SC creation
alexeyrodriguez Apr 29, 2025
dff582d
Hotfix: fix check for empty index in Search Configuration creation
alexeyrodriguez Apr 29, 2025
0f47cb0
Hotfix: attribute resolution was not preserving original query ordering
alexeyrodriguez Apr 29, 2025
b24f7f3
Revert introduction of CAPS constants to refer to end-points.
alexeyrodriguez Apr 30, 2025
f1f5594
change template card names to reflect the experiments we expect in th…
wrigleyDan Apr 30, 2025
8791bf2
Merge pull request #21 from o19s/revert-api-constants
alexeyrodriguez Apr 30, 2025
abf2273
Improve view Search Configuration
alexeyrodriguez May 5, 2025
45d0fe3
Further adjustments to view pages
alexeyrodriguez May 5, 2025
0e89351
Further improvements in view pages
alexeyrodriguez May 5, 2025
7a22c5f
Remove dead code
alexeyrodriguez May 5, 2025
950ef27
Merge pull request #24 from o19s/view-screens
alexeyrodriguez May 5, 2025
207a412
Limit query set inputs in experiment creation
alexeyrodriguez May 6, 2025
f66e77b
Limit search configuration inputs in experiment creation
alexeyrodriguez May 7, 2025
e6a952d
Merge pull request #25 from o19s/limit-inputs
alexeyrodriguez May 7, 2025
c778f0e
Remount query view component once queries are loaded
alexeyrodriguez May 12, 2025
65ff059
Use more typed datastructures for experiment view
alexeyrodriguez May 12, 2025
8d0ceda
Create component to view pairwise experiments in preparation for furt…
alexeyrodriguez May 12, 2025
3b59530
Merge pull request #27 from o19s/experiment-view-refactoring
alexeyrodriguez May 12, 2025
18624d1
These are different types experiments, and we dont really provide a t…
epugh May 6, 2025
eab81f1
Description that covers both manual entry and sampling
epugh May 7, 2025
b1b73d6
Help text needs to be complete sentences.
epugh May 7, 2025
d1514f5
Tweak text of button.
epugh May 7, 2025
763d915
Help text comes as sentences.
epugh May 7, 2025
1da3d48
Help text comes as sentences.
epugh May 7, 2025
ae4be1a
Be more specific on what the validation is about.
epugh May 7, 2025
36861dc
Fix the "cancel" feature,
epugh May 7, 2025
7240207
Wordsmith the results comparison card name
epugh May 7, 2025
a49bbca
Wordsmith the placeholder message
epugh May 7, 2025
0259378
Merge pull request #29 from o19s/ui_polishing_fixed
alexeyrodriguez May 13, 2025
47fe91a
Refactoring of experiment types and add forgotten file
alexeyrodriguez May 13, 2025
3bad8cf
Use of experiment type of experiment listing
alexeyrodriguez May 13, 2025
7b809b2
Merge pull request #30 from o19s/more-experiment-types
alexeyrodriguez May 13, 2025
e76284e
Move former relevance tool to single query comparison
alexeyrodriguez May 14, 2025
5d80265
Remove old dead components
alexeyrodriguez May 14, 2025
69049ec
Error handling
alexeyrodriguez May 14, 2025
1659b0a
Simplify passing of dependent values
alexeyrodriguez May 14, 2025
f7fa4b8
Merge pull request #32 from o19s/single-query-experiment
alexeyrodriguez May 14, 2025
f678ce1
The whole thing is relevance, but this is more specific tool
epugh May 14, 2025
a182897
Merge branch 'evaluation_lab' into template-and-text-improvements
epugh May 14, 2025
e4872d1
Merge pull request #22 from o19s/template-and-text-improvements
epugh May 14, 2025
92875ae
add judgment management pages
fen-qin May 14, 2025
4fd2ae4
support search pipeline selection
fen-qin May 14, 2025
18256b7
Merge pull request #31 from o19s/judgment_page
fen-qin May 14, 2025
e30ac1e
Fix for pairwise experiments for redo branch
alexeyrodriguez May 16, 2025
8328f77
Merge pull request #33 from o19s/adapt-to-redo-branch
alexeyrodriguez May 16, 2025
2dad4a7
First attempt at pointwise experiment view
alexeyrodriguez May 19, 2025
e541661
Update how experiment types are printed
alexeyrodriguez May 20, 2025
70c2e28
First working evaluation experiment creation
alexeyrodriguez May 20, 2025
c2731af
Simplify interface of components in create page
alexeyrodriguez May 20, 2025
c61dc24
Add forgotten view page for evaluation experiments
alexeyrodriguez May 20, 2025
a7ba0d3
Restrict number of search configurations for evaluation experiment
alexeyrodriguez May 20, 2025
1eeb825
Remove logging
alexeyrodriguez May 20, 2025
ca56249
Merge pull request #36 from o19s/basic-pointwise-experiment
alexeyrodriguez May 20, 2025
01495a3
Smarter help text, and require caller to specific number of search co…
epugh May 20, 2025
efee736
Review help text on forms
epugh May 20, 2025
dbd4cb1
respond to feedback on text help messages
epugh May 21, 2025
ae8d969
Lighter touch tweaking of labels for judgment: judgments and judgemen…
epugh May 21, 2025
3ef42e5
Summary tables
alexeyrodriguez May 21, 2025
2bd17bf
Lighter touch tweaking of labels for judgment: judgments and judgemen…
epugh May 21, 2025
92f0e4e
Merge pull request #39 from o19s/summary-tables
alexeyrodriguez May 21, 2025
b834dfb
support file picker to upload manual query set with or without refere…
fen-qin May 16, 2025
f3f51dc
Update how we define an optional parameter to meet modern OUI standard
epugh May 21, 2025
5c114eb
Merge pull request #37 from o19s/review_help_text
epugh May 21, 2025
8bca60e
Add optional tag for form fields that ARE optional.
epugh May 21, 2025
9eb2b4c
Merge pull request #41 from o19s/follow_standard_for_optional_form_va…
epugh May 21, 2025
dba600b
Attempt to come up with a tag line
epugh May 21, 2025
c158690
Feedback was that a small portion of query body isnt useful
epugh May 21, 2025
73fc29b
Try out different naming
epugh May 21, 2025
91c3792
Make the run experimient/cancel,back buttons work the same as other s…
epugh May 21, 2025
26a6457
Make the run experimient/cancel,back buttons work the same as other s…
epugh May 21, 2025
6b63f06
First version of sidebar navigation
alexeyrodriguez May 22, 2025
8065b89
Add EuiPanel to have a background for Experiment create pages
alexeyrodriguez May 26, 2025
36c607b
Add Template Type enum and add icon mapping
alexeyrodriguez May 26, 2025
27574cd
Experiments sub-menu
alexeyrodriguez May 26, 2025
23ba8b8
Merge pull request #47 from o19s/navigation-side-bar
alexeyrodriguez May 26, 2025
0d2a7bf
Easier to paremeterize colors and make colors of top legend with resu…
alexeyrodriguez May 27, 2025
d48e2dd
Further fixing of customization
alexeyrodriguez May 27, 2025
6133a96
search configuration to take whole DSL query instead of queryBody
fen-qin May 17, 2025
e45905a
fix selected search pipeline config parsing issue
fen-qin May 20, 2025
6366ea3
Further allowing adjusting eyeballing style
alexeyrodriguez May 27, 2025
185dcb6
Temporary style selection control
alexeyrodriguez May 27, 2025
af1afdb
fix inline search pipeline validation
fen-qin May 27, 2025
ab519b7
Expose hybrid optimizer in the left nav and expose place holder page
epugh May 27, 2025
90bc17b
Merge pull request #49 from o19s/introduce_hybrid_optimizer_ui
alexeyrodriguez May 28, 2025
90c6c19
Renaming of styles
alexeyrodriguez May 28, 2025
ec372c8
One more color scheme to evaluate
alexeyrodriguez May 28, 2025
95ecdac
Details pane is shown on click rather than on hover
alexeyrodriguez May 28, 2025
bb15e43
Use this term pointwise, it is not about the source of the judgements
epugh May 28, 2025
f620a23
seperate hybrid optimizer into its own form file, event though right …
epugh May 29, 2025
0312e61
update experiment type and index name
fen-qin May 28, 2025
ee8529c
add advanced setting for llm judgment
fen-qin May 28, 2025
2ce7683
Merge pull request #50 from o19s/align_experiment_names
epugh May 29, 2025
247778c
Merge pull request #45 from o19s/tweak_judgement_types
epugh May 30, 2025
dd05014
Merge pull request #43 from o19s/tagline
epugh May 30, 2025
075808f
Merge branch 'evaluation_lab' into narrow_search_config_listing_page
epugh May 30, 2025
6131e95
Merge pull request #44 from o19s/narrow_search_config_listing_page
epugh May 30, 2025
a3fba1e
Merge pull request #40 from o19s/introduce_list_term_for_judgements
epugh May 30, 2025
fb69604
Merge pull request #46 from o19s/fix_experiment_buttons
epugh May 30, 2025
8e20220
Merge branch 'main' into evaluation_lab
fen-qin Jun 1, 2025
6ec6f08
add access control via SRW feature flag in cluster settings
fen-qin May 30, 2025
528d456
Make style selection more discrete
alexeyrodriguez Jun 2, 2025
dda2f07
Merge pull request #48 from o19s/eyeballing-colors
alexeyrodriguez Jun 2, 2025
e024c5e
fix advanced setting bugs
fen-qin Jun 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .eslintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
extends:
- "@elastic/eslint-config-kibana"
- "plugin:@elastic/eui/recommended"
3 changes: 3 additions & 0 deletions .lintstagedrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"*.{js,jsx,json,css,md}": ["prettier --write", "git add"]
}
8 changes: 8 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.vscode
build
coverage
node_modules
npm-debug.log
yarn.lock
*.md
*.lock
5 changes: 5 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"singleQuote": true,
"trailingComma": "es5",
"printWidth": 100
}
16 changes: 16 additions & 0 deletions babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

module.exports = {
presets: [
require('@babel/preset-env'),
require('@babel/preset-react'),
require('@babel/preset-typescript'),
],
plugins: [
['@babel/plugin-transform-modules-commonjs', { allowTopLevelThis: true }],
[require('@babel/plugin-transform-runtime'), { regenerator: true }],
],
};
22 changes: 19 additions & 3 deletions common/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,35 @@
* SPDX-License-Identifier: Apache-2.0
*/

export const PLUGIN_ID = 'searchRelevance';
export const PLUGIN_NAME = 'Search Relevance';
export const PLUGIN_ID = 'searchRelevanceWorkbench';
export const PLUGIN_NAME = 'Search Relevance Workbench';
export const COMPARE_SEARCH_RESULTS_TITLE = 'Compare Search Results';

export enum ServiceEndpoints {
// OpenSearch node APIs
GetIndexes = '/api/relevancy/search/indexes',
GetPipelines = '/api/relevancy/search/pipelines',
GetSearchResults = '/api/relevancy/search',
GetSingleSearchResults = '/api/relevancy/single_search',
GetStats = '/api/relevancy/stats',
GetClusterSettings = '/api/relevancy/cluster_settings',

// Search Relevance node APIs
QuerySets = '/api/relevancy/query_sets',
SearchConfigurations = '/api/relevancy/search_configurations',
Judgments = '/api/relevancy/judgments',
Experiments = '/api/relevancy/experiments',
}

export enum BackendEndpoints {
QuerySets = '/_plugins/search_relevance/query_sets',
SearchConfigurations = '/_plugins/search_relevance/search_configurations',
Judgments = '/_plugins/search_relevance/judgments',
Experiments = '/_plugins/search_relevance/experiments',
}

export const SEARCH_API = '/_search';

//Query1 for the left search and Query2 for the right search page
// Query1 for the left search and Query2 for the right search page
export const QUERY_NUMBER_ONE = '1';
export const QUERY_NUMBER_TWO = '2';
4 changes: 2 additions & 2 deletions opensearch_dashboards.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
"server": true,
"ui": true,
"requiredPlugins": [
"navigation"
"navigation", "opensearchDashboardsReact"
],
"optionalPlugins": ["dataSource","dataSourceManagement","contentManagement"],
"supportedOSDataSourceVersions": ">=2.8.0"
}
}
39 changes: 27 additions & 12 deletions public/application.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -9,25 +9,40 @@ import { AppMountParameters, CoreStart } from '../../../src/core/public';
import { DataSourceManagementPluginSetup } from '../../../src/plugins/data_source_management/public';
import { SearchRelevanceApp } from './components/app';
import { AppPluginStartDependencies } from './types';
import { OpenSearchDashboardsContextProvider } from '../../../src/plugins/opensearch_dashboards_react/public';
import { ConfigProvider } from './contexts/date_format_context';

export const renderApp = (
{ notifications, http, chrome, savedObjects, application }: CoreStart,
coreStart: CoreStart,
{ navigation, dataSource }: AppPluginStartDependencies,
{ element, setHeaderActionMenu }: AppMountParameters,
dataSourceManagement: DataSourceManagementPluginSetup
) => {
const { notifications, http, chrome, savedObjects, application, uiSettings } = coreStart;
const props = {
notifications,
http,
navigation,
chrome,
savedObjects,
dataSourceEnabled: !!dataSource,
dataSourceManagement,
setActionMenu: setHeaderActionMenu,
application,
};

ReactDOM.render(
<SearchRelevanceApp
notifications={notifications}
http={http}
navigation={navigation}
chrome={chrome}
savedObjects={savedObjects}
dataSourceEnabled={!!dataSource}
setActionMenu={setHeaderActionMenu}
dataSourceManagement={dataSourceManagement}
application={application}
/>,
<OpenSearchDashboardsContextProvider services={coreStart}>
<ConfigProvider
uiSettings={uiSettings}
dataSourceEnabled={!!dataSource}
dataSourceManagement={dataSourceManagement}
setHeaderActionMenu={setHeaderActionMenu}
navigation={navigation}
>
<SearchRelevanceApp {...props} />
</ConfigProvider>
</OpenSearchDashboardsContextProvider>,
element
);

Expand Down
Loading
Loading