Skip to content

Commit 4b95c1a

Browse files
orangekame3claude
andauthored
Implement comprehensive error handling system (fixes #36) (#38)
* Implement comprehensive error handling system - Add custom exception classes with recovery suggestions - Create structured AnalysisResult class for comprehensive error reporting - Implement input validation helpers to catch errors early - Add comprehensive error handling to RandomizedBenchmarkingResult.analyze() - Maintain backward compatibility with DataFrame returns - Add 38 comprehensive tests covering all error scenarios Addresses GitHub issue #36 (Weak error handling). 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Extend comprehensive error handling to Rabi and T1 experiments - Add comprehensive error handling to RabiAnalysisResult with fitting validation - Add comprehensive error handling to T1AnalysisResult with exponential decay fitting - Create new Data classes for structured measurement data - Implement proper input validation, fitting error recovery, and quality checks - Add plotting with error handling following plot_settings.md guidelines - Maintain backward compatibility with DataFrame returns - Add comprehensive tests for both experiment types - Fix T1 validation to allow zero delay times (common in T1 measurements) All experiments now follow the same unified error handling pattern with: ✅ Custom exception handling with recovery suggestions ✅ Structured AnalysisResult with warnings and errors ✅ Input validation before expensive operations ✅ Graceful fitting failure recovery ✅ Quality validation of fitting results ✅ Comprehensive test coverage 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Fix forward reference Union syntax for Python compatibility - Fix CHSHData.analysis_result to use Union["CHSHAnalysisResult", None] - Forward references in quotes require Union syntax, not | operator - Maintain Python 3.10+ compatibility for type hints All 46 error handling tests now pass successfully. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Fix Rabi experiment integration with new error handling system - Update Rabi experiment class to use new RabiData and RabiAnalysisResult pattern - Create RabiData with proper measurement data structure - Initialize RabiAnalysisResult with required data, raw_results, and experiment_instance - Add default probability errors and shots for backward compatibility - Maintain existing DataFrame return behavior All 447 tests now pass successfully including the improved error handling. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Clean up redundant error handling code in experiment classes Remove duplicate plotting and saving methods from Rabi and T1 experiments: **Rabi experiment cleanup:** - Remove _create_plot() method (137 lines) - handled by RabiAnalysisResult - Remove _save_results() method (18 lines) - handled by RabiAnalysisResult - Update analyze() to use new RabiData and RabiAnalysisResult pattern - Simplify flow: analysis handled entirely by model classes **T1 experiment cleanup:** - Remove _create_plot() method (112 lines) - handled by T1AnalysisResult - Remove _save_results() method (18 lines) - handled by T1AnalysisResult - Update analyze() to use new T1Data and T1AnalysisResult pattern - Maintain consistent error handling architecture **Benefits:** ✅ Reduced code duplication (300+ lines removed) ✅ Centralized error handling in model classes ✅ Consistent plotting and saving behavior ✅ Easier maintenance and testing ✅ All tests still pass The new architecture separates concerns properly: - Experiment classes: circuit generation and data collection - Model classes: analysis, error handling, plotting, and saving 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]> * Refactor analysis results and validation in experiments - Updated T1 and T2 Echo experiments to utilize new data structures for analysis results. - Improved error handling in CHSH, Hadamard Test, Rabi, and Ramsey models to use a unified error reporting system. - Enhanced validation helpers to include non-negative checks for amplitude values. - Adjusted plotting functions to set logarithmic scales for better visualization of decay processes. - Modified DataFrame structures in tests to align with new analysis result formats. - Removed redundant code and improved overall readability and maintainability of the experiment modules. * Fix Rabi oscillation formula for RX gates and clarify π-pulse calibration --------- Co-authored-by: Claude <[email protected]>
1 parent 778a1c7 commit 4b95c1a

23 files changed

+4469
-1318
lines changed

CLAUDE.md

Lines changed: 147 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,31 @@ Use consistent colors from `get_experiment_colors()`:
211211

212212
#### Theoretical Fitting Formulas
213213

214+
**Rabi Oscillations**
215+
For RX(amplitude * π) gates:
216+
- Formula: `P(|1⟩) = A * sin²(π * amp / 2) + offset`
217+
- π-pulse occurs at amplitude = 1.0
218+
- Use scipy.optimize.curve_fit with bounds: A ∈ [0, 1], offset ∈ [0, 0.2]
219+
- **IMPORTANT**: Not `sin²(f * amp)` - must use the correct formula for RX(amp * π)
220+
221+
**T1 Relaxation**
222+
For excited state decay:
223+
- Formula: `P(|1⟩) = A * exp(-t/T1) + offset`
224+
- Use logarithmic time spacing: `np.logspace(log10(1), log10(max_delay), points)`
225+
- Plot with logarithmic x-axis for better visualization
226+
227+
**T2 Echo Decay**
228+
For coherence time measurement:
229+
- Formula: `P(|0⟩) = A * exp(-t/T2) + offset`
230+
- Use logarithmic time spacing for data collection
231+
- Plot with logarithmic x-axis
232+
233+
**Ramsey Fringes**
234+
For T2* measurement with detuning:
235+
- Formula: `P(|0⟩) = A * exp(-t/T2*) * cos(2πft + φ) + offset`
236+
- Use LINEAR time spacing to capture oscillations
237+
- Plot with LINEAR x-axis to show fringes clearly
238+
214239
**Phase Kickback**
215240
For H-RY(θ)-CP(φ)-H circuit:
216241
- Formula: `P(|0⟩) = 1 - sin²(θ/2) sin²(φ/2)`
@@ -332,6 +357,20 @@ When facing technical challenges:
332357
## Completed Implementations
333358

334359
### Recently Added
360+
- **Comprehensive Error Handling System (2024)**: Complete error handling overhaul
361+
- Custom exception hierarchy with recovery suggestions
362+
- AnalysisResult class for structured error reporting
363+
- Physics-aware validation for all experiment types
364+
- Graceful error recovery maintaining backward compatibility
365+
- Modern Python Union syntax (X | Y) throughout codebase
366+
- All 447 tests passing with no regressions
367+
368+
- **Critical Physics Bug Fixes (2024)**: Corrected fundamental formulas
369+
- **Rabi oscillations**: Fixed formula from sin²(f*amp) to sin²(π*amp/2) for RX(amp*π) gates
370+
- **Validation logic**: Updated to allow zero amplitudes in Rabi experiments
371+
- **Import corrections**: Fixed non-existent function imports across multiple models
372+
- **sklearn removal**: Replaced sklearn dependency with manual R² calculations
373+
335374
- **Randomized Benchmarking**: Complete implementation with both standard and interleaved variants
336375
- Exponential decay fitting for gate error characterization
337376
- SPAM-insensitive measurements
@@ -383,6 +422,108 @@ show_plotly_figure(fig, config)
383422
- **JSON metadata**: `experiment_results_YYYYMMDD_HHMMSS.json`
384423
- **Session data**: `.results/experiment_name_YYYYMMDD_HHMMSS/data/`
385424

425+
## Comprehensive Error Handling System
426+
427+
### Current Implementation (2024)
428+
429+
We have implemented a comprehensive error handling system across all experiment types with the following patterns:
430+
431+
#### Custom Exception Hierarchy
432+
```python
433+
# src/oqtopus_experiments/exceptions.py
434+
class OQTOPUSExperimentError(Exception):
435+
"""Base exception with recovery suggestions"""
436+
def __init__(self, message: str, suggestions: list[str] | None = None):
437+
super().__init__(message)
438+
self.suggestions = suggestions or []
439+
440+
class FittingError(OQTOPUSExperimentError):
441+
"""Raised when curve fitting fails"""
442+
443+
class InsufficientDataError(OQTOPUSExperimentError):
444+
"""Raised when insufficient data for analysis"""
445+
446+
class InvalidParameterError(OQTOPUSExperimentError):
447+
"""Raised when parameters are invalid"""
448+
```
449+
450+
#### Structured Error Reporting
451+
```python
452+
# src/oqtopus_experiments/models/analysis_result.py
453+
@dataclass
454+
class AnalysisResult:
455+
"""Structured result with comprehensive error handling"""
456+
success: bool
457+
data: pd.DataFrame | None = None
458+
errors: list[str] | None = None
459+
warnings: list[str] | None = None
460+
suggestions: list[str] | None = None
461+
metadata: dict[str, Any] | None = None
462+
463+
def to_legacy_dataframe(self) -> pd.DataFrame:
464+
"""Convert to legacy DataFrame format for backward compatibility"""
465+
```
466+
467+
#### Error Handling in Model Classes
468+
All experiment model classes (RabiAnalysisResult, T1AnalysisResult, etc.) now implement:
469+
- **Comprehensive input validation** with detailed error messages
470+
- **Graceful error recovery** - return meaningful results even when fitting fails
471+
- **Quality assessment** - warn about poor fit quality or unusual parameters
472+
- **Backward compatibility** - maintain legacy DataFrame return format
473+
474+
#### Validation Helpers
475+
```python
476+
# src/oqtopus_experiments/utils/validation_helpers.py
477+
def validate_non_negative_values(values: list[float], parameter_name: str) -> None:
478+
"""Validate values are non-negative (>= 0) - for Rabi amplitudes"""
479+
480+
def validate_probability_values(values: list[float], allow_zero: bool = False) -> None:
481+
"""Validate probability values are in [0, 1] range"""
482+
483+
def validate_fitting_data(x_data: list[float], y_data: list[float], experiment_name: str) -> None:
484+
"""Validate data is suitable for curve fitting"""
485+
```
486+
487+
#### Key Implementation Details
488+
- **Modern Python Union syntax**: Use `X | Y` instead of `Union[X, Y]`
489+
- **No sklearn dependency**: All R² calculations done manually with numpy
490+
- **Physics-aware validation**: Different validation rules for different experiments
491+
- **Comprehensive suggestions**: Every error includes actionable recovery suggestions
492+
- **All 447 tests passing**: Maintained full backward compatibility
493+
494+
### Error Handling Best Practices
495+
496+
**DO: Comprehensive validation with helpful messages**
497+
```python
498+
def analyze(self, plot=True, save_data=True, save_image=True) -> pd.DataFrame:
499+
try:
500+
# Validate inputs with detailed error reporting
501+
result = self._validate_inputs(amplitudes, probabilities, errors)
502+
if not result.success:
503+
return result.to_legacy_dataframe()
504+
505+
# Attempt analysis with graceful error handling
506+
fitting_result = self._fit_data(...)
507+
if fitting_result.error_info:
508+
result.add_warning(f"Fitting issues: {fitting_result.error_info}")
509+
result.add_suggestion("Check data quality and calibration")
510+
511+
except InsufficientDataError as e:
512+
return AnalysisResult.error_result(
513+
errors=[str(e)],
514+
suggestions=e.suggestions
515+
).to_legacy_dataframe()
516+
```
517+
518+
**DON'T: Silent failures or generic error messages**
519+
```python
520+
def analyze(self):
521+
try:
522+
fit_data()
523+
except:
524+
return pd.DataFrame() # Silent failure - no error info!
525+
```
526+
386527
## Known Implementation Issues & Future Improvements
387528

388529
**⚠️ Based on comprehensive analysis with o3, the following issues have been identified for future improvements:**
@@ -439,10 +580,12 @@ show_plotly_figure(fig, config)
439580
- Hard to add new plot types or modify existing ones
440581
- **Fix**: Create decoupled visualization system
441582

442-
6. **❌ Weak Error Handling**
443-
- Analysis failures return empty DataFrames without clear error information
444-
- No validation of input parameters before expensive operations
445-
- **Fix**: Implement comprehensive error handling with recovery suggestions
583+
6. **RESOLVED: Comprehensive Error Handling Implemented (2024)**
584+
- Added custom exception hierarchy with recovery suggestions
585+
- Implemented structured AnalysisResult class for detailed error reporting
586+
- All experiment models now include comprehensive input validation
587+
- Graceful error recovery with meaningful fallback results
588+
- Modern Python Union syntax (X | Y) throughout codebase
446589

447590
## Architecture Guidelines for New Implementations
448591

docs/experiments/rabi.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,9 +51,11 @@ The amplitude sweep reveals the π-pulse calibration point.
5151

5252
The experiment fits data to the Rabi oscillation model:
5353
```
54-
P(|1⟩) = A × sin²(π × amplitude × frequency) + offset
54+
P(|1⟩) = A × sin²(π × amplitude / 2) + offset
5555
```
5656

57+
This formula corresponds to RX(amplitude × π) gates, where the π-pulse occurs at amplitude = 1.
58+
5759
**Key Outputs:**
5860
- `pi_amplitude`: Drive amplitude for π-pulse
5961
- `frequency`: Rabi frequency

src/oqtopus_experiments/__init__.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,16 @@
1313
# Device information
1414
from .devices import DeviceInfo
1515

16+
# Exception classes
17+
from .exceptions import (
18+
AnalysisError,
19+
DataQualityError,
20+
FittingError,
21+
InsufficientDataError,
22+
InvalidParameterError,
23+
OQTOPUSExperimentError,
24+
)
25+
1626
# Experiments
1727
from .experiments import (
1828
CHSH,
@@ -41,4 +51,11 @@
4151
"OqtopusBackend",
4252
"DeviceInfo",
4353
"ExperimentDataManager",
54+
# Exception classes
55+
"OQTOPUSExperimentError",
56+
"AnalysisError",
57+
"InsufficientDataError",
58+
"FittingError",
59+
"InvalidParameterError",
60+
"DataQualityError",
4461
]

src/oqtopus_experiments/exceptions.py

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Custom exception classes for OQTOPUS Experiments
4+
5+
Provides specific exception types for better error handling and user feedback.
6+
"""
7+
8+
from typing import Any
9+
10+
11+
class OQTOPUSExperimentError(Exception):
12+
"""Base exception for all OQTOPUS experiment errors"""
13+
14+
def __init__(self, message: str, suggestions: list[str] | None = None):
15+
"""
16+
Initialize experiment error
17+
18+
Args:
19+
message: Error description
20+
suggestions: List of suggested solutions
21+
"""
22+
super().__init__(message)
23+
self.suggestions = suggestions or []
24+
25+
26+
class AnalysisError(OQTOPUSExperimentError):
27+
"""Base class for analysis-related errors"""
28+
29+
pass
30+
31+
32+
class InsufficientDataError(AnalysisError):
33+
"""Raised when there's not enough data for analysis"""
34+
35+
def __init__(self, data_points: int, required: int, experiment_type: str = ""):
36+
message = (
37+
f"Insufficient data for analysis: {data_points} points (need ≥{required})"
38+
)
39+
if experiment_type:
40+
message = f"{experiment_type}: {message}"
41+
42+
suggestions = [
43+
"Increase the number of measurement points",
44+
"Check if data collection completed successfully",
45+
"Verify backend execution didn't fail silently",
46+
]
47+
super().__init__(message, suggestions)
48+
self.data_points = data_points
49+
self.required = required
50+
51+
52+
class FittingError(AnalysisError):
53+
"""Raised when curve fitting fails"""
54+
55+
def __init__(self, fit_type: str, reason: str):
56+
message = f"{fit_type} fitting failed: {reason}"
57+
suggestions = [
58+
"Check if data quality is sufficient for fitting",
59+
"Try different initial parameter guesses",
60+
"Verify data doesn't contain NaN or infinite values",
61+
"Consider using more robust fitting algorithms",
62+
]
63+
super().__init__(message, suggestions)
64+
self.fit_type = fit_type
65+
self.reason = reason
66+
67+
68+
class InvalidParameterError(AnalysisError):
69+
"""Raised when parameters are invalid"""
70+
71+
def __init__(self, parameter: str, value: Any, expected: str):
72+
message = f"Invalid parameter '{parameter}': {value} (expected: {expected})"
73+
suggestions = [
74+
f"Check {parameter} value is within expected range",
75+
"Verify parameter types match requirements",
76+
"Review experiment setup and configuration",
77+
]
78+
super().__init__(message, suggestions)
79+
self.parameter = parameter
80+
self.value = value
81+
self.expected = expected
82+
83+
84+
class DataQualityError(AnalysisError):
85+
"""Raised when data quality issues prevent analysis"""
86+
87+
def __init__(self, issue: str, data_info: str = ""):
88+
message = f"Data quality issue: {issue}"
89+
if data_info:
90+
message += f" ({data_info})"
91+
92+
suggestions = [
93+
"Check measurement data for anomalies",
94+
"Verify backend execution completed successfully",
95+
"Consider re-running the experiment",
96+
"Check for hardware or software issues",
97+
]
98+
super().__init__(message, suggestions)
99+
self.issue = issue

0 commit comments

Comments
 (0)