Skip to content

Commit db98731

Browse files
JamesHabbenJamesHabben
authored
Merge pull request #852 from JamesHabben/main
feature docs
2 parents e76696d + 192abc6 commit db98731

File tree

3 files changed

+175
-0
lines changed

3 files changed

+175
-0
lines changed

‎admin/docs/features/media_management_architecture.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Architectural Decision Record: Media Management System
2+
3+
## Status
4+
Proposed
5+
6+
## Context
7+
We need a centralized system to manage media files (images, videos, audio) across different modules and across xLEAPP and LAVA projects. This system should handle metadata extraction, indexing, and provide a unified interface for modules to interact with media files, while avoiding duplication and maintaining references.
8+
9+
For ongoing discussion and updates related to this ADR, please refer to [Issue #851: Media Manager](https://github.com/abrignoni/iLEAPP/issues/851).
10+
11+
## Decision
12+
We will implement a Media Management Function that acts as a central point for "checking in" media files. This function will:
13+
1. Check if the file has already been added to the database
14+
2. Determine the media type (image, video, audio, etc.) for new files
15+
3. Extract appropriate metadata (resolution, duration, format, etc.) for new files
16+
4. Build and maintain a central media index for Lava
17+
5. Record references to media items from different modules and artifacts
18+
6. Return a MediaReference object to the calling module
19+
20+
## Consequences
21+
Positive:
22+
- Centralized media management improves consistency and reduces duplication
23+
- Pre-extracted metadata enables rich features like hover-over previews and advanced searching
24+
- Modules can easily reference media without handling low-level details
25+
- Automatic tracking of which modules and artifacts reference each media item
26+
27+
Considerations:
28+
- Need to implement efficient lookup for existing media items
29+
- May need to optimize for performance with large numbers of media files
30+
- Automatic parser in ilapfuncs.py and lavafuncs.py needs to handle MediaReference objects appropriately
31+
32+
## Implementation Notes
33+
- Modules will pass MediaReference objects as cell values in their data structures
34+
- Implement an automatic parser to extract necessary information from MediaReference objects
35+
- Design the system to support future media filtering and searching capabilities
36+
- Consider implementing a background process for enhanced metadata extraction (e.g., EXIF data) in the future

‎admin/docs/features/media_management_technical.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Media Management System: Technical Specification
2+
3+
## 1. System Components
4+
5+
### 1.1 Media Management Core
6+
7+
#### 1.1.1 `check_in_media(file_path: str, module_name: str, artifact_name: str) -> MediaReference`
8+
9+
Functionality:
10+
- Checks if the file has already been added to the database
11+
- If file exists, returns the existing MediaReference
12+
- If file doesn't exist:
13+
- Validates file existence and readability
14+
- Determines media type using `mimetypes` library
15+
- Extracts basic metadata (size, dimensions for images, duration for audio/video)
16+
- Generates a unique ID for the media item
17+
- Stores metadata in the database
18+
- Records the module name and artifact name referencing the media
19+
- Returns a `MediaReference` object
20+
21+
Error Handling:
22+
- Raises `FileNotFoundError` if file doesn't exist
23+
- Raises `PermissionError` if file isn't readable
24+
- Raises `UnsupportedMediaTypeError` for unrecognized file types
25+
26+
...
27+
28+
## 2. Database Schema
29+
30+
### 2.1 `media_items` Table
31+
32+
| Column | Type | Description |
33+
|-------------|---------|-------------------------------------|
34+
| id | TEXT | Unique identifier (UUID) |
35+
| type | TEXT | Media type (IMAGE, VIDEO, etc) |
36+
| path | TEXT | Original file path |
37+
| metadata | TEXT | Extracted metadata (JSON-encoded) |
38+
| created_at | INTEGER | Creation timestamp (Unix timestamp) |
39+
| updated_at | INTEGER | Last update timestamp (Unix timestamp) |
40+
41+
### 2.2 `media_references` Table
42+
43+
| Column | Type | Description |
44+
|---------------|---------|---------------------------------------|
45+
| id | TEXT | Unique identifier (UUID) |
46+
| media_item_id | TEXT | Foreign key to media_items.id |
47+
| module_name | TEXT | Name of the module referencing media |
48+
| artifact_name | TEXT | Name of the artifact referencing media |
49+
| created_at | INTEGER | Creation timestamp (Unix timestamp) |
50+
51+
...
52+
53+
## 3. Integration Flows
54+
55+
### 3.1 Media Check-in Process
56+
57+
1. Module calls `check_in_media(file_path, module_name, artifact_name)`
58+
2. Media Management Core checks if file already exists in database
59+
3. If file exists, Core returns existing MediaReference
60+
4. If file doesn't exist:
61+
a. Core validates file
62+
b. Core extracts basic metadata
63+
c. Core generates unique ID
64+
d. Core stores data in `media_items` table
65+
5. Core records reference in `media_references` table
66+
6. Core returns `MediaReference` to module
67+
7. Module stores `MediaReference` object as cell value in its data structure
68+
69+
...

‎admin/docs/features/timezones_architecture.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Architectural Decision Record: Timezone Handling in Project
2+
3+
## Status
4+
Proposed
5+
6+
## Context
7+
Our project deals with timestamp data from various sources, including SQL databases and plist files. We need a consistent approach to handle timezones across different components of our system, including LAVA, TSV, and KML outputs. The current implementation varies across modules, leading to inconsistencies and potential errors in timestamp processing.
8+
9+
For ongoing discussion and updates related to this ADR, please refer to [Issue #850: Timezone Handling](https://github.com/abrignoni/iLEAPP/issues/850).
10+
11+
Key issues:
12+
1. Some modules produce timestamps without timezone information.
13+
2. The `lava_insert_sqlite_data` function expects timestamps in ISO format with timezone.
14+
3. Different output formats (LAVA, TSV, KML) have different timezone requirements.
15+
4. The `timezone_offset` parameter is currently only supported in iLEAPP.
16+
5. The need to maintain compatibility with TSV and KML outputs, which benefit from timezone adjustments.
17+
6. Consideration of features in comparable commercial tools, which often include timezone adjustment capabilities.
18+
19+
## Decision
20+
We propose the following approach for handling timezones:
21+
22+
1. All modules should provide timestamps in one of two formats:
23+
a. Unix timestamps (assumed to be in UTC)
24+
b. ISO 8601 format with timezone specified (e.g., "2021-12-12 12:54:37+00:00")
25+
26+
2. Maintain the `timezone_offset` parameter as an input to iLEAPP:
27+
- This parameter will continue to be used for adjusting timezones in TSV and KML outputs.
28+
- It provides consistency with features found in comparable commercial tools.
29+
- For LAVA output, all timestamps will be converted to UTC, with timezone adjustments handled by LAVA's display features.
30+
31+
3. Update timestamp conversion functions to handle null or empty values gracefully.
32+
33+
4. For LAVA, all timestamps should be converted to UTC, with timezone adjustments handled by LAVA's display features.
34+
35+
## Consequences
36+
Positive:
37+
- Consistent timestamp handling across all modules
38+
- Improved compatibility with LAVA's requirements
39+
- Maintained support for timezone adjustments in TSV and KML outputs
40+
- Maintained feature parity with commercial tools regarding timezone adjustments in direct outputs (TSV, KML)
41+
- Flexibility for users to adjust timezones in non-LAVA outputs without requiring changes to the source data
42+
43+
Negative:
44+
- Requires updates to existing modules to ensure compliance with new standards
45+
- May introduce temporary inconsistencies during the transition period
46+
47+
Risks:
48+
- Potential for data misinterpretation if timezone information is not correctly propagated through the system
49+
50+
## Implementation
51+
1. Update `ilapfuncs.py` to include robust timestamp conversion functions that:
52+
- Convert Unix timestamps to ISO 8601 format with UTC timezone
53+
- Add timezone information to ISO format timestamps if missing
54+
- Handle null or empty timestamp values gracefully
55+
56+
2. Modify the `lava_insert_sqlite_data` function to accept both Unix timestamps and ISO 8601 format timestamps.
57+
58+
3. Update all modules to use the new timestamp conversion functions when processing data.
59+
60+
4. Ensure the `timezone_offset` parameter in iLEAPP is properly documented and its usage is clear for developers:
61+
- It should be applied to timestamp adjustments for TSV and KML outputs.
62+
- It should be ignored for LAVA output preparation, as LAVA handles timezone display internally.
63+
64+
5. Update the documentation for users to clarify how the `timezone_offset` parameter affects different output types.
65+
66+
## Notes
67+
- Further discussion may be needed to determine if LAVA should completely replace other report types in the future.
68+
- Consider creating a comprehensive test suite to verify correct timezone handling across all modules and output formats.
69+
- Regular code reviews should include checks for proper timestamp formatting to ensure ongoing compliance with this decision.
70+
- The decision to maintain the `timezone_offset` parameter balances the needs of different output formats and user expectations based on comparable tools in the industry.

0 commit comments

Comments
 (0)