[Lens] Make Lens Platform Self-Deployable and Cloud-Agnostic

[amrit3701]

Proposal

Title: Make Lens Platform Self-Deployable and Cloud-Agnostic

Problem Statement:

• The Lens platform currently has complex deployment requirements
• Heavy dependency on Amazon AWS services creates vendor lock-in:
  • S3: Required for CAD file storage and presigned URL generation
  • AWS Lambda: Required for running the FC-Worker (FreeCAD processing)
• Limited flexibility in deployment options
• Difficult to set up development and testing environments

Goals:

1. Create a cloud-agnostic version of the Lens platform that can be:
   • Self-hosted on any infrastructure
   • Easily deployed using container orchestration
   • Run multiple instances on a single machine or cluster
2. Eliminate direct dependencies on AWS services
3. Provide a streamlined deployment process
4. Enable easier development and testing workflows

Technical Architecture Changes:

1. Container Orchestration with Docker Compose (M, 1 week)

   • Create a unified docker-compose configuration for all components:
     • Frontend (Vue application)
     • Backend (API server)
     • MongoDB database
     • FC-Worker service
     • Redis for message queuing
     • Celery for task management
   • Include environment configuration templates
   • Add health checks and service dependencies
   • Enable volume mapping for persistent storage

2. Local Storage Service (M, 1 week)

   • Replace AWS S3 with a local storage solution
   • Implement file management APIs

3. FC-Worker Redesign (L, 2 weeks)

   • Replace AWS Lambda-based processing with scalable local solution:
     • Implement Celery-based task queue system
     • Add Redis for message broker and result backend
     • Create worker pools for parallel processing
   • Features:
     • Progress tracking

4. Documentation and Deployment Guide (S, 4-5 days)

   • Comprehensive deployment documentation:
     • System requirements
     • Installation steps
     • Configuration options
     • Troubleshooting guide
   • Technical documentation:
     • API documentation (existing Swagger)
     • Database schema and collections

Budget Breakdown:

Task	Description	Complexity	Time	Cost (USD)
1	Container Orchestration	Medium	1 week	1,500
2	Storage Service	Medium	1 week	1,500
3	FC-Worker Redesign	Large	2 weeks	3,000
4	Documentation	Small	4-5 days	500
Total			~5 weeks	6,500

Future Enhancements:

1. Platform Customization

   • Configurable feature sets per deployment tier

2. Version Control Integration

   • Git-based file storage backend

3. Collaboration Features

   • Real-time 3D model annotations in the web viewer
   • Comment/Review system on models
   • User activity timeline

Success Metrics:

• Deployment time reduced
• Zero AWS service dependencies
• Successful deployment verification on major Linux distributions
• Complete deployment possible within 30 minutes
• All existing functionality maintained

Source Code

• https://github.com/FreeCAD/Ondsel-Server
• https://github.com/FreeCAD/FC-Worker

[yorikvanhavre]

This looks good to me! Very well planned and the goal seems totally something we want.

[Reqrefusion]

This is great. Will FreeCAD be a lens copy that works here? Or are we just enough to read the results of development just from somewhere?

I also see you forget the "About You" part. I know you are one of the original developers of Lens, but I believe this should be.

The next questions are about FPA, but I think it needs to be asked. It will be included in the new grant system, or whether it is something related to conditional donation. In the vote here, a question mark ultimately makes the acceptance of the conditional donation seems to have been made in advance. Of course, how to examine this issue in the technical examination is a matter of curiosity.

[amrit3701]

sure @Reqrefusion.

About me

I'm a software engineer passionate about building solutions for CAD development. I have been using FreeCAD for 8 years and currently maintain the FreeCAD-Reinforcement workbench, which I initiated in 2017 as a Google Summer of Code project. I have mentored three Google Summer of Code projects under the FreeCAD organization. I also maintain the official FreeCAD Docker image, which has over 10,000 downloads. Throughout my career, I have developed numerous tools using FreeCAD to solve CAD-related challenges for various industries.

The concept of using FreeCAD as a backend CAD engine for web applications originated during my college days. When I shared this idea with Brad, it evolved from a prototype into LENS, a full-fledged CAD PDM web application. LENS is now open-source, enabling other developers to install and contribute to the project. However, LENS is currently tightly coupled with AWS services, making it challenging to set up development and testing environments. Through this proposal, I aim to simplify LENS installation to a single command process. Adding FreeCAD flavour to LENS would be the next steps after my proposal.

[chennes]

Thank you for submitting this grant proposal: it fits perfectly into the intended use for the Ondsel Onward fund, so should it be accepted the funds will be sourced from that budget line. This will be reviewed as part of the Q1 2025 grant program, with a submission deadline of 1 March 2025 and a final decision date of 31 March 2025.

[amrit3701]

I have created a couple of PRs:

• Feature: Allow to use local storage instead of AWS S3
• Revert company closing commits

@yorikvanhavre @sliptonic
What is the review policy? Could you grant me write access so I can merge PRs?

[yorikvanhavre]

@amrit3701 Let us discuss that at todays FPA meeting and we get back to you, but in principle I see no problem in giving you merge permissions.

[yorikvanhavre]

We decided yesterday that this decision, how we should handle the permissions of the new ex-ondsel repos, is not an FPA matter but should be decided by the FreeCAD maintainers, so it will be discussed next developers meeting: https://github.com/FreeCAD/FreeCAD-developer-meetings/blob/main/Agendas/agenda-2025-02-15.md

In the meantime, I'll merge your PRs above

[amrit3701]

@yorikvanhavre @sliptonic

PRs to allow users to run LENS using docker-compose:

FreeCAD/Ondsel-Server#3
FreeCAD/FC-Worker#1

[yorikvanhavre]

Merged!

[chennes]

For your planning purposes: the FPA has now voted to approve a budget of €20.000 per quarter, for a total of €80.000 in funds to be allocated by the grant program over the course of the year. This value may be revised in Q3 if donations significantly differ from the FPA's financial projections. This grant will be reviewed in the Q1 grant review cycle.

[amrit3701]

Hi @chennes, as far as I know, there's a separate grant for the LENS-specific proposal. Is it part of the €20,000 per quarter budget? Half of the proposed work is already completed and merged. Is decision still pending on this proposal?

[chennes]

No decisions on any proposals will be made until after the Q1 grant deadline (March 1, with voting expected to complete March 31). And yes, the €20,000 per quarter takes into account the additional funding: those funds will be used to prioritize the acceptance of Ondsel-related grants, but the grant process remains the same.

[pieterhijma]

@amrit3701 and @chennes, I added the development plan of Ondsel-related work that we set up with the Ondsel team and with direction from the anonymous donors in FPA issue #263 Ondsel Onward Fund. That issue tracks the spending on the Ondsel Onward Fund. I thought it would be helpful to list it there for reference and refer to it from here.

From my point of view, this grant proposal is highly aligned with that development plan.

[chennes]

Thank you again for your grant proposal: it was submitted in time to be evaluated as part of the Q1 2025 grant cycle, and is currently under committee review. In this quarter we received 12 grant requests totaling approximately 61.800 EUR in requests, and we expect to award approximately 15.000 EUR in grants. Because this grant will come from the Ondsel Onwards fund it will not count against the 15.000 budget if it is awarded. We appreciate your participation in the program, and you can expect to hear the results of the technical review committee's deliberations in two weeks.

[yorikvanhavre]

@amrit3701 you now have write access to the Ondsel-server and FC-worker repos, so you can continue to work on them and they stay maintained, later on we'll elaborate a better system to define who should have access, who will be responsible for reviewing/merging, etc...

[Reqrefusion]

No decisions on any proposals will be made until after the Q1 grant deadline (March 1, with voting expected to complete March 31). And yes, the €20,000 per quarter takes into account the additional funding: those funds will be used to prioritize the acceptance of Ondsel-related grants, but the grant process remains the same.

I did not fully understand what is being attempted to be conveyed here. Will the grant be included in the budgeted €20,000? This implies that the budget of €80,000 is incorrect.

[chennes]

@Reqrefusion see https://github.com/FreeCAD/FPA/blob/main/annualreports/2025.md#financial-plan -- "Ondsel Onward" is a separate line item.

[grd]

Local Storage Service (M, 1 week)

Replace AWS S3 with a local storage solution
Implement file management APIs

This is impossible to implement in one week. It took me half a year to implement my filesystem and it still has some points that could be improved. How do you want to implement this? What is the local storage solution? Git would probably the answer but the files that are stored on Git are publicly available, so I don't think that Git is secure for private file storage. SVN would be better IMO but SVN also has some problems.

Besides that, I think that if you really want to implement this, I think that the only thing what is possible is to implement a new version of Git that is private and suitable for CAD. Also NextCloud could work but file locking is not possible with NextCloud. If you really want to implement Git, for CAD we can discuss this, but Git is probably the wrong solution.

[amrit3701]

What is the local storage solution?

By "local storage," I mean a standard directory on your file system—no Git or SVN for now. While version control could be a great addition in a future proposal, the primary goal, as outlined in the current proposal, is to ensure a one-command deployment that operates independently of AWS.

[grd]

That is fine, except that this is then a one man operation, not a team, because you don't want to overwrite other men's files.

[amrit3701]

That is fine, except that this is then a one man operation, not a team, because you don't want to overwrite other men's files.

The platform itself ensures that no one can read or modify another user's files unless they have the appropriate read/write permissions.

[chennes]

The FPA's vote on the Quarter 1 2025 grant proposals is complete, and this grant was selected for funding. Congratulations, @amrit3701.

This work is being funded through the Ondsel Onward grant program using funds donated for and dedicated to making the software developed at Ondsel more widely useful to the Open Source community. This project is directly related to that effort.

[amrit3701]

Just a quick recap — I’ve already created PRs for the following tasks:

• Container Orchestration with Docker Compose: Add Docker Compose setup for local development Ondsel-Server#3
• Local Storage Service: Feature: Allow to use local storage instead of AWS S3 Ondsel-Server#2

I’ve also started working on the third task, which involves redesigning FC-Worker to use Celery and Redis for processing FreeCAD-related jobs.

[amrit3701]

I created a PR to implement Celery, Redis, and API services in FC_Worker. I also integrated these services into Ondsel-Server.

• Feature: Implement Celery Worker
• Integrate FreeCAD Celery Worker into docker-compose

[amrit3701]

Another set of PRs to enable hot reloading of FC-Worker code without the need to rebuild the Docker container repeatedly:

• Feature: Enable Hot Reloading for FC-Worker Code
• Feature: Add Development Environment for fc-worker

[amrit3701]

New PR to replace AWS-based Pre-signed URL Generation

[pieterhijma]

Hi @amrit3701, great work! Just for info, I'm still waiting for a decision on the NLnet grant.

[amrit3701]

The final PR related to the documentation has also been merged into the main branch:
FreeCAD/Ondsel-Server#11

@yorikvanhavre All stages of the proposal have been completed and merged into the main branch. Every task outlined in the proposal is now done. I will continue maintaining this project and address any issues reported by users.

I’ve also created a thread on the FreeCAD forum to continue the discussion about this project:
https://forum.freecad.org/viewtopic.php?t=96971

[sliptonic]

If I asked really nicely, could you publish docker images to docker hub or similar?

[amrit3701]

@pieterhijma
Issue related to initial model thumbnail are fixed in PR Fixed bug initial model thumbnail #15

[amrit3701]

If I asked really nicely, could you publish docker images to docker hub or similar?

@sliptonic Created a PR for that: FreeCAD/Ondsel-Server#16